2ª Calle 20-92, Zona 11 Col. El Mirador 1, 01011 Mobile.:(502) 5202 1982 Home: (502) 2472 8501 Guatemala, Guatemala E-mail: jmatute@bigfoot.com Jorge Matute, October 2011 Title II Food Security Program SHARE MYAP 2006-2011 Final Evaluation Report Final Evaluation MYAP 2011 Executive Summary i Asociación SHARE Guatemala (SHARE) hired the consulting company of JMatute-CIENSA to carry out the final evaluation for the Title II Improved Food Security Program (IFSP) (MYAP 2006-2011). I.1 Program Background SHARE Guatemala is a Guatemalan organization whose mission is to promote participative and sustainable development opportunities, as well as provide emergency assistance, so that the most vulnerable populations are able to improve their quality of life. SHARE was legally established in 1987, and since then has implemented various programs in several of the nation’s departments. Child malnutrition is one of Guatemala’s most serious and chronic problems. In rural areas, 51.8%1 of children under five suffer from chronic malnutrition. Of these, 64.8%1 live in Guatemala’s north-western region. SHARE currently implements the Improved Food Security (IFS) Program, a Title II Multi-Year Assistance Program launched on October 1, 2006, and officially ending on September 30, 2011. This program aims to promote development initiatives, offering various services to improve food security for vulnerable families in rural areas in the departments of Huehuetenango and Chimaltenango. The general objective of the IFS Program is to improve food security for highly vulnerable Guatemalan families in rural areas. SHARE Guatemala is certified to manage United States government funding for the development of this program. The findings of this evaluation show highly positive changes in chronic malnutrition rates in children under five. While this evaluation cannot demonstrate the program’s causality, it can shed light on the effects it has had on participating communities. It is also worth noting that SHARE complimentarily implemented the Emergency Assistance Program (SYAP 2009- 2010). SHARE was awarded additional funding to be implemented throughout 2010 in order 1 ENSMI 2008-2009. NCHS standards = 51.8% / WHO standards = 49.8% Final Evaluation MYAP 2011 Executive Summary ii to compliment the IFS Program in the face of the deteriorated food security situation and to mitigate the effects of intermittent draught and rains throughout these years. The calculations for the sample size to be used in this study were based on the parameters described in this section, and are also described in this study’s design document. Calculations were made following FANTA II instructions. The sample size used in the study was calculated according to estimates of chronic malnutrition rates in children under five (children whose height for age falls below minus two standard deviations). The calculated sample size was of 1,162 children under five, to be found in a sample of 775 homes, distributed among 31 communities (or clusters). The sampling design is stratified by the geographic location of the communities by municipality, and by the presence of the Program’s agricultural and livestock farming interventions. As such, out of a total 15 strata, two clusters (communities) were selected from each, using random and PPS sampling (selection probability proportional to size). The exception was stratum 10, out of which three clusters were selected (see table 1). Data gathering in the field was carried out using the data collection instruments or ballots agreed to by the three PVOs in the MYAP consortium. In the field, internal and external quality control methods were implemented. The consultant monitored the process internally by including “editors” in the field team. The main function of these editors was to review each of the instruments or ballots collected in the field while still in the communities. This step helped to catch any errors made by the surveyors, and the few inconsistencies were corrected in a timely manner. The rest of the internal monitoring was left to each respective supervisor in the three teams of surveyors. SHARE performed the external monitoring with the participation of their Monitoring and Evaluation unit, as well as participation of staff from the Program’s Health and Nutrition and Agriculture/Livestock initiatives. Community facilitators also closely monitored the proceedings. External monitoring also contributed to correcting systematic errors during interviews. SHARE staff participation also aided in providing timely translation when necessary. Final Evaluation MYAP 2011 Executive Summary iii Table 1: Sampling Design Strata Stratum # Health and Nutrition Intervention2 Agricultural Intervention Municipality Department YES NO YES NO 1 X X Santa Barbara Huehuetenango 2 X X Nenton 3 X X 4 X X San Antonio Huista / Concepción Huista 5 X X 6 X X Aguacatan 7 X X 8 X X Todos Santos Cuchumatan 9 X X 10 X X Chiantla 11 X X San Martin Jilotepeque Chimaltenango 12 X X 13 X X Tecpan 14 X X 15 X X San Juan Comalapa Field surveying resulted in 769 completed interviews. A total of 1,046 children were anthropometrically measured, given that every child in each family was evaluated. With the exception of the anthropometric indicators used to evaluate the children, all other indicators relevant to health, nutrition and agriculture were evaluated in the 769 interviews. The other exception pertains to the indicators in the commercial intervention, which were measured using a sub-sample of the participants in these activities. The reasoning behind this was that these activities were not considered to have population-wide effects. As such, it was decided (with FANTA approval) to measure the effects on actual participants from the communities selected for the study. 2 Health and Nutrition interventions are carried out in all of the communities in the program coverage area, while agricultural and livestock interventions are carried out in a percentage of the total coverage area. Final Evaluation MYAP 2011 Executive Summary iv The information was entered into the computer using a double data entry method with EPI INFO software. This double entry was validated to correct for typing errors until an error-free data base could be ensured. Statistical analysis took the sampling design into account, using EPI INFO’s CSAMPLE analysis module that performs calculations based on stratification and cluster selection. This analysis required the calculation of a variable to weight the observations against the probability of selection. This aided in ensuring that all of the observations had a similar probability of selection. Final Evaluation MYAP 2011 Executive Summary v III.A MYAP Indicators Table III.A.1 MYAP Indicators and their goals Thematic Area Code 3 Indicators Baseline value GOAL 2011 Percentage (Confidence Interval. 95%) Anthropometrics in children under 5 years 1a Percentage of children aged 6 to 59.99 months with <2.0 SD height for age, NCHS 69% 66% 62.5% (55.9, 69.0) 1a Percentage of children aged under 5 with <2.0SD height for age, NCHS 64% 61% 57.2% (51.2, 63.1) 1b Percentage of children aged under 5 with <2.0DS height for age, WHO 69% N/A 65.8% (60.3, 70.2) 2a Percentage of children aged 0 to 59.99 months with <2.0 SD weight for age, NCHS 35% 30% 29.1% ( 24.8, 33.5) 2b Percentage of children aged 0 to 59.99 months with <2.0 SD weight for age, WHO 28% N/A4 22.2% (18.7, 25.7) Anthropometrics in children under 36 months 2a Percentage of children aged 0 to 35.99 months with <2.0 SD weight for age, NCHS 35% 28% 26.7% (21.7, 31.8) 2b Percentage of children aged 0 to 35.99 months with <2.0 SD weight for age, WHO 27% N/A4 18.7% ( 15.2, 22.1) Household dietary diversity and food scarcity in the household 4 Number of food groups consumed in the household in the preceding 24 hours. 4 Food groups 6 Food groups 7.3 Food groups (7.1, 7.4) 5 Average number of months out of the year with adequate provisions of food in the household. 11 months 11 months 11.0 months (10.8,11.2) Child feeding 8 Percentage of infants aged 0 to 5.99 months who have been exclusively breastfed in the preceding 24 hours. 71% 77% 74.7% (64.4, 85.0) Warning signs of childhood illness. 18 Percentage of mothers and caregivers with children aged 0 to 35.99 months who recognize at least two warning signs of childhood illness that indicate the need to seek health services. 36% 50% 75.4% (70.1,80.8) Maternal health and neonatal warning signs. 12 Percentage of mothers with children aged 0 to 35.99 months who recognize at least two warning signs in pregnancy that indicate the need to seek health services. 13% 34% 71.3% (65.0,77.6) 15 Percentage of mothers with children aged 0 to 35.99 months who recognize at least two neonatal warning signs (<28 days) that indicate the need to seek health services. 15% 35% 61.1% (53.8,68.5) Agricultural and livestock farming practices 22 Percentage of farmers who adopt at least two improved agricultural practices. 18% 23% 30.9% (25.5, 36.3) 23 Percentage of farmers who adopt at least two improved livestock practices 9% 14% 42.0% (35.5, 58.5) Income improvement 30 Percentage of farmers who adopt at least two improved formal marketing practices 17% 22% 86.7% (71.4, 100) 3 Codification of the variables subject to research Final Evaluation MYAP 2011 Executive Summary vi Thematic Area Code 3 Indicators Baseline value GOAL 2011 Percentage (Confidence Interval. 95%) (Not comparable to the baseline) Local capacity building 31 Number of communities with functioning early warning systems for Food and Nutritional Security 0 84 72* (48, 93) * According to SHARE’s monitoring system information the exact number of communities is 91. Table III.A.1 illustrates that the populations within SHARE’s MYAP coverage area now present a different situation than before the program. The proposed goals have been met, with the exception of the exclusive breastfeeding indicator. Results that directly affect children’s health: A 7% reduction from the baseline in chronic malnutrition rates in children under five (Z < -2SD height/age) by NCHS standards (from 69% to 62% (P<0.05)). A 5.9% reduction from the baseline in general malnutrition rates in children under five (Z < -2SD weight/age) by NCHS standards (from 35% to 29.1% (P<0.05)). An 8.3% reduction in general malnutrition rates in children under 36 months (Z < -2SD weight/age) by NCHS standards (from 35% in the baseline to 26.7% in the final evaluation). Values are based on NCHS standards. It is worth noting that the differences are still in evidence by WHO standards. Final Evaluation MYAP 2011 Executive Summary vii Malnutrition in the final and baseline evaluations Dietary diversity was clearly increased from the baseline value, from an average consumption of four food groups to an average of 7.3 groups, resulting as well in exceeding the established goal of six groups (P<0.05). The average number of months with adequate food provisions was 11 months, a similar value to the baseline and to the proposed goal. Recognition of warning signs in pregnant women, neonatal infants and children far exceeded baseline values, and as such also exceeded the proposed goals (P<0.05). The percentage of women who acquired knowledge of warning signs easily doubled or even quadrupled baseline values, and as such now: 71.3% of women recognize warning signs in pregnancy; 61.1% recognize neonatal warning signs, and 75.4% recognize warning signs of childhood illness. Among the women who mentioned recognizing some warning signs in both pregnancy and infants and children, a percentage of them (around 50%) also mentioned having suffered at least one warning sign, 0% 10% 20% 30% 40% 50% 60% 70% NCHS% children under 5 with chronic malnutrition WHO% children under 5 with chronic malnutrition NCHS% children under 5 with general malnutrion WHO% children under 5 with general malnutrition NCHS% children under 3 with general malnutrition WHO% children under 3 with general malnutrition Baseline Final Final Evaluation MYAP 2011 Executive Summary viii either in pregnancy or with one of their children. It is relevant to note that almost all of them (over 95%) sought assistance with qualified service providers (hospitals, health centers/posts, or private clinics). Given that the study design and sample size were calculated in order to measure chronic malnutrition as the main indicator, the measure of the other indicators was necessarily subject to the sampling calculated for the main objective. For most of the indicators, this sample size worked perfectly. However, in the case of the exclusive breastfeeding indicator, the target population is children under 6 months, which constitutes a very small subset of the total population of children under 5. Likewise, the sample obtained to measure this indicator was also too small. This resulted in obtaining data with too high a margin of error. As such, as perhaps could be expected, the results obtained for exclusive breastfeeding do not demonstrate a significant change from the baseline, because of the small sample that was obtained. However, the value recorded is significantly higher than the national average in rural areas (ENSMI 2008-2009, 60.4%). The recommended age to begin complimentary feeding in children is 6 months, and the study found that 87.1% of the women surveyed implemented this good practice. The microbiological quality of water is important to staying healthy. 96.5% of families treat their drinking water in some way. Of these, 93.9% boil their water, 17.6% filter their water, 6% use the SODIS method, and 1.3% use chlorine. Agricultural and livestock practices have also improved from the baseline, and these indicators also exceeded the proposed goals (P<0.05). Now, 30.9% of agricultural farmers implement improved agricultural practices, and 42% of livestock farmers implement improved practices. With regards to the implementation of at least two practices in formal marketing, the baseline found that 17% of farmers implemented these measures. Accordingly, a goal of 22% was established for this Final Evaluation MYAP 2011 Executive Summary ix indicator. SHARE developed this initiative with a small number of farmers, around 500 individuals, and contemplated a small amount of advocacy among participating populations. As such, the sample for this indicator is a sub-sample of the evaluation sample4 that is nonetheless still representative due to the random sampling of communities. In this case, a sample of 43 participants was surveyed, yielding the following information regarding the program’s achievements: 86.7% of those surveyed adopted at least two formal marketing practices This percentage among the total participating population is at least 71.4%. In the area of Local Capacity Building (Risk Management), SHARE has met the goal of providing 84 communities with an early warning and alert system for potential food insecurity. The three sentinel sites evaluated demonstrate community leaders’ effective performance with regards to risk management training, managing the community monitoring bulletin board, use of the respective notebook, and use of the rainwater gauge, all relevant to the performance of the Monitoring Systems and Early Warning for Food and Nutritional Security. Further, according to SHARE’s monitoring and evaluation system, 91 communities are now monitored by 12 sentinel sites. 4 FANTA and the three PVOs: SHARE, Save the Children, and CRS, agreed in meetings held in July 2011, that the measure of this indicator would be carried out using a sub-sample of participants, and not using an entire population sample, which was used to measure the other indicators. MYAP 2011 Final Evaluation I Introduction .............................................................................................................................. 1 I.1 Background of the organization (PVO): .............................................................................. 1 I.2 Justification and study objectives. ...................................................................................... 5 1.3 Program Indicators ............................................................................................................. 6 II Methodology ............................................................................................................................ 7 I. Study design .......................................................................................................................... 7 I.1 Sample Size ...................................................................................................................... 7 Selection of study subjects (children under five) ...................................................................12 I.2 Sampling Design (sample selection) ..............................................................................12 I.3 List of selected clusters / communities .........................................................................14 Clusters M2: Selected and visited community clusters .........................................................14 II. Updating and Adapting the Data Collection Instruments ..................................................15 III. Field Staff Selection and Training. .....................................................................................17 III.1 Field staff selection and integration............................................................................17 III.2 Training and standardization ......................................................................................17 IV. The Field Study ..................................................................................................................19 IV. 1 Field work phase ........................................................................................................19 A. Introductions and requests for community cooperation .............................................19 B. Survey team management ............................................................................................19 C. Linguistic Aspects ..........................................................................................................20 D. Field information management ....................................................................................20 IV.2 Quality control in the field ..........................................................................................21 IV.3 Results of the Evaluation............................................................................................21 IV.3.1. Health and Nutrition and Agriculture and Livestock Interventions ....................................21 IV.3.2. Micro-Business Development Intervention ........................................................................ 23 IV.3.3. Capacity Building (Risk Management) ................................................................................23 V Processing, data analysis and report preparation .............................................................24 III Results ....................................................................................................................................26 III.A MYAP Indicators .......................................................................................................28 Graph 1: Malnutrition in the final and baseline evaluations .........................................3030 III.B 2011 Results. ...............................................................................................................32 III.B.1 Anthropometrics in children under 5 ......................................................................32 Graph 2: Malnutrition prevalence in the study ...............................................................344 Graph 2a: malnutrition prevalence found in this study, compared to the 2009 ENSMI . 355 III.B.2 Family dietary diversity and family food scarcity ..................................................366 Graph 3: Average Dietary Diversity Scores ......................................................................377 MYAP 2011 Final Evaluation xi Graph 4: Scarce foods and reasons for scarcity ................................................................. 37 Graph 5: Food group consumed............................................¡Error! Marcador no definido. III.B.3 Information, education and communication in health and nutrition ...................399 Graph 6: Information, education and communication in health and nutrition .................40 III.B.4 Child Feeding ..........................................................................................................411 Graph 7: Child Feeding .....................................................................................................422 III.B.5 Warning signs in maternal, neonatal and children’s health ..................................423 Graph 8: Knowledge of warning signs in pregnancy, neonatal and children’s health.....444 III.B.6 Water Treatment ...................................................................................................455 III.B.7 Agricultural and Livestock Practices.......................................................................466 Graph 9: Agricultural and Livestock Practices ..................................................................488 III.B.8 Income Improvement (Formal Marketing) ..............................................................50 Graph 10: Formal Marketing Practices ............................................................................500 III.B.9 Capacity Building (Risk Management) ...................................................................511 IV Conclusions ...........................................................................................................................533 V Annexes ................................................................................................................................545 ANNEX 1: III.C Detailed 2011 Results. ................................................................................55 III.C.1 Anthropometrics in children under five: ................................................................556 III.C.2 Family Dietary Diversity and Family Food Scarcity ................................................599 III.C.3 Information, education and communication in health and nutrition ...................633 III.C.4 Child Feeding ..........................................................................................................655 III.C.5 Warning signs in maternal, neonatal and children’s health ..................................666 III.C.6 Water treatment ......................................................................................................69 III.C.7 Agricultural and Livestock Practices .........................................................................70 III.B.8 Income Improvement (Formal Marketing) ..............................................................73 III.B.9 Community Risk .......................................................................................................75 ANNEX 2: III.D Design Effect on MYAP 2011 Indicators ......................................................78 MYAP 2011 Final Evaluation xii COCODES Community Development Councils CRS Catholic Relief Services ENSMI National Maternal Child Health Survey FANTA Food and Nutrition Technical Assistance CI SHARE Cooperating Institutions MSPAS Ministry of Public Health and Social Welfare MYAP Multi Year Assistance Program NCHS National Center for Health Statistics Odds Ratio Statistical measure describing the strength of association between illness and exposure to a factor WHO World Health Organization PPS sampling Probability Proportional to Size Sampling PVO Private Voluntary Organization IFS Improved Food Security STC Save the Children USAID United States Agency for International Development MYAP 2011 Final Evaluation Introduction 1 Asociación SHARE Guatemala (SHARE) hired the consulting company of JMatute-CIENSA to carry out the final evaluation for the Title II Improved Food Security Program (IFSP) (MYAP 2006-2011). The results of the study are presented in this document. I.1 Background of the organization (PVO): SHARE Guatemala is a Guatemalan organization whose mission is to promote participative and sustainable development opportunities, as well as provide emergency assistance, so that the most vulnerable populations are able to improve their quality of life. SHARE was legally established in 1987, and is an international non-profit organization without political or religious affiliation, headquartered in Guatemala. Over its 24 years of work, SHARE has implemented various programs and projects in different regions and departments of Guatemala in order to promote sustainable development, focusing on areas with greater vulnerability. One such program currently in implementation is the Improved Food Security (IFS) Program, a Title II Multi Year Assistance Program (MYAP), approved for implementation from 2006-2011. MYAP 2011 Final Evaluation Introduction 2 IMPROVED FOOD SECURITY PROGRAM BACKGROUND: Child malnutrition is one of Guatemala’s most serious and chronic problems. In rural areas, 51.8%5 of children under five suffer from chronic malnutrition. Of these, 64.8% live in Guatemala’s north-western region. The IFS Program currently in implementation was launched on October 1, 2006, and officially ends on September 30, 2011. This program aims to promote development initiatives, offering various services to improve food security for vulnerable families in rural areas in the departments of Huehuetenango and Chimaltenango. GENERAL OBJECTIVE: The general objective of the IFS Program is to improve food security for highly vulnerable Guatemalan families in rural areas. STRATEGIC OBJECTIVES: 1. Improve the health and nutrition of breastfeeding mothers, children aged 0-36 months and pregnant women. 2. Improve family food production, income and resource management. 3. Strengthen community capacities to manage their own development. 4. Promote basic community infrastructure. From its launch on October 1, 2006 through its end on September 30, 2011, the program has sought to improve food security by improving four main axes: 1. Food access 2. Food availability 3. Food utilization 4. Risk Management 5 ENSMI 2008-2009. NCHS standards = 51.8% / WHO standards = 49.8% MYAP 2011 Final Evaluation Introduction 3 IFS Program Interventions and Services: Each of the above mentioned axes entail specific interventions and each intervention provides a series of services to the beneficiary population. The program has been designed to provide participating families with every possible opportunity to improve their food security. The objective population is comprised mainly of women, who transfer the benefits of the services they receive directly to their families. However, the target population varies in each intervention, depending on the nature of the activities therein. Geographic Area: The program is implemented in seven municipalities in the department of Huehuetenango and in three municipalities in the department of Chimaltenango. These have been deemed priority municipalities, as they share the following characteristics: They are included in the list compiled by the President’s Secretariat for Planning and Programming (SEGEPLAN), as one of the 70 secondary priority municipalities to receive development aid (2005). Their chronic malnutrition indicators are higher than 56.5%. Their poverty indicators exceed 57%. IFS Implementation: The program has been implemented with a self-help strategy. There are differences in the working methods of each of the interventions, as well as in the development context of each municipality, and even among the target populations. Given these unique contexts, the program design devised two different implementation strategies: Implementation through Local Partners: SHARE partners with local organizations called Cooperating Institutions (CIs). These are provided with financing, trainings in technologies and technical assistance. CIs implement the IFS Program directly with a specified organizational structure. They organize and train the community staff members that then provide direct services to participating families. In this process, families are organized into interest groups, who then participate in activities designed to strengthen organizational capacities and MYAP 2011 Final Evaluation Introduction 4 increase their own capacities for development. The program coordinates closely with the Ministry of Public Health and Social Welfare (MSPAS). This strategy is applied to the following interventions: Food Security Risk Management Health and Nutrition Food Production Food Provision Capacity building processes are planned, organized and evaluated in conjunction with CIs, following a cascade methodology defined by the general IFS Program strategy: From SHARE headquarters to regional offices From regional offices to municipal / CI offices From municipal/ CI offices to community staff From community staff to participating families Direct Implementation Model: In this model, each intervention has a specialized technical team responsible for strengthening participants’ organization, providing technical assistance, monitoring implementation and transferring the required resources for each service. This model is applied to the interventions in: Sanitation and access infrastructure Village Banking Micro-business development SHARE Guatemala is certified to manage United States government funding for the development of this program. MYAP 2011 Final Evaluation Introduction 5 I.2 Justification and study objectives The program and the results attained therein must be evaluated in the months preceding the program’s closure. In addition, it is important to determine the effectiveness of the program’s implemented strategies and operative processes, given that human development processes are dynamic and influenced by internal and external factors. 1. OBJECTIVES OF THE EVALUATION: As part of any planning process, an evaluation responds to the need to place a value on attained results and to identify any factors that have hindered the accomplishment of proposed objectives, in order to improve future planning. Thus, the objective of this study is to evaluate the results obtained by the Improved Food Security Program in order to determine the effects and achievements to date. The evaluation will focus on evaluating the consistency of the program’s strategic design in its approach to improving nutritional status, as well as the progress in attaining the proposed goals (its effect and impact indicators). Specific Objectives: 1. Compare the results achieved to the proposed goals and objectives (efficacy). SHARE hired an external consulting team to meet the evaluation’s proposed objective, in accordance with Food for Peace guidance for Title II program evaluations. This helps to ensure that the results and findings of the study are objective and transparent. This document presents the findings of the study led by the consulting team. The results presented respond to the evaluation’s specific objective, and provide SHARE with valuable information. MYAP 2011 Final Evaluation Introduction 6 1.3 Program Indicators SHARE presented the following indicators for its approved MYAP, to allow the evaluation to determine the scope of the program’s achievements: Thematic Area Indicators Anthropometrics in children under 5 years Percentage of children aged 6 to 59.99 months with <2.0 SD height for age, NCHS Percentage of children aged 0 to 59.99 months with <2.0 SD height for age, WHO Percentage of children aged 0 to 59.99 months with <2.0 SD weight for age, NCHS Percentage of children aged 0 to 59.99 months with <2.0 SD weight for age, WHO Anthropometrics in children under 36 months Percentage of children aged 0 to 35.99 months with <2.0 SD weight for age, NCHS Percentage of children aged 0 to 35.99 months with <2.0 SD weight for age, WHO Household dietary diversity and food scarcity in the household Number of food groups consumed in the household in the preceding 24 hours. Average number of months out of the year with adequate provisions of food in the household. Child feeding Percentage of infants aged 0 to 5.99 months who have been exclusively breastfed in the preceding 24 hours. Warning signs of childhood illness. Percentage of mothers and caregivers with children aged 0 to 35.99 months who recognize at least two warning signs of childhood illness that indicate the need to seek health services. Maternal health and neonatal warning signs. Percentage of mothers with children aged 0 to 35.99 months who recognize at least two warning signs in pregnancy that indicate the need to seek health services. Percentage of mothers with children aged 0 to 35.99 months who recognize at least two neonatal warning signs (<28 days) that indicate the need to seek health services. Agricultural and livestock practices Percentage of farmers who adopt at least two improved agricultural practices. Percentage of farmers who adopt at least two improved livestock practices Income improvement Percentage of farmers who adopt at least two improved formal marketing practices Local capacity building Number of communities with functioning early warning systems for Food and Nutritional Security This evaluation used the same instruments as those created for the baseline to evaluate the achievement of the above indicators. However, for this final evaluation, they were subject to review and validation by the three Title II MYAP consortium PVOs, resulting in a very small number of adjustments. MYAP 2011 Final Evaluation Methodology 7 This evaluation was implemented in five important phases: 1. Study design 2. Updating and adaptation of the instruments used to gather data 3. Field staff selection and training 4. Field work and data input 5. Information analysis and report preparation I. Study design The study design process was carried out by a different consulting team6 . On that occasion, the design responded to the needs of all three MYAPs, including Save the Children, SHARE and CRS. The sampling design presented then was supervised and approved by FANTA II. This section provides a summary of that design. I.1 Sample Size SAMPLING METHOD The recommended sampling method was a combination of stratification and cluster sampling. The community sampling frame was stratified by its exposure to agricultural and micro business development activities. First level of stratification: Geo-political. The 10 participating municipalities in the program. Second level of stratification: Farming/agricultural activity in the program. In accordance with a meeting on June 10 with FANTA, USAID and the PVOs, the sample must be stratified according to agricultural participation. As such, each of the geo-political strata should state: o With agricultural activity / Without agricultural activity 6 Consulting team developed by Jorge Matute; contracted by CRS, SHARE and Save the Children, with the objective of designing the sampling of the MYAP final evaluation. The design followed FANTA guidelines, who also reviewed and approved the design document. Jorge Matute also prepared the document to design the sampling of the baseline in 2007-2008. MYAP 2011 Final Evaluation Methodology 8 The groups were distributed proportionally among the strata. Sampling within each stratum was comprised of three sampling stages. PPS sampling was used in the first stage to select the communities, simple or systematic random sampling was used in the second stage to select homes within the previously selected communities, and simple random sampling was used to select the family within the home in the third stage. Cluster sampling in three phases was used to obtain the sample within each of the last level strata (municipalities-activities). First level of selection: Community. The community-cluster is a population group that resides in the same geo-political sector that is also quite homogenous. Community￾clusters refer to communities or human settlements known as villages, places, hamlets, towns, cities, etc. Community-clusters were selected using PPS Sampling. Community￾clusters correspond to the statistical concept known as primary clusters. Second selection level: Households. Within each of the community-clusters selected, random sampling was used to select households. This phase of selection took into consideration the only factor for inclusion in the sample: that the household contain children younger than five years of age. It is important to note that the field team documented the number of homes that comprised the sampling frame for each community-cluster, as well as the number of homes visited. Third selection level: Families. It was highly probable that a single household contain more than one family that met the afore-mentioned inclusion criteria. In such a case, one of the families was randomly selected. A family is defined by the concept of “nuclear family”, consisting of a mother, a father and their offspring7 . These families comprised the sampling units subject to measurement: the mother, father and any offspring under five. It was important to document the number of families that comprised the sampling frame for each home-cluster in the field. A total of 74 7 It is important to note that this final evaluation measured the data for a nuclear family, which slightly differed from the information used for the baseline, which was that of HOUSEHOLD. (The latter included the nuclear family as well as any other family member who eats from the same pot). MYAP 2011 Final Evaluation Methodology 9 households contained more than one nuclear family that met the inclusion criteria, and as such one family in each of these was randomly selected for the study. It must be stressed that the selection process for clusters and sampling units was random. This ensured adequate representation as well as the evaluation’s external validity, making it possible to extrapolate the data to the population. DETERMINING THE SAMPLE SIZE In accordance with FANTA’s instructions, this design is an adaptation. As such, it uses the same sample size calculations in order to perform estimations (and not to prove the hypotheses). The sample size for a study such as this is defined by three basic standard parameters: variance, confidence level and the standard error expected in the estimations. Given that the sampling design is not simple random, the effects of the design on the variance should be added to the preceding parameters. This is known as “Design Effect”. FANTA has suggested using a value of “2” as the design effect8 . Another factor that influenced the sample size is the condition imposed to consider the total target population as finite. Finite is understood to be relatively small. In this case small is taken to mean a total population smaller than 5,000. However, for this design and given the populations in question, the population was instead considered to be infinite (greater than 5,000), both for the MYAP program as well as in the case of each of the three PVOs. As such, it was not necessary to make any adjustments for population size. The sample size required for the baseline was calculated considering the following: Confidence level: this was set at 95% (Z = 1.96). P (1-P): The study abided by FANTA’s recommendation of using the value of P (1-P) = 0.25. Design effect on sampling. The study used FANTA’s recommended value: 2 Standard Error. The value selected for standard error was 4%. Given that the above values are constants in the equation, the standard error of measurement is what conditions the 8 Meeting with FANTA-2 and the MYAP PVOs, Guatemala, June 10, 2011 MYAP 2011 Final Evaluation Methodology 10 sample sizes in the end. In SHARE’s evaluation, sample sizes were calculated with a standard error of 4. Calculation of sample size in the target population of children under five: According to the baseline information, the selected households with children under five were found to have an average number of 1.5 children per household. Therefore, in order to find 1,163 children under five, it is necessary to visit: 1,163/1.5 = 775 households with children under five. Given that at the time of gathering field data some families could reject being interviewed, or that there may be cause to rule out a given survey (or that losses may occur), the loss was considered in the calculation of the sample size in order to increase the selection and avoid the need for substitutions that could skew the sample, or the need to repeat the selection process. THE SELECTION PROCESS IN THE FIELD: The following procedure was used to find children under five within the communities: a. Household selection. Houses were randomly selected using maps or community diagrams. b. Case confirmation. Once a household was selected, the first step was to confirm that it met the inclusion criteria, which was that a child under five resided within. MYAP 2011 Final Evaluation Methodology 11 Upon this confirmation, the team needed to establish whether or not there was more than one family with children under five within that household. In the case of the presence of two or more families who met this criterion, one of them was randomly selected. Once selected, the family was required to give informed consent to participate in the study. This whole process led to the possibility that the selection process could result in not finding a household with children under five, or that a selected family would choose not to participate. This loss was then defined as “loss for lack of response”. Given the decision to begin the selection within communities with households, the size of the sample to be selected within the communities refers to HOUSEHOLDS. Loss for lack of response: This formula was provided by FANTA, and was used to adjust the necessary sample size according to the possibility of data loss in the field, especially due to selection of and visits to households without children under 5: Where lambda corresponds to the calculated factor as: Lambda = the average size of the families X the proportion of people in the target population (under five) = -(6 X 0.15) = -0.9 (values provided by FANTA at the June 10 meeting). Note: The baseline used a sample size adjusted for calculations of the number of community clusters sampled. The final evaluation used a non-adjusted sample size to establish the number of community clusters, in keeping with FANTA’s recommendation of making the adjustment within each community cluster. Therefore, the study planned to find 42 households in each community-cluster, rather than the 25 that were used in the baseline. MYAP 2011 Final Evaluation Methodology 12 Selection of study subjects (children under five) Once the sampling unit (household) had been selected, it was likely to find more than one nuclear family (mother, father, and children) within it, resulting in more than one family applicable for study in the household. This made it necessary to define selection criteria for families. The team was also required to document the number of families living in the household and to specify that they formed part of the sampling frame of the household-cluster. The same inclusion criteria used in the selection of households could also be used for the selection of families within the household. The team need only document the situation and to select the family to be studied by performing a drawing among the families with children under five. The MYAP Consortium and FANTA agreed that all children under five in a selected nuclear family should participate in evaluating Capacities, Attitudes and Practices (CAPs), as well as in anthropometric measurements. With regards to the former, the surveyor should ask questions pertaining to the expected CAPs and in accordance with the age of the children within the household. For instance, if there is a child under six months old, the surveyor should ask the question regarding exclusive breastfeeding for that child. If there is also a child aged 14 months, then the surveyor should ask the question referring to complimentary feeding for that child. I.2 Sampling Design (sample selection) The sampling design responds to the need to measure the impact indicator: Prevalence (in %) of children with low height for age (chronic malnutrition defined by the z-score: malnourished = Z < -2SD). To this end, the evaluation followed a design of stratified cluster sampling in three phases. The stratification here took into consideration two characteristics in the population groupings: The first consisted of the geo-political division of the country into municipalities. In this case, the communities subject to the SHARE evaluation are distributed among two departments and ten municipalities: o Huehuetenango: San Antonio Huista, Concepcion Huista, Todos Santos, Chiantla, Santa Barbara, and Aguacatan o Chimaltenango: Tecpan, Comalapa, and San Martin MYAP 2011 Final Evaluation Methodology 13 The second consisted of the presence of the agricultural or farming intervention that the program implemented in some communities. As such, the groups are divided in two: those with the agricultural intervention, and those without9 . The conjunction among the geo-political divisions and the presence or absence of the agricultural intervention generates the stratification. The following are the strata defined for SHARE: Table M1: Strata and number of primary clusters to be sampled. Stratum # Department Municipality Agricultural Activity Total stratum population (families) Number of clusters /communities to be sampled Proportional distribution Adjusted distribution * 1 Huehuetenango Santa Barbara NO 788 2 2 2 Nenton YES 1,099 3 2 3 NO 194 0 2 4 San Antonio Huista / Concepción Huista YES 1,056 3 2 5 NO 428 1 2 6 Aguacatan YES 1,056 3 2 7 NO 516 1 2 8 Todos Santos Cuchumatan YES 1,066 3 2 9 NO 257 1 2 10 Chiantla NO 1,781 44 3 11 Chimaltenango San Martin Jilotepeque YES 1,215 3 2 12 NO 471 1 2 13 Tecpán YES 1,118 3 2 14 NO 590 1 2 15 San Juan Comalapa NO 1,411 3 2 Total Population: 13,046 31 * Table extracted from the document “Title II Food Security Program Evaluation in Guatemala, MYAP 2006-2011, Final Evaluation Sampling Design” 9 Stratification agreement between FANTA, Save the Children, CRS and SHARE (MYAP Title II consortium) MYAP 2011 Final Evaluation Methodology 14 I.3 List of selected clusters / communities Table M2 presents the list of selected clusters that were presented to SHARE in the design document. Using this list, SHARE staff reviewed the viability and feasibility of visiting the selected areas, taking into account factors such as natural disasters. The evaluation was performed during the rainy season and as such some access roads were washed out, were subject to landslides or were at high risk levels. Another factor taken into consideration was that of social risk (rejection on the part of the community that could pose the risk of violence such as lynching to the group of surveyors working in the field. In the end, SHARE made a few changes to the presented list, switching some of the original communities for replacement communities. Clusters M2: Selected and visited community clusters Dept. Stratum Municipality Total Stratum Population Selected Cluster / Community Community Visited Huehuetenango 1 Santa Barbara 788 1 Tintonel YES 2 Cania Pequeño YES Replacement Community 3 Chicol NO 2 Nenton 1,099 4 Chanquejelve YES 5 Subajasun YES Replacement Community 6 Bilil NO 3 194 7 Canquintic YES 8 Tzojbal YES Replacement Community 9 Yalisjau NO 4 San Antonio Huista / Concepción Huista 1,056 10 Canton Union YES 11 Petatan YES Replacement Community 12 Ap NO 5 428 13 Canton Pozo YES 14 Ajul YES Replacement Community 15 El Tablon NO 6 Aguacatan 1,056 16 Rio San Juan Village YES 17 Climentoro Village YES Replacement Community 18 Pichiquil Village NO 7 516 19 Pajuil Pais Village YES 20 La Estancia Village YES Replacement Community 21 La Barranca Village NO MYAP 2011 Final Evaluation Methodology 15 Dept. Stratum Municipality Total Stratum Population Selected Cluster / Community Community Visited 8 Todos Santos Cuchumatan 1,066 22 Chicoy YES 23 San Martin YES Replacement Community 24 Tres Cruces NO 9 257 25 Las Lajas YES 26 Laguna Seca YES Replacement Community 27 Tuicobch NO 10 Chiantla 1,781 28 El Potrerillo YES 29 La Laguna YES 30 El Calvario YES Replacement Community 31 La Haciendita NO Chimaltenango 11 San Martin Jilotepeque 1,215 32 Los Osorios YES 33 Varituc YES Replacement Community 34 Choabajito NO 12 471 35 Estancia La Virgen YES 36 Rosario Conajal YES Replacement Community 37 Choatalun NO 13 Tecpán 1,118 38 Pachichiac YES 39 Pacacay Village YES Replacement Community 40 Pamezul NO 14 590 41 Paraxquin Village YES 42 Xecoxol YES Replacement Community 43 San Vicente NO 15 San Juan Comalapa 1,411 44 Simajuleu Village YES 45 Chipoc YES Replacement Community 46 Paxot NO The original design document is annexed to this document. II. Updating and Adapting the Data Collection Instruments Because the MYAP is carried out by three organizations (Save the Children, CRS and SHARE), evaluations such as the baseline and the current final evaluation are carried out jointly, with the inherent differences of each of the PVOs. Doing this in consortium and with joint agreements helps to ensure MYAP 2011 Final Evaluation Methodology 16 standardized results. Thus, the data collection instruments were defined from the beginning of the MYAP. The baseline evaluation already had validated and widely field tested instruments for gathering information. However, as each organization independently implemented its program, the technical staff of each organization met to review the baseline instruments and to determine whether to make minor changes to the format prior to gathering data for the final evaluation (given the particular information needs of each PVO). To the understanding of the consulting team, the changes made did not result in any significant variations to the original structure of the instruments. Rather, they maintained their quality of standardized instruments for data collection aimed at obtaining information and establishing the attainment of the indicators standardized by the MYAP consortium. (For instance, changes consisted of providing alternative answers to some of the questions.) As such, the consulting team that performed the evaluation already had data collection instruments with previously validated and field-tested questions at the time of the field study. In order to facilitate data gathering and input, the consultant divided the original instrument into several sections or forms, without changing the original questions. This allowed for more efficient data collection in the field and data input. The following is a complete list of the instruments, and they are also annexed to this document: 1. Form 1 Cover (filter).docx (one page) 2. Form 2 Family composition (FORM A).docx (one page) 3. Form 3 Health 0-60 (FORM B).docx (0 a <60 months) (six pages) 4. Form 4 Health 0-6 (FORM C).docx (0 a <6 months) (two pages) 5. Form 5 Health 6-24 (FORM D).docx (6 a <24 months) (two pages) 6. Form 6 Production (FORM E).docx (six pages) 7. Form 7 Marketing cover (filter-M).docx (one page) 8. Form 8 Marketing family composition (FORM F).docx (one page) 9. Form 9 Marketing (FORM G).docx (three pages) 10. Form 10 Anthropometrics (FORM anthropometrics).docx (one page) 11. Form 11 Risk management (FORM H).docx (one page) 12. Form 11a Risk management (FORM H1).docx (Community Strengthening)(three pages) MYAP 2011 Final Evaluation Methodology 17 More detailed information regarding the evaluation instruments is provided in the annex “First Progress Report: Training”. III. Field Staff Selection and Training. Whenever undertaking field research, one should keep in mind that some elements have a direct impact on the quality of information to be collected, subsequently conditioning the findings. It is therefore important to consider staff recruitment or selection, as well as other requirements such as training activities and standardization. III.1 Field staff selection and integration The staff, comprised of 21 members divided into three work groups, was recruited in accordance with the following basic characteristics: that they have a mid-level educational degree and previous experience. Surveying staff were expected to speak at least one of the Mayan languages of the regions to be visited. Another prerequisite was availability in accordance with the timetable in the initially drafted work schedule. Lastly, team members were expected to have positive attitudes toward the kind of activities that these types of quantitative studies require. The method used to document the survey responses in the field was through the instruments or paper ballots. As such, each of the three groups of surveyors had one team member identified as the editor. The role of the editor was to review and verify each and every one of the instruments completed in the field, especially in the communities, in order to ensure that these were properly and completely filled out. The editor also needed to verify the presence of the different “forms” in accordance with the information collected in each family. III.2 Training and standardization Training for the Survey Group consisted of a five-day workshop (August 8-12, 2011). Usually, in consultancies of a similar nature, the contracting organizations leave training and standardization to the consultant. However, in this case both SHARE and Save the Children requested direct participation in staff training, and to act in a supervising capacity rather than one of observation (as is usual). This was to ensure quality data collection and proper management of the information to be collected. As such and following the request of the technical staff for these organizations, the first activities in the training workshop were managed by them. There, they presented their programs and especially the questions in the instruments concerning health, production and marketing (commercialization). The first two and a MYAP 2011 Final Evaluation Methodology 18 half days of the workshop were dedicated to these topics. The remaining time was used for staff practice, and to standardize the management of the instruments as well as standardize anthropometrics for the staff designated to record weight and height. In the practice sessions for completing the instruments, techniques and activities were employed to work on direct “face to face” interviews, as this was the method to be used in the field. The training workshop was divided into three significant sections: Part One: Generalities and knowledge of the topics (managed by SHARE and Save the Children technical staff) Part Two: Familiarization with the instruments Part Three: Practice and standardization The second part of the workshop was used to review the data gathering instruments, and instruction was provided for the completion of each of the forms. In this way, the survey team could familiarize itself with the order and sequence of the questions, and the correct way to record the information (codification of the recorded information). The third part of the workshop took on the technical aspects of the face to face interview format (in pairs and systematically), and incorporated respective practice. This phase had two main purposes: a) Instructing the staff on the process of applying the data collection instruments b) Evaluating the learning level of each of the workshop participants with regards to proper instrument management. Determining the degree to which each of the team members learned enabled timely constructive feedback as to the correct application of each of the data gathering instruments. The final two days of the workshop was used to standardize the basic survey terminology in Mayan languages (Quiché, Ixil, Mam, K'aqchikel), and to apply the data collection instruments in these languages using the systematized interview. This was repeated until the teams reached a degree of standardization similar to the one demonstrated in the application of the instruments in Spanish. MYAP 2011 Final Evaluation Methodology 19 The third part of the workshop was also used to practice three activities for the standardization of weight and height for children under five10, in order to ensure that the staff was standardized prior to taking and recording these anthropometric measurements. In these exercises, Salter brand hanging scales in perfect working order were used. These had a weight capacity of 25 kilograms with “calibration/sensitivity=0.1kg (100gr)”. These were the same scales to be used in the field, as well as the measuring rods to measure children horizontally or vertically. The standardization of these methods was evaluated using the Habicht11 method. IV. The Field Study Upon completion of the training, the consulting team reviewed and defined the field logistics in accordance with the evaluation requirements. Field schedules and working routes were established according to the selected communities. SHARE helped to establish routes and to convene the clusters to be interviewed in advance. IV. 1 Field work phase Field activities were carried out in accordance with the plans made during the training and logistics phase. A. Introductions and requests for community cooperation Prior to gathering information in each community, each field team sought the support of local authorities/leaders. These mainly consisted of SHARE or Save the Children technical staff, institutional facilitators, community facilitators, lead mothers, COCODE members and auxiliary mayors. These individuals served as facilitators during the field phase and provided guidance within the community, served as translators and/or helped contact the community members. B. Survey team management The field teams began their part of the evaluation on August 13, 2011, arriving in the department of Huehuetenango to begin working in SHARE communities. Surveying in Huehuetenango municipalities took place from August 14 through August 20. Three field teams worked simultaneously (Groups 1, 2 and 3), so that three communities were visited per day. On September 4 the teams were transferred to 10 This practice was carried out with aid from the Integrated Attention Centers Program, which facilitated the entry of our staff to the center located in zone 13 (see annex 5 for a copy of the letter requesting cooperation). 11 The purpose of this standardization was to determine the degree of precision and accuracy with which participants weighed and measured the children under five. MYAP 2011 Final Evaluation Methodology 20 the Chimaltenango communities, where field work began on September 5 in the municipality of Tecpan. The work method described above was also used in Chimaltenango. The last community visited was Varituc in the municipality of San Martin Jilotepeque on September 8, and that same day the teams returned to headquarters. C. Linguistic Aspects Previous familiarization with the geographic regions and especially linguistic contexts was a determining factor in designating each team member to each team. Each team had at least one member who spoke one of the four dominant languages for the different regions. Thus there was always at least one surveying member of the team who could speak the local language. The field supervisor was responsible for finding and placing translators as needed. Hired translators were community members such as community facilitators, lead mothers and in some cases technical staff. Prior to using their services, they were provided with a brief introduction to ensure their understanding of the instruments to be used. Prior to hiring translators the teams explored the community to determine the general language preference of the mothers to be surveyed. This included questioning the local authorities. In each household, the team introduced itself and asked the surveyed mother her language preference for the interview. Each group supervisor was responsible for assigning translators and was careful to note when and if they were necessary, staying with the group at all times during the field phase. It is important to highlight the collaboration between local SHARE technical staff and the field teams. This was instrumental in facilitating community introductions, presenting the field team and explaining the reasons for their presence in the communities to the local authorities. SHARE also helped to find local community guides and translators. D. Field information management Within each community, the first step was to select the homes to be visited, according to the total number of homes in the community and the determined sample size of 35 homes. In each community, the samples were selected by systematic random sampling using the sampling frames (community mapping diagrams). Once the surveyors and anthropometrists were established in the community, the team supervisor transferred them to the homes assigned by the sampling. MYAP 2011 Final Evaluation Methodology 21 All of the information in the field was collected in home visits. Each interview lasted between 45 minutes and 1.5 hours. This depended on introductions, the number of instruments to be used (depending on the number and ages of the children present in the home), and the need for translators. In most communities the field staff was supported by local staff, who served as guides and companions. This contributed greatly to ensuring more effective field work. IV.2 Quality control in the field IV.2.1 Internal quality control During the field phase, several mechanisms for quality control were applied, including: a. Direct supervision in the surveyed homes; the supervisor randomly selected homes to be supervised in the course of the field activities. b. In depth review of each of the completed questionnaires (editing). This activity was the responsibility of the team editors and was done while still in the respective community. IV.2.1 External quality control Throughout the field work phase SHARE local and headquarters staff supervised the three field teams. SHARE headquarters personnel who continually supervised and escorted the teams were: the Monitoring and Evaluation Coordinator, the Health and Nutrition Sub-Manager, the Agriculture Sub￾Manager and the Human Development Manager. This is the staff that oversees the implementation of the IFS Program, in conjunction with field staff in the various municipalities. SHARE headquarters staff supervised the evaluation field work and helped to improve the field teams’ performance by providing constructive criticism regarding data collection. IV.3 Results of the Evaluation IV.3.1. Health and Nutrition and Agriculture and Livestock Farming Interventions The results of the evaluation corresponding to the surveys recorded in each community are presented in Table M3, in the order that the communities were visited. MYAP 2011 Final Evaluation Methodology 22 Table M3. Summary of surveys recorded by community: Health and Nutrition and Agriculture and Livestock Farming Interventions. Department Community Survey team # of selected homes # of effective interviews Huehuetenango Chanquejelve 2 35 22 Huehuetenango Subajasun 3 35 26 Huehuetenango Canquintic 1 35 25 Huehuetenango Tzojbal 2 35 26 Huehuetenango Canton Union 1 35 21 Huehuetenango Petatán 3 35 25 Huehuetenango Canton Pozo 3 35 25 Huehuetenango Ajul 1 35 28 Huehuetenango Chicoy 2 35 27 Huehuetenango San Martin 3 35 26 Huehuetenango Las Lajas 2 35 23 Huehuetenango Laguna Seca 1 35 29 Huehuetenango El Potrerillo 2 35 28 Huehuetenango La Laguna Paquix 1 35 24 Huehuetenango El Calvario 3 35 28 Huehuetenango Tintonel 1 35 24 Huehuetenango Canjá Chiquito 3 35 16 Huehuetenango Rio San Juan Village 3 35 24 Huehuetenango Pajuil Pais Village 1 35 21 Huehuetenango La Estancia Village 2 35 21 Huehuetenango Climentoro Village 2 35 14 Chimaltenango Pachichiac 1 35 28 Chimaltenango Pacacay 2 35 25 Chimaltenango Paraxquín Village 3 35 24 Chimaltenango Xecoxol 2 35 25 Chimaltenango Simajhuleu Village 3 35 26 Chimaltenango Chipoc 1 35 28 Chimaltenango El Rosario Canajal 1 35 28 Chimaltenango Estancia de la Virgen 3 35 25 Chimaltenango Los Osorios 2 35 25 Chimaltenango Varituc 1,2,3 35 32 It is important to note that, due to the dates and scheduling of the field portion of the study, several events took place in and out of the communities that at times limited data collection. Some examples include political rallies due to the election year, religious activities (activities prior to and during the Day of the Virgin’s Assumption and other religious activities particular to MYAP 2011 Final Evaluation Methodology 23 various communities), school activities for Independence Day, activities organized by the Mi Familia Progresa [a Guatemalan government program], COCODE meetings, and other community events. These events occupied the mothers outside of their homes for the better part of the day, delaying anthropometric measurements and interviews. It is likewise important to note that suburban communities presented frequent scheduling conflicts to the survey scheduling, as it was common to find the women away from their homes or unwilling to take the survey. IV.3.2. Micro-Business Development Intervention The research for this intervention took place among participating groups within the randomly selected communities. All community participants in the Micro-Business development intervention were convened for this purpose. Coordination and convening of the participants was the responsibility of SHARE’s technical teams. The results of the surveys obtained in each selected community are presented in Table M4, according to the order in which the communities were visited during the field study. Table M4. Summary of surveys obtained per community, Micro-Business Development Intervention. Department Community Survey Team # of Marketing Ballots Huehuetenango Las Lajas 2 14 Huehuetenango La Estancia Village 2 10 Chimaltenango Pacacay 2 10 Chimaltenango Simajhuleu Village 3 10 Chimaltenango Los Osorios 2 5 A total of 5 communities were sampled for this component, and a total of 49 surveys were completed. IV.3.3. Capacity Building (Risk Management) The Risk Management intervention was evaluated using questionnaires H and H1. The first was applied in 31 communities that comprised the sample and the second was used in three communities with sentinel sites. It should be noted that the sample of 31 communities included some communities in which the intervention was not implemented. MYAP 2011 Final Evaluation Methodology 24 The team interviewed community leaders (COCODEs). The field supervisor of each team was responsible for coordinating and convening the people to be interviewed, with the support of SHARE technical staff. In the case of the communities with sentinel sites, the survey activities were carried out in locations specified by the community leaders in order to observe the sites. This included a review of the documents that support the intervention and its implementation. Table M5 includes the names of the communities in which sentinel site information was compiled. Table M5. Summary of surveys completed in Sentinel Sites per community. Department Community Survey team Community Bulletin Boards Huehuetenango Chanquejelve 2 1 Chimaltenango Pacacay 2 1 Chimaltenango Los Osorios 2 1 The information compiled with both of these questionnaires provides the answer to the question posed by the indicator, “#of communities with functioning monitoring and early warning systems”. V Processing, data analysis and report preparation The data that was collected in the field was entered using EPI INFO version DOS 6.04d software. The information was entered using double entry of each of the completed questionnaires or instruments. Each of the double entries was performed by a different data entry operative. EPI INFO incorporates a function to ensure quality control during input, called CHECK. This function was enabled and thus allowed to check for errors such as inadvertent “jumping” between variables, extreme or invalid values (for instance, only two codes are accepted for sex: 1 = man and 2 = woman; the software did not allow the input of any other number). Once the mutually independent double entry was completed, these were compared through a process known as validation (using the software VALIDATE). Any discrepancies or errors in the file were identified and were corrected in both input files (the first and second). Once the files had been corrected, they were validated again. If errors were found, they were again identified and corrected. The process was repeated as many times as necessary until the two files were identical (without inconsistencies or errors). Anthropometric values were calculated using EPI INFO’s EPINUT module. This software provides Z-scores according to NCHS standards. Z-scores according to WHO standards were also calculated using WHO ANTHRO software version 3.2.2. It is important to note that upon processing the data, both software MYAP 2011 Final Evaluation Methodology 25 programs identified extreme cases or “outliers”. The ANTHRO program eliminates the values for these children, while NCHS software requires manual elimination through the consultant’s programming. Because there are children under five with extreme measurements in weight or height for their age, and because there is rarely a case in which the extreme measurement is for both weight and height, sample sizes for anthropometric indicators are not homogenous, nor are the indicators measured by the different software programs homogenous, because of their differing standards. Once the databases were free of errors, programs were created to construct the indicators (see the annexed databases for these programs; they have a PGM extension and are accessible through Word). EPI INFO’s CSAMPLE was used for statistical analysis. This program calculates percentages, averages and variable measurements (including confidence intervals), considering the sampling design. In this case, three variables were considered: STRATUM, CLUSTER and WEIGHTING. The strata is the same as is described in the design document, so each survey is easy to locate within its corresponding stratum. By the same token the CLUSTER corresponds to primary or community cluster, in other words, the community. WEIGHTING was calculated taking into consideration probability of selection (the same calculations that were made in the baseline): Weighting = 1 / probability of selection Where: Probability of selection = Prob. Community selection X Prob. Selecting a home in the community For the preparation of this report, SHARE and Save the Children were presented with a draft that identified the tables of indicators to be reported. MYAP 2011 Final Evaluation Results 26 This section details the results of each of the program indicators as measured by the study. The information is organized into thematic tables12. The tables present the information for the entire population as well as disaggregated by communities with/without the program’s agricultural farming interventions. The tables present the values for the sample (percentages and averages), as well as the values for the population (with a 95% confidence interval). The tables also detail the sample size with which each indicator was calculated, and the number of subjects in the sample that meet the indicator. It is important to note that due to the fact that the statistical analysis was performed considering the sampling design (stratified by cluster), it was necessary to weight each observation against its probability of selection. As such, the percentages don’t correspond to a simple division of the number of cases that meet the indicator by the sample size. For instance in the case of the indicator: Percentage of children under five with chronic malnutrition (Z < -2SD height/age) in communities with the program’s agricultural interventions. The finding is 58.1%, though the sample size is 389 children under five, and of these 223 meet the indicator of having less than two standard deviations. Sample indicator Sample Size Number of Cases % Percentage of children aged 0 to 59.99 with <2.0 SD height/age, NCHS 389 223 57.2% The operation of 223 divided by 389, equals 57.3%, and not 58.1%. The reason for this is because the 58.1% is calculated taking into consideration the sampling design, while the value of 57.3% does not consider the sampling design. As such, the latter is a skewed value (by analysis). The results are in five different sections of the document: III.A MYAP Indicators This section presents a summary table with the results of the main program indicators. The values are shown in contrast with regards to goals and values found in the baseline. 12 The tables gather all of the indicators of a particular theme. MYAP 2011 Final Evaluation Results 27 III.B 2011 Results Tables present summarized indicators, as well as a narrative of the main findings (discussion of the results). III.C Detailed 2011 Results The results of the indicators are presented in detail, for the total as well as disaggregated population (populations with/without the agricultural component). The tables include sample sizes as well as confidence intervals. This section is annexed to this document (Annex 1). III.D Design Effect This is detailed in Annex 2, in which the sampling design effect of each of the main indicators is presented. III.E 2011 Results by Municipality This section presents the results of the indicators disaggregated by municipality. This sections IS NOT PRESENTED as a part of this report, rather it is an attached document that contains the EPI INFO analysis outputs. This disaggregation was requested by SHARE, for use of the information for programming purposes. Confidence Intervals The tables present confidence intervals, which provide the values between which we find the value estimated for the total population. In other words, the percentages presented in the tables correspond to the value obtained from the sample, which is also the value for the population. However, this value doesn’t necessarily correspond to the value in the total population. Rather, the population value is calculated with error and is therefore presented as an interval. This interval presents the lowest and highest values, between which the values for the population can be found. For example, the Percentage of children aged 0 to 59.99 months with <2.0 SD weight for age, NCHS, is 29.1% for the sample, and in the total population may be as low as 24.8% or as high as 33.5%. Percentage of children aged 0 to 59.99 months with <2.0 SD weight for age, NCHS 29.1% ( 24.8, 33.5) MYAP 2011 Final Evaluation Results 28 III.A MYAP Indicators Table III.A.1 MYAP indicators and their goals Thematic Area Code 13 Indicators Baseline value GOAL 2011 Percentage (Confidence Interval. 95%) Anthropometrics in children under 5 years 1a Percentage of children aged 6 to 59.99 months with <2.0 SD height for age, NCHS 69% 66% 62.5% (55.9, 69.0) 1a Percentage of children aged under 5 with <2.0SD height for age, NCHS 64% 61% 57.2% (51.2, 63.1) 1b Percentage of children aged 0 to 59.99 months with <2.0 SD height for age, WHO 69% N/A14 65.8% (60.3, 70.2) 2a Percentage of children aged 0 to 59.99 months with <2.0 SD weight for age, NCHS 35% 30% 29.1% ( 24.8, 33.5) 2b Percentage of children aged 0 to 59.99 months with <2.0 SD weight for age, WHO 28% N/A4 22.2% (18.7, 25.7) Anthropometrics in children under 36 months 2a Percentage of children aged 0 to 35.99 months with <2.0 SD weight for age, NCHS 35% 28% 26.7% (21.7, 31.8) 2b Percentage of children aged 0 to 35.99 months with <2.0 SD weight for age, WHO 27% N/A4 18.7% ( 15.2, 22.1) Household dietary diversity and food scarcity in the household 4 Number of food groups consumed in the household in the preceding 24 hours. 4 Food groups 6 Food groups 7.3 Food groups (7.1, 7.4) 5 Average number of months out of the year with adequate provisions of food in the household. 11 months 11 months 11.0 months (10.8,11.2) Child feeding 8 Percentage of infants aged 0 to 5.99 months who have been exclusively breastfed in the preceding 24 hours. 71% 77% 74.7% (64.4, 85.0) Warning signs of childhood illness. 18 Percentage of mothers and caregivers with children aged 0 to 35.99 months who recognize at least two warning signs of childhood illness that indicate the need to seek health services. 36% 50% 75.4% (70.1,80.8) Maternal health and neonatal warning signs. 12 Percentage of mothers with children aged 0 to 35.99 months who recognize at least two warning signs in pregnancy that indicate the need to seek health services. 13% 34% 71.3% (65.0,77.6) 15 Percentage of mothers with children aged 0 to 35.99 months who recognize at least two neonatal warning signs (<28 days) that indicate the need to seek health services. 15% 35% 61.1% (53.8,68.5) Agricultural and livestock farming practices 22 Percentage of farmers who adopt at least two improved agricultural practices. 18% 23% 30.9% (25.5, 36.3) 23 Percentage of farmers who adopt at least two improved livestock practices 9% 14% 42.0% (35.5, 58.5) Income improvement 30 Percentage of farmers who adopt at least two improved formal marketing practices 17% 22% 86.7% (71.4, 100) (Not comparable to the baseline) 13 Codification of the variables subject to research 14 Note: In anthropometric indicators, the ascribed goals correspond only to the values estimated in NCHS standards, and not WHO standards. MYAP 2011 Final Evaluation Results 29 Thematic Area Code 13 Indicators Baseline value GOAL 2011 Percentage (Confidence Interval. 95%) Local capacity building 31 Number of communities with functioning early warning systems for Food and Nutritional Security 0 84 72* (48, 93) * According to SHARE monitoring information, the exact amount is 91 communities. III.A.1 MYAP Indicators and their Goals Main Findings: Table III.A.1 illustrates that the populations who are in the implementation area of SHARE’s MYAP now present a different situation than before the program, as most of the indicators demonstrate an improvement. Childhood indicators are particularly highlighted: reduced rates of chronic malnutrition in children under five (height for age at less than two standard deviations), from 69% to 62.5% (P<0.05), and in children aged 6 to 59.99 months (height for age at less than two standard deviations) from 69% to 62.5%, and reduced rates of general malnutrition in children under 36 months (weight for age at less than two standard deviations) from 35% to 26.7% (P<0.05). This demonstrates that the established goals for the reduction of chronic and general malnutrition rates were met. MYAP 2011 Final Evaluation Results 30 Graph 1: Malnutrition in the final and baseline evaluations Dietary diversity was clearly increased from the baseline value, from an average consumption of four food groups to an average of 7.3 groups, exceeding the established goal of six groups (P<0.05). Knowledge of warning signs in pregnancy, neonatal infants and children greatly exceeded the values found in the baseline, and as such also exceeded the goals (P<0.05). The percentage of women who acquired knowledge of warning signs easily doubled or even quadrupled baseline values, and as such now 71.3% of women recognize warning signs in pregnancy, 61.1% recognize neonatal warning signs, and 75.4% recognize warning signs of childhood illness. 74% of infants younger than 6 months were exclusively breastfed, while in the baseline, that value was 71%. However, due to having such a small sample to measure this indicator, it was not possible to establish a change of sufficient statistical significance from the baseline. Therefore it was also impossible to evaluate whether the indicator goal was met. Regardless, among the populations where SHARE works, the percentage of infants younger than 6 months who were exclusively breastfed is significantly higher than the ENSMI 2008-2009 findings for rural areas (60.4%). 0% 10% 20% 30% 40% 50% 60% 70% NCHS% children under 5 with chronic malnutrition WHO% children under 5 with chronic malnutrition NCHS% children under 5 with general malnutrion WHO% children under 5 with general malnutrition NCHS% children under 3 with general malnutrition WHO% children under 3 with general malnutrition Baseline Final MYAP 2011 Final Evaluation Results 31 Agricultural and livestock practices have also improved from the baseline, and these indicators also exceeded the proposed goals (P<0.05). Now, 30.9% of agricultural farmers implement improved agricultural practices, and 42% of livestock farmers implement improved practices. With regards to the implementation of at least two practices in formal marketing, at the time of the baseline 17% of farmers implemented these measures. Accordingly, a goal of 22% was established for this indicator. SHARE developed this initiative with a small number of farmers, around 500 individuals, and contemplated a small amount of advocacy among participating populations. Given this, the indicator does not adequately demonstrate the effort and accomplishments of these participants. The sample for this indicator is a sub-sample of the evaluation sample15, which is nonetheless representative due to the random sampling of communities. In this case, a sample of 43 participants was surveyed, showing that the program has accomplished that 86.7% of those surveyed adopted at least two formal marketing practices, and that the rate among the total participating population is at least 71.4%. 15 FANTA and the three PVOs: SHARE, Save the Children and CRS, agreed in meetings held in July 2011, that the measure of this indicator would be carried out using a sub-sample of participants, and not using the populational sampling used to measure the other indicators. MYAP 2011 Final Evaluation Results 32 III.B 2011 Results III.B.1 Anthropometrics in children under 5 The sample size of children under 5 was 1,046. Because some of these presented extreme values in their weight and/or height measurements in relation to their age or the relationship between them, these were identified as outliers. The software programs flag these values, and the ANTHRO program eliminates them. For this reason, the total sample size is less than 1,046 children, and the indicators present different sample sizes, depending on the outlying measurement. Table III.B.1a Anthropometrics in children under 5 Results Indicator Code Name Anthropometrics according to NCHS standards 1a Percentage of children of 0 to 59.99 months with chronic malnutrition (Z < -2SD height/age) 57.2% 1a Percentage of children of 6 to 59.99 months with chronic malnutrition (Z < -2SD height/age) 62.5% 1a1 Average Z score for height/age in children of 0 to 59.99 moths. -2.16 SD 1a1 Average Z score for height/age in children of 6 to 59.99 months. -2.28 SD 2a Percentage of children under 5 with general malnutrition (Z < -2SD weight/age) 29.1% 2a1 Average Z score for weight/age in children under five -1.37 SD Anthropometrics according to WHO standards 1b Percentage of children under 5 with chronic malnutrition (Z < -2SD height/age) 65.8% 1b1 Average Z score for height/age in children under five -2.35 SD 2b Percentage of children under 5 with general malnutrition (Z < -2SD weight/age) 22.2% 2b1 Average Z score for weight/age in children under five -1.21 SD Table III.B.1b Anthropometrics in children under 5 years by age group Results Indicator 0 to < 36 months 36 months to < 60 Code Name months Anthropometrics according to NCHS standards 1a Percentage of children under 5 with chronic malnutrition (Z < -2SD height/age) 50.5% 68.9% 1a1 Average Z score for height/age in children under five -2.00 SD -2.44 SD 2a Percentage of children under 5 with general malnutrition (Z < -2SD weight/age) 26.7% 33.4% MYAP 2011 Final Evaluation Results 33 III.B.1 Anthropometrics in children under five years Main Findings: Tables III.B.1a and III.B.1b (see Annex 1) show that chronic and general malnutrition rates are comparable among children in communities with agricultural interventions and those in communities without agricultural interventions (in accordance with NCHS standards), with chronic malnutrition at 62% and general malnutrition at 29%. However, acute malnutrition is lower in communities with the agricultural intervention than in communities without the agricultural intervention. It is also noteworthy that according to WHO standards, there was likewise no statistical difference in malnutrition rates among these strata (with or without agricultural intervention). The differences between the two systems for malnutrition classification are significant because the WHO system gives higher values than the NCHS system. 2a1 Average Z score for weight/age in children under five -1.22 SD -1.62 SD Anthropometrics according to WHO standards 1b Percentage of children under 5 with chronic malnutrition (Z < -2SD height/age) 60.0% 75.9% 1b1 Average Z score for height/age in children under five -2.22 SD -2.57 SD 2b Percentage of children under 5 with general malnutrition (Z < -2SD weight/age) 18.7% 28.3% 2b1 Average Z score for weight/age in children under five -1.03 SD -1.52 SD MYAP 2011 Final Evaluation Results 34 It is important to highlight the low numbers of children with severe acute malnutrition (lower than or equal to three standard deviations), being only a total of 5 cases according to NCHS standards and 16 according to WHO standards. Graph 2: Malnutrition prevalence in the study The National Survey for Maternal/Children’s Health for 2008-2009 (ENSMI) presented malnutrition results in accordance with WHO standards (for children aged 3 to 59 months). In the specific areas of Chimaltenango and Huehuetenango, the results for chronic malnutrition were 61.2% and 69.5% respectively, and 14.5% and 20.8% for general malnutrition respectively. These values coincide with the values found by this study in the evaluated populations. The ENSMI also presents national values for chronic and general malnutrition in accordance with WHO as well as NCHS standards, and it is worth noting that the population in this evaluation presents much higher values than those found by the ENSMI for national levels (see Graph 2a). Tables III.B.1b and III.C.1b (Annex 1) also show a series of important differences in terms of malnutrition rates by age group in accordance with NCHS standards. In this case, the rates of chronic malnutrition are lower in younger children (under 36 months), with a rate of 50.5%, while the older group (over 36 months) presented a rate of 68.9%. This is a statistically significant difference (P < 0.05) that demonstrates that children older than 36 months are 2.1 times more likely (odds ratio) to suffer from malnutrition than children under 36 months. This observation is consistent with the values that are supported by the average Z score. 0% 20% 40% 60% 80% Chronic NCHS Chronic WHO General NCHS General WHO 36 months to < 60 months 0 to < 36 months Under 5 years MYAP 2011 Final Evaluation Results 35 Graph 2a: Malnutrition prevalence found in this study, compared to the 2009 ENSMI Likewise, these results show that lower acute malnutrition rates are found in younger age groups (0.5% as compared to 3.9% on older children (P < 0.05). The probability of finding children with acute malnutrition is 8.2 times more likely (odds ratio) in children who are over 36 months. The results in this same table show that rates of malnutrition found in accordance with WHO anthropometric standards were lower (P<0.05) for the age group of 0 to < 6 months, a finding that is consistent with each of the Z score measures, respectively. Table III.C.1c (Annex 1) presents the results of the data analysis by sex. According to the results found in accordance with NCHS standards, no difference is found between the sexes. However, using WHO standards for the analysis results in a significant difference in the rates of chronic malnutrition, where boys/girls present a higher value than girls (68.5% compared to 62.7%, P < 0.05). Boys/girls are 1.3 times more likely (odds ratio) than girls to suffer from malnutrition. This finding is unusual, and it is difficult to find an explanation. 0% 10% 20% 30% 40% 50% 60% 70% 80% Chronic NCHS Chronic WHO General NCHS General WHO 36 months to < 60 months 0 to < 36 months Under 5 years ENSMI 2009 National ENSMI 2009 Hue ENSMI 2009 Chim MYAP 2011 Final Evaluation Results 36 III.B.2 Family dietary diversity and family food scarcity Due to the relevance of food production to this theme, the results are presented disaggregated by the presence or absence of the agricultural intervention. Table III.B.2 Family Dietary Diversity Family Food Scarcity Results Indicator Population WITH Agricultural Interventions Population WITHOUT Agricultural Interventions Total Population Code Name Family Dietary Diversity 4 Average Dietary Diversity Score (average number of groups consumed) 7.5 groups 7.0 groups 7.3 groups 4a Percentage of families who ate from the group: 4a3 C: Vegetables 48.0% 42.2% 44.8% 4a4 D: Fruits 71.9% 67.5% 69.5% 4a6 F: Eggs 66.8% 54.5% 60.0% 4a8 H: Legumes / leguminous plants / dried fruits 78.9% 71.3% 74.7% Family Food Scarcity 5 Average number of months with adequate food provisioning in the home 11.1 months 10.9 months 11.0 months 5a Foods that became scarce: 5a1 A: Corn 32.8% 29.3% 30.9% 5a2 B: Beans 26.9% 23.3% 24.9% 5b Reasons for food scarcity: 5b1 A: Lack of money 29.7% 35.0% 32.7% 5b2 B: Lack of employment 26.4% 27.6% 27.0% 5b5 E: There were no crops 13.6% 15.2% 14.5% 5b6 F: The harvest was lost 9.2% 5.5% 7.2% 6 Average Reduced Severity Index16 "What did you do for food during the months of scarcity" 1.7 points 1.8 points 1.7 points 6a What was done for food during the months of scarcity: 6a2 B: Sold animals 14.6% 14.2% 14.4% 6a4 D: Borrowed 13.6% 17.2% 15.6% 6a6 F: Someone in the family went elsewhere for work 21.5% 21.3% 21.4% 16 "Severity Index" known as the Coping Strategies Index (CSI). This index has been studied to a greater degree in Africa and was created by the World Food Program (WFP) Assisted Refugees in Western Tanzania. The index is presented in a document drafted by several organizations in 200816, and is defined as the sum of considered adaptations (or strategies used to confront food crises). In Guatemala, JMatute-CIENSA has calculated the index for UNICEF and World Vision. The index has a scale of 0-20 points, where “0” means that no strategy was employed and “20” is the maximum severity having employed all of the checked strategies. MYAP 2011 Final Evaluation Results 37 III.B.2 Family Dietary Diversity and Family Food Scarcity Main Findings: Tables III.B.2 and III.C.2 (Annex 1) demonstrate that the populations with agricultural interventions have a higher dietary diversity score, as compared to populations without agricultural interventions (7.5 points versus 7.0 points respectively, P<0.05). 99.9% of the families consume cereals, and a similarly high 97.8% consume sugar or honey. The foods that follow in terms of consumption rates are fats or oils (79.3%), legumes or leguminous plants (74.7%) and fruits (69.5%). The least consumed foods are fish and seafood (3%), followed by milk and dairy products (21.7%). It is worth highlighting that the population with agricultural interventions demonstrated a significantly higher consumption of the vegetable, egg and legume food groups, than the populations without the agricultural intervention. Therefore, this may be considered one of the program’s effects. Graph 3: Average Dietary Diversity Scores Graph 4: Food Groups Consumed It may also be noted that the number of months of the year in which adequate food provisions may be counted on is eleven. However, out of the 769 surveyed homes, 437 mentioned having had adequate provisions for 12 months of the year (56.8%17). Thus, 332 homes reported at least one month of scarcity, and it was the month of July that was reported as the month with the most significant scarcity (143 out of the 332 that reported scarcity). 17 Non-weighted value 6.7 6.8 6.9 7 7.1 7.2 7.3 7.4 7.5 7.6 With Agriculture Without Agriculture Group Total Average MYAP 2011 Final Evaluation Results 38 The food that is most reported as scarce is corn (30.9%), followed by beans (24.9%) and sugar (22.5%). The main reasons given by families who report food scarcity is “lack of money” (32.7%) and “lack of employment” (27.0%). It is also important to note, however, that the reasons “no crops” or “the harvest was lost” jointly added up to 22.8%. Graph 5: Scarce foods and reasons for scarcity The severity index quantifies the adaptations the family had to make in order to overcome the food crisis caused by food scarcity. In this case, the calculated index is on a 0 to 20 point scale, where “0” means the families felt no need to adapt, and “20” means that they employed all of the means considered in the study. The results show an average of 1.7 points, with which we may conclude that the studied population did not have very pressing needs of adaptation in the last year. This in turn is explained by the fact that, as previously mentioned, the majority of homes had adequate food provisions for 12 months of that year. It is also important to note that this study found that the families with the most severe food insecurity, whose children posed the greatest malnutrition problems, were supported by additional resources (including food, supplements, medical services, fowl, etc.). These were provided with additional funds in the context of a Single Year Assistance Program for Acute Malnutrition Recovery and Agricultural Reactivation. Further, not all of the possible adaptation mechanisms were measured, which means that this index does not adequately demonstrate this population’s food insecurity. The most employed adaptation mechanism was “seeking work in another area” (21.4%), followed by “borrowing” (15.6%), and “the sale of animals” (14.4%). Significantly, 3.6% 0% 5% 10% 15% 20% 25% 30% 35% 40% Corn Beans Lack of money Lack of employment There were no crops Harvest was lost Total Without Agriculture With Agriculture MYAP 2011 Final Evaluation Results 39 families mentioned that their children had to work (Annex 1), which may be interpreted as a sign of the magnitude of the crisis the family faced, given the strong campaign against child labor. III.B.3 Information, education and communication in health and nutrition Table III.B.3 Information, education and communication in health and nutrition Code Name of the Indicator Results 7 Percentage of women who confirm having received health and nutrition counseling in the preceding month 63.4% 7a Percentage of women who confirm having received counseling from MSPAS staff and/or from the IFS Program 62.0% Women received counseling from: 7a1 A: MSPAS health staff 21.6% 7a2 B: IFS Program community staff 50.6% 7b Topics dealt with in the counseling sessions: 7b1 A: Children’s health 49.7% 7b4 D: Food and nutrition 53.8% 7b5 E: Household hygiene 36.1% 7c Ways that messages or counseling were transmitted: 7c1 A: Counseling 26.9% 7c2 B: Educational meetings and/or demonstrations 54.8% MYAP 2011 Final Evaluation Results 40 III.B.3 Information, education and communication in health and nutrition Main Findings: 63.4% of the women confirmed having received health and nutrition counseling in the last month. It is evident that having received said counseling was unrelated to the disaggregation factor of whether or not a population participated in agricultural interventions (see Table III.C.3 in Annex 1). This makes it clear that the health and nutrition intervention was implemented equally among both populations, which was to be expected. There is a relatively high percentage of women who confirm having received counseling from qualified staff from either the MSPAS or the IFS Program, the latter having been the source of the majority (50.6%). The main counseling topics were “nutrition and feeding” (53.8%) and “children’s health” (49.7%). The main method employed for the transmission of these topics was through educational meetings and/or demonstrations (54.8%). Graph 6: Information, education and communication in health and nutrition 0% 10% 20% 30% 40% 50% 60% 70% MYAP 2011 Final Evaluation Results 41 III.B.4 Child Feeding Table III.B.4 Child Feeding Code Indicator Results Breastfeeding in infants under 6 months 8 Percentage of infants from 0 to < 6 months that were exclusively breastfed in the preceding 24 hours 74.7% Feeding in children aged 6 to 24 months 9 Percentage of women with children aged 6 to < 24 months who have breastfed their children 94.8% 9a Percentage of women with children aged 6 to < 24 months who are breastfeeding their children 79.6% 10 Percentage of mothers or caregivers of children aged 6 to < 24 months who know about and practice good weaning / complimentary feeding practices (having started complimentary feeding when the child was six months old) 87.1% 10a Average age at which children aged 6 to < 24 months started complimentary feeding 6.6 months III.B.4 Child Feeding Main Findings: The study found that 74.7% of infants are exclusively breastfed among the surveyed population in SHARE’s IFS Program. The rate for the total population could be as low as 64.4% and as high as 85% (confidence interval). The rate of exclusive breastfeeding found is significantly higher than the national average for rural populations (ENSMI 2009, 60.4%). With regards to general breastfeeding, with or without complimentary feeding in infants under 6 months, ENSMI reports rates of 96.6% for rural populations, which coincides with the rates found in this evaluation (97.8%). 94.8% of children between the ages of 6 and 24 months have been breastfed, and the percentage of this population currently being breastfed is 79.6%. In other words, almost 20% of children between the ages of 6 and 24 months are no longer being breastfed. The recommended age to begin complimentary feeding in infants is 6 months. The study found that 87.1% of the women surveyed implement this practice with their children. The average age at which mothers began this practice was 6.6 months. MYAP 2011 Final Evaluation Results 42 Graph 7: Child Feeding III.B.5 Warning signs in maternal, neonatal and children’s health Table III.B.5 Warning signs in maternal, neonatal and children’s health Results Code Indicator 12 Percentage of women who recognize at least two warning signs in pregnancy 71.3% 13 Percentage of women who had warning signs in pregnancy Of those who mentioned recognizing at least one warning sign 43.7% 14 Percentage of women who sought counseling or treatment because of a warning sign during pregnancy 90.4% 14a Where the counseling was sought 14a1 Hospital, Health Center/Post, Private Clinic 86.8% 14a2 MSPAS Health promotion convergence center, midwife 37.4% 15 Percentage of women who recognize at least two neonatal or newborn danger signs (under 28 days) 61.1% 16 Percentage of women who confirm having had a newborn with danger signs Of those who mention recognizing at least one sign 46.7% 0% 20% 40% 60% 80% 100% Exclusive breastfeeding < 6 months Breastfeeding < 6 months Have been breastfed 6-24 months Being breastfed 6-24 months Began complimentary feeding at 6 months MYAP 2011 Final Evaluation Results 43 Table III.B.5 Warning signs in maternal, neonatal and children’s health Results Code Indicator 17 Percentage of women who confirm seeking treatment when their newborn demonstrated danger signs 96.2% 17a Where they sought treatment 17a1 Hospital, Health Center/Post, Private Clinic 87.6% 17a2 MSPAS Health promotion convergence center, midwife 20.9% 18 Percentage of women who recognize at least two warning signs that indicate that the child needs medical treatment when ill 75.4% 19 Percentage of mothers who confirm that one of their children has shown warning signs during illness Of those who mention recognizing at least one sign 75.3% 20 Percentage of women who confirm having sought treatment when their child demonstrated warning signs 98.7% 20a Where they sought treatment 20a1 Hospital, Health Center/Post, Private Clinic 90.9% 20a2 MSPAS Health promotion convergence center, midwife 18.1% III.B.5 Warning signs in maternal, neonatal and children’s health Main Findings: The study found that 71.3% of women recognize at least two warning signs in pregnancy. Of those who recognize at least one sign, 43.7% mentioned having experienced one of the signs, and of them, 90.4% sought help (either counseling or treatment). A high percentage (86.8%) of these women sought assistance with qualified personnel (a hospital, health center/post or private clinic), and 37.4% sought assistance locally (in a convergence center, with a health promoter or with a midwife). The study also found that 61.1% of women recognize at least two warning signs in neonatal infant health. Of those who recognize at least one, 46.7% confirmed having witnessed a warning sign in their newborn, and almost all of these (96.2%) sought assistance. Of these, 87.6% sought assistance with qualified personnel (a hospital, health center/post or private clinic), while 20.9% sought assistance locally (in a convergence center, with a health promoter or with a midwife). Of the women surveyed, 75.4% recognize at least two warning signs when their child is ill. Of those who recognize at least one sign, 75.3% mentioned having witnessed a warning sign in their child, and of them, nearly all sought help (98.7%), seeking assistance mainly with qualified personnel (90.9%), while only 18.1% sought assistance locally. MYAP 2011 Final Evaluation Results 44 Graph 8: Knowledge of warning signs in pregnancy, neonatal and children’s health 0% 20% 40% 60% 80% 100% Sought assistance locally Sought assistance with qualified personnel Experienced a warning sign in pregnancy Recognizes at least two warning signs in pregnancy Sought assistance locally Sought assistance with qualified personnel Witnessed a warning sign in their newborn Recognizes at least two warning signs in neonatal health Sought assistance locally Sought assistance with qualified personnel Witnessed a warning sign in their child Recognizes at least two warning signs in children's health MYAP 2011 Final Evaluation Results 45 III.B.6 Water Treatment Table III.B.6 Water Treatment Results Code Indicator 21 Percentage of families that practice some form of water treatment (boiling, chlorinating, filtering or using the SODIS method) 96.5% Percentage of families who treat by: 21a Boiling 93.9% 21b Filtering 17.6% III.B.6 Water Treatment Main Findings: The microbiological qualities of water are important to staying healthy. It has been demonstrated that when water is contaminated by pathogens associated with diarrhea, children’s nutritional health bears a significant negative impact. This is even more exacerbated when the pathogen is something as virulent as, for instance, cholera, which could be fatal. This is even more of a risk among vulnerable populations who are living in poverty and with poor nutritional health. Over the past few years in Guatemala, the Nutritional Institute for Central America and Panama (INCAP) has studied the relationship between diarrhea episodes in children and nutritional states, demonstrating how even non-severe diarrhea episodes that are frequently repeated over long periods of time can negatively affect children and their nutritional states. As one of the main vectors for these pathogens is contaminated water, it is very important to ensure that household drinking water is treated prior to consumption. This study found that 96.5% of families treat their water. Of these, 93.9% boil it, 17.6% filter it, 6% use the SODIS method, and 1.3% use chlorine (see Annex 1). MYAP 2011 Final Evaluation Results 46 III.B.7 Agricultural and Livestock Practices A total of 769 homes were selected, visited and then surveyed. Of this total, some of the families do not practice agricultural or livestock farming. Therefore, the sample sizes shown in this section are smaller than in previous sections. Table III.B.7 Agricultural and Livestock Practices Results Indicator Population WITH Agricultural Interventions Population WITHOUT Agricultural Interventions Total Code Name Population Agricultural Practices Evaluated only among families who practice agriculture 22 Percentage of farmers who adopt at least two improved agricultural practices 41.1% 22.4% 30.9% 22a Adopted agricultural practices: 22a1 Percentage of farmers who adopt good storage practice (silos) 20.5% 21.3% 21.0% 22a3 Percentage of farmers who adopt the practice of organic fertilization 33.5% 25.0% 28.9% 22a4 Percentage of farmers who adopt the practice of crop diversification 36.1% 16.7% 25.6% 22a5 Percentage of farmers who adopt the practice of fruit tree management 43.9% 23.0% 32.6% Livestock practices Evaluated only among families who employ livestock practices 23 Percentage of farmers who adopt improved livestock practices (with fowl) Meets the following two: 23a1 and 23b1 49.7% 34.9% 42.0% 23a1 Percentage of farmers who enclose their livestock (fowl) 74.2% 64.4% 69.1% 23b1 Percentage of farmers who care for their livestock: Fowl vaccinations 63.0% 51.6% 57.4% 24 Percentage of families who consume at least one food provided by their animal husbandry (fowl) 95.0% 88.3% 91.4 % 25 Percentage of farmers who sell at least one of the foods produced by their animal husbandry (fowl) 68.1% 70.7% 69.5% Farming information, education and communication (considering individuals with agricultural and/or livestock practices ) 26 Percentage of people who have received agricultural/livestock farming counseling 42.0% 15.8% 27.6% 27 Frequency of counseling: 27a Once a month 74.4% 50.7% 66.9% 28 The counseling received in the past year has been about: 28a Agricultural farming improvement 88.8% 81.0% 86.3% 28d Livestock farming improvement 89.6% 81.6% 87.1% 29 Who have provided the counseling in most recent years: 29a Organizations in support of farming or 84.7% 70.0% 80.0% MYAP 2011 Final Evaluation Results 47 Table III.B.7 Agricultural and Livestock Practices Results Indicator Population WITH Agricultural Interventions Population WITHOUT Agricultural Interventions Total Code Name Population commercialization III.B.7 Agricultural and Livestock Practices Main Findings: Given that the majority of communities in SHARE’s coverage area practice agricultural farming, good practices are to be expected even areas without the agricultural and productive component of SHARE’s program. This is clear in Tables III.B.7 and III.C.7 (see Annex 1), as they show that good practice is found both in areas that benefit from the agricultural component as well as in areas that did not. The same is true for education in agricultural and livestock topics. However, the benefits of the program in areas with agricultural and productive interventions is clear, as the percentage of farmers who have adopted at least two improved agricultural practices is significantly higher in communities with the interventions (41.1%) than in communities without the interventions (22.4%). This is due to the positive reception and implementation of the practices introduced by the program. A farmer who lives in communities with the agricultural intervention is 2.42 times more likely (odds ratio) to adopt improved agricultural practices than one who lives in a community without the intervention. This reflects the high adoption rates of two of the introduced practices. One is crop diversification, for which the rate of farmers is much higher (P<0.05) in communities with the agricultural intervention (36.1%) than in those without (16.7%). The other is adoption of fruit tree management, for which the rate of adoption is 43.9% among communities with the intervention compared to 23% in communities without the intervention. MYAP 2011 Final Evaluation Results 48 Graph 9: Agricultural and Livestock Practices Livestock practices are limited in people who have adopted an animal for the purpose of raising it. In this case, SHARE has promoted and motivated people to raise fowl. Therefore, this is the type of animal husbandry in practice among all of the families. Of this total, less than half (218) live in the communities in which SHARE has developed livestock farming activities. The study found that 49.7% of those who own fowl and who live in communities where the farming intervention is implemented have adopted two good practices: enclosure of their animals (fowl), and caring for them (vaccinations). These rates are significantly lower (P<0.05) in areas where the farming interventions are not implemented, as only 34% in these latter communities have adopted these practices. A farmer who breeds fowl and lives in a community where the program implements the farming interventions is 1.84 times more likely (odds ratio) to implement these two practices than one who lives in a community without farming interventions. The adoption of both practices is important, not only because they help to ensure productivity, but also because they maintain a cleaner and healthier home environment. Adoption of the improved practices is a reflection of the educational processes involved in the program. The rates of people who have received agricultural and livestock farming counseling is significantly higher (P<0.05) in communities that have participated in the program’s farming interventions, compared to rates in areas where the farming interventions were not implemented. It is further important to note that this counseling is more frequent in areas where the farming intervention is implemented. The 0% 10% 20% 30% 40% 50% 60% 70% 80% 90%100% Adopt at least two improved agricultural practices Adopt storage practices Adopt organic fertilization Adopt crop diversification Adopt fruit tree management Adopt good livestock practices Enclose their animals Care for their animals Consume animal husbandry products Sell animal husbandry products Total Without Agriculture With Agriculture MYAP 2011 Final Evaluation Results 49 majority of the farmers (88.8%) confirm that this counseling has been provided by organizations that support farming and commercialization, such as SHARE. Lastly, it is worth mentioning that, because the sampling for this study was based on the total population, and not on participants, it is highly probable that the indicators are not met by 100% because of the farmers surveyed who do not participate in the program. III.B.8 Income Improvement (Formal Marketing) The data for this table was obtained through a sub-sampling of the original sample. This was achieved by evaluating all of the participants in marketing or income improvement activities that were found within the five communities selected for this study. Table III.B.8 Income Improvement (Formal Marketing) Code Indicator Results 30 Percentage of farmers who adopt two formal marketing processes (ind30a1 and/or Ind30a2) 86.7% 30b Percentage of farmers who adopt two formal marketing processes (ind30a1 and Ind30a2) 86.7% Formal marketing practices employed: 30a1 Percentage of farmers who calculate production and/or commercialization 99.0% 30a2 Percentage of farmers who record production and/or commercialization 86.7% 30b1 Percentage of farmers who calculate production and commercialization 99.0% 31 Percentage of farmers that receive institutional technical assistance for formal marketing 97.2% 31a Types of businesses run by the farmers: 31a1 Agricultural 100% 31a2 Livestock 2.8% 31b Family business 3.5% 32 Percentage of farmers who implement production and/or commercialization planning 96.5% 32b Percentage of farmers who implement production and commercialization planning 85.4% Places where their products are sold: 32a1 Communal Market 12.7% 32a2 Municipal Market 12.7% 32a3 Regional market, Guatemala City wholesaler, middleman, export market 86.6% MYAP 2011 Final Evaluation Results 50 Table III.B.8 Income Improvement (Formal Marketing) Code Indicator Results 33 Percentage of farmers who express having had problems that affected his or her business 72.0% 33a Type of problem: 33a1 Lack of marketing or advising 38.3% 33a2 Lack of money, capital or raw materials 33.5% III.B.8 Income Improvement Main Findings: Unlike the previously reported indicators that were evaluated among the general population, the income improvement indicator was evaluated with a small sample of participants in marketing intervention. Thus, a total of 49 people were surveyed. The study found that 100% of these individuals had an agricultural type of business, and very few of them (5%) had other types of businesses. In terms of activities related to income improvement, 86.7% of those surveyed had adopted two formal marketing practices. These were “calculating production and commercialization” and “recording production and commercialization”. 85.4% implement production and commercialization planning. Most of these (86.6%) sell their products in markets outside of their communities, seeking larger markets (regional markets, wholesalers in Guatemala City and export markets). 72% of these farmers stated that they had had some kind of problem that had affected their business, including a lack in marketing/counseling (38.3%) and lack of capital / raw materials (33.5%). Graph 10: Formal Marketing Practices 75% 80% 85% 90% 95% 100% Adopt two formal marketing practices Calculate production/commercialization Record production / commercialization Receive technical assistance Implement production/commercialization… MYAP 2011 Final Evaluation Results 51 III.B.9 Capacity Building (Risk Management) Within its commitments, one of SHARE’s goals is to ensure that 84 of the communities that participate in the IFS Program have an early warning and alert system for potential food insecurity. In order to gauge the scope of this goal, the units analyzed are the communities rather than the women, men and children that have been evaluated for the other indicators. The sample of 31 communities visited is representative of the total number of communities in SHARE’s IFS Program. SHARE has implemented the methodology of sentinel sites in the risk mitigation intervention. One sentinel site located in a community reflects the food and nutritional security status of the other communities it covers. SHARE has implemented a total of 12 sentinel sites. Thus, a total of 84 communities should be covered in accordance with the goal of these 12 sentinel sites. However, the IFS Program monitoring and evaluation system registers that 91 communities are covered by the 12 sentinel sites18. In order to evaluate the results, the consultant calculated the average number of communities that should be covered per sentinel site, being 84/12 = 7 communities per sentinel site; and subsequently calculated the same average within the sampled communities. SHARE’s progress in building local capacities (risk management) is examined in detail with the results of the indicators described below. Indicators and respective results: Number of communities that have participated in risk management activities, per active sentinel site Table III.b.9a (Annex 1), shows that three sentinel sites were found in the 31 sampled communities and their function was verified and confirmed. Additionally, Table III.b.9 (Annex 1) shows that 18 communities have received trainings regarding community risk management. This is equal to 58.1% of the communities visited, and considering the confidence interval this rate may be as low as 39% or as high as 75%. Of the 31 communities evaluated, 18 have participated in risk management through sentinel site coverage. This information yields a calculated average of communities covered per sentinel site: 18 / 3 = 6. Considering the confidence interval of 95%, this means that there are 4 to 7.75 communities covered per sentinel site. Based on the confidence interval, it may be concluded 18 Information provided by SHARE directly to the consultant MYAP 2011 Final Evaluation Results 52 that SHARE has met its goal of having an average of 7 communities covered per sentinel site, which translates to meeting the goal of 84 communities covered program-wide. The estimated total value based on sampled communities is 72, with a confidence interval ranging from 48 to 93 communities. This interval includes the target value. Further, SHARE’s monitoring system reports a total of 91 communities covered. Number of communities that have participated in activities related to an early warning and alert system for potential food insecurity. Table III.b.9 (Annex 1) shows that 13 communities have participated in activities related to an early warning and alert system for potential food insecurity. This means 41.9% of the communities visited and 72% of communities that have been covered by sentinel sites. This reflects the communities’ acceptance, awareness and ownership of the early warning and alert system for potential food insecurity. III.B.9 Capacity Building (Risk Management) Main Findings: SHARE has met the goal of providing 84 communities with an early warning and alert system for potential food insecurity. The three sentinel sites evaluated demonstrate community leaders’ effective performance with regards to risk management training, managing the community monitoring bulletin board, use of the respective notebook, and use of the rainwater gauge, all relevant to the performance of the Monitoring Systems and Early Warning for Food and Nutritional Security. Further, according to SHARE’s monitoring and evaluation system19, 91 communities are now monitored by 12 sentinel sites. 19 Verbal information provided by SHARE to the consultant. MYAP 2011 Final Evaluation Conclusions 53 Based on the findings here presented, the study has drawn the following conclusions: 1. This study demonstrates that SHARE has performed well in terms of meeting the goals of its various interventions. 2. There has been a significant improvement in the nutrition and health indicators for the communities in which the IFS Program was implemented, as compared to the baseline findings, with the exception of the exclusive breastfeeding indicator. Among them, the following stand out: a. Reduction of chronic malnutrition in children under five by 6.8%. b. Reduction of general malnutrition in children under 36 months by 8.3% c. Improvements in the recognition of warning signs in pregnancy, neonatal and children’s health d. Adoption of improved health and nutrition practices e. Increased family dietary diversity 3. In the communities where SHARE implements its agricultural and livestock farming interventions, there was a significant increase in the adoption of good agricultural and livestock farming practices as compared to the baseline findings. This could explain the increase in family dietary diversity and in maintaining a reduced incidence of months of food scarcity. 4. At the community level, risk management has been implemented in the number of communities established in the indicator goal. In these communities, the COCODEs confirm being knowledgeable in the topic of Disaster Risk Management, and Project Profiles. They also confirm having the necessary tools to manage community projects. MYAP 2011 Final Evaluation Annexes 54 1. Design effects on MYAP 2011 Indicators Databases (annexed documents) 2. Folder with the original EPI INFO databases (REC Files) 3. Folder with the EPI INFO databases that contain the indicators (REC files), as well as the programs with which they were constructed (PGM files) MYAP 2011 Final Evaluation Annex 1 55 ANNEX 1: III.C Detailed 2011 Results. III.C.1 Anthropometrics in children under five: Table III.C.1a Anthropometrics in children under five Results Indicator Population WITH the agricultural intervention Population WITHOUT the agricultural intervention Total Population Code Name Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) Anthropometrics according to NCHS standards 1a Percentage of children under 5 with chronic malnutrition (Z < -2SD height/age) 389 (223) 58.1% (49.9, 66.4) 645 (375) 56.4% (48.0, 64.8) 1034 (598) 57.2% (51.2, 63.1) 1a Percentage of children of 6 to 59.99 moths with < -2.0 SD height/age) NCHS 342 (217) 64.9% (54.3, 75.5) 587 (367) 60.6% (52.6, 68.7) 929 (584) 62.5% (55.9, 69.0) 1a1 Average Z score for height/age in children under five 389 -2.22 SD (-2.41, -2.03) 645 -2.11 SD (-2.29, -1.93) 1034 -2.16 SD (-2.29, -2.03) 1a1 Average Z score for height/age in children of 6 to 59.99 months 342 -2.36 DE (-2.59, -2.12) 587 -2.22 DE (-2.39, -2.06) 929 -2.28 DE (-2.42, -2.14) 2a Percentage of children under 5 with general malnutrition (Z < -2SD weight/age) 389 (107) 27.5% (20.0, 35.0) 647 (202) 30.4% (25.4, 35.4) 1036 (309) 29.1% ( 24.8, 33.5) 2a1 Average Z score for weight/age in children under five 389 -1.32 SD (-1.44, -1.20) 647 -1.40 SD ( -1.57, -1.23) 1036 -1.37 SD (-1.47, -1.26) 3a Percentage of children under 5 with acute malnutrition (Z < -2SD weight/height 387 (1) 0.3% (0.0, 0.9) 645 (17) 2.86% (0.1, 5.7) 1032 (18) 1.7% (0.1, 3.3) 3a1 Average Z score for weight/height in children under five 387 0.09 SD (-0.16, 0.34) 645 -0.11 SD (-0.25, 0.03) 1032 -0.02 SD (-0.16, 0.01) Anthropometrics according to WHO standards 1b Percentage of children under 5 with chronic malnutrition (Z < -2SD height/age) 389 (264) 69.3% (62.1, 76.5) 648 (422) 62.9% (55.1, 70.9) 1037 (686) 65.8% (60.3, 70.2) 1b1 Average Z score for height/age in children under five 389 -2.38 SD (-2.60, -2.16) 648 -2.32 SD (-2.51, -2.13) 1037 -2.35 SD (-2.49, -2.20) MYAP 2011 Final Evaluation Annex 1 56 Table III.C.1a Anthropometrics in children under five Results Indicator Population WITH the agricultural intervention Population WITHOUT the agricultural intervention Total Population Code Name Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) 2b Percentage of children under 5 with general malnutrition (Z < -2SD weight/age) 391 (80) 20.4% (15.2, 25.7) 648 (158) 23.6% (18.8, 28.3) 1039 (238) 22.2% (18.7, 25.7) 2b1 Average Z score for weight/age in children under five 391 -1.17 SD (-1.36, -0.98) 648 -1.24 SD (-1.41, -1.06) 1039 -1.21 SD (-1.34, -1.08) 3b Percentage of children under 5 with acute malnutrition (Z < -2SD weight/height 389 (4) 1.2% (0.0, 3.4) 648 (18) 3.0% (0.2, 5.8) 1037 (22) 2.2% (0.2, 4.1) 3b1 Average Z score for weight/height in children under five 389 0.32 SD (-0.10, 0.73) 648 0.13 SD (-0.01, 0.28) 1037 0.22 SD (-0.26, 0.62) Table III.C.1b Anthropometrics in children under five by age groups Results Indicator 0 to < 36 months 36 months to < 60 months Code Name Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) Anthropometrics according to NCHS standards 1a Percentage of children under 5 with chronic malnutrition (Z < -2SD height/age) 670 (347) 50.5% (44.8, 56.2) 370 (254) 68.9% (60.8, 76.9) 1a1 Average Z score for height/age in children under five 670 -2.00 SD (-2.14, -1.86) 370 -2.44 (-2.62, -2.27) 2a Percentage of children under 5 with general malnutrition (Z < -2SD weight/age) 673 (194) 26.7% (21.7, 31.8) 369 (117) 33.4% (25.6, 41.2) 2a1 Average Z score for weight/age in children under five 673 -1.22 SD (-1.33, -1.11) 369 -1.62 SD (-1.77, -1.46) 3a Percentage of children under 5 with acute malnutrition (Z < -2SD 669 0.5% 369 3.9% MYAP 2011 Final Evaluation Annex 1 57 Table III.C.1b Anthropometrics in children under five by age groups Results Indicator 0 to < 36 months 36 months to < 60 months Code Name Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) weight/height (6) (0.0, 1.0) (12) (0.0, 8.0) 3a1 Average Z score for weight/height in children under five 669 0.09 SD (-0.05, 0.23) 369 -0.22 SD (-0.40, -0.03) Anthropometrics according to WHO standards 1b Percentage of children under 5 with chronic malnutrition (Z < -2SD height/age) 674 (412) 60.0% (54.6, 65.5) 369 (278) 75.9% (68.4, 83.3) 1b1 Average Z score for height/age in children under five 674 -2.22 SD (-2.40, -2.03) 369 -2.57 SD (-2.74, 2.40) 2b Percentage of children under 5 with general malnutrition (Z < -2SD weight/age) 676 (137) 18.7% ( 15.2, 22.1) 369 (101) 28.3% (20.9, 35.8) 2b1 Average Z score for weight/age in children under five 676 -1.03 SD (-1.17, -0.90) 369 -1.52 SD (-1.69, -1.35) 3b Percentage of children under 5 with acute malnutrition (Z < -2SD weight/height 674 (9) 1.0% (0.0, 2.0) 369 (13) 4.2% (0.0, 8.5) 3b1 Average Z score for weight/height in children under five 674 0.32 SD (0.12, 0.51) 369 0.04 SD (-0.23, 0.31) MYAP 2011 Final Evaluation Annex 1 58 Table III.C.1c Anthropometrics in children under five by sex Results Indicator Boys/Girls Girls Code Name Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) Anthropometrics according to NCHS standards 1a Percentage of children under 5 with chronic malnutrition (Z < -2SD height/age) 520 (301) 59.1% (51.5, 66.8) 520 (300) 55.0% (49.6, 60.4) 1a1 Average Z score for height/age in children under five 520 -2.13 SD (-2.27, -1.99) 520 -2.18 SD (-2.33, -2.03) 2a Percentage of children under 5 with general malnutrition (Z < -2SD weight/age) 521 (148) 27.3% (21.7, 32.8) 521 (163) 30.9% (26.5, 35.3) 2a1 Average Z score for weight/age in children under five 521 -1.32 SD (-1.47, -1.17) 521 -1.41 SD (-1.50, -1.31) 3a Percentage of children under 5 with acute malnutrition (Z < -2SD weight/height 520 (6) 1.0% (0.0, 2.2) 518 (12) 2.4% (0.0, 4.9) 3a1 Average Z score for weight/height in children under five 520 0.01 SD (-0.16, 0.17) 518 -0.05 SD (-0.20, 0.11) Anthropometrics according to WHO standards 1b Percentage of children under 5 with chronic malnutrition (Z < -2SD height/age) 524 (352) 68.5% (62.1, 74.9) 519 (338) 62.7% (57.6, 67.9) 1b1 Average Z score for height/age in children under five 524 -2.35 SD (-2.54, -2.17) 519 -2.33 SD (-2.48, -2.18) 2b Percentage of children under 5 with general malnutrition (Z < -2SD weight/age) 524 (119) 21.5% (16.7, 26.4) 521 (119) 22.7% (18.6, 26.7) 2b1 Average Z score for weight/age in children under five 524 -1.18 SD (-1.38, -0.98) 521 -1.23 SD (-1.33, -1.13) 3b Percentage of children under 5 with acute malnutrition (Z < -2SD weight/height 524 (10) 1.9% (0.0, 4.0) 519 (12) 2.4% (0.0, 4.9) 3b1 Average Z score for weight/height in children under five 524 0.24 SD (-0.05, 0.53) 519 0.19 SD (0.03, 0.36) MYAP 2011 Final Evaluation Annex 1 59 III.C.2 Family Dietary Diversity and Family Food Scarcity Table III.C.2 Family Dietary Diversity and Family Food Scarcity Results Indicator Populations WITH Agricultural Interventions Populations WITHOUT Agricultural Interventions Total Population Code Name Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) Family Dietary Diversity 4 Average Dietary Diversity Score (average number of groups consumed) 295 7.5 groups (7.3, 7.7) 474 7.0 groups (6.8, 7.3) 769 7.3 groups (7.1, 7.4) 4a Percentage of families who ate from the group: 4a1 A: Cereals 295 (295) 100% (-) 474 (473) 99.9% (99.3, 100) 769 (768) 99.9% (99.9, 100) 4a2 B: Roots and tubers 295 (174) 58.7% (54.5, 62.7) 474 (277) 58.7% (48.6, 68.9) 769 (451) 58.7% (52.3, 64.6) 4a3 C: Vegetables 295 (133) 48.0% (42.0, 54.0) 474 (202) 42.2% (37.0, 47.5) 769 (335) 44.8% (40.9, 48.7) 4a4 D: Fruits 295 (206) 71.9% (66.7, 77.1) 474 (312) 67.5% (63.1, 72.0) 769 (518) 69.5% (66.2, 72.8) 4a5 E: Meat, chicken and offal 295 (136) 41.5% (34.5, 48.4) 474 (213) 40.3% (33.4, 47.3) 769 (349) 40.8% (35.9, 45.8) 4a6 F: Eggs 295 (194) 66.8% (56.0, 77.5) 474 (267) 54.5% (47.3, 61.7) 769 (461) 60.0% (53.4, 66.6) MYAP 2011 Final Evaluation Annex 1 60 Table III.C.2 Family Dietary Diversity and Family Food Scarcity Results Indicator Populations WITH Agricultural Interventions Populations WITHOUT Agricultural Interventions Total Population Code Name Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) 4a7 G: Fish and seafood 295 (13) 4.5% (2.1, 7.0) 474 (13) 1.8% (0.9, 2.7) 769 (26) 3.0% (1.8, 4.2) 4a8 H: Legumes / leguminous plants / dried fruits 295 (234) 78.9% (73.9, 83.9) 474 (348) 71.3% (66.7, 75.9) 769 (582) 74.7% (71.2, 78.2) 4a9 I: Milk or dairy products 295 (69) 23.1% (16.1, 30.1) 474 (99) 20.6% (13.8, 27.4) 769 (168) 21.7% (16.9, 26.5) 4a10 J: Oils / fats 295 (229) 82.3% (74.1, 90.4) 474 (353) 77.0% (69.6, 84.9) 769 (582) 79.3% (73.8, 84.9) 4a11 K: Sugar / honey 295 (288) 98.1% (96.3, 100) 474 (452) 97.5% (95.8, 99.2) 769 (740) 97.8% (96.6, 99.0) 4a12 L: Various foods 295 (233) 76.6% (72.4, 80.7) 474 (355) 72.3% (66.7, 77.9) 769 (588) 74.2% (70.6, 77.8) Family food scarcity 5 Average number of months with adequate food provisioning in the home 295 11.1 months (10.8,11.4) 474 10.9 months (10.6,11.2) 769 11.0 months (10.8,11.2) 5a Foods that became scarce : 5a1 A: Corn 295 (98) 32.8% (25.6,40.1) 474 (149) 29.3% (21.6,36.9) 769 (247) 30.9% (25.6,36.1) 5a2 B: Beans 295 (82) 26.9% (22.8,31.1) 474 (135) 23.3% (14.9,31.7) 769 (217) 24.9% (19.9,29.9) 5a3 C: Produce 295 (16) 5.5% (0.6,10.3) 474 (55) 10.5% (5.7,15.3) 769 (71) 8.3% (4.7,11.8) 5a4 D: Greens 295 1.5% 474 4.8% 769 3.4% MYAP 2011 Final Evaluation Annex 1 61 Table III.C.2 Family Dietary Diversity and Family Food Scarcity Results Indicator Populations WITH Agricultural Interventions Populations WITHOUT Agricultural Interventions Total Population Code Name Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) (6) (0.0,3.6) (26) (3.2,6.5) (32) (1.9,4.8) 5a5 E: Sugar 295 (59) 19.9% (12.5,27.3) 474 (117) 24.5% (16.5,32.6) 769 (176) 22.5% (16.8,28.2) 5a6 F: Oil 295 (15) 6.4% (3.5,9.3) 474 (24) 5.7% (2.3,9.0) 769 (39) 6.0% (3.8,8.2) 5a7 G: Rice 295 (33) 12.0% (10.1,14.0) 474 (71) 15.2% (11.2,19.1) 769 (104) 13.8% (11.3,16.2) 5a8 H: Eggs 295 (22) 6.01% (1.4,10.7) 474 (26) 5.9% (4.0,7.9) 769 (48) 6.0% (3.6,8.3) 5a9 I: Meat 295 (37) 11.6% (6.6,16.6) 474 (71) 16.2% (9.5,22.9) 769 (108) 14.2% (9.7,18.6) 5a10 J: Other 295 (13) 4.6% (2.3,7.0) 474 (34) 8.0% (3.8,12.3) 769 (47) 6.5% (3.9,9.1) 5b Reasons for food scarcity: 5b1 A: Lack of money 295 (87) 29.7% (26.3,33.1) 474 (173) 35.0% (25.2,44.9) 769 (260) 32.7% (27.0,38.4) 5b2 B: Lack of employment 295 (76) 26.4% (18.1,34.7) 474 (131) 27.6% (18.9,36.3) 769 (207) 27.0% (20.9,33.1) 5b3 C: Large family 295 (9) 2.8% (0.0,6.0) 474 (12) 2.4% (0.6,4.1) 769 (21) 2.6% (0.9,4.3) 5b4 D: Distant market/ food supply store 295 (3) 1.5% (0.0,3.2) 474 (5) 1.2% (0.2,2.2) 769 (8) 1.4% (0.4,2.3) 5b5 E: There were no crops 295 (45) 13.6% (8.2,19.0) 474 (83) 15.2% (12.4,18.0) 769 (128) 14.5% (11.6,17.4) MYAP 2011 Final Evaluation Annex 1 62 Table III.C.2 Family Dietary Diversity and Family Food Scarcity Results Indicator Populations WITH Agricultural Interventions Populations WITHOUT Agricultural Interventions Total Population Code Name Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) 5b6 F: The harvest was lost 295 (25) 9.2% (3.8,14.6) 474 (25) 5.5% (2.9,8.1) 769 (50) 7.2% (4.4,10.0) 5b7 G: Food too expensive 295 (6) 1.5% (0.0,3.5) 474 (25) 4.7% (1.8,7.5) 769 (31) 3.2% (1.3,5.2) 5b8 H: Other 295 (5) 1.2% (0.0,3.0) 474 (5) 1.3% (0.0,2.5) 769 (10) 1.2% (0.1,2.3) 6 Average Reduced Severity Index20 "What did you do for food during the months of scarcity" 295 1.7 points (1.4,2.0) 474 1.8 points (1.4,2.2) 769 1.7 points (1.5,2.0) 6a What was done for food in the months of scarcity 6a1 A: Sold an item 295 (13) 3.5% (0.9,6.1) 474 (34) 5.8% (0.9,10.7) 769 (47) 4.7% (1.8,7.7) 6a2 B: Sold animals 295 (37) 14.6% (11.7,17.4) 474 (60) 14.2% (9.0,19.4) 769 (97) 14.4% (11.2,17.5) 6a3 C: Sold land 295 (2) 0.4% (0.0,1.0) 474 (1) 0.0% (0.0,0.1) 769 (3) 0.2% (0.0,0.5) 6a4 D: Borrowed 295 (40) 13.6% (9.4,17.7) 474 (83) 17.2% (12.0,22.3) 769 (123) 15.6% (12.0,19.1) 6a5 E: Some of the children had to work 295 (11) 4.3% (1.4,7.1) 474 (17) 3.1% (1.0,5.1) 769 (28) 3.6% (1.9,5.3) 20 "Severity Index" known as the Coping Strategies Index (CSI). This index has been studied to a greater degree in Africa and was created by the World Food Program (WFP) Assisted Refugees in Western Tanzania. The index is presented in a document drafted by several organizations in 200820, and is defined as the sum of considered adaptations (or strategies used to confront food crises). In Guatemala, JMatute-CIENSA has calculated the index for UNICEF and World Vision. The index has a scale of 0-20 points, where “0” means that no strategy was employed and “20” is the maximum severity having employed all of the checked strategies. MYAP 2011 Final Evaluation Annex 1 63 Table III.C.2 Family Dietary Diversity and Family Food Scarcity Results Indicator Populations WITH Agricultural Interventions Populations WITHOUT Agricultural Interventions Total Population Code Name Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) 6a6 F: Someone in the family went elsewhere for work 295 (64) 21.5% (14.0,29.0) 474 (125) 21.3% (14.9,27.6) 769 (189) 21.4% (16.5,26.2) 6a7 G: Reduced the number of daily meals 295 (15) 4.0% (1.5,6.5) 474 (21) 5.4% (2.6,8.2) 769 (36) 4.8% (2.8,6.8) 6a8 H: Other 295 (14) 4.3% (0.0,9.5) 474 (17) 5.2% (0.9,9.4) 769 (31) 4.8% (1.4,8.1) III.C.3 Information, education and communication in health and nutrition Table III.C.3 Information, education and communication in health and nutrition Results Indicator Population WITH Agricultural Interventions Population WITHOUT Agricultural Interventions Total Population Code Name Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) 7 Percentage of women who confirm having received health and nutrition counseling in the preceding month 295 (171) 57.7% (51.7, 63.7) 474 (315) 68.1% (55.0, 81.1) 769 (486) 63.4% (55.7, 71.2) 7a Percentage of women who confirm having received counseling from MSPAS staff and/or from the IFS Program 295 (167) 56.7% (51.5, 61.8) 474 (308) 66.4% (53.7, 79.1) 769 (475) 62.0% (54.7, 69.4) Women received counseling from: 7a1 A: MSPAS health staff 295 17.4% 474 25.0% 769 21.6% MYAP 2011 Final Evaluation Annex 1 64 Table III.C.3 Information, education and communication in health and nutrition Results Indicator Population WITH Agricultural Interventions Population WITHOUT Agricultural Interventions Total Population Code Name Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) (54) (11.1, 23.8) (96) (17.6, 32.4) (150) (16.5, 26.8) 7a2 B: IFS Program community staff 295 (138) 48.2% (45.0, 62.6) 474 (253) 52.5% (42.3, 62.6) 769 (391) 50.6% (44.9, 56.3) 7a3 C: Families, neighbors or others 295 (14) 4.1% (0.0, 8.3) 474 (24) 6.0% (1.1, 10.9) 769 (38) 5.2% (1.8, 8.6) 7b Topics dealt with in the counseling sessions: 7b1 A: Children’s health 295 (137) 44.0% (35.0, 53.0) 474 (241) 54.3% (42.6, 66.0) 769 (378) 49.7% (41.9, 57.5) 7b2 B: Healthy pregnancy 295 (24) 8.8% (3.3, 14.3) 474 (42) 10.2% (3.9, 16.5) 769 (66) 9.6% (5.3, 13.8) 7b3 C: Newborn health 295 (27) 8.8% (4.1, 13.4) 474 (60) 13.5% (9.3, 17.8) 769 (87) 11.4% (8.2,14.6) 7b4 D: Food and nutrition 295 (141) 47.6% (39.8, 55.4) 474 (264) 58.9% (48.3, 69.5) 769 (405) 53.8% (46.8, 60.9) 7b5 E: Household hygiene 295 (99) 35.7% (31.7, 39.7) 474 (166) 36.4% (27.1, 45.8) 769 (265) 36.1% (30.6, 41.6) 7b6 D: Water treatment and care 295 (44) 15.8% (11.6, 20.0) 474 (95) 21.3% (12.4, 30.2) 769 (139) 18.8% (13.6, 24.1) 7c Ways that messages or counseling were transmitted: 7c1 A: Counseling 295 (59) 15.8% (5.0, 26.7) 474 (152) 35.9% (24.2, 47.6) 769 (211) 26.9% (18.1, 35.7) 7c2 B: Educational meetings and/or demonstrations 295 (153) 52.2% (44.5, 60.0) 474 (265) 56.8% (46.2, 67.5) 769 (418) 54.8% (47.9, 61.6) MYAP 2011 Final Evaluation Annex 1 65 Table III.C.3 Information, education and communication in health and nutrition Results Indicator Population WITH Agricultural Interventions Population WITHOUT Agricultural Interventions Total Population Code Name Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) 7c3 C: Home visits 295 (24) 8.6% (5.5, 11.7) 474 (35) 6.1% (1.6, 10.6) 769 (59) 7.2% (4.3, 10.1) 7c4 D: Public media (radio, posters, fliers, billboards, TV, loudspeaker) 295 (33) 10.0% (2.9, 17.1) 474 (47) 9.6% (2.0, 17.3) 769 (80) 9.8% (4.5, 15.1) III.C.4 Child Feeding Table III.C.4 Child feeding Results Indicator Population WITH Agricultural Interventions Population WITHOUT Agricultural Interventions Total Population Code Name Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) Breastfeeding in infants under 6 months 8 Percentage of infants from 0 to < 6 months that were exclusively breastfed in the preceding 24 hours 47 (36) 80.2% (67.1, 93.3) 59 (41) 68.9% (54.9, 83.0) 106 (77) 74.7% (64.4, 85.0) 8a Percentage of women with infants aged 0 to < 6 months who are breastfeeding 47 (47) 100% (-) 59 (58) 95.5% (86.4, 100) 106 (105) 97.8% (93.4, 100) Feeding in children aged 6 to 24 months 9 Percentage of women with children aged 6 to < 24 months who have breastfed their children 108 (104) 94.4% (90.5, 98.2) 242 (231) 95.0% (91.2, 98.8) 350 (335) 94.8% (92.0,98.0) 9a Percentage of women with children aged 6 to < 24 months who are breastfeeding their children 108 (95) 85.8% (78.5, 93.0) 242 (188) 76.0% (70.6, 81.4) 350 (283) 79.6% (75.4, 83.8) 10 Percentage of mothers or caregivers of children aged 6 to < 24 months who know about and practice good weaning / 108 (99) 91.7% (88.0, 95.4) 242 (206) 84.4% (79.9, 88.9) 350 (305) 87.1% (83.8, 90.3) MYAP 2011 Final Evaluation Annex 1 66 Table III.C.4 Child feeding Results Indicator Population WITH Agricultural Interventions Population WITHOUT Agricultural Interventions Total Population Code Name Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) complimentary feeding practices (having started complimentary feeding when the child was six months old) 10a Average age in which children aged 6 to < 24 months started complimentary feeding 105 6.5 months (6.4, 6.6) 232 6.6 months (6.3, 6.8) 337 6.6 months (6.4, 6.8) III.C.5 Warning signs in maternal, neonatal and children’s health Table III.C.5 Warning signs in maternal, neonatal and children’s health Results Indicator Population WITH Agricultural Interventions Population WITHOUT Agricultural Interventions Total Population Code Name Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) 12 Percentage of women who recognize at least two warning signs in pregnancy 295 (208) 67.7% (56.6,78.7) 474 (341) 74.2% (67.6,80.7) 769 (549) 71.3% (65.0,77.6) 13 Percentage of women who had warning signs in pregnancy Of those who mentioned recognizing at least one warning sign 227 (111) 47.5% (38.6,56.6) 379 (167) 41.0% (33.9,48.1) 606 (278) 43.7% (38.0,49.4) 14 Percentage of women who sought counseling or treatment because of a warning sign during pregnancy 111 (100) 85.5% (69.1,100) 167 (156) 94.6% (91.0,98.2) 278 (256) 90.4% (82.2,98.7) 14a Where the counseling was sought 14a1 Hospital, Health Center/Post, Private Clinic 100 (82) 84.7% (74.2,95.1) 156 (131) 88.5% (83.4,93.5) 256 (213) 86.8% (81.4,92.2) 14a2 MSPAS Health promotion convergence center, midwife 100 (41) 37.2% (13.3,61.0) 156 (58) 37.5% (27.6,47.4) 256 (99) 37.4% (25.5,49.2) 14a3 Other 100 2.4% (0.0,5.4) 156 3.6% (0.0,8.1) 256 3.1% (0.2,5.9) MYAP 2011 Final Evaluation Annex 1 67 Table III.C.5 Warning signs in maternal, neonatal and children’s health Results Indicator Population WITH Agricultural Interventions Population WITHOUT Agricultural Interventions Total Population Code Name Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) (3) (5) (8) 15 Percentage of women who recognize at least two neonatal or newborn danger signs (under 28 days) 295 (189) 58.8% (44.9,72.6) 474 (293) 63.0% (56.5,69.5) 769 (482) 61.1% (53.8,68.5) 16 Percentage of women who confirm having had a newborn with danger signs Of those who mention recognizing at least one sign 221 (108) 47.3% (38.8,55.9) 347 (156) 46.2% (40.4,52.1) 568 (264) 46.7% (41.7,51.7) 17 Percentage of women who confirm seeking treatment when their newborn demonstrated danger signs 108 (104) 95.1% (89.7,100) 156 (149) 97.0% (94.9,99.2 264 (253) 96.2% (93.5,98.9) 17a Where they sought treatment 17a1 Hospital, Health Center/Post, Private Clinic 104 (90) 88.2% (83.2,88.2) 149 (127) 87.1% (81.6,92.6) 253 (217) 87.6% (83.9,91.2) 17a2 MSPAS Health promotion convergence center, midwife 104 (26) 21.0% (14.6,27.3) 149 (31) 20.9% (13.6,28.2) 253 (57) 20.9% (16.0,25.9) 17a3 Other 104 (7) 5.4% (0.8,10.0) 149 (13) 5.4% (0.7,10.2) 253 (20) 5.4% (2.1,8.8) 18 Percentage of women who recognize at least two warning signs that indicate that the child needs medical treatment when ill 295 (224) 75.6% (68.0,83.3) 474 (363) 75.2% (67.9,82.6) 769 (587) 75.4% (70.1,80.8) 19 Percentage of mothers who confirm that one of their children has shown warning signs during illness Of those who mention recognizing at least one sign 252 (181) 74.1% (69.5,78.6) 416 (311) 76.2% (72.0,80.4) 668 (492) 75.3% (72.0,80.4) 20 Percentage of women who confirm having sought treatment when their child demonstrated warning signs 181 (177) 97.2% (94.5,99.9) 311 (309) 99.8% (99.4,100) 492 (486) 98.7% (97.4,100) 20a Where they sought treatment 20a1 Hospital, Health Center/Post, Private Clinic 177 (150) 86.4% (78.1,94.8) 309 (280) 94.2% (91.1,97.2) 486 (430) 90.9% (87.1,94.7) 20a2 MSPAS Health promotion convergence center, midwife 177 20.7% 309 16.1% 486 18.1% MYAP 2011 Final Evaluation Annex 1 68 Table III.C.5 Warning signs in maternal, neonatal and children’s health Results Indicator Population WITH Agricultural Interventions Population WITHOUT Agricultural Interventions Total Population Code Name Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) (43) (12.1,29.3) (50) (10.9,21.3) (93) (13.4,22.7) 20a3 Other 177 (17) 8.5% (1.7,15.4) 309 (21) 5.6% (1.2,10.0) 486 (38) 6.8% (3.0,10.7) MYAP 2011 Final Evaluation Annex 1 69 III.C.6 Water treatment Table III.C.6 Water treatment Results Indicator Population WITH Agricultural Interventions Population WITHOUT Agricultural Interventions Total Population Code Name Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) 21 Percentage of families that practice some form of water treatment (boiling, chlorinating, filtering or using the SODIS method) 295 (286) 97.1% (95.1,99.1) 474 (452) 96.0% (94.0,98.1) 769 (738) 96.5% (95.0,98.0) Percentage of families who treat by: 21a Boiling 295 (277) 92.3% (89.4,95.1) 474 (449) 95.2% (93.0,97.3) 769 (726) 93.9% (92.1,95.7) 21b Filtering 295 (44) 16.8% (11.8,21.2) 474 (85) 18.2% (11.7,24.7) 769 (129) 17.6% (13.4,21.8) 21c Chlorinating 295 (2) 0.5% (0.0,1.3) 474 (13) 1.9% (0.4,3.3) 769 (15) 1.3% (0.4,2.2) 21d SODIS 295 (19) 9.3% (1.5,17.1) 474 (15) 3.3% (1.3,5.2) 769 (34) 6.0% (2.1,9.8) MYAP 2011 Final Evaluation Annex 1 70 III.C.7 Agricultural and Livestock Practices Table III.C.7 Agricultural and Livestock Practices Results Indicator Population WITH Agricultural Interventions Population WITHOUT Agricultural Interventions Total Population Code Name Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) Agricultural Practices Evaluated only among families who practice agriculture 22 Percentage of farmers who adopt at least two improved agricultural practices 262 (101) 41.1% (35.9, 46.2) 415 (93) 22.4% (14.1, 30.6) 677 (194) 30.9% (25.5, 36.3) 22a Adopted agricultural practices: 22a1 Percentage of farmers who adopt good storage practice (silos) 262 (57) 20.5% (10.5, 30.5) 415 (78) 21.3% (11.3, 31.4) 677 (135) 21.0% (13.8, 28.1) 22a3 Percentage of farmers who adopt the practice of organic fertilization 262 (86) 33.5% (27.3, 39.6) 415 (100) 25.0% (16.1, 33.9) 677 (186) 28.9% (23.3, 34.5) 22a4 Percentage of farmers who adopt the practice of crop diversification 262 (85) 36.1% (29.2, 42.9) 415 (67) 16.7% (8.8, 24.8) 677 (152) 25.6% (19.8, 31.5) 22a5 Percentage of farmers who adopt the practice of fruit tree management 262 (107) 43.9% (30.1, 57.7) 415 (92) 23.0% (16.1, 29.3) 677 (199) 32.6% (25.3, 39.9) Livestock practices Evaluated only among families who employ livestock practices 23 Percentage of farmers who adopt improved livestock practices (with fowl) Meets the following two: 23a1 and 23b1 218 (116) 49.7% (38.5, 60.9) 330 (117) 34.9% (26.2, 43.6) 548 (233) 42.0% (35.5, 58.5) 23a1 Percentage of farmers who enclose their livestock (fowl) 218 (161) 74.2% (66.7, 81.7) 300 (200) 64.4% (54.5, 74.2) 548 (361) 69.1% (63.3, 74.8) 23b1 Percentage of farmers who care for their livestock: Fowl vaccinations 218 (149) 63.0% (52.8, 74.6) 330 (181) 51.6% (43.1, 60.0) 548 (330) 57.4 (50.6, 64.1) 24 Percentage of families who consume at least one food 225 95.0% 362 88.3% 587 91.4% MYAP 2011 Final Evaluation Annex 1 71 Table III.C.7 Agricultural and Livestock Practices Results Indicator Population WITH Agricultural Interventions Population WITHOUT Agricultural Interventions Total Population Code Name Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) provided by their animal husbandry (fowl) (217) (90.9, 99.1) (325) (83.5, 93.1) (542) (88.2, 94.6) 25 Percentage of farmers who sell at least one of the foods produced by their animal husbandry (fowl) 225 (154) 68.1% (60.1, 76.2) 362 (265) 70.7% (59.1, 82.2) 587 (419) 69.5% (62.2, 76.7) Farming information, education and communication (considering individuals with agricultural and/or livestock practices ) 26 Percentage of people who have received agricultural/livestock farming counseling 294 (112) 42.0% (35.0, 48.9) 468 (83) 15.8% (8.4, 23.3) 762 (195) 27.6% (21.7, 33.3) 27 Frequency of counseling: 27a Once a month 112 (78) 74.4% (64.5, 84.3) 83 (42) 50.7% (24.6, 66.7) 195 (120) 66.9% (57.1, 76.7) 27b At least once every two months 112 (10) 7.4% (4.7, 9.7) 83 (14) 14.5% (1.4, 27.5) 195 (24) 9.6% (4.6, 14.6) 27c Every three months 112 (6) 4.2% (0.0, 4.2) 83 (2) 2.1% (0.0, 7.9) 195 (8) 3.5% (0.0, 7.9) 27d Less than every three months 112 (8) 7.9% (2.9, 12.9) 83 (16) 16.9% (7.3, 26.5) 195 (24) 10.8% (6.7, 14.8) 28 The counseling received in the past year has been about: 28a Agricultural farming improvement 112 (96) 88.8% (81.3, 96.1) 83 (70) 81.0% (65.7, 96.3) 195 (166) 86.3% (79.2, 93.4) 28b Business improvement 112 (6) 3.8% (0.0, 8.6) 83 (10) 10.5% (4.2, 16.9) 195 (16) 5.9% (1.9, 10.0) 28c Guidance to generate more money 112 (2) 1.8% (0.0, 5.6) 83 (3) 2.1% (0.0, 6.5) 195 (5) 1.9% (0.0, 4.5) 28d Livestock farming improvement 112 (99) 89.6% (83.6, 95.6) 83 (67) 81.6% (73.0, 90.2) 195 (166) 87.1% (82.2, 91.9) MYAP 2011 Final Evaluation Annex 1 72 Table III.C.7 Agricultural and Livestock Practices Results Indicator Population WITH Agricultural Interventions Population WITHOUT Agricultural Interventions Total Population Code Name Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) Sample Size (# of cases) % (95% CI) 29 Who have provided the counseling in most recent years: 29a Organizations in support of farming or commercialization 112 (88) 84.7% (68.2, 100) 83 (61) 70.0% (44.3, 95.6) 195 (149) 80.0% (65.6, 94.5) 29b Relatives or neighbors 112 (9) 4.9% (0.0, 11.9) 83 (10) 15.3% (5.5, 25.2) 195 (19) 8.2% (1.9, 14.4) 29c Mass media (Radio, TV, print, loudspeakers) 112 (0) 0% (-) 83 (0) 0% (-) 195 (0) 0% (-) 29d Technical operatives or local promoters 112 (26) 21.3% (13.5, 29.1) 83 (20) 21.8% (16.2, 27.5) 195 (46) 21.5% (15.8, 27.1) MYAP 2011 Final Evaluation Annex 1 73 III.B.8 Income Improvement (Formal Marketing) Table III.B.8 Income Improvement (Formal Marketing) Results Indicator Code Name Sample Size (# of cases) % (95% CI) 30 Percentage of farmers who adopt at least two formal marketing processes (ind30a1 and/or Ind30a2) 49 (44) 86.7% (71.4, 100) 30b Percentage of farmers who adopt at least two formal marketing processes (ind30b1 and Ind30a2) 49 (44) 86.7% (71.4, 100) Formal marketing practices employed: 30a1 Percentage of farmers who calculate production and/or commercialization 49 (48) 99.0% (96.7, 100) 30a2 Percentage of farmers who record production and/or commercialization 49 (44) 86.7% (71.4, 100) 30b1 Percentage of farmers who calculate production and commercialization 49 (48) 99.0% (96.7, 100) 31 Percentage of farmers that receive institutional technical assistance for formal marketing 49 (47) 97.2% (91.6, 100) 31a Types of businesses run by the farmers: 31a1 Agricultural 49 (49) 100% (-) 31a2 Livestock 49 (2) 2.8% (0.0, 8.4) 31b Family business 49 (2) 3.5% (0.0, 7.9) 32 Percentage of farmers who implement production and/or commercialization planning 49 (47) 96.5% (92.1, 100) 32b Percentage of farmers who implement production and commercialization planning 49 (42) 85.4% (73.9, 96.9) MYAP 2011 Final Evaluation Annex 1 74 Table III.B.8 Income Improvement (Formal Marketing) Results Indicator Code Name Sample Size (# of cases) % (95% CI) Places where their products are sold: 32a1 Communal Market 49 (7) 12.7% (0.0, 29.3) 32a2 Municipal Market 49 (8) 12.7% (0.0, 30.6) 32a3 Regional market, Guatemala City wholesaler, middleman, export market 49 (42) 86.6% (66.5, 100) 33 Percentage of farmers who express having had problems that affected his or her business 49 (38) 72.0% (41.7, 100) 33a Type of problem: 33a1 Lack of marketing or advising 38 (16) 38.3% (2.7, 74.0) 33a2 Lack of money, capital or raw materials 38 (13) 33.5% (0.5, 66.4) 33a3 Lack of appropriate monitoring of operational records or lack of organization on the part of the farmers 38 (0) 0% (-) MYAP 2011 Final Evaluation Annex 1 75 III.B.9 Community Risk Table III.B.9 Community Risk Results Sample Size = 31 Indicator Code Name Number of Cases % 34 Percentage of communities in which one of its leaders were trained regarding community risk management 18 58.1% 34a Percentage of communities that have a community plan for risk management 7 22.6% 34b Percentage of communities that have participated in activities related to a monitoring and warning system or Sentinel Site for food scarcity 13 41.9% 34c Percentage of communities that are knowledgeable about early warning systems or Sentinel Sites 12 38.7% 34d Percentage of communities with a community Sentinel Site (COCODE confirms that there is an information table for the Sentinel Site and/or verified the existence of the table) 8 25.8% 34e Percentage of communities with a device to read the amount of rainfall in the community 4 12.9% 34e1 Percentage of communities that have a sheet or notebook to record daily rainfall 4 12.9% Table III.B.9a Community Risk (Sentinel Sites) Information obtained at the Sentinel Sites Indicator Results Sample Size = 3 Number of cases Number of communities in which the COCODE has received training regarding risk management 3 Number of communities in which the COCODE has received training regarding COCODE functions 3 Number of communities in which the COCODE has received training regarding risk factors of food and nutritional insecurity 3 Number of communities in which the COCODE has received training regarding risk of disaster 3 Number of communities in which the COCODE has received training regarding program profiles 3 Number of communities in which the COCODE has received training regarding food and nutritional security 1 Number of communities in which the COCODE has received training regarding program management 3 Number of communities that have a community risk management plan (the document was verified) 1 MYAP 2011 Final Evaluation Annex 1 76 Table III.B.9a Community Risk (Sentinel Sites) Information obtained at the Sentinel Sites Indicator Results Sample Size = 3 Number of cases Topics in which the community is most active Children’s health, children’s malnutrition, food security Children’s education Animal disease (including birds) Environment: household hygiene (waste management) Monitoring corn prices Program management Received training materials 3 The topics that the communities found most interesting or important Program management Disaster Risk Environment COCODE functions Monitoring of bird disease Health and nutrition Have raised resources at the municipal level to meet their needs 3 Have implemented some program to the benefit of their community 3 Multi-purpose room, classrooms, highway retaining wall Have participated in some activity or training related to monitoring and warning systems or Sentinel Sites for food scarcity 3 The members of the community participate in monitoring and warning systems or Sentinel Sites for food scarcity 3 Recognize a community monitoring board 3 Verification (by observation) of an existing community monitoring board 3 Verification (by observation) of a community notebook 2 Know how to use the colored happy face stickers 3 Know what the early warning system and Sentinel Site are for 3 MYAP 2011 Final Evaluation Annex 1 77 Table III.B.9a Community Risk (Sentinel Sites) Information obtained at the Sentinel Sites Indicator Results Sample Size = 3 Number of cases Have the community Sentinel Site monitoring board (verified by observation) 2 Have a notebook to record the Sentinel Site information 2 Know what a rain gauge is 3 Have a rain gauge installed 3 Have a sheet or notebook to record daily rainfall 3 MYAP 2011 Final Evaluation Annex 2 78 ANNEX 2: III.D Design Effect on MYAP 2011 Indicators Thematic Area Cod. Indicators Design Effect Anthropometrics in children under 5 years 1a Percentage of children under 5 with chronic malnutrition (Z < -2SD height/age), NCHS 3.866 1b Percentage of children under 5 with chronic malnutrition (Z < -2SD height/age), WHO 3.558 2a Percentage of children under 5 with general malnutrition (Z < -2SD weight/age), NCHS 2.485 2b Percentage of children under 5 with general malnutrition (Z < -2SD weight/age), WHO 1.931 Anthropometrics in children under 36 months 2a Percentage of children under 36 months with general malnutrition (Z < -2SD weight/age), NCHS 2.299 2b Percentage of children under 36 months with general malnutrition (Z < -2SD weight/age), WHO 1.377 Household dietary diversity and food scarcity in the household 4 Average score in dietary diversity 1.408 5 Average number of months out of the year with adequate provisions of food in the household. 1.913 Child feeding 8 Percentage of infants aged 0 to 5.99 months who have been exclusively breastfed in the preceding 24 hours. 1.547 Warning signs in maternal, neonatal and children’s health 12 Percentage of mothers with children aged 0 to 35.99 months who recognize at least two warning signs in pregnancy that indicate the need to seek health services. 3.880 15 Percentage of mothers with children aged 0 to 35.99 months who recognize at least two neonatal warning signs that indicate the need to seek health services. 4.565 18 Percentage of mothers and caregivers with children aged 0 to 35.99 months who recognize at least two warning signs of childhood illness that indicate the need to seek health services. 3.059 Agricultural and livestock practices 22 Percentage of farmers who adopt at least two improved agricultural practices. 2.431 23 Percentage of farmers who adopt at least two improved livestock practices 2.448 Income improvement 30 Percentage of farmers who adopt at least two improved formal marketing practices 2.589