CARE International in HONDURAS Title II Food Security Program DAP FY 2001-2005 FINAL EVALUATION QUANTITATIVE ANALYSIS Frank Sullivan, M.P.S. (Ag.) November 15, 2004 FINAL EVALUATION QUANTITATIVE ANALYSIS CARE HONDURAS FOOD SECURITY PROGRAM DAP FY 2001-2005 submitted to USAID/ Honduras and CARE International in Honduras by Frank Sullivan, M.P.S. (Ag.) Locust Grove, Virginia November 15, 2004 EXECUTIVE SUMMARY CARE International in Honduras’ second DAP has run from FY 2001 and will conclude in September 2005. The goal of the project is to improve the food and nutrition security of vulnerable households in three municipalities of western Honduras. Divided into mother-child, agricultural, and municipal development components, the target population of this $32+ million project is 85,000 beneficiaries annually over the life of the five-year project. This document is one of a series measuring project accomplishments at the end of YR 4 and is the quantitative analysis of project impact in health and in agriculture. An interview format used in the ’96 Baseline, and the ’00 Final of the previous DAP was applied to 1,600 families in this study. Data collection took place in May ‘04, and data analysis took place through the Summer. CARE Honduras hired an experienced DAP evaluator to interpret and report on the extensive data tables, 130 in all, that were produced from those interviews (Appendix B.) Many of the following Impact indicators represent statistically significant change at the .01 and .001 levels. Though the assertion cannot be made with complete confidence because of methodological difficulties, it is most likely the project has successfully reduced the average percentage weighted yield gap for three basis grains from 26.5% to zero, a substantial achievement. Agricultural income in local currency at Baseline-exchange rates has increased in excess of the planned goal; at current exchange rates unfortunately, it represents at least a 25% net fall in dollar-purchasing power. In spite of substantial improvement in agricultural yields, there has been a disastrous fall in project households accumulating productive assets, clearly a result of devastating macro-economic conditions in Honduras. Hoped-for gains in paid employment for project participants have not been achieved. In spite of these very difficult conditions, the project has achieved modest improvement in reducing chronic malnutrition in children 2-5 yr. olds, and in reducing of global malnutrition in children 12-23 months, though neither of these accomplishments reached their ambitious goals. Substantial improvements are documented in mothers’ knowledge of improved nutrition, and better diets are being offered to project children. Modern family planning usage has increased in project couples by 45%, though the goal was not achieved. Most Outcome indicators show 100% achievement or better. • Municipal watershed management plans have been prepared in 15 municipalities. • More municipalities and communities have contributed a higher portion of local inputs to DAP activities than planned. • 3,150 farmers have participated in the rotating agricultural funds exceeding the annual target. • The project has substantially exceeded its target in terms of FFW-assisted road￾building and road maintenance. • More children are receiving regular growth monitoring than was targeted (though fewer children than planned are showing adequate growth.) • More mothers take their children to the nearest health post in the case of diarrheal dehydrations and acute respiratory infection. • More women now understand infant feeding and nutrition better. • 100% of pregnant women received pre-natal counseling. • More reproductive health promoters were trained than planned. • More children were fully vaccinated than planned. Other indicators have been achieved at levels of statistical significance: • More villagers have upgraded their houses to adobe from thatch and improved their latrines. • More mothers know their child’s birth weight. • More 6-to-24 month-old children now consume cheese, yoghurt, bean mix, and other nutritious supplementary foods. • More mothers have used a growth card for their child, have a growth card in their possession, and their child’s weight has been recently registered. • More mothers with under-twos participated in nutritional counseling recently. • More women are now vaccinated with Tetanus Toxoid. • (Alarmingly) more mothers suspend, reduce, or make no change in breast feeding during their children’s diarrhea. • More farmers practice three and five-or-more improved agricultural techniques. Management findings in health are offered: better analysis of nutritional data; better selection of community health workers; and better incentives for them. In agriculture, finding are that a greater number farmers need to benefit from technical assistance; the sustainability of the revolving fund loan is questioned; more emphasis on grain storage and on drip irrigation is recommended; also recommendations are offered for more variety in food-for-work activities, less emphasis on marketing, and eliminating fertilizer loans for unimproved basic grains. General management findings are offered regarding leaving certain “old” municipalities, achieving a gender balance in field extensionists, and being more judicious in using Honduran and external consultants. Recommendations are offered. In mother-child health, pay more attention to the weaning period, hire more female extension workers, and provide production incentives to health volunteers. In agriculture, embark on a major effort in grain storage, redesign the program methodology, and collect agronomic data at the end of each harvest by crop￾cuttings. Overall, scale back on innovation and concentrate program focus. TABLE OF CONTENTS EXECUTIVE SUMMARY …………………………………………………… ii TABLE OF CONTENTS ……………………………………………………… iv ACRONYMS …………………………………………………………….. …. vi 1. INTRODUCTION and BACKGROUND …………………………….. 1 2. METHODOLOGY and TIMING ……………………………………… 1 3.0 IMPACT INDICATORS ………………………………………………. 3 3.1 SO 1: Increased Availability of Basic and Nutritious Foods …………. 3 3.2 SO 2: Increased Access to Food ………………………………………. 6 3.3 SO 3: Improved Biological Utilization of Food ………………………. 8 3.4 Summary ……………………………………………………………….. 12 4.0 MONITORING INDICATORS ……………………………………….. 14 4.1 SO 1 Monitoring Accomplishments …………………………………… 14 4.2 SO 2 Monitoring Accomplishments …………………………………… 15 4.3 SO 3 Monitoring Accomplishments …………………………………… 16 4.4 Summary ……………………………………………….......................... 17 5.0 PROGRAM ELEMENTS WITH X 2 SIGNIFICANCE ………………… 18 5.1 Households by House Type …………………………………………...... 18 5.2 Households according to Hygiene Conditions …………………………. 18 5.3 Households that have used MOH Health Services …………………….. 18 5.4 Mothers with Under-Two’s who know the Child’s Birth Weight ……… 18 5.5 Under-Two’s whose Breast Milk was suspended during Diarrhea …….. 18 5.6 Under-Twos who during Diarrhea had no change in Breastfeeding ……. 18 5.7 Under-Twos With Diarrhea in the last 15 Days ………………………… 19 5.8 Solid Food Consumed Yesterday by Age Group ……………………….. 19 5.9 Under-Twos and Growth Card Use ……………………………………… 19 5.10 Households with Under-Twos Participating in Educational Counseling .. 19 5.11 Women 10-49 Yrs. Vaccinated with Tetanus Toxoid …………………… 19 5.12 Households Consuming a More Balanced Diet …………………………. 19 5.13 Number of Farms Practicing Improved Agricultural Techniques ………. 19 5.14 Percentage Farmers with Improved Agricultural & Agroforestry Practices 20 5.15 Summary ………………………………………………............................. 20 6.0 PROGRAM OBSERVATIONS …………………………………………. 21 6.1 Health Observations …………………………………………................... 21 6.2 Agricultural Observations ………………………………………….......... 23 6.3 General Observations …………………………………………................. 26 6.4 Summary …………………………………………..................................... 29 7.0 CONCLUSIONS …………………………………………........................ 30 7.1 Impact Level Conclusions ………………………………………….......... 30 7.2 Outcome Level Conclusions …………………………………………...... 31 7.3 Management Conclusions ………………………………………….......... 32 7.4 Macro Conclusion …………………………………………...................... 32 8.0 RECOMMENDATIONS …………………………………………........... 33 8.1 In HOGASA, pay more attention to the weaning period ……………….. 33 8.2 Hire more women in order to emphasize this focus ……………………. 33 8.3 Provide production incentives to health promoters …………………….. 34 8.4 Embark on a major effort in grain storage ……………………………… 35 8.5 Redesign the EXTENSA program methodology ……………………….. 36 8.6 Collect agronomic data at the end of the harvest season ……………….. 36 8.7 Overall, scale back on program innovation and concentrate focus …….. 37 APPENDICES Appendix A Expanded Narrative on Methodology (in Spanish) Appendix B Charts ACRONYMS ADAI Honduran Data Collection Consultant Firm ASHONPLAFA Asociación Hondureña de Planificación Familiar alumno Farmer trainee AP Project Area BHRIS Rotating agricultural loan fund CODECO Community Development Committee DAP Development Assistance Program EXTENSA Agricultural Extension Component FFW Food-For-Work GO Government Organization HOGASA Maternal and Child Health Component INE Instituto Nacional de Estadísticas manzana 0.7 of one hectare MOH Ministry of Health NA New (Project) Area NGO Non-Governmental Organization PEPE Principal farmer with whom EXTENSA works PODER Municipal Strengthening Component quintal 100 lbs. SO Strategic Objective UCS Unidad Comununitaria de Salud (project-built health meeting center) USAID/H The United States Agency for International Development/Honduras FINAL EVALUATION QUANTITATIVE ANALYSIS CARE HONDURAS FOOD SECURITY PROGRAM DAP FY 2001-2005 1. INTRODUCTION and BACKGROUND CARE International in Honduras has been managing a Title II Food Security program since 1996 within the framework of the USAID Food Security Policy Paper. The first DAP ran from 1996-2001; the second DAP has run from FY 2001, will conclude in September 2005, and is the focus of this and other Final Evaluation documents. The aim of the project is to improve in a sustainable manner the food and nutrition security of vulnerable households in three extremely poor municipalities of western Honduras: Lempira, Intibuca, and La Paz. The program is divided into three components, HOGASA, the mother-child side; EXTENSA, the agricultural development side; and PODER, the municipal development side. The target of this approximately $33 million project is to improve the food availability, food access, and food utilization for approximately 85,000 beneficiaries annually over the life of the five-year DAP. This document is one of a series measuring the accomplishments of the Honduras DAP at the end of YR 4 of the project, the decision having been taken to assess the project earlier than originally planned in order to incorporate lessons learned into the follow-on proposal more effectively. Three qualitative Evaluations have been conducted, one each in Health, Municipal Strengthening, and Agriculture. Those texts should be read in conjunction with this one, which is the quantitative analysis of project impact in health and in agriculture. 2. METHODOLOGY and TIMING The quantitative evaluation has been carried out in several parts. The first was the application of an 1800 family Baseline data collection at the start of the project. Following the pattern established during the first DAP, these data were collected by an entity independent to CARE. The Honduran firm, ADAI, is experienced in quantitative data collection and processing, and participated in the Baseline data collection of the first DAP, and in its Final Evaluation. That report was submitted both as quantitative evaluation of the first DAP as well as Baseline information for the current DAP. For this Final Evaluation, ADAI also conducted interviews of 1,680 families during May, 2004; and that information will form the basis of this text. Repeating the pattern of the first DAP, interviewees were chosen from three geographic areas: areas where the project plans to continue working (N = 696 families) plus areas the project is thinking to withdraw from at the close of this DAP (N = 504)—together denominated Project Areas (AP in Spanish)— as well as areas the project has plans to enter in FY 2005 (N = 480) —denominated NA for New Areas. Project accomplishments will be measured by contrasting figures of the AP area compared to Baseline; NA data will generally be cited only by way of contrast. Adding NA data in this fashion also creates the Baseline for the FY 2011 Final evaluation. As in previous studies, the interview instrument was the one used in the 1996 Baseline, in the first DAP’s 1998 Mid-Term, and in the 2001 Final Evaluation. Themes of the questionnaire were: demographic composition of the household, characteristics of the dwelling, health data (including anthropometric measurement), feeding and consumption practices, agricultural production and commercialization, income, and others. Communities to be investigated were chosen at random, and participant interviewees in these communities were also chosen at random. A detailed explanation of the methodology is included (in Spanish) in Appendix A. Since the aim of the project is to bring overall uplift to the project area, no distinction was made between those who had participated directly in the project and those who had not. This was not a complicating factor in analyzing nutritional and health topics since over the course of four years, all but a few families would have had a child passing through the project. The situation with the agriculture component was more problematic since a limited number of farmers participated directly in the agricultural component. The report addresses, nevertheless, overall impact on both sides of the project. Data collection took place during three weeks in May. Data analysis took place during the Summer of ’04, and was largely completed by August 20, ‘04. An independent evaluator with experience in DAP program evaluation was hired by CARE Honduras to interpret the numerous data tables—131 tables in all, broken into 20 thematic areas, attached as Appendix B —and to produce a narrative of findings, conclusions and recommendations in time for the ideas to be incorporated into the write-up of the 2006-2011 DAP taking place in early October, 2004. The report will be divided into sections. Section One and Two, introductory comments, have now concluded. Section Three will address impact indicators as proposed in the original DAP. Section Four will discuss monitoring indicators, those proposed in the Indicator Performance Tracking Table. Section Five will discuss tables where statistical significance has been demonstrated using a Chi-square analysis. Section Six will address management and program observations that grow out of the evaluator’s brief field travel to the project. This section, particularly, should be read in conjunction with the qualitative evaluations mentioned above. Section Seven offers conclusions and Section Eight offers recommendations. The document has deliberately been written more concisely than previous years’ reports, thinking that a condensed analysis would be preferable to the hundreds of pages of text and tables of previous evaluations. The reader is invited to review the extensive tables of Appendix B should a more detailed study be of interest. 3. IMPACT INDICATORS The Goal of the project is to improve, in a sustainable manner, the food and nutrition security of vulnerable households in extremely poor municipalities of western Honduras. The DAP Results Report Table tracked lists nine Impact Indicators under three Strategic Objectives. SO1: Increased availability of basic and nutrition foods • Reduced average percentage weighted yield gap for maize, beans, and sorghum. • Increased net agricultural income/hectare. SO 2: Increased access to food • Increased percentage of households accumulating liquid and productive assets. • Increased average number of person-days paid employment per capita. SO 3: Improved biological utilization of food • Reduced percentage of malnutrition (height/age) in children age 2-5 years. • Reduced percentage of malnutrition (weight/age) in children 12-23 month-olds. • Increased percentage of households consuming a balanced and diversified diet. • Increased percentage of mothers using suitable feeding to children under 12 months of age. • Increased percentage of reproductive age couples using family planning methods. In the tables that follow, the indicator will be listed and five figures will generally be provided, three directly attributable to the DAP under review (in bold), and two reference figures: the first, comparable baseline data from 1996; and the last, data from areas recently entered into by the program in 2004. A summary table will be included at the end of the section. 3.1 SO 1: Increased Availability of Basic and Nutritious Foods Accomplishment of this Strategic Objective is measured by two impact indicators: reduced average yield gap in three basic grains, and increased net monthly agricultural income per hectare. Table 3.1.1: Reduced average percentage weighted yield gap for basic grains of corn, beans and sorghum. (Table 19.2) Indicator Baseline ‘96* Baseline ‘01 Goal Achieved New Area Reduced average percentage weighted yield gap for basic grains of corn, beans and sorghum (1,096 farms reporting) 23.4% 26.5% ?20% (22.08) -15.1% - 19.0 * DAP FY 1996-2000 Final Evaluation, Table No. 102 The weighted yield gap is the difference between potential yield, defined as the average agricultural yield reported by the Honduran Institute of National Statistics (INE) for the specific agricultural area, compared to actual reported yields per crop, weighted by the size of the area of cultivation. A twenty percent reduction from 26.5 in the 2001 Baseline was proposed, down to 22.08. Actual achievement based on evaluator collected reports was -15.1% meaning that project farmers had higher yields than those of the comparable surrounding area. Such improvement would represents a gain of almost forty percent, from 25.5% under-average production to 15.1% over-production. The figures disguise a bigger gain in gross production, because the 2001 INE averages for maize and beans (note: figures are reported in quintales per manzanas) were 20 and 7 respectively, while the 2004 INE figures are 22.4 and 10.3, (only sorghum reflecting a fall in production from 15 to 11.2.) Thus the production gap would have been eliminated and a surplus created against a backdrop of higher levels of comparable production in two of the crops. The increase is statistically significant at .001 level and statistical significance applies to each of the crops analyzed. However, the data merit further analysis as per the following table. Table 3.1.2: Average yield (x100 lbs.) per hectare (from Table 19.1) Indicator Baseline ‘96* Baseline ‘01 Goal Achieved New Area Average corn yield/ha. (1096 ’04 farms) 19.8 23.1 27.72 34.7 36.3 Average bean yield/ha. (812 ’04 farms) 8.5 6.4 7.68 16.1 18.7 Average sorgh. yield/ha. (264 ’04 farms) 17.7 13.4 16.08 24.0 26.1 * DAP FY 1996-2000 Final Evaluation, Table No. 101 Data in the table above taken from Table 19.1 report that corn yields have gone from 2,314 lb./ha. in 2001 to 3,471 lb./ha. in 2004, 125% of target. Bean production has apparently more than doubled from 640 lb. per hectare at Baseline to 1,614 lb./ha. at Final Evaluation, 209% of target. Sorghum production is reported to have increased from Baseline by eighty percent to 2,396 lb./ha., 150% of target. In all three crops, new program areas report figures higher than current program areas. Such yield figures need to be read with caution. It is hard to imagine that sorghum production could have increased so much—against an INE-reported deterioration in average production in comparable areas—without the topic being widely reported. Far higher increase in yields for beans than one could normally expect have also been reported by the evaluation team; the same for maize. To explore this issue further, the evaluator turned to CARE’s annual reporting of agricultural production. Note that this is internally generated information reported by agricultural extension staff, and is provided on “CARE farmers,” those who participate directly in the EXTENSA project rather than on all farmers in the area, as INE and evaluator data do. Thus it would not be unusual if CARE figures were somewhat higher than those of farmers nearby, since these farmers would have had the full benefit of technical assistance. In fact, the opposite in true: in all three crops, evaluation data are higher than for CARE farmers. Table 3.1.3: Comparison between CARE-annually reported yields and evaluator￾reported yields (x 100 lbs./ha.) Indicator CARE Reports (from Annual Results Tracking Table) Eval. reports Commentary YR1 YR2 YR3 YR4 YR 4 CARE reported maize yields (Baseline 23.1) 25.8 18.4 24.5 25.37 34.7 Reasonable annual variance. Evaluation data 35% higher CARE reported bean yields (Baseline 6.4) 11.1 7.0 15.4 13.92 16.1 Wide annual variance; 15% difference between CARE/evaluator data CARE reported sorghum yields (Baseline 23.1) 16.0 n/a 12.9 14.39 24.0 Reasonable annual variance. Evaluation data 66% higher Reviewing annual yield reports taken from the DAP Annual Results Tracking Table, maize yields have varied, but the figures seem reasonable. Variation in annual bean yields are much greater than maize—not necessarily a surprising finding from an agronomic point of view. CARE data for sorghum production vary from year to year but in YR 4 are in line with average yields reported by INE. Note that both CARE and evaluation data are based on verbal farmer estimates rather than on crop-cuttings—CARE’s data collected at harvest time, evaluator data collected in May ’04 on 2003 production. For these and a number of other reasons, the evaluator concludes that CARE data are more reasonable than evaluator data, and the following figures will be reported. Table 3.1.4: Comparison of CARE and evaluator figures regarding average yield increases from Baseline in three crops Indicator Baseline ‘01 Goal 20%? Achieved % incr. R* Maize Evaluator data: average corn yields/ha. 34.7 150 CARE data: average corn yields /ha. 23.1 27.7 25.3 110 v Beans Evaluator data: average bean yields /ha. 16.1 252 CARE data: average bean yields /ha. 6.4 7.7 13.92 218 v Sorghum Evaluator data: average sorghum yields 24 179 CARE data: average sorghum yields /ha. 13.4 16.1 14.39 107 v * Reasonable figure as per evaluator judgment In summary: as much as can be inferred from these anomalous figures, maize yields seem to have increased across the project area probably by about 10%. Bean yields have increased substantially, perhaps more than doubling. Sorghum yields seem to have increased by about 7.5%. A recommendation how to improve the collection of agronomic data will be offered. If this analysis is reasonable, it will be important to “back these figures” into the Weighted yield gap table (above) and the Increased net monthly agricultural income table (below) some time soon. This is an important exercise so as to have more accurate FY04 Baseline average-yield-gaps and income-per-hectare figures against which FY 2010 accomplishments can be measured. When the evaluator conducted a rough calculation of this revised yield gap based on CARE figures, the 23.4% negative yield gap of the Baseline was reduced to zero, but no positive yield gap resulted. To this observer, this is probably a close approximation of the real gains achieved by EXTENSA—substantial in any case. Table 3.1.5: Increased net monthly agricultural income/hectare Indicator Baseline ‘96* Baseline ‘01 Goal Achieved New Area Increased net agricultural income/Lps./ha. (Table 15.3) 501 489 687 616 Constant dollars at ’01 Baseline exchange rate (Lps. 10.5::$1.00) $47.72 $46.57 $50 $65.43 (@ 10:5::1) At current exchange rate (+/-Lps. 18.5::1) $37.13 $33.3 * DAP FY 1996-2000 Final Evaluation, Table No. 89 Average monthly income, Lempira per hectare, has improved from Lps. 489 in the Baseline to Lps. 663 at time of data collection, 131% of target if one uses a constant dollar value. It did not achieve a X 2 test of statistical significance however. (If one adjusts these figure based on estimations in the previous paragraph, the figure drops to $52.02 dollars [at 10.5::1]—still exceeding the original goal.) Alas, as all in Honduras know, the value of the Lempira has fallen substantially in the intervening years. At current exchange rates, +/- L. 18.5::$1.00, the monthly dollar value of DAP farmers’ production is $35.83, a fall of 25% from the Baseline of 2001. (If one adjusts these figure based on estimations in the previous paragraph, the figures drop to $29.52 [at 18.5::1].) It will be observed the monthly (dollar-value) production of DAP farmers has fallen one-third over the ten years of the two DAPs, notwithstanding substantial increases in agricultural yields. It would appear that a goal expressed in dollar terms seems unattainable under current Honduran macro-conditions. 3.2 SO 2: Increased Access to Food Accomplishment of this Strategic Objective is measured by two impact indicators: increased percentage of households accumulating liquid and productive assets, and increased number of day of wage labor. Table 3.2.1: Increased percentage of households accumulating liquid and productive assets. Indicator Baseline ‘96* Baseline ‘01 Goal Achieved New Area Increased percentage of households accumulating liquid and productive assets. (Table 13) 54 33.4 40 19.1 20 * DAP FY 1996-2000 Final Evaluation, Table No. 77 Nowhere is the deterioration that has taken place in the Honduran rural economy seen more clearly than in the data of this table. Because of the devastation of Hurricane Mitch; because of the vertiginous drop in the world price of coffee; because of Honduras’ high unemployment and underemployment; because of the increasing gap between the rich and the poor; and probably because of a number of other macro-factors— in spite of substantial gains in agricultural production—project farmers have substantially fewer liquid and productive assets than they had five years ago, and fewer than they had ten years ago too. These figures are statistically significant at .001 level. As will be seen by a glance at Table 16.1, the percentage of project households in indigence has risen from 85.9% to 87.8% in the period of the current DAP, up from 80.3% in the 1996 Baseline (DAP FY 1996-2000 Final Evaluation, Table No. 95.) Increases in agricultural production documented above have not stemmed the overwhelming tide of rural impoverishment that is taking place in Honduras. As the document will shortly analyze nutritional impact, this fact should be kept ever in mind. Table 3.2.2: Increased average number of person-days of paid employment, per capita Indicator Baseline ‘96* Baseline ‘01 Goal Achieved New Area Percentage days of salaried employment 59% 69% 50% 35% Increased average number of person-days of paid employment, per capita (Table 14) 8.1 9.8 16 10.9 12.3 * DAP FY 1996-2000 Final Evaluation, Table No. 78 The number of paid days worked by project participants as a percentage of total days worked has fallen from 69% in the Baseline to 50% in the Final, a deterioration in the job market at the >99% level of statistical confidence. However, modest gains in the average number of days per worker have occurred, from 9.8 to 10.9, though the finding does not have statistical validity. The goal of achieving 16 average days paid labor per capita was not achieved. The average number of days of paid work in the new programming area shows somewhat higher levels than current programming area, likely due to somewhat better road connections in the new area, thus better out-of-area opportunities. One could speculate this decrease in the percentage of salaried employment represents an increase in agricultural employment as a result of increased agricultural production. Data from other tables in this series suggest such an effect has been minimal. Table 14 also shows the average number of workdays per female worker has risen quite a bit more than for males, from 9.7 at Baseline to 15.4 in the Final Evaluation. Salaried women, though only half of the male cohort, report substantially higher wages. 3.3 SO 3: Improved Biological Utilization of Food Accomplishment of this Strategic Objective is measured by five impact indicators: reduced percentage of chronic malnutrition, reduced percentage of global malnutrition, increased percentage of households consuming a better diet, increased percentage of mothers providing suitable feeding practices to their under twelve-months old children, and increased use of (modern) family planning methods. Table 3.3.1: Reduced percentage of malnutrition (height/age) in children age 2-5 years in target areas. Indicator Baseline ‘96* Baseline ‘01 Goal Achieved New Area Reduced percentage of malnutrition (height/age) in children age 2-5 years in target areas. (Table 23.2) 54.9 61.0 53 58.4 64.6 * DAP FY 1996-2000 Final Evaluation, Table No. 115 Data collected on 1,095 children during the Baseline and 721 children during the Final Evaluation demonstrate the project has achieved a modest reduction in the percentage of 2 nd and 3 rd degree height-for-age malnutrition in children 25-to-59 months, from 61.0% to 58.4%. The goal of a reduction to 53% was not achieved. Also noted in Table 23.2, there was a modest decline in global malnutrition from 30.7% to 28.2%. Neither of these two findings achieve X 2 statistical significance. Table 24.2 shows all of this improvement in both classes of malnutrition has taken place for male children, though the sample size is not large enough to determine any statistical significance. In new project areas, percentages of chronic and global malnutrition are higher than in 2001 Baseline communities. At this point, it bears reflecting on the agricultural findings from the previous section. Agricultural data strongly suggest that living conditions in the project area have deteriorated substantially during the four years of the DAP, with rural impoverishment and the sale of rural assets increasing. Other things being equal, one would expect nutritional status of vulnerable children to deteriorate in such a climate. Instead, the data show nutritional status for 25-to-59 month old children improving, albeit modestly. This trend was given a quintessential summation by one nine-year community health promoter who said: “Before, when the children got sick, they died; now they lose weight, but they don’t die.” Said another in a different village, with great pride: “we haven’t had an infant death in over three years.” The quantitative finding and these anecdotal comments suggest strongly that project activities of growth monitoring, nutritional talks, community group mobilization, linkages with the Ministry of Health, and all other aspects of the multi-dimensional DAP are having nutritional impact. Table 3.3.2: Reduced percentage of malnutrition (weight/age) in children age 12-23 months in target areas. Indicator Baseline ‘96* Baseline ‘01 Goal Achieved New Area Reduced percentage of malnutrition (weight/age) in children age 12-23 months in target areas. (Table 23.3) 33.3 30.1 24 27.2 39 * DAP FY 1996-2000 Final Evaluation, Table No. 116 Data collected on 376 12-23 month-old children during the Baseline and 224 children during the Final Evaluation demonstrate that the project has been able to reduce the percentage of 2 nd and 3 rd degree weight-for-age malnutrition from 30.1% to 27.2%. The goal of a reduction to 24% was not achieved. Also noted in Table 23.3, there was a modest decline in global malnutrition from 30.7% to 28.2%. Neither of these two comparisons represent findings of X 2 statistical significance. In new project areas, percentages of chronic and global malnutrition are higher for these children than in 2001 the Baseline. When one analyzes Table 23.3 and related tables, interesting suggestive patterns appear. Almost a quarter (23.7%) of children develop chronic malnutrition before one year of age—an alarming figure. This figure then jumps to over half (52.2%) by two years of age, and then creeps up to 58.2% by the age of five. Table 6.1 shows that 92% of children are born with adequate weight, clearly indicating the finding is not due to heredity. Reviewing nutritional charts during the field travel showed that nutritional faltering occurs rather infrequently in the 0-6 months cadre; by nine months, about 50% of children have received “red dots,” signifying two consecutive months of stagnant weight or weight loss. Comparing data from several other tables: Table 6.3 reports that almost 20% of children are weaned by five months (far too early) and another 19% are weaned by 11 months. Table 6.6 shows that 77% of mothers report exclusive breast feeding, however Table 6.8 shows that less than half of them actually do so. Clearly, exclusive breastfeeding is less than desirable, and the effects in terms of early chronic malnutrition are evident—and are never recovered from. Global malnutrition is somewhat different: only 6.4% between birth and 11 months. Low weight-for-age jumps to over a quarter (27.2%) by two years old, and then stabilizes at that level until five. The figures are generally comparable between Baseline, program, and new communities. Programmatically, it would appear the next DAP could be concentrating on three ideas: • even more emphasis on exclusive lactation; • even more emphasis on extended lactation; and • even more emphasis on the critical period of 12-to-23 months when most weaning takes place and/or when a competing sibling is born. It is at this time that enormous jumps in both chronic and global nutrition are occurring. Mention should be made of DAPFY1996-2000 Final Evaluation comments in this regard: That (sic) fact that 34.6% of children age 12-23 months in the Program Area are not receiving any solid food at all highlights the seriousness of this problem in the targeted area. Clearly recommended complementary feeding practices should be the key focus for the HOGASA component. [Bold in original text.] All other key project indicators—such as complete immunization, receiving treatment for diarrhea and/or acute respiratory infection, and attending growth monitoring—are at least double the level of this indicator. 1 This evaluation’s findings suggest this comment is as valid in YR4 of the current DAP as it was at the close of the previous one. That having been said, as with comments in the previous section, against a backdrop of increasing rural impoverishment, even a modest decrease in global malnutrition in 12-23 month old children shows the DAP is having nutritional impact. It also seems useful to underline that this information has been collected at the end of the fourth year of a five year project, a year earlier than planned. In actual fact, the project has another 12 months of activity before it could be held accountable for under-performance of goals. Table 3.3.3: Increased percentage of households consuming a balanced and diversified diet. Indicator Baseline ‘96* Baseline ‘01 Goal Achieved New Area Increased percentage of households consuming a balanced and diversified diet. (Table 12.3) -- 66.4 74 80.3 70.9 *This indicator was refined between the first and the second DAP and comparable ’96 data do not exist. At Baseline, 66.4% of 1,785 households reported consuming 8 or more types of food, the redefined indicator of the current DAP. In the Final Evaluation, this figure jumped to 80.3% of 962 families. This represents a substantial improvement in household nutrition and the figure is statistically significant at the .001 level. The project’s efforts to diversify the rural Honduran diet are having considerable success. Table 12 in the Appendix demonstrates that project households are now consuming more animal products —up from 70.2% to 76%; more vegetables, fruits, and Vitamin A products—up from 28.3% to 36%; and more oils—up from 83.6% to 88.8%. All these findings are statistically significant at the .001 level. 1 Final Evaluation DAP FY 1996-2000, p100.. Table 3.3.4: Increased percentage of mothers with suitable feeding practices of children under 12 months of age Indicator Baseline ‘96 Baseline ‘01 Goal Achieved New Area Increased percentage of mothers with suitable feeding practices of children under 12 months of age. (Tables 9.3) < 6 mos. 6-8 months 9-11 months Subtotal: <12 months 81.1 n/a 2.9 40.2 (N= 372) ?30% 52.4 78.4 36.2 21.3 53.6 (N=207) 82.5 16.7 11.8 50.7 * Indicator newly adjusted for the current DAP; ’96 data not applicable. This table represents a composite of several data sets: mothers who report exclusive breastfeeding for their under-six month olds; the introduction of appropriate complementary feeding at 6 to 8 months; continued introduction of a more diversified diet for 9-11 month-old children; and a gradual introduction of carbohydrates, oils, and fruits and vegetables over the period 12-23 months. As will be seen elsewhere in this report, lactation has continued to be strong for under-six months children. Big DAP gains have been achieved in suitable feeding practices for 6-to￾8 month olds and in 9-to-11 month old children. The goal of a 30% increase in such practices has been achieved. This finding has X 2 significance at the p.<.01 level. Data from the new program areas convincingly support the need to concentrate on the 6-11 month period. Table 3.3.5: Increased percentage of couples of reproductive age using modern family planning methods. Indicator Baseline ‘96* Baseline ‘01 Goal Achieved New Area Increased percentage of couples of reproductive age using any family planning methods. (Table 11.9) 4.3 10.8 16.8 14.5 12.2 Increased percentage of couples of reproductive age using modern family planning methods. (Table 11.8) 2.5 8.7 12.7 11.4 * DAP FY 1996-2000 Final Evaluation, Table No. 73 and 72 At Baseline, 10.8% of 2607 not-pregnant, 12-49 year old women were using some form of family planning; at Final Evaluation, the figure rose to 14.5%, 253 women of 1750. Data regarding use of modern methods of family planning show 8.7% of these women using some method of modern family planning; at the Final Evaluation, this figure jumped to 12.7%. While the ambitious goal was not achieved, these figures represent a 34% and 46% jump in family planning practices and are significant at the p.<.001 level of confidence. Table 3.3.6: Summary Table of Impact Accomplishment Indicator Baseline 2001 Goal Achieved Commentary SO 1: Increased availability of basic and nutritious foods Reduced average percentage weighted yield gap for basic grains of corn, beans and sorghum (Table 19.2) 26.5% 22.1% 0 Best estimate: gap reduced to zero. p. <0.001 Increased net agricultural income/Lps./ha. (Table 15.3) 428 525 663 125% of goal. No statistical power. SO 2: Increased access to food Increased percentage of households accumulating liquid and productive assets. (Table 13) 33.4 40 19.1 Drastic impoverishment of rural Honduras seen in these figures. Increased average number of person-days of paid employment, per capita (Table 14) 9.8 16 10.9 Goal not reached; small increases in employment. SO 3: Improved biological utilization of food Reduced percentage of malnutrition (height/age) in children age 2-5 years in target areas. (Table 23.2) 61.0 53 58.4 Small improvement in spite of difficult macro￾climate. Reduced percentage of malnutrition (weight/age) in children age 12-23 months in target areas. (Table 23.3) 30.1 24 27.2 Small improvement in spite of difficult macro￾climate. Increased percentage of households consuming a balanced and diversified diet. (Table 12.3) 69.4 74 80.3 108% of goal. p. <0.001 Increased percentage of mothers with suitable feeding practices of children <12 months of age. (Table 9.3) 40.2 52.4 53.6 Goal achieved. p. <.01 Increased percentage of couples of reproductive age using modern family planning methods. (Table 11.8) 10.8 16.8 14.5 45% improvement though goal not reached. p. <0.001 3.4 Summary The project is having mixed success in its complicated array of activities, targets, and hoped-for impacts. In general terms, agricultural production has certainly improved; one infers the severe yield gap has at least been eliminated (even without conducting the “backing in” exercise for the Yield Gap table.) In spite of this, rural impoverishment has been exacerbated. Some modest improvements in reducing malnutrition have occurred in the face of a tide of negative macro-economic trends though large, hoped-for gains have not been achieved. Substantial improvements to diet have taken place, as have improvements in the use of modern family planning methods. 4. MONITORING INDICATORS The Indicator Tracking Table lists annual monitoring indicators that will now be discussed. Some of the data have been provided by project management staff; other data are found in externally collected Tables of Appendix B that will be referenced. Table 4.1: SO 1 Monitoring Accomplishment as reported by CARE (except where noted) Indicator Baseline 2001 FY04 Target Achieved Commentary SO 1: Increased availability of basic and nutritious foods Increased number of municipalities with micro watershed management plans being implemented 0 15 15 Achieved Increased percentage of families that implement three or more sustainable agroforestry practices (Table 15.4) 43% 70.5% FY04 target not specified; likely over achieved Increased percentage of households obtaining agriculture inputs and technical assistance from self-sustaining rotating banks (BRHIS) n/s 2688 3,150 farmers reported participating in BRHIS; see comments; banks as self-sustaining seems questionable at this stage. Increased average yields of basic grains (qq./hectare of land; corn, beans, sorghum). C 23.1 B 6.4 S 13.4 24.5 7.5 13.8 25.4 13.9 14.4 Achieved Over-achieved Achieved Increased percentage of households that plant two or more new crops n/s 1209 1308 Over achieved n/s = not specified Municipal watershed management plans have been prepared in 15 of the 19 DAP municipalities. The project has achieved nearly double the percentage of families that are implementing agro-forestry practices compared to Baseline. Somewhat more farm families are reported to have planted two or more new crops than had been targeted. A total of 3,150 farmers have participated in the rotating agricultural funds, up from the annual target of 2,668. However, participation in this financing varies from member to member—some few members availing of three or more loans, others availing of only one. The proposed “self-sustainability” of these funds will be discussed in the Management findings section. Table 4.2: SO 2 Monitoring Accomplishment as reported by CARE (except as noted) Indicator Baseline 2001 FY04 Target Achieved Commentary SO 2: Increased access to food Reduced percentage of communities reporting commercialization problems 80% 68% Accomplished Increased percentage of communities with increased access to markets 33% 46% 46% Accomplished Increased number of kilometers of roads constructed (cumulative) n/s 50.5 64.6 Over accomplished Increased number of kilometers of roads upgraded (cumulative) n/s 65 104.5 Over accomplished Number of municipalities (M) and communities (C) increasing resource contribution to support Title II program n/s M-15 C- 88 M-15 C-94 Over accomplished Increased percentage of AF2001 communities with sustainable Title II program phase-out plans n/s 35% Highly doubtful Increased percentage of municipalities coordinating development forums among CODECOs-NGOs-GOs and private sector. n/s 15 15 Accomplished n/s = not specified The project’s initiatives in agricultural marketing are reported as improving. The subject was not explored during the field travel in enough depth to offer comments, except to suggest that this is probably proceeding somewhat more slowly than project planners would like. It is an exceedingly complex subject after all. See also comments on marketing in the management section. The project has substantially exceeded its targets in terms of FFW-assisted interior road building and road maintenance (a subject which will be discussed further in the Programming Observations section.) More municipalities and communities have contributed a higher portion of local inputs to DAP activities than planned. Also reported is that 15 of the 19 municipalities have increased the dialogue among CODECOs, NGOs and Government agencies. CARE reports that preliminary work has taken place in strengthening municipal skills in Human Resource management, methodologies, and horizontal and vertical linkages; but it is early days to determine whether any municipality or community has acquired enough skills to carry on after the departure of the DAP. Table 4.3: SO 3 Monitoring Accomplishment as reported by CARE (except as noted) Indicator Baseline 2001 FY04 Target Achieved Commentary SO 3: Improved biological utilization of food Increased percentage of children with growth monitoring according to MOH guidelines n/s 90% 97% Over accomplished Increased percentage of children with adequate growth trends n/s 75% 66% Under accomplished Increased percentage of children taken for treatment when mothers identify two or more signs of diarrhea dehydration (Table 7.2) 61.7% 98% 69.3% CARE monitoring reports 98%; evaluation data report 69.3% Increased percentage of children taken for treatment when mothers identify two or more signs of acute respiratory infection (Table 8.2) 64.8% 90% 75% CARE monitoring reports 100%; evaluation data reports 72.9% Increased percentage of mothers in families receiving rations trained in proper infant feeding practices and nutrition n/s 70% 93% Over accomplished Maintained percentage of pregnant women with rations receiving pre-natal care-MOH norms 88.3% 100% 100% Accomplished Increased number of trained health advisors in reproductive health (cumulative) n/s 770 1032 Over accomplished Increased percentage of children with immunization according to MOH norms 86.8% 95% 95% Accomplished Number of target households (pregnant women, breastfeeding, < 6 months of age, children 6-24 months) receiving rations 7500 7500 8064 Over accomplished n/s = not specified More children are receiving regular growth monitoring than was targeted. Unfortunately, (discussed above) fewer children than planned are showing adequate growth. CARE monitoring suggests a higher figure than the independent data collectors do for mothers taking their children to the nearest health post in the case of diarrheal dehydrations and acute respiratory infection. Part of this is likely under-reporting by the mothers to the evaluation data collection team. The DAP has developed a system of referral and counter￾referral that is reported on monthly, and is likely the more accurate figure. More women than planned now understand the concepts of infant feeding and nutrition, a finding corroborated by several Impact tables already analyzed. Accomplishment of the indicator of 100% of pregnant women receiving pre-natal counseling is an important project accomplishment, as is the training of a larger number of reproductive health promoters than was planned, as is achieving the target regarding children with full vaccinations. The project has reached more vulnerable households with the HOGASA component than was planned in the original DAP narrative. 4.4 Summary Almost all monitoring indicators show 100% achievement—or more. Project management and staff are working quite hard and are achieving a large number of targets at the Output level. Several ideas that can be re-thought in order for staff to work “smarter” to achieve impact rather than working “harder” will be discussed in the Conclusions section. 5. PROGRAM ELEMENTS WITH X 2 SIGNIFICANCE Given that the Final Evaluation data collectors produced over 130 tables, it seems useful to look at those tables that resulted in data of Chi square significance. As will be seen, some of the information sheds light on what has already been observed; some of the information is new. The reader is encouraged to refer to Appendix B throughout the narrative for further detail. All X 2 values are between the baseline and the Project Area. For the sake of clarity, no X 2 comparison are made between the Baseline and new areas or between AP and new areas. Table 5.1: Number of Households by House Type (Appendix B, Table 3) Indicator Baseline ‘01 AP New Area Commentary Adobe houses 65.1% 74.2% 74.6% p.<.001 Table 5.2: Number of Households according to Hygiene Conditions (Appendix B, Table 3.3) Indicator Baseline ‘01 AP New Area Commentary With improved toilet 45.2 50.9 58.1 p. <.01 Table 5.3: Households using MOH Health Services (Appendix B, Table 4.1) Indicator Baseline ‘01 AP New Area Commentary Visits to UCS Visit to health post Visit to both 7.1 91.4 1.4 15.3 66.9 17.9 100 p. <.001 p. <.001 p. <.001 Table 5.4: Mothers with Under-Twos who know the child’s Birth Weight (Appendix B, Table 6) Indicator Baseline ‘01 AP New Area Commentary Mothers who know 39.3 59.4 53 p. <.001 Table 5.5: Under-Two’s whose Breast Milk was suspended during Diarrhea (Annex B, Table 7.4) Indicator Baseline ‘01 AP New Area Commentary Yes, suspended 3.2 11.6 11.1 p.<.01 worrisome Table 5.6: Under-Two’s who during diarrhea had no change in their Breastfeeding or Breastfeeding was reduced (Appendix B, Table 7.5) Indicator Baseline ‘01 AP New Area Commentary Giving same or fewer times 75.6 90.2 83.3 p.<01 using both Table 5.7: Under-Two’s with Diarrhea in the last 15 days (Appendix B, Table 8) Indicator Baseline ‘01 AP New Area Commentary Yes 64.9 50.4 56.3 p. <.001 Table 5.8 Solid Food consumed yesterday by Age Group (Appendix B, Table 9) Indicator Baseline ‘01 AP New Area Commentary 6-8 and 9-11 month-olds eating cheese, yoghurt, bean mix, etc. 12-23 month-olds eating cheese, yoghurt, bean mix, etc. 7.3 20.1 24.1 48.1 55.6 53.9 p. <.001 (Surprisingly high figures for the new areas) p. <.001 Table 5.9 Under-twos and Growth Card Use (Appendix, Tables 10, 10.1 and 10.2) Indicator Baseline ‘01 AP New Area Commentary Yes, used Card in their possession Weights registered in last 4 months 68.5 50.1 78.2 80.5 77.0 92.7 66.9 94.1 80.8 p. <.001 p. <.001 p. <.001 Table 5.10: Households with Under-twos which have participated in Educational Counseling in last three months (Appendix B, Table 10.4) Indicator Baseline ‘01 AP New Area Commentary Yes, participated 12.1 53.3 19.2 p. <.001 Table 5.11: Women 10-49 yrs. vaccinated with Tetanus Toxoid (Table 11.4) Indicator Baseline ‘01 AP New Area Commentary Yes 55 99 94.9 p. <.001 Table 5.12: Households that consume more balanced diet (Appendix B, Table 12) Indicator Baseline ‘01 AP New Area Commentary Animal products Vegetables and fruits Oils 70.2 28.3 83.6 76.0 36.0 88.8 76.6 21.3 89.7 All at p. <.001 Table 5.13: Number of farms practicing improved Agricultural Techniques (Appendix B, Table 17) Indicator Baseline ‘01 AP New Area Commentary Zero burn Green manure Organic fertilizer Inter-cropping Crop rotation Live barriers Minimum tillage Chemical fertilizer Management of residue Contour planting 69.3 0.4 6.0 38.5 4.2 11.0 16.8 74.1 56.9 1.2 94.1 9.2 10.5 53.5 10.6 31.6 52.3 85.5 43.1 6.3 92.2 3.8 7.1 22.3 2.7 30.7 25.8 88.2 31.8 2 All at p. <.001 Table 5.14: Percentage of farms with improved Agricultural and Agro-forestry practices (Appendix B, Table 17.1) Indicator Baseline ‘01 AP New Area Commentary 3 or more ag. practices 3 or more agro-f. practices 5 or more agro-f practices 69.3 62.3 21.6 94.1 93.5 54.0 92.2 82.6 32.7 Similar for male- and female-headed farms 5.15 Summary Of the hundreds of activities the project is undertaking, it is achieving statistically significant change on an important number of them. When linked to the impact and monitoring data of previous sections, this finding implies the project is well designed and managed, a topic to which we now turn. 6. PROGRAM OBSERVATIONS This evaluator traveled to project sites for three days. Though the period was short, a number of program observations became possible. These are not number-driven comments as were previous sections, but are reflections of a experienced outside observer. They will be divided into Health, Agriculture, and General observations. Before beginning such a discussion, it is important to reiterate that much of what is going on in the DAP is being done well. It is easy to focus on the “holes” in a list of suggestions, but many of the program concepts of the current DAP are well thought out. Among the most salient appear the following: • Good emphasis on the role of community health volunteers. • Good lesson plans and training materials for them. • Good knowledge by mothers of the nutritional status of their children. • Programming the “family ration” instead of an individual one. • Construction of community health centers in isolated villages. • Good tracking of births and weights for thousands of children; good reporting up of these data. • Good mix of health/agricultural/natural resources interventions; good experimentation. • Good program presence in a given area for an extended period of time. • Well conceived monitoring measurements and good reporting. • Sufficient staffing for the complicated program mix. • Good relationship and interaction with the municipality. • Good relationship and interaction with the Ministry of Health • Good adjustment to the MOH’s Atención Integral de Nutrición charts. 2 On each of these topics, some text could be prepared. Suffice it to say only that the Honduras DAP is a well designed Title II program. Beyond these, a number of additional programming comments seem possible. 6.1 Health Observations Plethora of nutritional data; less analysis? One cannot help but be impressed with the quantity of nutritional data collected in the DAP. Most infant births are captured 3 , also birth weights (bravo!), most infant deaths, monthly growth monitoring for thousands, children adequately gaining weight, those not gaining, those suffering from acute respiratory infection, etc. The list of monthly monitoring is almost fifty. However, could it be that so many nutritional trees are being counted that sight is being lost of the nutritional forest? Recognition of how the project was doing overall may be somewhat less than optimal than if the information was being looked at more holistically. Why, for instance, has the growth faltering in the 12-23 month period not become a project priority before now, if the pattern is so widely evidenced? Why is the lower-than-hoped for exclusive lactation not yet a program focus? It is as if the data review process has been overtaken by record keeping: number- collecting instead of analysis. This could be an area to strengthen in the new DAP. 2 A Honduran Ministry of Health adaptation of the traditional growth card that is being well explained by health volunteers and well understood by village mothers. 3 Under-registering of home births may be an issue requiring further attention. Quasi-illiterate community health volunteers One of the DAP’s key health strategies is relying on community health volunteers: there are almost a thousand such volunteers. These people are the “arms and legs” of the project, carrying out project activities under the tutelage of the CARE health extensionist. The principal criterion for such work, naturally, is a desire to help. However, desire is not sufficient in and of itself. The project has a large list of health themes that volunteer needs to become proficient in. In addition, s/he needs the ability to express her/himself in public venues and to promote improved health practices. Though the “sample size” was only about a dozen, it was clear that three or four of these well￾intentioned young people were barely literate. Their laborious line-by-line reading during a “nutritional counseling session” with village mothers was painful to watch: at times, it seemed the mother knew more of the content than did the health volunteer. The issue of who is selected as a health volunteer—and how effective that service is—is an area that seems unexplored in the current DAP and should be addressed in the next one. Better incentives for volunteers When several key staff were asked the question: “What would you most like to change in the new DAP?” the answer was “a better system of incentives for the promoters.” Until now, the project has not created any system of recompense for community health volunteers; in fact, several of these exhibited certain resentment toward the agricultural volunteers who were reported to receive free seed, fertilizer, etc. The work of the health promoter can take hours and days, and the DAP’s somewhat unthinking assumption that someone will do this for years on end without any form of remuneration seems to be taking unfair advantage of people’s desire to contribute. In fact, the project succeeds or fails based on these volunteers’ ability to perform their function as transmitter of project concepts. Monetary compensation is not suggested. However, staff came up with several excellent suggestions. What is the possibility of the project providing a free grain storage silo (thereby contributing to improve nutritional stability) for volunteers who have contributed X years of service? What is the possibility of expanding the agreement with the Ministry of Health so that health volunteers receive some sort of additional free service at the health post or the hospital beyond what they already receive? What is the possibility of the project financing volunteers’ attending sewing course (corte y confección) as suggested by several women, or attendance at somewhat more advanced health trainings in the capital? These incentives should never become the reason a person contributes to one’s community; however, they could become a way for the project to reward people for such service, and to encourage the good ones to continue. 6.2 Agricultural Observations Small number of farmers in EXTENSA This subject has been addressed in the qualitative evaluation of EXTENSA. It appears the agricultural promotion package has been working with a quite small number of farmers for quite a long time. Discussions with agricultural extensionists suggested primary beneficiaries of technical assistance, called PEPEs, have mostly been no more than seven or eight farmers per extensionist for almost ten years. It is true that each PEPE has supposedly been replicating what he learned to another six or sever peers, the so-called “alumnos” (schooled ones?), giving the extensionist a ‘client load’ of about 40-50. However, it seemed quite likely the alumnos did not receive the same inputs as the PEPEs in terms of improved seed, fertilizer, etc., nor did they receive intensive on-farm supervision as had their more favored colleagues. The “ripple-effect” of EXTENSA has thus not been large. This is not only an issue of less-than-full impact. It also has overtones of unbalanced equity. One energetic farmer was visited who under project auspices had received ten years of subsidized inputs, plus an irrigation system, plus an “experimental” solar-driven pump and 1000 liter water storage tank free of charge, and likely other inputs as well. Nearby farmers were not nearly so fortunate. This issue needs serious realignment in the new DAP. BRHIS sustainability The long-term sustainability of the project’s rotating agricultural funds seems questionable. Loan repayment rates vary greatly from one BRHIS to another. Though time did not permit a review of the figures, many appear to run about 70% while some BRHIS were reported to have been “restructured” entirely. Moreover, the interest rate on these loans varies from village to village and is set by the farmers not CARE. CARE reports that the average interest rate for 104 BRHIS is 34% annually, not bad in terms of sustainability. However the long-term financial viability of these funds is yet in doubt: new agricultural inputs farmers receive with the promotion of this season’s this-or-that are added to BRHIS books, meaning there is a steady infusion of additional capital which completely obscures the self-sustainability issue. More grain storage Several PEPEs interviewed have built small tin grain storage bins, (18 qq. or 0.8 MTs) and reported great success. An off-the-cuff analysis conducted with these farmers suggested two such silos would allow a family of eight to store enough maize to be self-sufficient for an entire year. Also, these farmers reported their grain losses were minimal (compared to losses of 30% or more worldwide with unimproved on-farm storage), as well as being able to sell their grain several months after harvest at a price considerably higher than the post￾harvest low. Such anecdotes mirror the experience of a highly successful grain storage project promoted by the FAO in Nicaragua where gains of up to 400% have been reported. 4 The current DAP reports having built 923 such silos, the LOP target. Conducting a Cost￾Benefit analysis of this activity would provide enormously useful information on its potential for further “scale-up.” Such an analysis could be conducted by an Ag. economist contracted from of one of the Honduran universities. If the new DAP is hopeful of having a dramatic impact on childhood nutrition, the Nicaraguan and Honduran experiences suggest an expanded silo construction component could bring substantial nutritional gains for a great number of children. Drip irrigation The project is to be congratulated for its emphasis on, and experimentation with low-cost, low-tech drip irrigation. This is an important program component that will bring substantial long-term gains, and should clearly remain a program focus in the new DAP. Rainwater harvesting The visitor was pleased to see EXTENSA also experimenting with collecting rainwater runoff by means of gutters connected to small underground cisterns. This technology has been used in arid Arab lands for centuries 5 and is currently being promoted in Nicaragua and in the Dominican Republic. It is an excellent way to avail of Honduras’ heavy seasonal rain for all-year-long benefit. Careful cost-benefit analysis of this intervention should be undertaken to look at the possibility of further expansion in the new DAP. Food for work and road maintenance One of the “default options” for the Food-for-Work component has been establishing work brigades to build and maintain interior village roads. This was a good decision, complementary as it is to the project’s initiatives on farm-to-market commercialization. However, other uses of the food-for-work have been under-utilized. For instance, FFW is also an excellent resource to create productive infrastructure such as terraces, forestry plantations, dead and live barrier soil conservation structures on community land, and others. DAP reporting shows 86% of the FFW resource budgeted for roadwork, only 8.6% for reforestation. Indeed some of the roads traveled on showed quite good maintenance. However, the new DAP could use FFW more creatively in larger developmental ways. One could envisage a 60/40 mix. Over-focus on commercialization 4 Agricultural Recovery and Reconstruction Project Assessment; USAID/Nicaragua; Chemonics Evaluation; Managua, October, 2000. 5 It carries the Arabic-influenced name aljibe—though the term is not well recognized in Honduras. In discussions with agricultural staff, another “mental model” seemed evident. The comment was: “The first DAP concentrated on basic grains; the second DAP on expanded production; the third should concentrate on marketing.” There is some truth to this: a small number of farmers are ready to move into marketing, and the project can provide limited technical assistance in this area. It is also true, in defense of the ag. staff’s perception, that this comment has been put forward as recommendation in both the last Final Evaluation and in the recent Mid-term. However, this evaluator would contend that the subject needs to be addressed more carefully. As has been seen from the quantitative results in production (and in the sale of farming assets,) most farmers are still in the basic-grains/subsistence mode of production; and a heavy emphasis on marketing for these farmers is premature. The comment is offered not to convince EXTENSA to abandon the marketing idea, but to suggest the project should not jump ahead to marketing for a few, if it means leaving behind the majority who are still struggling to make ends meet. The idea is not theoretical: to this observer, the DAP should be concentrating much more on grain storage (for production and self-sufficiency) than on selling vegetables to urban supermarkets—one of the “star projects” visited by the U.S. Ambassador some months ago. One suggestion might be for the project to hire a marketing specialist to work with the few farmer groups at that stage, leaving the rest of the extensionist cadre to remain focused on the grain production/ storage stage. Finding agronomic-and-marketing skills in one field staff per municipality is likely quite difficult; it may be more appropriate to focus staff time and energy on crop production and storage for the majority, while only one or two staff concentrate on marketing for the more advanced minority. Chemical fertilizer for basic grain production In discussions with field staff and farmers, it became clear the project has been financing the purchase of chemical fertilizer for use with traditional varieties of corn. Cost-benefit analysis from other Latin countries suggests this is an ill-conceived program strategy. In fact, one Cooperating Sponsors of a Title II project in Nicaragua has gone so far as to adopt a policy of “zero fertilizer for traditional variety grains.” 6 In this writer’s experience in another Latin country, increases in harvest brought about by chemical fertilizers, when sold at rock-bottom post-harvest prices, did not justify the cost of the input. Focused as they are on obvious increases to production, this hidden de-capitalization is sometimes hard for farmers (and extensionists) to see, but a net de-capitalization it is nonetheless. A careful cost-benefit analysis is required to determine what costs of production are with and without chemical fertilizer under various production and sales price scenarios. Such an analysis could be conducted by an Ag. economist contracted from of one of the Honduran universities. It is recommended the DAP cease promoting fertilizer loans of this nature until such a Cost-Benefit analysis has been conducted. 6 ibid. The PVO cited is World Relief. 6.3 General Observations Departure from “old” municipalities Project planners and the donor should be congratulated for the project’s working uninterruptedly for ten years in the same geographic area. All too frequently, a project is pushed by political or other factors to leave a project area before it has been given enough time to show results. Well done. At the same time, one of the programming decisions that project staff appear to be drifting toward is withdrawing from some municipalities where the DAP has been working. On the surface, this seems an appropriate decision: with limited resources, after ten years in an area, it is time to think about moving to new areas. Several ideas have not been thought through, however. First, in many of the larger municipalities, the DAP does not work in all communities but only three or four. The mayor of Santa Ana said the project has only worked in 11 of her 27 communities, 41%. Thus, a project decision to withdraw from Santa Ana as an “old project area” disguises the fact that the project has never entered over half of Santa Ana communities. Exiting an area characterized by high municipal collaboration, as in this case, before having worked in all villages, in favor of establishing a new presence (and traversing a new learning curve) with a new municipality seems worth further reflection. Relatedly, it may be worth asking whether the DAP has sometimes been working in the closer-in of such communities. When a project selects only a limited number of communities in a given geographic area, the tendency is to favor those that are most accessible, or all villages along a given road or two. Such a decision frequently brings an unconscious bias against those that are the neediest, the furthest away from the municipal center and the most inaccessible. An idea that can be considered is working in every village of a given municipality. 7 Third, as quantitative data have demonstrated clearly, the project has not yet achieved a number of its developmental and nutritional goals. One can ask oneself if it is appropriate to leave a project area when the job is only half done. It appears project designers should think carefully about which areas to work in for the new DAP, and not slip into overly facile decisions regarding withdrawing from “old” areas. 7 The author shared with several staff his personal experience where a donor insisted that a project work in every community of the Province, a decision he resisted mightily. Having the opportunity to visit that province ten years after the project closed, he is convinced the donor was right: the impact of that project, working in every village in that isolated part of the Bolivia Altiplano, is still easily observable years later. Return to some communities to understand real sustainability issues. At the same time, it is time for the project to move out of some areas. However, it appears to the outsider that no real thought has been given to a meaningful “Transition Plan.” (Indeed, there are few projects where this issue has been adequately planned.) CARE reports early work on training municipal and community members in themes of Human Resources, transfer of some equipment and manuals, and working to strengthen horizontal and vertical linkages; but one infers these are seminal efforts. In the new DAP, budget should be set aside, visits should be planned, and research should be carried out to project areas that will be exited to see what really has been created that is “sustainable.” That information could then inform adjustments of the Midterm of the upcoming DAP in FY 2007. Staffing intensity and gender balance The new DAP may want to rethink some of its staffing patterns. The current DAP called for one health extensionist per municipality, one agricultural extensionist per municipality, and one municipal extensionist for every two municipalities, plus two others recently added. The Health component has recently reduced the number of health field staff without negative effects; agricultural extensionists so far work with a limited number of farmers (discussed above); and municipal extensionists may be under-utilized. Project designers may want to give more thought to staff workloads in light of these observations. While on the subject of field staff, the visitor was quite surprised to find that only two of the forty-eight field workers are women. For several years, CARE Headquarters has conducted a campaign to bring more gender balance and equity to the organization, and Honduras is probably one of the leaders in terms of gender balance at the mid- and upper￾management levels. The DAP proposal talks proudly of a gender balance in health promoters of 47% male and 53% female. However at the level of CARE’s field staff— through whom project participants’ needs are voiced—overwhelming male representation is a significant obstacle to women’s concerns being aired. Given CARE Honduras’ participation in a Central American Gender Gap Analysis two years ago, the continuation of male domination at this level of the organization seems all-but unacceptable. The new DAP should resolve this gender gap even if it means undertaking a special hiring campaign in order to do so. Good use of external assistance in nutritional data collection CARE Honduras has used an independent consulting firm for the collection of nutritional impact and agricultural data, for the Baseline and the Final Evaluations for both DAPs. The Honduran firm, ADAI, has provided a good service in the collection of independent and unbiased anthropometric data. Such data collection has been carried off objectively without burdening staff—a not insignificant comment. The same thing cannot be said of the collection of agricultural data, unfortunately. Reliance on farmers’ verbal reports in an interview conducted in May ‘04, based on memory recall of production during the previous calendar year, has not produced trustworthy data. Several important impact indicators beyond that of increased agricultural production are also affected by this flawed methodology. At the same time, there has been a “hermetic seal” between project staff and data collectors. No HOGASA staff has been allowed to accompany impact collection staff at any time. While one wants to avoid project influence during an evaluator’s forming subjective judgments, surely the risk is eliminated when one is collecting objective anthropometric data. The policy has had several negative consequences. First, field staff have not been able to see how conducting rigorous anthropometric takes place, and compare it with their own way of doing things. Second, errors of data collection may have taken place— particularly in reported agricultural yields—that staff could have provided a knowledgeable eye on. Third, there is an ignorance at the level of project staff for months on the results of the data collection. This was reflected by the non-infrequent comment: “Oh, we [project staff] have nothing to do with impact data.” It would appear that project staff’s accompanying evaluation data collectors could bring benefits to both organizations. Over-involvement of an external observer As noted, the Honduras DAP has availed of the services of several well qualified consultants; but, over time, appropriate “consultant distance” has likely been lost. One consultant appears to have participated in the formulation of at least one project strategy paper during the first DAP. This was followed by this same individual’s authorship of a qualitative assessment of the effects of the road construction component in June 2000. The same individual was the primary author of the 2001-2005 DAP proposal in June 2000. This individual was co-author of the January 2002 Midterm Evaluation, and authored the Quantitative section of the Final Evaluation of the first DAP in February 2002. The same person would have authored this Quantitative Final Evaluation except that she assumed a permanent position elsewhere in CARE. The individual’s unquestioned competence notwithstanding, this appears to be overly close involvement and a loss of needed outsider perspective, particularly in the role of evaluator. CARE Honduras is to complimented for its use of qualified external experts, but in this case, the relationship likely went too far. Generalized hunger in 2005? The following comment is not really part of this section, but is difficult to fit in anywhere else. Several farmers interviewed as part of this study told of serious crop losses during the now-ending agricultural season, and, indeed, numerous plots were visited where this was evident. Farmer comments were: “This has not only been a bad year for low-zone production because of rains, but also it has been a bad year for the high zones because of frost.” Agricultural supervisors confirmed the impression. If this is a generalized problem throughout much of Honduras, it is possible that next year the rural areas could be facing: 1) considerably more hunger than this year; 2) considerably higher prices for subsistence basic grains (with consequent impact on childhood nutrition); and 3) considerable scarcity of quality seed for next year’s planting. It is recommended that project and donor staff create a committee to investigate this situation and think about pro-active responses if the worst-case scenario materializes. 6.4 Summary Much of the DAP shows a well-organized, effectively run program. In Health, more nutritional analysis can be conducted and program adjustments made based on such analysis. Project volunteers can be chosen more carefully and some system of incentives could be established to reward these people for their dedicated service. In Agriculture, ways must be found to work with more farmers. Sustainability of the revolving agricultural loan fund needs a serious look. An invigorated focus on on-farm grain storage, expanded drip irrigation, and rain water harvesting are potential areas of new project emphasis. Re-focusing the food-for-work program, the marketing component, and the grain fertilization campaign may be in order. In management, further thought is proposed regarding the withdrawing from “old” municipalities; returning to exited communities during the new DAP to study carefully the issue of sustainability; addressing gender bias in the field extensionist cadre; and exercising more care in the hiring of Honduran and expatriate consultant advisors. 7. CONCLUSIONS 7.1 Impact Level Conclusions As noted at the end of the section on agronomic data, the project has had success in increasing agricultural production, 10% in maize, up to 200% in beans, and 7-8% in sorghum. As a result, the FY01 yield gap for DAP farmers of minus 26.5% has been eliminated. This is an important project accomplishment. Unfortunately, these gains have not resulted in increased income for farm families in dollar purchasing power. While increases in Lempira earnings have been more than fifty percent, the devaluation of the Lempira has effectively resulted in a fall in dollar purchasing power of 25% or more. In a country as dollar denominated as Honduras, this represents a substantial loss of purchasing power—in spite of increased production. This fact, along with other debilitating macro-economic trends such as the fall in the price of coffee, has had an enormously negative effect on farm asset acquisition. The evaluation shows with stark clarity that farmers have been selling off farm assets at an alarming rate: tangible farm assets have dropped from 33% to 19% over the four years of the current DAP. Indigent households in the DAP area have risen from 85.9% to 87.8% in the current DAP, up from 80.3% in the 1996 Baseline. It will take a concerted, well-designed effort to reverse this long-term trend. Off-farm employment would seem to be a partial solution, but in the absence of substantial program campaign directed to such an end, the impact has been modest. The number of days of salaried work by project participants as a percentage of total days worked has fallen from 69% in the Baseline to 50% in the Final Evaluation. Modest gains in the average number of days per worker have occurred. The average number of workdays per female worker has risen more than for males; salaried women report substantially higher wages. Performance in the nutrition realm must be measured against this bleak backdrop. The project has achieved small gains in reducing chronic malnutrition, 2 nd and 3 rd degree height￾for-age malnourishment in children 25-to-60 months: from 61% to 58.4%. Regarding global malnutrition, weight-for-age in 12-23 month-old children, the project has achieved a similar modest reduction, from 30.1% to 27.2%. Small steady improvement in global malnutrition over the life of two DAPs from 33% to 30% to 27%, has not been mirrored by a reduction in chronic malnutrition of the older children: the figure was 54.9% in 1996, it is 58.4% in 2004. This comparison suggests that the program priority should become the chronically malnourished. The project has achieved substantial success in promoting the consumption of a balanced and diversified diet, from 69.4% of families to 80.3%. In addition, the project has been successful in working to improve mothers’ feeding practices for children under 12 months of age, as attested to by the steady reductions in global malnutrition just cited. Though not a family planning program, the project has been successful in promoting the use of family planning in the DAP area. Couples of reproductive age using any form of family planning have increased by 34%; couples using modern family planning methods have increased by over 45%. 7.2 Outcome Level Conclusions Municipal watershed management plans have been prepared in 15 of the 19 municipalities. The project has achieved nearly double the percentage of families implementing agro￾forestry practices compared to Baseline. Somewhat more farm families are reported to have planted two or more new crops than had been targeted. Over 3,100 farmer have participated in rotating agricultural funds, 118% of target. However, participation in complementary Technical Assistance varies, a far smaller number getting lion’s share. Also the proposed “self-sustainability” of these funds seems questionable. The project has substantially exceeded its targets in terms of FFW-assisted road-building and road maintenance. More municipalities and communities have contributed a higher portion of local inputs to DAP activities than planned. However, it is doubtful any municipality or community is meaningfully prepared to carry on DAP activities after the departure of the project. More children are receiving regular growth monitoring than targeted. The number of mothers taking their children to the health post in the case of diarrheal dehydrations and acute respiratory infection has increased. More women than planned now understand concepts of infant feeding and nutrition, a finding corroborated by several Impact tables. An important project accomplishment is that all pregnant women received some pre-natal counseling. The project trained more reproductive health promoters than were planned. Also, more children received full vaccinations than planned and more vulnerable households participated HOGASA than planned. The following indicators have been achieved at levels of statistical significance compared to Baseline: • villagers who have upgraded their houses to adobe from thatch; • villagers who have improved their latrines; • villagers who have visited a (project-built) community center or MOH health post; • village mothers who know their child’s birth weight, (though the knowledge does not seem to result in any long-term differences in nutrition); • 6-to-24 month-old children who are now consuming cheese, yoghurt, bean mix, and other nutritious supplementary foods; • village mothers who have used a growth card for their child, have a growth card in their possession, and whose child’s weight has been registered sometime in the last four months; • mothers with under-twos who have participated in nutritional counseling in the last three months; • village women who are now vaccinated with Tetanus Toxoid. Alarmingly, a statistically significant number of mothers suspend, reduce, or make no change in breast feeding during their children’s diarrhea. There was also a statistically significant number of program children who had diarrhea during the last 15 days compared to Baseline. Compared to Baseline, a statistically significant number of farmers practice improved agricultural techniques: zero burn, green manure and organic fertilizer, intercropping, crop rotation, live barriers, minimum tillage, and others. Also, a number of program participants practice three-or-more and five-or-more of these techniques. 7.3 Management Conclusions Many of the characteristics of DAP programming are well thought out and well applied. However, improved data analysis rather than simple number collection may be an area to look at. Also, one of the program pillars is the community health volunteer. More care should be exercised in who is accepted for this important role, and thought could be given to providing non-monetary stimulus to reward these dedicated volunteers. Too few farmers receive on-going technical assistance from CARE extensionists. Also, the project can devote more attention to grain storage and drip irrigation. It may need to devote somewhat less attention to food-for-work roads (in favor of agro-forestry and productive infrastructure), the use of chemical fertilizers on unimproved basic grain varieties, and to commercialization for those who still are in the subsistence mode of production. The new DAP should carefully weigh departure from “old” municipalities, since approximately half the villages of current municipalities have not had the services of the project. Some villages need to be exited, but there should be a well-articulated activity to return to these communities several times during the next DAP to look at the issue of what is really sustainable when the DAP departs. CARE has made good use of national data collectors, though adjustments need to be made, and of international consultants; but no relationship should continue so long that due consultant objectivity is sacrificed. 7.4 Macro Conclusion At the macro level, perhaps the most difficult recommendation is that the project needs to become more efficient in combating malnutrition and hunger in a Honduran rural context that only seems to be getting worse as time goes on. 8. RECOMMENDATIONS Recommendations arising out of the preceding narrative that seem the most important are as follows. 8.1 In HOGASA, pay more attention to the weaning period From evaluation data, it is clear the program should pay more attention to the weaning period. Small steady gains have been made in children’s nutritional status from birth to 6 months over the life of the two DAPs that have not occurred in the 7-24 month-olds. It is at this time that both global and chronic malnutrition take significant jumps that never improve. Among the ideas program nutritionists may wish to consider are the following: • re-sensitize field staff to the critical importance of this period; • do the same for all community health volunteers; • develop an enhanced protocol to respond to the first “red dot” episode, strengthening the MOH referral which is the current practice; field staff to follow-up aggressively to see whether the mother has visited the MOH; • strengthen the home visit system to encourage mothers to exercise special care at this time in the child’s life; • emphasize the importance of more-but-smaller feeding episodes; • re-emphasize the important of more frequent breast feeding during diarrhea; • create a “state of alert” when a competing sibling is born, stressing to the mother the importance of staying focused on the older child also; • give more program emphasis to good hygiene habits as the child learns to walk. None of this is much different than what is already in place. Mostly it only means emphasizing current practices, but in a more concentrated way. Overall, the program should begin a focused campaign to catch 7-24 month olds before they fall into malnutrition. If one were to express a “stretch” goal for such a program, it would be something like: “in DAP communities reduce chronic malnutrition to 15% and global malnutrition to 25% by 2010.” 8.2 Hire more women in order to emphasize this focus The point has been made in the narrative that the cadre of field extensionists is completely male-dominated. Notwithstanding that many men make excellent health workers, women are usually more sensitive to children’s need than men, (female extensionists too). Also, rural women generally open up more easily and respond more deeply to a female extensionist than a male one. The over-concentration of men results in mothers’ perspectives and unexpressed aspirations likely not being reported up the DAP line of responsibility nearly as much as is needed. In order to carry out this recommendation, CARE Honduras will have to make a special effort to attract female field workers. It will mean among other things: • deliberately reaching out to learning institutions that favor “female careers,” such as auxiliary nurses, social workers, and child pedagogues; • creating a “trainee” program and hiring more than the needed number of female staff, assuming that some will not make it to permanent status; • doing away with unconsciously prejudicial job requirements such as “ability to drive a motorcycle” and “previous field experience required;” • buying smaller motorcycles for smaller (female) staff; • creating a training program for motorcycle driving for all new employees; providing the time and space so these skills are learned before the person heads into the field on her own; • expecting and accepting that female candidates will have less paper qualifications than men, and hire them anyway; • when recruiting and interviewing, weighing “nurturing skills” and “outgoing personality” as much as, or even more than, academic qualifications and paper skills. Though an outsider is hesitant to establish a numerical target, DAP managers should shoot for a substantial number of female extensionists by the end of the first year of the new project, perhaps as much as 40%—twenty young women—and recognize the challenge of such an ambitious target. 8.3 Provide production incentives to health promoters (similar to incentives in place for agricultural promoters) Some health volunteers have been working for their community for nine years without recompense. The project should find a way to reward these people in a meaningful fashion without making the incentive so big that people are attracted by the incentive instead of by the service. Some ideas are the following: • After X years of service, give the volunteer a grain silo in recognition of his/her service to the community. • Award the health volunteer some of the subsidized agricultural inputs that are routinely part of the ag. extension package. • Offer financial support for volunteers’ attendance at courses on additional health themes. Some volunteers expressed an interest in First Aid, and there must be many more course offerings of a basic nature offered by institutions that would deepen volunteers’ skills. • Finance (or co-finance) similar skill up-grading for MOH nurse auxiliary staff. • Negotiate a protocol with the MOH that would provide health volunteers with special treatments or some additional free MOH service beyond that offered to the general public. • Perhaps in coordination with ASHONPLAFA, pay for some ASHONPLAFA services for these health volunteers. • After X years of service, offer project financing for sewing classes (corte y confección.) • Discuss with volunteers other promoter-identified aspirations and look for ways to finance them. 8.4 Embark on a major effort in grain storage (to move farmers out of subsistence farming) In order to move DAP farmers, 87.8% of whom are indigent, into commercial production, it is first necessary to assure grain self-sufficiency. Gains in field production in maize and sorghum over the last four years have been at the 7-to-10%, a reasonable achievement in multi-dimensional ag./nutrition program like the DAP. Literature on small farmer grain storage, and current experience in Nicaragua and with selected DAP farmers, suggest that gains from a grain storage activity (in reduced crop losses) could be 30%. Also, two grain silos of 18 quintales reportedly can make a family of eight self-sufficient in maize for an entire year. The project should immediately contract the services of a Honduras university agricultural economist to confirm the accuracy of these anecdotal impressions, and begin a major planning session with the agricultural extension team how to incorporate an expanded grain silo program in the DAP. If needs be, senior project agriculturalists should travel to the FAO project in Nicaragua to do a Lessons Learned field study. One would encourage project planner to formulate the next DAP in terms like: “75% of farm families are self-sufficient in basic grain consumption by the end of 2010.” Such a strategy would have a noticeable effect on reducing rates of malnutrition also. 8.5 Redesign the EXTENSA program methodology The new DAP cannot continue to work only with seven farmer per village, as some suggest the pattern has been for the last two DAP cycles. The number of active project participants must take an exponential jump. However, this is easier to say than to do. To accomplish the target, it is likely EXTENSA will have to re-think its program methodology entirely. It does not seem possible, for instance, that the project can continue to rely on the one-to-one program model: there are too few staff to rely exclusively on on-farm technical assistance visits as has been the pattern in the past. The project will have to do many things differently, including: • training sessions for a number of farmers, not just for one; • field visit days with ten or fifteen farmers in tow instead of just one; • agronomic courses at centralized venues rather than individual farm visits; • the primary skill of the field staffer as teacher rather as agronomist. This is a fundamental re-thinking of the DAP role in agriculture and is not going to be an easy change. As a result, one would posit more modest goals than in some of the other recommendations. One may think of something like this: “Effectively reach 6,000 farmers with a revised agricultural model, such that self-sufficiency in grain production is achieved by three-quarters of farmers by 2010.” 8.6 Collect agronomic data at the end of the harvest season with trained agronomic investigators The confusing yield data this evaluation has had to rely on need to be improved. Though this document has ratified the validity of external data collection as a concept, it seems clear the current firm needs to have its efforts strengthened by more agricultural expertise, or another firm needs to be hired. The following recommendations are offered: • Collect agricultural production data at harvest time, not verbal reports five months later. • Hire a Honduran firm specializing in agricultural studies to carry out the work, which would be complimentary to the nutritional work. • Rely on randomized crop-cuttings not on verbal reports. • Train project staff in randomized crop-cuttings; at harvest evaluation time, transfer field staff from their area of work to another area so they are not evaluating their own performance; supervise the crop cutting carefully; create a prize for the most rigorous data collected (not necessarily for the best production); • Conduct the first of such exercises now at the close of the YR4 agricultural calendar. • Compare CARE-staff collected data and evaluator-collected data at the end of each year in a formal workshop. Make program adjustments. More scientific rigor will make DAP claims of improved productivity more reliable. 8.7 Overall, scale back on program innovation and concentrate focus Having congratulated the project on a number of innovations, the evaluator now finds himself in the paradoxical position of recommending that the project scale down and return to basics. This means avoiding activities things that will not pay large-scale dividends in terms of improved lives of project participants, and cease other activities, interesting in themselves, that distract from the overall focus of the project. The overall focus should be to reduce the percentage of indigent families in DAP areas in the face of a deteriorating macro-economic climate. The recommendation in this paragraph is to hire a broadly experienced agricultural economist/consultant to study with project staff where the most likely big-payoff items are in agriculture. To this writer ideas that need to be re-thought are the following: • What is the scale-up potential of home reconstruction grants? If not large, thought should be given to doing away with this activity. • What is the replicability of the rainwater catchment tanks? If not large, thought should be given to doing away with the activity. • Chemical fertilization of traditional grain varieties should cease. • What is the sustainability of the BHRIS? How long can these constantly-refinanced artificial structures survive? How could the project engage in seed multiplication and distribution without this unsustainable mechanism? • How much time and money does the municipal component take? How much value￾added do municipal field extensionists bring to reducing indigence in DAP areas? Can some of these funds be used more effectively elsewhere in the project? • How can the FFW ration be used to create income-generating structures (for instance: earthen dams, cattle watering holes, agro-forestry plots, terraces, etc.) instead of being largely used for inter-village road maintenance? • At first glance, the scale up potential of drip-irrigation seems obvious, but what is the cost per capita of this activity? How far can it be expanded within the DAP budget? Can farmers repay part of this cost (to CARE) so that such repayments can be rolled over to finance more? • Can the grain storage structures—which this evaluation is recommending strongly— be programmed on a partial subsidy basis, and reflows (to CARE) used to build more within a given budget? • What is the cost of seed stock improvement? How can it be scaled up throughout the DAP area? How can thousands of farmers avail of this genetic improvement instead of a few hundred? SUMMARY: The 2001-2005 Honduran Title II Food Security Program, likely like its predecessor, has been well-designed and professionally managed and is showing important impact in maintaining—even somewhat reducing— rates of malnutrition in a deteriorating macro environment. The overall vision of the project is sound, and it appears the program can be tightened in a number of small ways so that its impact can be substantially enhanced. APPENDIX A METHODOLOGY I. INTRODUCCION En 1996, CARE Internacional en Honduras puso en marcha la reorientación del Programa de Seguridad Alimentaría, Título II, para coadyuvar a aliviar de forma más efectiva el problema de inseguridad alimentaría del país. El objetivo central de ese nuevo enfoque fue preparar a los hogares para procurarse los medios necesarios con los cuales asegurar una alimentación básica adecuada y sostenible. Los recursos del Programa de Seguridad Alimentaría Titulo II financian la ejecución de tres proyectos específicos en los departamentos de Intibucá, Lempira y la Paz, mismos que fueron seleccionados mediante una estrategia de intervención encaminada a aliviar las condiciones de vida de las regiones más empobrecidas de Honduras. Dichos proyectos son: Hogares Gestores de Atención en Salud (HOGASA), Extensión para la Seguridad Alimentaría (EXTENSA) y Proyecto de Oportunidades de Desarrollo y Empleo Rural (PODER). Precedió a la ejecución del programa de seguridad alimentaría un estudio de línea de base, realizado en 1996 con el propósito de establecer datos reales sobre las condiciones socioeconómicas prevalecientes entre las familias residentes en el Área de Influencia de CARE. Asimismo, la Línea de Base proporcionó los indicadores de impacto y monitoreo, necesarios para las evaluaciones que se realizarían a mitad y final del Programa, habiéndose realizado la evaluación de Medio Término en 1998 y la Final el 2001. La Evaluación Final del 2001, sirve actualmente como Línea de Base para comparar los datos de la actual Evaluación Final 2004. II. DESCRIPCION DEL ESTUDIO El objetivo central de la actual Evaluación Final 2004, es evaluar el grado de avance e impacto del Programa de Salud Alimentaría, Título II en las condiciones socioeconómicas de las familias en las comunidades beneficiarias de la zona de influencia a tales fines, la evaluación pretende determinar si en el área de influencia, se ha logrado: Incrementar la disponibilidad de alimentos, e incrementar el acceso y utilización biológica de los mismos. III. ASPECTOS METODOLOGICOS 1. Diseño del Estudio de Evaluación Final 2004 La Evaluación Final se realizó en los tres tipos de Áreas en que CARE ha dividido sus intervenciones: Áreas a Continuar (AC) significa que ahí se están ejecutando proyectos del Programa, pero que aún requieren intervención por más tiempo. Áreas de Salida (AS) son las que han tenido participación intensiva en los proyectos de CARE, desde los inicios del Programa y a criterio de la institución ya se alcanzaron los cambios planeados. Ambas áreas constituyen en el presente estudio el AREA DE PROYECTO (AP). Finalmente, las AREAS NUEVAS (AN) que son en las que CARE aun no está desarrollando ninguno de los tres proyectos y se tomarán entonces como el nuevo “Grupo Control”, para la Línea de Base 2006-2010 . Para la presente evaluación se utilizaron diferentes factores de ponderación para presentar los datos tanto del área de proyecto de la Evaluación Final 2004 como en los datos para la Línea de Base 2006-2010, mismos que de acuerdo al tamaño de la muestra han sido proporcionados por CARE. Los factores de ponderación antes citados son los siguientes: AREA DE PROYECTO 2004 : AP 04= (AC+AS) Áreas a Continuar =0.986 Áreas de Salida = 1.021 LINEA DE BASE 2004 : LB 2004= (AC+AN) Áreas a Continuar = 0.917 Áreas Nuevas = 1.127 2. Instrumento y Equipo para la Encuesta: Para este estudio se aplicó la misma boleta utilizada en el estudio de Línea de Base 1996, Evaluación de Medio Término 1998 y Evaluación Final 2001, la cual fue sometida a revisión y actualización acorde a las prioridades revisadas a partir de reuniones de discusión entre el equipo de CARE y el de ADAI. La boleta comprende las siguientes secciones: ? Composición del Hogar ? Características de la Vivienda ? Ingreso ? Producción Agrícola ? Comercialización ? Salud ? Prácticas alimentarías en niños menores de dos años ? Consumo de alimentos ? Antropometría en niños menores de cinco años ? Participación Ciudadana La composición del hogar incluye aspectos tales como número de personas en la vivienda, por sexo, edad, alfabetismo y escolaridad. Las características de la vivienda incluyen, material de las paredes, piso, techo, número de piezas, servicio de electricidad, clases de estufa o fogón más usados para cocinar, sistemas de eliminación de excretas, fuentes de agua para beber y uso diario, tratamiento del agua y presencia de animales sueltos dentro de la vivienda. La sección de salud incluye el uso de servicios de salud y nutrición, estado de vacunación, prácticas de salud familiar, enfermedades diarreicas, infecciones respiratorias agudas, asistencia a control de crecimiento y salud reproductiva. La sección de prácticas alimentarías comprende información sobre lactancia materna, tipo de líquidos y alimentos sólidos dados al niño el día anterior a la entrevista, frecuencia diaria de consumo de alimentos semisólidos. La sección de consumo de alimentos consiste en una forma simple de estimar el patrón de consumo de alimentos consumidos el día anterior, sin establecer las cantidades ingeridas, con el objeto de conocer que grupos de alimentos están incluidos en la dieta familiar. La sección de comercialización se incluyó esta vez para conocer el tipo de productos comercializados, fuentes de información para saber donde vender los productos, dificultades en la comercialización y forma de organización para vender los productos. La sección de participación ciudadana se introdujo para conocer sobre las organizaciones existentes, razones para que funcionen más que otras, planes comunitarios, participación familiar, convocatorias y relación con la municipalidad. La sección de antropometría incluyó la toma de peso y talla de los niños menores de cinco años. Para la toma del peso se utilizaron Balanzas Salter con capacidad de 25 Kgs. y una precisión de 0.1 Kg, diariamente estas balanzas eran calibradas con pesos conocidos de 1 Kg. para verificar la exactitud de la misma, además se utilizaron otros accesorios como calzones y pañales de tela para colocar a los niños y lazos para colgar las balanzas. Para la toma de la talla se utilizaron tallímetros portátiles de madera, contando para la medición con cintas métricas graduadas en centímetros y milímetros. La sección de ingresos comprende la acumulación de activos líquidos y de producción, trabajo por cuenta propia, trabajo por salario y otros ingresos familiares. La sección de producción comprende lo referente a producción y destino de cultivos temporales ó anuales y de cultivos permanentes, disponibilidad de especies menores. La cantidad y complejidad de información a recabar con esta boleta hizo necesario que sus diferentes componentes fueran relacionados mediante un número de orden asignado a cada miembro de la familia, y poder controlar de esa forma la consistencia de los datos a través de los diferentes cruces de variables. 3. Tamaño de la Muestra El tamaño de la muestra fue proporcionado por CARE, mismo que fue establecido en 1680 hogares para toda el área del proyecto, asignados de la siguiente manera: Areas a Continuar 696 Areas de Salida 504 Areas Nuevas 480 Total 1680 De igual manera CARE nos proporcionó la cantidad de segmentos que cubrirían cada uno de los tipos diferentes de aldeas según la intervención de CARE, haciendo un total de 70 segmentos de 24 viviendas cada segmento. Los segmentos estaban distribuidos de la siguiente forma: Areas a Continuar 29 Areas de Salida 21 Areas Nuevas 20 TOTAL 70 3.1 Selección de la Muestra CARE proporcionó un listado de aldeas del área de influencia en el programa de Seguridad Alimentaría Titulo II, dividido en municipios y aldeas, que constituye el área geográfica del programa. Estos municipios y aldeas correspondían a los departamentos de la Paz, Intibucá y Lempira. En el listado también se encontraba identificado el desglose de los tipos de intervención que utiliza CARE actualmente para canalizar los recursos del programa. Basado en el listado de aldeas del área geográfica del programa se procedió a la elaboración del Marco de Lista el cual incluyó el desglose de los tres tipos de intervenciones de CARE por departamento, municipio, aldea, número de vivienda y segmento en el cual se detectaba la cantidad de viviendas participantes en cada aldea o caserío según intervención. Tanto el listado de viviendas como de segmentos se obtuvieron del marco muestral del censo del año de 2001, del Instituto Nacional de Estadística de la República de Honduras. Posteriormente se construyó el Marco de Area, siguiendo el mismo procedimiento de desagregación, por departamento, con cada municipio, luego en cada municipio con sus aldeas y caseríos, viviendas y segmentos se desglosa el número de viviendas por departamento, por municipio, por aldea, por caserío, lo mismo se va desglosando a nivel de segmento. (Ver Anexos) A este nivel, con los segmentos identificados se elaboró el Marco de Lista de segmentos siguiendo el mismo procedimiento anterior de desagregación, por departamento, municipio, aldea, caserío dentro de cada intervención en forma proporcional de acuerdo a la estructura de la población. El siguiente paso fue efectuar la distribución de la muestra dentro de cada intervención por departamento y por municipio, distribuyendo los departamentos de manera proporcional, según el porcentaje de viviendas de cada uno de ellos. Similar es el caso para la selección de los segmentos por comunidades. A continuación se detalla el, resultado de la distribución de segmentos por departamento, en la cual se aplicó la estructura de viviendas del Censo de 2001 y en la que se determinaron las viviendas por departamento y se calculó el número de segmentos; cada segmento estaba conformado por 24 viviendas. AREAS A CONTINUAR (29 segmentos asignados) Departamento Cantidad de Viviendas 1/ % Viviendas de La Muestra Total Segmentos La Paz 692 30.6 216 9 Intibucá 1,078 47.6 336 14 Lempira 493 21.8 144 6 TOTAL 2,263 100.0 696 29 1/Fuente: Censo de 2001 Total de viviendas: 696 muestra AREAS DE SALIDA (21segmentos asignados) Departamento Cantidad de Viviendas 1/ % Viviendas de La Muestra Total Segmentos La Paz 645 30.6 168 7 Intibucá 805 38.3 192 8 Lempira 656 31.1 144 6 TOTAL 2,106 100.0 504 21 1/Fuente: Censo de 2001 Total de viviendas: 504 muestra AREAS NUEVAS (20 segmentos asignados) Departamento Cantidad de Viviendas 1/ % Viviendas de La Muestra Total Segmentos La Paz 1,703 49.7 240 10 Intibucá 168 4.9 24 1 Lempira 1,550 45.3 216 9 TOTAL 3,421 100.0 480 20 1/Fuente: Censo de 2001 Total de viviendas: 480muestra La selección de segmentos se efectuó de manera aleatoria dentro de cada estrato. 3.2 Factores de Ponderación Para Evaluación Final 2004: EF04= (AC+AS) Se denomina Area del Proyecto (AP) a la suma de las Areas a Continuar más las Areas de Salida AP*= (AC+AS) a las cuales se le aplicó un factor de ponderación de acuerdo a la muestra así: AC = 0.986 , AS=1.021 . Area Nueva AN= 1. Para Línea de Base 2006-20010 LB04=(AC04+AN04),son (las Areas a Continuar del 2004 + Areas de salida del 2004), a las cuales se les aplicó el factor AC =0.917 y AN= 1.1270 3.3 Cartografía Una vez seleccionada la muestra por cada tipo de intervención se adquirió la cartografía del área en el Instituto Nacional de Estadística (INE), basada en el censo nacional del año 2001, con lo que se procedió a actualizar los segmentos seleccionados, para lo cual los supervisores de grupo procedieron a la escogencia de las viviendas de la muestra, con sus respectivas sustituciones. Con la actualización cartográfica se logró establecer con precisión el número de viviendas habitadas y deshabitadas en cada segmento y la ubicación de las mismas. Dicha actualización permitió, además, determinar exactamente el área de trabajo, la distancia entre comunidades, establecer de manera eficiente las rutas de trabajo, el tiempo y logística necesarios para el levantamiento de los datos. 4. Universo y Unidades de Análisis El universo del estudio lo conforman el total de familias del área de influencia de la zona a la cual se infieren los resultados obtenidos en la muestra. El grupo control, aunque no forma parte del universo, es de suma utilidad a efecto de comparar si los resultados del Programa de Seguridad Alimentaría, Titulo II se deben exclusivamente a sus intervenciones, o si por el contrario, está influenciado por otros factores externos. En general, la unidad de análisis es el hogar. Esta unidad de análisis se evalúa con el enfoque de hogares dirigidos por hombres y los dirigidos por mujeres. Adicionalmente, se consideran los niños menores de cinco años como grupo enfoque para el caso específico del estado nutricional y los niños de 0-23 meses para las secciones relacionadas con salud y nutrición. En el caso de la sección agrícola, se toman como unidad de análisis los hogares se reportan uno o mas cultivos, osean, los hogares agrícolas. 5. Capacitación La complejidad y tamaño del cuestionario requerían de un proceso intensivo de capacitación para asegurar la correcta aplicación del mismo en el campo. A ese fin, se implementó una fase de capacitación de encuestadores, la cual tuvo una duración de tres semanas. El curso de capacitación se impartió a 40 personas, de las cuales se seleccionarían 24 encuestadores, 8 supervisores de grupo y 2 codificadores. Vale la pena destacar que en dicho curso participaron algunas personas que ya habían trabajado en el estudio de Evaluación 2001 y también en Línea de Base 1996, lo cual fue de mucha utilidad para facilitar la comprensión del cuestionario por parte de los candidatos nuevos. El curso de capacitación consistió en un taller teórico-práctico sobre el contenido del cuestionario, técnicas de entrevista y llenado correcto de las secciones de la boleta. Debido a que cada una de las secciones del cuestionario tenían sus características particulares, en el taller se cubrió por separado cada sección, de acuerdo a su complejidad (ingreso, producción, etc.) para cada área se contó con la participación del personal técnico especializada en cada rama. En todos los casos se realizaron pruebas teóricas y prácticas en el aula a fin de asegurar la debida comprensión de la boleta. Todos los candidatos fueron estandarizados en el manejo de las técnicas antropométricas en niños menores de cinco años, así como en el debido uso del equipo a utilizar (balanza y tallímetro). La práctica se realizó en una guardería infantil en donde habían niños de todas las edades que iban a ser objeto del estudio. La estandarización se realizó con grupos de 10 niños, quienes se pesaban y medían 2 veces, para después hacer un análisis de las diferencias. Para la realización de estas prácticas los encuestadores se organizaron en diferentes grupos de trabajo. Durante la capacitación se hizo especial énfasis en las técnicas correctas de toma de peso y talla, lectura y registro exacto de los datos. En la parte correspondiente a la producción agrícola se hizo hincapié en los conceptos de, formas del producto, unidades de medida, ventas, autoconsumo etc, debido fundamentalmente a que la producción agrícola que se analizó es la que correspondía del 1º al 31 de diciembre del año 2003 y por otra parte, que de esta información depende el correcto cálculo de los ingresos agrícolas de las familias. La evaluación para seleccionar el personal de campo consistió en dos tipos de prueba: un examen teórico y una prueba práctica (Encuesta Piloto). Al primero se le dio una ponderación de 40% y la prueba práctica se ponderó en 60%. La calificación mínima que se debía acumular en las dos pruebas era 70%. La Prueba Piloto tuvo una duración de cuatro días y se realizó en comunidades con características similares a la muestra seleccionada. Esta prueba se desarrolló con todas las secciones de la boleta y se aplicó completamente la metodología a implementar en la encuesta definitiva (destreza cartográfica, distribución de grupos, supervisión de campo, revisión de boletas en el área de trabajo, codificación, etc). Posterior a la prueba piloto, se efectuaron algunos ajustes a la boleta para proceder a la edición de la boleta definitiva y se realizó un reforzamiento a los a los encuestadores, para consolidar algunos conceptos en los que se observaron debilidades en la prueba de campo, lo cual fue consensuado en la reuniones de evaluación sostenida en el campo conjuntamente entre el personal de CARE y el de ADAI. 6. Levantamiento de Datos El trabajo de campo se realizó del 6 al 25 de mayo de 2004 con 8 grupos de tres encuestadores. Cada grupo contó con un supervisor quien realizaba el control de calidad de la información recabada. Después de la jornada del levantamiento, los cuestionarios eran revisados diariamente por los supervisores, el jefe de campo y el equipo de apoyo en el terreno y se realizaban re-visitas al azar para verificar los datos y en otros casos, corroborar datos que parecieran inconsistentes. Dicho control fue a su vez supervisado directamente por el Coordinador General de campo por el equipo Técnico de ADAI y por el Enlace Técnico de CARE, quienes permanentemente se trasladaban a las distintas comunidades de la encuesta para verificar el proceso de control de calidad de las boletas y asegurar que las mismas retornaran a la sede en el más breve tiempo, para ser analizadas, codificadas y accesadas al programa de entrada de datos. Los codificadores también recibieron la capacitación y participaron en la encuesta piloto, manejaban listados correlativos de paquetes de encuestas, rotulaban y empaquetaban las boletas de acuerdo al departamento, municipio, aldea y caserío, número de segmento, número secuencial y se le asignaba número de paquete para facilitar su búsqueda durante el proceso de digitación y limpieza. Fueron contratados dos transcriptores, a quienes se les dio una capacitación de tres días y era el encargado del control de calidad y cumplimiento de metas de los codificadores. Realizaron labores de capacitación y se les asignó una tarea mínima de 40 boletas diarias promedio por transcriptor. Para los aspectos de programación se contrato un encargado de programación, un asistente y un programador especial para la sección de Antropometría. Adicional al levantamiento de las 1680 boletas de los hogares, se ejecutó simultáneamente, el trabajo de recopilación de datos de una boletas más de las que se levantaron. 70 Boletas de Comunidades que fueron asignadas a los supervisores de grupo para ser llenadas mediante entrevistas con los líderes de cada comunidad encuestada dentro de la muestra. Las boletas antes mencionadas tenían el objetivo de conocer la situación organizativa de comités de desarrollo comunitario, planes de acción comunitaria y problemas existentes dentro de las comunidades. Posteriormente se procedió a la limpieza, digitación y procesamiento de la información correspondiente a este instrumento. 7. Procesamiento y Limpieza de Datos Para la entrada de datos se utilizó el programa Data Entry de CSPRO 2.5 el cual genera archivos en ASCII. Para los datos finales de salida se usó el programa SPSS. Para darle el formato de presentación se trasladó a EXCEL. En el caso de Antropometría se generó la Base en Dbase para convertirlo a ASCII y luego reprocesarlo con el paquete ANTRO I y generar los puntajes Z de peso y talla. La limpieza de datos se hizo en varias etapas, así: ? Se hizo una revisión completa de cada cuestionario durante el proceso de codificación, a fin de reducir al mínimo la entrada de datos equivocados. ? Al momento de digitar los datos, se utilizaron saltos y rangos establecidos en el programa de entrada, para cada variable, de acuerdo a los pases y códigos del cuestionario. Ello permitió que los datos que no correspondían a una pregunta en particular fueran rechazados y nos obligaran a verificar los mismos. ? Se aplicó, de igual manera al programa una serie de validaciones que permitieron verificar datos como por ejemplo que la edad de un niño fuera mayor que la del jefe del hogar, ó que en un hogar no pueden existir dos jefes de hogar. ? Finalmente, se hicieron distribuciones de frecuencia para las variables, estimando sus valores máximo, medio y mínimo, así como su desviación standard; a fin de identificar valores extremos. Dichos extremos se revisaron nuevamente en la boleta respectiva y posteriormente si era del caso, se hicieron las correcciones necesarias. ? Los programas de Línea de Base estaban en SYS; para unir las bases de datos Línea de Base 2001 con Evaluación Final 2004, se fusionaron estos archivos Sav.