Food for the Hungry Multi‐Year Assistance Program  2008 – 2011 Katanga Province, Democratic Republic of the Congo FINAL EVALUATION REPORT 12/20/2010 Author: John Wiater Independent Consultant, Development and Humanitarian Affairs The final evaluation of the FH-implemented 2008-2011 MYAP was conducted from August 18 through September 10, 2010 in order to understand the program nature, objectives, focus and requirements so as to assess progress made against stated objectives, identify lessons learned and to make recommendations for program enhancement that could be of value in future programming 1 Acknowledgements The author of this report would like to express his gratitude to the Food for the Hungry DRC staff for their support and contributions to survey preparations, training, logistical support and implementation. In particular, he would like to acknowledge the dedicated to field-level operational and administrative coordination prior to, during and after the evaluation survey exercise which was provided by Esther Wong, as FH survey manager. She, along with Pierre Djomo, Dr. Jean-Pierre Okitakoy and Kingsly Mforteh provided valuable assistance and support in the refinement of/revisions to the survey questionnaire and the organization of a training workshop for survey numerators as well as training of data entry personnel. The latter two also provided participated in, and lent valuable points of view with regard to analyzing information provided by focus group respondents. Finally he wishes to acknowledge and thank all of the survey and data entry team members for their hard and diligent work during the course of the exercise. 2 List of Acronyms AIDS Acquired Immune Deficiency Syndrome BM Beneficiary Mother CBO Community-Based Organization CDC Community Development Committee CFW Cash For Work CG Care Group CHW Community Health Worker CI Confidence Interval DIP Detailed Implementation Plan DRC Democratic Republic of Congo EFSA Emergency Food Security Assessment EHA Essential Hygiene Action ENA Essential Nutrition Action EHB Essential Health Behavior EWS Early Warning System FANTA Food and Nutrition Technical Assistance FAO Food and Agriculture Organization FFLG Farmer Field and Life Group FFP Food For Peace FFW Food For Work FH Food for the Hungry GAM Gross Acute Malnutrition GoDRC Government of the DRC Ha. Hectare HAZ Height for Age Z-score HDDS Household Dietary Diversity Score HIV Human Immunodeficiency Virus IPTT Indicator Performance Tracking Table ITBN Insecticide Treated Bed Net Kg. kilogram Km kilometer KPC Knowledge, Practice, Coverage LQAS Lot Quality Assurance Sampling M Months MAHFP Months of Adequate Food Provisioning ML Mother Leader MOH Ministry of Health MYAP Multi-Year Assistance Program NGO Non-Governmental Organization 3 OCHA Office for the Coordination of Humanitarian Affairs ORS Oral Rehydration Salts ORT Oral Rehydration Therapy P4P Purchase for Progress POU Point of use PPS Probability proportionate to size SD Standard deviation SENASEM National Seed Service (English translation of the French acronym) SO Strategic Objective SPR Seed Protection Ration UNDP United Nations Development Program UNHCR United Nations High Commission for Refugees UNICEF United Nations Children’s Fund US United States USAID United States Agency for International Development WASH Water and Sanitation/Hygiene WATSAN Water and Sanitation WAZ Weight for Age Z-score WFP World Food Program WHO World Health Organization WHZ Weight for Height Z-score 4 Table of Contents 1.Introduction............................................................................................................................................... 7 2.Objective and Methodology of the Evaluation ....................................................................................... 8 3. Brief Description of the Program .......................................................................................................... 10 4.Detailed Analysis of Findings ................................................................................................................. 12 4.1 Strategic Objective 1:  Improved Livelihood Capacity of Vulnerable Households............................ 12 4.1.1 Number of households which received seeds and tools........................................................... 13 4.1.2 Quantity of seed and tools distributed...................................................................................... 14 4.1.3 Number of households which received Seed Protection Rations ............................................. 15 4.1.4 Number of associations which engaged in multiplication activities.......................................... 16 4.1.5 Quantity of high quality seed and cuttings distributed to associations for multiplication ....... 16 4.1.6 Quantity of small livestock distributed to associations for multiplication ................................ 16 4.1.7 Percentage of trained beneficiaries who score 70% or higher on post training tests............... 17 4.1.8 Percentage of FFLGs which conduct a participatory learning exercise ..................................... 17 4.1.9 Percentage of beneficiaries who use a minimum of 3 sustainable agricultural practices ........ 19 4.1.10 Percentage of beneficiary households which planted at least 70% of seed received............. 21 4.1.11 Average yields of selected crops per beneficiary household .................................................. 22 4.1.12 Total area planted by association for high quality seed/cutting multiplication...................... 23 4.1.13 High quality cassava cuttings produced by multiplication associations.................................. 24 4.1.14 High quality seed produced by multiplication associations .................................................... 24 4.1.15 Percentage of beneficiary associations that use a recommended livestock technologies ..... 24 4.1.16 Percentage of households in the area which own small livestock .......................................... 25 4.1.17 Average number of small livestock produced per beneficiary association ............................. 28 4.1.18 Number of soil and water conservation structures created.................................................... 29 4.1.19 Number of sustainable soil and water conservation management technologies transferred 29 4.1.20 Percentage of beneficiary farmers who use sustainable soil and water technologies ........... 29 4.1.21 Number of soil and water conservation structures that are still intact or repaired ............... 31 4.1.22 Area under soil and water management................................................................................. 31 4.1.23 Number of producer associations that have developed statutes and bylaws........................ 32 4.1.24 Number of producer associations using market information for decision‐making................. 32 4.1.25 Total income of small livestock multiplication associations.................................................... 33 4.1.26 Total income of seed multiplication associations.................................................................... 35 5 4.1.27 Number of months of adequate food provision...................................................................... 35 4.1.28 Household dietary diversity score ........................................................................................... 38 4.2 Strategic Objective 2:  Improved Human Capabilities of Vulnerable Households............................ 40 4.2.1 Total number of care groups currently active in health promotion.......................................... 40 4.2.2 Average number of health extension workers trained in ENA and EHA ................................... 40 4.2.3 Average number of community meetings held to share what CG members learned............... 41 4.2.4 Average number of direct beneficiaries who received health promotion on health            and child nutrition………………………………………………………………………………………………………………..42 4.2.5 Percentage of beneficiary children 6‐23.9 m wgi received Vitamin A supplements................. 42 4.2.6 Percentage of beneficiary children 0‐5.9 m who are exclusively breast fed............................. 42 4.2.7 Percentage of beneficiary children 6‐23.9 m who have been provided 3 or more infant           and young child feeding practices............................................................................................. 43 4.2.8 Number of MOH Child Day Campaigns...................................................................................... 44 4.2.9 Percentage of CG members who regularly meet with a health promoter................................ 45 4.2.10 Percentage of CG members who know signs of childhood illness .......................................... 46 4.2.11 Pre‐natal care........................................................................................................................... 48 4.2.12 Use of insecticide‐treated bed nets......................................................................................... 48 4.2.13 Regular de‐worming of young children ................................................................................... 49 4.2.14 Diarrhea treatment.................................................................................................................. 49 4.2.15 Knowledge and attitudes with regard to HIV/AIDS ................................................................. 50 4.2.16 Number of constructed water points which pass water quality tests..................................... 51 4.2.17 Number of school/market latrines constructed a safe distance from water sources............. 52 4.2.18 Percentage of households with year‐round access to improved water sources..................... 52 4.2.19 Percentage of beneficiary households adopting improved hygiene behaviors...................... 54 4.2.20 Percentage of underweight beneficiary children 0‐23.9 m..................................................... 59 4.2.21 Percentage of underweight children 0 ‐59.9 m....................................................................... 60 4.2.22 Percentage of stunted children 6 – 59.9 m ............................................................................. 62 4.3 Cross‐cutting Indicators:  Improved Human Capabilities of Vulnerable Households....................... 63 4.3.1 Number of assisted communities with improved community capacity as a result of MYAP assistance............................................................................................................................................ 63 4.3.2 Percentage of Community Development Committees whose members are women............... 64 4.3.3 Number of Community Development Committees trained in Early Warning Systems ............ 64 6 4.3.4 Percentage of Community Development Committees that have a long‐range community development plan............................................................................................................................... 65 4.3.5 Percentage of Community Development Committees that have used early warning data to assess and analyze risk........................................................................................................................ 65 4.4 Mission‐Required Final Indicators:  Improved Human Capabilities of Vulnerable Households....... 65 4.4.1 Number of organizations receiving USG assistance .................................................................. 65 4.4.2 Number of individuals who have received USG‐supported long‐term agricultural sector productivity training ........................................................................................................................... 66 4.4.3 Number of people trained in child health and nutrition through USG‐supported health programs............................................................................................................................................. 66 4.4.4 Number of rural households benefiting directly from USG interventions................................ 67 5. Summary of Findings – By Strategic Objective ..................................................................................... 69 6. Qualitative Evaluation Findings ............................................................................................................ 74 7. Recommendations ................................................................................................................................. 85 8. Appendices 1.   Scope of Work 2.   Sampling  Frame 3a. Survey Modifications 3b. Health Survey Form 3c. Agriculture Survey Form 4.    Evaluation Training Workshop Schedule 5a.  Health Focus Group Guide 5b.  Agriculture Focus Group Guide 5c.  External Stake Holder Focus Group Guide 6.    List of Sites Visited 7.    IPTT – Year II ‐ Final 8.    List of Key Informants 7 1. Introduction An external evaluation of the 2008-2011 MYAP was conducted from August 18 through September 10 in order to understand the program nature, objective, focus and requirements so as to assess progress made against stated objectives, identify lessons learned and to make recommendations for program enhancement that could be of value in future programming. The Scope of Work is found as Appendix 1. The exercise was carried out by an international consultant with extensive experience in the programmatic areas of concern, in food aid management for development and in relevant humanitarian and development issues in the DRC. He was assisted by FH/DRC staff in Kalemie and Moba in the organization of data collection, data entry and first level tabulation, as well as in all logistical organization. The exercise consisted of both quantitative and qualitative elements. The quantitative component took the form of a comprehensive survey that was conducted along the same lines as that which established the program baseline for SO1 (agriculture) and SO2 (health and nutrition). It was conducted in both the Kalemie and Moba sites, inclusive of all target areas. Qualitative information was gathered by the consultant through meetings will all concerned program staff as well as focus group meetings with program beneficiaries, community leaders as well as key stakeholders in both Kalemie and Moba (concerned representatives of government line ministries, United Nations Organizations and International and Local NGOs). Prior to, during and after field work, the consultant reviewed key program documents such as: the program proposal, annual results /periodic monitoring reports, the baseline survey report, IPTT, yearly DIP, and internal field monitoring reports. The evaluation was conducted during a very limited time frame, considering the broad nature of the program, geographic dispersion, and logistical constraints in the region. To make the most cost-effective use of the time allocated for collecting the most relevant data, the original baseline survey was adapted for the most part, with some additions in the agriculture section to take into account new elements which came into effect after the BLS, and the presence of Farmer Field and Life Groups (FFLG), a core element of the project. The evaluation was conducted just as the main harvest of maize was concluding in Moba and had already concluded in Kalemie, whereas the baseline survey was conducted in January/February which represented the tail end of the “hungry season”. As a result, crops were, for the most part out of the ground at the time of the evaluation, making a physical crop assessment impossible, and the availability of food from the harvest affected the comparability of anthropomorphic data. 8 2. Objective and Methodology of the Evaluation The objective of the evaluation was to review progress made in terms of the impact indicators chosen to determine program success. To arrive at that point, output and results indicators were reviewed along with the opinions suggestions of participants/beneficiaries, program agents and key external stakeholders. Indicator achievement was measured by way of field surveys of randomly selected community members in the areas of health/nutrition/hygiene and agriculture and livestock. Survey questions paralleled those used for the MYAP baseline for the purposes of consistent measurement of progress. In addition, monthly and quarterly reports, as well as other documents on file with FH in Kalemie and Moba were consulted to obtain information on certain activities and achievements. Interviews were conducted with concerned FH staff in Kalemie, Moba and Bukavu, as well as with community and external stakeholders through focus group discussions in villages, Kalemie and Moba. Those discussions helped determine outside perceptions of the program, understand the operating environment of the MYAP, gauge levels of knowledge, attitudes and practices of program beneficiaries, and understand better beneficiary and program limitations and gaps. The discussions with beneficiaries and stakeholders helped gain insights as to lessons to be learned and identify, if they were present. Likewise presence in villages and conversations with beneficiaries could unveil unanticipated results and impacts. Field survey Separate surveys were conducted for farmers and for mothers of young children under the age of two years. A two-staged cluster sampling methodology was used to for both health and agriculture surveys with 30 cluster samples and ten households per cluster for a total of 300 households sampled for each cluster. Probability proportionate to size (PPS) was used to select the clusters (the detailed sampling frame can be found in Appendix 2). Starting households for clusters were selected using a spinning bottle method. Respondents for the health and nutrition survey were primary caregivers of children 0-23 months of age, and where there was more than one child under the age of 24 months in the household, the youngest child was used as the index child. In households where more than one child would qualify for the parallel sample, a coin was tossed to randomly select which child to measure for the survey. Respondents for the agricultural survey were adult household members, residents in their community for a period of at least one year, with intimate knowledge of household agricultural production and marketing practices. The requirement for one year of residence in the community for the agricultural survey was necessary in order to ensure that crop production and marketing data gathered were relevant to the project area. 9 For the health and nutrition survey a two-staged cluster sample methodology similar to the population-level agricultural evaluation was carried out to select and interview lots of at least 19 mothers of children 0-23m of age (primary caregivers) in each of the supervision areas, with a total sample size of 300 children. For anthropometric indicators, the health and nutrition evaluation survey included a sampling of heights and weights of 300 children 0-23m of age and a parallel sample of 300 children 24-59m of age. The sample size for the survey allowed a detection of at least a seven point change from baseline to final on the anthropometric indicators with a 95% CI and 80% power. The format and content of the questionnaires developed for the baseline survey were used as the basis for the evaluation survey. The survey instruments were not translated into Swahili. However, enumerator training included sessions in which Swahili translation of key questions and phrases was reviewed, discussed and standardized. Use of the two survey questionnaires allowed FH to gather information related to a wide scope of food security related indicators and topics including household characteristics, agricultural production, agricultural marketing, maternal and child health, water and sanitation, and perceptions related to broader social and physical health related issues. Each survey took approximately one hour to complete. Copies of the two survey instruments are included in Appendices 3b and 3c. Several modifications were made to the agriculture survey to improve clarity and to include new information relevant to the evaluation of project indicators. The health questionnaire remained the same as that of the baseline survey, with the inclusion of one additional question set. The precise nature of the survey modifications is included in Appendix 3a. As in the case of the baseline survey, FH agricultural, health and nutrition team members were assigned the task of administering the surveys. For the purpose of objectivity, staff from Kalemie were assigned to Moba to administer the survey, while their colleagues in Moba were assigned to Kalemie. Because most had prior experience with the format which had not changed significantly, and had already received five days of training before conducting the baseline survey, it was determined that only a three-day refresher training workshop would be required (the workshop schedule is included in Appendix 4). Once again, the training curriculum from the baseline workshop was used. The external consultant acted as facilitator of the event. Senior programming personnel, Dr. Jean Pierre Okitakoy , Chief of the Health and Nutrition Section; Mr. Kingsly Mforteh, Chief of the Agriculture and Livestock Section; and Ms. Esther Wong, Program Officer took the lead in taking trainees through the various elements of the survey form and methods for administering it. Agriculture, health and nutrition supervisors and enumerators were trained together in general topics related to the administration of the survey, but received separate training related to the testing and application of their respective survey instruments. For example, one day of training for health and nutrition staff focused on reviewing and refining anthropometric techniques to ensure accuracy of height and weight measurements, while supervisors and enumerators 10 involved with the agricultural survey focused on improving the quality of crop yield estimates and estimates of areas of production. In addition, data entry supervisors received training on controlling the quality of data entry prior to initiating data entry efforts. Twelve survey teams conducted the survey, six for agriculture and six for health. Each team consisted of a supervisor and two enumerators. In addition, each health team included two trained assistants to help with collecting anthropometric data. Data collection required five days. Data entry and analysis was conducted using EPI-Info 3.5.1. Beneficiary and Stake-holder Focus Groups (FG) The time frame allocated for qualitative investigation was extremely tight, particularly in light of the programmatic and geographic scope of the MYAP. Within the six days allocated for field work in the villages, the consultant decided to conduct focus group discussions in randomly selected villages, simultaneous to the implementation of the evaluation survey. Though randomly selected on the basis of PPS, health and agriculture activities had to have been implemented in the communities eventually chosen in order to optimize the use of time and minimize logistical complications. Where the random sampling resulted in the selection of a village in which only one component functioned, the next village in the sampling frame list that contained both components was chosen as an alternate. Ultimately 6 villages (3 in Kalemie and 3 in Moba) were selected as locations for FGs. Instructions were given to field promoters to invite 10 randomly selected FFLG members and 10 participants in the health component (ideally 5 Mother Leaders and 5 Beneficiary Mothers). Special invitations were extended to the presidents of Community Development Committees (CDCs) to attend both focus groups in each village. Needless to say, village members tended to show interest in the discussions and more than the allotted 10 often showed up and were not turned away. Emphasis was placed on inviting female members of FFLGs. Though the discussions were guided by a pre-determined list of questions (Appendices 5a - c), the sessions were conducted as semi-structured conversations, which sometimes took courses somewhat outside of the scope and sequence of the established guide. External stakeholders were invited to focus group discussions in Kalemie and Moba. In attendance were representatives of government line ministries concerned with the MYAP, international organizations – particularly UN agencies, international and local NGOs which implement programs akin to those of the MYAP. 3. Brief Description of the Program Food for the Hungry/DRC (FH/DRC) implements a MYAP targeting 27,306 vulnerable households in the territories of Kalemie and Moba, in Tanganyika, Katanga. The official 11 implementation period is from August 1, 2008 to July 30, 2011. The scope of the activities includes agriculture recovery and development, health and nutrition, and water and sanitation. In addition, cross cutting indicators focus on developing community capacity and gender mainstreaming. FH’s intervention activities which are designed to improve the livelihoods of beneficiary households through agricultural and small livestock development are grounded in the establishment and orientation of Farmer Field and Life Groups (FFLGs). Through those associations farmers have been trained and provided capital assistance for: the multiplication of improved varieties of seeds and cassava cuttings; the multiplication of goats and ducks; and the production of tree seedlings for agro-forestry production. All beneficiary farmers have received training for employing improved agricultural and soil and water management technologies, post￾harvest storage management practices and value chain analysis. The project has made marketing information available to whole participating communities through the posting and maintenance of market price billboards outside of markets in all villages in the MYAP area of implementation. In addition, households deemed to be particularly vulnerable/food insecure were provided improved seeds and tools through MYAP organized seed fairs and were provided three monthly seed protection (food aid) rations (SPR) to ensure that they had enough food for consumption to allow them to save seed from their harvests for future planting.; The cornerstone of the project’s actions to achieve this strategic objective is the formation of Farmers Field and Life Groups (FFLG). Farmers are selected to participate in the project on the basis of their expressed level of commitment to the objectives of the project, willingness to work together in achieving those objectives and on the basis of their level of poverty against several vulnerability criteria. The MYAP implements interventions to improve the health and nutritional status of mothers and children with a focus on pregnant and lactating women and children 0-59m of age with an intensive emphasis on the most vulnerable children ages 0-23m. Behavior change at the household level related to Essential Hygiene Actions, Essential Nutrition Actions, and disease prevention and care is created and maintained through the successful Care Group model. The program also supports the MOH to carry out preventive health measures in the target areas. With regards to water/sanitation activities, FH is constructing and/or upgrading water points and school/market latrines, while ensuring WATSAN committees to help sustain and care for the new and existing structures. Within a livelihoods context, the above-mentioned elements are integrated. They support each other and contribute to their achievement of their respective objectives. In that light, however, the success in terms of improving agricultural production is of fundamental importance to providing households (especially mothers) the means with which to put into practice what is promoted and learned to address health and nutrition needs. 12 The MYAP’s two strategic objectives and intermediate results are: SO1. Improvement of livelihood capacity of vulnerable households IR 1.1: Increased agricultural production. IR 1.2: Improved natural resource base. IR 1.3: Improved market linkages. SO2. Improved human capabilities of households IR2.1 Improved use of Essential Nutrition Actions (ENA) by pregnant women and mothers of young children IR2.2 Improved mother's ability to prevent, diagnose and manage common childhood diseases that exacerbate malnutrition R2.3 Improved Access to Clean Water, Sanitation Facilities and of Essential Hygiene Behaviors (EHB) The original three-year LOA budget was set at $11,497,600. It consisted of 10,000 metric tons of wheat to be monetized to cover program investments as well as FH direct program operating expenses, plus 3,258 MT of assorted Title II commodities for direct distribution in the form of seed protection rations (SPR). In addition $1,892,394 was provided to cover Inland Transport, Shipping and Handling (ITSH) costs. The designated commodities arrived as approved, but the revenue generated from the monetization of wheat with MIDEMA through the port of Matadi during years I and II proved to be significantly below the projected value. Consequently the original budget was drastically cut to an anticipated $7,199,892 of which $5,152,842 corresponded to Years I and II. Virtually all budget cuts involved project investments and direct program operating expenses which were linked to monetization revenue. Key elements of the agricultural component, notably seed/cassava cuttings multiplication and water conservation management investments were pushed back into Year II. The former’s late start has had a direct and detrimental effect on the program’s ability to achieve certain revenue-based impact targets, and the program’s expansion of beneficiary coverage based on an organic growth model. It can be noted, however, that a subsequent significant 202e increase in Year II served to offset partially the program budget shortfall. 4. Detailed Analysis of Findings 4.1 Strategic Objective 1 : Improve Livelihood Capacity of Vulnerable Households The cornerstone of this SO is the formation of FFLGs through which to implement the training, technical assistance and seed/manioc cuttings multiplication and distribution. To this effect, the project has met its target of establishing 200 FFLGs (100 in the Kalemie District and 100 in Moba District). While the model calls for each FFLG to be composed of from 15-20 members, actual total participation is in excess of 4,300 farmers. The training and technical assistance 13 being provided to members of the FFLGs is highly valued by the communities at large, resulting in requests for information on how to form an FFLG, to participate in technical training events and for ad hoc technical assistance from project agriculture extension agents. Intermediate Result 1.1: Increase Agricultural Production Output Indicators:  Number of households receiving seeds and tools  Quantity of seed and tools distributed  Number of households receiving seed protection rations (SPR)  Number of associations engaged in multiplication activities (seeds/cassava cuttings)  Quantity of high quality seed and cassava cuttings distributed to associations for multiplication  Quantity of small livestock (goats and ducks) distributed to associations for multiplication 4.1.1 Number of households receiving seeds and tools Table 1: Number of households which received seeds and tools FY 1 Target FY 1 Achieved FY 2 Target FY 2 Achieved Cummulative Target Cummulative Achieved 2,400 2,393 3,000 2,998 5,400 5,391 Through the first two fiscal years of the project, virtually 100% of the target was achieved. These beneficiaries qualified for assistance, in the form of three-month SPR along with a hoe and a voucher valued at about $10 for the purchase of quality seed at FH-organized seed fairs, on the basis of need determined by vulnerability criteria. All other members of the community have continued to obtain seed through a variety of usual means at their disposal. The evaluation survey revealed that most farmers believe that they do not have sufficient seed to plant farm land available to them. Slightly less than 28% of all farmers acknowledge that they have had sufficient seed for their last planting season. A slightly higher percentage of FFLG members acknowledged having sufficient seed, possibly owing to their higher participation in project-sponsored seed fairs. There was hardly any variation in responses between Kalemie and Moba. Table 2: Farmers who acknowledge having had sufficient seed to plant All Farmers (N=298) FFLG (N=138) Non-FFLG (N=160) Kalemie (N=151) Moba (N=147) # % # % # % # % # % 83 27.9% 45 32.6% 38 23.8% 43 28.5% 40 27.2% 14 Overall, farmers’ source of seed is relatively balanced between saving from their last harvest, buying seed at the local market and from FH Seed Fairs (based on qualification according to vulnerability criteria). In the case of members of FFLGs, a lower percentage saved seed from their prior harvest, while a higher percentage received seed at FH Seed Fairs. Very few farmers have received seed and tools assistance from either government or other NGO sources. This reflects the wind down of humanitarian programs in the region (except for newly repatriated refugees) and the lack of transitional and development programs being implemented. It also underscores the important role that the MYAP plays in filling a widespread fundamental gap. The low percentage of FFLG members who acknowledged in the survey that they saved seed likely relates to the start of last planting season, at which time those farmers had just received both seed and a SPR from the project. FFLG members stated in evaluation focus groups that they have set aside seed from their next planting from their recent harvest. However the poor harvest has limited the amount they could save, while still allocating produce for household consumption as well as sale for other necessities. Without baseline data in this regard, progress cannot be assessed. Therefore, the source of seed used in the next planting cycle should be carefully monitored in order to determine the source of seed and to determine if more farmers are developing capacity to save seed for future plantings. Table 3: Source of seed for planting Source All Farmers (N=298) FFLG (N=138) Non-FFLG (N=160) Frequency % Frequency % Frequency % Saved from last harvest 106 35.6% 40 29% 66 41.5% Bought from the market 113 37.9% 42 30.4% 71 44.7% Received from friends/family 58 19.5% 19 13.8% 39 24.5% FH Seed Fair 100 33.6% 74 53.6% 26 16.4% Donation from the Govt. or another NGO 7 2.3% 6 4.3% 1 0.6% 4.1.2 Quantity of seed and tools distributed As confirmed above, the targets for the number of farmers who received seed vouchers and a hoe from the project. Focus group respondents stated that the seed fairs organized by FH were well organized and that the quality of seed/cuttings was good. They stated that the price of seed was slightly higher than that which they could find in area markets where seed/cuttings are sold. Upon further discussion, however, respondents did recognize that the prices were probably fair owing to the fact that merchants had to travel to and set up in outlying villages, saving farmers the time and expense (sometimes) of going to central markets. They also acknowledged that the quality of the seed had been screened and that it probably was higher than some of the seed available in local markets for lower prices they had cited. For the most part, farmers felt that the value of the voucher did not permit them to purchase as much seed as they would have liked. In that regard, they preferred high value (higher priced) 15 peanut and bean seed to be used for cash cropping. Farmers were, therefore, obliged to consider the trade-offs between purchasing less of higher-value seed vs. more of less costly seed (maize) for staple crops. Despite having to make trade-offs, focus group participants who received seeds/tools recognized the value of the assistance and the role it played in being able to cultivate/produce enough to feed their families and meet some of their other needs. When asked how they would obtain seed were the assistance to come to an end, respondents acknowledged that they hoped to be able to produce enough to be able to set aside sufficient seed stock, or they would purchase seed from area suppliers. To finance the purchase, many respondents stated that they would have to sell an animal (if they had them), would seek day labor to obtain cash, or would go and fish to sell/barter their catch for seed from merchants. The conclusion, here, is that farmers do have options and will exercise them as circumstances require. 4.1.3 Number of households which received seed protection rations (SPR) In that SPRs were provided to beneficiaries along with seeds and tools, the project has fully met its target for households served in this regard during the first 2 years of implementation. During Year I only beneficiaries in Kalemie received three complete one-month rations. In Moba, FH experienced delays in accessing in country commodities for distribution to beneficiaries (a force majeur caused by errors committed by GoDRC customs and phyto-sanitary inspectors who temporarily refused to release cargo on the grounds that it was unfit for consumption). By the time the error was corrected and the commodities released, it was too late to make the second and third distributions. Had the commodities been distributed, the timing could have had adverse effect on market prices, as distribution would have occurred at harvest time. In Year II, commodities were available and three full one-month distributions were carried out according to the anticipated schedule. Beneficiaries reported in focus groups that the size of the ration was not enough to completely feed their families for the entire month in which it was received. As a result, many acknowledged that they were unable to set aside enough seed to plant all of their available land. Farmers also noted that their lack of effective seed grain storage limited how much they could set aside without risk of loss to pests and animals. Other than the FH program, there is very little other assistance of this nature from other humanitarian and/or development organizations in the area. The survey revealed that only 2.8% of respondents received some form of SPR from another organization during the previous 12 months (3.6% of FFLG members and 1.9% of non-members). Of what little assistance was provided, Kalemie farmers benefited more than their cohorts in Moba (4%, compared to 1.4%). 16 4.1.4 Number of associations engaged in multiplication activities (seeds/cassava cuttings) Because of monetization shortfalls, multiplication activities were not conducted in Year I. However, with the availability of funding starting in Year II, the project has fully met its Year II target of establishing, training and setting up 34 associations, from the 200 FFLGs, to implement seed and cassava cutting reproduction (14 dedicated to seed and 20 dedicated to cassava cutting reproduction). In partnership with SENASEM, which is based in Lubumbashi, a systematic schedule of training and on-site technical assistance has been provided to the associations’ members. 4.1.5 Quantity of high quality seed and cassava cuttings distributed to associations for multiplication The project has successfully and effectively procured the required seed and cassava cuttings for the designated number of reproduction associations on the basis of 1 hectare of land cultivated per association. Participants in focus groups acknowledged that from their observation, the quality of seed/cuttings procured and plants is superior to traditional varieties that they are presently planting in their fields. 1,400 kg of maize, peanut and bean seed have been procured, distributed and planted by participating associations. Similarly, 54,000 linear meters (LM) of cassava cuttings were procured, distributed and planted. 2,000 LM of additional cassava cuttings above the target represents a margin for loss and damage during transport and handling. 4.1.6 Quantity of small livestock (goats and ducks) distributed to associations for multiplication 48 FFLGs have been organized to implement small livestock reproduction – 20 raise goats and 28 raise ducks. There are 24 groups in Moba and Kalemie, respectively. Table 4: Small Livestock Multiplication Groups and Livestock Distributed Small Livestock Distributed Beneficiary Associations Total Distributed Total Year I Year II Year I Year II Associations Goats 204 120 12 8 324 20 Ducks 120 370 8 20 320 28 The project has fully met its targets for farmers group (FFLG) participation. Each goat multiplier group has received 15 animals (one male and 14 females). Duck multiplier groups have received 20 birds in Moba (2 male and 18 females), but only 17 (1 and 16) in Kalemie in Year II due to a higher-than-expected price in that region. Effect (Results) Indicators:  Percentage of trained beneficiaries who score 70% or above on post-training test scores  Percentage of FFLGs which conduct a participatory learning exercise at least once a month in the last twelve months 17  Percentage of beneficiaries who use a minimum of three sustainable agricultural technologies  Percentage of beneficiary households which planted at least 70% of the staple seed received through direct distribution or seed fairs.  Average yields of selected crops (maize, cassava, beans and groundnuts) per beneficiary household  Total area planted by associations for high quality seed and cutting multiplication (hectares)  Quantity of high quality cutting produced by multiplication associations (linear meters)  Quantity of high quality seed produced by multiplication associations (kgs)  Percentage of beneficiary associations that use a minimum of four recommended small livestock technologies  Percentage of households in the area which own any small livestock (goats or ducks)  Average number of small livestock produced in the last year per beneficiary association household. 4.1.7 Percentage of trained beneficiaries who score 70% or above on post-training test scores In Year I, post training tests were administered to participating farmers. The results were discouraging despite the fact that during training farmers seemed to demonstrate an understanding of the material covered. The program team believes that the low level of literacy of farmers adversely affects written test scores, giving an impression that marginally literate farmers have learned less than they may have. In fact written tests are absolutely impossible for the completely illiterate. It makes little sense to invest time and effort to administer tests that provide less-than-reliable findings. The external consultant concurs with the opinion of the program team based on his experience. Substituting oral tests to individual farmers in place of written tests has been considered, but has been deemed too time-consuming to be cost effective in light of the number of farmers involved. A third option could be to set up a protocol of randomly sampling training individual participants in a way that proved cost-effective in terms of the burden it places on field staff and the quality/reliability of the results generated. However, the project would be better served if the time spent on post-training testing were invested in supporting and monitoring actual practices employed by farmers in their fields. 4.1.8 Percentage of FFLGs which conduct a participatory learning exercise at least once a month in the last twelve months In both Years I and II, up to 1,600 monthly training exercises were conducted each year for members of the 200 FFLGs. Through the third quarter of Year II, 1,268 training events had been organized, but the pace during the fourth quarter seems to have been sufficient to meet the project’s yearly target and cumulative targets through Year II. 18 Because Year I seed/cuttings multiplication activities were postponed due to funding shortages, there was no activity in this regard in the first year. In Year II, 34 monthly participatory learning exercises were scheduled for FFLGs participating in multiplication activities. Whereas all 20 cassava multiplication associations have carried out monthly learning exercises, the 14 seed multiplication associations have been limited to learning exercises associated with the harvest. During those sessions, participating farmers had the opportunity to assess, by way of direct inquiry and observation, the quality and advantages of the seed/cuttings that were planted . In all cases, participants reported satisfaction with the improved nature of the seed/cuttings which they would eventually be harvesting. This same sentiment was voiced by participating farmers during the course of focus group discussions. This level represents a 59% of full achievement of the indicator as set expressed, though all 34 associations have carried out the planned activity to one extent or another. To the extent that monthly training activities are participatory and based on farmer inquiry related to their on-going farming/multiplication activities, those sessions could reasonably be considered valid as accomplishments against this indicator. However, it was beyond the scope of this evaluation to actually evaluate the nature and methods of the training carried out. At the personal level, approximately 75% of FFLG members have received training in several key agricultural and small livestock technologies associated with improving productivity. Among the topics which were imparted to farmers were the following: Table 5: FFLG participants in key training activities (information through 6/2010) Topic of Training Annual Target Achieved Improved Soil and Water Management and Practices 3,000 1,540 Integrated Pest Management Practices 3,000 2,900 Improved Post-Harvest Storage Management Practices 3,000 2,492 Dry Season High-Value Crop Production 3,000 2,981 Agroforestry Practices 3,000 1,910 Participants in focus groups cited the importance that training has played in improving their productive capacities. Training was generally one of the first project services sited - and with conviction. Virtually all respondents acknowledged the high value they place on training and its usefulness to them. On average, focus group respondents were able to cite 5 lessons from the training they had received. Survey responses showed that greater percentages of members of FFLGs were able to cite improved agricultural, livestock management and soil and water management technologies which they remembered from training. This is not surprising considering that only some non￾FFLG community members take advantage of opportunities to sit in on training and at the same time demonstrates the learning opportunities that the training events represent. However, the very fact that sessions are open to anyone who has an interest in the subject, and that non-FFLG 19 members do participate, is a positive element and accomplishment of the program, contributing to additional multiplier effects. Table 6: Improved practices training recalled by farmers Technology All Farmers FFLG Non-FFLG Frequency % Frequency % Frequency % Inter-cropping 107 36% 90 65.2% 17 10.6% Crop Rotation 103 34.6% 89 64.5% 14 8.8% Post-harvest Storage 105 35.2% 97 70.3% 8 5.0% Pest control 27 9.1% 26 18.8% 1 0.6% Agro-forestry 86 28.9% 79 57.2% 7 4.4% Improved seed varieties 49 16.4% 45 32.6% 4 2.5% Natural resource management 35 11.7% 33 23.9% 2 1.3% Small livestock management 46 15.4% 43 31.2% 3 1.9% Seed multiplication 39 13.1% 34 24.6% 5 3.1% Value-added chain 41 13.8% 39 28.3% 2 1.3% Post-harvest storage, inter-cropping, crop and agro-forestry practices are those most frequently recalled by FFLG and non-FFLG farmers, alike. Among all survey respondents, the average number of improved practices that were known/recalled per farmer was 2.1 (N=297; CI: 1.49-2.81). FFLG members knew/recalled an average of 4.2 practices (N=138; CI: 3.00- 5.33) while Non-FFLG farmers averaged only 0.4 practices (N=158; CI: 0.29-0.51). Based on focus group conversations, post-harvest storage is high on the minds of farms. Farmers site it as a significant limiting factor for storing enough food for household consumption, storing enough seed for planting, and taking advantage of/controlling seasonal market fluctuations when they sell crops. Integrated pest management (pest control) came in well behind as the least recalled practice during focus group discussions. This may be a reflection of the relative emphasis placed on this subject early in the project’s history. 4.1.9 Percentage of beneficiaries who use a minimum of three sustainable agricultural technologies At the start of the project, only 26% of baseline survey respondents acknowledged use at least three of the 10 FH-recommended improved agricultural technologies on their farms. The Year II target was for 80% of farmers would reach that threshold and that by the end of the project 90% would do so. At the time of this evaluation (virtually the end of Year II), levels have significantly improved. Among all farmers, 59% acknowledge practicing at least three technologies. The percentage of all farmers who admit to not practicing any improved technologies dropped from 22% to 12%. 20 The median at the time of the baseline was 1 practice. By the end of Year II, it has risen to 3 for all farmers (5 among FFLG members). Most telling, however, is the fact that among FFLG members 80% practice three or more technologies and only 3% practice none. Whereas no farmers in the baseline acknowledged practicing more than five technologies, 38% of FFLG members report doing so at the 2-year mark of the project. Though non-FFLG farmers lagged in terms of this indicator, their levels have also demonstrated improvement in comparison to the baseline, probably owing to the demonstration effect of FFLG activities as well as participation of non-FFLG members in training events. Table 7: Number of improved agricultural technologies practiced by farmers in their fields # Tech Practiced Baseline All Farmers (N=298) FFLG (N=138) Non-FFLG (N=160) % # % Cumm # % Cumm # % Cumm 10 0% 1 0% 0% 1 1% 1% 0 0% 0% 9 0% 2 1% 1% 2 1% 2% 0 0% 0% 8 0% 4 1% 2% 3 2% 4% 1 1% 1% 7 0% 24 8% 10% 18 13% 17% 6 4% 4% 6 0% 30 10% 20% 28 20% 38% 2 1% 6% 5 1% 35 12% 32% 22 16% 54% 13 8% 14% 4 6% 38 13% 45% 23 17% 70% 15 9% 23% 3 18% 41 14% 59% 14 10% 80% 27 17% 40% 2 20% 49 16% 75% 14 10% 91% 35 22% 62% 1 33% 37 12% 88% 9 7% 97% 28 18% 79% 0 22% 37 12% 100% 4 3% 100% 33 21% 100% Table 8: Types of improve technologies practiced by farmers in their fields Baseline All Farmers (N=298) FFLG (N=138) Non-FFLG (N=160) % Frequency % Frequency % Frequency % Improved seed/cutting 8.4% 38 13% 30 22% 8 5% Use of green manure 0.7% 42 14% 21 15% 21 13% Use of compost 1% 19 6% 14 10% 5 3% Crop rotation 13.7% 139 47% 83 60% 56 35% Inter-cropping 35.7% 182 61% 97 70% 85 53% Incorporation of organic material 21.1% 175 58% 95 69% 80 50% Mulching 6% 77 26% 47 34% 30 19% Grain storage technologies 26.9% 121 41% 88 64% 33 21% Planting densities 37.4% 127 43% 99 72% 28 18% Pest management 10.4% 73 24% 55 40% 18 11% Survey responses seem to indicate that there has been an appreciable increase in the use of improved technologies across the board, with the greatest positive change among FFLG members. Positive news aside, such high degrees of improvement need to be substantiated by 21 objectively verifiable field observations. Unfortunately, the time allocated for field work during the exercise and the timing of the evaluation (for the most part, crops had already been harvested from the fields) made such a verification of observable/measurable signs difficult. 4.1.10 Percentage of beneficiary households which planted at least 70% of the staple seed received through direct distribution or seed fairs Based on post-planting questionnaires, 68% of farmers who received seeds at seed fairs or through direct distribution actually planted the seed during Year I. In Year II that percentage improved slightly to 74%, representing a 92% achievement of the program’s proposed target. The fact that a farmer would/could not plant free seed, while also counting on a three-month seed protection food ration indicates one of several possible scenarios: a) There has been a miscommunication in the way the post-planting questionnaire posed the question. During focus group discussions farmers frequently cited that there was insufficient availability of seed at seed fairs. Upon further probing, it came to light that what they were really saying was that the value of the seed voucher was insufficient for them to buy the desired quantity of the types of seed that they had wished. During focus groups, participants in the seed fairs confirmed that they had planted what they had “purchased”. This inconsistency with the survey should be investigated further and more closely by the program team in Year III within the scope of monitoring of seed fairs and SPR distributions. b) Farmers had anticipated that the customary “hungry season” would be more severe in future months and/or had determined that their customary coping mechanisms would not be sufficient to cover their consumption needs during the lean period While forward-looking logic might dictate that a failure to plant would guarantee that households experience worse hunger in the future, survival logic of farmers struggling to just subsist would call for them to address immediate needs as they happen, leaving the future to be addressed at a later time. Eating seed would be viewed as one of a farmer’s last resort. One would imagine, however, that day labor, fishing and hunting/gathering options would have to have been fully depleted to arrive at that point. c) Farmers assumed that humanitarian type responses (free seed, food ration packages, etc.) would continue in the future, therefore making a decision to eat their seed or sell it for short-term financial liquidity, making such action a viable livelihoods option. d) Farmers already had sufficient seed from other sources, therefore feeling confident enough to sell seed received at the seed fair In any case, the project had met its indicator target during the course of Year II. Nevertheless, for the sake of long-term sustainability of the project’s results and impact, this matter should be 22 investigate further in an appropriately rigorous manner to determine if SPRs are actually serving effectively their anticipated purpose or whether other modalities would be more appropriate (i.e address household food gaps through other forms of food aid – FFW/CFW or supplemental feeding). 4.1.11 Average yields of selected crops (maize, cassava, beans and groundnuts) per beneficiary household As in the baseline survey, obtaining reliable information on crop production and areas planted on which to base crop yield calculation is problematic in a survey of this nature. During the course of the evaluation survey, data resulted in improbably yields, particularly on the high end of the spectrum. It is difficult, if not impossible, to precisely determine the cause/source of the unreliable data – whether it results in communications confusion over the use of a unit of measure for the area of land cultivated or weight of a crop harvested; or a result of unconscious or conscious under/over reporting. FH did not carry out a systematic and controlled sampling of crop cuttings just prior to harvest as a means of obtaining reliable data for calculating yields, as recommended in the baseline survey report in the wake of similar data probability issues. Therefore, farmer responses, to the best of their recollection, were the only information available at the time of the evaluation. During the course of conducting focus groups farmers unanimously signaled that due to the unfavorable rains during the recently-ended agricultural cycle, yields were significantly lower than prior years. Through indirect probing, the focus group facilitator could only deduce that the range of drop which they mentioned may have been from 30%-75% of certain crops. At first glance, using unadjusted data crop yields calculated from responses provided by farmers, yields would have appeared to have risen from baseline levels (considerably in the case of cassava and peanuts; and with the exception of beans), with FFLG farmers appearing to outperform non-FFLG farmer in all crops except rice. However, those calculations were based on a number of improbably high yields. Similar abnormalities were noted, as well, in the baseline survey. In keeping with an adjusted yield calculation put forward as a possible way of viewing the baseline data, a similar upper limit thresholds were used as grounds for disqualification. While a number of low end yields were calculated, all the way down to zero production, they were not excluded because any number of unknown factors could have brought about extremely poor harvests in individual cases. However, in the case of cassava, 105 out of 160 valid responses (66%) for farmers reporting cassava cultivation were reported as zero production. This, in the estimation of the external evaluator seemed improbable given that the “total crop loss” reported was totally out of proportion with similar loss in other crops. Only in the case of cassava were low-end (zero) yields excluded. 23 Table 9: Mean select crop yields (adjusted for improbable yield data) – Kg./Ha. Crop Adjusted Baseline (Kg/Ha) Evaluation Survey Disqualification Range (Kg/Ha) All Farmers (Kg/Ha) FFLG (Kg/Ha) Non-FFLG (Kg/Ha) Maize 961 >3,330 (N=9) – 3.9% 638 (N=221) CI: 548-727 648 (N=99) CI: 515-781 630 (N=122) CI: 509-751 Rice 1,385 >3,499 (N=3) – 8.3% 929 (N=33) CI: 806-1,053 724 (N=19) CI: 440-1,008 1,208 (N=14) CI: 0-3,623 Cassava 4,754 =0 >240,000 (N=106) – 66.3% 2,211 (N=54) CI: 646-3,776 3,157 (N=25) CI: 580-5,733 1,396 (N=29) CI: 0-3,180 Sweet Potato 3,125 No outliers 866 (N=37) CI: 711-1,021 1,004 (N=16) CI: 0-3,087 761 (N=21) CI: 504-1,019 Beans 425 No outliers 252 (N=97) CI: 129-374 257 (N=45) CI: 76-437 247 (N=52) CI: 81-413 Peanuts 558 >1,499 (N=11) – 11.8% 216 (N=82) CI: 90-342 204 (N=40) CI: 28-381 227 (N=42) CI: 48-406 In all cases, calculated yields were below those registered in the baseline. Drops in yields ranged from 33% to 82% among all farmers. In the case of cassava and sweet potato, FFLG members appear to have out-performed non-members, while in rice the table was turned. Undoubtedly, poor rains/harvests during the last agricultural cycle (which were not only reported by farmers, but also confirmed by Ministry of Agriculture and NGO representatives) explain to some extent the drop in yields from baseline levels. Given the scope of the evaluation, a rigorous and precise explanation is not possible The data generated from the evaluation survey is inconclusive from a statistical point of view. At this time, however, it is not possible to consider that the project has achieved its goal of increasing yields. Nevertheless, in light of adverse climatic conditions and the lack of reliability of the data, one should not jump to the conclusion that the project has failed in this regard. More time and a more rigorous methodology for measuring yields over time, while trying to account for certain climatic variations will be required in this regard if statistical evidence is sought. 4.1.12 Total area planted by associations for high quality seed and cutting multiplication (hectares) 33.2 hectares out of a planned 34 hectares (1 ha./group) have been planted in seed/cuttings for multiplication – a 98% completion. In one case, land promised by the CDC was not made available and in two other cases, FFLG members failed to weed properly for seed to be planted. The former, which represents half of the shortfall, was an unforeseen and uncontrolled occurrence. All cases occurred in Kalemie. 24 Most farmers have opted to farm peanuts for seed, with 11 of 14 associations planting 10.2 hectares in that crop. Beans and maize are raised by the remaining associations (2 and 1 associations, respectively). The importance that farmers place on quality seed for cash crops is clearly evident. 4.1.13 Quantity of high quality cutting produced by multiplication associations (linear meters) The budget shortfall that resulted from reduced proceeds from monetization forced the project to start implementing seed/cutting multiplication activities only in Year II. At the time of the evaluation, cassava had still not reached maturity in order to be harvested. Harvest is scheduled for November/December of 2010. Nevertheless, reports from field promoters, accounts from farmers in focus groups and a random visit to a field in association with a focus group discussion provide reason to be optimistic that the harvest will be at anticipated yield levels, and surely levels that are superior to traditional varieties presently planted. This indicator could not be measured. 4.1.14 Quantity of high quality seed produced by multiplication associations (kg.) Seed multiplication activities only began during Year II of the project. During this period, 7,000 Kg. of seed stock was harvested against a target of 11,200, representing a 63% achievement rate. Table 10: Seed produced by re-producer associations Site Peanuts Beans Maize # Assoc. Kg Produced Yield Per Ha. # Assoc. Kg. Produced Yield Per Ha. # Assoc. Kg. Produced Yield Per Ha. Kalemie 4 2,530 632.5 2 956 478 1 560 560 Moba 7 2,954 476.5 0 0 0 0 0 0 Total 11 5,484 537.6 2 956 478 1 560 560 Farmers participating in focus groups confirmed that the poor rains had greatly affected the development of their crops. Those participating in seed reproduction associations reported the same poor results. The yields obtained from seed reproduction plots are close to baseline yields for peanuts and beans, while that of maize was significantly below. In light of the adverse growing conditions, peanut and bean yields may possibly be viewed as positive, under the circumstances. Only by observing yields over time, in which periods of “normal” and “abnormal” rain levels are documented, will actual yield performance be observed with accuracy. 4.1.15 Percentage of beneficiary associations that use a minimum of four recommended small livestock technologies Through the first two years of project implementation 91% of FFLGs which participate in small livestock multiplication have demonstrated that they practice at least four recommended technologies. This is just ahead of a LOA target of 90%. 25 4.1.16 Percentage of households in the area which own any small livestock (goats or ducks) In pursuit of its objective to increase the number of households which raise small livestock as a part of their productive livelihoods options, the project has chosen to pursue an “organic” growth strategy through the dissemination of livestock to individual households through reproduction association/farmers groups. This, by its very nature is a gradual process based on livestock reproductive cycles. All reproduction associations formed by the project only began breeding in Year II. At the time of the evaluation animals from the first breeding cycle were still too young to separate from their mothers, though the process was due to commence during the final two months of 2010. This is reflected in survey data, which does not show increases in the number of households raising small animals. Only in the case of sheep and pigs was there a very modest rise, though from a very low baseline. Overall, the percentage of households which raise goats appears to have dropped by nearly two percentage points from the baseline, though FFLG farmers remained on par with the baseline while non-FFLG farmers dropped by more than 3%. Drops in the number of households currently raising small livestock may be the result of the more vulnerable households having been forced to sell the last of the few head of livestock they owned in order to compensate for the recent poor harvest. This, however, is only speculation and is not supported by data. The same can be said for ducks. In addition, reproduction group members signaled during focus groups (and project monitoring reports corroborate the fact) that duck breeding has proved very disappointing. Ducks are either falling ill and dying or just not breeding for reasons that even experts in the Ministry of Agriculture and Livestock and the FAO have yet to determine. Similar breeding projects supported by those two organizations have experience similar or worse results. During focus group discussions, numerous participants signaled that they may want to concentrate on raising goats, which is proving less problematic in terms of breeding. Table 11: Households raising small livestock Livestock Baseline Survey All Farmers FFLG Non-FFLG % Raising Frequency % Frequency % Frequency % Goats 14.3% 37 12.4% 20 14.5% 17 10.6% Ducks 23.7% 54 18.1% 21 15.2% 33 20.6% Chickens 53.7% 102 34.2% 57 41.3% 45 28.1% Sheep 0.3% 2 0.7% 2 1.4% 0 0.0% Cows 0.0% 0 0.0% 0 0.0% 0 0.0% Pigs 1.0% 6 2.0% 5 3.6% 1 0.6% Other 6.7% 4 1.3% 2 1.4% 2 1.3% 26 With the exception of pigs, the survey found that those households which raised small livestock are presently raising fewer animals than at the time of the baseline. Table 12: Mean number of small livestock raised by households Livestock Baseline Survey All Farmers FFLG Non-FFLG Goats 7.4 4.2 (N=37) CI: 0-34 4.4 (N=20) CI: 1-7 4.06 (N= 17) CI: 1-7 Ducks 8.3 4.7 (N=54) CI: 3-7 6.1 (N=21) CI: 3-9 3.82 (N= 33) CI: 1-6 Chickens 15.0 4.4 (N=102) CI: 0-23 4.7 (N= 57) CI: 3-7 3.96 (N= 45) CI: 2-6 Sheep 15.0 1.5 (N=2) CI: 0-8 1.5 (N=2) CI: 0-8 - Cows - 0.0 - - Pigs 6.6 9.0 (N=6) CI: 6-12 8 (N= 5) CI: 3-13 14.00 (N= 1) CI: 0-110 From the survey data, alone, it is impossible to determine with certainty the cause of the decline. However, some of the data that follows for goats and ducks may shed light on the dynamics that have existed during the 12 months prior to the evaluation. A significant number of households which raised goats and ducks in 2009 reported in the survey that they no longer raise those animals. To one degree or another, more than 50% of households raise fewer animals than in 2009. There were major drops in the levels for ducks, which would seem to be linked to breeding problems cited by focus groups participants. Selling and consuming others were expected actions that households carried out within the scope of their livelihoods strategies. The average numbers, if they were to be combined represent a significant proportion of the average number of goats and ducks that households reported having a year ago. That fact that within the data there are farmers who experienced a drop in herd/flock size while reporting neither sales nor consumption, demonstrates that farmers also “lost” animals to either death or theft. Therefore it is likely that a combination of sales, consumption, deaths and loss due to theft probably negated most or all natural propagation when it did occur. 27 Table 13: Household goat herd evolution – 2009 to 2010 FFLG Non-FFLG Households reporting raising goats in September 2009 18.8% 9.4% Average number of goats raised per household 4.8 5.1 Of those who had goats in 2009, number which reported “0” in 2010 30.7% 15% Percentage of households with an increased herd 31% 40% Percentage of households with a decreased herd 54% 50% Percentage of households whose herds stayed the same 15% 10% Percentage of households which sold goats in the last 12 months 26.9% 35% Average number of goats sold per household 2.1 5.1 (2.2) Percentage of households which consumed goats in the last 12 months: 18.5% 10% Average number of goats consumed per household 1.1 1% Table 14: Household duck flock evolution – 2009 to 2010 FFLG Non-FFLG Households reporting raising ducks in September 2009 19.6% 24.4% Average number of ducks raised per household 10.9 8.2 Of those who had ducks in 2009, number which reported “0” in 2010 33.3% 22.2% Percentage of households with an increased flock 14.8% 16.2% Percentage of households with a decreased flock 85.2% 78.4% Percentage of households whose flocks stayed the same 0% 5.4% Percentage of households which sold ducks in the last 12 months 25.9% 33.3% Average number of ducks sold per household 3 3.8 Percentage of households which consumed ducks in the last 12 months: 29.6% 48.7% Average number of ducks consumed per household 3.1 3.5% Based on the data and on the organic growth methodology implemented within the scope of the project, it is too early to expect the project to achieve its target of a 5% increase in the number of households which raise small livestock by the end of Year II. Both the “organic” growth strategy employed and livelihood factors beyond the project’s immediate and short-term control (i.e. using the sale of animals as a livelihoods coping mechanism to compensate for poor harvest or to meet other household financial needs) makes the target unrealistic as it presently stands. The target of increasing the total number of households raising small livestock should dropped in favor of concentrating on output indicators associated with the growth in the number of reproduction associations and the number of association member households receiving livestock distributions in the coming few years. Also, the project should invest significantly in training 28 and technical assistance in efficient livestock management so that farmers can maintain, and even grow their herds/flocks. 4.1.17 Average number of small livestock produced in the last year per beneficiary association households Because livestock reproduction did not begin until Year II, no livestock has been spun off from reproduction associations to member farmers for their future personal management or to other reproductions associations for the purposes of replication and growth. Monitoring reports and information provided by participants during focus groups discussions show that goat reproduction is progressing satisfactorily. Birthing is going smoothly, deaths are well within natural levels and thefts have not been a problem for the reproduction associations. In the case of ducks, breeding has so far been disappointing. With mortality exceeding the rate of hatchings 5 of 14 associations, and being only slightly above hatchings in the rest (overall: 91 ducks hatched and 100 deaths) the number of ducks in the flocks of the first 14 reproducers associations has hardly grown in most associations (definitely not enough to commence re￾distributions). In the latter regard, the project staff should, in concert with Ministry of Agriculture and Livestock and FAO experts, assess the cost-benefit of duck raising based on probable causes of poor breeding results and the prospects of reasonable feasibility of obtain productive results at both the reproducers association and private farm levels. If not found to be reasonably feasible, it would be in the best interest of the beneficiaries to transfer resources to increasing investments in goat raising to achieve more rapid advances in what seems to be a more viable option. Intermediate Result 1.2: Improved Natural Resource Base Output Indicators:  Number of project assisted communities with improved physical infrastructure to mitigate the impact of shocks  Number of soil and water conservation management committees created Activities associated with this intermediate result were suspended during Year I in response to the significant shortfall in monetization revenue. Activity and investments in Year II have been concentrated in awareness-building and training, rather than investment in infrastructure. Effect (Results) Indicators:  Number of soil and water conservation structures created  Number of sustainable soil and water conservation management technologies transferred  Percentage of beneficiary farmers who use the minimum number (3) of sustainable technologies 29  The number of soil and water conservation structures that are still intact or suffered minor damage, but have been successfully repaired one year after construction  Area under soil and water management 4.1.18 Number of soil and water conservation structures created At the time of the evaluation, no infrastructure had been constructed by MYAP beneficiaries. However, FH has indicated that actions are programmed to begin early in Year III. 4.1.19 Number of sustainable soil and water conservation management technologies transferred As noted in Table 7, beneficiary recall of soil and water conservation technologies that were covered in training events was lower than many agricultural technologies. Not surprisingly, far more FFLG members were able to recall one or more technologies compared to non-FFLG farmers (23.9% vs. 1.3%), given that FFLG members were the primary beneficiaries of training. This disappointing level of recall is likely linked to the fact that natural resource management training and investments in tree planting and investments in infrastructure were cut back in Year I due to the monetization shortfall. Once activities were reinstated in Year II, only 64% of the targeted 3,000 beneficiaries were trained in agro-forestry practices. While Moba met hit 98% of its target, Kalemie came in just shy of 29% due to the departure its Site Agriculture Coordinator and in delays in the development of a training module. 4.1.20 Percentage of beneficiary farmers who use the minimum number (3) of sustainable technologies Responses to the agriculture evaluation survey indicate that considerable progress has been made in promoting improved natural resource management practices among farmers. The percentage of farmers practicing three or more technologies has increased from 17.1% to 33.6%. More FFLG members have met that threshold than Non-FFLG farmers (43.5% vs. 25%), representing a marked improvement from the baseline and an indication that promotion and training activities are achieving their anticipated effect. Noteworthy is the fact that among FFLG farmers, a small percentage actually broke new ground by practicing seven technologies compared to none at the time of the baseline. The average number of practices employed by respondents were: All Farmers: 1.77 (CI: 1.16-2.38) FFLG Members: 2.27 (CI: 1.28-3.26) Non-FFLG Farmers: 1.35 (CI: 0.60-2.10) 30 Table 15: Number of improved natural resource management technologies practiced by farmers in their fields # Tech Baseline All Farmers (N=298) FFLG (N=138) Non-FFLG (N=160) Practiced # % Cumm # % Cumm # % Cumm 11 0.0% 0 0.0% 0.0% 0 0.0% 0.0% 0 0.0% 0.0% 10 0.0% 0 0.0% 0.0% 0 0.0% 0.0% 0 0.0% 0.0% 9 0.0% 0 0.0% 0.0% 0 0.0% 0.0% 0 0.0% 0.0% 8 0.0% 0 0.0% 0.0% 0 0.0% 0.0% 0 0.0% 0.0% 7 0.0% 3 1.0% 1.0% 3 2.2% 2.2% 0 0.0% 0.0% 6 1.0% 2 0.7% 1.7% 2 1.4% 3.6% 0 0.0% 0.0% 5 2.7% 7 2.3% 4.0% 5 3.6% 7.2% 2 1.3% 1.3% 4 5.7% 16 5.4% 9.4% 12 8.7% 15.9% 4 2.5% 3.8% 3 7.7% 72 24.2% 33.6% 38 27.5% 43.5% 34 21.3% 25.0% 2 8.0% 65 21.8% 55.4% 37 26.8% 70.3% 28 17.5% 42.5% 1 29.3% 49 16.4% 71.8% 19 13.8% 84.1% 30 18.8% 61.3% 0 45.7% 84 28.2% 100.0% 22 15.9% 100.0% 62 38.8% 100.0% The aforementioned achievements still fell considerably short of the Year II target of 80%. Nevertheless, given the relatively short span of implementation, the improvement probably represents a more realistic target than that established at the outset of the project. In the final year of the project, continued training along with more intensive extension work in farmers’ fields could result in a similar rate of improvement in Year III so as to reach the 80% threshold, especially among FFLG members.. Responses during the evaluation survey reveal that more farmers are practicing an array of soil and water management technologies across the board. FFLG members, on all counts, are doing so in greater numbers than their non-FFLG neighbors. As in the case of agricultural technologies, improvements are noted among non-FFLG farmers as a result of training spill-over or multiplier effects from examples set by FFLG members. 31 Table 16: Types of improve natural resource management technologies practiced by farmers in their fields Technology Baseline All Farmers (N=298) FFLG (N=138) Non-FFLG (N=160) % Frequency % Frequency % Frequency % Terracing 10.3% 26 8.7% 20 14.5% 6 3.8% Plant Ground Cover 0.7% 31 10.4% 17 12.3% 14 8.8% Check Dams 13.7% 41 13.8% 24 17.4% 17 10.6% Micro-Basins 1% 28 9.4% 14 10.1% 14 8.8% Bunds 6% 42 14.1% 27 19.6% 15 9.4% Contour Lines or Ridging 31.4% 149 50.0% 83 60.1% 66 41.3% Live Barriers 8.3% 34 11.4% 25 18.1% 9 5.6% Relay Planting (Leguminous Trees) 0.3% 9 3.0% 8 5.8% 1 0.6% Trees in Fallow Fields 1.7% 14 4.7% 11 8.0% 3 1.9% Water Capture Irrigation 20.7% 71 23.8% 44 31.9% 27 16.9% Improved Drainage 16.3% 75 25.2% 39 28.3% 36 22.5% The project is successfully implanting knowledge among not only their core FFLG members, but also the farming community at large. More importantly, that knowledge appears to be translating into gradual change in farming attitudes and practices, which, if sustained, should translate into positive impact over the medium and long term. 4.1.21 The number of soil and water conservation structures that are still intact or suffered minor damage, but have been successfully repaired one year after construction Soil and water conservation measures that farmers may be putting in place on their own land has not been monitored so as to be able to quantify or qualify in any detail they types of investments being made and maintained by beneficiaries. 4.1.22 Area under soil and water management According to evaluation survey responses, 68% of farmers (203 of 298) acknowledge placing some land under soil and water management. Respondents report that a total of 55 ha. have some form of soil and water conservation measures on them for an average of 0.27 hectare per farmer. This would mean that the Year II target of 14 ha. has been exceeded by a significant margin. 81.2% of FFLG members acknowledge having land under conservation for an average of 0.298 hectares per farmer. 56.9% of non-FFLG farmers have invested in soil and water conservation measures on their land at an average of 0.238 ha. per farmer. FH agricultural promoters have physically verified 17 ha. under soil and water management by members of FFLGs. This also qualifies as surpassing the MYAP target for Year II. 32 Intermediate Result 1.3: Improved Market Linkages Output Indicator:  Number of bulletins/boards exhibited for market information dissemination The project has put in place and maintains up-to-date market price information boards in all villages where markets are organized within the project zone. According to information provided by farmers who participated in focus groups, the boards are widely consulted by persons patronizing the markets to either sell or purchase produce. Effect (Results) Indicators: 4.1.23 Number of producer associations that have developed statutes and bylaws The projects have provided assistance to 65 farmers associations to draft and put into practice statutes and bylaws. This exceeds the targeted number of 50. 4.1.24 Number of producer associations using market information for decision-making The evaluation survey revealed that households rely overwhelmingly on information supplied by other farmers on market prices for produce – one of the two main sources cited in the baseline survey. In fact, that source seems to have taken on even more importance for households in the project zone. Whereas the relative importance of “other buyers in the local market” seems to have declined since the baseline, “market bulletin boards” have begun to emerge as an important source. To this effect, the project has had a marked positive effect on farmers’ access to accurate and up-to-date information on market prices by setting up and maintaining a system which did not exist before. There is no significant difference between FFLG members and non-members. According to information provided by farmers who participated in focus groups, the boards are widely consulted by persons patronizing the markets to either sell or purchase produce. In fact, participants unanimously confirmed that they consult the boards before their buying or selling. This differs notably from the survey responses, but does not detract from the success of the project in establishing the service and promoting it. Nevertheless, project staff should continue to monitor closely the extent to, and the ways in which, the service is being used, as well as any feedback on ways to improve it. 33 Table 17: Sources of market information Technology Baseline All Farmers (N=298) FFLG (N=138) Non-FFLG (N=160) % Frequency % Frequency % Frequency % a. Other buyers in the local market 49% 57 29.5% 27 28.7% 30 30.3% b. Other farmers 41.7% 122 63.2% 63 67.0% 59 59.6% c. Other Farmers Association Members 0% 2 1.0% 1 1.1% 1 1.0% d. Radio 0% 0 0.0% 0 0.0% 0 0.0% e. From an external agent (Govt. or NGO) 0.3% 0 0.0% 0 0.0% 0 0.0% f. From market bulletin boards 0% 54 28.0% 26 27.7% 28 28.3% g. Local merchants/traders 8% 29 15.0% 14 14.9% 15 15.2% h. External merchants/traders 7.3% 14 7.3% 6 6.4% 8 8.1% i. Others 0 0.0% 0 0.0% 0 0.0% Impact Indicators:  Total income of small livestock producer associations and cooperatives from the sale of targeted small livestock (US$)  Total income of seed multiplication associations and cooperatives from the sale of targeted crops (US$)  Number of months of adequate food provisioning  Household dietary diversity score (HDD) 4.1.25 Total income of small livestock multiplication associations and cooperatives from the sale of targeted small livestock (US$) When the evaluation was carried out, animals (goats only) from the first cycle of birthing were still too young to be distributed or sold; therefore achievement could not be measured and is considered at this moment as not achieved. Project staff and beneficiary associations have devised a protocol on how the association herds and their offspring will be managed. This involves: 1) providing each member with at least one female and male animal with which to start (or augment) their own herds; 2) donating animals for the formation of new reproducers associations; and 3) sell animals as required and based on multiplication herd growth. Animals for #2 will only be “spun off” after all members have obtained their initial starter animals. 34 As a growth strategy, this represents an “organic” approach which, by its nature, will require time to achieve a degree of considerable coverage in the project zone – more time than is available during this three-year MYAP. Nevertheless, based on births and mortality figures for 6 FFLGs which received goats in Year I, and assuming that gender of newborns will be split 50/50, it can be projected that the associations will only be able to distribute as “dividends” to its members anywhere from 1pair (doe and buck) to 6 pairs of animals. At this rate of organic growth (barring either improvement or deterioration of birth/mortality rates) it will take at least four years for all association members to receive just one pair of animals. During that time, there will be no cash revenue for the association from the sale of animals. Neither should there be enough animals for donation to form new FFLG livestock reproducer associations through organic growth until the first phase of internal distribution is completed. Table 18: Year I goat reproduction FFLG Total Received Births Deaths Sales Actual Herd New Does New Bucks MWENDE 10 12 0 0 22 6 6 KISAKALA 15 5 3 0 17 1 1 KAPAMPA 17 6 2 0 21 2 2 LUNGALABA 16 10 2 0 24 4 4 MWELE 17 3 1 0 19 1 1 KAKERA 17 11 2 1 25 4 4 At present rates of duck hatchings and mortality, appreciable duck flock growth and distribution among members in any appreciable number are highly unlikely during the life of this MYAP. During the interim, there will likely be male goat offspring which will exceed the numbers required for reproduction which can be sold. However, the likely numbers of animals available for sale may result in sales proceeds that meet the Year II annual income target of US$7,000, but will unlikely be able to generate the Year III target of US$20,250. At a current approximate market price for a young goat of US$55, the $7,000 target will represent the sale of 127 goats. This will average out to about 6 goats per association per year. To more than double that revenue stream will require a large increase in association herds, which cannot happen during the life of this project – particularly if livestock offspring are to be distributed to members. The target for Year III is not feasible and should be kept at the year II target of $7,000. What revenue is generated will be distributed/invested by each association according to their own criteria. During focus group discussions, some suggested they might: buy new female stock to increase the production herd; buy new female stock to increase the number distribute to members; and distribute the cash to members as a form of dividend. Needless to say, decisions will be reached by each association. This is laudable and should be supported fully within the scope of community and organizational development. 35 In this light, to achieve a significant coverage in terms of the number of households raising small livestock as well as the number of livestock each raises (emphasis on the former), this project and any follow-up project should increase the level of investment in “capitalizing” more households so that they can accumulate a reasonable start-up stock of animals. Needless to say, training and technical assistance will have to keep pace with the scale of livestock capitalization. 4.1.26 Total income of seed multiplication associations and cooperatives from the sale of select crops (US$) At the time of the evaluation, cassava plants were still maturing and would not be harvested until November/December of 2010. Therefore, revenue had not been obtained at that time. As noted previously (Table 14), poor rains have reduced first-year yields for maize, peanut and bean seed crops. At current production levels, and considering the producer associations will have to reserve harvests for: first, the next planting; second, distribution among its members for planting in their own fields; and third, donation for the formation of at least one new producer association, the amount of seed that will be available for sale to the general public will be rather limited. Under these conditions, the program may not it will not reach the targeted levels of US$9,100 in Year II and the Year III target of US$17,400 established in the IPTT. Present yield levels will also slow the organic growth rate of coverage for placing improved seed into the hands of non-FFLG farmers in the MYAP zone. 4.1.27 Number of months of adequate food provisioning To determine achievements with regard to this indicator, both farmers participating in the agriculture program and mothers participating in the health and nutrition program were asked to indicate for how many months of the year do their households have adequate food provision. By the end of Year II there has not been hardly any progress in increasing the number of months in which households enjoy adequate food provision. According to the survey responses, households enjoy adequate food provision for an average of 8.14 months of the year (CI: 7.68- 8.60), compared with 8.07 at the time of the baseline survey. The average response for mothers participating in the health program was 8.3 months while farmers participating in the agriculture program came in at 7.97 months. 36 Table 19: Months of Adequate Household Food Provision – Agriculture Respondents # of Months Baseline All Farmers (N=271) FFLG (N=123) Non-FFLG (N=160) % Frequency % Frequency % Frequency % 12 8.2% 4 1.5% 0 0.0% 4 2.7% 11 6.2% 16 5.9% 10 8.1% 6 4.1% 10 19.1% 33 12.2% 18 14.6% 15 10.1% 9 18.9% 56 20.7% 21 17.1% 35 23.6% 8 14.7% 71 26.2% 29 23.6% 42 28.4% 7 12.6% 41 15.1% 17 13.8% 24 16.2% 6 4.7% 28 10.3% 15 12.2% 13 8.8% 5 4.9% 13 4.8% 8 6.5% 5 3.4% 4 2.5% 3 1.1% 1 0.8% 2 1.4% 3 2.2% 0 0.0% 0 0.0% 0 0.0% 2 1.7% 1 0.4% 0 0.0% 1 0.7% 1 0.7% 0 0.0% 0 0.0% 0 0.0% 0 3.7% 5 1.8% 4 3.3% 1 0.7% Non-FFLG members actually report 8.08 of MAHFP (CI: 6.81-8.87) compared with 7.83 (CI: 7.18-8.98) among FFLG members. This difference is not significant, however, taking into account that many FFLG members have been drawn from among households whose livelihoods are deemed to be vulnerable. Table 20: Months of Adequate Household Food Provision – Health Respondents # of Months Baseline Health Respondents (N=280) % Frequency % 12 8.2% 7 3% 11 6.2% 16 6% 10 19.1% 51 18% 9 18.9% 84 30% 8 14.7% 38 14% 7 12.6% 37 13% 6 4.7% 26 9% 5 4.9% 13 5% 4 2.5% 2 1% 3 2.2% 2 1% 2 1.7% 1 0% 1 0.7% 0 0% 0 3.7% 3 1% More mothers participating in the health component of the MYAP than members of FFLGs appear to believe that they have had MAHFP for more than 8 months (the average). Normally, one might assume that the women of the household have a better grasp of the dietary and overall food situation of the family. The definition of what constitutes “adequate” may differ between 37 males and females. Also, one could speculate that the male respondents tended to under-report in order to try to generate more assistance. It is impossible to determine the cause of the difference, but project staff should find ways to monitor food availability in the household. When farmers were asked what contributed to their household food shortfall, those most cited were:  Drought - 38.6%  Sickness - 21.8%  Flooding - 17.1%  Crop Infestation/Disease - 16.1%  Lack of Seed - 12.1%  Soil Infertility - 11.7%  Poor Productivity - 6.0% Based upon the survey data, the project has not yet been able to achieve its target of ensuring that households enjoy 10 months of adequate food provision. By the end of Year II virtually no improvement has occurred. However, this target was ambitious from the start since substantial impact progress in increasing agriculture/livestock production and other livelihoods sources rarely occurs in the brief span of 1.5 years of effective implementation, given production cycles as well as time required for changing attitudes and practices with regard to farming practices and levels of capitalization required for certain technological improvements conspire against quick results. While there is still one more year left in the MYAP during which to achieve the program’s target, the significant short-fall in food production from the last planting season makes achieving the necessary increase in MAHFP unlikely. In the case of this progress, a significant part of the roll-out of improved seed/cuttings varieties is based on “organic growth” through local seed multiplication, instead of outright grants or loans of seed. Whereas the most vulnerable households did receive high quality seed via seed fairs, the majority of households in the project zones continued to use traditional seed saved from harvests or bought in local markets. Focus group participants indicated that out of necessity, many farmers would opt for cheaper, but lower-yielding traditional varieties of seed to stretch their limited budgets. As mentioned above regarding seed/cuttings multiplication, within the limits of the scale and scope of the project the organic approach could not get enough improved seed to enough farmers in time to generate significant change in this impact indicator. The effect of seasonal rains along with other external household livelihoods factors such as disease or injury, a death of a family or an extended family member, or theft or acts of violence against a household or community are external factors which can have adverse effects on livelihoods, even to the point of off-setting gains that might be realized in production. Likewise, whether the baseline occurred during a good, poor or average agricultural season could affect appearance of progress or lack thereof. 38 4.1.28 Household dietary diversity score To determine the number of food groups being consumed by households, the evaluation surveyed only mothers participating in the health and nutrition program, unlike in the case of the baseline survey which questioned both mothers and farmers participating in the agriculture program. Experience indicates that women-of-the-household, who are the family member responsible for planning and preparing daily meals, are the most knowledgeable in this matter and have more accurate recall of foods used in the preparation of family meals. According to survey respondents, the mean number of food groups consumed the day before the survey was 5.95 (CI: 5.15-6.70), compared with 3.78 (CI: 3.58-3.98) acknowledged by health survey respondents at the time of the baseline. At the time of the baseline line 50% of households consumed two or fewer food groups per day. Currently, that median has risen to five, owing to major increasing in the number of households consuming six and seven groups in a day. Table 21: Household Dietary Diversity Score # of Food Groups Baseline Health Respondents (N=307) % Frequency % 1 4.4% 1 0.33% 2 26.5% 13 4.23% 3 30.2% 14 4.56% 4 19.1% 41 13.36% 5 10.2% 56 18.24% 6 5.2% 71 23.13% 7 1.5% 56 18.24% 8 1.5% 22 7.17% 9 0.5% 22 7.17% 10 0.3% 8 2.61% 11 0.3% 1 0.33% 12 0.2% 2 0.65% Not surprisingly, grain makes up the bulk of the household diet and is consumed daily by the vast majority families. While relative rankings have not varied to a significant degree, there has been an across-the-board considerable increase in the percentage of households reporting consumption of all food groups during the course of a day. Though vegetable consumption (essentially manioc leaves and cabbage) has remained basically stable (slightly down from baseline levels, though still robust), it has been replaced by fish in the #2 spot at a very high 87.9% rate of consumption. This shift might also be viewed within a framework of a livelihoods coping strategy as fish represent a hedge against drops in harvest due to poor rains. Focus group farmers reported having to go out and fish to feed their families (direct consumption and as barter for staple food) as well as to sell for cash in the face of poor harvests. 39 Focus groups reported that, for the most part, fruits are not produced on the household farm, and have to be purchased in the local market for home consumption. This indicates that awareness building and training as to the nutritional value of fruit consumption is taking hold. Most farmers grow beans as a cash crop. However, the increase in the number of households reporting consumption probably means that more families are retaining a part of their harvest for personal consumption instead of selling all for cash. That noted, however, when women participating in health focus groups were asked what their family had eaten the day before, no one mentioned beans and only a few noted that they served fruit, which they had to buy and the market and could not always serve due to cost constraints. Invariably, women responded that they served “fufu” (cassava), cassava greens and, perhaps, some fish (essentially a traditional daily diet before the advent of the MYAP). There was absolutely no mention of meat, eggs or dairy products. This inconsistency might be explained by survey responses being driven by a recall of nutrition lessons on the part of survey respondents who were providing what they learned to be the “correct answers” (as the list of groups was read by the survey taker), or factual recall limitation during focus group discussions, which were less guided. In the opinion of the external evaluator, the former is a more likely explanation. To get a better handle on “what is going on in the kitchen”, the project health teams should build in periodic monitoring processes for subtle visual inquiry during the course of daily activities in villages in order to record actual practices and analyze them against self-reporting on the part of beneficiaries. Table 22: Types of Food Groups Consumed (ordered by percentage) Evaluation Survey Baseline Food % Food % Grains 89.3% Grains 84.6% Fish 87.9% Vegetables 79.7% Tubors/Roots 86.6% Tubors/Roots 46.6% Vegetables 77.2% Fish 41.9% Oils 69.4% Oils 25.2% Legumes 52.8% Fruits 21.3% Fruits 45.6% Legumes 19.1% Sugar/Honey 30.9% Sugar/Honey 7.9% Other (tea, coffee, condiments) 24.1% Other 6.9% Meat 16.6% Meat 4.5% Eggs 10.7% Eggs 3.4% Dairy 3.6% Dairy 2% The project seems to have achieved beneficial impact in the form of household food provision and dietary diversity. These apparent gains, however, are still fragile since they are highly dependent on productive capabilities (farming and livestock) which are still in their early stages of development. Households are sure to face food shortages between the time of the evaluation 40 and the next harvest in September, 2011 due to the poor recent harvest. Household food consumption will likely be adversely affected as families tighten their belts in response. Therefore, the role that the project plays in increasing agricultural and livestock production through improvements in techniques and the use of improved varieties seed/cuttings takes on even more importance. FH should, in the final year of the current project, as well as in subsequent programming, look for ways to accelerate the rate of coverage in getting improved seed/cuttings into the hands of farmers and providing training/technical assistance through more FFLGs to backstop the gains achieved so far. 4.2 Strategic Objective 2: Improved Human Capabilities of Vulnerable Households Intermediate Result 2.1: Improved practice of Essential Nutrition Actions (ENA) by pregnant women and mothers of small children Output Indicators:  Total number of Care Groups (CG) currently active in health promotion  Average number of Health Extension Workers trained on ENA and EHA  Average number of community meetings held to share what CG members learned after modules are completed  Average number of direct beneficiaries who received health promotion on maternal and child nutrition through CG Teaching 4.2.1 Total number of Care Groups currently active in health promotion The project has fully achieved its target of establishing 188 CGs in all of the 179 villages comprising the Kalemie and Moba districts. In a few cases of particularly large villages, more than one CG functions. The project cascade model has been effectively adhered to, with up to 10 Mother Leaders (MLs) in each CG taking responsibility to pass on training to beneficiary mothers assigned to them, provide motivation/promotional information to mothers of small children and growth monitoring of infants and channeling nutritional and health status information back to FH health promoters and agents for further action with local health centers. The project has successfully achieved full geographic coverage and, through its cascade model, been able to identify and reach all pregnant women and mothers of children who are two years old or under in their villages. This level of coverage is fundamental for achieving and sustaining improved attitudes and practices in matters of health, hygiene and nutrition. 4.2.2 Average number of Health Extension Workers trained on ENA and EHA The project has fully achieved its target of recruiting and training 1,870 women to carry out the role of Health Extension Workers, referred to as Mother Leaders (ML). MLs are volunteers who 41 are recruited on the basis of their respect in the community, their spirit of service and their commitment to the goal of improving health and nutrition levels among their fellow mothers and their young children. Over the course of the project, FH has modified the its target of organizing training monthly meetings with MLs to a schedule of providing one full training three times per year (once for each module). Given that project field promoters are in periodic contact with MLs and their communities throughout the year, there are numerous opportunities for MLs to receive “on the job” refresher assistance and backup on the subject matter in which they are trained. This is a more rational use of ML time (which is tight given their volunteer status and personal obligations in the home and community) and a more action-based way of reinforcing knowledge and practices. During focus group discussions, MLs emphasized the amount of time and effort required of them, even under this rationalized schedule. Nevertheless, MLs are committed to and take pride in their work and accomplishments. 4.2.3 Average number of community meetings held to share what Care Group members learned after modules are completed This output indicator got off to a slow start in Year I, as the project team spent that year identifying MLs and organizing CGs in communities throughout the two districts. An FH Training and Curriculum Specialist developed a 3-module curriculum (each consisting of six EHA/ENA topics), along with their corresponding lesson plans and didactic materials, especially flip charts to be used by MLs in carrying out training with mothers in their villages. All materials were developed specifically for the context of project zone (terminology, translation and illustration). By the time cost-effective printing was procured, training in only the first module could be conducted for MLs. The remaining two modules of training MLs were conducted in Year II. During the second year, MLs and the health promoters set out to organize 2 meetings with community members in each of the 188 CG communities (376) in order to share with them what MLs learned in the training. In the face of labor obligations (particularly planting and harvesting), family obligations (funerals – a strict social obligation) and other inconveniences, scheduling sessions in all communities was difficult. Consequently, only 213 sessions could be organized for a 57% rate of completion. The afore-mentioned inconveniences which make full attendance impossible will continue to exist. Therefore, the target should be re-articulated to document/measure that all beneficiary mothers receive the full series of training at one time or another. This would entail organizing more sessions (albeit with smaller groups) in order to offer all beneficiaries convenient opportunities for attendance. The additional cost of time and effort of project staff should result in greater coverage, which is essential for realizing impact in the project zone. 42 4.2.4 Average number of direct beneficiaries who received health promotion on maternal and child nutrition through CG Teaching According to project monitoring records, the project is successfully reaching practically all (98.18%) of its targeted 15,000 direct beneficiaries. 90.7% of evaluation survey respondents acknowledged health training sessions offered through the CGs. 14,877 women (averaging 79 women per CG) have been receiving training in the three modules of EHA/ENA over the course of the first two years of the project. Effect (Results) Indicators:  Percentage of beneficiary children 6-23 months of age who have received Vitamin A supplements in the past six months  Percentage of beneficiary children 0-5.9 months of age who were exclusively breastfed in the last 24 hours  Percentage of beneficiary children 6-23.9 months of age who have been provided three or more infant and young child feeding practices (continued breast feeding, age-appropriate dietary diversity, and age-appropriate frequency of feeding) 4.2.5 Percentage of beneficiary children 6-23 months of age who have received Vitamin A supplements in the past six months The evaluation survey revealed that 70.8% of children in the designated age group had received a vitamin A supplement in the prior six months. This represents 83% of the project’s established Year II target of 85% of young children. However, a LQAS conducted by FH in June 2010 found that 84.7% of those surveyed acknowledged that their children had received the supplement. While the discrepancy is of some significance, and merits further monitoring, even the lower level of achievement still represents a 32 percentage point improvement from baseline the level of 38.7%, nearly double in two years. In any case, the project has been successful in positively changing mothers’ practice of an essential health action. 4.2.6 Percentage of beneficiary children 0-5.9 months of age who were exclusively breastfed in the last 24 hours According to the evaluation survey, 70 out of 71 children between the ages of 0 and 5.9 months were being breast fed at the time of the survey (98.6%). Of the 69 mothers who could recall all liquids which there children had consumed, 52 (75.4%) had been exclusively breast-fed during the preceding 24 hours. This level of practice falls slightly short of the 80% target set for Year II and the LOA ( 93.25% achievement rate). Considering the baseline starting point of 28.4% for this ENA, it can be considered that the project has been successful in positively transferring knowledge that is being practiced by mothers of newborn children. 43 Along the lines of infant breast-feeding, MLs have been building awareness among women of the importance of beginning breast-feeding immediately after child birth, rather than wait extended periods as per traditional practice. The evaluation survey found that 77.5% children from 0 to 11.9 months of age (chosen to reduce recall bias and to coincide with the time that the project has actually been implementing awareness-building/training) commenced breast-feeding within two hours of birth. This represents a significant improvement from the 39.9% rate recorded at the baseline. For all children from 0-23.9 months of age, the rate dropped to 71.2%, perhaps as a result of recall bias or early births prior to the start of awareness-building/training. 4.2.7 Percentage of beneficiary children 6-23.9 months of age who have been provided three or more infant and young child feeding practices (continued breast feeding, age-appropriate dietary diversity, and age-appropriate frequency of feeding) Survey respondents revealed that more mothers were still breastfeeding young children up to two years of age. 96.6% of mothers of children 12 to 17.9 months of age were breastfeeding at the time of the survey (up from a baseline of 88.1%). Similarly more mothers of 18 to 23.9 month old children (82.9%) were breastfeeding compared with 50.9% at the time of the baseline. According to evaluation survey responses children seem to receive more food groups within their diet, particularly in the 4-5 group range and among the 6-11 and 12-17 month age groups. The average number of food groups consumed for all age groups was 4.5 (CI: 3.5-5.2), with the average increasing as children got older. The average 18-23 month old child is now consuming 5.07 groups (CI: 2.9-7.2). Overall, 73.2% of all children met the recommended level of consumption of 4 or more food groups (90.4% for children 18-23 months), surpassing by more than double the baseline percentage of 31.6% Table 23: Number of food groups consumed by children 6-23.9 months (by age group) 6-11 months 12-17 months 18-23 months # of Groups Frequency (N-79) % (Baseline) Frequency (N=89) % (Baseline) Frequency (N=41) % (Baseline) 0 2 2.5% (13.6%) 3 3.4% (4.5%) 0 0.0% (0%) 1 10 12.7% (30.7%) 1 1% (10.4%) 0 0.0% (7%) 2 6 7.6% (20.5%) 4 4.5% (19.4%) 1 2.4% (26.3%) 3 13 16.5% (14.8%) 13 14.6% (29.9%) 3 7.3% (22.8%) 4 9 11.4% (13.6%) 12 13.5% (11.9%) 9 22.0% (29.8%) 5 18 22.8% (5.7%) 28 31.5% (14.9%) 9 22.0% (10.5%) 6 20 25.3% (0%) 21 23.6% (0%) 17 41.5% (0%) 7 1 1.3% (0%) 7 7.9% (0%) 2 4.9% (0%) Across the board, more children are being given more nutritious foods. The consumption of dairy products and eggs is still only practiced in a very small percentage of households. Meanwhile vegetables rich in vitamin A and foods rich in protein are readily are commonly available, and traditionally eaten in the form of cassava leaves and dried fish. 44 Table 24: Dietary diversity among children 6-23.9 months Food Groups Evaluation Survey (N=209) Baseline Frequency % % Dairy Products 16 7.7% 4.2% Grains and Tubors 203 97.1% 76.4% Vitamin A Rich Fruits and Vegetables 183 87.6% 68.4% Other Fruits and Vegetables 121 57.9% 19.3% Eggs 11 5.3% 2.4% Meat, Poultry and Fish 164 78.5% 49.5% Legumes and Nuts 106 50.7% 23.6% Oils and Fats 122 58.4% 24.1% Based on survey responses, 61.22% (N=196) of children between the age of 6 and 23.9 months were provided all three of the practices mentioned in the target (continued breast feeding; age￾appropriate dietary diversity – 4 or more food groups consumed; and age-appropriate frequency of feeding – feeding 2 or more times a day for children 6-8 months; and 3 or more times for children 9-23 months). This slightly exceeds the Year II target of 60% and the level of 58% which was recorded by a mini-KPC conducted by FH. Intermediate Result 2.2: Improved mothers’ ability to prevent, diagnose and manage common childhood diseases that exacerbate malnutrition Output Indicator: 4.2.8 Number of MOH Child Days Campaigns supported and facilitated With FH technical and financial support, the MOH have organized three Child Days Campaigns to date – one during Year I and two in Year II in each of the villages in which the MYAP has been implemented. During those events MOH staff organized awareness-building and training on de-worming and vitamin supplements for mothers and caregivers of young children. In addition de-worming medicine and Vitamin A supplements were provided to children in attendance. Those supplies were provided by FH from the MYAP grant. During the two session organized during Year II, 49, 832 children received de-worming treatment (18,733 in Moba and 31,099 in Kalemie) and 55,110 children received Vitamin A supplements (20,764 in Moba and 34,346 in Kalemie). These beneficiaries represent close to 100% coverage of children in the villages served. Effect (Results) Indicators:  Percentage of beneficiary care-givers of children 0-23 months of age who regularly meet with a health promoter to learn about health (bi-weekly or more often) 45  Percentage of beneficiary care-givers of children 0-23 months of age who know at least three signs of childhood illness that indicate the need for treatment 4.2.9 Percentage of beneficiary care-givers of children 0-23 months of age who regularly meet with a health promoter to learn about health (bi-weekly or more often) By systematizing promotional meetings/trainings with MLs and Community Health Workers (CHW), the project has been successful in increased the number mothers/caregivers who receive awareness-building and training on essential health practices. The evaluation survey found that 75.4% of mothers/caregivers have met with a CHW and/or ML in the last month to learn about health practices, compared to 9.3% at the time of the baseline. Over one-half of respondents acknowledged meeting with promoters twice or more times. Table 25: Percentage of beneficiary care-givers of children 0-23 months of age who regularly meet with a health promoter to learn about health (bi-weekly or more often) Evaluation Survey (N=212) Baseline Number of times Frequency Percent Percent Once per month 59 21.0% 6.0% Twice per month 93 33.1% 3.3% 3 times per month 47 16.7% More than 3 times per month 13 4.6% In terms of frequency, the survey revealed that 54.4% of respondents met with a CHW/ML bi￾weekly or more often, compared with only 1.7% at the time of the baseline. While this represents a very significant improvement, it only represents a 78% achievement of the project’s target of a 70% rate of coverage by the end of Year II. Work (planting, tending to fields, and harvest) and family obligations (illnesses and funerals) have proven to be real limiting factors for beneficiary mothers/care givers and MLs being able to fully meet schedules. The project staff has been mindful of this limitation and has built in training redundancy (visiting absent mothers to repeat lessons that were missed to try to achieve the most coverage possible of women receiving the full array of lessons. This has increased the work load of volunteer MLs, who have so far accepted the responsibility with a high level of commitment. In light of limiting factors that are not always under the control of the project team, and the team’s response to try to mitigate those factors, this target should be considered met to the extent that it is reasonably possible under the circumstances. Having met regularly for awareness-building and training, the next concern should be mothers’/caregivers’ recall of the types of training their received over the course of the past year. The topics of “breast-feeding” and, “hygienic food preparation” were those most cited by 46 mothers, while those of “sharing health messages with friends/family” and “nutrition screening” were notably low on the recall scale. Table 26: Number/Percentage of health training topics recalled by mothers/care givers Topic Evaluation Survey (N=284) Frequency % Breast-feeding 259 91.2% Nutrition screening of young children 52 18.3% Hygienic food preparation 259 91.2% Foods that are rich in Vitamin A 209 73.6% Hand-washing 234 82.4% Malaria and other parasite transmission/control 116 40.8% Sharing health messages with family/friends 22 7.7% Hygienic food storage 134 47.2% Appropriate foods/feeding of sick children 160 56.3% 81% of respondents recalled more than four topics, with 66.9% recalling between 4 and 6, and 14.1% recalling 7 or 8 topics. More mediocre retention rates for “malaria transmission/control” and “hygienic food storage” do not bode well for high rates of practice in actual fact . 4.2.10 Percentage of beneficiary care-givers of children 0-23 months of age who know at least three signs of childhood illness that indicate the need for treatment Awareness building and training seem to be having a positive effect on mothers/care givers awareness of and knowledge about childhood illnesses which should indicate to them a need to seek treatment. The percentage of women who can cite at least three illnesses has risen to 72.6% from the baseline level of 21.7%. This level would place achievement above not only the Year II target (40%), but also the LOA target of 60%. Table 27: Percentage of beneficiary care-givers of children 0-23 months of age who know at least three signs of childhood illness that indicate the need for treatment Evaluation Survey (N=281) Baseline # of Signs Frequency % Rev. Cumm % Rev. Cumm 0 1 0.4% 100.0% 5.3% 100.0% 1 15 5.3% 99.6% 33.0% 94.7% 2 61 21.7% 94.3% 40.0% 61.7% 3 61 21.7% 72.6% 16.0% 21.7% 4 104 37.0% 50.9% 3.7% 5.7% 5 25 8.9% 13.9% 2.0% 2.0% 6 12 4.3% 5.0% 0.0% 0.0% 7 2 0.7% 0.7% 0.0% 0.0% 47 Mothers/care givers have gained greater awareness of all signs of illness across the board compared to the start of the project. This can be attributed to the broad base of training provided by the project. Table 28: Number/Percentage of mothers/care givers who know particular signs of childhood illnesses Signs of Illness Evaluation Survey (N=281) Baseline Frequency % % Looks unwell or not playing normally 199 70.8% 34.3% Not eating or drinking 148 52.7% 20.3% Lethargic/difficult to wake 38 13.5% 3.7% High fever 259 92.2% 83.0% Fast or difficult breathing 36 12.8% 10.0% Vomiting 64 22.8% 9.0% Convulsions 46 16.4% 5.3% Other valid signs (incessant crying, diarrhea, cough, palmor pallor, conjunctivitis, bloody stools, wasting, groaning, generalized edema, and refusal to breastfeed) 133 47.3% 17.7% During focus group discussions, MLs and beneficiary mothers/care givers spoke often and highly of the value they place on the training and the beneficial effects they see in the health of children in the community. Frequently participants were able to cite 6-9 signs of illness which have been the topic of training carried out by the project (these responses were from the group as a whole, not any one particular participant). It is safe to conclude that the project has been successful in raising the awareness among and training beneficiaries of signs which they should know that signal childhood illness which requires medical attention. Other mother/caregiver practices associated with improving the health of infants and young children Though not specifically established as effect indicators associated with awareness-building and training received by mothers and care-givers from the project, several fundamental and basic practices, as well as knowledge and attitudes with regard to HIV/AIDS were tracked as part of the evaluation survey:  Prenatal care  Use of Insecticide-treated bed nets to control malaria  Regular de-worming of young children  Diarrhea treatment  Knowledge and attitudes with regard to HIV/AIDS 48 4.2.11 Pre-natal care The evaluation survey found that 100% of mothers of children from 0-11 months of age had received pre-natal care at some time during their pregnancy. This represents an improvement from an already high baseline of 98.1%. In terms of the number of times that care was provided, there has been no significant increase from the high levels noted in the baseline. Over 52% of mothers reported having received care more than 3 times during their most recent pregnancy. Table 29: Number of times mothers of children between 0-11 months of age received pre￾natal care during pregnancy. Number of Times Evaluation Survey (N=281) Baseline Frequency % % 1 time 2 2.8% 3.8% 2 times 10 13.9% 12.2% 3 times 22 30.6% 29.5% More than 3 times 38 52.8% 52.6% Don’t know/blank 0 0.0% 1.9% While basic coverage levels are now where they should be, there is still plenty of room for improving the number of times that mothers seek/receive pre-natal treatment. Similarly, the project has yet to make improvements to the existing rates at which pregnant women receive Iron supplements. 74% if mothers of children 0-11 months received this basic treatment (compared to a baseline of 73.5%). 4.2.12 Use of Insecticide-treated bed nets (ITBN) to control malaria Malaria is one of the most prevalent illnesses affecting the population of the zone. During focus group discussions its prevalence was confirmed, yet households’ ability to practice the lessons learned about the beneficial effect ITBNs can have on controlling the contraction of malaria is hampered by cost constraints. The evaluation survey found that only 46% of households acknowledge having ITBNs, an increase from 24.8% at the time of the baseline survey. Of those who responded positively, 79.2% stated that “the survey child (under 2 years of age) and others” slept under the bed net. From focus group responses, “others” were exclusively parents. All other children were said to sleep un-protected. However, direct observations conducted by survey takers found that, in fact, a smaller percentage (34.2%) of surveyed families actually had all or some beds covered by nets. This, too, is an improvement from similar direct observation conducted during the baseline survey, which revealed 20.5% coverage. The discrepancy underscores the possibility that self￾reporting in the survey may be based on theory as if respondents were “answering test questions. 49 Table 30: Direct observation of the number of households which were seen to have beds covered with ITBNs. Existence of ITBNs Evaluation Survey (N=281) Baseline Frequency % % Bed nets over all beds 60 19.2% 3.6% Bed nets over some beds 47 15.0% 17.9% No bed nets 155 49.5% 74.6% Not permitted to enter or blanks 51 16.3% 3.9% 4.2.13 Regular de-worming of young children At the time of the baseline, 27.4% of children between 12 and 23.9 months had been de-wormed. The evaluation survey found that 72.3% of the children of that age group had received the treatment (94 of 130 children). This level of coverage was achieved through support for Ministry of Health organized Child Day Campaigns. 4.2.14 Diarrhea treatment The evaluation survey found a very small positive change in the percentage of survey children who experienced at least one case of diarrhea in the previous two weeks (36.4% compared to the baseline of 37.7%). Where significant improvement has occurred is in the percentage of mothers/care givers who acknowledge having treated their children with either ORS packets or with recommended home solutions. Table 31: Use of oral re-hydration solutions to treat diarrhea in young children Incidence of Diarrhea Type of Treatment Evaluation Survey (N=281) Baseline Frequency % % Children who have had a case of diarrhea in the last 2 weeks 114 36.4% 37.7% Children treated with prepared ORS paquettes 61 53.5% 27.5% Children treated with recommended home solutions 12 19.7% 7.5% Children treated with both 10 8.8% One should take in mind that the evaluation survey was conducted in September, at the very end of the dry season, whereas the baseline survey was conducted in February, when rains and cases of diarrhea tend to be more prevalent. This calls into question the comparability of the data, though the lack of significant improvement at the time of the evaluation reinforces the probability that the program has yet to achieve positive impact in lowering the incidence of diarrhea. However, it would be unreasonable to expect appreciable positive impact in the short span of less than two years of effective implementation. What is notably positive is improvements in mothers/caregivers’ ability to appropriately treat cases of diarrhea. 50 4.2.15 Knowledge and attitudes with regard to HIV/AIDS According to evaluation survey responses, there has been improvement in the number/percentage of mothers/care givers who possess knowledge and attitudes with regard to the transmission and prevention of HIV/AIDS. Awareness of the disease has increased from a baseline of 72% to 84.7% of survey respondents. Across the board, greater numbers of mothers/care givers were knowledgeable of 5 key pieces of information associated with HIV/AIDS and its transmission. Table 32: Number/Percentage of mothers/care givers who are knowledgeable of HIV/AIDS and its transmission Knowledge and Common Beliefs Evaluation Survey (N=281) Baseline Frequency % % Having one uninfected and faithful partner can reduce the risk of being infected 228 72.8% 50.5% Use of a condom during intercourse can reduce the risk of contracting HIV 182 58.1% 35.9% An apparently healthy person can be infected with HIV 155 49.5% 34.1% A person cannot be infected with HIV from a mosquito bite 121 38.7% 22.6% A person cannot be infected with HIV by sharing meals with an infected person 146 46.6% 43.6% Mothers/caregivers with a comprehensive knowledge of HIV/AIDS 47 15.0% 1.7% Knowledge with regard to the fact that fidelity to an uninfected partner and the use of condoms can decrease the risk of contracting HIV represented the largest improvements, reaching levels above 50%. Fortunately, these two issues deal with pro-active actions that persons can take to reduce their risk of infection. Levels of change of common beliefs about how HIV is transmitted to a lesser degree. Surprisingly, still only 38.7% of respondents were aware of the fact that HIV cannot be transmitted through mosquito bites and less than 50% knew that one cannot contract HIV by sharing meals with an infected person. The fact that 15% of mothers/caregivers now demonstrate a comprehensive knowledge of HIV/AIDS (correct knowledge of all five facts) represents a significant improvement over the baseline, though that level still very low, leaving much work to be done. It is worth noting that a full Care Group training module on HIV/AIDS has not been taught as of the end of Year II. It is reasonable to assume that once a module of this nature is available to care group members, the level of comprehensive knowledge should increase even more. In contrast, there has been no improvement in the percentage of mothers/caregivers who know about transmission risk from mother to infant. The percentage of mothers/caregivers who possess knowledge of the four basic pieces of information has dropped slightly in each case. Very few respondents are aware of the fact that retro-viral drugs are available to reduce the risk of transmission. 51 Table 33: Beliefs and opinions about the risk of transmission of HIV from mother to infant Knowledge and Common Beliefs Evaluation Survey (N=281) Baseline Frequency % % The virus can be transmitted from an infected mother to infant during pregnancy 163 52.1% 55.3% The virus can be transmitted from an infected mother to infant during delivery 172 55.0% 55.3% The virus can be transmitted from an infected mother to infant through breast-feeding 178 56.9% 63.1% There are special drugs that a doctor/nurse can give to an infected mother to reduce the risk of transmission 50 16.0% 20.3% Number of caregivers with a comprehensive knowledge about HIV transmission from infected mother to infant 23 7.3% 6.7% Intermediate Result 2.3: Improved access to clean water, sanitation facilities and of essential hygiene behaviors Output Indicators:  Number of constructed water points which pass water quality tests  Number of school/market latrines constructed a safe distance from water sources 4.2.16 Number of constructed water points which pass water quality tests The project has met its targets for Years I and II to install 50 protected water points. These were constructed in 48 villages (two large villages received two sources each) in as many villages. The macro and micro selections processes included the involvement of community leaders to determine the localities of particular need and to help prioritize potential sites to address the needs of most vulnerable community members. 25 water points were constructed in the two project sites – Kalemie and Moba. In each site 14 wells were dug were dug/protected and 11 natural springs were captured/protected. All sources were tested before construction commenced and then after construction in July. All sources were found to meet quality standards for potability. In each community where a water point was constructed, a village WASH committee was either strengthened/supported or one was set up to manage the care and maintenance of the source. It was learned during focus group conversations that each community has been left to devise a cost-recovery scheme to cover repairs and maintenance that require cash outlays. Some communities have opted for a modest monthly fee for all users (usually in the case of sources that are located in particularly convenient locations and are thus highly sought by users). Others have elected to make a general community assessment when an outlay must be made. The 52 approach on the part of the project team is meant to reinforce community “ownership” and to stimulate learning opportunities with regard to management and sustainability. This output target has been successfully met through Year II of the project. 4.2.17 Number of school/market latrines constructed a safe distance from water sources 28 latrine units have been constructed against the target of 30. However two, at the request of the community on the basis of the size of schools which would be served by them, were six-door units instead of the standard four-door units. Therefore, the 40 doors targeted for the first two years was met by the project. Schools selected to receive the units were prioritized on the basis of their deficit in latrines in proportion to their student body as well as the condition of existing latrines. School administrations and the school’s parent associations have accepted responsibility for the units’ upkeep. Public places, such as schools and markets, which attract numerous persons were targeted as the limited number of sites for constructing latrines for their promotional effect. According to conversations with focus groups, virtually all households in the project area already have family latrines. If they do not have them it is because their pits had caved in (a factor of soil type and the depth of the water table) and the families were in transition toward building replacements. This output target has been successfully met through Year II of the project. Effect (Results) Indicators:  Percentage of households with year-round access to improved water sources, where access means direct connection to the home or public facility within 400 meters of the home  Percentage of beneficiary households adopting at least three improved hygiene behaviors 4.2.18 Percentage of households with year-round access to improved water sources, where access means direct connection to the home or public facility within 200 meters of the home The evaluation survey revealed that 72.8% of households draw water from an improved primary source. This represents a marked improvement from the 47.4% level at the time of the baseline. In addition, 62.8% of households have sources that are more accessible (200 meters or less from their houses). At those levels, the project will have exceeded its target LOA threshold of 55%. 53 Table 34: Percentage of households with access to improved water sources (regardless of distance from house) Consolidated (N=309) Kalemie (N=159) Moba (N=150) Baseline Improved Source Frequency % Frequency % Frequency % % Public Tap 76 24.6%  ‐    0.0% 76   50.7% 24.6% Protected Well 90 29.1% 59 37.1% 31   20.7% 8.5% Protected Spring 58 18.8% 25 15.7% 33   22.0% 14.0% Bottled Water 1 0.3% 1 0.6%  ‐    0.0% 0.3% All Protected Sources 225 72.8% 85 53.5% 140   93.3% 47.4% Table 35: Percentage of households with year-round access to improved water sources within 200 meters of the home Consolidated (N=309) Kalemie (N=159) Moba (N=150) Improved Source Frequency % Frequency % Frequency % Public Tap 49 15.9%   ‐     0.0%  49   32.7% Protected Well 68 22.0%  45   28.3%  23   15.3% Protected Spring 30 9.7%  9   5.7%  21   14.0% Bottled Water 1 0.3%  1   0.6%   ‐     0.0% All Protected Sources 148 47.9%  55   34.6%  93   62.0% When all criteria are taken together, 47.9% of households acknowledge that they get their drinking/cooking water from a protected source that is accessible (200 meters or less from the house) and available year-round. This is a marked improvement from the baseline rate of 15.7%. In this regard, the program has exceeded its year II target of 45% and has achieved nearly 89% of its LOA target of 55% comprehensive coverage. In all villages where focus groups were conducted, participants noted that they had protected sources of water prior to the initiation of the project. However those sources were inadequate to meet the needs of the population of the village. The construction of 50 new protected sources has contributed to bringing protected sources of water closer to a greater percentage of the population of the project sites. Considerably more households in Moba utilize protected primary sources of water than those in Kalemie. The concentration of public taps as a source in the Moba region may be the result of past humanitarian transition projects in support of refugee returns. In the first two years of the project, the number of water sources constructed were evenly distributed between the Moba and Kalemie sites. Going forward, however, FH should consider shifting more investments to villages in Kalemie district in order to bring it more in line with coverage levels demonstrated in Moba. 54 4.2.19 Percentage of beneficiary households adopting at least three improved hygiene behaviors Training of MLs and beneficiary mothers has focused on five basic, but essential, hygiene behaviors/practices that are associated with reducing the incidence of illness, particularly among children. The five have been:  Regular treatment (purification) of drinking water at the point of use (POU)  Covering containers used for storing drinking water  Regular hand-washing  Proper disposal of family fecal material  Proper disposal of the fecal material of infants and young children In all matters, the project appears to be having the positive effect of inducing more families to practice more improved hygiene behaviors compared with levels found at the time of the baseline. Regular treatment (purification) of drinking water at the point of use (POU) The evaluation survey revealed a notable increase in the percentage of households which practice some form of water purification treatment at the POU. The percentage has risen to 38.5% from the baseline of 14%. Within the universe, however, households in Kalemie are implementing improved practices in greater percentages than their cohorts in Moba (64.8% vs. 10.7%). Whereas the vast majority of those who practice POU purification in Moba boil water, those in Kalemie choose to add chlorine or chlorine products. Table 36: Households which practice POU treatment of drinking water Consolidated (N=309) Kalemie (N=159) Moba (N=150) Baselin e Practice Frequency % Frequency % Frequency % % Boiled before using 31 26.1% 18 17.5% 13 81.3% 7.3% Add chlorine or chlorine product 77 64.7% 74 71.8% 3   18.8% 92.7% Filtered before using 3 2.5% 3 2.9%  ‐    0.0% 0.0% Purified using the sun 1 0.8% 1 1.0%  ‐    0.0% 0.0% Other (Aquatabs, PUR, etc.) 7 5.9% 7 6.8%  ‐    0.0% 0.0% Households Which Practice POU Treatment 119 38.5% 103 64.8% 16 10.7% 14.0% Household Which Do Nothing 184 59.5% 53 33.3% 131   87.3% 85.7% While Kalemie’s progress is very encouraging, the low percentage of households who treat their drinking water in Moba (below baseline levels) is particularly troubling, despite promotional and educational efforts of the project. Further investigation should be carried out to determine the nature and causes of the variance. The difficulty associated with obtaining fuel wood in 55 Kalemie, could possibly explain the low numbers of households which boil their water as a purification method in Kalemie In terms of the regularity/frequency of treatment, again the project has resulted in improvement in the percentage of households that treat their drinking water daily or every other day (of those which acknowledge practicing treatment). Compared to a baseline figure of 39%, the evaluation survey found that 89.8% of households had treated their water on the day of the survey or the day before (84.4% in Kalemie vs. 50% in Moba). Covering containers used for storing drinking water The project has succeeded in increasing the number of households that practice this low-cost means of protected drinking water from being contaminated once brought back to the house. Direct observation carried out by evaluation survey takers revealed that 59.5% of households covered all of their water containers. This compares favorably baseline findings that showed that 53.8% of households covered all of their water containers. An additional 35.6% of homes observed during the evaluation survey covered at least some containers. Moba households were more advanced in this regard, though the considerable number of Kalemie households observed to cover some containers could provide some optimism that continued awareness-building and training could result in Kalemie catching up. Table 37: Households which use covered drinking water containers Consolidated (N=309) Kalemie (N=159) Moba (N=150) Baseline Frequency % Frequency % Frequency % % All Covered 184 59.5% 83 52.2% 101 67.3% 53.8% Some Covered 110 35.6% 67 42.1% 43 28.7% None Covered 7 2.3% 5 3.1% 2 1.3% Not permitted to see 3 1.0% 1 0.6% 2 1.3% Blanks 5 1.6% 3 1.9% 2 1.3% Regular hand-washing Within the scope of the hygiene module being taught to Care Groups hand-washing features prominently. During focus group discussions, women frequently cited hand washing as a practice they and their families carry out. Nevertheless, when survey enumerators carried out direct observation to determine if hand-washing materials were in evidence, they could find such evidence in only a small percentage of the households surveyed. Up from a negligible 0.3% of households at the time of the baseline, 7.1% of households show evidence of practicing hand-washing. Soap was the sole product seen for the purpose. The percentage achieved is still too low to considered the project’s hand-washing promotion/training a success. 56 Table 38: Evidence of hand-washing supplies in houses Consolidated (N=309) Kalemie (N=159) Moba (N=150) Baseline Type of Supplies In Evidence Frequency % Frequency % Frequency % % Soap 22 7.1% 7 4.4% 15 10.0% 3.7% Detergent 0 0.0% - 0.0% - 0.0% Ashes 0 0.0% - 0.0% - 0.0% 0.7% Sand 0 Other 0 0.0% - 0.0% - 0.0% None 4 1.3% 3 1.9% 1 0.7% Blanks 283 91.6% 149 93.7% 134 89.3% 95.5% Households with Supplies 22 7.1% 7 4.4% 15 10.0% 4.4% The very high number of “blanks” is likely the result of caregivers not being able to produce the supplies they supposedly use in a reasonable amount of time to be considered valid (this is to avoid respondents borrowing products in order to appear that they practice the behavior. Direct observation failed to identify in a very large majority of the houses a specific place where hand-washing was being carried out. In less than 10% of the houses, hand-washing materials were located in one of the usual primary locations – in or near toilets/latrines or in or near kitchens. Table 39: Places where hand-washing is being carried out Consolidated (N=309) Kalemie (N=159) Moba (N=150) Location Frequency % Frequency % Frequency % In or near toilets/latrines 7 2.3% 0 0.0% 7 4.7% In or near the kitchen 10 3.2% 6 3.8% 4 2.7% Other 10 3.2% 4 2.5% 6 4.0% No specific place 270 87.4% 143 89.9% 127 84.7% Not permitted to see 3 1.0% - 0.0% 3 2.0% Blanks 9 2.9% 6 3.8% 3 2.0% The percentage of households which demonstrate all three signs that regular hand-washing is being practiced (a designated location existed; hand-washing supplies were present; and water was present) has increased significantly since the initiation of the project. Households in Moba have made greater advances in this regard than their cohorts in Kalemie. Nevertheless, there is considerable work left to be done to ensure that hand-washing is widely practiced among the majority of the population. Similarly, it would seem that extra attention will have to be paid to awareness-building/training activities in Kalemie to increase practice adoption rates. 57 All households: 6.15% (19/309) Kalemie households: 3.14% (5/159) Moba households: 9.33% (14/150) Baseline (all households): 0.30% It is worth noting that participants in focus groups for the most part gave prominent reference to their families’ practice of hand-washing; even noting such natural materials such as ashes and sand as supplies that they use. The direct observations reveal a different reality. In this light, project staff must be mindful of the ability of beneficiaries to “respond for the test questions” successfully (telling the inquirer what he/she is looking to hear as the “correct answer”), while not having internalized the practice. Focus group participants frequently made note of their inability to afford container (“bidones”) for fabrication of “tippy taps”, this alone does not explain the low rate of hand-washing practice. While the affordability limitation in the minds of beneficiaries should be addressed by the project going forward, more attention will needs to be paid to reinforcing the message that hand-washing is by far the most fundamental and most cost-effective disease prevention practice. Proper disposal of family fecal material Starting from a high baseline of 84.7% latrine usage by households, the project has contributed to making marginal improvements. More noteworthy might be the finding that even fewer households simply allow fecal material to be left out in the open. The percentage has dropped from a baseline of 6.8% to 0.3% according to survey responses. Table 40: Means by which households dispose of fecal material Consolidated (N=309) Kalemie (N=159) Moba (N=150) Baseline Means Used Frequency % Frequency % Frequency % % Toilet or latrine 277 89.6% 138 86.8% 139 92.7% 84.7% Drain or ditch 6 1.9% 3 1.9% 3 2.0% 0.0% Thrown into garbage 4 1.3% 2 1.3% 2 1.3% 0.0% In a hole/buried 2 0.6% 1 1 0.7% 4.7% Left in the open 1 0.3% 1 0.6% - 0.0% 6.8% Other 9 2.9% 8 5.0% 1 0.7% 3.1% Don't know and blanks 10 3.2% 6 3.8% 4 2.7% 0.7% Proper disposal of the fecal material of infants and young children Project awareness-building and training is having a positive effect on care-givers taking greater care in disposing of infant and young children’s fecal material. More children are being taught and allowed to use latrines. That, combined with simply rinsing the material down into a sanitary latrine, result in over 80% practice of the most sanitary means of disposal (compared to the baseline of 57.9%). Notable, once again is the drop in the percentage of households which 58 simply permit fecal material to be left out in the open. The percentage has dropped from a baseline of 30.1% to 1.9%. Table 41: Means by which households dispose of the fecal material of infants and young children Consolidated (N=309) Kalemie (N=159) Moba (N=150) Baseline Means Used Frequency % Frequency % Frequency % % Child uses toilet or latrine 73 23.6% 51 32.1% 22 14.7% 14.7% Put/rinsed into toilet or latrine 177 57.3% 75 47.2% 102 68.0% 43.2% Put/rinsed into drain or ditch 6 1.9% 3 1.9% 3 2.0% 1.4% Thrown into the garbage 18 5.8% 5 3.1% 13 8.7% 4.1% Buried in a hole 15 4.9% 11 6.9% 4 2.7% 6.5% Left in the open 6 1.9% 5 3.1% 1 0.7% 30.1% Other 7 2.3% 4 2.5% 3 2.0% 0.0% Don't know and blanks 7 2.3% 5 3.1% 2 1.3% 0.0% Taking into account the above mentioned findings, the project has been successful in increasing the percentage of households which practice three or more improved behaviors. That percentage has more than doubled to 66.3% from a baseline of 31.4%. It also represents an achievement of the projects LOA target of 65%. Of note also is the finding that the percentage of households practicing no behaviors has fallen to 1.9%. Table 42: Percentage of beneficiary households adopting at least three improved hygiene behaviors Evaluation Survey (N=309) Baseline # of Behaviors Frequency % Rev. Cumm % Rev. Cumm 0 6 1.9% 100.0% 6.0% 100.1% 1 16 5.2% 98.1% 21.7% 94.1% 2 82 26.5% 92.9% 41.0% 72.4% 3 134 43.4% 66.3% 29.7% 31.4% 4 66 21.4% 23.0% 1.7% 1.7% 5 5 1.6% 1.6% 0.0% 0.0% In both sites, the percentage of households practicing improved behaviors have exceeded that of established in the baseline. Those practices appear to have taken greater hold in Kalemie compared with Moba, leaving Moba still shy of the overall LOA target of 65%. 59 Table 43: Percentage of beneficiary households adopting at least three improved hygiene behaviors – disaggregated by project site Consolidated (N=309) Kalemie (N=159) Moba (N=150) # of Behaviors Frequency % Cumm Frequency % Cumm Frequency % Cumm 0 6 1.9% 100.0% 3 1.9% 100.0% 3 2.0% 100.0% 1 16 5.2% 98.1% 7 4.4% 98.1% 9 6.0% 98.0% 2 82 26.5% 92.9% 35 22.0% 93.7% 47 31.3% 92.0% 3 134 43.4% 66.3% 65 40.9% 71.7% 69 46.0% 60.7% 4 66 21.4% 23.0% 46 28.9% 30.8% 20 13.3% 14.7% 5 5 1.6% 1.6% 3 1.9% 1.9% 2 1.3% 1.3% Impact Indicators:  Percentage of underweight (WAZ<-2) beneficiary children aged 0-23m (FFP)  Percentage of underweight (WAZ<-2) children aged 0-59m (FFP)  Percentage of stunted (HAZ<-2) children aged 6-59m (FFP) Anthropomorphic measurements were randomly taken of children between the ages of 0 to 59 months to determine their nutritional status. 606 children between those ages were weighed and their length/height were measured by survey staff. Weight-for-Age (WAZ) Z-scores for the entire universe ranged from +4.93 to -27.78, with an average WAZ score for the entire universe of sampled children between the age of 0 to 59.9 months of -1.56 (prior to cleaning outliers). Height-for-Age (HAZ) scores for the same age range ran from +7.9 to -55.87 with an average Z￾score of -1.63. To eliminate distortions due to weighing and measuring errors, data was cleaned to purge outliers at the extremes using 2006 WHO recommended cut-off levels. Samples that resulted in WAZ scores above +5 and below -6 and HAZ scores of more than +6 and less than -6 were excluded. The following were the number of outliers eliminated for the purpose of analysis: Table 44: Outliers purged from data tabulated WAZ Outliers Number Purged HAZ Outliers Number Purged Greater than +5 0 Greater than +6 3 Less than -6 5 Less than -6 1 4.2.20 Percentage of underweight (WAZ<-2) beneficiary children aged 0-23m (FFP) The project has set out to reduce infant and early childhood malnutrition from a baseline level of 29.2% of children between the ages of 0 to 23 months to 26% by the end of Year II (the time of this evaluation) and 21% by the end of the project. Anthropomorphic measurements taken of children during the course of the evaluation survey, indicate that Global Acute Malnutrition (GAM), evidenced by Z-scores of <-2 have worsened to 37.6% (CI: 30% - 45%), with an 60 average Z-score for the group of -2.89. Moderate malnutrition (-3 < WAZ < -2) was seen in 22.2% of children while extreme levels (WAZ < -3) were found in 15.4%. Table 45: Percentage of underweight (WAZ<-2) beneficiary children aged 0-23m (FFP) – Categorized by GAM, Moderate Malnutrition and Extreme Malnutrition Weight for Age Z-Score Categories All Children Percentage/(Frequency) (N=306 ) Baseline Percentage GAM (WAZ < -2) 37.6% / (115) CI: 30% - 45% 29.2% CI: 21.9%-36.5% Moderate (-3 < WAZ < -2) 22.2% / ( 68 ) 16.1% Extreme (WAZ < -3) 15.4% / ( 47) 13.1% Table 46: GAM (WAZ < -2) by Age and Sex Categories (children aged 0-23m) Age Categories (Months) Males (Percentage) Females (Percentage) 0 – 5.9 5.6% 4.9% 6 – 11.9 36.1% 21.6% 12 – 17.9 60.4% 57.1% 18 – 23.9 52.2% 79.2% Of note is the considerable spike in GAM among both boys and girls in the 12-17.9m and 18- 23.9m age categories. It would seem that this spike is associated with mothers’/care-givers’ current weaning practices, exacerbated by a recent poor harvest. The nutritional status of male and female newborns (0-5.9m) are comparable, though 6-11.9 month old males appear to be in a poorer state than their female cohorts. By the time children reach 18-23 months of age, the pattern reverses. It was not within the scope of the evaluation to determine possible reasons for these patterns. It will, however, be advisable that FH health staff and field promoters monitor and assess this situation more closely and with more rigor to determine how best to improve nutritional status among those gender groups during their physical growth cycles. To date, the project has been unable to achieve impact in improving the nutritional status of infants and young children as measured by WAZ scores. While it could be expected that significant improvements in this impact indicator would not be seen after just about 1.5 years of effective project implementation, a worsening nutritional situation should be cause for concern and investigated to trace the source of the setback. In all likelihood, recent poor rains/harvests in the region have had a detrimental effect on household livelihoods and food consumption. Nevertheless, feeding practices and increases in illness should not be discounted. 4.2.21 Percentage of underweight (WAZ<-2) children aged 0-59m (FFP) 35.6% of children tracked during the survey demonstrate global acute malnutrition – GAM - ( Z￾scores < -2). Rather than improve from the time of the baseline, the nutritional status of the group as a whole appears to have declined. 61 Table 47: Percentage of underweight (WAZ<-2) beneficiary children aged 0-59.9 m consolidated and weighted (FFP) Weight for Age Z-Score Categories All Children Percentage/(Frequency) (N=601 ) Baseline Percentage GAM (WAZ < -2) 35.6% / (216) 28.9% Moderate (-3 < WAZ < -2) 21.6% / (129 ) 16.8% Extreme (WAZ < -3) 14.8% / ( 88) 12.1% Because this data has been weighted on the basis of an equation, no confidence intervals have been calculated. The level of extreme malnutrition over 10% should be viewed with concern and consideration for whether therapeutic feeding interventions presently in place are adequate to address, what appears to be, a worsening situation. Nutritional levels between the two age bands rests at a relatively tight range. However, the level of deterioration of the 24-59.9 m age groups appears to have deteriorated by a modestly lower number of percentage points. Nevertheless, the deterioration is troubling and should be investigated further and with some urgency. Table 48: Percentage of GAM (WAZ<-2) beneficiary children aged 0-23.9 m and 24-59.9 m with Confidence Intervals (FFP) Age Groups Evaluation Survey Percentage Baseline Percentage 0-23.9 m (N=306) 37.6% / (115) (CI: 29.9-45.3) 29.2% (CI: 21.9-36.5) 24-59.9 m (295) 34.2% / (101 ) (CI: 26.6-41.9) 28.5% (CI: 21.3-35.8) Nutritional deterioration was noted in all age categories and among both sexes. As was the case at the time of the baseline, infants (0-5.9m) who were, for the most part, being exclusively breast-fed demonstrated a considerably fewer cases of GAM. Spiking, which occurred after six months and especially 12-17.9 months probably relates to inappropriate weaning practices carried out by mothers. 62 Table 49: Percentage of children demonstrating GAM (WAZ < -2) by Age and Sex Categories (Children aged 0-59 m) Age Categories (Months) Males Females 0 – 5.9 5.6% 4.9% 6 – 11.9 36.1% 21.6% 12 – 17.9 60.4% 57.1% 18 – 23.9 52.2% 79.2% 24 – 29.9 39.4% 39.3% 30 – 35.9 28.6% 23.5% 36 – 41.9 33.3% 36% 42 – 47.9 28.6% 55.6% 48 – 53.9 42.1% 26.1% 54 – 59.9 30.8% 35.7% In light of the above, to date the project has been unable to achieve impact in improving the nutritional status of young children as measured by WAZ scores. 4.2.22 Percentage of stunted (HAZ<-2) children aged 6-59m (FFP) As in the case of WAZ, Height-for-Age (HAZ) measurements revealed similar deterioration, though to a less significant extent that that demonstrated in WAZ. That said, stunting is a evolutionary problem , whose effects are manifested well beyond their onset. In this light, it would have been unlikely from the onset that significant impact could be obtained as a direct result of just 1.5 months of effective implementation. Consequently, the project has been unable to achieve its target of reducing the percentage of children who are stunted to 37%. Table 50: Percentage of stunted (HAZ<-2) beneficiary children aged 6-59 m (FFP) – consolidated and weighted Height for Age Z-Score Categories All Children Percentage/(Frequency) (N=525 ) Baseline Percentage Global (HAZ < -2) 46.1% / (236) 43.5% Moderate (-3 < HAZ < -2) 25.8% / (133 ) 27.0% Extreme (HAZ < -3) 20.3% / ( 103) 16.5% 63 Table 51: Percentage of children demonstrating Global Stunting (HAZ < -2) by Age and Sex Categories (Children aged 6-59 m) Age Categories (Months) Males Females 6 – 11.9 36.1% 22.4% 12 – 17.9 54.7% 27.9% 18 – 23.9 52.2% 45.8% 24 – 29.9 51.5% 42.9% 30 – 35.9 53.6% 47.1% 36 – 41.9 61.5% 40% 42 – 47.9 42.9% 57.9% 48 – 53.9 55% 43.5% 54 – 59.9 38.5% 57.1% 4.3 Cross-cutting Indicators : Improved Human Capabilities of Vulnerable Households CC Indicator 1: Community Capacity Output Indicators:  Number of assisted communities with improved community capacity as a result of MYAP assistance  Percentage of Community Development Committees (CDCs) whose members/beneficiaries are women  Number of CDCs trained in early warning systems 4.3.1 Number of assisted communities with improved community capacity as a result of MYAP assistance The project has effectively promoted the concept and structure of Community Development Committees (CDC) as a mechanism for participatory planning and organization of community development initiatives. The anticipated target of 28 CDCs has been slightly exceeded with the formation of 29 CDCs. Each CDC brings together neighboring villages that are situated a reasonable distance from each other. On average each CDC is comprised of 14 respected members chosen from all of the participating communities on the basis of their leadership qualities and abilities. The presidents and other CDC representatives who participated in health and agriculture focus groups provided a positive impression of being engaged and concerned with the development of the communities which the CDCs oversee. 64 4.3.2 Percentage of Community Development Committees (CDCs) whose members/beneficiaries are women Of the 401 CDC members of record, 146 are women, representing a 36% participation rate. Though slightly short of the Year II target of 40% and the LOA target of 45%, women’s participation is still respectable, given the time normally required to change deeply entrenched gender attitudes and practices and the relatively short period of project execution. The scope of the evaluation and the short time available for field work did not permit an assessment of the inner workings of the CDCs and the quality of participation of individual members. Just as important as hitting a numerical target, the project should continue to work toward strengthening the workings of the CDCs and creating a quality participatory experience for the members, particularly female members. 4.3.3 Number of CDCs trained in early warning systems (EWS) CDCs have not been trained in EWSs. At the time of the evaluation a framework for an EWS adapted to the particular realities of the region had not been developed by FH, though staff informed the evaluator that it is being developed. Without a framework, training is premature. In the greater scheme of things, an EWS represents a highly deductive approach to addressing the livelihoods needs of community members. They already have too little time and too few resources to plant, produce and care for their families. Many participants in focus groups voiced a desire to plant more acreage in staple and cash crops in order to improve their families’ livelihoods if they could obtain the technology and resources to do so. From the “inductive” approach practiced by the rural poor, it is hard to imagine that investing the time and effort to implement an EWS would be a higher priority that producing more and caring better for family members. This is not to say that an EWS is not important. It must, however, be grounded in the real “here-and-now existence” that community members face as they struggle to overcome malnourishment and meet their general livelihood needs on a sustainable basis. In light of the above and given the amount of work already on the plates of field staff, it is advisable to concentrate the rest of the limited time left in the current MYAP and any follow-up project to consolidate and strengthen the basic workings of the CDCs and take the first steps toward actual planning and implementing quick impacting community development plans. EWSs should be rooted in the operational capacities of FH, like-minded NGOs, international organizations and host-government line ministries in the immediate future. Even then, the resources will have to be available for them to take on the task while still implementing the array of programming which must still be carried out for the foreseeable future. Effect (Results) Indicators:  Percentage of CDCs that have a long-range community development plan 65  Number of assisted communities with disaster early warning and response systems in place as a result of project assistance 4.3.4 Percentage of CDCs that have a long-range community development plan So far no CDCs have developed and put into practice formal community development plans. Members of CDCs who participated in focus groups made reference to a variety of needs, but none were expressed in a structured manner which resembles a plan. Starting in June, CDC members were trained in Participatory Rural Appraisal (PRA). Up until the time of the evaluation six CDCs (3 each in Moba and Kalemie) had carried out that exercise with project support using the array of techniques associated with PRA and are now in the process of structuring a plan. At this pace, the project may not achieve its target of helping 21 (75%) CDCs establish community development plans, barring a major push in the final year of implementation. That said, all concerned need to remember that the process of community development planning is “organic” and time/resource consuming. Once again, just as important as the number of plans is the quality of those plans based on their technical feasibility, economic viability, prospect of sustainability and “fundability” in the real context in which the beneficiaries live and work. Impact Indicator: 4.3.5 Percentage of CDCs that used Early Warning data in the past year to assess and analyze Food Security Risks As of the close of the second year of the MYAP, this impact has not been achieved is not likely to be achieved during the year that is left in the current plan. Time and resources should be focused on maximizing achievements in more basic elements of livelihoods and community development that will impact the lives of beneficiaries in the short-to-medium term. Given the precarious state of livelihoods and its effect on the health and well-being of households, there is still much work to be done (and investments to be made) in raising household revenues, improving the utilization of those revenues to achieve improved health, nutrition and education, as well as increasing the coverage of households reached and which benefit from those gains. 4.4 Mission-required Final Indicators : Improved Human Capabilities of Vulnerable Households SO 1: Improved Livelihood Capacities of Vulnerable Households 4.4.1 F Indicator 1: Number of producer organizations, water user associations, trade and business associations, and community-based organizations receiving USG assistance The project has channeled financial, material and training/technical assistance to 445 formal organizations categorized as follows: 66  FFLGs - 200  Village Water Committees - 28  CDCs - 29  Care Groups - 188 The afore-mentioned organizations represent a 99% achievement of the target set forth in the MYAP through Year II. Beyond these organizations, there is growing interest among farmers in forming FFLGs in order to establish seed/cutting and livestock multiplication groups. Further growth in the number of such organizations that can be supported by the current MYAP and any further program will dependent on the amount of resources available to FH to allow it to implement larger scaled training, follow-up and material support/capitalization. Based on an organic growth model, growth will be gradual, resulting in only a limited number of new FFLGs coming on board. At this point it is advisable to concentrate time, effort and resources on strengthening the organizations already supported, unless significantly increased resources become available. During the course of focus group discussions, representatives of these organizations demonstrated commitment sustaining the work that they are presently carried out for the value that it represents to their communities. While all vowed to continue the work of their organizations regardless of the availability of future external assistance, they did stress that their need for training, technical support and capital infusions still exists in order to consolidate the gains they have made so far. 4.4.2 F Indicator 2: Number of individuals who have received USG-supported long-term agriculture sector productivity training Through Year II, 6, 998 farmers have received one or more types of training and material support designed to improve their productivity. 3,000 members of FFLGs have been trained in a number of productivity measures (described previously). Likewise, 2,998 farmers heading vulnerable households benefited from improved varieties of seeds through seed fairs. This represents a more than 79% achievement of the MYAP’s LOA target after the second year of implementation. 4.4.3 F Indicator 3: Number of people trained in child health and nutrition through USG￾supported health programs 14,877 persons have been trained through Year II of the MYAP – a 99% rate of achievement of the anticipated target of 15,000. Overwhelmingly, the caregivers who have benefited have been women, with just 5 men listed as having attended sessions. The project has been very effective at getting health and nutrition messages to all mothers and caregivers of children between the ages of 0-5 years of age. 67 In the face of absenteeism the project team and MLs have had to make extra efforts to “back fill” training sessions with individualized sessions to be sure that all beneficiaries receive the full array of training provided. While not the easiest road, it is proving to be the more effective one. Overall Program Impact 4.4.4 F Indicator 4: Number of rural households benefiting directly from USG interventions The MYAP has touched the lives of 26,905 households with goods and services (training) designed to improve their lives and livelihoods. The breakdown of households by the types of benefits was:  Water and Sanitation Infrastructure 3,637  Agriculture Seeds, Tools and Seed Protection 5,391 Food Assistance  Agriculture Training and Technical Assistance (FFLGs) 3,000  Health, Hygiene and Nutrition Training, and Nutrition 14,877 Monitoring This universe of beneficiaries represents a significant degree of coverage of the area households in the areas where the MYAP is being implemented. The MYAP has established significant coverage of the villages in the health zones in which it operates within the Kalemie and Moba districts (211 out of 211 villages). However, the two components, which should be working in tandem to achieve intended impact, do not actually operate together in all villages. The health, hygiene and nutrition component operates in 179 out of 213 villages (84%), while the livelihood component operates in 117 out of 213 (55%). There is convergence of programming in only 85 villages (40% of the universe). This means that the MYAP cannot develop the synergies between those two components in 59% of villages (126). The health component is implemented in 94 villages without the presence of livelihoods, while the livelihoods component is being implemented in 32 villages without the presence of Health. In terms of households which benefit, the health, hygiene and nutrition component is designed to reach all households which have a child of between the ages of 0-23.9 months. The 14,877 LMs and beneficiary mothers who have been touched by the project represent approximately 67% of the estimated number of households in the villages served. Needless to say, not all households in the universe have children in the age group assisted by the project, so the percentage of coverage will actually be higher. The agricultural and livestock training and technical assistance component beneficiaries who belong to FFLGs represent approximately 18% of the households in the villages in which the component operates. Similarly, seeds, tools and seed protection food assistance beneficiaries (considered the most vulnerable) represent 32% of households in those villages. 68 The 48 villages which benefited from expanded availability of protected/potable water sources represent 27% of the villages in which the MYAP operates and 23% of the total universe of villages in the health zones served. Some Comments on Survey Data Limitations As in the case of practically all field surveys of beneficiaries at the village level, data collection and analysis required special attention to the quality of data that was obtained. Cases if questionable responses involved, in some way, cases of limitations in the capacity of respondents to adequately comprehend the questions posed or in their capacity of accurate recall and the inability of some survey takers to identify certain instances of questionable responses at the very time of the survey. Raw data was reviewed to isolate highly improbable responses and outliers were purged during the process of tabulation. Such cases were, by no means, inordinately frequent, and tended to occur in the agricultural survey related to crop yields. Some limitations were as follows:  Missing data was present in both agriculture and health surveys. After the fact, it cannot be determined what element of the question was not understood by respondents or why survey takers were unable to provide further clarification of questions so that respondents could provide an answer. In any case, unanswered questions were usually excluded from the analysis, unless their presence needed to be accounted for in an analysis calculation. For this reason, some of the percentages representing the answers to the questions from the survey may not total 100%. This is also the reason behind inconsistent denominators in the analysis calculations.  Even though they received refresher training, the enumerators and assistants, measuring heights and lengths were, nevertheless not expert and consistent measurements were not always obtained. The anthropometric data was reviewed and corrected prior to tabulation. Tabulated data was reviewed to ensure that there were no systematic rounding errors. Outliers were removed from the analysis, and often the outlying data appeared to be length or height measurements.  High variability in crop yields calculated from the baseline data, in addition to a number of improbable and impossible yields based on farmer provided data, indicate that errors in quantitative estimates have likely affected the quality of the results in estimating crop yields. This result is not surprising. A number of studies indicate that farmer recall for estimating crop production is reasonably reliable, but that farmer-provided yield estimates are often less reliable. Also, there may have been confusion between enumerators and respondents over the units of measure used for the area of land under cultivation and how those units were identified on the survey forms. While enumerators were instructed to obtain land area data in the customary unit of measure of the area (“Ares” – equal to 0.01 hectare), some amounts of land under cultivation recorded on survey forms are unlikely to have been reported as Ares, but rather directly in hectares, 69 resulting in a 99% reduction of reported area and a spike in calculated yield beyond probable levels. Other cases may have involved the inverse, perhaps resulting in highly improbable low yields.  Data analysis and entry (including, in some cases where it was feasible, rectification back in the field) took considerably longer than anticipated. Final data sets were not completely ready by the time the consultant departed the field and were forwarded to him after his arrival back in the US. Clarifications after that were conducted virtually by email. 5. Summary of Findings - By Strategic Objective Strategic Objective 1: Improved Livelihoods Capacities of Vulnerable Households I.R. 1.1: Increased Agricultural Production 1. The scope and sequence of training foreseen in the MYAP has been carried out on schedule and benefiting the anticipated number of farmers who have organized into the planned number of FFLGs. Key improved agricultural practices and techniques are being transferred to farmers. 2. The strategy of “open training” for all farmers who are interested in attending training events (not just FFLG members) is being taken advantage of by farmers outside the immediate universe of farmers belonging to FFLGs. In addition, FFLG members are willingly sharing what they have learned with inquisitive friends, family and neighbors who have expressed interest. 3. The training seems to be paying off as indicated by the number of farmers employing improved practices/techniques and the number that they are employing. Overall, more farmers are employing more practices than at the time of the MYAP baseline. For the most part, more FFLG members employ more practices than their non-FFLG cohorts. However, “open training” has broadened success in this regard. 4. The provision of quality seed to vulnerable farmers and Seed Protection food rations has effectively reached the targeted number of farmers and their families. The impact of that assistance has probably been diminished by the fact that only 74% of recipients planted at least 70% of their allocated seed. If, indeed, household food production has been so impacted by poor harvests to create a food deficit for a period considerably greater than three months (for which normal coping mechanisms may be adequate to bridge), then the 3-month SPR may have to be re-thought in terms of its adequacy in terms of length of provision. At that point, other modalities such as food-for-work/cash-for-work might be more appropriate. 5. The creation and training of seed/cuttings re-production associations is on track. Technically, the associations appear to be employing improved means of production, according to FH technical guidelines. However, adverse growing conditions associated 70 with poor rains have kept yield lower than normal and anticipated. After making provisions for new planting and allocating seed stock for the creation of at least one new producer association, the amount of seed available for sale to the general population will be limited until yields return to normal. This will result in slower “organic” growth of beneficiaries of get access to improved varieties of seed. 6. Impact in the form of increased agricultural yields has yet to be realized, due to adverse growing conditions during the recent agricultural cycle. While the evaluation survey did not reveal improvement in yields, but rather significant drops, there are too many uncertainties and factors that make survey data on crop yields unreliable to use as an indicator compared to other direct means of observation and measurement. 7. As in agriculture, the creation and training of small livestock reproducers associations is on track. Livestock producers appear to be learning improved practices/techniques and employing them within their association production schemes. 8. Whereas, goats appear to be reproducing at a rate to provide farmers with stock with which to build their own herds, that of ducks has been very disappointing. With flocks not growing for unexplained reasons, ducks seem to be less viable as a productive means for farm families. I.R. 1.2: Improved Natural Resource Base 1. Due to monetization short-falls, investments in soil and water conservation infrastructure have been seriously curtailed. In response, MYAP resources have been concentrated in training focused on improved practices and techniques which could contribute to improved farm productivity. Training is being effectively provided to FFLG members as well as community members interested in attending. 2. Farmers are learning improved practices and greater numbers of farmers are employing more practices then at the time of the baseline, attesting to the efficacy of awareness￾building and training. Based on farmers’ self-reporting through the evaluation survey, 55 hectares have been subjected to various forms of soil and water conservation practices, exceeding the MYAP target. How this translates into productive impact, however, may not be so evident during the life of this MYAP. To achieve impact, a level of effective coverage must be achieved at both the farm and community/area levels, neither of which have been achieved at this time. I.R. 1.3: Improved Market Linkages 1. The simple, but most important output that the MYAP has produced with this I.R. has been the placement and maintenance of market price bulletin boards outside of all markets operating in the project zone. While only 28% of survey respondents acknowledged receiving market information from market bulletin boards, this level represents a absolute increase from zero before the initiation of the MYAP. On the other 71 hand, all focus group participants acknowledged being aware of the bulletin boards and using the information prior to their buying or selling activities. 2. Regardless of the precise level of usage, it is clear that the MYAP has impacted positively in contributing to the market knowledge base of farmers. 3. Even more important to farmers’ market decisions seems to be their limitations in terms of post-harvest storage. Having very limited capacity, many farmers acknowledge having to sell a large portion of their crops right after harvest, for fear of loss during poor storage. As a result, they are unable to take advantage of natural market fluctuations for their own benefit by selling when prices are more advantageous for them. Overall Impact of SO 1 The evaluation is unable to find evidence that the MYAP has achieved the achieve the impact set forth by the project’s indicators. In that regard, 1.5 years of effective implementation are too short a time to realistically observe and measure such change. 1. Neither livestock nor seed reproduction associations have begun generating revenue that will benefit their members. Instead of revenue in the initial year or two, associations will be distributing in-kind “dividends” to their members as improved start-up stock. That and eventual donations of start-up seeds/cuttings and animals to new producer associations should be considered as impact for current and future beneficiaries just as much as the initial indicator of a revenue stream. 2. The MYAP has not been able to increase the number of months of adequate household food provision. Over the first 2 years, the average has remained static at approximately 8 months. This should not be viewed as unexpected given the various external variables beyond the scope of the MYAP that influence harvest levels. 3. Improvements in Household Dietary Diversity Scores have been recorded by the survey of mothers/caregivers. The mean score has risen from a baseline level of 3.43 to 5.93. Increased number of families reporting consumption of tubers/roots, fish, oils and legumes contributed to the gain. This positive impact should be credited as much, if not more, to enhanced awareness on the part of mothers/caregivers as to the importance of improved diet in the welfare of the household and training in the value of various types of foods. Nevertheless, improved productive capacity has a role to play in decisions involving dietary diversity. Strategic Objective 2: Improve human capabilities of vulnerable households I.R. 2.1: Improved practice of Essential Nutrition Actions (ENA) by pregnant women and mothers of young children 1. The MYAP has been successful in identifying women to act as MLs, organize them into functional Care Groups and provide them with the necessary training and motivation to be effective leaders and trainers at the grassroots. 72 2. The MYAP has been equally successful in identifying and reaching out to all mothers of young children and expectant mothers to incorporate them into care groups for promotion, training, nutrition screening and follow-up by MLs. Coverage of the universe of women in this category is virtually 100% 3. The evaluation survey revealed that notable improvements have been achieved in the practice of ENA by mothers and caregivers. The percentage of children receiving Vitamin A supplements, being breastfed immediately after birth and exclusively during the first the first 6 months of life increased from baseline levels. Also, more mothers are continuing to breastfeed their children longer while improving weaning practices which incorporate more age-appropriate dietary diversity from 6 to 23.9 months. I.R. 2.2: Improved mothers’ ability to prevent, diagnose and manage common childhood diseases that exacerbate malnutrition 1. The MYAP has achieved its targets of assisting the MOH in organizing Child Days in all of the villages served by the MYAP. The de-worming treatment and Vitamin A supplementation provided by FH with MYAP resources provided close to 100% coverage of all children in those villages. 2. While the percentage of women meeting bi-weekly or more frequently with health promoters (54.4%) fell below the MYAP’s target of 70% it represented a major improvement from the 3.3% baseline rate and a credible advance toward the MYAP’s target in the face of very real labor and social obligations of rural women. To their credit, FH staff have built in training redundancies to try to achieve as high a coverage of training topics as possible. This has required a sacrifice of time and effort on the part of staff. 3. The MYAP has amply exceeded its target of increasing awareness and knowledge of at least three signs of childhood illness. The 72.6% rate by the end of year two, compares favorably to the 40% set at the start of the MYAP and the 21.7% baseline rate. 4. Though not formally listed as MYAP indicators, the project has achieved notable advances in percentages of children receiving de-worming treatment, proper oral rehydration therapy at home and in the use of ITBNs. Levels of awareness/knowledge with regard to HIV/AIDS have also shown improvements. The coverage of pregnant women who have received some level of pre-natal care reached 100% (up from an already high 98.1%). Frequency of care improved ever so slightly in the process. I.R. 2.3: Improved access to clean water, sanitation facilities, and of essential hygiene behaviors (EHB) 1. The project has successfully met its targets for establishing water and sanitation infrastructure in key villages. When the number of villages that have benefited from this infrastructure is viewed as a percentage of all villages served by the MYAP, coverage is still limited. Only 26.8% of villages benefited from an additional water source, while 73 only 15.6% received improved public latrines. Admittedly, there are some among the 179 villages that have adequate water and sanitation infrastructure, thus requiring no further assistance. However, there are still likely communities which are under-served with potable sources of water and adequate public sanitation infrastructure. This is not a reflection of FH performance. Rather the shortfall reflects the limited resources available for investment in such basic infrastructure. 2. More households now report year-round access to improved water sources within 400 meters of their house compared to the baseline. The percentage of households reporting access has risen to 67.6% from a baseline of 47.4. Of all households acknowledging access to an improved water source, 93% of those are within 400 meters. The 48 village water sources constructed by the MYAP likely contributed to higher accessibility levels. However, Moba households report significantly higher access levels than their cohorts in Kalemie (88% vs. 48.4%). 3. The MYAP has been successful in creating awareness among mothers and caregivers with regard to the importance of adopting improved hygiene behaviors. The evaluation survey revealed that 66.3% of mothers/caregivers demonstrate knowledge of at least 3 hygiene behaviors, compared with 31.4% at the time of the baseline. While POU treatment of water has increased somewhat, the practice has not yet reached even 50%, leaving considerable work to be done. The very low rate of practice in Kalemie must be evaluated further and steps taken to bring it more in line with that of Moba. While the practice of hand-washing has shown improvement since the onset of the project a troubling low percentage (less than 10%) actually practice it. Overall Impact of SO 2 Despite the MYAP’s achievements in outputs and certain effects, the forecast impact on the nutritional status of children 0-23.9 months (and the extended group of 24-59.9 months) has not been achieved. This might be expected from a project which has been implemented for a relatively short time. Troubling, however, is the worsening of the nutritional status of children as measured by WAZ and HAZ. The project team should study further and carefully the factors that are contributing to this decline. Climate change and poor harvests should be considered as a primary factor to be considered. Also, the fact that activities associated with SO1 are carried out in 85 out of 179 villages where SO2 activities are carried out (slightly less than half), means that a large portion of the household universe are not being provided knowledge, technical assistance and investment assistance to take measures to improve productivity and mitigate some of the effects of climate variation. The nutritional survey data should be further cross-checked against agriculture data to determine if there are any correlations. 74 Cross-cutting Indicators CC.I 1: Community Capacity 1. The MYAP has been successful in setting up a CDC structure in support of the communities it serves, with the number of CDCs meeting anticipated targets. Female participation has not quite reached the targeted 45% (presently 36%), but advances are on track, evidenced by the 7 percentage point increase from Year I to Year II. With continued emphasis and the important role that Care Groups are playing in the lives and livelihoods of communities, the target may still be achieved by the end of the MYAP. As important as hitting the numerical target, however, is the MYAP’s success is promoting quality/effective participation by all members of the CDCs, especially women. 2. Early Warning has not be a major emphasis of work with the CDCs and is not be practiced at the CDC level. Given the time and effort required in not only establishing community development organizations like the CDC, but strengthening its performance, it should be of no surprise that little time has been left for introducing such a complex task on fledgling organizations so early in their development cycle 3. While CDCs have yet to establish any development plans, work is underway and the first six may have them within the next few months. Further monitoring of those plans should determine if the slow progress has resulted in quality plans that bear fruit. CC.I 2: Gender Mainstreaming The MYAP has made credible advances in promoting and establishing a greater role for women in the development processes of their communities. The important role that MLs play and the respect that they have garnered in their communities have made them key actors. The role women play in CDCs has already been addressed above. Women currently participate as members of FFLGs, though only as a small percentage. However, they are free, and do, attend training events, as well as work side-by-side with their husbands in the fields practicing techniques learned. 6. Qualitative Evaluation Findings Agriculture and Livestock Beneficiaries 1. Overwhelmingly, focus group participants were grateful for and satisfied with the training component of the MYAP. As a group, they were able to recall a wide array of farming and animal husbandry techniques, which they said that they practiced on seed/cutting multiplication plots and with their animals for reproduction. Similarly, many reported that they have started to practice some techniques on their own land. 75 2. Beneficiaries of SPRs acknowledged that in Year II they have received their full 3-month ration on schedule. However, they noted that the size of the ration was not enough to sustain their families for an entire month. All confirmed that they did set aside seed from their most recent harvest for next year’s planting. Their needs for produce for consumption and sale, in the face of poor harvests, has limited the amount that they could save for planting. Therefore, respondents claim that they will not be able to plant all of the land available to them next season. 3. Seed Fair participants were satisfied with the organization of the events, and the quality of seed available. While they noted that prices were slightly higher than prices found in markets in the area, they did come around to acknowledge that the quality of seed was probably better overall and that merchants were justified in asking for a higher price to compensate having to transport stock right to their villages. 4. Seed Fair beneficiaries reported that the value of their vouchers was not enough for them to purchase all of the types of seeds which they wanted. Cash crop seeds such as peanuts and bean was highly sought, but was also more expensive. Consequently farmers had to make decisions about whether to purchase less of a “high value” seed or obtain more basic seed, such as maize, at a lower cost. 5. Members of seed and cuttings multiplier associations expressed satisfaction with the training and technical assistance that they have been receiving. They are confident that the techniques and the quality of seed/cutting stock that they are employing in their association fields are superior to that which they have traditionally planted in their own fields. However, all groups warned that the recent poor rains have negatively affected crop development, leading to lower than expected harvests. Nevertheless, farmers are conscious of the benefits of the improved varieties of seed/cuttings. In particular, cassava reproducers acknowledged that they view the variety that they have planted as more yielding than traditional varieties and appears to be resisting cassava mosaic. 6. Associations that reproduce small livestock (goats and ducks) have experienced mixed results. Reproduction and the growth of goat herds have been good (65.8% births vs. 7% mortality). Ducks, however, are experiencing serious problems of mortality which was over 50% for ducks received by associations in the first year of the project. As a result flocks of 3 out of 4 associations have dropped in number and one has increased by 8 animals (53%). 7. The first cycle of goats will be old enough to separate from their mothers and to be distributed among the members as their first “dividend”. While the associations have made a noble commitment to provide offspring to form new producer associations, as noted earlier in this report, there is little likelihood that natural rates of herd growth will be sufficient to make good on such commitment for the next few years, until all members have received as dividends at least one pair of animals for their own start-up herds. 8. One focus group noted that while they received training in the construction of proper pens for keeping their animals safe, the members are limited by their inability to build 76 sturdy pens to keep their goats. They claimed that they lacked material. Given that only one group brought up a problem of this nature, such a claim may have represented “fishing” for assistance. Most groups did, however, site a problem with finding grazing land close to their villages. They note some tension with neighboring farmers over alleged damage their herds are causing to neighbor’s crops. A few have taken steps to take their herds farther out into the countryside where there is more room to roam and less risk of conflict. 9. Duck reproduction associations are experiencing more serious problems of mortality. Some admitted that they would like to switch to goat-raising, which they view as more economically viable and less prone to risk. While no one is sure of what is causing the levels of mortality, some speculate that it may be the result of the transition from free￾range to penned animals. Along that line, the animals be under stress as a result of a change in diet during the transition. 10. FFLGs engaged in agro-forestry (tree nurseries) are satisfied with the training provided them and the results of their labor. They acknowledge the production has been good and that they have distributed seedlings free-of-charge for planting in public locations around their villages. A few, but definitely not a majority, have planted trees in and around their own fields but not extensively. Unfortunately, extensive planting in and around the fields of farmers who are not members of agro-forestry FFLGs has still not taken off, remaining something to work toward. Consequently, associations are not yet selling seedlings to generate revenue. One FFLG noted that neighbors are starting to take an interest in agro￾forestry and have signaled a desire to obtain seedlings for their fields. Where public interest seems to be growing is in fruit trees, and FFLGs wish to expand their production to include them for distribution/sale. 11. In general, members of FFLGs demonstrated enthusiasm and recall/knowledge of appropriate production techniques. The offer an impression that their confidence in working together on productive endeavors is growing with experience (a valuable achievement in a region which is recovering from violence and conflict). 12. FFLG members noted that friends and neighbors approach them spontaneously for information on what they are doing and how they are doing it. This is a confidence booster and a sign that they are achieving observable results. The members willingly and freely share their knowledge with anyone who inquires. Training events are open to anyone who has interest. FFLG members and field promoters believe that there is strong demand for training and program services among the communities at large, based on the number of inquiries about how to form an FFLG and requests for training. 13. Virtually all participants in focus groups acknowledged consulting FH Market Price Bulletin Boards posted outside of village markets. They confirm that the information is up-to-date and useful to both buyers and sellers. 14. Though FFLG members have been trained in appropriate grain storage techniques, many signaled in the focus groups that they lack grain storage capacity in their homes. While 77 quite a few acknowledge practicing “no-cost” techniques to protect their stored grain from pests, they do not construct storage infrastructure to protect produce from the elements and from animals. They acknowledge that they cannot afford the materials required for construction. The lack of effective storage capacity limits the amount of produce households can store for any significant amount of time without risking loss. This governs the amount that needs to be sold immediately after the harvest, limiting the amount that is saved for consumption beyond a few months and the amount that can be sold later in the year when market prices are more favorable. This “chicken-or-egg” situation needs to be sorted out and solutions implemented if households are to both improve their nutritional situation as well as their overall economic livelihoods. 15. FFLG members highly value the training they have received and would like to have written training and technical materials on the practices they have been taught for future reference. Health, Hygiene and Nutrition Beneficiaries 1. MLs have been trained and evidence retention of an array of health, nutrition and hygiene lessons which they have, in turn, provided on a regular basis to beneficiary mothers under their tutelage. During focus group discussions MLs displayed great pride in their learning and enthusiasm for and commitment to their work, despite the time and effort asked of them. 2. The MLs value the tokens of motivation that they receive (T Shirts, badges) as well as training and seeds for Care Group gardens (dual-purpose: demonstration and produce). However, those benefits have often been construed as “payment” or a “benefit” by beneficiary mothers, who are demonstrating frequent signs of envy. This, according to MLs, is making their job harder. Nevertheless, MLs continued to request more assistance (more seeds and tools to expand their CG farming) during focus group discussions, despite the fact that it may further exasperate their problem. While BMs may tend to envy MLs for their perceived “compensation” from the project, they are still respected by BMs and their communities at large. However, that respect does not yet translate into any form of community initiative to recognize MLs’ time and effort with some form of community support. 3. With regard to the above, numerous CG members expressed that vegetable gardening was too labor and water-intensive, given all of their other home and CG responsibilities. Many suggested (sometimes lobbied) switching to cassava or maize cultivation or raising chickens or ducks. Their rationale concerning labor intensity is surely valid and should be duly noted. However, the project team should weigh carefully the proposed alternatives to safeguard against duplicating the work that households are already doing on their own land and which are being supported through FFLGs. CGs and FFLGs should not be in competition, but rather should be complimenting each other in an effort to improve the livelihoods of beneficiary families in a well-rounded manner. 78 4. Beneficiary mothers showed that they have learned an array of health, hygiene, and nutrition lessons which they are able to cite when asked about them. BMs confirm that MLs provide them with regular training, the types of which they do not receive from any other source at present. 5. While knowledge has been imparted and seems to be retained by MLs and BMs, practice lags behind in some notable areas. When asked in focus groups for the reasons for not practicing certain lessons, women invariably cited a lack of resources with which to cover associated costs. 6. While MLs and BMs offered confirmation that they and their family members practice hand-washing on a regular basis, survey findings noted previously tell a different story. One tell-tale indication of difficulties encountered in practicing hand-washing occurred as women noted that they have been unable to construct “Tippy Taps” as per their lessons because they cannot afford the price of the containers (“bidones”) used in the device. The project teams will have to study this apparent problem to determine if the impediment is as large/significant as the women implied or whether their comments during focus groups was a form of “fishing” for assistance. 7. 50% of focus group participants acknowledged using mosquito nets. All also acknowledged that they had received their nets from sources outside of the project and perhaps prior to its advent. In all cases, they said that the parents and the youngest child slept under the nets and that other family members slept in the open. When asked in focus groups how families obtained their bed nets, all who had them acknowledged having received them as part of a humanitarian package or when they gave birth at a health center, as part of government effort to promote attended births at health centers. When asked if they have considered buying more, all replied that they couldn’t afford them. Those who did not use mosquito nets cited that they do not have them because they are “no longer being given out free” and because they cannot afford to buy them. Even though care-givers acknowledged that each time a family member has fallen ill to malaria in recent months they have dispensed from 5,000 – 15,000 Congolese Francs ($5.50 - $16.50) for medication, they could not appreciate the economic value in paying CF 5,000 for a bed net, let alone the value of prevented illness. Clearly, there is considerable work still to be done in the area of linking awareness to action and helping mothers appreciate the economic value of prevention compared to cure. 8. Mothers report having improved the diversity and quality of the food they feed their families (especially children). They report providing their family food from at least 3 food groups. Besides staples such as foufou and cassava greens, a women also report integrating more peanuts, oil and fruit into their meals. Those who do not always eat more nutritious foods (fruit, oil, beans, and soy) cite the cost and lack of means as the reason. 9. Mothers expressed a belief that the health and nutrition of their families (especially children) have improved somewhat since the project began. However, illness and 79 malnutrition still persist. Most acknowledged that malaria, respiratory illness (cough) and gastro-intestinal illness (stomach) have been experienced by someone in most families over the last two of months. Though the survey indicated that nearly 56% of infants have suffered diarrhea during the last two weeks, mothers in the Focus Groups expressed belief that the number of cases have come down in their families. When an open ended question was posed to mothers, diarrhea was rarely cited as a health problem experienced by members of their families. 10. The cost of medical treatment (health center fees and medication) is considered expensive and a burden on household finances. One case was cited of a mother not taking a child with “fever” (malaria) to the health center because she could not afford the fees. Other respondents questioned the benefit of going to a health center that sometimes cannot provide the required treatment. One mother made the poignant comment: “How is it possible that we get free service and medicines when we are not sick, but we have to pay when we are sick.”. This sums up the challenge ahead for this project and subsequent ones with building/increasing awareness of the importance and value of “prevention” as it co-exists with the reality of seeking sustainability and cost-recovery. 11. MLs and BMs confirmed that screening for malnutrition is being carried out regularly and systematically. This seems somewhat at odds with the survey finding that slightly more than 51% of children have had their arms measured in the last four months. MLs expressed belief that the numbers of children suspected of malnutrition has come down from past years. Similarly, this does not appear to agree with survey findings that the percentage of children found to be malnourished has increased since the baseline. Most cases referred to health centers have been confirmed after subsequent weighing, confirming that MLs appear to be correctly carrying out the measurements as they were trained to do. 12. Villages have at least two protected water sources (well or spring) at a reasonable distance from respondents’ homes, but, according to focus group participants, they provide insufficient amounts of water to fully meet demand. All villages have organized water committees to oversee the care and maintenance of the water sources (most established before and outside of the project). Some committees have opted to set up a monthly user fee system, while others have decided to make special assessments of all village households when an expense for maintenance or repair is required. 13. Most households have and use sanitary latrines (except in Masembe and Mamba, where sandy soil affects the durability latrine pits. Mothers report that the latrines are used by all members of the family. They also report that they take recommended appropriate measures to keep the latrines clean and sanitary (though a few mothers claimed that hole covers were too expensive for them to afford, so therefore they do without them). This was confirmed by the findings of the survey. 80 14. Mothers report practicing appropriate ways to keep water potable (clean/covered containers). Some report boiling unprotected water and use of purification tablets or chlorine – however, their availability and cost limit that practice. 15. When questioned about the need to sustain project services to mothers and their children even after the project concludes in any one village (so as to allow the project to serve other communities) MLs committed themselves to continue providing training and screening to BMs and their young, children. They noted that with the training they have received they can continue to carry on their duties. However, MLs felt that they could be more effective if they receive for a while longer continued training to reinforce their knowledge, and, some motivational support (flip charts, T Shirts, badges, and, of course, some productive support). 16. There does not appear to be a problem of ML attrition in the face of the time and effort asked of them, as they still receive small tokens of support from the program. It is hard to say at this time whether the same level of enthusiasm would actually continue without program support. In the opinion of the external consultant, most volunteer-based programs require numerous years of structured follow-up support to “anchor” volunteers into a structured program, lest they revert to the demands of their regular livelihoods. So long as they find that anchor, volunteers tend to stay sufficiently motivated. Key in the case of the CG model developed under this program will be the ability and willingness to the MOH to provide such a structured follow-up and support. Local representatives of the MOH health zones appear to have the will and desire to provide this. However, the financial and management capacity of the MOH is still weak and dependent on external assistance for even basic program activities. 17. Both MLs and BMs expressed an opinion that their communities have moved beyond an expectation of humanitarian assistance and are able and ready to contribute their time and efforts in projects to find long-term solutions to their needs. They believed that training was the primary input that they would require, but were very quick to urge the project to provide more in the way of seeds, tools, and animals. There were also notable cases of BMs asking that the project provide them with “kits” and with “farine” (their way of saying “food aid”) because their families did not have enough to eat. Apparently, in deed, humanitarian dependency is still a factor to be dealt with as the transition continues toward longer-term sustainable community development Cross-cutting Indicators Discussed with Both Sets of Focus Groups Presidents of local CDCs participated in all of the focus groups concerned with agriculture and health. Also some of the FFLG and CG participants also happened to be CDC members. They confirmed that CDCs exist in support of all villages in which FGs were conducted. Presidents voiced the support of CDCs for the activities of FFLGs and CGs. The CDCs stand ready to help resolve any issues that may affect the success of those two structures. 81 CDCs are still consolidating their structures and role in coordinating development efforts in the village. No CDCs reported having a village development plan, though all had a list of projects they would like to implement. It was evident from discussions that Early Warning was not an issue in which they were conversant. Focus Group Discussions with External Stakeholders in Moba and Kalemie The focus group discussion in Kalemie was attended overwhelmingly by GoDRC line ministry representatives and representatives of UN agencies (FAO and UNICEF). In Moba, however, there was a broader participation of organizations, which included representatives of several international and national NGOs. Regional Planning and Coordination 5-Year Regional (“territoire”) development plans have been developed in both Kalemie and Moba – Kalemie’s has just commenced and Moba’s dates back to 2007. In neither case does it appear that the plans have been circulated to all concerned stakeholders or widely “socialized”. Representatives of NGOs were generally uninformed of the plans contexts, while U.N. agencies were better aware. In this light, it appears that development planning is a top-down process among the “big players” and has not benefited from the insights and “ground truthing” of concerned NGOs (local and international) as well as Community-Based Organizations (CBO). UNICEF has been back-stopping the Ministry of Health in its sectoral planning. UNICEF wishes to work toward integrating concerned actors into an inter-agency plan to create a sustainable health response. It wishes to conceptualize a sustainability strategy for when project financing ends and NGOs exit the region. Kalemie’s plan was developed with the technical support of UNDP, but UNDP has just closed its field office in Kalemie. This does not bode well for follow-up and technical assistance in the actual implementation of the plan. Coordination tends to focus on humanitarian response. There is no effective coordination structure for recovery and development programming and interventions. In Kalemie OCHA still implements humanitarian coordination and some attention to early recovery. In some ways, it also offers a latent opportunity for transformation into longer-term development coordination. With UNDP departure, a natural “heir” for the coordination lead is lacking. UNICEF has actively supported the Ministry of Health in developing its sectoral plan within the scope of the regional plan. They may be able to fill the void. In Moba, even humanitarian coordination has suffered with the closure of the OCHA field office. UNHCR has taken over the role as humanitarian lead, but its actions appear to be somewhat “casual”, and focused on their particular operational concerns according to some NGO observers. 82 Refugee returns continue though they are scheduled to wind down by the close of 2010 (returns now average 500 persons per month). In that context, humanitarian funding has begun to diminish and humanitarian-oriented NGOs are concluding their operations. Unfortunately, development financing is not coming on line at the same pace to fill the void. With little in the way of funding, the number of NGOs willing and able to engage in development programming/implementation is still limited. In this light, one could question the operational viability of the regional development plans without donor support and without vibrant NGO community with capacity to implement community development. In Moba, adequate road infrastructure and water for drinking and agriculture still remain needs that are not fully met. Different comments were made along the lines that there is a need for programming that builds more sustainable development of productive capacities. It was observed by local NGOs that they have an important role to play in local development, but are under-supported and under-funded. Agriculture and Livelihoods 1. Some participants expressed the view that the populace still has the tendency to “ask for things” (seeds, tools, animals), when what they should be doing is organizing themselves for investments in development. This represents a hold-over of a humanitarian assistance mentality. Unfortunately, humanitarian responses have tended to be small, isolated interventions which have not been able to generate real impact, 2. What is required for improving livelihoods in the region through agriculture is farmer access to improved seeds and technologies. More importantly, however, programs with a capacity to provide close and consistent follow-up and technical assistance are required. 3. It was noted by the Ministry of Agriculture that current market systems still represent a “buyers’ market”. Farmers are not sufficiently organized, nor do they have the ready means to make it more competitive by taking better advantage of market seasonality. In this regard, the need for more and better produce storage capacity was cited. However, nowhere in the conversation was the issue of household need for liquidity brought up as a reason for farmers’ obligation to sell when the market terms are to their disadvantage. 4. A representative of the Ministry of Agriculture in Moba offered the observation that INGOs working in Food Security do not do so in close cooperation and “synergy” with GoDRC line ministries. 5. FAO announced that it has carried out a “Who Does What Where” of actors working in Kalemie region in the field of agriculture and livestock. Based on participant reactions, it is clear that the results have not been circulated or widely socialized. FAO also noted that they are supporting the promotion of farmers’ organizations with a management responsibility for resources entrusted to them. 6. An ACTED participant in MOBA noted that during the planning and early implementation phase of the MYAP, contact between FH and ACTED technical staff was very close, with fruitful exchanges on strategic orientation, geographic coordination and 83 harmonization of needs assessment tools. Lately, however, the contacts have “cooled off”. He nevertheless felt that there is much to be gained by re-kindling such coordination, perhaps to the extent that it could generate broader coordination among like-minded organizations. 7. A livestock expert with the Ministry of Agriculture and Livestock in Kalemie was aware of FH’s MYAP programming and expressed an opinion that the program may not have followed a “logical path”. In his opinion, the program has provided training and “free aid”, but “the training stops before addressing the task of building farmers’ productive capacity”. Shortly after, the same expert recommended that FH provide more “means” (“moyens”) to beneficiaries. He also suggests that FH may have targeted the wrong interventions to groups which may have actually wanted to do something else. This view, he claims, comes from reports filed by some of his field agents. No one in the discussion remotely shared his views, which could lead one to wonder what agenda lies behind his opinions. 8. A valid recommendation which the same expert brought up, and which is worthy of further analysis, was that FH work to increase access to balanced animal feed which is locally produced. 9. Another representative of the Ministry of Agriculture offered the suggestion that FH establish “more follow-up of farmers in order to build marketing systems and infrastructure”. By that he suggested forming crop transformation enterprises (mills and oil presses) as well as produce refrigeration infrastructure. However, his ideas entailed large-scale, not necessarily community-level infrastructure and enterprises. He recommended increased investment in “storage and conservation” as well an increase in FH geographic coverage to include other another “axe”. The representative was reminded that an organization’s capacity to increase its scale and/or scope of programming depends on its operational capacity, which, in turn, is determined by sufficient available financing. 10. The FAO participant made the interesting suggestion that FH explore possibilities of partnering with WFP on its P4P (Purchase For Progress) program. Such an arrangement could provide an interesting market for certain crops being produced by FFLGs supported by FH. Presently P4P is being implemented with Kabalo with DANCHURCH Aid as a partner. WFP has a current target to procure 4,000 MT of produce (2,500 MT of which is already contracted), which should increase to 9,000 MT in the near future. 11. Numerous organizations are implementing projects/programs which could represent common ground for coordination and the sharing of best practices, as well as a potential for geographic/demographic overlap. o ACTED is providing seeds and farming inputs o CARITAS/CRS is conducting seed fairs and supporting farmers groups in the multiplication of cassava cuttings o GTZ is providing farmers with seeds and farming inputs 84 o FAO is providing farmers with seeds and farming inputs, as well as supporting 5 farmers association in seed multiplication. In the latter case, FAO will begin buying seed from those associations for use in its other assistance programs. With regard to seeds and inputs, the representative noted that beneficiaries have expressed discontent, because they believe that other organizations are offering more generous/advantageous packages of assistance. o The Ministry of Agriculture and Solidarite implement seed multiplication initiatives through producer associations. Also, they are promoting and assisting in the organization of “Comites des Gens du Developpement” (People’s Development Committees) Health, Hygiene and Nutrition 1. A Ministry of Health participant in Kalemie recognized the value of the FH Health and Nutrition Component of the MYAP, particularly the role that MLs and Care Groups are playing in nutritional screening in villages as well as in providing awareness-building and training. He called for FH to consider increasing its coverage to additional health zones. 2. The same participant was of the opinion that emphasizing improved livelihoods (improving production and how it is used at the household level) is key to “breaking the chain of malnutrition and illness”. 3. It was noted that Solidarite is implementing a Health Program along with Livestock and Fishing Development in Nyunzu. However, in the opinion of one participant, the programs only partially cover the population of the Health Zone. It was also noted that awareness-building and health response (mise en charge) have been weak. This comment may have been made to bolster the suggestion made in point 1. 4. In Moba most organizations operating in the health sector are concentrated in the plateau region. Communities along Lake Tanganyika and the northeast are under-served. The Ministry of Health participant also noted that considerable needs still exist in rehabilitating health centers, equipping them and covering their running costs. Of particular interest to the Ministry of Health are interventions to address malaria in the zone. Interventions which involve screening, treatment and prevention (bed nets) are key elements. 5. GTZ’s health programming concentrates only on the immediate needs of returning refugees and the communities they are returning to. CARITAS, too, works in health, but only out of the Central Hospital. They do have several potential projects on the drawing boards which are waiting for funding approval. 6. The Ministry of Health acknowledges the good working relationship with FH and the MYAP. Coordination has been good from the very beginning. The Care Group model is highly valued as an effective channel for providing women and mothers with essential training and nutritional screening. The model, with its reliance on community participation, is viewed as an important contribution to building bridges to communities 85 by way of which certain interventions can be channeled in a sustainable manner. The main question here is whether the MOH has the necessary financial and managerial capacity to take full advantage of the model within the scope of their program. 7. Recommendations In light of the fact that implementation of the third and final year of the MYAP is currently underway, some recommendations can be implemented within the scope and time frame what remains of the current MYAP. Those and others requiring more analysis and planning should be taken into account when developing a possible Phase II of current programming. General Programming Recommendations 1. The next MYAP should continue only in its current geographic locations to consolidate achievements made to date and to expand livelihoods activities to the 94 villages in which health currently operates alone. If resources permit, more FFLGs should be supported in the 117 villages in which livelihoods activities are presently underway to accelerate the expansion of coverage in those communities ahead of the organic growth anticipated in the first phase. 2. The next MYAP should more tightly link health and nutrition programming and implementation with that of livelihoods. In this regard beneficiaries should be tracked in one consolidated data base in order to track achievements in both so as to identify causality relationships between the two (particularly in the direction of livelihoods toward health/nutrition. By tracking both in the same data base, the program should avoid duplication and foster synergies. 3. FH should incorporate into its monitoring system real time assessment of food insecurity among households in the MYAP zone of implementation. Useful data should be drawn from multiple sources: Crop assessments, Direct Observation with regard to food stocks and consumption patterns, and Quick Surveys of Households. In this regard, FH might consider working with WFP to adapt its EFSA methodology and tools for this task. 4. Considering the food insecurity revealed in the evaluation survey, FH should conclude as soon as possible the EWS framework which is currently under development for use in real-time programming and program adjustments moving forward into the new MYAP. In this regard, FH should not expect more from the CDCs than they can reasonably deliver at this point in their growth and development and should concentrate their work with CDCs to understanding/appreciating the concept and to obtain their valuable assistance in obtaining useful data for FH, likeminded international organizations and the concerned institutions of the GoDRC to analyze and incorporate into their programming. 86 5. FH should consider incorporating into the next MYAP productive schemes for helping communities bridge their food gap (of 4-6 months) by means of:  FFW tied to the capitalization of FFLG food banks (post-harvest storage)  FFW/CFW for projects that result in productive activities used for community social investment (food for supplemental a feeding program; defraying hospital fees and/or school fees in a form of “mutual” arrangement). 6. Within the scope of the next MYAP, FH should develop monitoring systems and tools for following indicators of household livelihoods which view in an integrated manner: farm food production, income generation, consumption and uses of household revenue. 7. FH should conclude participatory planning with CDCs and the development of Long￾Term Community Development Plans in the time remaining in the current MYAP. At this early stage in their development, CDCs should concentrate on their plans being technically feasible, economically viable and fundable. FH should assist CDCs in emphasizing the concept of community investment within the scope of those plans (as opposed to a simple list of community infrastructure and capital inputs to be provided from external funding). 8. Within the scope of the next MYAP, FH should consider replacing a final evaluation with a system of continuous real-time external evaluation. The evaluation should be conducted by a multi-disciplinary external team (as opposed to a single evaluator as was the case of this evaluation) to accompany the project throughout its implementation on an on-and-off basis. In that way, key evaluation data can be collected during key moments of the project, when that information is readily available and can be most valuable in making adjustments to programming and implementation on a timely basis. Livelihoods Component 9. FH should ramp up follow-up with and provision of technical assistance to FFLG farmers in their own fields so as to link training to actual practice. In this way, reliance on post-test scores can be eliminated as an indicator. It can be replaced by direct observation of the types of improved agricultural techniques being employed in farmers’ fields and directly link performance in that regard with improvements in farm productivity/yields. 10. Within the scope of the MYAP agro-forestry component, FH should consider introducing fruit tree/plants (citrus, papaya, and banana) cultivation as an option which will likely be well received by both FFLG farmers and women/mothers in the health and nutrition component. 11. FH should consider dropping duck reproduction to concentrate all available capital and technical assistance on goat reproduction as a higher risk/reward option for FFLG farmers. 87 12. Unless a significant increase in resources is available for providing farmers with small livestock start-up animals from “non-organic” sources, the MYAP indicator calling for an increase in the number of families raising small livestock, should be dropped in favor of an output indicator which measures the number of reproducers associations and/or number of farmers households which have received small livestock. 13. FH should consider increasing investment in supplying small livestock to farmers through other modalities besides “organic growth” in order to accelerate the increase in coverage of this productive activity within the farming universe. 14. The MYAP LOA target for livestock reproduction revenue should be modified to stand at US$7,000 (the $20,250 indicator is unrealistic). 15. FH should analyze creative post-harvest storage infrastructure options and seek/invest greater resources in support of increasing farmer access to viable infrastructure options. FH should analyze the viability of establishing “grain banks”, even using food assistance (see #5 above) as a form of start-up capital for such ventures with provisions for farmers “repaying” the start-up infusion over time. 16. FH should establish enhanced quantitative and qualitative methods and systems for monitoring farmers’ practice of saving and planting seed from their own harvest. If non-saved seed is planted, the source of the seed and its monetary value should be documented. 17. In order to create more significant coverage among the farmer universe in the project zone, the MYAP and future MYAP should increase investment should be allocated for getting improved seeds into the hands of farmers who are presently not members of FFLGs. This can be accomplished by increasing more seed multiplication FFLGs or by opening up participation in seed fairs with provisions for an appropriate level of subsidy on the basis of farmers financial capacities. 18. In order to more accurately measure MYAP beneficiary crop yields, the project team should organize a system of randomly sampled crop cuttings just prior to each harvest (also recommended in the MYAP baseline report). 19. The MYAP agricultural team should set up and implement a monitoring system for observing and documenting soil and water conservation measures being implemented in beneficiary farmers’ own fields. Both quantitative and qualitative information should be accumulated for evaluative purposes. 20. In light of projected seed production and the protocols the associations have put in place for the allocation of harvests, the impact indicator target for revenue generated from seed multiplication should be re-articulated to track the amount of seed redistributed to members as “dividends” as well as any seed stock provided to FH for the establishment of new FFLG associations. The unit of measure could be established by weight (kg.), which could then be extrapolated to areas to be planted in improved seed varieties. 88 21. FH should investigate with WFP whether the latter’s P4P modalities could be a viable fit for FH’s support to FFLGs and whether WFP’s P4P roll-out would permit participation by FFLGs possible. 22. Within the scope of current and future MYAP programming and implementation, FH should seek closer operational coordination with other concerned actors (local and international NGOs, other international organizations, and GoDRC line ministries) in order to unify technical and programmatic strategies, share best practices, provide mutual technical support when required, and avoid overlap, while seeking maximum coverage of the universe of potential beneficiaries. This would be particularly relevant in Moba, where FH and ACTED have had a fruitful working relationship. The two might seek to kindle the emergence of an inter-agency coordination mechanism to take over from humanitarian coordination once carried out by OCHA and subsequently UNHCR. Seed multiplication and the organization of seed/tool fairs could be first areas of focus. Health, Nutrition and Hygiene Component 23. FH should establish monitoring procedures and tools to carry out systematic direct observation monitoring of what MLs and BMs are actually practicing in their daily lives as a real-time ground-truthing of beneficiaries’ self-reporting. Among the practices which proved to be problematic in the evaluation survey and which should receive special attention should be:  Use of ITBNs (including which family members use them)  Hand-washing  Household food consumption and dietary diversity (“what’s in the pot”)  Quality and use of household latrines 24. FH should revamp its awareness-building and training curriculum move from broad concepts to targeted practical training in the application of practices that are observed to be weakest among MLs and BMs and in need of reinforcement (see #23). The training plan should be practical and need-based. 25. FH should modify the output indicator that targets the average number of community meetings that are organized to one which targets/measures the number of women/mothers who receive a full series of health/hygiene and nutrition training (the direct link to results and impact is not through the number of meetings, but the coverage achieved through training. 26. FH should further investigate probable causes for the deterioration of nutritional status among young children targeted by the MYAP. The investigation should consider structural (poor livelihoods), punctual (drought and other shocks), and poor health and nutrition practices. 27. FH should consider putting in place a monitoring and evaluation model for dietary diversity based on weighted value for different food groups according to nutritional 89 value. In this regard, FH should approach WFP on assistance in adapting their EFSA tools for FH’s development approach. 28. FH training and technical assistance to BMs should focus more specifically on helping BMs increase, in reasonable, cost-effective ways, their families’ consumption of legumes/nuts, eggs and dairy products within their diets. 29. FH should approach sources for commodities appropriate for Therapeutic Feeding (UNICEF, WFP and/or USAID) to replenish current dwindling stocks of fortified rice to ensure that children who are identified as severely malnourished during screening carried out by MLs are provided the appropriate attention when their cases are confirmed at local health centers. 30. In light of recent poor harvests, probable upcoming household food insecurity and survey evidence of deteriorated nutritional status among young children, FH should consider developing an immediate supplemental feeding intervention for moderately malnourished children (perhaps in partnership with WFP) and a follow-up program within the scope of a new MYAP. A take-home family ration could be employed, or child nutritional status could be used as a criteria for participation in FFW assistance. 31. The MYAP should increase and intensify awareness-building in the importance of use of ITBNs and of hand-washing. In addition, the FH team should work more directly with MLs and BMs in seeking solutions for increasing the application of those two practices in homes. In this regard, simple elements of cost-benefit should be introduced into awareness-building. Likewise the team should work with MLs to devise appropriate and acceptable means by which families can obtain ITBNs without recourse to outright grants (i.e. “nets-for-work”, CFW to by nets or a simple price subsidy – perhaps in association with seed fairs or through vouchers for redemption with pre-vetted merchants). 32. The FH MYAP team should further investigate the cause of the apparently low rate of POU water purification practice in Moba site communities. Depending on whether it can be determined that survey error, weak awareness/knowledge, or a weak household economic capacity are at the heart, practical steps should be taken to assist households in addressing the problem. 33. The MYAP team should carefully consider future motivational assistance that the program provides to MLs for the possible unintended side effects it might have on BMs perceptions of MLs as beneficiaries favored with tangible benefits as opposed to just training. In this regard, support for income-generating activities should be considered only within the scope of activities designed to benefit the population of communities in general. 34. In order to provide MLs and BMs with income-generating options for their families, the MYAP should consider those which are the least labor intensive possible, which keep women/mothers as close to the home as possible, and which enhance the quality of consumption in terms of the nutritional value. Activities should not duplicate what 90 is already being addressed by FFLGs within the scope of the MYAP and which men and women already collaborate in household fields. In this regard, fruit tree and plants cultivation could represent options (i.e. citrus, papayas, banana and amaranth which can be grown in close proximity to houses).