CATHOLIC RELIEF SERVICES MADAGASCAR PROGRAM FINAL FELANA EVALUATION (FFE): GLOBAL REPORT FOOD SECURITY TO ENHANCE LIVELIHOOD THROUGH AGRICULTURE AND NUTRITION ACTIVITIES (FELANA), FY 2004 – FY 2008 PREPARED BY TANGO INTERNATIONAL L. MORINIÈRE (GLOBAL) AND B. SAGARA (QUANTITATIVE) September, 2008 i TABLE OF CONTENTS Table of Contents ................................................................................................................... i List of Tables ......................................................................................................................... ii Acronyms ............................................................................................................................. iii Executive Summary ............................................................................................................... 2 1. Introduction ...................................................................................................................... 4 2. Methodology ..................................................................................................................... 6 4. Interventions (Quantitative and Qualitative Achievement) .............................................. 13 5. Program Management .................................................................................................... 35 6. Unanticipated results (not captured by IPTT Indicators) ................................................. 41 7. Lessons learned and Conclusions ..................................................................................... 42 6. Recommendations .......................................................................................................... 45 Annexes: ............................................................................................................................. 47 References .......................................................................................................................... 48 ii LIST OF TABLES Table 1: EValuations ...................................................................................................................................................... 6 Table 2: % of Households that had enough food to cover all needs & Number of months with enough food .......... 13 Table 3: Percent increase in value of HH production and domestic assets ................................................................. 14 Table 4: Ownership of Domestic Assets ...................................................................................................................... 14 Table 5: Ownership of tool assets (% of households that own) ................................................................................. 15 Table 6: Ownership of Livestock assets (% of households that own) ......................................................................... 15 Table 7: Impact of natural disasters on felana areas ................................................................................................... 15 Table 8: Coping strategy categories by FELANA Diocese ............................................................................................. 16 Table 9: Percent of HHs adopting various Coping strategies by Zone (see diocese-level disclaimer above) .............. 17 Table 10: Type of agricultural training received (selected training above 3%) ........................................................... 17 Table 11: Known Agricultural Techniques and abandonment rate (known by 3%+ of the sample) ........................... 18 Table 12: Percent increase in crop yields (kg/ha) ........................................................................................................ 19 Table 13: Type of seed used ........................................................................................................................................ 19 Table 14: Crops on which improved seeds were used ................................................................................................ 19 Table 15: Type of fertilizer used (% of plots where used) ........................................................................................... 19 Table 16: Type of pesticide used (% of plots where each type is used) ...................................................................... 20 Table 17: Livestock holdings ........................................................................................................................................ 20 Table 18: Known livestock practices and abandonment rates (known by 3%+ of the sample) .................................. 20 Table 19: SO1 IR2 Output indicators ........................................................................................................................... 21 Table 20: Increased revenue from commercial sales .................................................................................................. 21 Table 21: Type of marketing training received ............................................................................................................ 22 Table 22: Marketing techniques known and abandonment rates (known by 3%+ of the sample) ............................. 22 Table 23: Percent of HHs that lost any crops............................................................................................................... 23 Table 24: Percent and quantity of crop losses* ........................................................................................................... 23 Table 25: Type of NRM training received (PACOM) .................................................................................................... 24 Table 26: Anthropometrics .......................................................................................................................................... 26 Table 27: Breakdown by age and sex of children in final survey ................................................................................ 26 Table 28: Breastfeeding Practices................................................................................................................................ 26 Table 29: Sanitation in and around the household (% of households) ....................................................................... 27 Table 30: Water availability compared to last year in 3 most critical months of the dry season (% of households) . 28 Table 31: Average quantity of water consumed per household and per capita by season ........................................ 29 Table 32: Source of drinking water during rainy vs. dry season (Main source only) (%HHs) ...................................... 29 Table 33: Mean time in minutes to fetch drinking water by season .......................................................................... 29 Table 34: Final Vaccination Coverage rate .................................................................................................................. 30 Table 35: Baseline Prevalence of Childhood Illnesses ................................................................................................. 31 Table 36: Final Evaluation Prevalence of Childhood Illness (N) ................................................................................... 32 Table 37: Number of centers and beneficiaries during FY07....................................................................................... 33 Table 38: Number of Beneficiaries .............................................................................................................................. 37 Table 39: Integration and distribution of effort by component .................................................................................. 38 Table 40: Felana achievements ................................................................................................................................... 42 Table 41: Final PITT ........................................................................................................................................................ 2 iii ACRONYMS ART ARI Administrative, Religious or Traditional (Local Authoriteis) Acute Respiratory Infection BCC Behavior change communication CRS/MG CRS Madagascar DAP Development Assistance Proposal DPMP Disaster Prevention, Mitigation, and Preparedness FARN FELANA Foyers d’apprentissage et rehabilitation nutritionnelle (HM/PD in English) Food Security to Enhance Livelihood through Agriculture and Nutrition Activities FFE FFS Final Felana Evaluation Farmer Field Schools GoM Government of Madagascar HM/PD Hearth Model/ Positive Deviance JPBS Justice, Peacebuilding and Solidarity JBS IGA Joint Baseline Survey Income Generating Activities IMCI Integrated Management of Childhood Illnesses IPTT Indicators Performance Tracking Table (or PITT) IR ITN Intermediate Result Insecticide Treated Nets LOA Life of Activity MAP MTE Madagascar Action Plan Mid Term Evaluation MoA Ministry of Agriculture MoH Ministry of Health NRM Natural Resource Management PMI President’s Malaria Initiative PSI Population Services International RDM SN SO Risk & Disaster Management (GRC in French), Safety Net Strategic Objective SRI UFE Système Rizicole Intensif Utilization Focused Evaluation 2 EXECUTIVE SUMMARY The FELANA Program (2004-08) of CRS/Madagascar is a successful example of an integrated Food Security platform targeting the country’s most vulnerable with a package of agriculture, marketing, risk management, health / nutrition and safety net interventions. Its success is built upon decades of field work in the country, solid approaches, strong technicians and a committed implementing partner network. FELANA succeeded despite many hurdles: • a challenging learning curve to master the chosen approaches, • difficulty in managing four independent partners held accountable to multiple ‘masters’, • a period of high inflation and economic shocks (huge fluctuations in the value of an operational budget pinned to the US dollar) and • numerous natural hazards (cyclones, floods, hail storms, livestock epidemics, etc.) affecting the areas of intervention. The FELANA Program success has been measured not only by the careful tracking of 42 contractual Performance Indicators (PITT) but also by a solid qualitative appreciation by external evaluators canvassing the six dioceses. The Final evaluation was conducted between June and September 2008, guided entirely by TANGO (Technical Assistance for NGOs). Out of the 42 indicators, 20 (process and impact indicators combined) have been achieved in full and another 14 have improved since the 2003/4 baseline. This gives a total achievement rate of at least 80%. Although touted as the signature of FELANA, complete integration of activities (all four interventions) exists within only 30% of the 144 targeted communities. The communities that benefit from the full FELANA package boast generously of its impact. As articulated by one technical coordinator: “entire families have benefitted from FELANA integration: the father attends the FFS, the mother, a FARN, the older son participates in PACOM-FFW or markets the surplus, the younger son attends school (due to the increased household revenue) and the baby is breastfed longer”. Other noteworthy FELANA strides providing strong evidence for CRS impact on food security include: • Increase in the average number of months with adequate access to food from 5 to 6 months; • Increase in total value of assets by 50%; • Improved adoption rates for almost all agricultural and NRM techniques; • Appreciable increase in exclusive breastfeeding; 50% of mothers start within one hour of delivery; • Full vaccination (children 12-24 months) up 21%, to an overall rate of 51%; • Net increase in use of mosquito nets for children (now 90%); • Safety net beneficiaries at 22,094, almost double the stated target of 12,500. All FELANA approaches were found to be featured in colorful success stories. Beneficiaries go on to claim that their communities are now working together, more unified than ever before and that a climate of confidence has returned to the village. Authorities claim that the FELANA villages are more manageable now, thereby attracting actors from other agencies and sectors. CRS implementing partners have learned from their trials, and (if the experienced staff do not slip away during the present difficult transition) are well seasoned to attract other donors, thereby facilitating an exit for CRS, if need be. 3 In conclusion, CRS-partner relations are besought with tension; internal consensus begs to recognize major differences between the two types of partners (Caritas versus independent). There is reason to believe that enhancements to CRS Monitoring & Evaluation efforts will help implementing partners be more engaged in producing results while providing regular information to assist in monitoring. Although FELANA significantly contributes to the MAP, very few visible links are found between CRS/Felana/Partners and the Government of Madagascar. Safety Net Centers make an important contribution to Food Security in Madagascar –one unique among USAID partners; the respective administrations, however, require particular assistance in reaching for independence by expanding IGAs. Risk and Disaster Management is the least integrated but most visible component of FELANA. The main recommendations in the present report encourage CRS to: • fine-tune the PITT indicators and CRS monitoring and evaluation system. • increase the PITT sample size to have representativity at the level of each implementing partner, thus engaging them more in the quest for measurable results and accountability. Dare to collect only that which can be analyzed and used immediately. • develop a rapid monthly mini ‘early warning’ system among CRS zones to monitor both progress and impact. • engage the Government of Madagascar as a more visible partner reaping benefits of the efforts. • promote Safety Net Centers to status of main CRS partner, aiming to permit their respective rosters to include those not given entrance (and even more vulnerable). • alongside the Ministry of Justice, target prisons more systematically in future Safety Net Center development. • mainstream Risk/Disaster Management and Marketing activities to achieve systematic integration. • make a strategic decision to systematically strengthen Caritas in all program areas or organize an (albeit partially limited) Call for Proposals for the next program. Although a complete exit strategy and measurable sustainability are yet premature concepts, the seeds for a more food-secure future have been solidly planted within these 144 FELANA communities. 4 1. INTRODUCTION A. PROGRAM DESCRIPTION The Title II-funded FELANA program was launched in October 2003. The new program was a dramatic departure from the previous DAP, abandoning the distribution-based MCH program and top-down agricultural extension approach for the participative Hearth / Positive Deviance and Farmer Field School methodologies. In addition, the agro- enterprise marketing approach piloted in the USDA-funded Mihary program has been fully integrated into FELANA. The Risk / Disaster Management (GRC) work conducted by CRS since the 1970s was also pulled in, with both prevention/mitigation activities and targeted relief to communities exposed to disaster events during the life of the program. Also an integral component, the distribution-based vulnerable group interventions became the new “Safety Net” component of FELANA. As an integrated Food Security program, FELANA focused on agricultural activities such as increasing production, diversification, and agricultural revenue, coupled with intensive small-group-based village and household-level complementary nutrition education targeting young children. The integrated package also includes rural infrastructure rehabilitation and environmental protection through food for work, disaster preparedness/planning as well as response, and a critical safety net component. Program activities have been carried out within 144 communities in 4 Dioceses: Antsirabe, Farafangana, Mananjary, Toamasina (managing sites in Fenerive Est). See map. The program goal is the “sustained improvement in household food security of vulnerable families in 4 regions1 of Madagascar”. The program has five major components: Agriculture, Marketing, Health/Nutrition, Risk/Disaster Management and Safety Net. The first four are conducted at the community level within the four Dioceses and the last, at the level of 104 centers within six Dioceses. FELANA folds out indicators under three strategic objectives, as portrayed in the text box. These are increased revenue resiliency, improved health status and promotion of human dignity for the most vulnerable. 1 The term ‘region’ here does not refer to the administrative division of Madagascar, of which there are 22. SO1: Increased resiliency of household revenue IR 1: Increased agricultural productivity. IR 2: Increased revenue from commercial sales. IR 3: Better organization for disaster prevention and preparedness IR 4: Reinforced partner and community capacities to enhance household resiliency SO2: Improved health status of the most vulnerable members of rural families IR 1: Improved nutritional practices of the family, especially young children. IR 2: Improved household prevention and management of priority diseases for children. IR 3: Reinforced partner/community capacity to support nutrition, and preventative health care. SO3: Promotion and maintenance of basic human dignity for the most vulnerable of society IR 1: Increased levels of food availability to people served in 100 safety net centers. IR 2: Improved income-generating skills of safety net center beneficiaries. IR 3: Reinforced partner capacity to support safety net activities. 5 B. OBJECTIVES OF THE FINAL FELANA EVALUATION (FFE) The overall objectives of the DAP Final Evaluation are to measure program impacts at the beneficiary level and to identify best practices that will be capitalized on, in order to scale-up future integrated development programs that are in line with and contribute to the Madagascar Action Plan (MAP), especially with regards to integrated food security. The specific objectives of the final evaluation are to: Assess Performance: Measure program results and impacts, comparing them to the target values set initially in the program documents. More focus will be given to intermediate results and impacts results included in the program, commonly called Performance Indicator Tracking Table (PITT). Also, analyze how program activities contributed to enhancing the Madagascar Action Plan (MAP) objectives and concerned national policies. Assess Program Efficiency and Sustainability: Analyze sustainability of program impacts and program related activities that may require continued efforts. This should be done within the cultural context and values. At the same time, study the effectiveness and efficiency of structures meant to take over activities. Identify Best Practices: Identify best practices, successes, constraints and weaknesses that occurred during the program implementation. Assess Relevancy and Effectiveness of Approaches: Assess various approaches used in the program (FFS, Territorial Approach to Marketing, Hearth/Positive Deviance) and the global strategy for implementation, including relevancy compared to the population’s needs, effectiveness compared to targeted results, and strengths and weaknesses. Determine Complementarity / Coherence of Strategies and Approaches: Assess complementarity between various approaches and strategies; identify gaps in order to propose an alternative strategy for future actions. Conduct Stakeholder/Partnership and Capacity-building Analyses: Assess to what extent technical capacity building and partnership have been effective in terms of: (1) reinforcing technical skills of implementing partners; and (2) ensuring sustainability of program impacts at the beneficiary level. Assess the Management Process: Analyze (1) the extent to which program organization and management process allowed an adequate implementation of program activities in order to reach targeted objectives and (2) the strengths and weaknesses of the organization and propose recommendations. Make Recommendations: Provide recommendations based on lessons learned and best practices, in order to improve CRS future actions related to: (1) integrated food security sectors; (2) other sectors that are identified to be relevant to make greater impacts on the livelihoods of the population; (3) and program design, organization and implementation that are consistent with the CRS agency guiding principles. 6 2. METHODOLOGY The methodology employed for the Final FELANA Evaluation (FFE) draws on classic evaluation protocols combining quantitative and qualitative techniques. The quantitative collection was organized by CRS in June/July 2008 (for final analysis by TANGO) and the qualitative field work was organized directly by TANGO in August/September 2008 under the management of an International Consultant visiting Madagascar for 42 days. A. QUANTITATIVE The quantitative methods for the FFE respected those employed in the joint baseline survey (JBS) conducted in December 2003 / January 2004 by FANTA for the implementing agencies (CARE, CRS and ADRA). The Mid Term Evaluation (MTE) was conducted between January and March 2006 under the supervision of an external consultant. The final quantitative survey was organized in June/July 2008 by CRS with external surveyors. For many indicators, the difference in collection dates is inevitably problematic due to seasonal differences. See Table 1. The final survey instrument was the same as the baseline with the exception of both the Family Planning and AIDS modules being removed. The data collection model used was the same as the Baseline and Mid-Term Evaluations. Made popular by FANTA, it is based on the plausibility concept (Habicht and Victoria, in Magnani 1999). The sampling provided data representative at the global (agency) level --allowing a comparison between USAID￾funded Consortium partners. It collected data from all regions where FELANA works. A sufficiently high number of respondents was surveyed to yield a margin of error of 5% (i.e., a 95% confidence interval). A classic, randomized sampling was used, with communities2 as primary (N=28), households as secondary (N=728), and infants (N=380, aged 6 to 59 months) as tertiary groups, all proportional to the percentage share of beneficiaries per Diocese. To obtain data comparable to those collected previously, communities were purposefully included in the FFE survey if they had been surveyed in the baseline and were also FELANA beneficiaries. Four communities were chosen as control samples for the FFE. Limitations of the methodology: Due to the limited sample size, there is no diocese-specific validity that permits a comparison across the four main intervention zones. The excessively long questionnaires (a total of 72 pages for per household, mother and farmer combined) entailed a huge investment of time and resources when only 42 indicators were actually required in the Performance Indicator Tracking Table (PITT). Furthermore, there is no CRS institutional memory regarding the exact definitions and calculations concerning the quantitative baseline; this has significantly complicated the comparison between baseline and final evaluations. Many questions were posed during the FFE which found insufficient answers; a CRS document was apparently issued in 2004 that addressed, in vain, many of the same questions to FANTA. 2 A community is a term specific to CRS that groups between 250 and 500 households from 1- 3 fokotany (villages). TABLE 1: EVALUATIONS 7 B. QUALITATIVE The qualitative dimension of the Final FELANA Evaluation (FFE), and the global final evaluation report, are both managed by TANGO. The qualitative methodology included a survey of, and a short workshop with, CRS personnel and field work in each of the four dioceses. Each of these is described in detail below. • Survey / Interviews of CRS Personnel Following in-depth literature review, the first qualitative method involved a simple tally of the opinion of key informants inside CRS. This was especially important because many key CRS staff were on leave or had important missions linked to the MYAP preparation schedule. Six main questions were posed by email and responses were received from ten senior and technical CRS staff. This questionnaire and interviews with key CRS program staff helped greatly to develop the strategy and scope of qualitative field work and to set the tone of the FFE. • Workshop with CRS Personnel Once the preliminary quantitative results were available, based on the pre-approved set of indicators, a 3-hour workshop was held with CRS Monitoring and Evaluation and Technical personnel to get their reactions to the preliminary quantitative data. Each participant was asked to present the main strength and weakness of FELANA and these were debated. Then they broke into component teams to review their respective indicators, to express their concerns or to propose various explanations of the trends identified. They were also asked to estimate which of the four dioceses was most likely to be the most/least successful in reflecting the trend. This exercise was essential both to check the definitions used in calculations of the quantitative data and to help plan the qualitative portion of the evaluation. • Qualitative Field Work Qualitative field work was conducted by teams of external (2 agents) and CRS internal (1 agent) staff between August 15 and 30, 2008 in the four dioceses where FELANA interventions were targeted. Each team organized 18 qualitative sessions (key informant interviews or focus groups) to grasp the dynamic and impact of FELANA in their respective dioceses. The list of sessions is featured in the inset, right. Selection and Sampling: Partners and Dioceses were chosen during the design of FELANA. The choice of the government entity interviewed was based on the partner informing the team as to which respondent (typically at the level of the commune) was the most familiar with FELANA activities. The choice of the two safety net centers was made to have at least one ‘A’ Center (benefitting from an Income Generating Activity supported by FELANA). The choice of 18 Qualitative Sessions: Each team (Diocese) organized the following: Interviews (3): 1, GoM Agent 2, Safety Net Center Administration Focus Groups (15): 1, Partner Coordination Team 2, Safety Net Center Beneficiaries and within each of 4, Communities: 1, Local ART Authorities 1, Mothers 1, Farmers 8 the four communities was made based on the following criteria: ideally two “B-I-F” communities (those that had been surveyed for the baseline and quantitative final and had benefitted from the FELANA intervention) and ideally one that had benefitted from all four intervention components (Agriculture, Marketing, Health/Nutrition and Risk Management). The teams were also asked to ascertain that they not visit only the most successful intervention communities, but also at least one that was known by partners to be a problematic program site. External team and Training: CRS proposed four three-person teams made up of two external analysts accompanied in the field by one CRS technical person. The profile of the analysts was set by the international consultant and demanded a certain level of education and a command of the dialect(s) to be used to gather qualitative information. A two-day training session in Antananarivo was organized for the qualitative team (external and internal members). After an introduction to Qualitative Research Design and implementation (following WFP/EFSA Technical guidance, 2007), the training engaged the CRS personnel and the external analysts in an exploration of the program to identify the key respondents that need to be consulted for a complete understanding of the program in each diocese. They perfected and practiced seven qualitative discussion instruments (see Annex). In a participative manner, the teams finalized the criteria for site (center and community) selection and the list of sample communities. They divided specific roles and responsibilities between members, created lists of materials needed and developed schedules to be followed. After their return to Antananarivo, each team had five days to submit a full evaluation report in French, following an agreed outline --in addition to their Triangulation Matrices in Microsoft Excel. The writing was entirely the responsibility of the external team to maintain evaluation independence. These reports and matrices are very insightful and fed intricately into the present global final report. As they are written in French language, CRS may decide to share them with their partners at the Diocese level or they may be available upon request to CRS. The international consultant also made a two week trip, visiting the four dioceses to gain a global vision of the program. While backstopping the field teams, she interviewed the four partner heads (Development Coordinators), visited four additional Safety Net Centers and two control communities (where no FELANA intervention was allegedly held). Limitations of the methodology: Time was insufficient to visit a greater number of communities –an inclusion that may have resulted in visiting one where FELANA interventions might have been very different (set of interventions, charisma of leaders, etc.) from those visited. A huge strength of the field work was the use of analysts that had already been involved in the quantitative field collection in July 2008. This made most of the analysts familiar with the breadth of FELANA in at least one area different from the qualitative field work. Having spoken at-depth with a few FELANA beneficiary households, the qualitative analysts had a keen perspective of the program that permitted them to triangulate with new more qualitative information received. 9 3. APPROACHES In the design of FELANA, a set of four approaches were proposed and applied over the past five years that represent an important change from the past. The former DAP focused largely on food distributions (school feeding, etc.) and predominately health interventions whereas the second DAP, FELANA, had an integrated approach featuring food for work. The classic CRS “partner” approach was maintained (whereby CRS leads the management and lends technical guidance, but implements the program through a network of partners --historically those linked to the Catholic Church), and the rural Malagasy ‘household’ remains the prime focus. Technical approaches, however, were featured in the program – some of which had been ‘tested’ in the previous DAP. These inter-related approaches --a short description and an evaluation of each-- are found below. Although the approaches can be used interchangeably, there is one approach for each rural FELANA component: FFS for Agriculture, T/AE for Marketing, FARN for Health/Nutrition and PACOM for Risk & Disaster Management. A. FARMER FIELD SCHOOL (FFS) APPROACH The Farmer Field School (FFS) approach was initially developed in Asia to train farmers in Integrated Pest Management (IPM). The approach, often called “green school”, begins with the precepts that farmers have a wealth of capacity and experience. Learning must be hands-on, and the learner decides what is relevant and meaningful. The approach entails weekly meetings by a self-selected group of farmers. Instead of listening to lectures or watching demonstrations, these farmers observe, record and discuss what is happening in their own fields from the time of planting to the time of harvest. This discovery￾learning process generates an understanding of ecological concepts and their practical application. Typically, new techniques are applied personally by all participants in their fields (or sometimes in a common delegated field). CRS used Farmer Field Schools (FFS) composed of self-selected farmers to facilitate knowledge and information sharing based on their own experiences with their fields. Program FELANA provided technical assistance as farmers identified needs and requested interventions throughout the program. Evaluation: Among the CRS/MG senior staff, FFS is generally accepted as pertinent (it earned an average score of 1.75/5, 1 being the highest) to the FELANA intervention zones. CRS critiques of the approach were mainly concerned with the delays in start-up due to the loss of an entire agricultural cycle prior to achieving visible results, thereby losing the interest of many participants. Some beneficiaries in the Dioceses, on the other hand, find that the training sessions, as well as the new techniques, monopolize too much time. This is proof that some farmers remain unconvinced that the end result (increase yield and/or revenue) merits the investment of time. Others complain about the difficulty to obtain seeds and entrants, that access to adequate water for certain agricultural techniques is insufficient, or that, in Madagascar, it is difficult to identify a practice field that is not private or whose owner will not insist on exclusively reaping the fruit of the efforts of many. In summary, the greatest challenge lies with the partner staff and volunteers who must entirely transform themselves from experts into ‘mere’ facilitators. Facilitation is an art-form mastered or naturally inherited by few individuals. Furthermore, in a country historically exposed to hierarchical top- 10 down methods of instruction, this green FFS is at first a shock, and later a long awaited blessing. Although off to a grand start, FFS has yet to fully bear fruit. B. TERRITORIAL AGRO-ENTERPRISE (T/AE) APPROACH CRS/MG is participating in an East-African marketing "Learning Alliance" supported by CRS, CIAT and USAID-financed Foodnet. The Learning Alliance is promoting the CIAT-developed territorial agro￾enterprise approach to marketing. The “territorial” approach has provided farmers with the capacity to better manage their commercial endeavors linked to local resources and their personal production and animal husbandry. For any marketing endeavor, participating farm groups are educated about the entire marketing process - investigating the market, performing simple economic analyses, identifying market opportunities, selecting potential crops, planning of production, controlling quality, storing, processing, transportation, and negotiation. Marketing in groups will give farmers access to inputs at wholesale prices, better leverage with buyers, and also enables contracting transportation to reach distant markets at more favorable prices. The approach consists of three main phases: 1.) identify opportunities via situation analysis and market studies, 2.) form interest groups (working towards an action plan), and 3. Conduct integrated agro￾enterprise projects. Market opportunities are identified according to what products show a strong demand, which can be produced locally, and which are of interest to small holders. Evaluation: Among the CRS/MG senior staff, Territorial Agro-Enterprise is accepted as being even more pertinent than FFS (average score of 1.56/5, 1 being the highest) to the FELANA intervention zones. In the field, despite initial surprise at the esteem for local crops, the approach is almost systematically considered relevant and sustainable. The qualitative field work clearly demarcated, however, the dependence of this and other approaches on charismatic leadership, the dynamic nature of exchanges between peers and the general resourcefulness of group members. C. FOYERS D’APPRENTISSAGE ET REHABILITATION NUTRITIONELLE (FARN) (known under the English name, Hearth Model and Positive Deviance, HM/PD) The core approach within the health objective is integrated nutrition and health-related behavior change communication (BCC) targeting rural families. This strategy is guided by household and community components of the Integrated Management of Childhood Illnesses (IMCI) approach with complementary strengthening of partnerships between health facilities, traditional health providers, and the communities they serve. Project FELANA has adapted the community IMCI approach recommended by WHO to focus on the community and household level. At the household level, paired with IMCI, the Hearth Model (known locally under the French acronym FARN) was utilized, based on the positive examples (thus Positive Deviance) of successful mothers in the same vulnerable socio-economic strata as other project participants. The Hearth Model and Farmer Field School are perfectly complementary approaches since they are built on the same principles of valuing indigenous knowledge of people who are often illiterate or have low levels of formal education. Both methods recognize that communities are the best managers of their resources and are actually very resourceful, and promote sustainability 11 through community discovery and ownership of solutions. The entire package builds upon culturally appropriate BCC and improving hygiene and sanitation facilities and practices, by using locally developed, innovative IEC tools. Evaluation: Among the CRS/MG senior staff, FARN is accepted as being the most pertinent (average score of 1.29/5, 1 being the highest) of approaches to the FELANA intervention zones. Most beneficiaries not only use the term ‘FARN’ but rave about the successes of the approach and every community has a colorful story about FARN having saved a child’s life. Some beneficiaries in the Diocese, on the other hand, find that the weighing sessions monopolize too much of their time and bring little to bear on their children’s health, and that certain products (such as the oil to which group members must contribute) are beyond their means. This is proof that some mothers remain unconvinced that the end result (improved nutrition) merits the investment of time and minimal resources. D. PACOM (PLAN D’ACTION COMMUNAUTAIRE) Drawing on lessons learned over the past ten years, CRS/MG applies effective Risk & Disaster Management (RDM) strategies that strengthen community organization to address natural hazards and to help communities build productive assets and conserve/develop their natural resources so that household vulnerabilities are reduced. Often referred to by the dated term ‘DPMP’ (Disaster Prevention, Mitigation, and Preparation), community level action plans (well-known in the field as ‘PACOM’) have been implemented as a critical component in limiting crop and asset losses, which significantly impact food security of families in vulnerable zones. Although more a tool than an approach, the PACOM focuses on rehabilitating irrigation systems to improve drainage and reduce flood damage, rehabilitating access roads, reinforcing roofing, improving grain storage, reducing soil erosion and runoff, and expanding potable water and sanitation program to mitigate the impacts of future emergencies and rapidly rehabilitate the damage, allowing for communities to quickly recover to pursue their pre-disaster development activities. Many of the PACOM activities are integral to the overall development of the community - improving potable water and sanitation and improving roads and access. Food For Work assistance will be provided for a variety of community-identified projects appropriately timed to occur during the off-season, when the hunger period strikes annually. The FFW will provide an additional resource for the participants and is designed to complement the existing tradition of community volunteer work. Evaluation: Qualitative field work produced a clear signal that the term ‘PACOM’ was synonymous with ‘infrastructure’. While many community residents have little knowledge of the existence of a plan, as such, if the RDM component was active in or near a community, they all knew about the visible and tangible ‘by-products’ embodying the PACOM. Another appreciation was that the PACOM infrastructure ‘package’ is an easy ‘carrot’ to attract the buy-in of regional and local government authorities, with the hopes that it will catalyze a more permanent commitment thus assuring the sustainability of program interventions or impact. In one diocese, PACOM was considered to be simultaneously the component most likely to reinforce solidarity and community responsibility (notably regarding the infrastructure maintenance organizations) and the most problematic of all (requiring costly external inputs, expertise, and buy-in from many parties as well as the risk of more visible failure). 12 General comments: Back in 2003, the approaches were new to most CRS and partners staff in Madagascar, and without surprise, took time to be understood and to transform into visible and flourishing efforts in the field. After five years of FELANA, and just barely feeling ‘master’ of the approaches, however, most partners suggested that they not be increased or changed for a future program. Partners also requested more flexibility in the way that approaches are implemented and controlled. See text box. As a singular example, it may be more useful to allow partners to organize Farmer Field Schools with fewer than 10 households, as achieving this number has led at times to the ‘forced’ inclusion of less-than motivated households. On the other hand, CRS personnel feel that, although systematically starting with what one has on hand locally (the so-called ‘territorial’ approach) is both inherently common sense and best practice (“we really couldn’t go wrong using this approach”), an ‘approach’ is nothing more than “a means to an end”, and that in certain zones, there may be other means that are equally appropriate. Some examples proposed include model farmer and more heightened commercialization (perhaps beyond the means of typical CRS interventions). Most CRS staff are frustrated that approaches and/or their names change systematically every few years within the international and NGO community (“approaches à la mode”) and that all approaches would benefit from appropriate in-country pre-testing before scaling-up to the level of FELANA. A more general comment was that the approaches applied during this DAP were inherently better because they are all bottom-up --avoiding the classic top-down initiatives of yesterday. We need more flexibility to implement approaches and interventions. FELANA was conceived as a practice-based program, and yet often neglects field realities. A Partner 13 4. INTERVENTIONS (QUANTITATIVE AND QUALITATIVE ACHIEVEMENT) Program FELANA was built upon a long history of previous CRS experience in Madagascar and aimed to target the most vulnerable in remote areas. Following a report on progress made regarding the general objectives, quantitative results and qualitative achievement are reported, by intervention: Agriculture, Marketing, Risk & Disaster Management, Health / Nutrition and Safety Net. A. GENERAL Table 2 below provides information regarding the number of months a household had enough food to cover all needs in September 2007-August 2008. The average number of months with adequate access to food increased from five to just over six months over the life of the program. Large gains were made in September through November, as well as significant gains in January through April, the most food insecure months according to the baseline survey. The typically abundant period from May to July, appears to have been negatively affected by Cyclone Ivan, which made landfall in February 2008 and severely affected the Toamasina Region (OCHA 2008). TABLE 2: % OF HOUSEHOLDS THAT HAD ENOUGH FOOD TO COVER ALL NEEDS & NUMBER OF MONTHS WITH ENOUGH FOOD Baseline Final Differenc e During all of last year 10.9 8.6 -2.3 September 2007 30.6 43.7 13.1 October 2007 24.6 37.1 12.5 November 2007 31.2 35.4 4.2 December 2007 53.9 51.2 -2.7 January 2008 39.0 47.4 8.4 February 2008 26.5 42.3 15.8 March 2008 26.9 42.8 15.9 April 2008 35.7 49.6 13.9 May 2008 63.9 72.7 8.8 June 2008 72.4 73.6 1.2 July 2008 59.9 62.7 2.8 August 2008 40.6 46.8 6.2 Mean # of months with enough food 5.00 6.05 1.05 Program FELANA sought to increase the resiliency of household revenue through supporting agricultural production, marketing and risk and disaster management. Table 3 below illustrates a 30% increase in household production values, a 13% increase in value of domestic assets, a 57% increase in livestock values and a 62% increase in productive assets. The total value of assets is up by 50%, exceeding the LOA target of 10%, providing strong evidence for CRS impact on agricultural productivity. 14 TABLE 3: PERCENT INCREASE IN VALUE OF HH PRODUCTION AND DOMESTIC ASSETS Baseline Final % increase LOA Target (%) Value of household production (Ar) 76,736.64 99,909.54 30.2 25 Value of domestic assets (Ar) 85,892.20 97,350.08 13.3 n/a Value of livestock assets (Ar) 383,683.2 603,153.69 57.2 n/a Value of productive assets (Ar) 49,392.20 79,829.56 61.6 n/a Total value of assets 518,967.60 780,333.33 50.4 10 TABLE 4: OWNERSHIP OF DOMESTIC ASSETS (% OF HHS THAT OWN ) Baseline Final Difference Vehicle 0.7 0.6 -0.1 Bicycle 14.2 25.3 11.1 TV 2.3 3.0 0.7 Cassette or CD player 17.8 13.3 -4.5 Radio 29.6 57.5 27.9 Watch/clock 27.3 35.6 8.3 Kettle or tea pot 97.2 96.5 -0.7 Table 57.8 65.0 7.2 Chair 42.1 55.0 12.9 Wooden bed 60.1 74.9 14.8 Metal bed 4.3 5.1 0.8 Armoire 9 15.6 6.6 Trunk 41.9 49.8 7.9 Gas stove 1 2.5 1.5 Boat 8.2 11.4 3.2 Stool 18.5 33.3 14.8 Canape 4.5 6.9 2.4 Table 4 describes domestic asset ownership. Households in general appear to have acquired more domestic assets over the life of the program, with the greatest gains being radio, chair, bed, and bicycle ownership. These gains are particularly important to note, given that radios constitute a significant source of information (especially regarding imminent hazards) for rural households and bicycles are often a primary mode of transportation in rural contexts. There was a very minor erosion of ownership of household assets (vehicles, cassette or CD players, and kettles). This is important to keep in mind when analyzing the coping strategies used by households, specifically with regards to divestment. The data suggest that households are more likely to sell productive equipment or livestock to cope with shocks. Tables 5 and 6 describe tool and livestock asset ownership. More households reported owning sickles, watering cans, and wheelbarrows by the end of the program, but shovel, machete and axe ownership was eroded over time. Modest gains were made on livestock ownership, although the percentage of households owning chickens or ducks decreased over the life of the project. These numbers suggest that households are increasingly selling their productive assets, more so than household or livestock assets in order to cope with shocks. 15 TABLE 5: OWNERSHIP OF TOOL ASSETS (% OF HOUSEHOLDS THAT OWN) Final Baseline Difference Cart 6.9 2.7 4.2 Plow 5.0 2.6 2.4 Motor pump 0.0 0.3 -0.3 Hand/footpump 0.0 0.3 -0.3 Thresher 2.1 1.2 0.9 Sprayer 1.2 0.3 0.9 Weeder 9.5 6.0 3.5 Shovel 56.1 66.5 -10.4 Engade 86.7 81.3 5.4 Watering can 22.5 5.1 17.4 Pick 23.7 24.9 -1.2 Machete 59.0 69.5 -10.5 Handsaw 8.9 10.6 -1.7 Axe 55.4 70.6 -15.2 Sickle 36.3 25.1 11.2 Fishing net 4.8 4.8 0.0 Fishing basket 8.9 5.6 3.3 Fishing pole 4.2 *** *** Wheelbarrow 3.6 7.1 -3.5 Brouette 38.5 1.5 37.0 Other 3.3 5.4 -2.1 TABLE 6: OWNERSHIP OF LIVESTOCK ASSETS (% OF HOUSEHOLDS THAT OWN) Baseline Final Difference Oxen (Pull carts) 3.6 6.3 2.7 Zebu 12.1 12.8 0.7 Milk cows 0.4 1.1 0.7 Other cows 14.2 18.6 4.4 Heifers 7.8 8.0 0.2 Calves 6 7.8 1.8 Sheep 0.3 0.0 -0.3 Goats 0.1 0.2 0.1 Pigs 15.9 17.3 1.4 Rabbits 2.6 3.9 1.3 Chicken 76.7 63.5 -13.2 Ducks 15.6 10.7 -4.9 Geese 2.7 2.4 -0.3 Fish 0.8 2.3 1.5 Beehives 1.6 2.7 1.1 Other 0.8 26.7 25.9 Exogenous Shocks to households within FELANA Dioceses The 5-year FELANA program witnessed many impediments to achieving targeted results. Here, we speak of exogenous shocks that are for the most part natural: cyclones, drought, flooding, hail storms and livestock disease. Using officially-compiled CRED data alone (see Table 7), at least 545,466 individuals inhabiting the FELANA dioceses were affected by these shocks. In addition to these, qualitative research also brought to light and was able to confirm destructive hail storms and flooding in the Diocese of Antsirabe (a region typically considered outside the realm of natural hazards) and avian and swine diseases throughout the FELANA zones that killed numerous chicken and swine holdings, also during the past five years. Although FELANA specifically worked on animal husbandry, partners confirmed that there were serious problems in obtaining and distributing vaccines. Start End Location Type of Disaster Name Killed Tot. Affected 17/02/2008 3/3/2008 Fenoarivo Atsinanana,... Tropical cyclone Cyclone 'Ivan' 93 524,153 3/1/2007 3/1/2007 Mananjary, Nosy-Varika Tropical cyclone Cyclone ‘Clovis’ 1 7,313 00/11/2005 00/12/2006 Vangaindrano... Drought 14,000 2004 Madagascar… Epidemiological African Swine Fever (SOURCE: CRED, Universite de Louvain, 2008) TOTAL 94 545,466 TABLE 7: IMPACT OF NATURAL DISASTERS ON FELANA AREAS 16 Since January 2008, Madagascar has been hit by cyclones Ivan and Fame. Cyclone Ivan struck the northeast coast (including CRS areas) and was the more severe of the two cyclones, affecting 228,000 people; 147,000 of which lost their homes. More than 225,000 ha. of agricultural land were damaged (OCHA 2008). Coping Strategies Given the magnitude of Cyclone Ivan, already vulnerable households seem to have been forced to adopt more severe coping strategies as tables 6 and 7 below illustrate. The baseline survey categorized coping strategies in the following manner (Bergeron & Deitchler 2004): People’s first and most common way to cope with food insecurity is to reduce food intake, either by reducing the number of meals or reducing the quantity of food in each meal. Consuming inferior foods is also very common. One way to define this first stage of severity in coping is as “tightening the belt”. Those are the least damaging of the coping strategies, as they only involve temporary inconveniences. The next degree is represented by a set of practices that, while apparently unrelated, often cluster closely together empirically. These include selling firewood, borrowing cash or grain, migrating for work or work outside the farm—the common thread being that they all imply reaching out and using unusual means to obtain income or food from sources outside the farm. We thus refer to this second, intermediary stage of coping as “reaching out”. The next stage in coping, the most severe one, is when people start selling their assets. People usually delay this type of strategy as long as possible. When no other choices remain, however, they begin to sell household assets, personal effects or productive assets; or may resort to maternal buffering. We refer to this third degree of coping as “divesting”. Ninety-three percent of surveyed households used at least one coping strategy, with the most commonly used strategies being “belt-tightening” strategies. One of the most alarming statistics below is the 21 percent of households engaging in divestment – erosion of household assets has severe negative implications for future productive capacity and ability to cope with future shocks. As alluded to previously, the significant increase across all coping categories is likely a result of Cyclone Ivan. TABLE 8: COPING STRATEGY CATEGORIES BY FELANA DIOCESE Antisirabe Farafangana Mananjary Toamasina Baseline Final % of HHs (NB: Diocese-level sample size insufficient for statistical validity in this numbers) Tighten the belt 76.6 90.6 95.8 87.5 54.5 86.1 Reach out 49.5 54.2 54.2 59.9 37.7 55.4 Divest 20.7 20.8 30.2 18.8 7.8 21.3 N 184 96 96 287 Table 9 breaks down the individual coping strategies used by surveyed households, beginning with the most commonly used coping strategy of reduced household consumption. With regards to divestment strategies, it is important to note that nine percent of households sold or ate reserve seeds, six percent sold productive assets, and three percent practiced maternal buffering. Only two percent of the sample sold off household goods, which helps explain the increase in household asset ownership in combination with increased rates of divestment. 17 TABLE 9: PERCENT OF HHS ADOPTING VARIOUS COPING STRATEGIES BY ZONE (SEE DIOCESE-LEVEL DISCLAIMER ABOVE) Strategy Antisirabe Farafangana Mananjary Toamasina Total% Reduced quantity of food per meal 72.9 87.2 88.5 77.2 79.1 Ate fewer meals per day 49.7 58.5 63.5 51.2 53.7 Ate less preferred foods 52.5 48.9 45.8 55.9 52.5 Ate wild foods 6.6 46.8 49 53 38.7 Sought daily work outside farm 38.7 35.1 26 35.2 34.8 Borrowed cash or grain 15.5 31.9 26 19.9 21.3 Migrated for work 8.8 8.5 6.3 12.8 10.1 Sold or ate reserve seeds 8.3 7.4 14.6 7.8 8.9 Sold firewood 3.9 11.7 14.6 6.4 7.7 Made charcoal for sale 3.3 5.3 8.3 9.3 6.9 Sold productive assets 8.8 7.4 2.1 4.3 5.7 Rented out land 3.3 7.4 7.3 5 5.2 Withdrew children from school 2.8 6.4 4.2 1.1 2.8 Maternal buffering 1.1 4.3 7.3 1.8 2.8 Relied on help from com. organiz. 2.2 2.1 4.2 2.5 2.6 Sold personal effects 1.7 1.1 1.0 2.8 2.0 Sold household goods (e.g. utensils) 0.6 3.2 1.0 1.8 1.5 Sold jewelry 1.1 1.1 0.0 1.8 1.2 B. AGRICULTURE SO1: Increased resiliency of household revenue IR 1: Increased Agricultural productivity Increased agricultural productivity was a significant component of SO1. CRS used Farmer Field Schools (FFS) composed of self-selected farmers to facilitate knowledge and information sharing based on their own experiences with their fields. Project FELANA provided technical assistance as farmers identified needs and requested interventions throughout the project. CRS/MG FFS activities align well with Commitment Four of the MAP working towards rural development and a green revolution. The breakthrough reform initiative under this commitment seeks to facilitate the green revolution through provision of seeds, fertilizers, and technical support to farmers as well as establish cooperatives to collect and distribute harvests. Activities under this commitment will also include expanding land-use, mechanizing and industrializing agriculture, and encouraging crop rotation and diversification. The mid-term identified limited participation in the Farmer Field Schools (FFS) initially, despite generally positive perceptions of the groups. As the program progressed, however, annual reports suggest that farmers became more likely to participate and adopt new technologies and practices. Table 10 above lists the most common trainings received. SRI and vegetable production are the most common, followed by row TABLE 10: TYPE OF AGRICULTURAL TRAINING RECEIVED (SELECTED TRAINING ABOVE 3%) Training Baseline Final SRI (intensive rice) 2.1 21.0 Vegetable production 1.9 16.7 Row planting 1.3 9.9 Fertilizer applications 1.6 9.1 Planting techniques 0.5 4.9 Crop diversification 0.3 4.7 Composting 0.3 3.8 Soil preparation 0.8 3.6 18 planting and fertilizer applications. Seventy-five percent of farmers who received agricultural training received it from CRS. Table 11 below provides evidence of farmers actually adopting agricultural techniques. Across almost all techniques (except soil preparation, sarclage frequent, and crop diversification), households are more likely now to sincerely adopt new practices. This is a sign of real progress, as the Mid￾Term Evaluation prioritized achievement of adoption over coverage of message transmission/understanding. Fertilizer application appears to be particularly adhered to by farmers. TABLE 11: KNOWN AGRICULTURAL TECHNIQUES AND ABANDONMENT RATE (KNOWN BY 3%+ OF THE SAMPLE) # of HHs know technique # of HHs not practicing JBS* Abandonment rate Final Abandonment rate % change Fertilizer applications 205 19 42.3 9.3 -33.0 SRI (intensive rice) 172 38 71.8 22.1 -49.7 Row planting 160 27 46.2 16.9 -29.3 Vegetable production 114 26 32.0 22.8 -9.2 Soil preparation 77 3 3.7 3.9 0.2 Planting techniques 70 13 33.8 18.6 -15.2 Composting 55 7 28.6 12.7 -15.8 Sarclage frequent 41 3 5.1 7.3 2.2 River diversion 40 1 5.1 2.5 -2.6 Repriquage en espacement 40 3 23.4 7.5 -15.9 Inter/mixed cropping 35 2 9.3 5.7 -3.5 Residue incorporation 30 1 7.1 3.3 -3.8 Seed preparation/selection 29 1 8.2 3.4 -4.7 Crop rotation/pruning 27 3 11.3 11.1 -0.2 IPM (integrated pest management) 25 5 39.3 20.0 -19.3 Improved seeds 25 4 25.0 16.0 -9.0 Crop diversification 23 2 3.1 8.7 5.6 Total 1168 158 *** 13.5 *** *Please note that the baseline values are to be interpreted as a general reference as these values include all CS areas and do not represent CRS operational areas exclusively FELANA focused primarily, but not exclusively, on the increase of rice yields, specifically through SRI. Only two of the principal crops identified by CRS showed increased yields over the life of the program. Although rice showed a modest 16 % increase, it did not meet the anticipated goal of a 50% gain. Taro was the only crop that exceeded the goal of a 50% increase over the life of the program. These numbers should be interpreted within the post-cyclone context however. Over 225,000 ha felt the impact of Cyclone Ivan in affected regions and the MoA estimated crop losses to be near 80% (OCHA 2008). Given the figures in Table 12, it is not surprising that crop yields were negatively affected; in fact, it is more surprising that there was any crop increase at all in FY08. When tracking these indicators across the IPTT, rice, beans, and vegetables were the only consistently increasing yields of the selected crops. One explanation proposed for the success in rice yield was the national rice farmer competition in which many FELANA farmers participated and won (Toamasina especially). 19 TABLE 12: PERCENT INCREASE IN CROP YIELDS (KG/HA) Baseline Final % increase Rice 1,807.67 2,092.13 15.7 Maize 2,642.55 1,002.26 -62.07% Manioc 3,972.37 1,906.43 -52.01% Sweet Potato 4,847.08 3,203.10 -33.92% Beans 1,358.63 1,002.26 -26.23% Voandzou 1,315.48 836.64 -36.40% Taro 2,000.00 3,512.54 75.63% Tables 13 and 14 below present information on seed types used by farmers. Data suggest that there has been a slight improvement in household access to improved seeds. This was not confirmed by the qualitative research which noted repeated concerns with inaccessible seeds. Most commonly improved seeds are used on rice and vegetable crops by 30 and 27 % of households respectively. Please use caution when interpreting diocese level data due to low sample sizes. TABLE 13: TYPE OF SEED USED Baseline Final % change Traditional 97.3 89.7 -7.6 Improved 2.7 10.3 7.6 TABLE 14: CROPS ON WHICH IMPROVED SEEDS WERE USED Crops w/ more than 3% response rate are reported Antisirabe Farafangana Mananjary Toamasina Baseline Total Final % of HHs Rice 21.5 5.0 31.3 36.7 42.4 30.0 Maize 16.9 0.0 3.1 4.7 3.8 7.1 Manioc 6.2 0.0 6.3 2.7 N/A 3.7 Potato 18.5 0.0 0.0 0.7 N/A 4.9 Beans 16.9 5.0 6.3 3.3 4.5 7.1 Vegetable crops 1.5 70.0 25.0 32.0 17.7 26.6 Cucumber 0.0 0.0 3.1 8.7 N/A 5.2 N 65 20 32 150 267 Rural farmers are now using greater quantities of fertilizer on their crops. Compared to the baseline, farmers are using fertilizer on 13 % more of their plots. Table 15 below substantiates the findings that households are using the training they have received in fertilizer application. Chemical fertilizer use increased slightly, but access appears to remain limited. Pesticide use is being practiced on 9% more plots than the baseline, with most households using chemical pesticides on their crops. TABLE 15: TYPE OF FERTILIZER USED (% OF PLOTS WHERE USED) Baseline Final Difference None 77.4 64.3 -13.1 Manure/Cowdung 19.9 27.9 7.9 Compost 2.0 4.8 2.8 Chemical/Others 0.7 2.9 2.2 20 TABLE 16: TYPE OF PESTICIDE USED (% OF PLOTS WHERE EACH TYPE IS USED) Baseline Final Difference None 95.9 87.1 -8.8 Chemical 3.9 9.67 5.8 Mechanical .2 0.03 -.17 Organic .0 2.1 2.1 Other .0 1.1 1.1 The FELANA program had a small livestock component composed of small-animal husbandry techniques. The techniques centered on health care, feeding practices, and housing of poultry and swine. Table 17 below shows a slight increase in the number of chickens and rabbits owned per household, but the overall value of livestock, as stated above, decreased by nine percent. TABLE 17: LIVESTOCK HOLDINGS Baseline Final Difference # Pigs 2.7 1.83 -0.87 # Geese 3.7 4.00 0.3 # Ducks 5.4 6.76 1.36 # Chicken 11.3 14.41 3.11 # Rabbits 6.6 12.19 5.59 # Beehives 2.6 4.00 1.4 Fish kg/ha 151.10 187.47 36.37 Mean value of livestock (Ar) 383,683.20 349,686.70 -8.9 % As was expected, most households know poultry raising techniques, improved poultry breeds, and improved pigs. Table 18 below shows a decrease in the abandonment rate for both improved poultry breeds and improved pigs, signifying that households are subscribing to the trainings they have received. There was a very slight increase (2.8%) in the amount of households abandoning known poultry raising techniques. CRS was the most common trainer of households, reaching 75 % of households that received any training. TABLE 18: KNOWN LIVESTOCK PRACTICES AND ABANDONMENT RATES (KNOWN BY 3%+ OF THE SAMPLE) # of HHs know technique # of HHs not practicing Abandonment rate JBS* % change Poultry raising 144 18 12.5 9.7 2.8 Improved poultry breeds 78 24 30.8 43.94 -13.2 Improved pigs 65 14 21.5 53.36 -31.8 Zero grazing/stall feeding 49 8 16.3 36.14 -19.8 Other (specify) 63 2 3.2 28.57 -25.4 Other small husbandry 24 1 4.2 19.05 -14.9 Fish pond or fish culture 22 10 45.5 61.82 -16.4 Improved beehives or keeping 21 4 19.0 82.76 -63.7 Total 466 81 17.4 *** *** *Please note that the baseline values are to be interpreted as a general reference as these values include all CS areas and do not represent CRS operational areas exclusively 21 C. MARKETING IR 2. Increased revenue from commercial sales FELANA sought to increase market revenues through the territorial approach which seeks to identify marketing opportunities within a region so that farmers can increase their revenues (CRS 2004). One of the challenges addressed by Commitment Four of the MAP is the promotion of market-oriented activities. Through their marketing activities, FELANA has contributed to several aspects of this initiative. Specifically, FELANA has enabled farmers to conduct market analyses, establish infrastructure that enables farmers to access the market and store goods, and strengthen the capacity of farmers’ associations. The key premise is that the farmers will make a shift from the traditional “sell what you grow” mentality to “grow what is determined by the market in order to sell” mentality. The Mid-Term Evaluation found challenges to implementation as FELANA was beginning marketing activities. Specifically, there was confusion due to “the novelty of the approach; low levels of instruction on the part of technicians and comprehension on the part of farmers; and increased time demands on technicians from promoting two program components (AG and MKT)…” (Patterson 2006). In addition, this component began late and suffered from ill-prepared and equipped technician-trainers. Despite initial setbacks, final evaluation data shows a substantial increase in marketing revenue. In 2004, CRS activities largely centered on providing training on the territorial approach to 153 staff members of implementing partners. Table 19 below shows the number of trainings provided to rural farmers over the life of the program. These trainings included food storage techniques and simplified economic management at the household level. Throughout the life of the program, CRS organized several “National Learning Alliance” workshops to share experiences and coordinate marketing strategies among partner institutions. The success of CRS/MG marketing activities is evidenced through the fact that they have been solicited by other government and NGO entities to assist in the training of their agents on the Marketing Territorial Approach. In addition, CRS/MG has provided support to other country programs to develop agricultural marketing focused programs. TABLE 19: SO1 IR2 OUTPUT INDICATORS Indicator Year FY04 FY05 FY06 FY07 FY08 # of AETs formed 3 2 *** *** *** # farmers trained *** 3366 5914 *** *** # communities conducting participative marketing study *** 29 83 *** *** KM of road rehabilitated through FFW 29.1* 44.5 76 *** *** *as reported in FY04 annual results report under IR3 Table 20 below compares the commercial sales revenue over the life of the program. Revenue reached a high of 368,522 FM in FY 05 but steadily decreased in successive years to 261,682 FM. Despite the steady decrease after the initial spike, CRS/MG still exceeded their LOA target of a 50% increase in commercial revenue. TABLE 20: INCREASED REVENUE FROM COMMERCIAL SALES Baseline Final % increase LOA objective Revenue from commercial sales (FM) 164,524.40 261,682.90 59.1 50.0 22 As identified by the annual reports, most farmer training focused on inventory management and identification of market opportunities. Twenty-five percent of sampled households cited that they have received training to identify market opportunities, followed by 24% of households who received general marketing training (Table 21). When asked who provided the training, 90% of respondents stated CRS/MG. Abandonment rates remained relatively low with the major trainings received. Just two percent of households have abandoned storage techniques they have learned; 12 and 10 percent of households have abandoned market opportunity identification and marketing techniques respectively (Table 22). TABLE 22: MARKETING TECHNIQUES KNOWN AND ABANDONMENT RATES (KNOWN BY 3%+ OF THE SAMPLE) # HHs Know technique # of HHs Abandoned technique Abandonment rate Post harvest storage at village community 109 2 1.8 Identifying market opportunities 98 12 12.2 Marketing 68 7 10.3 Product commercialization 65 10 15.4 Post harvest packaging 49 1 2.0 Post harvest storage at market input supply center 46 9 19.6 Credit use 29 15 51.7 Post harvesting pest control 23 2 8.7 Total 487 58 11.9 TABLE 21: TYPE OF MARKETING TRAINING RECEIVED Type of Training % of cases Identifying market opportunities 24.5 Marketing 23.5 Post harvest storage at market input supply center 8.7 Product commercialization 7.7 Post harvest storage at village community 7.7 Post harvesting pest control 5.1 Post harvest packaging 3.1 Value added production 2.6 Credit use 2.6 Organizational development 2.6 Other technique (specify) 2.6 Numeracy 2.0 Cost/benefit analysis 2.0 Agribusiness development 2.0 Other post harvest technique (specify) 1.5 Market linkages 0.5 Organizational strengthening 0.5 23 D. RISK & DISASTER MANAGEMENT IR 3. Reduced crop and asset losses A significant part of risk reduction is ensuring that crop losses are minimal during the harvest. Seven percent of surveyed households suffered from any sort of storage loss (Table 23). Table 24 below presents the percentage of the crops lost in storage as well as the mean quantity of crops lost of households that incurred any losses at all. The most commonly lost crop was rice, followed by maize. Overall, these indicators suggest that CRS/MG efforts to reduce crop losses through construction of storage facilities and training on storage techniques were successful. (*Only crops that constitutes more than 3% of lost crops are included in this table). IR4. Strengthened partner and community capacity to reinforce household resiliency activities PACOM (RDM) Plans Risk and Disaster Management (RDM) activities constituted a significant part of FELANA. Commitment Eight of the MAP addresses National Solidarity through the challenge of improving support to the very poor and vulnerable populations. FELANA helps support this commitment through food distribution, FFW activities, improving service access through infrastructure rehabilitation and development, and ensuring that communities have plans in place to respond to emergency situations. The plans focused on rehabilitating irrigation systems, rehabilitating access roads, reinforcing roofing, improved grain storage, reduced soil erosion, and expanded potable water and sanitation. Using FFW, CRS/MG supported infrastructure rehabilitation and development programs as designated by community￾developed PACOM plans. Natural resource management A component of IR4 was natural resource management which entailed the transfer of natural resource management from the state to the community level and reforestation. This was accomplished through the training of communities on management of natural resources, the implementation of natural resources TABLE 23: PERCENT OF HHS THAT LOST ANY CROPS N % % of HHs incurring losses 1886 6.7 TABLE 24: PERCENT AND QUANTITY OF CROP LOSSES* Crop Unit N Value Std. Deviation Rice % 71 4.0 .06241 Kg 71 28.9 73.08071 Maize % 24 6.5 .07149 Kg 24 10.6 26.48369 Manioc % 6 33.3 .41102 Kg 6 111.0 194.23957 Sweet potato % 6 6.8 .05762 Kg 6 25.0 37.65103 Beans % 4 8.5 .03073 Kg 4 1.2 .68845 Banana % 5 8.4 .00416 Kg 5 45.0 37.08099 Other legume % 5 10.3 .06519 Kg 5 12.8 20.90933 All Crops % 127 6.7 .1184 Kg 127 30.4 73.56202 24 management plans and a pilot structure or NRM and the elaboration of 4 pilot plans for the management of hydrographic basins. Table 25 below illustrates that tree planting is the most common NRM training received by households. Most (71%) households reported receiving their training from CRS, although 14 percent of households reported receiving training from other NGOs. Over the life of the program, CRS fell just shy of their intended goal of 250 ha of land protected through NRM activities. They achieved the protection of 213 ha, meeting 85 percent of their intended goal. Through their reforestation measures FELANA has contributed to Challenge Two of the MAP that seeks to cherish the environment through promotion of reforestation among other conservation measures. TABLE 25: TYPE OF NRM TRAINING RECEIVED (PACOM) Training % of cases Tree planting 34.0 Bund construction 9.3 Area enclosure management 9.3 Slash and no burn 9.3 Live or vegetative barriers 8.2 Other (specify) 7.2 Contour plowing 5.2 Agro forestry 4.1 Tree nurseries 3.1 Cut-off drainage 3.1 Bund stabilization 2.1 Slash and burn 2.1 Terracing 1.0 Grass strip 1.0 Cover crops 1.0 25 E. HEALTH/NUTRITION SO2: Improved health status of the most vulnerable members of rural families During initial program planning activities, malnutrition was cited by participants as a main concern. Baseline data showed higher than average levels of malnutrition in the areas of CRS/MG program focus; by age 15 -18 months of age the rate of stunting and underweight had reached 64.7 and 42.9%, respectively. Baseline anthropometric data caused FELANA to center the introduction of appropriate health and nutrition practices on children under 24 months of age and the practices of caregivers of this targeted group. In addition to malnutrition, focus group priority concerns were disease, contaminated water sources, limited access to health facilities due to long distances, and high drug and treatment cost. Responses to the mentioned concerns that could not be addressed directly through the program’s strategy of education and BCC, were addressed through coordination with the Ministry of Health (MoH) and USAID bilateral programs. CRS has had considerable success in the field in some areas of intervention; others lag behind the targeted achievements. Overall, CRS has been very successful promoting full courses of immunization, encouraging latrine use, increasing caregiver’s ability to recognize when sick children need treatment, and ensuring the use of bed nets to prevent malaria. These successes contribute to and enhance Commitment 5 of the Madagascar Action Plan (MAP) which includes strategies to increase the focus on child health prevention programs and to promote child nutrition and vaccinations (IMF 2007). The percentage of stunted children aged 6 months to 24 months as well as stunted children between 6 and 59 months fell by an average of only 2 percentage points from baseline levels (Table 26). The program made some progress reducing the frequency of stunting for male children six - twenty four months of age (- 9%), yet the frequency of stunting for girls in this same age group increased by nearly five percent. Similar changes occurred for the group of children 6 - 59 months of age; the proportion of stunting in boys was reduced by more than 5 percent; the prevalence of stunting for girl children increased by 1.5 percent. The results are disappointing - stunting is the result of long-term chronic malnutrition and is one of the main determinants of food security used by CRS. If stunting is not addressed by the time a child reaches 24 months of age, it may not be reversible. Lack of progress in this area suggests that CRS will need to look at additional and alternative methods to prevent child malnutrition before its effects become permanent. CRS made some progress reducing the number of children that are underweight. In contrast to the gender difference prevalent in the rate of stunting, data show that female children fared better than male children with regards to a reduction in underweight. The program’s goal was to reduce the prevalence of underweight children by 15 %; CRS was most successful at nutritionally rehabilitating children in the 6- 59 month age group (9%), specifically girls of this age (13 %). 26 TABLE 26: ANTHROPOMETRICS Age Group Stunted (%<-2) Wasted (%<-2) Underweight (%<-2) Baseline Final % change LOA target Baseline Final % change Baseline Final % change 6-24 months 48.2 46.0% (n= 100) -2.2 15% decrease 11.2 5.1 (n=99) -6.1 38 32.5 (n =114) -5.5 Boys 55.2 45.9 -9.3 8.1 5.3 -2.8 34.8 30.2 -4.6 Girls 41.1 46.0 + 4.9 14.3 4.9 -9.4 41.2 33.8 -7.4 6-59 months 54.7 52.8 (n=246) -1.9 15% decrease 8.1 6.6 (n=243) -1.5 43.9 35.0 (n=266) -8.9 Boys 56.0 50.5 -5.5 6.7 7.3 -0.6 40.1 35.7 -4.4 Girls 53.2 54.7 + 1.5 9.4 6.0 -3.4 47.3 34.4 -12.9 IR 2.1 Improvement of family nutritional practices Data show that over the life of the program the percentage of mothers that exclusively breastfed their children from birth up to six months of age rose appreciably. The program reached 95% of its goal, increasing the rate of exclusive breastfeeding for children less than six months by almost 16 percentage points. Also notable is the increased number of children that are being breastfed within one hour of birth. At the program’s inception, the initiation of breastfeeding was unfavorably delayed for 75% of children. Final evaluation data show that 50% of mothers now breastfeed their children within the recommended one hour of delivery, reflecting a 25% increase in this positive behavior (Table 28). Nearly all mothers in surveyed area provided their children with nutrient-rich colostrum at the time of the baseline; this practice has increased by four percent following program activities. The success of FELANA with respect to early infant feeding is quite likely due to the promotion of breastfeeding through group-based education that is targeted at mothers of young children. Participating communities have been able to share health and nutrition information and effective practices using HM/PD methodology. TABLE 28: BREASTFEEDING PRACTICES Baseline Final % change % exclusively breastfed < 6 months 69.40 85.00 15.6 % of children 0-6 months breastfed w/in 1 hour of birth 25 50 25 % of children 0-6 months fed colostrums 82.4 86.5 4.1 Evidence suggests that behavior change practices to reduce diarrhea were not as successful as breastfeeding awareness activities. Baseline diarrhea rates were near 22 %. The final evaluation shows that the rate had decreased to near 12 %, hitting 50 % of its achieved target. Review of the annual reports, which survey only program beneficiaries, shows very low rates (< 2%) of diarrhea. Although annual monitoring activities are TABLE 27: BREAKDOWN BY AGE AND SEX OF CHILDREN IN FINAL SURVEY 0-5 months 6-24 months 6-59 months Male 48.2 38.5 44.4 Female 51.8 61.5 55.6 N 114 117 275 27 conducted during the dry month of October, indicating the possibility of lower frequencies of the disease, these figures suggest that the program has had commendable success with program participants. It appears that Program FELANA activities have encouraged caregivers to increase liquid intake for children with diarrhea; final evaluation data indicate a 7% increase from the baseline. In addition to health and nutrition education using HM/PD approaches, key components of CRS’ efforts to reduce the prevalence of diarrheal diseases were to increase household access to potable water and household access and use of sanitation facilities. Latrine use (Table 29) appears to have increased dramatically, although even the baseline figures seem high for Madagascar. IR 2.2 Improved household prevention and management of priority diseases for children CRS has been successful and made considerable impact in health promotion and the establishment of illness management practices. Four of six indicators exceeded their targets and one came very close, reaching 90 percent of its goal. The indicator which did not reach its mark, percent of families drinking potable water, may be due to the damage wrought by Cyclone Ivan, which greatly compromised further access to water in program regions. Prior to program FELANA, many Malagasy families in the participating communities did not use health services. Long distances, poor services, and distrust of the modern health system were all contributors to limited reliance on basic health services. CRS addressed this challenge through reinforced, concrete educational health care campaigns directed at families. In addition to advancing hygienic knowledge and practices of caregivers, program attention was placed on improving household water and sanitation resources for the purpose of reducing the risk of childhood illness and disease. Latrine use Access to sanitation is limited for most households in Madagascar —baseline data show that only 40% of households in participating areas had access to an adequate latrine prior to program interventions. CRS surpassed, by four percentage points, its goal of increasing the number of households that use latrines; the proportion rose to 61%, representing a substantial gain in latrine use. The greatest increase occurred in households that moved from “defecating in the bush” to using improved latrines. Table 27 shows the results of the final evaluation in comparison to baseline values. TABLE 29: SANITATION IN AND AROUND THE HOUSEHOLD (% OF HOUSEHOLDS) Latrine Type Baseline Final Difference No toilets/in the bush 51.1 38.9 -12.2 Open/rudimentary pit latrine 9 11.3 2.3 Improved pit latrine 39.4 48.0 8.6 Ventilated pit latrine with shade 0.1 1.7 1.6 W.C. *** 0.0 Other 0.4 0.2 -0.2 Garbage Disposal (% of households) Throw it out close to home (within patio area) 44.8 21.3 -23.5 Throw it out far from home( outside patio area) 36.8 32.4 -4.4 Community dump 1.6 5.0 3.4 Bury it 14.5 36.0 21.5 Burn it 1.5 3.5 2 Other 0.7 1.8 1.1 28 The success of this intervention can be attributed to a combination of CRS’ provision of materials for the construction of pit latrines, community contributions of local materials and labor for the construction, and integrated educational sessions and behavioral change awareness campaigns. The program encouraged community “buy-in” by recruiting volunteer mothers to assist health educators with the promotion of hygiene, as well as by enlisting the aid of village elders and traditional health providers in awareness raising activities. After exposure to model program latrines, a number of communities used inexpensive, locally available materials to construct their own additional latrines. Program awareness campaigns were successful in encouraging behavior change with respect to safe garbage disposal. Baseline data showed that few households used proper methods to dispose of garbage. Unaware of the risk of environmental contamination, nearly all (80%) families simply threw their garbage outside of the home, within the patio or far from home. A considerable number (27%) of households moved to burying, burning, or using a community dump. The program appears to have had greater influence on families that were throwing household waste close to home, and not as much influence on households that were throwing garbage into nature far from home (See Table 30). Continued positive change will quite likely contribute to improved environmental health of households. CRS is making good progress in promoting changes in household disposal practices, but may need to alter its implementation strategy to explore awareness messages targeted at households that are disposing of garbage far from homes. Increase in Use of Potable Water The majority (73%) of Malagasy households with children between the ages of six and 60 months have limited access to potable water (Bergeron, 2001. p 153). Awareness of the necessity to appropriately treat and prepare potable water is, however, common. Baseline data indicates that 74% of respondents used potable water. For the purpose of improving health and preventing childhood illness and disease, CRS directed program efforts at improving, by 20%, the number of families that use potable water. Although CRS did not achieve its final goal, improvement (12%) had occurred at the time of the mid-term when about 86% of households reported using potable water. The FFE confirms potable water use for 77.1% of surveyed households. This increase was primarily due to increased knowledge on water preparation, and access to Sur’eau and parallel socially-marketed products. Limited progress has been achieved in access to clean protected wells (Patterson 2006). In spite of increased use of protected sources (7%) and an increase in the percentage of water needs covered (8%), final evaluation data indicate only a slight increase (3%) in the number of households using potable water (a 9% drop from the midterm). There is concern that this indicator from the start does not reflect the reality in rural Madagascar, and recall may be affected by month of data collection (different for each evaluation). Tables 31 and 32 detail household and per capita water information. In the baseline survey, 63% of households surveyed relied primarily on non-protected surface water for their drinking water; following program activities that number has dropped by less than one percent. Other results pertaining to household water access and use have seen little change; a slightly lower percentage of TABLE 30: WATER AVAILABILITY COMPARED TO LAST YEAR IN 3 MOST CRITICAL MONTHS OF THE DRY SEASON (% OF HOUSEHOLDS) Baseline Final Difference Improved 18.6 26.5 7.9 Deteriorated 42 17.1 -24.9 No change 39.4 56.3 16.9 Total 100 100.0 29 TABLE 31: AVERAGE QUANTITY OF WATER CONSUMED PER HOUSEHOLD AND PER CAPITA BY SEASON Daily water consumption (liters) per household Daily water consumption (liters) per capita % of Water needs covered Baseline3 Final Baseline Final Baseline Final Difference Dry season 56.4 76.0 11.7 15.9 23.4 31.8 8.2 Rainy season 50.5 68.2 10.4 14.6 20.8 29.2 8.4 Yesterday 51.6 68.8 10.7 14.8 21.4 29.6 8.2 Total 52.8 71.0 10.9 15.1 21.7 30.2 8.3 households now use non-protected wells or taps (32% in the baseline compared to 25% in the final evaluation) and, the number of households that rely on non-protected surface water for their main source of drinking water has increased slightly. Table 32 below shows the main source of drinking water for households surveyed in the final evaluation. Additionally, the time spent fetching water has increased by a small amount (avg. 1 minute) for all sources of water in both rainy and dry seasons (See Table 33). TABLE 32: SOURCE OF DRINKING WATER DURING RAINY VS. DRY SEASON (MAIN SOURCE ONLY) (%HHS) Baseline Final Difference Rainy Season Protected Sources 6.4 13.5 7.1 Non-protected source, surface water 63.3 61.7 -1.6 Non-protected source well or tap 30.3 24.8 -5.5 Total 100 100.0 Dry Season Protected Sources 6.9 12.8 5.9 Non-protected source, surface water 59.9 63.0 3.1 Non-protected source well or tap 33.2 24.2 -9.0 Total 100 100.0 Combined Protected Sources 6.65 13.2 6.6 Non-Protected source, surface water 61.6 62.4 .8 Non-protected source well or tap 31.75 24.5 -7.3 TABLE 33: MEAN TIME IN MINUTES TO FETCH DRINKING WATER BY SEASON Baseline Final Difference Rainy Season Protected Sources 3.2 4.2 1 Non-protected source, surface water 8.4 10.8 2.4 Non-protected source well or tap 4.4 4.2 0.2 Average of All sources 5.3 8.3 1.2 Dry Season Protected Sources 5.3 3.9 1.4 Non-protected source, surface water 11.9 11.5 0.4 Non-protected source well or tap 4.5 5.6 1.1 All sources 7.2 9.3 1 3 When conducting the final evaluation analysis, small discrepancies were noted with the baseline figures for household water consumption. Upon reviewing the baseline data, it became apparent that there was a coding error resulting in artificially low baseline numbers. These figures represent the re-created baseline numbers for household water consumption. 30 Increase in Vaccinated Children Full immunization is a key element to the prevention of child specific illnesses. While many children had received some immunizations at the time of the baseline survey, few (30%) had been fully immunized. The percentage of children 12-24 months that are fully vaccinated is now up 21% from baseline levels, to an overall rate of 51%. This net increase surpasses the program’s goal of a 15% increase. Contributing to the success was the promotion of vaccination through HM/PD methodology and integrated IMCI-related education, as well as collaboration with the Ministry of Health in the implementation of Vaccination Days. Table 34 below shows vaccination coverage for households surveyed in the final evaluation. The first value indicates children with and without cards. The second value, in parenthesis indicates children with confirmed vaccination cards, and the third value indicates caregiver recall. TABLE 34: FINAL VACCINATION COVERAGE RATE Baseline Full Vaccination Final Full vaccination % change 12-24mo 29.8 (43.0) (10.3) 50.6 (52.9) (33.3) 20.8 24-36mo 30.0 (39.4) (16.3) 32.8 (31.9) (36.4) 2.8 36-60mo 30.1 (44.6) (19.6) 54.3 (55.4) (50.0) 24.2 12-60mo 30.0 (42.4) (16.0) 47.2 (48.4) (41.7) 17.2 Increase in Children using Mosquito Nets One of the key successes of the DAP was increasing preventative practices against malaria - the number one cause of under-five mortality. CRS’ impact in this area is a great achievement and contributes to the MAP’s goal of effectively controlling this disease that leads to death for approximately 20,000 Malagasy children each year (PMI 2007). There was a net increase in the number of children using mosquito nets which brought the program to 106% of the final objective. This can be attributed to CRS’ sensitizing and educating caregivers about the benefits of treated mosquito net adoption, in collaboration with Population Services International’s (PSI) provision of treated nets. Program FELANA provided model mothers with nets for the purpose of promoting and demonstrating their use during HM/PD sessions. While the majority of caregivers protected their children under five with a bed net at the time of the baseline survey (60%), CRS has made substantial progress in improving this preventative practice against malaria, bringing the rate up to 90% of responding households. Moreover, prior to program intervention, the vast majority (96%) of children were not sleeping under insecticide treated nets (ITN), limiting the effectiveness (Bergeron, 2004). The nets provided by PSI and CRS are impregnated with insecticide. In 2006, activities to support the MAP included the distribution of more than 1 million bed nets in vulnerable zones by the MoH. CRS’ awareness activities surely enhanced this MAP strategy and future efforts in this area will serve to support the President’s Malaria Initiative (PMI) and the newly adopted Malarial Operational Plan (MOP) (PMI 2007). 31 Mother and child using mosquito nets to prevent malaria The result of mosquito net use by families to prevent transmission of malaria to mothers and children is below CRS’ expectations, yet still represents 68% of the targeted goal. CRS Annual Report for FY07 states that the strong resistance to net use by the population of one of the four dioceses has contributed to the consistent difficulty in meeting this objective. This resistance was resounded in the qualitative FFE and payback of nets by middle-men sellers to FELANA was often problematic. Stories were told of entire village vanishing when agents came to collect the cost of the nets! Unpaid nets can difficultly be made known to enumerators! Continued efforts to combat this major health problem are imperative. The vast majority of the Malagasy population (90%) is at risk and in the coastal areas of Program FELANA, transmission is perennial. Most vulnerable to the disease are children under five and pregnant women (PSI 2007). Future CRS interventions can complement the PMI and MOP designated activities for the eastern part of Madagascar; continued ITN distribution, community education, case management, intermittent preventative treatment of pregnant women, and home-based management of fever (PMI 2007). Mothers Seeking Health care for Fevers Roughly two of every three mothers sought appropriate care for children with fevers prior to program interventions. Following five years of BCC activities and health and nutrition education, that number has been increased to close to 95%. This suggests that caregiver’s knowledge and recognition of the signs and symptoms related to fever has increased. Since the midterm, CRS has substantially increased the proportion of mothers that seek appropriate medical care for their children; the figure at that time had dropped to 39%. Programmatic and technical obstacles contributing to the disappointing midterm results have been resolved. Furthermore, the prevalence of children with fevers has been decreased by 15% since the baseline survey. Differing months of data collection may partially explain these fluctuations. With respect to other childhood illnesses, progress has been less than CRS programmed. Tables 35 and 36 show that the prevalence of most sicknesses has dropped only slightly (5%), and that the prevalence of Acute Respiratory Infections (ARI) has risen by close to 6%. Again, differing months of data collection may partially explain these fluctuations. TABLE 35: BASELINE PREVALENCE OF CHILDHOOD ILLNESSES Age of Child Diarrhea4 ARI5 Fever Any Sickness6 Multiple Sicknesses (months) % 6-12 43.9 15.2 48.5 66.7 33.3 12-24 27.6 9.2 48.3 62.6 19.7 24-36 23.4 12.5 45.0 58.6 19.7 36-60 9.2 10.8 37.4 45.7 10.9 Total 6-60 22.4 11.2 43.9 56.5 18.4 4 Diarrhea was defined as having three or more runny stools per day or blood in his/her stools in the last two weeks. 5 To be classified as having ARI, the child had to have been reported to have a cough and difficulty or fast breathing. 6 Any sickness refers to a child having been reported to have an episode of cough, diarrhea, or fever in the two weeks prior to data collection. 32 TABLE 36: FINAL EVALUATION PREVALENCE OF CHILDHOOD ILLNESS (N) Age of Child Diarrhea7 ARI8 Fever Any Sickness9 Multiple Sicknesses (months) % 6-12 28.6 (28) 20.6 (34) 28.9 (28) 59.4 (32) 19.2 (26) 12-24 18.0 (61) 8.8 (68) 28.6 (61) 50.0 (62) 12.7 (55) 24-36 10.7 (56) 21.0 (62) 36.8 (56) 55.0 (60) 21.6 (51) 36-60 7.0 (71) 19.0 (84) 26.0 (71) 47.3 (74) 5.8 (69) Total 6-60 13.9 (216) 16.9 (248) 28.9 (270) 51.8 (228) 13.4 (201) E. SAFETY NET CENTERS SO: 3 Promote and maintain basic human dignity of the most vulnerable members of societies Initially in response to the virtually non-existent state support of Madagascar’s most vulnerable - orphans, mentally and physically handicapped persons, the homeless, prisoners, the aged and dying with no family support, and the many single-parent families that exist on less that US 30 cents per day - CRS included the provision of critical nutritional and institutional support to safety net (SN) centers as a key area of focus for Program FELANA. The word ‘critical’ is important here, and the term ‘safety net’, if correct, does not do justice in describing the FELANA life line offered to so many over 22000 beneficiaries. Dynamic changes in the GOM have resulted in a commitment to National Solidarity; part of this commitment is outlined in the MAP’s objective to Improve Support for the Very Poor and Vulnerable Populations (IMF 2007). Designed to guarantee equitable and adequate services for those that have difficulty meeting their basic needs, SO3 augments the national policy. The existing presence of solid safety net institutions was a contributor to the program’s geographic, partner, and individual selection. Over one hundred, mainly urban, institutions were targeted in six dioceses. SO3 intended to build on the existing relationship that CRS had established with 74 general relief centers by expanding the provision of food assistance and organizational support to 25 additional centers. In addition to ensuring minimum nutrition of those residing at the safety net centers and institutional capacity building activities, the program designed its assistance to provide education and skill training for the purpose of developing income-generating activities that could increase the financial sustainability of the centers as well as the financial independence of individuals. Safety net beneficiaries Program FELANA had achieved and substantially surpassed its LOA target by the end of FY07. Final evaluation data indicates the total number of safety net beneficiaries at 22,094, almost double the stated target of 12,500. CRS had additionally surpassed its final objective of 100 SN centers by the end of FY06. The 104 centers that Program FELANA supports are diverse: orphanages, hospitals, prisons, women’s shelters, leprosariums, handicapped facilities, mental asylums, and hospices for victims of AIDS. The majority of the centers and beneficiaries are located in the diocese of Antananarivo (Table 37). 7 Diarrhea was defined as having three or more runny stools per day or blood in his/her stools in the last two weeks. 8 To be classified as having ARI, the child had to have been reported to have a cough and difficulty or fast breathing. 9 Any sickness refers to a child having been reported to have an episode of cough, diarrhea, or fever in the two weeks prior to data collection. 33 Beneficiaries consistently provided with food assistance All who receive benefits from SN centers in CRS/MG’s area of implementation reside at the centers and receive a cooked meal on site; food assistance is vital to the beneficiaries’ daily nutritional requirements and sustainable health benefits. There are several organizations contributing to SN programming in Madagascar, yet CRS is the sole provider of food assistance to the vulnerable populations residing at SN centers. By the midterm, CRS had surpassed its target of consistently providing 16,000 beneficiaries with food assistance. FY08 data indicate that the program has burgeoned to reaching more than 22,000 beneficiaries. Corn soy blend, rice, pulses and vegetable oil were common commodities distributed. IR 3.1 Increase level of food availability to people served in 100 Safety Net centers Rations distributed The substantial increase in beneficiaries resulted in an increase in the number of distributed rations. At the end of FY07, the programmed had already reached 138% of its initial LOA target, representing 919,119 distributed rations. Credit goes to both the CRS Commodities Unit and the JPBS Unit for competently analyzing and managing the food distribution necessary to meet the high demand of the many new beneficiaries. IR 3.2 Improve income-generating skills of SN Centers beneficiaries Beneficiaries trained and involved in Income Generating Activities (IGA) Creating confidence and opportunities for economic development is a main element of CRS’ support to the vulnerable populations residing at SN centers. This objective is directly in line with MAP’s Commitment 6 to High Growth Economy via the promotion of full employment and the strengthening of domestic enterprises (IMF 2007). In addition to institutional capacity building and ensuring nutrition for SN center residents, the program designed its assistance to provide education and skill training for the purpose of developing income-generating activities that could increase the financial sustainability of the centers as well as the financial independence of individuals. The number of beneficiaries trained and involved in IGAs has steadily increased since the program’s inception. Almost 470 participants were developing income generating skills in year one; by FY07 that number had increased to 1944. Even with budgetary constraints, this indicator has shown a growth rate of near 4% for the past two years suggesting that it will quite likely reach its target of 2000 trained and involved beneficiaries by the end of the program cycle. Income generating activities have included establishing a fair trade relationship between a US-based crafts organization and one SN center that manufactures backpacks - opening the international market to the participants; assisting vulnerable women with courses on sewing, embroidery, cooking and basket weaving; and providing professional training in aviculture and agriculture. TABLE 37: NUMBER OF CENTERS AND BENEFICIARIES DURING FY07 Diocese Number of Centers Number of Beneficiaries Toamasina 16 1,634 Antananarivo 33 6,992 Antsirabe 16 3,847* Fianarantsoa 19 4,551 Farafangana 10 3,429 Mananjary 10 1,641 TOTAL 104 22,094 *Corrected typographical error from FY07 annual report 34 IR 3.3 Reinforced partner capacity to support SN activities Safety Net centers receiving assistance from Program FELANA are divided into three categories: • Category A- those that currently include IGAs and training activities; • Category B- centers that have the potential to include a developmental component; and • Category C - welfare centers whose purpose is to provide only assistance due to the severe state of vulnerability of the residents. IR 3.3 revolves around CRS’ commitment to help 48 Category B centers that were not currently undertaking development and training activities progress to a level “A” status. These centers typically had resources and geographical proximity to enabling conditions (i.e. useable farmland, workshop space, etc.), but had not yet undertaken a development component to benefit participants and help offset operational costs. In FY07, the number of SN centers Program FELANA had helped to reach level A status was 47, representing 98% of the LOA target. The most pressing final concern, however, of CRS and her partners near and far (some not at all associated with CRS but aware of their impact in the sector), is that most of these 104 centers will in a few days be falling into a transition with no food pipeline. CRS, despite valiant efforts to identify a fall-back, albeit temporary solution, is unable to continue distributions to these SN centers after September 2008 until the next DAP is confirmed. The centers’ fledgling IGAs (for the A centers) unable to sustain the costs of daily rations, most administrators will be forced to make extreme cut-backs, feeding fewer or less, until other safety nets are identified. For many concerned with vulnerable populations, this is a humanitarian outrage. 35 5. PROGRAM MANAGEMENT A. RELEVANCE The bearing FELANA brings upon the expressed needs and desires of both communities and the national commitment to development is important. CRS/Mg created FELANA within a vision of building upon the experiences gained during the first DAP proposal to improve food security. Out of that vision for a more food secure future, the program has woven together a set of integrated interventions that stresses agricultural and marketing development as well as risk reduction and nutritional education. At the national scale FELANA has relevancy as it responds to elements contained in the National Poverty Reduction efforts (DSRP). It specifically responds to that document’s Strategic Objective No.2 to “create and promote economic growth on a wide social scale;” and SO No.3 to “create and promote systems for human security, both materially and socially.” For SO No.2, the specific goals are to: develop agricultural production; improve food security; and preserve the environment and the sustainable management of renewable natural resources. The goal for SO No.3 is to promote the health of mother and child. These all clearly fall within the scope of FELANA activities. FELANA also fits within the Rural Development (PADR) framework, also at the national level. PADR and FELANA share agreement among objectives and orientations, namely ensuring food security, expanding and promoting agricultural production with optimal use of resources and infrastructure, implementing legal and institutional reform for better management of rural sector and developing social infrastructure to ensure access to social services. Most recently, the Madagascar Action Plan (MAP) makes eight commitments to assure rapid development between 2007 and 2012. FELANA already contributes to each of these, and the impact is likely to grow now that the seeds are planted. The eight MAP commitments are below, each followed by FELANA’s current contribution: 1. Responsible governance, FELANA engages local and communal authorities in the process of risk reduction and general development 2. Connected Infrastructure, FELANA engages communities in the creation and restoration of local infrastructure, for both agricultural and sanitation 3. Educational Transformation, FELANA enhances agricultural, marketing, health and other knowledge and builds skills in new sustainable techniques 4. Rural development and green revolution, FELANA generally builds-back-better from the bottom up and has improved rice yields and increased vegetable gardens 5. Health, Family Planning and the Fight against HIV/SIDA, FELANA reduces malnutrition and improves health via vaccination, breastfeeding and mosquito netting 6. High Growth Economy, FELANA increases the value of household production, assets and commercialization 7. Cherish the Environment, FELANA protects agricultural resources crops reforesting and embellishing the national heritage 8. National Solidarity, FELANA builds pride in locally-available resources fit for both consumption and commercialization. 36 CRS/MG FFS activities align well with Commitment Four of the MAP working towards rural development and a green revolution. The breakthrough reform initiative under this commitment seeks to facilitate the green revolution through provision of seeds, fertilizers, and technical support to farmers as well as establish cooperatives to collect and distribute harvests. Activities under this commitment will also include expanding land-use, mechanizing and industrializing agriculture, and encouraging crop rotation and diversification. B. PARTNERS FELANA depends on multiple networks of implementing partners. The main partners are those at the Diocesan level who manage FELANA activities for the entire zone (from 15-60 communities, each). Another set of partners is the administration unit of each Safety Net Center (N=104). The Government of Madagascar (GoM), especially at the regional and communal levels, is also a strategic and key partner –but one often side-tracked-- in achieving sustainable results. Dioceses: There is only one main CRS partner for each of the four FELANA Dioceses: two of these are managed by Caritas (within the diocesan hierarchy) and two are newly erected quasi-independent structures. The diocesan partners and regions are listed below: 1. CARITAS / SFK Antsirabe 2. CARITAS / Farafangana 3. BDEM / Mananjary 4. ODDIT / Toamasina Safety Net Centers: Over one hundred centers within 6 Dioceses are active partners (and beneficiaries). Within the four Dioceses mentioned above, the Safety Net Centers are most often managed by the implementing partner designated by the respective Diocese. Evaluation: In the qualitative questionnaire administered to 10 CRS senior staff, 30% felt that the management processes, as they are now, have been inadequate to fully meet the objectives of a program such as FELANA. Efficient management structures, according to the qualitative survey of CRS/MG senior staff, is the weakest link of the FELANA program (averaging a score of only 2.5 out of 5, where 1 is the best). As an impediment to achieving certain results, this issue is of serious concern to CRS. The worldwide dilemma is a daily one for CRS and concerns how to capitalize on the existing accessible network (the Catholic Church, complete with volunteerism and benevolent charity) to meet the needs of some of the most poor communities in the world while being fully and transparently accountable and professional in attracting and responding to donors to produce specific processes and impacts. In general, the four partners and their teams have increased their skills 100 times over during the past five years. They have been taught (often against their instincts) to jump through difficult administrative hurdles, characterized by heavy procedures based on theories that are not always grounded in practice. Furthermore, partners lived the reality of needing to satisfy many ‘masters’: the Diocesan Bishop and the Catholic hierarchy, USAID-driven CRS requirements and the urgent call of needy communities. It is not surprising that there were upsets, difficulties and misunderstandings. 37 Especially in (but not limited to) the most rural and distant of these Dioceses, the additional difficulty of finding and maintaining (under both budgetary and diocesan restraints) serious, well-trained and experienced technical agents (Coordinators) was a recurrent problem. The impact of frequent changes in technical personnel was felt even at the level of community commitment and evolution. Partners were curious to learn if a partner agent playing the same role in another Diocese earned a higher salary or if the other diocese had the same level of impact. Regarding Government of Madagascar buy-in of FELANA, the results were disappointing, especially at regional or wider levels. It was difficult to find agents who could speak intelligently about the FELANA efforts beyond the RDM/PACOM Infrastructure –the program’s most restrained component (with only 42 communities and 5100 households benefitting). Certain partners seemed convinced, however, that the inclusion of communal authorities in trainings and ceremonies would result in their taking over the program interventions, but this is very uncertain without a clear exit strategy stipulating their role or a protocol confirming their commitment. C. EFFECTIVENESS AND EFFICIENCY According to the MTE, effectiveness (here, defined as the degree of success in achieving indicators), was weak due to slow start-up and scaling-in of interventions. Although the process indicators were roughly on target for the mid-term, it was feared that even slight delays in process would hinder achievement of impact for the life of activity (LOA). In Table 38 below, the FFE finds that FELANA has largely maintained the process planned, and has achieved global coverage roughly equivalent to that desired. The exceptions are two: fewer beneficiaries than targeted in the health component (17% fewer than planned), and fewer Safety Net Centers that were able to develop IGAs, and thus be classified as an “A” Center (29% fewer). The health deficiency is all the more surprising because of the large volume of field workers attributed to that component, by partners. By 15 September 2008, CRS had committed all FELANA funds – a 100% level of expenditure. Given the total number of PITT indicators at 42, 20 (process and impact combined) have been achieved in full and another 14 have improved since the baseline. This gives a total achievement rate of at least 80%. The ratio between levels of expenditure and achievement from MTE is significantly better in the final (65:35 to 100:80). TABLE 38: NUMBER OF BENEFICIARIES 38 D. INTEGRATION AND COMPLEMENTARITY OF INTERVENTIONS / APPROACHES ‘Integration’ has been the most frequent term used to describe FELANA during the FFE, over and over again. As an index of integration, the percentage of total program communities in which all four FELANA activities were implements was calculated (Table 39). Out of the 142 communities, 42 (or 30%) are fully integrated, compared to 11 and 8% respectively for Farafangana and Mananjary at the time of the Mid-Term Evaluation. TABLE 39: INTEGRATION AND DISTRIBUTION OF EFFORT BY COMPONENT It appears that this type of inter-program integration is most forthcoming when the geographical targets are smaller or more compact. Again in the FEE, the greatest level of integration occurred in the Diocese of Mananjary, the Diocese with the smallest number of total communities (N=16). The majority of the marketing or RDM activities also represent the most recent add-ons in the five-year program. It remains to be assessed quantitatively if a fully-integrated community intervention systematically produces a measurably better impact. These data only speak to the level of coverage of components in the different regions; they cannot be used to infer anything about participation or adoption in communities. In fact, the program was designed as an integrated approach to food security and the value of this complementarity rings forth from every sector and partner, even from the community-level beneficiaries (see Text box). When prodded, these same communities provided some colorful examples of how this ‘integration’ manifested itself: • RDM (PACOM)-driven rehabilitation of rural roads that both protect crops and encourage mothers to finally vaccinate their children (access to the health center now much more manageable). Inevitably, they open up opportunities for product marketing. • FFS training produced vegetable gardens that both supplied foods to be used by FARN mothers and additional revenue (from vegetables sold) for households. • Improved rice yield permits larger volume to be commercialized, thus reinforcing revenue. • Safety net centers benefit from agricultural efforts that in turn provide more sustainable food supply chains to beneficiaries. Huge increases in manioc yield at Penal Camps in Mananjary result in surplus supplies which, when sold, can help purchase soap or other IGA linked materials. Household-level integration: “Entire families have benefitted from FELANA integration: the father attends the FFS, the mother, a FARN, the older son participates in PACOM-FFW or markets the additional produce, the younger son attends school (due to the increased hhld. revenue) and the baby is breastfed longer”. 39 In summary, although integration is not an end in itself, in most areas of Madagascar it still makes good sense. The absence of certain components in a FELANA community has even resulted in a good dose of jealousy. As an example, when marketing efforts were neglected in a diocese, a partner had to provide very judicious on-the-spot explanations so as not to discourage the motivated households. E. EXIT STRATEGY Typically an exit strategy (phasing down, phasing out or transfer) is one that allows a program to plan for the end or, inevitably better, the transfer, of efforts at a given date in the future. The premise is that interventions, by definition, are unsustainable and must come to an end. Entire programs are evaluated by proving that the efforts and better yet, their impact, will continue once the ‘intervention’ stops. The big challenge is finding a way to prove this. Rarely is an intervention evaluated five years after it ended –thereby producing a more thoughtful and careful measure of the true impact. Development is a long-haul journey and sustainability (see next chapter) is an art-form. In 2006, CRS made a clear move to encourage the partners to conceive of, document and implement their own ‘exit strategy’. In July of the same year a workshop in Mananjary was organized to catalyze the ‘exit strategy’ process. In the end, it appears that the definitions or interpretations of the term ‘exit strategy’ are numerous and it was most simple for a partner to chose one so flexible that at the end of the day, “there is no exit at all” (See Text box). On the other hand, it is true that a program of such intensive training and capacity building such as FELANA is very favorably setting itself up for an easy exit, thus the quote (see inset) “the entire program was conceived as an exit strategy”. While CRS was most careful not to issue an ‘order to exit’ to partners, however, the results are diluted and confusing. Confusing for the evaluators very likely means more confusing for the beneficiaries. Some communities visited had loudly heard and clearly understood the message : « we have started taking over the activities ourselves ». Others might hear but never listen to such ‘unfavorable news’, demanding external assistance ad infinitem. To each community, her own character and perception of the world. In other communities, all interventions appear to have entirely been put on hold at the time of the final evaluation. What become sorely clear, however, is that the (preliminary desire to) use a hasty or ill-prepared exit strategy to explain a few deficient indicators was just not going to happen. Many FELANA exit strategies appear to quite simply comprise the following efforts: - Transfer of competence by certified theoretical and practical training of volunteers and leaders - Daily (regular?) supervision of beneficiaries and their leaders - Strategic engagement of key local and communal authorities in the program from Day 1 - Committee development around key personalities that are trusted and likely to resist turnover - Supply manuals / documents / calendars to community agents at the appropriate level of understanding - Recognition and graduation ceremonies to volunteers and leaders. “What exit strategy? We are still here! and CRS or not, we’re not going anywhere!” A partner Or “The entire program was conceived as an exit strategy!” A partner Or “With or without an exit, our life-line may never make another entrance…” A Safety Net Center 40 Many learning initiatives have been conducted that reinforce management and a potential exit: o Rencontre Nationale Intégrée: Completed at the end of each fiscal year, the main objective is to define the Annual Work Plan. It is also a rich opportunity for partners to share their experiences. o Rencontre Technique Nationale Intégrée: Performed in the middle of fiscal year: to share technical aspects and reorient activities. o CRS Integrated field visits (at the beginning of fiscal year, after annual assessment): All components technicians made a visit to each partner aiming to share and discuss finding from annual survey, conduct quality control on activities and to establish a common workplan (IP CRS) to improve quality of intervention. o FFS workshop with ministries and other organization implementing or promoting FFS (FIFAMANOR, FAO, IPs, …). What is most evident, however, is that FELANA or not, the partners are in place and have teams with new sets of useful skills --ready to continue development work with other donors. Given the recent ambiguous transition underway, however, many of these key newly trained agents will disappear into other efforts. All in all, this is completely understandable and not necessarily negative; CRS should take credit for this as it offers CRS, itself, an exit strategy -- into other dioceses or with other partners, should the future strategy so dictate. F. SUSTAINABILITY AND DURABILITY OF IMPACT According to the qualitative survey of CRS/MG senior staff, durability of FELANA’s impact in 2008 is the least convincing of the program characteristics (it scored 2.75 out of 5, 1 being the best). Most respondents feel that the necessary efforts are slowly aligning, but that five years have been insufficient to ascertain lasting impact (See text box). Lining up alongside the less than convincing sustainability, according to the same respondents, are issues of inefficient management structures and inadequate (even if slowly improving) technical capacity of partners. One example of sustainability nonetheless surfaced when a ‘control’ community was visited during the qualitative evaluation. In fact, it was discovered that the community had benefitted from FELANA interventions two years prior, but that the efforts had ended prematurely. The President of the fokotany and three women described in great detail the successes of the efforts ending in 2006: FFS, FARN and PACOM. Only the apiculture seemed to have failed due to hives provided without skill training. Still reeling from the effects of FELANA two years back, the villagers detailed the improvements in malnutrition and cyclone destruction across the community. Many other examples of sustainable impact were visible throughout the four dioceses: continued efforts to enhance product commercialization, associations created ad hoc and household contributions collected to maintain infrastructure. Most areas manifested a clear increase in not just mastery but also adoption of techniques (see above) and in general knowledge about health and nutrition. It remains to be seen, however, if this new knowledge will withstand the test of time, especially if resources prove insufficient to apply the skills on a regular basis. SUSTAINABILITY? “the first year we learned the ropes, this last year, we have had to orchestrate our exit -- FELANA only really lasted 3 years.” A partner 41 6. UNANTICIPATED RESULTS (NOT CAPTURED BY IPTT INDICATORS) Sometimes the most interesting results of a program are those that were unplanned. Planned and quantitative results only go so far in understanding the true effect and impact of a 5-year program such as FELANA. This chapter describes some of the unanticipated results –both favorable and unfavorable- of the program. At times, it is difficult to discern an unanticipated result from one logically resulting due to the approaches and interventions chosen. In this case, however, the results below portray at the very least those that are not typically captured or well-appreciated by the 42 PITT Indicators. A. FAVORABLE The word fihavanana is legacy in Madagascar. Those who had the good fortune to wander around the remote and rural corners of the country since the early 1990s will have indubitable stumbled upon this word repeatedly. It represents the pristine image of solidarity despite extreme poverty, of a Malagasy who is compelled by generations of benevolence to help his neighbor, cousin or a stranger even when life is tough enough for him/herself. It was often thrown up as a way to explain many a humanitarian act --other times to proudly unite the Malagasy, set aside from the individualist and more self-centered vazaha. Times appear to have changed in Madagascar. Fihavanana is no longer common or even systematically visible. In fact, one of the unanticipated results repeatedly expressed by qualitative research teams in every Diocese is that fihavanana is back, due in part to programs like FELANA (see Text Box). As mothers are now ’forced’ to work together on the FARN meetings, and farmers in the FFS, the concept of working together is back in fashion. In every community newly formed groups are working together, taking on creative personalities and novel initiatives. As one example, FELANA’s has catalyzed a group of FARN mothers to develop an association to further their aims. Within a very individualistic community, a group of mothers amplified their collaboration around the health of their kids. FFW efforts catalyzed a coordinated effort never before noticed in a village. Neighboring communities that have the opportunity to appreciate FELANA results are often filled with envy and many copy FELANA techniques. On another slant, the presence and renown of FELANA activities has brought visibility and clout to the implementing partner, and other donors are now courting them, too. Some communities refuse any other actor to develop interventions without the partner’s agreement. Furthermore, the community groups organized under FELANA serve also to advance interventions of other actors in the area. Many single mothers exist in Southeast Madagascar and according to local tradition, they cannot own or inherit land. Although they used to travel vey far to work the fields of distance villages, with their FELANA￾learned skills, many now can find better paying agriculture work closer to home. Our community is more unified than ever before. A climate of confidence has returned to the village and crime has diminished. A FELANA Community 42 It is recognized that the inhabitants of FELANA communities are now better equipped to organize themselves and be managed, and this greatly assist local authorities. Another favorable effect of FELANA was the development of a listing of each household in the community. This list is kept by local authorities and serves multiple purposes. B. UNFAVORABLE Fewer unfavorable effects were identified than favorable ones. In many cases these entail infrastructure that is difficult or impossible to maintain because the procedures did not follow set norms. Other times, a village was so disappointed because of the absence of FELANA that they catalyzed a “social division among the Catholic Church” in the commune. Politicians have been known to take credit for the FELANA infrastructure. Food distribution at Safety Net centers continue to create a sense of dependence and high expectations. 7. LESSONS LEARNED AND CONCLUSIONS CRS‘s FELANA has been a successful integrated food security program in Madagascar. Overall, FELANA has a total achievement rate of at least 80% (Table 40) and has made a few remarkable strides towards a better future for many. Given the total number of PITT indicators at 42, 20 (process and impact combined) have been achieved in full and another 14 have improved since the baseline. The two components with the lowest number of indicators had the highest achievement rate: Marketing and Risk/Disaster Management. The Safety Net Component had a full achievement rate of 60% while Agriculture, with the highest number of indicators to achieve, had a rate of 47%. TABLE 40: FELANA ACHIEVEMENTS Component INDICATOR STATUS VIS A VIS LOA TARGET Process Impact Total Fully Achieved Improved Needing Improvement TOTAL 21 21 42 20 14 8 Global Indicators 1 1 2 0 2 0 Agriculture 8 11 19 9 3 7 Marketing 1 1 2 2 0 0 Health / Nutrition 6 6 12 4 8 0 Risk / Disaster Mgmt. 1 1 2 2 0 0 Safety Net 4 1 5 3 1 1 Noteworthy FELANA strides providing strong evidence for CRS impact on food security include: • Increase in the average number of months with adequate access to food from 5 to 6 months • Increase in total value of assets by 50% with: o 30% increase in the value of household production, o 13% increase in value of domestic assets, o 57% increase in value of livestock , and o 62% increase in value of productive assets. • Improved adoption rates for almost all agricultural and NRM techniques • Appreciable increase in exclusive breastfeeding; 50% of mothers starting within one hour of delivery • Full vaccination (children 12-24 months) up 21%, to an overall rate of 51% • Net increase in use of mosquito nets for children (now 90%) • Safety net beneficiaries at 22,094, almost double the stated target of 12,500 43 The main lessons learned / conclusions are organized under the following headings: M&E, Governance, Safety Net, Risk and Disaster Management, Integration, and Management / Technical. MONITORING AND EVALUATION Baseline surveys are useful in so much as they are conducted by agencies that have an intimate understanding of the local context. They cannot be copied / pasted from another setting. The baseline in Madagascar was much more in-depth (and therefore more costly in budget and time/effort) than needed either for programming or to produce the contractually obligated indicators. In fact, CRS programming was largely set into motion prior to the baseline collection in December 2003. Most importantly, no document was left in CRS memory (or that of the contracted evaluators) to guide future users and M&E personnel to understand the exact definitions and measures used. Although the sample size was developed to provide indicators representative for CRS/Madagascar globally, there is good reason to collect less information from an even wider sample representing the individual partner areas, thus engaging each partner in the quest for a measureable impact of effort. Finally, CRS is ready to manage a simple rapid monthly information system that can use technology to monitor both process and impact more carefully. Needs Assessments, must be grounded in reality (rather than theory) and would benefit from further strengthening. They have insufficiently demarcated the populations where interventions are to be planned. Annual assessments were conducted to assess impact at the level of beneficiaries. In 2007, results of this survey were disaggregated by each partner to allow them to learn about impact. Nonetheless, limits of these annual surveys were that they concern only direct beneficiaries level (figures are not valid for whole communities) and malnutrition indicators collected during regular weighting sessions (not representative). GOVERNANCE Although communities are pruned and guided to take increasing responsibility of their own futures, government buy-in at regional and national levels is largely lacking in FELANA. The greatest impact of FELANA within the Government of Madagascar is at the commune level, where commune level agents are often trained and engaged in the program. At the Regional or National levels, however, there is very little awareness of CRS and partner efforts that underline the huge contribution FELANA makes towards the MAP, in the numerous areas of intervention. SAFETY NET CENTERS AND UNMET URBAN AREAS As the name “Safety Net” does not underline the importance of the component, it is essential to recognize the unique contribution of the Safety Net Center approach. What CRS and partners contribute to 104 centers is not a safety net, but rather a lifeline, one that, in the majority of cases, cannot be replaced permanently by any other entity. This is the only component reaching the poorest or most vulnerable with almost 100% certainty. The entities supported merit attention as more primary partners of CRS and the huge effort to create and maintain income generating activities among these needs to be featured more loudly in any future Food Security programming. Progress has been made and much is left to be done, on a greater scale and more systematically. 44 A particular unmet need and ‘niche’ for CRS is the hundreds of prisons in Madagascar; a set of which are already targeted with success by CRS. This urban target is largely neglected if not addressed haphazardly in less than efficient ways, by national and international NGOs. RISK & DISASTER MANAGEMENT (RDM) Natural hazards continue to plague Madagascar and CRS has a long history of experience in responding to them. The inclusion of RDM as a main component of FELANA was a laudable and important move demonstrating the desire to prevent, mitigate and reduce risk. The use of PACOMs and natural resource management alongside infrastructure enhancements was also strategic. Although a heavy and logistically-sensitive component, it would now benefit from being mainstreamed in all zones of intervention, even where cyclones are least frequent. INTEGRATION On the tip of many tongues, this term means many different things to many different people. Integration is often used synonymously with complementarity. Evaluators got the clear impression that a package of food security efforts made inherently more sense to the communities than a single technical effort. Although this may be due mainly to good targeting of FELANA interventions (to the poorest of the poor in a very poor country), most respondents agree that integration is not the end in itself, but, like approaches, a means to an end: improved food security. It is essential that to assure integration, appropriate needs assessments are conducted at the community level (see above). MANAGEMENT & TECHNICAL CRS-partner relations are besought with tension. Consensus seems to recognize major differences between different types of partners (Caritas versus independent). Partners have learned a lot, but not always what they felt they needed most. They seemed to consider technical visits from CRS sometimes too frequent, other times too infrequent and occasionally an unpleasant surprise. Partners felt that too much information was requested with procedures that ill-reflected the realities of the field, but that despite all the challenges “we will do our best to improve our skills and satisfy CRS and the donor”. The main technical components of CRS not mentioned above are on the right track. The Agriculture, Marketing and Health components have a strong set of technicians whose skills could be channeled to further strengthen the field work. Additional enhancements to component programming may include a central stock (seed bank or vet-pharmacy) at the level of every Diocese to facilitate interventions relying on the provision of time-oriented supplies and a competition between FELANA communities or FFS may be a motivating factor to improve their mastery of each technique. Awareness messages may need to be targeted to households that are disposing garbage far from homes. “We host teams of data collectors, but rarely see data that portray the fruit (impact) of our own efforts -- here in this Diocese” A Partner 45 6. RECOMMENDATIONS The following recommendations arise from the global results. A. MONITORING & EVALUATION • Insist that technical agencies involved in designing and collecting the baseline survey have the responsibility of either 1.) carefully documenting the definitions, calculations and metadata behind each indicator (and leaving this document carefully embedded in the institutional memory for reference 3 and 5 years thereafter) or 2.) being present during the mid-term and final evaluation. • Ensure that the three stages of evaluation (baseline, midterm and final) are collected in exactly the same month. • Alongside an external needs assessment, using the IHD framework, engage the communities systematically in an internal resource assessment: what do communities have to offer their own development, what do they esteem as their priority needs? What can they do themselves in the absence of any external intervention? Ensure that the results focus more on local assets, structures and systems to guide a clear demarcation of target zones, identifying exactly which populations are to be included in an intervention. • Lighten up on evaluations: scale down number of PITT indicators. Identify impact, and not just process, indicators for each component. Dare to collect only that which can be analyzed and used immediately. • Increase sampling of evaluations (beyond the annual) to measure impact at the level of implementation, not only at the level of CRS/M. Enable partners to track impact (malnutrition and revenue, for example) of their own efforts beyond process indicators. • Insist on one M&E staff to be a skilled qualitative facilitator. Plan qualitative training for all CRS partner staff and educate donors about the need for both approaches. Numbers are not everything. Although qualitative data have been always part CRS/MG M&E system, the regular monitoring and reporting systems (from partners to CRS) focus mainly on quantitative indicators. Future capacity building must be conducted at partner level and the monitoring and reporting system should be reoriented to take this into account. • Hire a consultant to do an in-depth analysis on the current FELANA dataset. There is a huge volume of data unused. Assess how the set of control communities differ from the rest. Consider the role of inflation in the increased value of household portfolios. Analyze time-series data to describe the increases until 2006 followed by sharp decreases thereafter. • Develop a rapid monthly mini ‘early warning’ system among CRS zones to monitor both progress and impact. Pre-test the system using PDAs. B. SAFETY NET, A STRATEGIC NICHE FOR CRS • As a very unique niche for CRS, Safety Net Centers should become a major partner in the next program –one that addresses a growing need for urban programming. IGA work needs to be reinforced and actors educated to seek funding elsewhere. Admit, however, when a safety net is a true lifeline and plan not to exit. 46 • The Ministry of Justice is currently developing a Coordination Structure to standardize and ensure coverage of prison-based interventions across the nation, supported by ICRC. CRS should be a major player here, carefully articulating goals, resources and choosing the set of centers transparently. • Propose a way to target/care for those even less vulnerable in the proximity of safety net centers. • Develop a contingency plan that assures the pipeline / lifeline of these most needy. C. GENERAL MANAGEMENT AND PROGRAMMING • At national and regional levels, strategic efforts should lobby to obtain a more systematic buy-in of from the Government of Madagascar. Donors beyond USAID should bear witness to the contribution FELANA has made to the MAP. • Organize a restrained call for proposals within the Catholic hierarchy and/or articulate a clear strategic commitment to systematically enhance Caritas throughout Madagascar (in the priority sites). • Whatever the future partner network, plan for intensive training on new approaches and/or interventions before program starts up. Conduct a participative training-need analysis for all in￾coming partners. • Organize and provide recurrent refresher training in facilitation skills, especially for all agents active in the FFS. • Maintain and enhance current approaches, allowing more flexibility in their implementation and adding to them when appropriate. • Capitalize on the in-house technicians and component leaders to propose the most site-appropriate additions to future programming. • Take both the TAE/Marketing and the PACOM/Risk and Disaster Management components to a new dimension. Make them a systematic component of every future effort. Archive the PACOMs and compare them between communities. • Focus on quality, not quantity. If CRS plans on FELANA expansion do so carefully applying lessons learned. 47 ANNEXES: Bibliography / References Consulted Final PITT Indicators Map showing distribution of program components Map showing sample (Baseline, Midterm and Final Quant. and Qual.) Individuals Contacted (both Lead and Qualitative Teams) Terms of Reference of Consultant Work Plan of Lead Consultant (1 page) Field Schedule of Lead Consultant (1 page) Qualitative Collection Tools (7 pages) Qualitative Matrices (available from CRS in XLS upon request) Diocese Level Reports in French (available from CRS upon request) Quantitative Survey Instruments (available upon request from CRS) 48 REFERENCES Bergeron, G. & Deitchler, M. 2004. Report of the 2004 Joint Baseline Survey in the Targeted Areas of the PL480, Title II Program in Madagascar. CRS Madagascar. 2003. Cooperating Sponsor FY 03 Results Report CRS Madagascar. 2004. Cooperating Sponsor FY 04 Results Report CRS Madagascar. 2005. Cooperating Sponsor FY 05 Results Report CRS Madagascar. 2006. Cooperating Sponsor FY 06 Results Report CRS Madagascar. 2007. Cooperating Sponsor FY 07 Results Report C-SAFE, 2005. What We Know About Exit Strategies: Practical Guidance For Developing Exit Strategies in the Field. IMF. 2007. Republic of Madagascar: Poverty Reduction Strategy Paper. IMF Country Report No. 07/59. February 2007. OCHA. 2008. Madagascar: Cyclones as of 10 March 2008. Accessed 9.4.2008: http://reliefweb.int/rw/fullMaps_Af.nsf/luFullMap/D61D39A1FD1D22D9C125740A00373AA4/$File/rw _TC_mdg080312.pdf?OpenElement Patterson, D. 2006. FELANA Mid-term evaluation. Prepared for CRS/MG. President’s Malaria Initiative. 2007. Malaria Operational Plan FY08. Madagascar. Accessed 9.4.2008 athttp://www.fightingmalaria.gov/countries/madagascar_mop-fy08.pdf 2 TABLE 41: FINAL PITT Code Indicator Unit Baseline Survey Final FE Achieved Resultat escompte LOA GOAL Goal: Sustained improvement in household food security in 4 regions of Madagascar by 2008 GO-01 # of months of food availability Nb 5.00 6.03 8.00 GO-02 # of households reached by FELANA Nb 0 100,711 103,900 OS1 SO1: Increased household revenue resiliency OG1-01 % increase of household assets (ag+dom+livestock) Ar 518,967.60 486,750.38 570,864.36 OG1-02 % increase of household production Ar 383,683.20 499,547.70 479,604.00 IR 1.1 Increase in agricultural productivity OS 1.1-00 # of HH reached by agricultural activities Nb 0.00 15,392 16,000 OS 1.1-00 # of HH adopting new technics Nb 0.00 10,282 9,600 OS 1.1-01 % increase in yield of rice Kg/ha 1,807.67 2,262.18 2,711.51 OS 1.1-02 % increase in value of livestock holding OS 1.1-02-01 Porciculture Nb de tete 2.70 1.83 3.38 OS 1.1-02-02 Oie Nb de tete 3.70 4.00 4.63 OS 1.1-02-03 Canard Nb de tete 5.40 6.76 6.75 OS 1.1-02-04 Poule Nb de tete 11.30 14.41 14.13 OS 1.1-02-05 Cuniculture Nb de tete 6.60 12.19 8.25 OS 1.1-02-06 Apiculture Nb ruche 2.60 4.00 3.25 OS 1.1-02-07 Pisciculture Kg/ha 151.10 187.47 188.88 OS 1.1-03 % increase in yield of various crops OS 1.1-03-01 Mais Kg/ha 2,642.55 1,002.26 3,963.83 OS 1.1-03-02 Manioc Kg/ha 3,972.37 1,906.43 5,958.56 OS 1.1-03-03 Patate douce Kg/ha 4,847.08 3,203.10 7,270.62 OS 1.1-03-04 Haricot Kg/ha 1,358.63 1,002.26 2,037.95 OS 1.1-03-05 Voandzou Kg/ha 1,315.48 836.64 1,973.22 OS 1.1-03-06 Taro Kg/ha 2,000.00 3,612.54 3,000.00 OS 1.1-03-07 Cultures maraicheres Kg/ha 4,837.10 1,459.90 7,255.65 IR 1.2 IR1.2: Increased revenue from commercial sales OS 1.2-01 %ge increase in marketing revenue Ariary 164,524.40 261,682.90 246,786.60 OS 1.3 IR1.3: Reduction of post harvest losses OS 1.3-00 # of HH reached by GRC activities Nb 0 5,100 5,100 IR 1.4 IR 1.4: Increased partner & community capacity to reinforce resiliency OS 1.4-01 # of targeted communities that implement DPMP plans Nb 0.00 40 40 OS 1.4-02 # of hectares protected through NRM measures Ha 0.00 283 250 OS 2 SO2: Improved health status of the most vulnerable members of rural families OG.2-00 # of families reached by health activities Nb 0 58,155 70,300 OG2-01 Decrease % children 6-24 mos who are STUNTED % 60.60% 46.00% 45.60% OG2-02 Decrease % children 6-59 mos who are STUNTED % 54.70% 52.80% 39.70% IR 2.1 IR2.1: Improvement of family nutritional practices OS 2.1-01 increase % moms that exclusively breastfeed < 6mos % 69.40% 86.00% 89.40% OS 2.1-02 reduce% children 6-59mos with diarrhea % 22.40% 12.46% 2.40% OS 2.1-03 increase % children 6-59mos that receive increased amount of liquids during diarrhea % 73.50% 80.56% 93.50% IR 2.2 IR2.2: Prevention and improved management of child specific illnesses by household (C-IMCI) OS 2.2-01 increase % HH using latrines % 39.70% 59.43% 54.70% OS 2.2-02 increase % families using potable water % 74.20% 77.10% 94.20% OS 2.2-03 increase % children 12-24 mos vaccinated % 29.80% 40.80% 44.80% OS 2.2-04 increase % children using mosquito nets % 59.50% 89.67% 84.50% OS 2.2-05 increase % families where mother & children use mosquito nets to prevent malaria % 58.90% 75.96% 83.90% OS 2.2-06 increase % mothers that seek appropriate care for children with fevers % 57.90% 95.74% 77.90% IR 2.3 IR2.3: Reinforced partner & community capacity to support nutrition and preventative health care OS 3 SO3: Promote and maintain basic human dignity of the most vulnerable members of societies OG 3-00 # of Safety Net beneficiaries Nb 0 22,064 12,500 OG 3-01 # of beneficiaries provided with food assistance consistently Nb 0.00 22,064 16,000 IR 3.1 IR3.1: Increased level of food availability to people served in 100 SN Centers OS 3.1-01 # of rations distributed Nb 0.00 1,446,355 665,600 IR 3.2 IR3.2: Improved income-generating skills of SN Centers beneficiaries OS 3.2-01 # of beneficiaries trained and involved in IGAs Nb 0.00 1,870 2,000 IR 3.3 IR3.3: Reinforced partner capacity to support SN activities OS 3.3-01 # of SN Centers at level A Nb 10.00 41.00 58.00 3 Annex: Individuals Contacted (Lead Consultant, not including team contacts) Antananarivo Eddy RASOANAIVO, USAID Maria Cruz GONZALES, USAID Olivier GILLABERT, Chef de Mission, ICRC Patrick RAKOTOMAHEFA, Design, M&E Reporting Systems Manager, CRS Gabriella RAKOTOMANGA, Programming Officer CRS Felicien RANDRIAMANANTENASOA, Deputy Programming Officer, CRS Rijasoa RAKOTOARINORO, Monitoring/Evaluation Officer, CRS Lalaniaina ANDRIAMANARIVO, Technical Advisor: Ag, CRS Lala RATRIMOSON, Senior Project Officer, CRS Andrianambinintsoa RANDIMBIARISON, Project Officer Santé/Nutrition CRS Jaona RAJAONERY, Monitoring and Evaluation Officer, CRS Jocelyn RANAIVOSOA, Technical Advisor: Health, CRS Fanantenana, Sustainable Livelihoods Specialist, CRS Falihery RASANDIMANANA, P.O. GRC, CRS Avo RATOARIJAONA, CRS Nivo, CRS Toamasina Marie Ange RASOAMANANTENA, Coordinateur Development, ODDIT William FREDERIO, Coordonateur Safety Net, ODDIT Dieu-Donne RANOMBARY, Coordonateur Agriculture, ODDIT Sœur Jacqueline, Maison de Charite Communauté Marofarihy, President et 4 femmes Antsirabe Celestin Pierre RAKOTONDRANAIVO, Coordinateur Development, Caritas Dr. Mamy RAJAONARISINA, Coordinateur Sante, Caritas Freres Kiram et Remy, Mission de la Charite Albertine RAZAFINANTATRA, Directrice, Association Zazakely Farafangana Aurelie RAHANTAMALALA, Coordinateur Development, Caritas Richard Gilles CHAN LON CHING, Coordonateur Agriculture Communauté « Mahasoa » : President, 8 femmes, 9 hommes Mananjary Hery RAKOTONDRAMANANA, Coordinateur Development, BDEM Roger RASAMOELA, Guardien Chef, Prison Mananjary, prisonniers en campenal Communauté : Anilavimamy (Mahela Commune), President, 1 femme, 8 hommes Fianarantsoa Hary RAVELOJAONA, CRS Field Accountant Joelson RANDRIANANDRASANA, CRS Warehouse Clerk/Safety Net Etienne RANDRIAMANANTSOA, Directeur Orphelinat Catholique