FINAL QUALITATIVE EVALUATION: Food Security Initiative in Niger Executed in collaboration with Africare, CARE International, Catholic Relief Services, and Helen Keller International January 24, 2005 Transfer Authorization Award No. FFP-A-00-00-00087-00 External Evaluation Team Coordinators, CARE-Niamey Elizabeth Adelski, Team Leader Ali Béty Issifou Bissala Abdou Garba Ali Mahamadou Abdou Moha Resource People Idrissa Amadou, M&E Coordinator, CARE-Konni Pascal Payet, M&E Coordinator, Project Coordination Unit, Africare-Niger Kwanza Price, PCU Financial Coordinator, Project Coordination Unit, Africare-Niger ACKNOWLEDGEMENTS This report is the result of many people's time and effort that the evaluation team recognizes with great appreciation. We thank all the consortium's staff who contributed to our safe travels, to providing hospitality in the guest houses, to organizing our work in the offices and the various field sites, and above all to answering our endless questions about the program's many facets. The traditional and modern administrators, the government's technical services who were program partners, and above all the men and women in the rural communities that we visited all deserve our thanks for giving us time for interviews during a busy time in the agricultural season. The NGO directors and administrative staff who resolved all types and sizes of problems have our sincere thanks. We thank all of you and sincerely hope that this report proves useful for your current and future work. i ACRONYMS ADB African Development Bank AKG African kitchen gardens BSG Breast-feeding Support Group CARE Cooperative for Aid and Relief Everywhere CCA Food Crisis Cell CC/SAP Coordination Cell/Early Warning System CEWS-UR Community Early-Warning System-Urgent Response CGV Community Management Committee CILSS Inter-State Committee for Fighting Drought in the Sahel COSAN Community Health Committee CRS Catholic Relief Services CS Cooperating sponsor CSI Integrated Health Center CSO Civil society organization CSR4 Cooperating Sponsor Results Report and Resource Request CUSA Unit Committee for Food Security Intervention (also: CSA) CVD Village Development Committee DAP Development Assistance Proposal DIP Detailed Implementation Plan DSC Development Support Committee (Catholic Church) EU Emergency Response Unit (Unité d'Urgence) EBF Exclusive breast-feeding EVS Community Health Experts EVN Community Nutrition Experts EW&ER Early warning and emergency response FANTA USAID Food and Nutrition Technical Assistance Project FARN Center for Nutritional Learning and Rehabilitation FEWSNET Famine Early Warning System Network FFP Food for Peace FFW Food-for-Work FSIN Food Security Initiative in Niger GON Government of Niger GSA and GS2A Community Support Group for Breast-feeding/Child Feeding Practices HKI Helen Keller International IEC Information, education, communication IGA Income-generating activities IPTT Indicator Performance Tracking Table HC Community Health Committee (COSAN) HMC Health Management Committee (COGES) HNRP Household nutrition recuperation program (FARN) HKI Helen Keller International LISN Local Initiative Support Network (RAIL) M&E Monitoring and evaluation MMD Women on the move (Mata Masu Dubara) NGO Non-governmental organization ii ACRONYMS, continued NRM Natural resource management PCU Project Coordination Unit SO Strategic Objective TA Technical assistance TBA Traditional birth attendant TOR Terms of Reference TS Technical Services (GON) USAID United States Agency for International Development VDC Village Development Committee VHE Village health educator VMC Village Management Committee (Africare) VNE Village nutrition educators WFP World Food Program i EXECUTIVE SUMMARY The Food Security Initiative in Niger (FSIN) is a response to the food insecurity that affects the rural population in parts of in Niger. The program currently is in the beginning of its fifth fiscal year and will end in mid-2005. It is funded by United States Agency for International Development/Food for Peace (USAID,FFP), through monetization, and supported with Food￾for-Work (FFW) commodities. To date, FSIN has distributed 17,473 metric tons of bulgur wheat to 366,000 program participants. FSIN is implemented by a consortium of four nongovernmental organizations (NGO): Africare, CARE International, Catholic Relief Services (CRS), and Helen Keller International (HKI). Each NGO contributes its particular expertise to the consortium: Africare is responsible for monetization; CARE for organizing surveys and studies; CRS for FFW; and HKI for nutrition. FSIN works in seven departments in Niger, with 300 communities and approximately 300,000 people. Ninety-five percent of the participating households are classified as "very vulnerable" or "vulnerable" in terms of food security; only 5% are "moderately vulnerable." The program's overall goal is to improve the participants' food and nutritional security, through its three strategic objectives (SO). SO1, strengthening community capacity to manage food security, is based on training, education, and the establishment of community structures (committees, support groups, input and cereal banks). SO2 is designed to increase sustainable agricultural production by promoting environmentally sound cultural techniques. This includes sustainable natural resource management (NRM) and cultural techniques for rainfed cereal crop production and irrigated vegetable production. FFW is a cross-cutting FSIN activity. Forty days of FFW during the hungry season enables participants to implement new practices such as rehabilitating degraded fields with demi-lunes and enlarging seasonal ponds for irrigation. SO3 works toward improving the nutritional status of women and children under five, who are the most vulnerable to malnutrition. Its activities include health and nutrition education, breastfeeding support groups, and constructing rural health centerss. FSIN works to strengthen links between the communities and the government technical services in order to build sustainability into the program. Evaluation Methodology This final, qualitative evaluation was done early in FSIN's fifth year in order to comply with the USAID/FFP requirement to submit it before submitting their second DAP, which is due in early 2005. The overall goal of the evaluation was to collect information about FSIN's people-level impact, the lessons learned during its implementation, and recommendations for the current and the second DAP. The evaluation team conducted its fieldwork in the four districts where the cooperating sponsors (CS) work: in Agadez with Africare, in Tanout with CRS/HKI, in Konni/Illela with CARE, and in Dogondoutchi with CRS/HKI. The team spent 17 days in the field and conducted interviews in a total of 20 communities. Separate groups of women and men were interviewed in each community so that each had the opportunity to give its perspective on the program. The evaluators used a topical guide with open-ended questions to conduct the group interviews. FSIN staff, the government of Niger (GON) Technical Services (e.g. Health, Community Development, Environment), and FSIN's partners (e.g. Caritas and the African Development Bank) in each district also were interviewed. ii People-level Impact The evaluation team's overall conclusion is that FSIN has had a positive impact on participants' food and nutritional security. This is the result of achievements in the program's three SOs that have enabled the participants to make positive changes at the household and community levels. The changes include increased community capacity to address constraints on food security; participants' learning about and using sustainable NRM and agricultural production techniques; and participants' positive behavior changes in health and nutrition through learning as well as experiencing the advantages of changed behavior. FFW definitely has contributed to FSIN's achievements as it simultaneously provided food and enabled participants to implement new technologies. It also enabled people to invest their labor in activities that they prioritize that also affect food security, such as building roads and livestock corridors. Participants' evaluation of FSIN's impact on their lives is quite homogeneous, despite the program's different social and ecological environments. They consistently reported that SO1's training and education were among the best-liked and most useful aspects of the program, particularly in health and NRM. SO1 has built community capacity to work together and women's increased participation in community affairs: "The best thing FSIN has done is to organize committees; they are something new, and enable the community to implement activities. We have learned that one person alone cannot make a decision; a group of all types of people must assemble—men, women, and the poor." Participants have increased their arable landholdings by rehabilitating degraded fields, increased off-season vegetable production and thus revenues, and doubled their millet production in years with good rainfall as the result of SO2. Rehabilitating degraded land for pasture and agriculture is universally reported as an important impact by both participants and the Technical Services (TS). FFW is essential in operationalizing SO2 and particularly for the poor, because it enables them to invest their labor in improving their own fields instead of investing it in others' in order to eat. According to both men and women, SO3 has made visible improvements in people's health: exclusively breastfed children are fatter and healthier; pregnant women who take iron are stronger and have fewer problems at childbirth. People also have learned about the need for micronutrients in the diet and the local foods that supply them. Lessons Learned Some key lessons learned about FSIN are: • Capacity-building should be seen as a cross-cutting element that is essential for the success of other program components. • The program should recognize and plan for the fact that raising participants' awareness and behavior change are long-term processes. • All the CSs should provide literacy classes, particularly for women, because it is necessary for engaging the population in implementing all three SOs. • FSIN should promote women's participation because due to their traditional social role they are reticent about participating in community affairs. iii Sustainability The evaluation team, FSIN staff, and the TS concluded that sustainability inevitably rests with the participants, because the state does not have the resources to meet the rural population's need for social and technical services. The participants are the most optimistic about this: "We can continue after the project ends; we can do what we learned on our own, we have seen that it is in our own interest." The TS in all four districts, that are meant to provide sustainability by continuing to support and promote community activities when FSIN ends, consistently reported that they have the competence but not the resources to do so. Their lack of resources—vehicles, funds for gas and perdiem, human resources—varies by district but is a serious constraint on getting their work done. Linking communities to local NGOs and civil society organizations (CSO) is an option for building sustainability into the program, particularly if there is a second phase. Recommendations The evaluation's recommendations for FSIN include: 1. Work with FFP to start FFW activities earlier because currently they conflict with crucial, rainy-season agricultural activities that become secondary because people prioritize FFW. 2. Include a literacy component in FSIN-2, because illiteracy is a universal constraint on implementing the program. 3. Link communities with local NGOs and civil society organizations that can help them sustain their activities when FSIN ends. 4. Identify site-appropriate income generating activities (IGA), particularly for the dry season, when both men and women need to generate income. 5. Improve the availability of and access to health services by improving the functionality of the existing rural health centerss, and by constructing and equipping new ones. 6. Standardize the program's animal husbandry activities, because animal husbandry is an important component of household economic systems in all four districts. Cross-Cutting Activities: Management, Food-for-Work, Monitoring and Evaluation, and the Emergency Response Unit Management FSIN's cross-cutting activities—management, FFW, monitoring and evaluation, and the Emergency Response Unit—also were evaluated. Management was assessed in terms of FSIN's relevance to national policy and the program's integrated structure as a consortium. The civil servants interviewed about the former consistently stated that FSIN's objectives and implementation strategy fit well within Niger's current policy context, namely the National Poverty Reduction Plan, working toward gender equity, and the Ten Year Health Plan (2001- 2011). The fact that FSIN works with the population to address its basic needs is seen as appropriate in the GON's long-term orientation toward community-based development and in the newer context of decentralization. iv Overall, staff's evaluation of FSIN’s integrated structure as a consortium, and their evaluation of the advantages and disadvantages of working in a consortium, was positive. The consortium is seen as an innovative organization with a positive effect on management. Staff reported that the advantages of working in a consortium outweigh the disadvantages. The major advantages are exchanging information and experience with a network of colleagues; the CSs' complementary expertise, which produces good work; and learning about other management systems and approaches to rural development by working with other NGOs. The major problems with working in a consortium are busy colleagues and an overloaded work schedule that make it difficult to keep on schedule. Food-for-Work FFW is an essential component of FSIN because it enables participants to implement both household- and community-level activities to improve food security. These activities included building roads, rehabilitating wells, and reforestation. FFW’s people-level impacts include: raising participants' awareness about their capacity to protect the environment; mobilizing people for collective, community action; building social solidarity through collective work; avoiding household decapitalization from livestock sales to buy food; and enabling people to use improved NRM techniques to rehabilitate their fields and increase their arable land area. Participants reported that FFW replaced the men's traditional, dry￾season coping strategy of going to neighboring countries to earn money to support their families. FFW enabled people to stay home and invest their labor in improving their household and community resources, and thus their food security. The major problem with FFW is that its activities are implemented in the hungry season and thus conflict with households' need for labor for agriculture. Participants generally prioritize FFW at the expense of their own fields, which is counterproductive to some extent. The other weaknesses with FFW are that it creates an attitude of expectancy, if not dependency; and that communities do not always maintain the structures they build with it (e.g. roads). Nevertheless, the evaluation team definitely agrees with using FFW in FSIN. It is a constructive means of mobilizing people who live in chronically food-insecure areas and has contributed to achieving FSIN's strategic objectives. Monitoring and Evaluation FSIN has a standardized and integrated monitoring and evaluation (M&E) system. This system allows the aggregation and disaggregation of data across time and space, which meets the CSs' and USAID's need for information at both the district and the consortium levels, respectively. Each CS has an M&E unit with one staff member, who is responsible for data input and management. At the district level the M&E system is based on community participation and periodic community capacity assessments, and monthly reports from the field staff. Staff cited information-exchange, the standardized M&E system, and quarterly meetings to resolve M&E issues as strengths of FSIN's M&E system. Making the TS a partner in data collection and in conducting studies is positive because it helps build their capacity. Overall, staff consider the M&E system to be a good one, despite the weaknesses they also identified. The need for data inputters in the district M&E units has been a weakness throughout the program's lifetime. Other weaknesses are related to data quality: the SO heads should be v more involved in data collection because it improves data quality, and FSIN needs to be able to pay more in order to hire better data collectors. The Emergency Unit (Unité d'Urgence) CARE's strategic plan includes a component to strengthen the consortium's capacity for early warning and emergency response. The Emergency Unit (EU) is responsible for coordinating the numerous activities in early warning and response with donors and CARE's partners. The evaluation focused on the EU's work at the community level, where Community Early Warning System-Urgent Response (CEWS-UR) committees are in the process of being set up in FSIN's intervention zone. The CEWS-UR is a system for communities to monitor and report on potential threats to their well-being. There are five CEWS-UR committees in Agadez and Tanout, where the system is still in an experimental stage; there are 10 pilot CEWS-UR sites in Konni/Illela; and about 40 CEWS-UR committees set up in the Dogondoutchi district. The process of establishing the CEWS-UR committees has been a laborious one and not well understood by the consortium, the TS, or FSIN’s participants. However, the system seems to be a useful and accepted mechanism at the community level. Participants have used it to report locust invasions and measles outbreaks to the appropriate TS. The potential constraints on its use are the detailed monthly reports that the CEWS-UR committees are required to write; the cost of responding to emergencies that a program could support, but not communities; and the CEWS-UR’s integration into the state’s early-warning and emergency￾response system. i TABLE of CONTENTS Section Page I. Introduction 1 A. Purpose of the consultancy 1 B. Methodology II. Summary and Conclusions 4 A. FSIN and the national political context 4 B. Impact 4 1. SO1 5 2. SO2 6 3. SO3 7 4. FFW 8 5. The most vulnerable households 9 C. Strengths and weaknesses 9 D. Lessons learned 11 1. Key lessons learned 11 2. Women: lessons learned 11 3. Change is a slow process: lessons learned 11 4. Programmatic lessons learned 12 5. Training: lessons learned 12 E. Sustainability 12 F. Recommendations 13 1. Recommendations for the current program 13 2. Recommendations for the DAP2 14 III. Strategic Objective One 14 A. Summary of SO1 activities 15 B. Impact 16 1. SO1 impact indicators 16 2. SO1 impact 17 a. Participants 18 b. Technical Services and FSIN staff 20 c. HKI’s Community Radio intervention 20 C. Strengths and weaknesses 22 1. Participants 22 2. Technical Services and FSIN staff 23 D. Sustainability 25 E. Lessons learned 26 F. Recommendations 29 1. For the current program 29 2. For the DAP2 32 ii TABLE of CONTENTS, continued Section Page IV. Strategic Objective Two 30 A. Summary of key activities 30 B. Impact 32 C. Strengths and weaknesses 35 1. Strengths 35 2. Weaknesses 36 D. Sustainability 37 E. Lessons learned 38 F. Recommendations 39 1. For the current program 39 2. For the DAP2 39 V. Strategic Objective Three 40 A. Summary of key activities 40 B. Impact 42 1. Quantitative impact evaluation 42 a. Stunting in children 43 b. Breast-feeding newborns 45 c. Vitamin A consumption 46 2. Qualitative impact evaluation 47 a. According to women 47 b. According to men 47 c. According to the Technical Services and FSIN staff 48 C. Strengths and weaknesses 48 1. Strengths 48 2. Weaknesses 49 D. Lessons learned 49 E. Sustainability 49 F. Recommendations 50 1. Global recommendations 50 2. Recommendations for the DAP2 50 VI. Cross-Cutting Activities: Management, Food-for-Work, 51 Monitoring and Evaluation, and the Emergency Response Unit A. Management 51 1. Institutional management organization 51 2. Human and financial resources 51 3. Strengths and weaknesses 52 4. Evaluation of the consortium's structure for program management 52 5. Strengths and weaknesses of working in a consortium 52 6. Lessons learned 53 7. Recommendations 53 8. Implementation of the midterm recommendations 53 iii TABLE of CONTENTS, continued Section Page B. Food-for-Work 57 1. Summary of activities 57 2. Impact 58 3. Strengths and weaknesses 59 4. Sustainability 59 5. Lessons learned 59 6. Recommendations 60 C. Monitoring and Evaluation 60 1. Summary of the M&E system 60 2. Advantages and disadvantages of working in a consortium 61 3. Strengths and weaknesses of the M&E system 62 4. Lessons learned 63 5. Recommendations 65 D. The Emergency Response Unit 66 1. Summary of activities 67 2. Findings 67 3. Recommendations 68 References cited 70 Annex 1. Fieldwork and Activities Schedule and Topical Guide Annex 2. Terms of Reference Annex 3. Contact List Annex 4. Indicator Performance Tracking Table 1 I. Introduction and Methodology A. Purpose of the evaluation The FSIN program currently is in the beginning of its fifth fiscal year and will end in mid￾2005. The CSs plan to submit a Development Assistance Proposal (DAP) to the USAID/FFP office, for a second phase of FSIN. The timing of this evaluation was determined by the USAID/FFP requirement to submit an evaluation of FSIN before submitting the DAP, which is due in early 2005. The overall goal of this final, qualitative evaluation was to collect information about FSIN's impact on participants' lives, the lessons learned during its implementation, and recommendations to contribute to the second DAP. The evaluation's specific objectives included documenting the participants' views of the program's impact and the changes it made in their lives; assessing the change in food insecurity among the targeted vulnerable groups, particularly women and children under five; documenting changes in participants' knowledge and practices related to improving their food security; evaluating FSIN's operational strategy, including the benefits and costs of working as a consortium; and investigating the implementation of the midterm evaluation's recommendations. The Terms of Reference (TOR) for the evaluation are in Annex 2. B. Methodology The evaluation team conducted its fieldwork with the four CS in four districts: in Agadez with Africare, in Tanout with Catholic Relief Services (CRS) and Helen Keller International (HKI), in Konni/Illela with the Cooperative for Aid and Relief Everywhere (CARE), and in Dogondoutchi with CRS/HKI. The team spent 17 days in the field and conducted interviews in a total of 20 communities. Table I-1 below shows the departments and communities in each district where fieldwork was conducted, and the partners and government Technical Services that were interviewed. Three to seven communities were visited in each district. Due to constraints of time for the evaluation in FSIN's vast intervention area, driving time to reach the communities was a major factor in determining the sample for the evaluation. The other criteria for the sample were communities: 1) in the districts' different agroecological zones; 2) with varying performance levels, based on the FSIN staffs’ judgement; 3) of different generations with FSIN, in Africare's and CRS/HKI's districts, because they implemented their programs progressively. In brief, the reasons for choosing the communities for the final evaluation were: 1. Africare, Agadez: All the Africare communities visited for the evaluation in Agadez are in the department of Tchirozerine due to 1) the insecurity that made travel to Arlit inadvisable; 2) the travel-time required to reach the communities in Arlit, which was four to eight hours; and 3) the travel-time allocated for the fieldwork. The team conducted interviews in: • Four of the 48 communities in the Air agroecological zone; • Two of the 24 communities in the Irhazer agroecological zone; and • One of the 12 communities in the Tadress agroecological zone. 2. CRS/HKI, Tanout and Zinder: The communities that had made significant progress with CRS/HKI and could be reached within the travel-time allocated for the fieldwork were chosen for site visits. 2 3. CARE, Konni/Illela: The community of Cheta was chosen because staff consider it a moderately successful program site. It is in the valley where there is more access to water, there are numerous FSIN activities, and the vegetable-gardening activity has been successful. Awanchala was chosen as an example of a marginalized community in the arid, northern area with mediocre performance, one of 10 functional CEWS-UR, and where FSIN has emphasized nutritional activities. Goumbi Kano was chosen because it is included in the District Plan for Health Development and as an example of successful health/nutrition activities. 4. CRS/HKI, Dogondoutchi: The communities in this district were chosen based on the types of program activities that have been implemented and travel-time. Angoual Magagi Doka was chosen because it developed vegetable gardening with FSIN. In Dogon Kiria a group of men was interviewed about HKI's radio intervention; no other FSIN activities were addressed in this interview. Separate groups of women and men were interviewed in each community so that each sex had the opportunity to give its perspective on the program. The evaluators used a topical guide with open-ended questions (Annex 3) and the group interviews lasted approximately two hours. All of the evaluators used translators in Agadez because this enabled the Tamashek speakers to express themselves easily. In the other three districts the three Nigerien evaluators conducted their interviews in Hausa and the American consultant worked with a translator. It should be noted that the self-reported information from the FSIN participants is subjective, unverified, and subject to bias. These are universal caveats on any self-reported information, qualitative or quantitative. A potential bias in this evaluation is that the poorest people did not always have the opportunity to respond directly to the interview questions, as group discussions often are dominated by the more important people in the community. The evaluation team interviewed three other categories of people in each district: FSIN staff, the GON Technical Services (e.g. Health, Community Development, Environment), and FSIN's partners (e.g. Caritas and the African Development Bank). Two CARE projects also were interviewed: “Women on the Move” and “AIDS in Migration (SIDA en Exode).” The team made courtesy visits to the traditional chiefs, the Prefets, and the Sous-Prefets, which generally provided little in-depth information about FSIN. 3 Table I-1. The Final Qualitative Evaluation Site Visits and Interviews Interviews Districts Dates Partners Technical Services Communities Africare: Agadez, department of Tchirozerine Sept. 16-20 None Health, Literacy, Environment, Education, Rural Development, Hydrology. Air1 : Goffat, Boughoul, Egandawel, Boudari. Irhazer: Tamaghate, Tiguidan Tagueite. Tadress: Abalama. CRS/HKI: department of Tanout and region of Zinder Sept. 21-24 Caritas Tanout: Health, Community Develop￾ment, Agriculture, Environment, Rural Development Tanout: Sabon Kafi, Garin Bagala, Yagagi. Zinder: Dania Maikogo, Bakin Birji CARE: departments of Konni and Illela Sept. 25-29 "Women on the Move;" the Micro￾Project project, African Development Bank, Tahoua; CARE project “AIDS in Migration (SIDA en Exode) Konni: Community Development, Literacy, Environment, Livestock Resources, Land Tenure Commission, Agriculture Illela: Inspection of Primary Studies, Social Development, Commu￾nity Development, Literacy, Environment, Agriculture, Livestock Resources, Land Tenure Commission Konni: Cheta, Goumbi Kano. Illela: Awanchala. CRS/HKI, department of Dogondoutchi Sept. 29- Oct. 3 Committee for Development Support (Catholic Church), and the Local Initiative Support Network (RAIL) Health, Community Development, Agriculture, Environment, Rural Development Angoual Magagi Doka, Samia, Kaiwa Fako, Dogon Kiria2 , Balessa, Rijia Samna 1 Air, Irhazer, and Tadress are three different agroecological zones. 2 The interview in Dogon Kiria was to collect information only on HKI's Radio Club intervention. 4 II. Summary and Conclusions A. FSIN and the national political context All the civil servants interviewed during this evaluation stated that FSIN's objectives and implementation strategy fit well within Niger's current policy context. That context includes the National Poverty Reduction Plan, working toward gender equity, and the Ten Year Health Plan (2001-2011). Both civil servants and traditional chiefs appreciated the fact that FSIN works with the population to address its basic needs and has a participatory, grassroots approach. Building communities' capacity to manage their food security, and their being proactive about it, is seen as appropriate in the GON's long-term orientation of community￾based development ("developpement à la base") since the seventies, and in the newer context of decentralization. FSIN also has initiated a community-managed early warning and urgent response system that can be linked to the GON's departmental and regional systems. The goal of this system is to help communities to identify the potential risks to their food security, advise the Technical Services of level of the risks, and be proactive in local-level risk mitigation. B. Impact The overall conclusion of this evaluation is that, after four years, FSIN has had a positive impact on participants' food and nutritional security. The positive impact is the result of achievements in FSIN's three Strategic Objectives (SO) that, taken together, have enabled the program participants to improve their food and nutritional security at the household and the community levels. The improvement is due to communities' increased capacity to organize themselves to address their constraints on food security collectively; participants' increased knowledge and use of sustainable, improved NRM and agricultural production techniques; positive behavior changes in health and nutrition as the result of learning as well as experiencing the advantages of changed behavior; and FFW. FFW definitely has contributed to FSIN's achievements as it has simultaneously provided food and enabled participants to learn and use new technology. It also has enabled people to invest their labor in activities that they prioritize, such as building roads and livestock corridors that also affect food security. According to participants, FFW is an important factor in their improved food security. Another is the increase in their arable farmland due to the recuperation of degraded fields. As one woman said, "Food security has increased and the need for out-migration (exode) has decreased, mainly due to the rehabilitation of farmland and FFW." At this point it is not possible to determine if the improvement in participants' food security will be sustainable when FFW ends. FSIN works in the most food-insecure areas in Niger. Periodic drought, poorly distributed rainfall, marginal soils, pests, population pressure, and lack of government technical services all compromise agricultural production. A real lack of government social services also compromises the population's health and educational status, which are important factors in food security. It is reasonable to expect only limited improvement in food security in this tough ecological and social context. The participants are optimistic about increasing their food security through the combination of learning and activities that FSIN offers, while they recognize the limitations they face. The evaluation team shares their attitude. 5 It is interesting to note that participants' reports on the changes in their lives from working with FSIN were quite homogeneous, despite the ecological and social differences of the four districts where the program operates. The minor variations in their statements about FSIN's impact on their lives reflected the site-specific constraints on food security and the specific actions taken to address it in the different districts. 1. SO1 Participants' statements illustrate FSIN's impact in terms of its three SOs and FFW. SO1, strengthening community capacity to manage food security, was based on training, education, and the establishment of community structures (committees, support groups, and structures such as input and cereal banks). The positive results of SO1 commonly reported by participants included increased social solidarity; women's integration into community affairs; people's increased awareness of their own potential to help themselves; and participants' learning about improved practices in a range of areas, from democratic decision-making to health to crop production. Following are some participants' statements about the positive impacts of SO1. Social solidarity: • "Before FSIN individualism reigned and each family took care of itself; now, there is social solidarity and decision-making is done together." • "There is wider social solidarity now; the people from the outlying hamlets come and participate in community activities." Community organization: • "The best thing FSIN has done is to organize committees; they are something new, and enabled the community to implement activities." • "We meet and make our own decisions; neither the project nor anybody else can impose a decision on us. Learning to meet and make decisions is one of the most useful things we have learned with FSIN." Women's participation in community affairs: • Women: "Men used to make the community decisions, but now women sit beside them and take part in the decision-making. This fact is important: women are part of the community and should be part of the decision-making." • Men: "Now men and women meet to plan and implement activities related to food security; they work together. Women attend community meetings and speak up; sometimes they have better ideas than the men." Building capacity: • "Before FSIN, we wanted to manage our food security but we did not know how. Now, with technical assistance from the project, we can implement our ideas." • Women said that "It is easier to mobilize women now and we have acquired the confidence that we will succeed at whatever we work on." • A staff member said that "Participants learned that they are capable of getting major projects like road-building done; they were surprised at their collective capacity." 6 Education: • "Training and teaching have contributed to the decrease in people's poverty. They also have changed behavior: women practice EBF and monitor children's growth." • "We have learned to better manage our food security because we have learned to rehabilitate our land and techniques such as reforestation to protect the environment. We can teach others." 2. SO2 SO2 was designed to increase sustainable agricultural production by promoting environmentally sound cultural techniques. This includes sustainable NRM and cultural techniques for rainfed cereal crop production and for irrigated vegetable production. According to participants, the impacts of transferring improved practices that participants can easily use are: a significant increase in vegetable production, which generates revenue and diversifies the household diet; the regeneration of pasture grass and trees in rehabilitated land, particularly in the pastoralist area of Agadez; and an increase in cereal production in the demi-lunes1 . Both men and women stated that, as the result of demi-lunes and improved seed, their millet production has doubled in years with good rainfall. This increase is not reflected in SO2's impact indicator, but it was reported in three districts (Dogondoutchi, Konni/Illa, and Tanout). According to participants, SO2's positive impacts also included an increase in households' arable landholdings from rehabilitating degraded fields; diversification in vegetable production; and decreasing conflicts between pastoralists and farmers as the result of establishing livestock corridors and defining pasturelands. Participants' statements about the positive effects of SO2 include: Increases in production: • "Agricultural production has increased due to the use of new techniques such as demi-lunes, compost, and improved cultural practices." • "The rehabilitation of degraded fields and the use of demi-lunes that conserve moisture have increased millet production; the millet produces better in the demi-lunes than in sandy soil." Increases in revenue from vegetable production: • "Learning improved production techniques for onion production has made it quite profitable; many people have shifted from grain to onion production for that reason." • In Agadez, one man's production of onions increased almost seven-fold. He attributed this increase to the program’s training and the access it provided to chemical fertilizer, pesticides, and agricultural tools. Conflict reduction: • "Conflicts with the pastoralists have decreased because with FFW we made a livestock corridor and defined pasturelands." 1 Demi-lunes are shallow holes in the shape of a half-moon, about a meter long, designed to conserve moisture. 7 Rehabilitation of farmland: • "There are trees and pasture grass growing on rehabilitated land, where nothing has grown for years." • "The poor can cultivate more of their farmland because they have rehabilitated some of it, so their production has increased." 3. SO3 SO3 is aimed at improving the nutritional status of women and children under five, who are the most vulnerable to malnutrition. Community groups to support improved feeding practices and breastfeeding, recuperation centers for malnourished children, vaccination campaigns, the construction of rural health centerss, and health education were used to work toward this objective. The impact indicators for SO3 show that FSIN has had a positive impact on nutritional status. Stunting in children 24-59 months has decreased from 51% to 47% in four years, the proportion of communities where vitamin A consumption is above the risk level has increased from 10% to 30%, and the proportion of children breastfed within eight hours of birth has increased from 30% to 73% (IPTT, Annex 1). The positive impacts of SO3 that participants generally cited were the construction of health centers, access to health education information through HKI's Radio Club and Africare's Community Radio, learning how to rehabilitate malnourished children and treat diarrhea, and learning about EBF and hygiene. Both men and women stated that FSIN's health education was one of the best￾liked and most useful aspects of the program. Education: • "Women learned many new things about breastfeeding, hygiene, anemia and how to prevent it with pills and vegetables, how to treat children who are malnourished or have diarrhea, the value of vegetables in children's diets." • Men said that "Hygiene promotes health: dirty children get sick." Micronutrients: • "People have learned about the importance of different types of local foods that are nutritious." • "We learned about foods that have vitamin A and iron." Behavior change: • "EBF children are fatter and healthier; they have fewer illnesses and less diarrhea." • "Women's and children's health is better; both go to the clinic when necessary and women go for prenatal checkups." Impact on women and children: • "Women have stronger blood; they are healthier and childbirth is easier as the result of taking iron pills. There is less risk of hemorrhage and the placenta descends when newborns are put to the breast immediately. Women recover faster from childbirth because their blood is stronger." • "Healthier children mean less time and money spent on caring for sick children." 8 4. FFW FFW is a cross-cutting activity used to improve food security during the hungry season, usually May-July, and to enable participants to implement their priority activities related to food security. To date, FSIN has distributed 17,473 metric tons of bulgur wheat to 366,000 program participants. Participants reported that it provided food for two or three months, depending on the size of the household. They also universally said that it replaced the men's traditional coping strategy of seasonal migration (exode) to other countries (Algeria, Ivory Coast, Nigeria) to earn money to support their families during the hungry season. The activities implemented with FFW include land rehabilitation (demi-lunes, reforestation, dikes) road-building, repairing wells, construction of community buildings (warehouses, literacy and training centers), livestock corridors (delimited with trees). According to both participants and staff there is a conflict for labor between FFW and household crop production in the rainy season. The latter loses, as households allocate their stronger labor (men) to FFW. This is counterproductive and should be avoided, but evidently the combination of USAID's regulations on transporting and storing commodities and the variable onset of the rainy season perpetuate the problem. Participants consistently rated FFW as among the best-liked and useful components of FSIN. The summary statement about FFW in all districts is that it has the double advantage of providing food and rehabilitating the natural resource base. This is particularly important for the poor, because it allows them to invest their labor in improving their own fields instead of working in others' fields to earn money to eat. It also eliminates the poor’s need to take credit during the hungry season and repay it with their harvest. FFW thus helps to alleviate the vicious cycle that entraps the more vulnerable households: investing their labor in other people's crop production, taking credit to survive, and losing some of their production to repay the credit Participants said that: • "FFW decreases the number of men who go on exode and protects the environment. It has the double advantage of providing food and enabling people to learn NRM techniques." • "FFW helps prevent household decapitalization as people are not obliged to sell their livestock to buy food." • "FFW enables the poor to work in their own fields and improve them with new NRM techniques, instead of working for others in order to eat." • "FFW in the hungry season enables the poor to survive, and builds useful structures for the community and for individuals." • "Conflicts with the pastoralists have decreased because with FFW we made a livestock corridor and defined pasturelands." • "Before we ate chaff; now we eat wheat." The other impacts of FFW that the evaluation team noted were: • Participants learned that they have the capacity to protect their natural resources. • FFW mobilized people for collective work that benefitted the community, such as building roads and health centers. • The regeneration of vegetative cover and trees as the result of rehabilitating land. 9 • Building social solidarity through working together on collective activities. 5. The most vulnerable households The evaluation team was asked to assess FSIN's impact on the most vulnerable households. Ninety-five percent of the households that FSIN works with are classified as vulnerable to food insecurity: 70% are very vulnerable and 25% are vulnerable to food insecurity. Women-headed households generally are the poorest. Our overall conclusion is that participants' vulnerability has decreased somewhat during the life of the program, but as the participants themselves said, they are still poor. Four major factors have contributed to improving the poor's situation: FFW; increasing their arable area by rehabilitating degraded fields; collective work by community members to rehabilitate their fields using FFW; and the program’s use of a traditional means of building up a household's livestock holdings, in which the most vulnerable households are given a goat or sheep, keep the first two offspring, and give the mother to another poor household. The significant advantage of FFW is that, for a limited period, it enables the poor to invest their labor in improving their own fields and production instead of working for others in order to eat. However, FSIN inevitably has differential impact on participants, depending on their resource levels. Some better-off households hire the poor to make demi-lunes in the formers' fields, having seen the technique's impact on production, which is a long-term benefit for the well-off but not for the poor. For example, one community chief hired labor to make over 500 demi-lunes in his fields. However, differential program impact is balanced by the fact that only 5% of the participants are defined as "moderately vulnerable" and thus likely to have the resources for such investments. Following is what participants themselves said about FSIN’s impact on the poorest people's lives. • "The poor can cultivate their own fields because FSIN provides seed and FFW. Now they can keep their harvest, instead of using part of it to repay the credit they needed during the hungry season." • "Their lives have changed; they are not so hungry now because of FFW. Before FSIN they went on seasonal migration. But the majority is still poor." • "The poor benefitted from FSIN's teaching and training, which are for everyone, but they are still poor." • "Their food security has improved but they cannot say that they have enough to eat. [Poor rainfall this year is a major factor]." • "There is not a great difference in their lives. However, poor children's health is better due to women's learning about health care. One problem is that poor women do not have enough to eat so they do not have enough breast milk for EBF." C. Strengths and weaknesses Staff, the GON Technical Services, and FISN partners were asked about the program's strengths and weaknesses. The former outweighed the latter and were more consistent; the weaknesses tended to be idiosyncratic. FSIN's participatory approach, large geographical and programmatic scope, and the good working relations and partnership that it has forged with its partners—the GON administrators, traditional chiefs, the TS, local NGOs—consistently were reported as the program's major strengths. The recognized value of the participatory approach is that it "made the communities part of the program." As one TS staff member said, "Communities did their own Action Plans and feel responsible for them." Another said 10 that "The difference with FSIN is that the communities are responsible and manage the FFW and the work tools, not the Environment Service." FSIN's accent on training and education also was reported as a strength, because it is the basis for all of the program's interventions. The TS's overall assessment was that "FSIN got participants mobilized and taught them things" and that there is a "strong involvement of the participants and the TS in all phases of the program." The government administrators in all four districts reported that FSIN fits perfectly into the national political context, is an excellent program because it addresses people's basic needs, and should be expanded. The fact that FSIN has resources and has provided some support to the local TSs by involving them in the program most likely influenced the civil servants' assessment of the program. As many of them said, it is the major (or only) program in their areas, and they rely on it to address their serious food insecurity problems. Participants' high illiteracy rates, the staff's need for training, and an overbooked work schedule are the main weaknesses that staff and the TS reported. Lack of a literacy component in FSIN is a universal program weakness that both staff and the TS identified. High illiteracy rates are a constraint on program implementation across the board, and particularly on women's engagement and awareness-raising. Africare in Agadez is addressing this problem in its current DAP, but elsewhere it is a problem. Staff also identified the need for more training for technical staff, and the need for training in M&E activities for technical staff and the TS. The TS consistently reported that FSIN's work schedule and its staff are overloaded, which creates a cascade of problems: staff are not easily available to work with, it is not possible to adhere to the work schedule, so activities are not implemented on time, and there is not enough time to do all the activities well. Staff themselves identified some of these as weaknesses, so they should be taken into account in the second DAP. The evaluation team also documented staff's opinions of the advantages and disadvantages of working in a consortium. The overall opinion is that working in a consortium is advantageous, because: • It enables people to share information and experience, including best practices, so that there is a community-level benefit. • There is a diversity of expertise that is mutually enriching, and the complementary expertise improves the quality of the work. • It gives FSIN a large and diverse geographical scope, which one NGO could not cover. • Staff have learned from the different NGOs' approaches, and about their different management systems. The major disadvantage that staff identified already has been cited as a systemic weakness: an overbooked work schedule and overbooked colleagues. As a result it is difficult to find time to meet colleagues and to keep the work schedule on track. Staff noted that problems with working in the consortium in the beginning—confusion about the different NGOs’ roles, lack of understanding of other NGO's approaches and management systems, the need to standardize the M&E system—have disappeared over time. Overall, staff feel that the advantages of working in a consortium outweigh the disadvantages. 11 D. Lessons learned 1. Key lessons learned Staff, FSIN documents, and the TS reported most "lessons learned" in the form of conclusions. Some of these conclusions have been reformulated as lessons learned; the rest are included in this section in their original form. The universal and most salient lesson learned is that FSIN should have a literacy component. High illiteracy rates in the program zone (up to 90%) are a constraint on FSIN's implementing all three SOs and a constraint on participants' ability to learn and to manage their committees and activities. Illiteracy is particularly a barrier to women's participation in community affairs and in committees, as well as a constraint on activities such as monitoring children's growth and training matrons. The other key lessons learned that generally apply to the consortium as a whole are: • Capacity-building is a cross-cutting element essential for the success of other program components. • Volunteerism has its limits; people will drop out of committees if they are not paid. • Committee members travel extensively during the hungry season [to find work] and large numbers of able-bodied people migrate after a poor agricultural season. This affects performance in communities' committees and activities. 2. Women: lessons learned Another universal lesson learned is that FSIN should be proactive in building women's capacity and promoting their participation in community activities, including committees. This is necessary to change women's traditional social role and increase gender equity. Staff have learned that: • FSIN should support women's participation because they are reticent about participating in community affairs. • The program needs to push to get both sexes on committees. • Women's participation in community affairs and decision-making increases over time, with training. It is a process that takes time. 3. Change is a slow process: lessons learned Staff in some districts reported that they have learned that raising awareness and changing behavior is a slow, gradual process. According to some staff, the participants have learned a similar lesson: that their fields have degraded over time, and that rehabilitating them will be a gradual process that takes time. A few staff reported the following lessons learned about time and change: • The program should recognize and plan for the fact that raising participants' awareness is a process that takes time. • Behavior change is a long-term process. • There is lack of social cohesion in communities that have more than one chief, and it takes time to get people to work together as one community. 12 4. Programmatic lessons learned • Working with communities and their satellite communities and hamlets (unités) that have different ethnic groups in order to design one Action Plan contributes to social solidarity and increases people's participation. • One technical agent can work effectively in six or seven communities and their satellite communities (unités). • Working to improve nutritional status requires addressing literacy, access to water, and controlling and preventing infection. • It is increasingly difficult for the public health service to assume the responsibility for and the continuation of FSIN activities. • Technical Services: FSIN should make an Action Plan and adhere to it so that participants do not get discouraged. The program needs to keep on schedule. 5. Training: lessons learned • Teaching and training contribute to the emergence of community leaders. • The number of trainings in all subjects increased people's capacity to manage their food security. • Traditional birth attendants should have three months of training; they only get one week. • FSIN should build on existing agricultural techniques; it should not impose new ones. • Disadvantaged populations will make positive changes in health behavior if appropriate messages and strategies are used. E. Sustainability The evaluation team asked the participants, TS, and staff about the sustainability of FSIN's program after it ends. The conclusion, with which we agree, is that sustainability rests with the participants. They recognize this and are optimistic about it: "Yes, as the result of everything we have learned and implemented with FSIN, our capacity to manage our food security has increased. When the project leaves we will try to continue and do even better." "We can continue after the project ends; we can do what we learned on our own, we have seen that it is in our own interest." There are many acquisitions in knowledge and practices that participants can use independently, from rehabilitating malnourished children with local foods to seed multiplication to organizing democratic and inclusive committees to address community issues. However, two critical factors take precedence in the discussion of sustainability: first, it can be discussed but not assessed at this point in time; and second, the overall conclusion is that the participants cannot depend on the GON's social or technical services to continue or support the participants' activities when the program ends. FSIN currently is in its fifth year and running well, so the discussions about sustainability are theoretical. Sustainability only can be determined two or more years after the program has ended. FSIN's second phase could track some or all of the Indicator Performance Tracking Table (IPTT) indicators in the communities that have graduated from the program as a sound basis for assessing sustainability, for example. At this point in time there are only opinions. 13 Overall, the staff and the TS recognize that sustainability ultimately and inevitably rests with the participants, because the state does not have the resources to meet the rural population's need for services. This is certainly true in the chronically vulnerable areas where FSIN works, where due to poverty the population's needs are great and its ability to pay for services is not. The effect of decentralization on this equation remains to be seen but it is fundamentally a matter of the state's budget. The TS in all four districts, that are meant to be the key factor in sustainability by continuing to support and promote communities' activities when FSIN ends, consistently reported that they have the competence but not the resources to do so. Their lack of resources—vehicles, funds for gas and perdiem, human resources— varies by district but is a real constraint on getting their work done. FSIN's support amounts to 80% of the TS's budget in one district where the program works, so that the TS there have the means to work with the program. "Communities have to become responsible for themselves because the TS do not have sufficient funding to cover their territory" were the words of one TS staff member. The evaluation team's conclusion is that the TS cannot really contribute to sustainability when FSIN ends. The evaluation team believes that household-level activities such as EBF and improved production techniques are more likely to persist over time than community activities such as cereal banks or tree nurseries. Without the program's social and technical support, and without FFW, people's motivation and their capacity to invest in their communities are likely to diminish. Linking communities to local NGOs and civil society organizations (CSO) is an option for building sustainability into the program. FSIN could help community committees become formal structures recognized by the state as their communities' representatives (“agrément”). This would facilitate their working independently with local NGOs, CSOs, and banks after FSIN ends. F. Recommendations This section contains the evaluation team's key recommendations for the current program, which is in its final year, and those for a second phase of FSIN. The complete set of recommendations for each SO, FFW, and the program's cross-cutting activities are in their respective chapters. 1. Recommendations for the current program 1. Increase the length and frequency of community training sessions, based on consultation with the participants, in order to respond to participant demand. 2. Provide regular training to improve the program technicians' and the TSs' technical expertise, in order to improve technology transfer. 3. Expand vegetable-gardening activities throughout the intervention zone by creating new sites and expanding the existing sites, because vegetable production contributes to households income and micro-nutrient consumption. 4. Identify a mechanism for participants to compensate community volunteers (teachers and local experts), in collaboration with the participants, in order to reduce turnover, because volunteerism has its limits. 5. Teach women to rehabilitate malnourished children in Tanout and Dogondoutchi districts, in order to standardize these activities throughout the intervention zone. 6. Apply partial cost-recovery measures for health care in the rural health centerss, to conform to national health policy. 14 7. Continue the FFW-supported activities of identifying and delimiting pastures in Tanout, Dogondoutchi, and Konni/Illela, in order to decrease conflicts between pastoralists and farmers. 8. Increase the staff in the M&E units: each unit should have one or two data inputters and an assistant to the unit head. The M&E units currently have only one person, who cannot get all the work done on time. 9. Provide solid training so that field staff and the TS partners use their M&E data￾collection forms correctly and understand why the information is needed, in order to improve data quality. 2. Recommendations for the DAP2 1. Include a literacy component in FSIN-2, because illiteracy is a universal constraint on implementing the program. "Food for education" may be an option for supporting women's attendance and reducing dropout rates. 2. Work with FFP to start FFW activities earlier because currently they conflict with crucial, rainy-season agricultural activities that become secondary because people prioritize FFW. 3. Identify site-appropriate IGAs, particularly for the dry season, when both men and women need to generate income. Working with organizations that provide micro-finance is one option. 4. Link communities with local NGOs and civil society organizations that can help them sustain their activities when FSIN ends. 5. Integrate the control of surface water into NRM activities, because the potential to use it exists throughout the intervention zone. 6. Look for partners such as local NGOs that work in animal husbandry, in order to complement FSIN's activities in Tanout where it is a major component of household livelihood strategies. 7. Improve the availability of and access to health services by improving the services in existing health centers, constructing new ones, and building rural roads. 8. Streamline the M&E system by focusing on the donor's and CSs' information requirements in order to improve the quality of work at all levels. "Focus on the information requirements" means providing only the information that is required by USAID and the CSs' headquarters, until the M&E system can do that well. 9. Standardize indicators and data collection and analysis methods in the beginning of the program, before any of them are used, in order to avoid data aggregation and comparison problems later. III. Strategic Objective One (SO1) SO1 is "To strengthen communities' capacities to manage their food and nutritional security." Its purpose is to build people's capacity to identify and manage community activities related to food security. SO1 also includes building the capacity to implement SO2 and SO3 activities, which are agriculture/NRM and health/nutrition activities respectively. Capacity￾building thus is a cross-cutting activity that, as staff noted, is essential for the implementation of and progress in the other SOs. The strategy for building capacity focuses on three areas: 1) establishing community structures (e.g. committees, support groups, cereal/input banks); 2) providing training to improve skills (fishing, agriculture, literacy, children's feeding); and 3) providing education (“sensibilisation”) to raise awareness and change behavior in key areas including health and nutrition (EBF, the consumption of vitamin A-rich foods) as well 15 as agropastoral land management. SO1 takes into account participants' expressed needs for training and teaching in specific areas. HKI's "Radio Clubs," which are a form of self￾teaching (“auto-encadrement”) to teach people about health, are included in SO1. Establishing and managing democratic, inclusive committees (“la vie en association”) is a basic element of SO1 in all four districts. It includes training committee members about democratic processes of group participation and governance, such as the concept that they are responsible to those who elected them. Teaching participants about social diversity and inclusiveness in terms of ethnicity, age, sex, social class also is a basic part of SO1 in all districts. FSIN participants set up numerous community groups; there are core groups such as the Village Development Committee and the Food Security Committee, although they may have different names in different districts. Community radios or Radio Clubs to teach people about health and nutrition are another structure common to all four districts. The following lists of the different committees in the four districts therefore are not exhaustive. FSIN has detailed information on SO1's structures and training—for example, the number and sex of people who participated in training sessions—that will be documented when the program ends. A. Summary of SO1 activities by district 1. Africare, Agadez: • Support and training for community groups: the Village Management Committee, the Food Security Committee, the FFW Committee, the CEWS-UR, health committees (COSAN and COGES); the phytosanitary brigade, the veterinary technicians, and the Breastfeeding Support Group (BSG). • Support for the development of community Action Plans. • Establishment of community savings funds. • Training in a range of subjects: agriculture, animal husbandry, pasture management, health and nutrition (including TBAs, health agents, and model mothers), and in organizational capacity (management of cereal and agricultural input banks). • Support and training for women's groups which manage credit for IGAs. • Literacy training for the members of the "self-training" groups (information diffusion through the training of community trainers). • Establishment of 59 literacy centers. • Establishment of one rural radio station which broadcasts information on relevant subjects (human rights, health, education, agriculture) and reaches approximately 3,250 people. 2. CRS/HKI, Tanout/Zinder: • Support and training for community groups: Food Security and Village Development committees, health committees (COSAN, COGES); CEWS-UR; committees for Vegetable Production, First Aid, Sustainable Agricultural Production, Tree Nursery, and the Radio Club. Traditional birth attendants’ group. • Training in staple crop and vegetable production, use of phytosanitary products, NRM, health and nutrition, exclusive breast-feeding (EBF), knowledge and consumption of micronutrients, • Support and training for the development of community Action Plans and committees. 16 3. CARE, Konni/Illela: • Support and training for community groups: committees for food security, dry￾season production, CEWS-UR, health (COSAN and COGES), livestock-rebuilding (“habbanaye”), fishing, and roads. • Support and training for the Breastfeeding and Child-Feeding Support Group, children's nutrition rehabilitation centers, developing community Action Plans, and community savings funds. • Support and training to improve agricultural production including vegetable gardening; health and nutrition, micronutrient knowledge and consumption, diarrhea treatment; and child-to-child activity. • Support and training for participants to manage health kits, pasture lands, and community grain stocks. • Training and refresher courses for all committee and group members including TBAs, the health and food security committee members, literacy teachers, veterinary technicians, and tree nursery experts. • Teaching through cultural activities: songs, skits, field days. • Support for grain production in collective fields, creating community cereal stocks, and building granaries. 4. CRS/HKI, Dogondoutchi: • Support and training for community groups: Village Development Committees and FFW committees, Breastfeeding Support Groups (BSG), COSAN, GSA, and CEWS-UR committees. • Training of local experts in: committee formation and management, sustainable agricultural production techniques, tree seedling/nursery production, improved agricultural practices, vegetable gardening techniques, accounting, gum arabic production, gender and development, early warning and urgent response, fishing, seed multiplication, TBAs; and community educators. • Support and training for community Action Plans, • Five study trips for 96 participants. • Support for opening five bank accounts to obtain loans. B. Impact 1. SO1 impact indicators SO1's two impact indicators and their change over time are shown in Table III-1 below. The evaluation team was asked for a qualitative assessment of the quantitative impact indicators. This was not feasible for the SO1 indicators, because 1) they are qualitative and 2) the time allotted for interviewing the participants was not sufficient to assess these indicators and cover all the other evaluation questions, so the latter was prioritized. A good assessment of indicator 1.2 alone would have taken one of the two hours allocated for the participant interviews. The midterm survey in 2003 showed that only 61 of the intervention communities had "designed their food security plan democratically and take into account gender and equity" (indicator 1.1). This number falls short of the midterm target of 110 communities. Most of 17 the participant groups interviewed for this evaluation reported that they were in compliance with this indicator, but the Action Plans were not reviewed in detail with the groups so their response is general. The group interviews did show that, outside Agadez, women generally constituted half of the members on a committee rather than FSIN's required 25%. Reviewing the written Action Plans indicated that most took gender into account, in terms of having women's activities and support groups in the plan. Indicator 1.2, the food security capacity index, consists of eight variables and 29 indicators. Time did not allow a qualitative assessment of this indicator. Table III-1. SO1 Impact Indicators SO1 Impact indicators Level Baseline 2000 Midterm target Midterm achieved 2003 1.1 Number of communities that designed their food security plan democratically and take into account gender and equity. FSIN 0 110/142 (78%) 61/171 (36%) 1.2 Food security capacity index. FSIN 35 40.6 50.1 2. SO1 Impact FSIN definitely has had a positive impact on participants' capacity to manage their food security, according to the people interviewed for this evaluation (participants, staff, the TS, and program partners). This positive impact is attributed mainly to transferring knowledge and improved practices in community organization, NRM, agricultural production, and health to the program participants. Participants consistently reported that SO1's training and education were among the best-liked and most useful aspects of the program, particularly in health and NRM. As a result of this training, and with program support, there are behavior changes at both the community and household levels. Communities have established integrated committees, designed Action Plans to improve their food security, and are implementing the activities in those plans, partly with the support of FFW. At the household level, people reported behavior changes such as EBF, increased vegetable consumption, the adoption of improved NRM techniques such as demi-lunes, and the adoption of improved cultural practices for both grain and vegetable crops. The Technical Services (TS), staff, and program partners agreed that FSIN's approach to capacity building was appropriate and effective. They believe that training and education for technology transfer builds sustainability into the program. These people saw SO1's impact in terms of changes in participants' behavior: being more proactive about organizing community-level health care (e.g. contacting the Health Service to come and give vaccinations), being more proactive about agricultural problems (e.g. reporting locusts), building their relationship with the TS, and implementing improved NRM practices with FFW. The people-level impact of SO1 is best expressed in the participants' own words. Participants' answers about changes in "community capacity to manage food security" were primarily in terms of concrete changes in their health and agricultural production. Having community committees and Action Plans were secondary answers. It was not really possible 18 to get answers about the "change in community management capacity" as such, perhaps because it is abstract. It is interesting to note that people's responses about impact are quite homogeneous, despite the social and environmental differences in the areas where the consortium works. The minor variations in their statements about SO1's impact on their lives reflect the site-specific constraints on food security and the specific actions taken to address it. a. Participants Women made the following statements about the impact of SO1 on their lives: • Women no longer hide, socially; now they get out and participate in community activities. It is easier to mobilize women now and they have acquired the confidence that they will succeed at whatever they work on. Women work together better now. • Women are on community committees, but in general their participation is limited due to illiteracy and their workloads. In some communities literacy is not a requirement but it is preferred. • We have learned to better manage our food security because we have learned to rehabilitate our land and techniques such as reforestation to protect the environment. We can teach others. • Before we ate chaff; now we eat wheat. • FFW decreases the number of men who out-migrate; it also protects the environment. It has the double advantage of providing food and enabling people to learn NRM techniques. • The poor are a little better off but they still face difficulties. They still sell firewood and charcoal to earn money for food. Their lives have changed; they are not so hungry now because of FFW. Before FSIN they out-migrated. But the majority are still poor. • Women have learned about many health risks and health-care practices, including children's and household hygiene, EBF, better child-feeding practices, and rehabilitating malnourished children. • Women are healthier and childbirth is easier as the result of taking iron pills. • Exclusively breastfed children are fatter and suffer less from diarrhea and sickness. • The MMD micro-credit fund raised women's spirits: they got training in how to manage and use it, and now they no longer have to go door to door asking for credit and telling everybody their problems, they can get credit from the MMD fund. A woman in Kaiwa Fako, Dogondoutchi district, was very proud of her daughter and was showing her off to the evaluator, much to the amusement of the women in the interview group. The child is about three years of age and the first of this woman's five children to be exclusively breastfed, which she learned from FSIN. The mother said that as a result of EBF her daughter is stronger and healthier than any of her siblings, and there is not a stronger girl in the community. 19 Men made these statements: • The best thing FSIN has done is to organize committees; they are something new, and enabled the community to implement activities. Before FSIN individualism reigned and each family took care of itself; now, there is social solidarity and decision-making is done together. • We have learned that one person alone cannot make a decision; a group of all types of people must assemble—men, women, and the poor. • The community knows how to organize itself now and implement activities, as the result of forming committees. People can continue these activities when the project ends. • Now men and women meet to plan and implement activities related to food security; they work together. Women attend community meetings and speak up; sometimes they have better ideas than the men. • Training is an acquisition; it is an inheritance for the community. • As a result of project training people can meet to deal with food security issues. They can establish or stock community cereal banks, for example. • Millet production has doubled due to new cultural techniques and improved seed, when the rainfall is sufficient. • Food security has increased due to technology transfer and FFW, which allows people to work in their own fields and not others'. • Increased vegetable production due to FSIN's technology transfer has increased incomes and decreased the need for dry-seaon out-migration for the majority of men. • The poor can cultivate more of their farmland because they have rehabilitated some of it, so their production has increased. • The poor can cultivate their own fields because they get seed and FFW. Now they can keep their harvest, instead of using it to repay the credit they had to take during the hungry season. • Pregnant women have fewer problems and deliver in good health as the result of taking iron pills and training for the matrons. EBF protects newborns from illness. People have learned about the importance of different types of local foods that are nutritious. In Boudari village in Agadez, a man said that as the result of FSIN's technical assistance for onion production, his production increased from about 30 sacks on two hectares to 230 sacks. He attributed this 6.7-fold increase to the program’s training and the access it provided to chemical fertilizer, pesticides, and agricultural tools. One result of his increased income was that his wife became the best-dressed woman in the village. An indirect benefit was that he hired five poor people in the community to harvest his crop. A group of men in Agadez said that they used to think that there were things in life that they could not live without, namely their herds and their pastoralist lifestyle. They thought that if they were no longer pastoralists that life was finished. They have learned that they can overcome that significant loss and live with other things. They have learned this with FSIN. Those who lose all their herds and are dispirited can come and work with FSIN and FFW, and stay in their homeland. 20 b. Technical Services and FSIN Staff The evaluators interviewed personnel from the Health, Literacy, and Community Development services and the heads (“responsables”) of SO1 about its impact. The TS's and staff's reports of SO1's people-level impact definitely were positive, like the participants', although the former tended to be more general. For example, one staff member said that "Participants learned that they are capable of getting major projects like road-building done; they were surprised at their collective capacity." The TS pointed out that participants have become more proactive: "There has been a big impact on people's awareness and their mobilization to recognize and address their problems." Representative statements from these two groups of people are in Table III-2 below. c. HKI's Community Radio intervention One interview in Dogondoutchi was focused exclusively on HKI's community radio intervention. It must be noted that this is the only in-depth source of information on this intervention and therefore cannot be generalized to any other communities. The interview was conducted with a group of men that included those responsible for operating the Radio Club based in the community of Dogon Kiria. Unfortunately, the women who had come to talk with the evaluation team were not presented to the team until the interview was finished, so their perspectives on the intervention are missing. The team's appointment in another community did not allow time to talk with the women. The Radio Club in Dogon Kiria has been operating for 16 months. During the agricultural season when people are busy it operates for two and a half hours in the evening; the rest of the year it operates from 9 a.m. until noon. The radio broadcasts reach communities up to 35 kilometers distant that also are part of the HKI intervention and have radios. The radio program's major purpose is to operate "Radio Clubs" in each community, which assemble groups of people to listen to cassettes in order to learn about and discuss health topics (nutrition, hygiene, malaria). The men interviewed in Dogon Kiria said that their cassettes also addressed the fight against poverty. Initially they got cassettes on health from HKI and now they buy their own, on topics such as polio and malaria. The radio program's second purpose is to broadcast public health announcements, such as disease outbreaks or agricultural problems (e.g. locusts). The TS use the radio to transfer information in their technical areas such as health, livestock care, forestry practices, and agriculture. The radio in Dogon Kiria also broadcasts announcements and music, for which people pay about one dollar (500 CFA). The announcements are local news of births, deaths, marriages, and greetings to people who are not in their home communities. People also can request "concerts," which means that, for the same fee, they choose music and dedicate it to their friends, and the radio broadcasts it. The radio's revenue is from the fees paid for announcements and music; the latter was reported to be very popular, especially among young people, and generate most of the revenue. There are three radio agents (“animateurs”) in each of the participating communities who collect people's requests and fees. According to the men interviewed the radio staff's payment is 35% of these fees. 21 Table III-2. SO1 Impact, According to Staff and Technical Services Respondent SO1 Impacts FISN staff • Communities' awareness and taking charge of their development has increased; there are positive changes in community organization, decision-making, and social solidarity. • Participants learned that they are capable of getting major projects like road-building done; they were surprised at their collective capacity. • Village chiefs no longer monopolize community decisions. • Participants now talk to the TS as equals and call on them to do their work. • Participants have come to understand that rehabilitating their fields is a gradual and long-term process, as their soil degradation also occurred over the long term. • FFW enables people to work in their own fields and increase their production, instead of selling their labor to others and having to neglect their fields. • Decrease in the proportion of malnourished children; women know how to treat this problem with local products. Technical Services • There has been a big impact on people's awareness and their mobilization to recognize and address their problems. • Organizing in committees has given people awareness of their own potential to help themselves. • People are better at organizing themselves now. • People can take the initiative and report problems through their CEWS-UR. • People's reticence to deal with the TS has decreased now that they have committees. One result is that more women go for prenatal checkups. • Men and women got literacy training and FFW provided food during the hungry season. • FFW has decreased the need for seasonal out-migration. • The rehabilitation of farmland, enlargement of seasonal ponds, and increase in vegetable production have improved agricultural production and people's lives. • There is a growing awareness of the need for literacy, particularly among women. • EBF is widely practiced now. The main weaknesses with the radio intervention reported by the men in Dogon Kiria were: • The monthly revenue is variable, which makes management difficult in terms of paying good salaries and having funds for maintenance. • Staff want higher salaries. • The community agents (animateurs) need more training; they have had only one session. • They want the equipment necessary to capture BBC and VOA. 22 The men in Dogon Kiria as well as radio club members in many other communities (outside Agadez) reported that they had problems with their equipment. The former said that some of their equipment is broken and that they are not generating sufficient revenue for maintenance and repair. Many of the latter said that they do not have batteries to operate their radios since FSIN stopped supplying them, as long as a year ago. It is contradictory but understandable that, when asked about the Radio Clubs' sustainability, both types of respondents reversed their positions and said that they had the skills to maintain the equipment and the Radio Clubs. Overall, however, the men in Dogon Kiria reported that the radio intervention is a success. FSIN participants in other communities were not asked about the radio program in detail, but in general their comments also were positive. Participants in communities with Radio Clubs reported that they listened to cassettes, discussed the information, and learned about health. The men in Dogon Kiria reported three positive impacts of the radio program: • An increase in social cohesion, particularly among young people who can keep in touch by sending greetings and dedicating music to each other even when they are kilometers apart. • People have learned about health and health care; now they go to the health center when they are sick instead of ignoring the problem. The participants interviewed in other communities with Radio Clubs also reported that they learned about health in the clubs. • It is no longer necessary to travel to the different communities to invite people to social celebrations such as baptisms, because they can be announced on the radio, which saves time and money. Participants in other communities agreed that saving time and money was a real benefit of the radio intervention. C. Strengths and Weaknesses 1. Participants Participants generally were not good at assessing FSIN's strengths and weaknesses as a program. They perceived its strengths in terms of the material support and technical assistance (TA) it provided, such as FFW, cereal banks, and technology transfer that improved crop production. Weaknesses tended to be identified in terms of what they needed from the program. However, they did consistently identify one programmatic weakness: FFW activities started late in theannual agricultural production cycle and conflicted with their rainy-season agricultural production. The latter is the loser in this conflict as, according to staff, people tend to prioritize the FFW activities at the expense of their crop production, which is counterproductive. The other weaknesses that participants reported were: • The nutritional centers need program support because the women who do not have the food to participate are ashamed and stay at home. • There should be more focus on animal husbandry. • Women said that the FSIN technicians (“animatrices”) should come to work with them once a week rather than once a month. • Training sessions are not long enough. 23 2. Technical Services and FSIN Staff It was difficult to get the TS and the staff to focus on SO1 as one component of FSIN and assess its strengths and weaknesses; their responses tended to apply to the program as a whole. The TS's overall evaluation of FSIN was positive; as one person said, "Globally, all the TS are satisfied with the program." The fact that FSIN is in harmony with the GON policies of working toward poverty reduction and decentralization was consistently noted by all the TS and the local administrators. Good collaboration between the TS and FSIN was cited as a strength in all four districts. FSIN's other strengths included the "Strong involvement of the participants and the TS in all phases of the program, the diversity of program activities," and the fact that "The project is participatory and large in scope" (Table III-3 below). The TS seemed to appreciate the program's participatory approach because it engaged the participants and increased communities' responsibility for their development. The participatory approach, forging partnerships, building on traditional social structures, and focusing on the most food-insecure households are some of FSIN's strengths reported by staff (Table III-3). The first two were the most commonly cited strengths. Illiteracy and the fact that "volunteerism has its limits" emerged as constraints on program implementation in all four districts. Lack of community-level M&E systems to monitor progress with the Action Plans also was cited as a common weakness. The two weaknesses that the TS commonly reported were that FSIN staff and its schedule were overloaded, and that illiteracy was a constraint on the program across the board. 24 Table III-3. SO1: Strengths and Weaknesses Respondent Strengths Weaknesses FISN staff • The participatory approach makes the community part of the program. • Working in partnership with those outside the consortium such as Caritas and the TS. • Integration of TS from program inception. • Focus on the most vulnerable households, based on communities' identification of them. • The staff are from the area so they know the language and the customs. • The program is based on existing, traditional social structures. • Self-teaching works because it is done by social equals. • Not having integrated two of the area's major problems into the program: lack of potable water and illiteracy. • Community volunteers (committee members) who are not paid eventually quit. • Need to increase the number of community-based technicians. • The technical staff need more training. • FSIN needs to address illiteracy, which is a constraint, especially for women. • There should be M&E at the community level, done by participants. Technical Services • FSIN has no weak points; the structures and tools and M&E system are all in place to work. It should expand. • FSIN works very well with the TS and has strengthened the collaboration among all the partners. • Strong involvement of the participants and the TS in all phases of the program. • The diversity of program activities. • The project is participatory and large in scope. • The difference with FSIN is that the communities are responsible and manage the FFW and the work tools, not the Environment Service. • Communities did their own Action Plans and feel responsible for them. • FSIN got participants mobilized and taught them things, and extended their work to the central communities' satellite communities, which is good. • Program implementation is done with rigor and there are the means to do it. • Insufficient focus on the area's major problem, lack of access to water. • The pastoralists in the northern part of this area need support and should be included in the program. • Insufficient focus on protecting vegetation for animal and human consumption. • Insufficient support for vitamin A distribution. • FSIN staff are too busy to be easily available to work with; they are hard to find. • As a result of the above, the work calendar is disrupted and so are the activities that were scheduled. • Need to implement activities on time; i.e., adhere to the work calendar • There are too many activities to implement, and not time to do them all well. • Committee members are illiterate, which creates a functional problem for the committees. • FSIN uses forms that illiterate participants cannot fill out. 25 D. Sustainability Participants consistently said that they will be able to continue what they learned to do with FSIN when it ends. As one group of men stated, they will continue to work toward improving their food security by combining their own experience and the knowledge they have acquired with FSIN. However, it must be noted that virtually all the interview groups pointed out that they depend on FFW for two to three months of the year, and that if it ends they will have to resort to their traditional coping strategies: the men will migrate out during the dry season (exode) and the women will stay home and scrape by, engaging in petty commerce and selling their labor to earn money for food. According to some staff members, it is the competent men who leave, which affects community committees and activities. FFW clearly is essential in enabling people to work with FSIN because it replaces these coping strategies. Its significant benefit is that it enables people to invest their labor in improving their own farmland and community resources, instead of others'. Over the long term this should improve their food security. But given where FSIN works—in regions that are structurally or chronically food insecure—out-migration may always be necessary for part of the population. The conventional wisdom in Niger is that rainfall accounts for 80% of the variation in crop production, so in FSIN's districts rainfall is a major determinant of production and therefore food security. Improved NRM and cultural practices can counteract rainfall problems, to some extent, but assessing to what extent—i.e., the sustainability of FSIN's program—must be done over the long term, perhaps five years. The effects of food insecurity and out-migration on communities' implementation of their Action Plans over the long term, without program support, remains to be seen. This is particularly true of the more vulnerable households who, without support, can afford only limited investments in NRM in their own fields and thus are trapped in the negative cycle of low production and the need to sell their labor. Both the women and the men pointed out this problem in field interviews. In a few communities some men reported that they hire labor to make demi-lunes in their fields because it increases production, but this is certainly the less￾poor hiring the poorer. Any future evaluation of sustainability therefore should be done using the three classes of vulnerability that FSIN defined. The participants generally were the most optimistic about FSIN's sustainability. In all four districts their statements about continuing without program support were quite homogeneous: • Yes, due to everything we have learned and implemented with FSIN, our capacity to manage our food security has increased. When the project leaves we will try to continue and do even better. • We want the project to continue but if it does not we will continue to work with what we have learned. • We can continue after the project ends; we can do what we learned on our own. • We will continue the activities because we have seen that they are in our interest. In principle the TS should contribute to sustainability because their job is providing TA to their areas' communities, but they themselves recognize that that is problematical. Their summary statement in all four districts was that they have the competence but the not the means to do their work. The FSIN staff's overall appraisal was the same: that the TS could not provide the same level of TA without program support, due to lack of means. The FSIN staff were positive about the program's sustainability, but their statements should be taken 26 with a grain of salt because they also know, based on other projects' experiences, that many of the activities they are supporting—cereal and input banks, community savings funds (“caisse epargne communautaire”), forming committees, phytosanitary brigades— disintegrate when projects end. Table III-4 on the following page contains the staff's and the TS' statements about FSIN's sustainability. E. Lessons Learned Most of the "lessons learned" reported in the interviews for this evaluation, like those in FSIN's pre-evaluation reports, actually are conclusions. The conclusions about SO1 collected during this evaluation have been rephrased as "lessons learned" to the extent possible. People's lessons learned and conclusions did not always focus on SO1; they often addressed the program as a whole. (A lesson learned is something that the program learned it should or should not do, and generally includes the word "should," e.g. "Women should not be prioritized for FFW because socially men have priority to do FFW;" "The program should plan for the fact that awareness-raising is a slow process." A conclusion is a statement of fact that the program recognizes: "Capacity-building is essential for the success of other program components.") High illiteracy rates and the constraints that illiteracy places on both communities' and individuals' capacity were noted by both staff and the TS in all districts. The general conclusion was that FSIN2 needs to address illiteracy in order to promote progress in all its sectors. Many of the staff's lessons learned (Table III-5) reflect gender issues: women's literacy increases their participation in community decision-making and committees; women need support to participate in community affairs; FSIN should work with men and women separately to build capacity, until they are ready to work together; the program needs to push in order to ensure that women are put on committees. Staff also have learned that raising awareness and changing behavior are long-term processes. The TSs other lessons learned included the need to include water and animal husbandry components in FSIN (Table III-5). After four years of FSIN, both staff and the TS concluded that "volunteerism has its limits" and both were thinking about how to address this issue (Table III-6). One TS person said that participants are asked to contribute money, time, and labor to a plethora of activities and committees, and that it becomes a burden. Staff noted that capacity-building is a cross￾cutting program component necessary for success in the other components. The need to support women's development shows up in people's conclusions, just as it does in lessons learned: with time and training women's social participation increases, this process takes time, and women who generate income get more respect at home (Table III-6). 27 Table III-4. SO1: Sustainability Respondent Sustainability Factors FSIN staff • Sustainability is built into the program as its activities, for example cereal banks, function independently. • People will continue the economic activities they started with FSIN. • The activities that communities decided to implement themselves, like cereal banks, will continue. • Community links with the TS and the inclusion of traditional chiefs are factors in sustainability. • Linking community CEWS-UR with local TS, and with departmental and regional systems, will contribute to sustainability. • It is questionable: will NRM continue without FFW? • The customary chiefs said that they will support community activities. • The TS will work with the communities to do M&E and report progress on activities when FSIN ends. • People probably will continue the activities that are most important to them and their community, e.g. roads, vegetable gardening. • 75% of the communities will have the capacity to continue some of their activities, because 1) they have democratically-elected committees and 2) women and the poor are the most numerous in the communities and they are involved in the committees. • FSIN provides training, and it is the basis of knowledge, the implementation of new techniques, and their diffusion, that can continue when FSIN ends. • Participants have come to understand that resolving their problems (e.g. soil degradation) is a long and gradual process. • An example of sustainability: training on EBF now is done by the women themselves, the program technicians are no longer necessary. Technical Services • In general, the TS have the competence but not the means (vehicles, gas, funds for per diem) to do their work. The Agadez/Arlit area is vast, and we need program support in order to cover the area. Our technicians should visit communities more frequently, so they will need more resources from FSIN. • Communities have to become responsible for themselves because the TS do not have sufficient funding to cover their territory. • The TS are in place to continue FSIN's work but they need strengthening. • Participants have learned how to collect money and pay for what they need, such as health-care visits from the Health Service. This service can provide healthcare as long as people pay for their gas and meals. • The participation of the other TS is more problematical because they have less funding than Health. • Decentralization is a factor in sustainability. • How participants will manage their food security when the program ends remains to be seen; it is problematical. • Sustainability is built on the program's training and the communities' elected committees. • People's increased awareness will contribute to sustainability. Capacity-building makes participants more proactive in addressing their problems. People's knowledge about health will remain with them. • Committees such as COSAN and CVA will always exist and do their work. • Participants pay for healthcare now, but will they do so when FSIN ends? • Training is a primary factor, such as training for the committees: it is the development of human capital. 28 Table III-5. Lessons Learned Respondent Lessons Learned FISN staff • The program should recognize and plan for the fact that raising participants' awareness is a process that takes time. • FSIN should recognize that behavior change is a long-term process. • FSIN should recognize that literacy is necessary for the CEWS-UR to function. • The program should continue to support literacy classes, especially for women, because literacy is essential for committees to function well. It also is necessary for key activities such as monitoring children's growth. • FSIN should support women's participation because they are reticent about participating in community affairs. • The program should provide numerous training sessions in all subjects because it increases people's capacity to manage their food security. • The program should be prepared to push community participants in order to get both sexes on committees. • FSIN should build on existing agricultural techniques; it should not impose new ones. • FSIN should work with communities' satellite communities (unités) that have different ethnic groups in order to make one Action Plan, because this approach contributes to social solidarity and increases participation. • FSIN should work with men and women separately to build community capacity, until they are ready to work together. • Other projects' experience shows that input banks disintegrate when the projects end; FSIN should determine how to increase access to agricultural inputs, including seed. Technical Services • FSIN's entry point should be the population's primary problem, which in Tanout is lack of water.. • FSIN should make an Action Plan and adhere to it so that participants do not get discouraged. • FFW should be used in the hungry season. • Matrons should have three months of training; they only get one week. (Note: ISAN provided 10 days of training because only some of the state’s two-week training module is for midwives). • FSIN should include animal husbandry activities. • FSIN should address illiteracy because it is a serious constraint on participants' capacity to manage their committees and activities. 29 Table III-6. Conclusions Respondent Conclusions FISN staff • Volunteerism has its limits; people will drop out of committees if they are not paid. • Capacity-building is an essential element necessary for the success of other program components. • There is lack of social cohesion in communities that have more than one chief, and it takes time to get people to work together as one community. • Communities can be divided by people's allegiance to different political parties, but FSIN can overcome that as it is apolitical. • With training, women's participation in community affairs and decision￾making increases over time. • Large numbers of able-bodied people out-migrate after a poor agricultural season. • Teaching and training contribute to the emergence of community leaders. • One technical agent can work in 6-7 communities and their outlying hamlets (unités). • Increasing women's participation in community planning and decision￾making is a process that takes time. • Committee members travel extensively during the hungry season [to find work]. • Illiteracy levels are high in community groups. • Women who engage in IGA and generate income have more influence in their households and with their husbands. Technical Services • Volunteerism has its limits; the program needs to identify some type of compensation. • Literacy is necessary for committees to function, for matrons to function, and for participants to fill out the forms to monitor their progress. • Women's dropout rate in literacy classes is high because learning is more difficult and takes longer than they realized. F. Recommendations 1. For the current program 1. Continue training and education (“sensibilisation”) for capacity-building, technology transfer, and behavior change because it is essential for progress in the other SOs, there is strong participant demand, and it contributes to sustainability. 2. Increase the length and frequency of community training sessions, based on consultation with the participants, in order to respond to participant demand. 3. Provide regular training to improve the program technicians' and the TS’s technical expertise, in order to improve technology transfer. 4. Explore options to compensate community volunteers (committee members) for their work in order to reduce turnover. 5. Provide training in machine maintenance and repair for the HKI Community Radio staff. 6. Consolidate the community committees in order to reduce their number. The CEWS-UR should be part of the food-security committee, for example. 30 7. Implement a surveillance system to ensure that FFW commodities and the community recipients are not misused by the community distributors. . 2. For the DAP2 1. Work with FFP to start FFW activities earlier because currently they conflict with crucial, rainy-season agricultural activities that become secondary because people prioritize FFW. 2. Invest the time in designing a sound detailed implementation plan (DIP) so that the program has a solid basis for its annual plans. 3. Invest the time in meeting with partners to make a feasible annual work schedule and then adhere to it, so that activities are implemented and completed on time. 4. Link communities with local NGOs and civil society organizations that can help them sustain their activities when FSIN ends. 5. Include a literacy component in FSIN2, because illiteracy is a universal constraint on implementing FSIN's program. "Food for education" may be an option for supporting women's attendance and reducing dropout rates. 6. Complete the community activities that were not completed in FSIN1. Include a "nomad school" education component in Agadez because it is a feasible means of providing education for pastoralist's children. 7. Identify site-appropriate IGAs, particularly for the dry season, when both men and women need to generate income. 8. Expand MMD's ("Women on the Move") role in FSIN2 because it is successful and women need access to microfinance for IGAs. 9. Look for partners such as local NGOs that work in animal husbandry, in order to complement FSIN's activities in Agadez and Tanout where it is a major component of household livelihood strategies. 10. Incorporate education on HIV/AIDS into health education because seasonal migration, and improved roads are risk factors. The "SIDA en Exode" project is a potential partner. 11. Include education on good governance, in order to help people work with decentralization. 12. Increase the number of community-based program technicians (“animateurs”) in order to meet participant demand for technical assistance. IV. Strategic Objective Two (SO2) A. Summary of the key activities in the districts Cooperating sponsors implemented several types of activities to achieve SO2, "To increase sustainable agricultural production by promoting environmentally sound cultural techniques." Most of these activities were protecting or maintaining existing productive capital; training also was done. The principal activities that the CSs implemented are summarized in the table below. 31 Table IV-1: Summary of Activities in Each District Zone Key FSIN Activities Agadez • Establishment of community input and agricultural equipment banks, cereal banks, and livestock feed banks. • Training and equipping veterinary technicians • Training and equipping phytosanitary brigades • Constructing and rehabilitating wells for vegetable production • Provision of draft animals for irrigation • Construction and rehabilitation of livestock wells • Installation of traps for jackals • Rehabilitating land using NRM techniques Tanout • Constructing wells for vegetable production • Installation of irrigation systems (niyya da kokari pumps) • Provision of improved seed for millet, sorghum, beans, and vegetables • Creation of community tree nurseries • Rehabilitation of land using NRM techniques (demi-lunes, zaïs2 , mulching) • Training in sustainable agricultural techniques (use of manure, protection of natural regeneration, improved pruning, mulching, composting) • Enlargement of ponds • Stocking fish in ponds Konni/Illela • Building up community seed stocks • Rehabilitation of land using NRM techniques • Disseminating improved techniques (improved seed varieties, alternative strategies for pest control, use of pesticides) • Promotion of irrigated crops (installation of wells and the construction of thresholds (“seuils”) • Habbanayé3 • Establish of livestock input banks • Training, retraining, and equipping of veterinarian technicians • Stocking fish in ponds • Training and equipment of fishermen • Delimitation of corridors for livestock Dogondoutchi • Construction of wells for vegetable production • Installation of irrigation systems (niyya da kokari pumps) • Provision of improved seed for millet, sorghum, beans and vegetables • Creation of community tree nurseries • Rehabilitation of land using NRM techniques (demi-lunes, zaï, mulching) • Training in sustainable agricultural techniques ( use of manure, protection of natural regeneration, improved pruning, composting) 2 Zaïs : shallow round holes dug about a meter wide which conserve moisture and are treated with manure and other organic matter ; also called « tassa. ». 3 Habbanaye: giving an animal to an individual for rebuilding his livestock holdings. This practice is common among the nomadic populations in Niger. 32 • Enlargement of ponds • Stocking of fish in ponds SO2 has had a strong level of achievement in all the CSs and their activities, and the strong participation of local communities in the activities. Activities were designed and implemented based on Action Plans set up by mutual agreement between the communities and FSIN. The communities' strong participation also is due to ISAN’s participatory approach; the contribution of the communities to funding and building infrastructure; training and awareness-raising; and "self-teaching" through the program's training of local experts, community development committees, and food security committees. FFW also contributed to community participation. SO2's various activities made a considerable impact on participants' livelihoods, as a result of community participation and motivation. B. SO2 Impact The FSIN program produced useful results for local communities as well as for the consortium. Despite the difficult context in which the program operated, the field teams have shown the capacity to adapt and to make remarkable innovations in program implementation. The program operates in a context characterized by: • Marked climate risks; • High illiteracy rates; • High rates of seasonal out-migration (exode) in the target populations, which often affects the capacity for community-level self-teaching; • A low potential in natural resources, particularly water resources and irrigable land, coupled with severe environmental degradation; • The intervention zone's lack of roads and therefore communities’ isolation. • Poor social cohesion in some communities and frequent intra- and inter-community conflict. All of these factors contributed to limiting the program's impact. SO2's impacts can be summarized specifically as: • The gradual adoption of the technologies disseminated by the program (i.e., the replication of improved cultural techniques in participants' fields) and an "opening up of the spirit," as participants frequently reported, as a result of the program's training. For example, according to the person in charge of Rural Development (Génie Rural) in the department of Tchirozerine, communities have learned how to make rock cordons (correct spacing, placement, the correct direction and height to resist bad weather), as well as how to construct demi-lunes. The community of Sabon Kaffi in the department of Tanout had the following adoption rates for improved agricultural techniques: 60 % for “zaïs”; 100% for using manure on the fields; 80% for making demi-lunes; and 100% for improved clearing and protecting natural regeneration. The participants' growth in awareness enabled FSIN to develop the participants’ management capacities, to improve local governance through planning meetings, and to push to increase women's involvement in community decision￾making. These factors contribute to the program’s sustainability. 33 • An increase in agricultural production due to (1) an increase in the area of arable land (land rehabilitated with demi-lunes, zaïs, the establishment of new vegetable-gardening sites or the rehabilitation of old ones) ; (2) improvements in productivity (increase in yields due to the introduction of improved varieties and agricultural techniques) ; (3) securing production through water control, as illustrated by the testimony below: My name is « Wamachecka, » that means "the smallest" in Tamashek. Before I worked with FSIN my well collapsed each year during the rainy season, which made me lose two to three months of vegetable production. With FSIN’s support I benefitted from a well, enlarged my field, and diversified my vegetable production by adding onions, tomato, and Greek fennel. I sold these crops and bought my own draft animal for irrigation. Today I have become « the biggest ». The increase in agricultural production, including vegetable production that provides food for consumption and for sale, contributed to improving the food security of participants in all the districts. FSIN's indicators, ”the number of months of food the participants produce themselves" and "the volume (MT) and value (CFA and USD) of agricultural products chosen by the participating producers," have increased slightly over time, as Table IV-2 shows. Table IV-2: Level of Achievement of SO2 Impact Indicators Indicator Baseline (2000 & 2001) Level of achievement in year four, 2004 Change, baseline to 2004 Final target Number of months of food the participants produce themselves 6.11 6.44 +5.4% 8.16 Volume (MT) and value (USD) of the agricultural products chosen by the participating producers 1.321 MT/HH* $129 1.394 MT/HH $313 +5.5% 1.630 MT//HH (Source: S. Bakari, 2004) *HH = household This table shows a weak increase in FSIN's two impact indicators, which increased only about 5% during the program's four years. However, these figures do not reflect the opinions of the participants interviewed by the evaluation team. The participants reported notable improvements in crop production when rainfall was good and when they used the new techniques (e.g. mulching, zaïs, manure application). They reported that these techniques are effective when there is sufficient rainfall. The small change in these impact indicators underlines the strong link between agricultural production and rainfall in Niger. The evaluators found that the areas of Agadez, Tanout, northern Dogondoutchi, and Illela have had practically four years of drought. Without the control of water that irrigation provides for production, it is difficult to improve agricultural production in these unfavorable climatic conditions. Therefore, it is illusory or impossible to reach the target levels set for these impact indicators, without an emphasis on irrigation. 34 • The diversification of agricultural production, mainly of vegetable production (“culture maraichere”). FSIN introduced a range of vegetables: onions, cabbage, Greek fennel, lettuce, watermelon, melon, tomato, carrots. For example, in the Tamaghate community in Irhazer (Agadez), that historically is a livestock-production area, participants working with FSIN produced a variety of vegetable crops such as watermelon, onions, peppers, sweet peppers, tomatoes, squash, and Greek fennel. Before ISAN, they had produced only melon. A similar change occurred in Air, where cultivating onions became possible during the rainy season because FSIN helped install wells for vegetable production. Some communities in the department of Tanout did not have the capacity for vegetable production before FSIN’s intervention. Vegetable production contributes to improving households’ micro-nutrient consumption. • Increases in income were reported to the evaluation team everywhere where vegetables are produced and sold. Significant onion production was reported in Agadez and Konni, although the problem of marketing also was reported. The areas of Tanout and Dogondoutchi specialized in cabbage production, which also produced significant revenues. Similarly, fish production by fishermen trained by FSIN provides income as well as a source of protein. In some cases an increase in producers’ income is followed by their investment in irrigation, in particular the purchase of motor-driven pumps. This has occurred in the community of Egandawel (Agadez) and in the Konni/Illela area. For example, according to the Imam of Egandawel, one year his son harvested 70 bags of onion during the rainy season, due to FSIN’s technical assistance. He sold each bag for 8,000 francs, and with the income bought a motorized irrigation pump and a bicycle for his brother who had worked with him. • Positive impacts on the environment were reported by the communities in all four districts and by the TS. Different types of vegetative cover and trees have regenerated as the result of FFW activities. This environmental regeneration and the support from FFW enabled households to maintain their livestock, instead of selling them to buy food. • FFW and FSIN's use of the traditional "habbanaye" practice helped rebuild household livestock holdings, particularly among the most vulnerable households. FFW supported the poor so that they did not have to sell their livestock during the hungry season in order to buy food. One result was a change in the percent of households in the different vulnerability classes, including an increase in the number of the “moderately vulnerable” and the “vulnerable,” and a decrease in the percent of the “most vulnerable.” Table IV-3 below shows these changes. Table IV-3: Change in Vulnerability Classes in Agadez Vulnerability category Percent at baseline, 2000 & 2001 Percent in year four, 2004 Change, baseline to 2004 A, Vulnerable 5 12.7 +7.7 B, Moderately vulnerable 25 32.1 +7.1 C, Most vulnerable 70 55.2 -14.8 Source : Africare, 2004 35 • Building roads with FFW helped to open up the intervention areas, which facilitated the transportation and marketing of agricultural products. • There was a reduction in the number of conflicts between pastoralists and farmers in areas where forestry and improvements to manage land were made: delimiting livestock corridors and pasturelands, enlarging ponds for livestock, digging or rehabilitating livestock wells. C. Strengths and Weaknesses 1. Strengths The FSIN program has several strengths: • A strong emphasis on training: the program based all its activities on training and raising the awareness of the participants. This contributed to facilitating the implementation of program activities and their replicability in the intervention zone. In most cases the training was designed to create local experts (veterinary technicians, seed multipliers, phytosanitary technicians, well diggers, nursery technicians), who should continue their activities when FSIN ends. These local experts were operational when this evaluation was conducted, and they had mastered practically all the technologies that FSIN had disseminated. In certain cases there was also an exchange of local experts and thus experience-sharing among communities. • Use of participatory approaches: all the activities planned at the community level were made by mutual agreement between FSIN and the community members, by discussing and renewing community action plans each year. This facilitated the communities' participation in implementing the plans, because their concerns were largely taken into account. The organization of community forums and in certain cases inter-community forums allowed the sharing of best practices in the intervention zone. • Partnership with the TS and the administrative and traditional authorities facilitated the program’s visibility and credibility with the districts and the state. In several cases, the involvement of the TS in program implementation enabled them to do their regular government work in their zones as well as to implement development activities with FSIN. According to the head of the TS’s Environment Department in Dogondoutchi, working with FSIN enabled them to do 80% of their government activities. Certain local NGOs like « GYARA » in Konni/Illela and CAD in Dogondoutchi, in addition to the TS, also have developed their capacities to supervise development strategies and develop project proposals by working with FSIN. • Food for Work was an excellent entry point for the program and one of the strengths most frequently cited by the participants. FFW gave many communities access to markets and transportation by supporting road-building, which facilitated participants' marketing, particularly of their vegetable production. • Targeting the most vulnerable households through the traditional practices of “gayya” and “habbanaye” is another FSIN strength. "Gayya" is a day of communal work in a person's field for which the voluntary labor is given meals; FSIN adapted FFW to this custom. The targeting for “gayya” ensured that FSIN worked with the neediest households, 36 particularly those headed by women. For example, the community of Goumbi Kano in the Konni/Illela district took the initiative to organize a “gayya” for 40 very vulnerable households in the first year of the program, 30 households in the second year, and 30 households in the third year. These “gayya” consisted of making demi-lunes in the recipients' fields. • Community training by qualified program agents. • Providing the participants with seed that is short cycle, very productive, and well adapted to their environments (millet, sorghum, beans). This enabled communities to build up their grain stocks, which are evolving into seed banks. • "Study trips" between communities allowed participants to exchange experiences and provided positive learning opportunities • Installing irrigation equipment made it possible for participants to increase their vegetable production in FSIN's intervention zone, mainly in the district of Agadez. • Delimiting livestock corridors and pastureland, and involving the Land Tenure Commission in these activities, was highly appreciated by the communities, although not all the CSs implemented these activities. Delimiting corridors and pastureland helped regulate conflicts between farmers and herders, and there is a demand for them. 2. Weaknesses Despite the program’s strengths there are some weaknesses that should be taken into account to improve implementation: • Delays in promoting vegetable production: Although much has been done in this area, the evaluators noted some delays in implementation. In Konni/Illela there is strong water potential in pond outlets that is not yet exploited. In Agadez, the most vulnerable households’ access to irrigation infrastructure is limited by FSIN’s requirement that they make a financial contribution to the program’s investment. Even households with access to irrigation infrastructure have difficulting financing their vegetable production, so they generally seek loans from usurers. In the CRS/HKI zones the small size of plots and the lack of fencing (allowing damage by animals) are constraints on vegetable production that participants often cite. The CSs generally are not addressing post-harvest issues such as processing and marketing, despite their strong value-added potential for producers. However, the CSs did conduct a study on how to address those issues in order to understand what actions to take. Input supply problems arise where the program does not provide inputs to the communities4 . Therefore the problems of organizing the producers for acquiring inputs and processing and marketing their production remain. • Insufficient or inconvenient training: many participants regret the very short duration of training. Local experts express the need for more retraining. The timing of training sometimes conflicts with agricultural work, which makes participation in the sessions problematic. This in turn limits the training’s scope and effectiveness. 4 Africare provided capital for agricultural inputs at the community level which is managed directly by a committee in charge of renewing the stock each year; CARE contributed to establishing ties with approved suppliers of inputs; and CRS/HKI provides seeds directly to market-gardening sites. 37 • Inconvenient FFW timing: the timing of FFW activities also is not always favorable, especially in the southern zone (Konni/Illela, Dogondoutchi) where the rainy season arrives earlier than in other areas of the intervention zone. The timing of the FFW activities generally conflicts with agricultural work, forcing participants to choose between the two. • Partnership issues: although no major problems are reported in the FSIN program’s partnerships with local communities and administrative and traditional authorities, both the TS and local NGOs deplored the lack of FSIN’s logistical support for their operations. Evidently this type of support was not planned in the DAP, but as FSIN bases its sustainability on partnership, the weakness deserves mention here. Partnership with FSIN apparently did not include specific training to strengthen the local NGOs’ management capacity and autonomy. In many cases the partnership with FSIN was limited to the local NGOs’ providing only services, that in some cases was limited solely to paying participants their perdiem. • Limits reached on volunteerism: one of the strengths of the program has been to train community experts to continue to diffuse the techniques they have learned with FSIN without program support. These experts were chosen by their communities for their human qualities and have worked as volunteers since the program began. The evaluation team found that many of them were dissatisfied with working for no pay, and their communities were starting to ask themselves how to compensate them. A program is very often designed to cover such a commitment. The limits of volunteerism are clearly being reached in the FSIN population. There is also a strong tendency of the local experts to migrate in the dry season, that was specifically reported in the Tanout district. This poses a problem for the sustainability of local experts’ work and the dissemination of their expertise. • Dissemination of improved seed: the evaluation team found weaknesses related to (1) delays in supplying seed at the community level; (2) insufficient quantities of seed provided; (3) lack of organization of the seed producers; and (4) difficulties in monitoring demonstration fields. • Finally, the communities expressed concerns about issues that are not yet sufficiently taken into account by FSIN, that have limited its impact and effectiveness. These are: (1) questions related to literacy and education in general, (2) the obvious problem of lack of water in the district of Tanout, (3) questions relating to IGA, and (4) the need for animal husbandry activities in the districts of Tanout and Dogondoutchi. D. Sustainability The evaluation team is pleased that sustainability is part of all the CSs’ action strategies. Sustainability generally is based on: (1) training and installing local experts; (2) having communities contribute to building infrastructure; and (3) partnership with the TS and local NGOs. The CSs aimed to legitimize local strategies and match program activities with the needs expressed by local communities. Overall, the community training produced excellent results. The communities interviewed reported that there has been a gradual adoption of all the techniques and sustainable agricultural technologies initiated in the intervention zones: making demi-lunes, zaïs, 38 composting, improved pruning, protection of natural regeneration, and the use of manure. The evaluation team observed that demi-lunes were systematically dug in private fields at all the sites visited. Understanding and replicating these techniques constitute a basis for the sustainability of FSIN’s SO2 activities. However, these activities do not eliminate the effects of climate risk, and are only effective in periods of good rain. FSIN should take this into account in disseminating the technologies, so that participants are not discouraged after a poor production year. The fact that communities have participated in and contributed to building infrastructure also constitutes a basis for sustainability, because the communities have adopted the the infrastructure. The evaluation team believes that the elements of cost recovery should be introduced for all the SO2 activities, particularly those that require annual supplies (seed, small agricultural equipment, inputs). The basis of cost-recovery already is being implemented by Africare in Agadez, by setting up Community Development Funds with program support and community participation. This should be done throughout the intervention zone. The capacity of the TS to ensure FSIN sustainability is doubtful because they support this goal, they generally lack the logistical, financial, and in some cases even the human resources to ensure the continuity of the program activities. The TS in some districts are only operational when there is a program in their area that supports their work. FSIN also must take into account that their adaptive capacity is problematical. The local NGOs have greater adaptive capacity than the TS, as well as the advantages of being more flexible and closer to the communities in terms of social relationships. Partnership with the local NGOs is a good basis for sustainability, even if there have been weaknesses in collaboration with FSIN. The collaboration should be maintained and reinforced if possible. FSIN’s second phase could include a major component for strengthening the local NGOs’ institutional and implementation capacities. E. Lessons learned Both the participating communities and the consortium benefitted from FSIN in terms of lessons learned: • Increasing agricultural production is a difficult objective in an environment where the climate is a major risk factor and plays a dominant role in production, regardless of the agricultural production techniques used. Controlling water—irrigation—is the only sure means to increase production. • If the constraint of unpredictable rainfall were removed by increasing the land under irrigation, the dissemination of simple agricultural technologies that people can easily master could contribute to increasing agricultural production, and to protecting and rehabilitating land. Positive results would strengthen community support of such technologies. 39 F. Recommendations 1. For the current program 1. Reinforce vegetable production activities throughout FSIN’s intervention zone by creating new sites, enlarging existing ones, installing irrigation, and protecting the sites, in order to supplement household micro-nutrient consumption and incomes. 2. Address producers’ needs for agricultural loans by helping them access a formal micro￾finance network, which would allow them to avoid usurers and increase their production profits. These micro-finance networks exist in each district of FSIN—for example, the Rural Credit Intermediation System (SICR KOKARI) in Agadez. It is important to take into account the experience with rural credit in each district, particularly with regard to interest rates in a Muslim society. 3. Organize producers in cooperatives to improve marketing, input supply, and post-harvest processing activities. Agadez has an established tradition of cooperatives so the FSIN producer cooperatives could join those that already exist. 4. For rain-fed crops, reinforce communities’ supply of short-cycle seeds and better organize their system for reproducing seed. 5. Help producers acquire agricultural equipment, to reinforce sustainable agricultural improvements. A system of cost recovery for the equipment should be set up through negotiation with communities in order to ensure the sustainability of this activity. 6. In collaboration with participants, identify a mechanism to compensate community volunteers (teachers, local experts) in order to reduce their drop-out rates. 7. Reinforce support for pest control, especially in the zones where pest infestations are frequent (Tanout, northern Dogondoutchi and Konni/Illela), by training and equipping phytosanitary brigades. 8. Help newly-established community groups (CVD, CUSA, CGV) obtain official recognition (“agrément”) as community representatives, in order to strengthen their role as official community representatives and make it sustainable to work with other programs in the future. 2. For the DAP2 1. Support IGA for women and men to address the chronic food deficit and the weak irrigation potential in the intervention zone. 2. Implement activities related to eliminating illiteracy. 3. Standardize the consortium’s activities for animal husbandry and in three related areas: preventing and managing farmer/herder conflicts, delimiting livestock corridors, and livestock production and health (training veterinary technicians and installing livestock input banks). The consortium could seek a partner for the implementation of this recommendation. 4. Integrate the issue of lack of water in Tanout into program activities. This could be done through a partnership or developing a project exclusively focused on water, due to its high cost. 5. Integrate activities to mobilize surface water to exploit the existing potential in the intervention zone (mini-dams, banks to contain manure [seuil d’épandage ] ) in order to increase irrigation and agricultural production. 6. Support local NGOs and civil society in general to make FSIN activities more sustainable. 40 V. Strategic Objective Three (SO3) SO3 aims at improving the nutritional and health status of households, in particular that of children less than 5 years old and women in the FSIN intervention zone. A. Summary of key activities The following key activities were implemented to achieve SO3: • Creating community groups to support immediate and EBF and to support improved children feeding practices; • Establishing community rehabilitation centers for malnourished children in Agadez and Konni/Illela; • Distributing iron tablets at the community level; • Establishing health committees (COSAN) and training their members, as well as training for traditional birth attendants (TBA), community experts for health (EVS) and nutrition (EVN), and local experts who sell drugs; • Training health personnel; • Establishing community radios which disseminate messages to increase women’s awareness about improved health and nutritional practices; • Building and equipping health infrastructures. A marked characteristic of SO3 is that the same health/nutrition activities were not implemented in all of the program’s districts. For example, rehabilitation centers for malnourished children have been established in the districts of Agadez and Konni/Illela, but not in the districts of Tanout and Dogondoutchi. The key activities implemented in each district are summarized in the table below. 41 Table V-1. SO3: Key Health/Nutrition Activities in the FSIN Districts Zone Key activities Agadez • Promoting best practices for breast-feeding and child nutrition through the creation of community support groups; • Treating diarrhea at home through Information, Education, Communication (IEC) and training mothers; • Promoting production and consumption of food rich in micronutrients through IEC, training and creating vegetable gardens (AKG); • Distribution of iron tablets at the community level; • Establishment and training of community health committees (COSAN), and local health and nutrition experts (EVS and EVN); • Training of health personnel; • Equipment of several rural health centers ; • Development of a partnership with the government health services. Tanout • Promoting best practices for breast-feeding and child nutrition through the creation of community support groups; • Treating diarrhea at home for children under five; • Improving access to health and nutritional education through the mass media; • Establishment and training of COSAN; • Training of health personnel and community health volunteers; • Equipment of several rural health centers; • Development of a partnership with the government health services. Konni/Illela • Promoting best practices for breast-feeding and child nutrition through the creation of community support groups; • Promoting the consumption of micronutrients for children under five, pregnant women, and women who breast-feed their children; • Treating diarrhea at home for children under five; • Improving vaccination coverage for children under five, particularly for measles. Doutchi • Developing a partnership with the government health services; • Promoting best practices for breast-feeding and child nutrition through the creation of community support groups; • Treating diarrhea at home for children under five; • Improving access to health and nutritional information through the installation of 2 community radios and 60 “radio club” volunteers; • Establishment and training of community health committees (COSAN); • Training of health personnel and community health volunteers; • Construction and equipment of several rural health centers. 42 B. SO3 Impact SO3’s overall impact is quite positive. This conclusion is based on a quantitative evaluation of SO3’s three impact indicators, and a qualitative evaluation by participants. The impact indicators are: 1) the prevalence of stunting among children (defined as 24-59 months of age); 2) the proportion of newborn infants breast-fed within eight hours of birth; and 3) the proportion of communities in which vitamin A consumption is above the risk level. There is positive change in all three indicators at the program level, and indicator number two reached 99% of its final target by 2004. During FSIN’s four years the proportion of stunted children stunting has decreased by almost 9% and the proportion of communities whose consumption of vitamin A is above the risk level has almost tripled. These are positive impacts, although continued progress is still necessary to meet the program’s final targets for indicators one and three. Participants’ were clear and enthusiastic about SO3’s positive impacts on their lives. They have acquired knowledge about how to improve women’s and children’s health and nutrition, and they have applied it. For example, both sexes have learned about EBF, that vegetables provide vitamin A and other micronutrients, that “dirty children get sick more often,” and that malnourished children can be rehabilitated with local foods and expertise. Participants reported that they have made corresponding behavioral changes; they were particularly enthusiastic about EBF’s effects on children’s health. Many of the behavioral changes should be sustainable as they are based on knowledge and local expertise (midwives, support groups for EBF). FSIN also has helped participants and their COSANs get motivated to organize and help finance the local health services’ community visits, which has spurred people’s interest in them for more health care. 1. Quantitative impact evaluation Table V-2 below summarizes the changes in SO3’s impact indicators from the baselines (that were done in different years in different districts) to year four of FSIN. The table also shows the amount of change still required for the indicators to meet their final targets. The figures show that all three indicators are moving in the right direction. 43 Table V-2. Change in SO3 Indicators, Baselines to Year Four (Source: PCU, IPTTFY04, draft version) SO3 Indicators Baselines, 2000 and 2001 Year Four, 2004 Percent change, baselines to 2004 Final program target for 2005 Percent of final target reached in 2004 1. Prevalence of stunting in children 24-59 months. 51.1% 46.6% -8.8% (the right direction) 43.54% 107%: the indicator needs to decrease by 6.6%* to reach its final target. 2. Proportion of newborn infants breast-fed within eight hours of birth 30.4% 72.6% +239% (the right direction) 73.15% 99% 3. Proportion of communities in which vitamin A consumption is above the risk level. 10.2% 28.7% +281% (the right direction) 36.8% 78%: the indicator needs to increase by 28%** to reach its final target. *The difference between the indicator in 2004 (46.6%) and its 2005 target is 3.06 percentage points, which is equal to 6.6% of the 46.6% **The difference between the indicator in 2004 (28.7%) and its 2005 target is 8.1 percentage points, which is equal to 28% of the 28.7%. a. Indicator 1: stunting in children The prevalence of children’s stunting has decreased by 8.8% from the baselines in 2000/2001 to 2004 (Table V-2). However, the stunting rate needs to decrease an additional 6.6% in approximately one year in order to meet its final target in FSIN’s last year, 2005 (Table V-2). Figure 1 below shows the program-level change in the prevalence of stunting over time; figures for the individual districts for the SO3 indicators are in the IPTT in Annex 4. Given the rate of change in stunting since the baselines it is doubtful that this indicator will achieve its final target, although the program has made good progress. Indicator one should be interpreted cautiously due to the difficulty of determining children’s exact ages in Niger. This indicator measures the relationship between children’s height and their age. Height was measured objectively in FSIN’s surveys but children’s ages usually were estimated by their mothers, and generally could not be objectively verified due to the lack of birth certificates and birth records. Women often over- or under-estimate their children’s ages, which potentially biases this indicator. However, although interviewers cannot distinguish between these two errors, it is fair to assume that they are approximately equal in a survey sample. This assumption means that the two potential errors cancel out each other and the indicator’s potential bias. The 44 conclusion thus is that FSIN has contributed to reducing the proportion of stunted children in its intervention zone. Figure 1. Indicator 3.1: Percent of Stunted Children 24-59 Months in FSIN’s Intervention Zone (Source: PCU, 2004: IPTT FY04, draft version) 51.1% 47.8% 46.6% 43.54% 0 10% 20% 30% 40% 50% 60% Baselines 2000 & 2001 Midterm Final survey 2004 Target 2005 45 b. Indicator 2: breast-feeding newborns There has been a significant increase in the proportion of newborn infants breast-fed within eight hours of birth (indicator two). The values for this indicator rose from 30% in the baselines to about 73% in 2004. This means that when FSIN began only three out of 10 newborn infants were breast-fed within eight hours of birth; four years later, the figure was seven out of 10 infants. Indicator two thus reached 99% of its final target in 2004 (Table V￾2). The cautionary note here is that this indicator is based on self-reported information, that is subjective and unverified. Figure 2. Indicator 3.2: Percentage of Newborn Infants Breast-fed within Eight Hours of Birth (Source: PCU, 2004: IPTT FY04, draft version) 30.4% 64.7% 72.6% 73.15% 0 10% 20% 30% 40% 50% 60% 70% 80% Baselines, 2000 & 2001 Midterm Final survey 2004 Target 2005 46 c. Indicator 3: vitamin A consumption Indicator three, the proportion of communities in which vitamin A consumption is above the risk level, has almost tripled (+281%) from the baselines to 2004 (Table V-2). As Figure 3 shows, in 2004 about 29% of FSIN communities were above this risk level. However, this indicator’s year-four level still needs to increase 28% in the program’s one remaining year in order to reach its final target Figure 3. Indicator 3.3: Percent of Communities with Vitamin A Consumption above the Risk Level (Source: PCU, 2004: IPTT FY04, draft version) 10.2% 30.0% 28.7% 36.8% 0% 5% 10% 15% 20% 25% 30% 35% 40% Baselines 2000 & 2001 Midterm Final survey 2004 Target 2005 47 2. Qualitative impact evaluation Discussions with a range of respondents showed that SO3 had numerous impacts on participants’ lives. Overall, the respondents unanimously declared that FSIN’s health/nutrition component has resulted in better health for children and women, good breast￾feeding practices, better feeding practices for children, and the improved use of health services. a. The impact of SO3 according to women The majority of women reported the following impacts of SO3 on their and their children’s lives: • Obvious improvement in their children’s health, who are stronger and healthier. Due to the iron supplements that women receive from FSIN, they declared that their babies are now healthier and appear stronger than those they had before FSIN. • Clear improvements in women’s health, as the result of receiving iron supplements and receiving more regularly the health services they need, including vaccinations. • The reduction of health risks related to anemia for pregnant women: due to the increased coverage of iron supplements, women declared that they no longer suffer from the problems of weakness and chronic fatigue they used to experience after childbirth. • Women now give birth to babies in good health and under the supervision of TBAs: women stated that now they systematically request the TBAs to help deliver their babies. FSIN retrained the TBAs who, according to the women, are doing a good job and are well-equipped to assist women during childbirth. • A reduction in the number of diarrhea episodes for children who are exclusively breast-fed: women stated that they noticed that children who are exclusively breast-fed are healthier and have fewer diarrhea episodes than children who are given water. • Nutritional rehabilitation for malnourished children and diarrhea treatment: women reported that now they know how to treat children with diarrhea at home, and how to rehabilitate a malnourished child using locally available foods. b. The impacts of SO3 according to men The majority of men interviewed reported the following impacts of SO3 on their and their families’ lives: • Men improved their knowledge about health/nutrition, and have learned better feeding practices for children’s health, particularly those related to immediate and EBF, as well as the nutritional recovery and rehabilitation of malnourished children. • Men are more aware of the utility of pre-natal check-ups for pregnant women, vaccinations, iron supplements, and local foods rich in vitamin A. • Improvement of night vision: men said that many of them had suffered from poor night vision before FSIN. They have improved their consumption of foods rich in vitamin A, based on advice from FSIN agents, so that the majority of them do not have this problem any more. 48 • Men as well as women understand the advantages of practicing immediate and EBF: the children who are immediately and exclusively breastfed are in better health than those who were not. c. Impacts of SO3 according to TS and FSIN personnel According to the TS and FSIN personnel the impacts of SO3 included: • The progressive and effective involvement of the communities in addressing their health problems, through the establishment or reinforcement of community health groups (COSAN, COGES, EVS/EVN, TBAs, community support groups for breastfeeding, nutrition, model mothers, local drug experts). • Improved community participation in managing their own health, by assuming responsibility for contributing to certain expenses (e.g. fuel and perdiem for nurses) in order to organize the state’s mobile clinic services for their communities. • FSIN’s partnership with government health services contributes to reinforcing their capacity of to supervise the activities of integrated health centers and rural health centers (“cases de santé”). Involving the government health services staff in training community health workers has reinforced the government staff’s responsibility for monitoring health activities carried out at the community level, including FSIN health and nutrition activities. C. Strengths and weaknesses of SO3 1. Strengths The main strengths of SO3 include: • The emphasis on involving community health groups (COSAN, TBAs, community support groups for breastfeeding, nutrition, local drug experts): from the start of the program, the majority of SO3’s activities were centered on community involvement through community health groups. By reinforcing these community health groups and building their capacity and functionality, FSIN facilitated the communities’ implementation and ownership of SO3 activities. • The participation and involvement of communities in planning, implementing, and monitoring SO3 activities: this participatory approach facilitated communities’ involvement in the activities, such as their financial participation in organizing mobile clinic visits for themselves. For example, in Dogondoutchi and Konni/Illela, the communities are contributing money to finance mobile clinic activities (fuel and perdiem for nurses) and the COSAN members are organizing the mobile clinic’s visits. • The partnership with the TS and the administrative and traditional authorities: government health services personnel in the FSIN intervention zone were extensively involved in planning, implementing, supervising, and monitoring SO3 activities. The administrative and traditional authorities also were regularly involved in implementing these activities. 49 • Food for Work and the complementarity between the agricultural sector and the health/nutrition component was an excellent springboard for the nutritional education and rehabilitation activities. Women reported that the availability of enriched wheat from FFW made it possible to improve the nutritional status of the whole family, particularly children less than five years old. 2. Weaknesses The implementation of SO3 activities revealed a number of weaknesses that need to be addressed in order to improve program results: • The lack of permanent availability of certain food products needed for cooking demonstrations and children’s nutritional rehabilitation, coupled with the lack of vegetable gardens in the great majority of the FSIN communities Despite the appreciable efforts made by FSIN to promote better feeding practices for children, the lack of availability of certain foods poses a problem for nutritional rehabilitation. • The voluntary work of the community health workers: FSIN’s health/nutrition component relies primarily on community groups and community health workers. The community health workers are chosen by the communities for their high moral qualities and their motivation to work. They often work voluntarily, without any compensation. Participants everywhere want FSIN to take charge of compensating the health workers. This shows that the limits of volunteerism have been reached and that alternative means of motivating community health workers should be explored. D. Lessons learned Three important lessons emerge from the evaluation of the SO3 activities: • Addressing the nutritional problems of vulnerable populations very often exceeds the strictly nutritional aspects and includes other, indirect factors that contribute to malnutrition (the elimination of illiteracy, water supplies, controlling and preventing infectious diseases). FSIN should consider taking these issues into account more. • Behavior changes are possible even for the most vulnerable populations as long as the strategies and messages used are appropriate for and adopted by these populations. • Government health services are having increasing difficulty in ensuring the take￾over and sustainability of FSIN activities. E. Sustainability The sustainability of the impact of SO3 activities rests primarily on five main factors: the adoption of sustainable and beneficial health practices and behavior; partnership with government health services; the involvement of community groups; the involvement of communities in planning, implementing and evaluating health activities; and the financial contribution of communities to health activities including mobile clinic services. • The adoption of sustainable and beneficial health practices and behavior: the evaluation team noted with great satisfaction that women have very clearly adopted new practices and sustainable and beneficial health behavior, particularly regarding feeding practices and immediate breast-feeding. Women in all the communities reported that they 50 understand and have adopted immediate and EBF. Their statements were the same about child nutrition practices and using the three basic food categories of food to feed children. These changes in behavior will be the cornerstone of the sustainability of SO3 activities because these behaviors, once understood and adopted, generally persist through life and across generations. • Involving the existing community groups (COSAN, TBAs): the health/nutrition component works toward continuity after FSIN ends by reinforcing and revitalizing community groups instead of creating new ones. • The participation and financial contribution of the participants in building health infrastructure, and their financial contribution to organizing mobile clinic services for themselves, also are sustainability factors for SO3 activities. Communities are more inclined to adopt and continue activities to which they have contributed physically and financially. • Partnership with the government technical health services can only provide a conditional sustainability for SO3 activities, given their lack of material and financial resources to ensure a satisfactory continuation of FSIN activities. F. Recommendations 1. Global recommendations 1. Support the extension of the Centers for Nutritional Learning and Rehabilitation (FARN) where needed, through technical cooperation between communities and the existing local expertise in the FARNs. 2. Support, catalogue, and disseminate women’s new recipes that use local food products to rehabilitate malnourished children. 3. Extend and support the establishment of community support groups for breast-feeding and child-feeding practices (GS2A) in all communities in the intervention zone ; 4. Reinforce and give full responsibility to the COSANs for all health activities to be carried out at the community level, including FARN. 5. Involve the government health services more in planning, monitoring, and evaluating SO3 activities, particularly in Agadez. 6. Consider broadening the community-based distribution of iron supplements to include other products (chloroquine, oral rehydration solution), and the application of cost-recovery mechanisms. 7. Support local initiatives aimed at coating iron supplement tablets to make them easier to take, while intensifying public awareness campaigns about their utility. 8. Continue and reinforce the use of mass media for raising awareness about health and nutrition. 2. Recommendations for DAP 2 1. Teach women in Tanout and Dogondoutchi to rehabilitate malnourished children in order to standardize this activity throughout the intervention zone and contribute to reducing malnutrition rates. 2. Continue constructing rural health centers and integrated community health centers in order to support SO3’s objectives. 3. Improve the availability and accessibility of food by developing vegetable gardens where possible. 51 4. Improve the availability and accessibility of basic health services by constructing and equipping rural health centers and integrated health centers, and continue mobile clinic visits in the districts of Konni/Illela, Tanout, and Dogondoutchi. 5. Apply cost-recovery mechanisms in the intervention zone. 6. Reinforce FARN by prioritizing the use of locally available food products. VI. Cross-Cutting Activities: Management, Food-for-Work, M&E, and the Emergency Response Unit A. FSIN Management 1. Institutional framework The institutional framework for the implementation of FSIN is composed primarily of the Management Committee, the Program Coordination Unit (PCU), and the NGO members’ regional offices. FSIN activities are decentralized and implemented at the regional and departmental levels. The Management Committee, chaired by Africare, gives overall direction to the program and ensures coordination and communication between the CSs. Quarterly meetings are held to discuss progress achieved in program implementation, identify opportunities and potential problem areas, and to make recommendations. Ten quarterly meetings have been held to date. These meetings have kept all the CSs informed about activities undertaken and difficulties encountered. The meetings also allowed the exchange of information, expertise, and experience. The PCU is responsible for monitoring and evaluation (M&E), producing reports, coordinating activities among the CSs, sharing information, and disseminating best practices. It is also the consortium’s liaison. The PCU is staffed by the head of the unit, the M&E coordinator, the coordinator for local monetization, the financial coordinator, and support staff. Each CS is represented by its staff in charge of implementing activities at the regional and departmental levels. This institutional framework is appropriate in theory but cumbersome in practice, leading to delays in decision-making and activity implementation. For example, implementing various decisions made during the coordination meetings was not followed up by the PCU, although one of its most important roles is to ensure that important decisions made by the Management Committee and during quarterly meetings are implemented. This would increase the consortium’s effectiveness and smooth operation. 2. Human and financial resources The CSs were satisfied with the human resources initially allocated for implementing FSIN’s activities. The positions of district M&E coordinators in Tanout, Dogondoutchi and Agadez were not initially planned but were added to the program. No major problem was reported to the evaluation team about financial resources. All the interviewees reported that financial flows were excellent. There were no instances of fund shortages blocking program implementation during the program’s four years. Minor 52 problems related to the risks of handling large amounts of cash were reported in Dogondoutchi, due to the absence of a reliable banking system there. 3. Strengths and weaknesses Strengths of FSIN’s management system: • Standardizing the Css’ different approaches; • Standardizing data gathering, analysis, and reporting; • Ensuring FSIN’s visibility to other organizations; • Providing a coherent institutional framework; • Decentralized management of the CSs’ activities; • Defining the roles and tasks of the CSs so that their work is continuous, precise, complementary, and does not overlap; • Management is based on results with appropriate technical support in the field; • Personnel are recruited and assigned by each CS for optimal effectiveness and results; • Motivated and hard-working teams are in place in each district, which have been effective in implementing the program in the field; • No overspending has been reported in any budget category; Weaknesses of FSIN’s management system: • The PCU initially was mistakenly viewed as a division of Africare rather than as unit for all the CSs. • The PCU initially was not perceived as an “independent entity” with equitable access for all CSs. 4. Evaluation of FSIN’s structure for program management The heads of the CSs’ field offices unanimously reported that the consortium’s structure has had a positive impact on managing the FSIN program. These respondents pointed out the consortium’s innovative structure, as this is the first time that four international NGOs have worked together. The field staff also unanimously noted that the exchange of expertise and experience among the different NGOs’ staff was a major advantage for the consortium 5. Strengths and weaknesses of working in consortium a. Strengths of working in consortium Staff reported several strengths of working in a consortium, including: • Diversity in the programmatic approaches and intervention areas: each CS maintained its own best practice management methods, allowing the others to benefit from their experience. • The four NGOs intervene in different areas, which gives the very favorable impression that the consortium is conducting four different experiments at the same time. • The availability of diversified expertise that each CS can use (health and nutrition, survey and research methodologies). 53 • Sharing experience among the CSs, especially regarding “best practices” and the difficulties encountered implementing program activities. • Having one NGO deal with USAID for all the other NGOs, because it streamlines submitting required documents. • Having one request for funding for the four NGOs, because it decreases the competition among them and reinforces their unity. • Weaknesses of working in consortium Despite the strengths noted above, some weaknesses also were noted: • Confusion about roles among the four NGOs, especially at the beginning of FSIN. The NGOs began program implementation before there was an explicit definition of their respective roles. This gave something of a “who’s doing what?” aspect to the beginning of the program. • The CSs did not always submit the information necessary for reporting to USAID on time. One of the strengths of working in a consortium is that only one NGO is in charge of reporting to USAID, but this requires the timely submission of information from each CS. This is not always the case. 6. Lessons learned There are more problems than advantages when one of a consortium’s members is responsible for program management. 7. Recommendations 1. Reinforce the PCU’s independence and promote equal access to it for all the CSs. 2. Improve the process of sharing information between the PCU and the CSs. 3. Make the PCU responsible for monitoring the implementation of important decisions that the consortium makes at its meetings. 8. Implementation of the midterm recommendations FSIN's midterm evaluation produced a total of 23 cross-cutting recommendations. The consortium rejected one because it risked creating confusion in the program ("Standardize FSIN's terms and concepts to better integrate them with the national policies of rural development and decentralization;" Table VI-1 below). Note that recommendation 16 is divided into two parts (16a. and 16b. in Table VI-1). Over half of the midterm recommendations have been implemented (12 out of twenty-two), and nine others are in the process of being implemented. Only one recommendation, concerning support for partners, is under consideration and neither implemented nor in process: "Analyze the trends to identify the zones that are chronically vulnerable and the types of problems that they face regularly." FSIN thus has made excellent progress in implementing its cross-cutting midterm evaluation recommendations. 54 Table VI-1. Status of FSIN’s Cross-Cutting Midterm Recommendations Midterm recommendation Implemented or not Justifications 1. SO1: Strengthen the support and monitoring of COSAN that is implicit in FSIN's strategy, and add an indicator. Yes 2. SO1: Standardize FSIN's terms and concepts to better integrate them with the national policies of rural development and decentralization. No Recommendation rejected. 3. SO1: Facilitate the exchange of best practices to analyze the participation of women and vulnerable groups in reports, the evaluation of FFW and in coordination meetings. Yes 4. SO2: Design and implement a short-term action plan for vegetable production and marketing. Yes 5. SO2: Develop economic alternatives (IGA) for communities that cannot produce vegetables. In process Several initiatives are in process in the program areas. 6. SO3: Organize exchanges among the experts in health and nutrition about their common experiences and specific experiences with FSIN, and they should be responsible for the recommendations about health/nutrition. Yes 7. Management: Encourage the PCU to prepare and execute a plan of action for training to increase the value-added to training for communities and program experts. In process 8. Management: Promote information-exchange at the program level by: • creating 3 networks; • reinforcing the PCU's backstopping; • presenting results by objective and not by CS; • develop a framework for reporting based on the CSR4 to facilitate comparing reports at the district level and each CS preparing annexes. Yes All four components of this recommendation have been implemented. 55 Table VI-1. Status of FSIN’s Cross-Cutting Midterm Recommendations Midterm recommendation Implemented or not Justifications 9. Management: Reinforce the existing management discussions and the mechanisms for reporting to the ministries, by formalizing annual district-level meetings. Yes 10. Management: Clarify the roles and responsibilities of the PCU in finance, reporting, communication, M&E, and coordination to satisfy the needs identified by the consortium. Yes 11. Management: Reinforce the analysis of monetization and its links with the trends in national food security and the investments of FSIN in different regions, and share the information with customs, clients, elected officials, and the regional and national administrative authorities. In process 12. M&E: Reinforce community self-monitoring and auto￾evaluations, based on the active role played by the Committees of Food Security in data collection and tracking program impact. In process The basic steps are underway, including literacy training. 13. M&E: Revise the IPTT by: • Disaggregating the data for Dogondoutchi and Tanout. • Integrating the data from the baseline and the midterm evaluation, and stratifying the data by vulnerable group in indicators 2.1 and 2.2. • Add indicators for FFW to the IPTT. Yes 14. M&E: Study a sample of at least one household per community and per vulnerability level as a tool for developing impact indicators for SO2 that are more realistic and reliable. In process The process to implement this recommendation is being put in place. 15. FFW: Simplify the annual reporting for FFW, facilitate the comparative analysis of the FFW activities, and add FFW indicators to the IPTT. Yes 56 Table VI-1. Status of FSIN’s Cross-Cutting Midterm Recommendations Midterm recommendation Implemented or not Justifications 16a. Partner support: Analyze the data from the Emergency Unit (information management) to provide regular reports to FSIN partners. Yes 16b. Partner support: Analyze the trends to identify the zones that are chronically vulnerable and the types of problems that they face regularly. No Under consideration. 17. Partner support: Disseminate the Operation System for the use of food commodities. Ensure that each consortium member allocates funds for transportation, distribution, and monitoring of activities, and that all the field agents understand the process well. In process The Operation System has been disseminated but the allocation of transportation funds is not certain. 18. Support to FSIN: Disseminate the Operation System for the use of food commodities. Ensure that each consortium member allocates funds for transportation, distribution, and monitoring of activities, and that all the field agents understand the process well. In process The Operation System has been disseminated but the allocation of transportation funds is not certain. 19: Support to FSIN: Define clearly the types of support that the Emergency Unit can provide to FSIN partners, given its limited staff. Yes 20. Training: Training on "Needs Evaluation," including the analysis of capacity and vulnerability, and the CEWS-UR, to benefit all the FSIN partners. Yes 21. CEWS-UR: The CEWS-UR models that CARE and CRS have tested and implemented have shown their merit, and now discussions and evaluations should be done to improve the system so that eventually it can be replicated. In process The workshop to discuss and evaluate the CEWS-UR model has been held; the model has not yet been diffused. 22. GON Emergency Plan: The Emergency Unit should continue its participation in the process of establishing a GON emergency plan to better understand the process occurring in Tahoua and to support its recognition, but above all to seek synergy. In process The process of establishing a GON emergency response plan is in process and CARE is participating. 57 B. Food-for-Work 1. Summary of activities by district Table VI-2. Summary of Main FFW Activities by District District Main Activities Agadez • Demi-lunes, stone cordons for water control, enlargement of ponds, fences for irrigated land. • Production and planting of seedlings. • Protecting river banks and (banquettes en terres). • Building walls for community buildings (schools, health centers) • Rehabilitation of wells. • Rural roads. Tanout • Demi-lunes. • Enlargement of ponds, stocking ponds with fish. • Production and planting seedlings. • Building walls for community buildings(health centers, schools). • Rural roads. Konni/Illéla • Demi-lunes, zaïs, stone cordons for water control. • Protection of ponds against silting up. • Construction of community buildings (grain warehouse, literacy center, training center). • Building walls for community buildings (health centers, schools). • Construction of banks to contain manure (seuils d'épandage). production and planting of seedlings. • Rural roads. • Delimitation of livestock corridors. Dogondoutchi • Demi-lunes, zaïs, stone cordons for water control. • Enlargement of ponds. • Treatment of river beds (koris). • Production and planting of seedlings. • Building rural roads. • Rehabilitation of community cement wells. • Building walls for community buildings. 58 2. Impact Table VI-3. Impact of FFW District Impact Agadez • Raise communities’ awareness about their capacity to protect and conserve their environment • Mobilize communities for implementing community actions • Improvement in vegetative cover • Dialogue between communities resulting in social cohesion ; • Preserving households’ livestock by eliminating annual sales and the resulting decapitalization • Increase in amount and availability of food stocks Tanout • Adoption of NRM techniques • Change in food consumption patterns • Decrease in cereal prices (millet, sorghum) during FFW activities • Dialogue between communities resulting in reinforced social cohesion • The availability of and the increase in food stocks particularly during the three-month hungry season • Raise communities’ awareness of their capacity to protect and conserve the environment Konni/Illéla • Learning and adoption of NRM techniques • Demonstration of active community solidarity, in particular towards vulnerable households • Change in food consumption patterns • Keeping participants at home (decreasing seasonal migration) which increases labor availability for agricultural production during the rainy season • Dialogue among communities • The availability of food in particular during the hungry season • Building roads to open up certain zones not easily accessible before • Improved coverage of food needs • Contribution to local governance through the installation of the FFW committee Dogondoutchi • Dialogue between communities with reinforced social cohesion • Change in food consumption patterns • Improved coverage of food needs • Development of local expertise particularly regarding NRM • Building roads to open up communities and add value to local economies (Konni and Dogondoutchi) • The reduction of seasonal out-migration • The availability of and access to food for the most vulnerable households during the hungry season • Stabilization of cereal prices when speculation is greatest (May-July) • Enabled the TS to carry out some of their state responsibilities • Support to local governance through the FFW committee 59 3. Strengths and weaknesses a. Strengths • A great mobilization of the participants for FSIN community development actions. • The involvement of all social classes and women. • The involvement of the TS. • The participatory approach combined with taking gender and vulnerability status into account. • Participants’ learning, adopting, and replicating NRM techniques in their individual fields. b. Weaknesses • The population’s priority is to satisfy their food needs; the work is of secondary importance. • The participants’ demands are directed towards FFW activities much more than toward activities that are done without FFW (e.g. composting, improved pruning). • FFW creates an attitude of dependence: communities demand FFW even when they have their own food stocks. • A weak adoption of certain activities (pasture lands, roads): some activities done with FFW are not wholeheartedly owned by the communities. 4. Sustainability The factors below are the basis for sustainability: • Use of the participatory approach. • Communities’ learning, adopting, and replicating NRM techniques. • The adoption of certain improved techniques (improved pruning, composting, tree nurseries). • The creation of local expertise in NRM and in seedling production. • The training provided. • The establishment of community groups (CVD, FFW committees). • The partnership with the TS and some local NGOs. 5. Lessons learned • Participants perceived FFW as an end in itself, not as a means to improve production. • FFW contributed significantly to the achievement of the program’s strategic objectives. • Food consumption patterns changed without major difficulties with FFW. 60 6. Recommendations a) Global recommendations 1. Implement FFW activities before households begin their rainy-season agricultural activities because participants prioritize FFW, to the detriment of household production. 2. Continue and intensify FFW activities because FSIN’s intervention zone is chronically food insecure and production most likely will be poor in 2004. 3. Continue to inform and raise communities’ awareness that FFW is a means and not an end in itself. 4. Provide sufficient tools so that communities can participate extensively in FFW activities. b) Recommendations for the DAP2 1. Negotiate with communities to being FFW activities earlier (April, May) in order to avoid conflict for labor for their household production, and distribute commodities later (June, July). 2. Provide sufficient tools so that communities can participate extensively in FFW activities. 3. Continue and intensify FFW activities because FSIN’s intervention zone is chronically food insecure. C. Monitoring and Evaluation 1. Summary of the M&E system The consortium, coordinated by the PCU, developed FSIN's standardized and integrated M&E system, and supported the CSs' use of common M&E indicators and data-collection tools. The standardized system was based on the M&E plans in each CS's DIP, which were developed in collaboration with USAID and the FANTA project. (Food and Nutrition Technical Assistance project. FANTA provides technical assistance to integrate food security and nutrition into development programs, in order to improve women’s and children’s health). The creation of a standardized M&E system allows the aggregation and disaggregation of data across time and space, which meets the partners' and USAID's need for information at both the consortium and the CS (district) levels. The common set of 16 monitoring and evaluation indicators in the IPTT is used for the annual report that USAID requires; in addition, each CS has its own, specific indicators. The consortium holds quarterly meetings in which each CS reports on its progress and results and M&E issues are addressed. The PCU M&E Manager is responsible for aggregating the CS data to the program level and reporting it to FSIN's partners and the GON both quarterly and annually. FSIN just recently installed a common software system for data input and management. The lack of this system has hampered data management. The current task is to train staff to use the new software. Each CS has an M&E unit with one staff member, who is responsible for data input and management. At the district level the M&E system is based on community participation and periodic community capacity assessments, and monthly reports from the field staff. These staff use FSIN's standardized data-collection forms to collect the M&E information in the communities, under the supervision of the SO heads. District health staffs also submit 61 monthly reports to the M&E unit. The district M&E units are responsible for entering and analyzing their data, which are used in the quarterly reports. The PCU is the central repository for copies of all the CSs' monitoring forms and monthly reports. It is worth noting that, despite the midterm evaluation's recommendation to increase their personnel, the district-level M&E units are still understaffed. FSIN also is setting up community-based M&E systems. One of these is the CEWS-UR, which some communities already are using to monitor potential threats to their livelihood security and to identify appropriate responses. This system is discussed in detail in section D below. Setting up community-run M&E systems that will enable communities to track progress in their Action Plans and plan future activities is under discussion. 2. Advantages and disadvantages of working in a consortium Staffs' views of the advantages and disadvantages of working in a consortium were a key topic in this evaluation. M&E staff reported that working in the consortium was positive because "the exchange of experience and knowledge enriches the M&E work." Sharing information and experience clearly is the primary advantage of working in a consortium as it was consistently reported by these and all other types of staff. Other advantages cited were having a standardized system that all the M&E staff understood and that facilitated sharing information; that collaborating on surveys improved the quality of the work; and that combining the complementary expertise in M&E among the CS produced a good result (Table VI-4 below). Some staff said that when FSIN began it was difficult to work with the different CSs' systems, but by now that has become an advantage, as they have acquired experience with those systems. The advantages of working in the consortium thus considerably outweigh the disadvantages, according to M&E staff. The major disadvantage reported by most staff is the busy M&E work schedule, that makes finding the time to meet colleagues and adhering to that schedule difficult (Table VI-4). These same problems were reported by many SO1 staff as well as the TS, which indicates that they are common problems, and that the DAP2 should have more realistic and coordinated DIPs. 62 Table VI-4. M&E: Advantages and Disadvantages of Working in a Consortium Advantages Disadvantages • We have common tools so all the M&E staff understand the system. • There is a common method of calculating the indicators. • Standardized M&E work is an advantage for sharing among the CS. • The exchange of experience and knowledge enriches the M&E work. • Collaborating on surveys improves the quality of the work, through planning and analyzing together. • There is complementary expertise in M&E among the CS; combine it and there is a good result. • Exposure to and experience with other management systems and other donors' systems. • It is difficult to find time in different people's calendars to meet. • It is difficult to get people together for training because everyone is busy. • People cannot always adhere to the work schedule because there is often someone who cannot come to a meeting or make a decision, which disrupts the rest of the schedule. • People are obliged to obey group decisions, even if they do not agree with them. • Initially it was difficult to collaborate because of different M&E systems and expertise. • The consortium needs a central office to do the accounting and streamline the administration of organizing surveys and hiring interviewers. 3. Strengths and weaknesses of the M&E system Staff cited information-exchange, good quality data, the standardized M&E system, and quarterly meetings to resolve M&E issues as strengths of FSIN's M&E system (Table VI-5 below). Making the TS a partner in data collection and in conducting studies is another strength because it helps build their capacity. The availability of funds to collect qualitative data and to conduct surveys also was reported as a strong point. Overall staff considers the M&E system to be a good one, despite the weaknesses they also identified. Many of the M&E system's reported weaknesses reflect four years of experience on the ground that the second DAP should be able to address. These weaknesses potentially affect data quality and therefore bear consideration in the DAP2. For example, staff reported that some of the IPTT indicators need revision to be valid site-specific measures, that the SO heads should be more involved in data collection because it improves data quality, and that FSIN needs to increase its daily rate for interviewers so that it can hire more competent people and improve the quality of the data collected. (Table VI-5). Inadequate staffing in the district M&E units was reported as a universal weakness; this is not new. Some staff questioned the utility of the plethora of data-collection forms that exist and pointed out the need for streamlining them, as well as the need to train field agents use them correctly. Table VI-5 below shows the M&E staff's assessment of their system's strengths and weaknesses. 63 Table VI-5. Strengths and Weaknesses of the M&E System Strengths Weaknesses • The exchange of information among the consortium members. • There is a standardized system of M&E and indicators for all for CS. • Common tools for data collection and analysis. • Good quality data. • The collection of qualitative data, people's testimonies. • Quarterly meetings with all CS to resolve M&E issues. • All the CSs collaborate to conduct studies; we define the methodology together. • The TS also participates in our studies, which builds their capacity. They participate in data collection. • There are numerous reference documents. • Funds are available to conduct surveys that do not use the IPTT indicators to assess impact, in addition to tracking the IPTT indicators. • The M&E units are under-staffed; one person working alone cannot get all the data input and analyzed on time. The midterm evaluation recommended hiring data￾inputters but this has not been done. • The IPTT indicators need to be streamlined and revised so that they are better measures of reality, which is district-specific. • The heads of the SOs do not take responsibility for the data collection for their SOs; they need to be more involved. • The software for data management was just now installed; it is late, we needed it earlier. • It is difficult to hire good interviewers for data collection due to the salary that FSIN can pay. This affects data quality. • We have learned that we have to define criteria for hiring interviewers in order to get competent ones. • There are too many forms to collect information. There are as many as 30 forms but only about six are used. • Self-M&E in the communities is not really taken into account so it is not done. It should be; it is important and part of capacity-building. • The community TBAs fill out the forms that document women's intake of iron pills, but there are no other community-level forms that participants can do themselves. • Each CS should have its own indicators, perhaps at the district level, in addition to the IPTT. • The consortium does not share the quarterly activity reports with the CS; the reports go to the PCU. Staffs need to see these reports before the quarterly meetings. 4. Lessons learned Several of the "lessons learned" point to the need for training related to M&E (Table VI-6 below). Field agents' need for training to use forms correctly and to collect valid data was a common theme, which indicates that it is an important lesson to address. Other lessons learned were that FSIN needs to limit the size of its "training of trainers" workshops to ensure that the training quality is good, and that participants need capacity-building to do their own 64 M&E. One staff member made the interesting statement that field agents and the TS "should get feedback on the data they collect so that they understand what they and FSIN are doing. Otherwise they are operating in a vacuum, which does not promote good work" (Table VI-6). These lessons learned, like some of the weaknesses noted above, point to the serious need to address the factors that affect data quality, namely the need for training and supervising data￾collection staff. The evaluator's sense, after interviewing a range of FSIN staff including those responsible for M&E, is that the M&E system would benefit from being streamlined and focusing on the essentials—the donor's and CSs' information requirements—in order to improve the quality of its work at all levels. It is clear that there are data- and training-quality issues that need to be addressed in FSIN's fifth year, as well as taken into account for the DAP2. Table VI-6. M&E: Lessons Learned Lessons Learned • FSIN should invest three months in planning in order to have a work plan and an M&E system in place before rushing out to start the activities. • The M&E unit makes field agents work with more rigor: when they know that there is a full-time M&E person checking their data, they do a better job of data collection. • Field agents should be well trained in data-collection methods so that the data quality is good. • Program technicians and the TS should get feedback on the data they collect so that they understand what they and FSIN are doing. Otherwise they are operating in a vacuum, which does not promote good work. They should get feedback because it would help them do a better job. • M&E tools should always be designed in collaboration with program participants. • The TS is not a factor in sustainability due to their lack of means, including lack of salary and perdiem that add up to lack of motivation. They have the competence, which varies by office, but not the means to continue M&E after FSIN ends. • Workshops for "training of trainers" should be limited to small numbers of people so that the quality of the training is good. • Participants need capacity-building so that they can implement their work plans and do their own M&E. • Illiterate TBAs can use the illustrated form to record women's intake of iron pills. • Surveys should not be done when participants are busy. 65 5. Recommendations a. For the current program 1. Increase the staff in the M&E units: each unit should have one or two data inputters and an assistant to the unit head. The M&E units currently have only one person, who cannot get all the work done on time. 2. Provide training for M&E staff on the new software system for data management. This includes training the program heads and the district and program coordinators. 3. Train the SO heads (“responsables”) how to use the new data-management system so that they understand how their data are input and analyzed, can recognize errors, and can use the system. 4. One CS should provide annual, standardized training for all program technicians and TS partners on data-collection and recording for M&E, so that both use their data-collection forms correctly and understand why the information is needed, in order to improve data quality. 5. Increase the salary that FSIN can pay interviewers (“enqueteurs”) so that the program can hire competent ones and improve data quality. 6. Continue to involve the TS in data collection and surveys, in order to build their capacity and maintain the partnership. 7. Limit the number of people who attend the "training of trainers" workshops so that the quality of the training is good; determine the optimum number based on experience. 8. The PCU needs to ensure that all the CSs receive all the quarterly reports before the quarterly review meetings, so that they are prepared for the meetings. 9. Systematically field-test the forms currently used by participants (e.g. the CEWS-RU forms), including those designed for illiterates (e.g. the TBAs’ form for iron pills), to assess their usability and the quality of data produced. 10. Improve the monitoring of the number of participants that are involved in the program's different activities. 11. Set up community-level M&E so that participants can track progress in their work plans, identify problems, and make plans. Design a system for illiterate people, where necessary. b. For the DAP2 1. Invest three months in planning in order to have a sound work plan and M&E system in place before starting the program activities. 2. Streamline the M&E system by focusing on the donor's and CSs' information requirements in order to improve the quality of work at all levels. "Focus on the information requirements" means provide only the information that is required by USAID and the CSs' headquarters, until the M&E system can do that well. 3. To do the above: for the DAP2, design the consortium-level, standardized M&E system and indicators, focusing it on information requirements. That is the core M&E system for the DAP2 and the DIP2. CSs can add their specific indicators/extras to the core system for their individual DIPs. 4. Standardize indicators and data collection and analysis methods in the beginning of the program, before any of them are used, in order to avoid data aggregation and comparison problems later. 5. Ensure that the IPTT indicators are valid measures of site-specific reality in the different districts in the beginning of the program, in order to have useful data. 66 6. Train the SO heads (“responsables”) how to use the new data-management system so that they understand how their data are input and analyzed, can recognize errors, and can use the system. 7. Provide solid training so that staff and the TS partners can use their data-collection forms correctly and understand why the information is needed, in order to improve data quality. 8. The district M&E units should provide regular feedback to program technicians and the TS about the data they collect, so that they understand what they and FSIN are doing. 9. Increase the salary that FSIN can pay interviewers (“enqueteurs”) so that the program can hire competent interviewers. 10. Design the forms for community-level M&E in collaboration with the participants and then systematically field-test them for usability and data quality. 11. Use round percentages (no decimal points) in reporting. D. The Emergency Unit CARE made three observations related to Niger’s adverse environment that triggered the establishment of the Emergency Unit (EU). • Household livelihood systems are precarious; • Communities are little involved in the state’s formal early warning system; • Responses to crises are neither timely nor appropriate. These observations led CARE to consider, in its multi-annual strategic plan, the objective of reinforcing “the capacities of CARE and its partners in the prevention of and response to emergencies.” FSIN provided the means to pursue this objective and the EU was set up with the following objectives5 : • Define and implement a plan for the early warning of and responding to emergencies; • Define a mechanism for monitoring household vulnerability in the FSIN and CARE intervention zones; • Train CARE and FSIN personnel and other partners’ staff in the early warning of and responding to emergencies; • Establish and maintain contact with partners and donors. The EU should coordinate “the design of all essential tools for responding to emergencies, and also ensure the centralization, processing, and the dissemination of information at the level of the intervention areas. A strategy thus was established, based on monitoring vulnerability, developing emergency early-warning and response plans, partnership with other organizations (NGOs and the state’s sub-regional crisis prevention and management committees), and support to communities to help them identify and set up local early warning indicators.” 5 See pre-evaluation final report of CARE (August 2004) 67 1. Summary of activities The principal activities carried out in the EU’s framework are: • The establishment of a mechanism for the early warning of and response to emergencies, composed of a National Task Force, Regional Task Forces, and CEWS-UR committees in FSIN’s intervention zone. Roles and responsibilities were defined for each level. • The development of emergency plans: these include CARE-Niger’s national plan for early warning and response to emergencies, regional plans for all the regions of Niger, and community emergency plans. Training workshops were organized for the regional task force teams. Some of these teams have drafted outlines for their regional emergency plans. It should be noted that the CARE-Niger plan is not yet finalized and that drafting the community-level plans has not begun. • The establishment of a mechanism to monitor vulnerability and early warning: the EU has done this through an iterative and participatory process of discussion with all the CARE projects and FSIN partners. This process led to the design of several forms (e.g. for early warning and for evaluating victims); an operational mode of emergency food relief; and a guide to collect information about food security and to evaluate agropastoral production. The committee also requested the establishment of CEWS-UR committees in FSIN communities. These committees will be responsible for collecting community-level information and reporting it to the sub-regional or the FSIN level. Above all, the CEWS-UR committees will be responsible for responding to local-level emergencies. They will seek external help when the responses required are beyond their capacity. • Training in early warning and emergency response for the ISAN and CARE staff, and for their partners, the TS. • Participation in the network, notably with the Inter-State Committee to Fight Drought in the Sahel (CILSS), the Famine Early Warning System Network (FEWSNET), and the World Food Program (WFP). • Conducting emergency operations: support to flood victims in Konni department in August 2002, operations to support communities at risk of food insecurity in the north and northwest of Tahoua, support to the evacuees from Ivory Coast, support to flood victims in Boughoul (Agadez), and support to the TS of Konni/Illela to mitigate animal disease. 2. Findings The analysis of the EU’s activities shows that: • The process of establishing the EU was laborious and initially not well understood by FSIN’s or other partners, namely the TS and FSIN’s participants. The process is still underway. The system is not standardized in Agadez or Tanout; it is being piloted in five sites in those districts. In Konni/Illela the system is being piloted in ten sites. Dogondoutchi district is more advanced and has forty community CEWS-UR. The TS find the system sound enough but note that it uses neither the same approaches nor the same indicators as the state’s formal system. FSIN participants do not understand the “early warning” aspect. 68 • Additional discussions have led to the establishment of a mechanism that is more or less operational, depending on the case. The mechanism includes the various “Task Forces” and the community-level CEWS-UR committees. The state’s emergency response system quickly learned about this mechanism through the EU’s actions. • The CEWS-UR committees function well enough in the communities where they were set up, but their sustainability is not guaranteed because of the following problems: (i) the laborious reporting system6 ; (ii) the inherent costs of emergency response that only a project could support, even if community emergency funds are programmed; and (iii) their overlap with (rather than their inclusion in, at this point) the state’s formal system. • The CEWS-UR mechanism is useful for effective response to crises at the community level. • The CEWS-UR mechanism is well accepted by communities, contrary to the TS statements. • The impact of the CEWS-UR has begun to be felt at the community level, namely by their rapid reaction to human and animal health problems, as well as in conflict prevention (herder/farmer problems). For example, the CEWS-UR committees played an essential role in disseminating information about locust attacks in northern Dogondoutchi. The same was occurred in Awanchala (Konni/Illela), where a measles epidemic was avoided because the CEWS-UR quickly reported information to the health services. • There is no mechanism for the consortium’s CSs to take direct charge of emergencies. The EU based in CARE-Konni manages the support mechanism for emergency victims. This limits the other CSs’ intervention capacities, particularly the common problem of transporting food. 3. Recommendations The CEWS-UR, as they operate in FSIN’s intervention zone, is a good tool for early warning about crises. It complements the state’s early-warning and disaster-management system. Therefore it should be maintained and extended to all the FSIN communities, taking into account the following recommendations: 1. Simplify the community-level reporting system by having only two alert levels: normal and abnormal. 2. Reinforce community capacity to collect and analyze data, and promote information￾sharing among the CEWS-UR committees in the same area 3. Intensify discussions with the TS at all levels to connect the CEWS-UR system with the state’s system. 4. Continue the consortium’s discussion to identify the types of support that the EU can provide each CS, and to standardize their methodology (data collection and analysis, reporting systems, crisis-response practices). 6 The reporting system consists of a set of indicators for food security, health, social relations, and the environment, that communities monitor and report on each month to the program and the sub-regional Early Warning System offices. The committees fill out a form with four warning levels (normal, alert, alarm, urgent). 69 5. Promote the establishment of community emergency funds in all the FSIN communities. 6. Each CS should have an emergency fund for dealing with disaster victims. 70 References Cited Africare, "Pre-Evaluation Report on the Food Security Initiative Project in Agadez," September 2004. Bakari, S., "Final Quantitative Study," final report, Food Security Initiative in Niger (FSIN), July 2004. CARE-Niger, "Pre-Evaluation Report on the Food Security Initiative Project in Niger," September 2004. McMillan, D.E., D. Jaharou, A. Mahamadou, D. Mariko, and H. Chano, "Mid-Term Evaluation, Food Security Initiative in Niger (FSIN)," July 7, 2003. Project Coordination Unit, FSIN, "Indicator Performance Tracking Table FY04, draft version” October 2004. ANNEX 1 Fieldwork and Activities Schedule and Topical Guide FSIN Final Qualitative Evaluation: Fieldwork and Activities Schedule Date Activity 11 September Briefing - Meeting with M&E point people from each organisation and team members, as well as PCU and DAP coordinators. 12-13 Septmber Elizabeth and team plan, discuss, review and prepare. M&E people available for discussions. 14 September Evaluation team presents a draft Table of Contents to M&E people, DAP Coordinators and PCU. 15 September Team work continues; meeting with Jennifer Peterson and Madame Chano Hadiza – M&E counterpart from the Ministry of Community Developpement. 16 September Depart for the field, 10 hour drive to Agadez; details and accomodation to be organized by CARE (lodging) and Africare. 17 September Visit Agadez – details to be provided 18 September Visit Agadez – details to be provided 19 September Visit Agadez – details to be provided 20 September Visit Agadez - details to be provided 21 September Depart for Tanout, 6 hour drive; accomodations and details to be organized by CARE (lodging) and CRS/HKI. 22 September Visit Tanout – details to be provided 23 September Visit Tanout/Zinder – details to be provided 24 September Visit Zinder– details to be provided 25 September Depart for Konni, 6 hour drive. Accommodation and details to be organized by CRS. 26 September Visit Konni – details to be provided 27 September Visit Illela – details to be provided 28 September Visit Illela/Tanout – details to be provided 29 September Depart for Dogondoutchi 2 hour drive- Accommodation and details to be organized by CRS. 30 September Visit Dogondoutchi – details to be provided 1 October Visit Dogondoutchi– details to be provided 2 October Visit Dogondoutchi – details to be provided 3 October Depart for Niamey – five hours drive 4 October Day off. 5-10 October Evaluation team writes up the first draft report and prepares for two debriefings. 11 October First debriefing with the consortium; second debriefing with USAID and the Ministry of Community Development. 12 October Brief presentation to the Ministry of Community Development; continue writing. 13-14 October Evaluation team continues writing. 18 October First draft, in French and English, sent to those who requested to have it before the French version is complete. 1 November Dr. Moha begins translating the English sections of the report into French. 1 TOPICAL GUIDE FSIN: Final Qualitative Evaluation Septembre, 2004 I. Project personnel 1. Program impact : global, by SO. 2. Strengths and weaknesses of the program? 3. Implementation of the midterm evaluation recommendations ? If not : why not? 4. Sustainability? 5. Lessons learned? 6. Recommendations? 7. Evaluation of FSIN’s overall structure in terms of program management? 8. Appropriateness of FFW? - Contribution to achieving results? - Impact on sustainability of program’s activities ? 9. Changes in vulnerability of the target groups? II. Local NGOs 1. Project impact : global, by SO ? 2. Strengths and weaknesses of the program ? 3. Sustainability ? 4. Lessons learned ? 5. Recommendations? 6. Appropriateness of FFW? - Contribution to achieving results? - Impact on sustainability of program’s activities ? 7. Changes in vulnerability of the target groups ? III. Administration (civil servants) 1. Project impact ?. 2. Program strengths and weaknesses? 3. Sustainability ? 4. Recommendations ? IV. State Technical Services 1. Program impact : global, by SO. 2. Strengths and weaknesses of the program? 3. Implementation of the midterm evaluation recommendations ? If not : why not? 4. Sustainability? 5. Lessons learned? 6. Recommendations? 7. Evaluation of FSIN’s overall structure in terms of program management? 8. Appropriateness of FFW? - Contribution to achieving results? - Impact on sustainability of program’s activities ? 9. Changes in vulnerability of the target groups? 2 TOPICAL GUIDE, p. 2 V. Beneficiares A. Impact 1. Have there been any changes in your lives in the past few years ? - What changes ? - Why? 2. Changes in your vulnerability to food insecurity ? - Why, or why not? - Changes in the different social groups here ? 3. Changes in nutritional status at the household level, among women and children, the most vulnerable people ? 4. What parts of this program function well, in your opinion ? Why ? 5. What parts of this program do not function well ? Why not? 6. Can you recommend changes in the program to improve it in the future ? B. KAP 7. Have you learned anything with the program ? What? 8. In your opinion, what are the most useful things that you have learned with the program? 10. Do you use any techniques or practices that you have learned with the program ? Which ones ? 11. What are the most useful techniques/practices that you use that you learned with the program ? C. Community capacity related to food security 13. What is the capacity of your community to manage your own food security? 14. Has this capacity changed during the past few years? 15. What are the roles of men and women in managing community food security ? 16. Do you have committees ? how many men/women on your committees? how are they chosen ? 18. What is your opinion of thes committees and your food security management plan (Plan d'Action)? 19. Recommendations to improve your capacity to manage your food security? 20. Have you received support from FSIN for a community initiative? - What initiative? - When? - Outcome? 21. FFW: have you received it ? - Your opinion of FFW ? - Advantages and disadvantages? - Impact on your life ? ANNEX 2 TERMS of REFERENCE 1 CARE INTERNATIONAL AU NIGER INITIATIVES DE SECURITE ALIMENTAIRE AU NIGER (ISAN) EVALUATION FINALE TERMES DE REFERENCE CARE International au Niger en collaboration avec Africare ; CRS/HKI Key Contacts: Dr. Amadou Sayo ACD CARE Niger Ali Bety ANR Sector Coordinator CARE Niger Véronique Praz Country Representative Africare Niger Kwanza Price PCU Manager Africare Niger Pascal Payet PCU ME coordinator Africare Niger Abdou Garba CARE Konni, DAP coordinator Idrissa Amadou CARE Konni, ME coordinator 2 I. INFORMATIONS GENERALES Programme : Initiatives de Sécurité Alimentaire au Niger (ISAN) Durée du Programme : 2000 – 2005 Financement : USAID (Titre II) Exécution : Africare ; CARE ; CRS/HKI Zones d’intervention : Agadez (Arlit, Tchirozérine) ; Tahoua (Konni, Illéla) ; Dosso (Doutchi), Zinder (Tanout, Matameye) Durée de l’évaluation : 34 jours Démarrage de l’évaluation : 1er Septembre 2004 Fin de l'évaluation : 4 Octobre 2004 Responsable direct : Coordonnateur Programme ISA CARE II. CONTEXTE Le Niger est un vaste pays sahélien enclavé de 1.267.000 km2 . Onze pour cent (11%) des terres sont cultivables, mais seulement le quart est exploité, bien que 90% de la force du travail soit engagée dans l’agriculture. Les conditions climatiques dures et les sols marginaux résultent en une productivité agricole faible et placent des limites sérieuses sur les pratiques de culture alternative. La condition de pauvreté de la plupart des ménages rend plusieurs solutions inabordables. La pluviométrie demeure le principal déterminant de la production agricole au Niger, mais la quantité de pluie et sa distribution ne sont pas prévisibles. Même au cours des années où les pluies sont jugées abondantes, une très grande proportion de ménages ne produit pas assez pour couvrir leurs besoins alimentaires. La dégradation des sols force plusieurs agriculteurs à se déplacer vers les terres traditionnellement réservées aux pasteurs. Ces terres marginales sont très peu productives et, leur utilisation par les agriculteurs peut provoquer des violentes confrontations avec les éleveurs dans les zones agricoles et agropastorales. En zone pastorale, la gestion du pâturage et des points demeure la principale source de conflits entre éleveurs. La pression démographique (3,3% par an), l’utilisation abusive du bois comme source d’énergie domestique, le surpâturage et la rareté des terres agricoles ont forcé les agriculteurs à adopter des stratégies de survie qui ne sont pas toujours compatibles avec la gestion durable des ressources naturelles. Aussi l’agriculture nigérienne souffre-t-elle beaucoup d’une absence de rotation de cultures, d’une pratique inappropriée de défrichage (coupe abusive, brûlis), d’une réduction voir la disparition de la pratique de la jachère, d’une faible utilisation de la fumure (organique et minérale), des techniques d’irrigation, etc. Cette situation qui perdure d’année en année place le Niger en situation d’insécurité alimentaire chronique laquelle a un impact mortel sur les plus vulnérables et particulièrement les enfants et les femmes. Selon l'indice de développement humain calculé par le PNUD, le Niger est parmi les pays les plus pauvres du monde; il se classe au 174ème rang sur 175 pays. La population en deçà du seuil de pauvreté représente 63% de l'ensemble. La majorité de ces pauvres se trouve en milieu rural (Document de Stratégie de Réduction de la Pauvreté DSRP, 2002). Dans ce milieu rural les ménages les plus touchés sont ceux 3 dont le chef n'est pas salarié et parmi ces derniers ceux qui soufrent le plus sont ceux dont le chef de ménage est une femme au foyer ou une personne inactive. La pauvreté limite l'accès aux services sociaux et aux autres satisfactions matérielles et morales. La couverture éducative des enfants et des jeunes est faible et croît moins que la population, dans ce domaine le Niger est le dernier pays de la sous région. Le taux brut global de scolarisation est estimé à 34%, pour les filles il n'est que de 27%. En matière de santé la situation appelle des efforts et des améliorations importantes, malgré les investissements faits jusqu'ici. Les indicateurs suivants sont à cet égard illustrants: - taux national de couverture sanitaire : 42% - taux de mortalité infantile : 12,6% - taux de mortalité infanto juvénile : 28% - Taux de couverture vaccinale : 15% - pourcentage des enfants non vaccinés en milieu rural : 47% - pourcentage des enfants non vaccinés en milieu urbain : 7% - pourcentage des enfants non vaccinés à Niamey : 2% - espérance de vie à la naissance : 45,6 ans - insuffisance pondérale ou malnutrition (poids/âge) des enfants de 0 à 5ans : 40%; ce taux est plus élevé dans les régions de Diffa, Zinder et Maradi. - retard de croissance des enfants de 0 à 5 ans: 40%; ce taux est plus élevé dans les régions de Diffa, Zinder et Maradi. - Pourcentage des femmes ayant reçu des soins prénatals : 36% en milieu rural (85% en milieu urbain) - Pourcentage des femmes rurales ayant été assistées par un personnel qualifié lors de leur accouchement : 9% (65% en milieu urbain). - Mortalité maternelle : 0,7% (le plus élevé au monde). - Le nombre des malades du SIDA estimé à 65.000 personnes par l'OMS en 1997, les cas signalés aux services de santé se chiffrent à 5624 en 2000. - Seulement 14% des hommes et 3% des femmes reconnaissent avoir utilisé le condom pour se protéger contre le SIDA. La situation de santé de la population est aggravé par les insuffisances des conditions du cadre de vie, la sous alimentation et la malnutrition. Le système de santé souffre des charges récurrentes élevées, des faiblesses de recouvrement des coûts, de la mauvaise allocation des ressources humaines, des insuffisances de gestion des ressources matérielles et de la centralisation excessive des pouvoirs de décision et d'initiative. La malnutrition est l'une des principales causes de mortalité des enfants nigériens. 41% des enfants sont dans un état de malnutrition chronique et 20% sont dans un état de malnutrition chronique sévère. A peine 50% des nigériens ont accès à l'eau potable et seulement 4,9% des ménages ont un robinet privé. Plus de 80% des nigériens n'ont pas de lieu d'aisance, ils font leurs besoins dans la nature ou dans des endroits non indiqués; en milieu rural plus de 94,1% des ménages n'ont pas de lieu d'aisance; seulement 2,8% des ménages urbains ont des toilettes personnelles avec chasse d'eau. 4 La sécurité alimentaire des populations reste précaire et préoccupante, et cela d'autant plus que la croissance démographique se maintient à un taux élevé. Presque une année sur trois le Niger accuse un déficit céréalier de 200.000 à 300.000 tonnes sur un besoin d'environ 2,5 million tonnes de céréales. Le système de sécurité alimentaire mis en place par le gouvernement et les donateurs atténue les effets des crises certes, mais des poches sociales touchées par le dénuement alimentaire total existent dans certaines zones du pays. Depuis les indépendances, le Niger tente de définir et de mettre en œuvre une politique de sécurité alimentaire susceptible d’inverser cette tendance. Les partenaires extérieurs, gouvernementaux et non gouvernementaux, sont de plus en plus impliqués dans la recherche d’une réponse adéquate à ce défit. C’est dans le cadre de cette mobilisation générale que en mai 1999, quatre (4) Organisations internationales Non Gouvernementales (ONG) - Africare, CARE International, Catholic Relief Services (CRS) et Helen Keller International (HKI) - ont soumis à l’Agence de Développement International des Etats Unis (USAID) une proposition d’activités (Development Activity Proposal, DAP) pour la mise en œuvre d’un vaste programme de sécurité alimentaire dit «INITIATIVES DE SÉCURITÉ ALIMENTAIRE AU NIGER (ISAN)». L’USAID a approuvé le programme en août 2000. Ce programme couvre une période de cinq ans (11 août 2000 à 31 septembre 2005). Il a touché trois cent (300) villages au niveau de sept (7) arrondissements du Niger à savoir Arlit, Tchirozerine, Dogon Doutchi, Konni, Illéla, Tanout et Matamèye. La population cible est estimée à environ trois cent milles (300 000) personnes. Le but final du programme est l’amélioration de la sécurité alimentaire et nutritionnelle dans les zones cibles. Trois (3) Objectifs Stratégiques (OS) sont définis autour de ce but : 1. OS1 : Renforcer les capacités des communautés à prendre en charge leur sécurité alimentaire et nutritionnelle ; 2. OS2 : Augmenter la production agricole durable des ménages par la promotion des techniques culturales respectueuses de l'environnement ; 3. OS3 : Améliorer l’état nutritionnel des membres du ménage, notamment les plus vulnérables (les femmes et les jeunes enfants). Le cadre logique présenté à l’annexe1 indique les indicateurs contractuels (dits également indicateurs communs) d’impact et de suivi retenus au niveau de chacun des trois objectifs stratégiques. Ces indicateurs constituent la base du plan de suivi de la performance du programme (annexe 2). Il a également été inscrit pour les cinq années d’exécution du programme des opérations de Food For Work (FFW) à travers la distribution du blé bulgur. Les produits destinés à cette distribution ont permis de soutenir les activités menées dans les zones d'intervention du programme, en apportant une aide significative aux participants au cours de la période de soudure pour les aider à faire face au déficit alimentaire grave. Les effets attendus d’une telle stratégie est d'améliorer à long terme la production agricole et sa commercialisation par l’accès aux services et aux infrastructures socio-économiques de base (marchés, centre de santé, services agricoles, vétérinaires, etc.). 5 Enfin, il a été confié à l’Unité d’Urgence de CARE Niger basée à Konni, la définition et a mise en œuvre d’un mécanisme de prévoyance et de réponses aux urgences à travers notamment : • la définition des plans d’urgence régionaux ; • la formation du personnel en matière d’urgence ; • la mise en place d’un dispositif de suivi de vulnérabilité et d’alerte précoce au niveau communautaire ; • l’établissement et le maintien de contact avec les partenaires oeuvrant dans le domaine. Les activités relatives aux trois objectifs stratégiques, celles conduites au cours des opérations FFW, et les activités de l’Unité d’Urgence constituent les cinq (5) axes d’intervention (secteurs d’activités) du programme. La mise en œuvre du programme est assurée par les quatre ONGs participantes. Chacune d’elle est responsable de la mise en œuvre des activités au niveau des sous zones dont elle a la charge. Ainsi : - Africare exécute son programme au niveau des arrondissements d’Arlit et de Tchirozérine (Agadez) ; - CARE International Niger met en œuvre son programme au niveau des arrondissements de Konni, Illéla (Tahoua) et de celui de Matamèye (Zinder) - CRS/HKI mettent en œuvre conjointement les activités de leur programme au niveau des arrondissements de Dogon Doutchi (Dosso) et de Tanout (Zinder). L’intervention au niveau de chaque district est spécifique d’une part aux réalités de chaque zone d’intervention et, de l’autre à la stratégie d’intervention de chaque ONG. Ainsi, chaque ONG a complété les trois objectifs stratégiques pour prendre en compte ces spécificités (voir cadre logique spécifique adopté au niveau de chaque district). Cependant, l’ensemble des composantes du programme est mis en œuvre selon une approche participative. Pour faciliter le processus de coordination du programme, une Unité de Coordination du Programme (PCU) dirigée par Africare est mise en place. Le PCU est responsable de la centralisation des rapports pour en faire une synthèse globale et coordonne le système commun de suivi et évaluation du programme. Il assure également la coordination et le partage des leçons apprises ainsi que la liaison externe assignée. Cette unité assure aussi la supervision générale des ressources fournies par l'USAID et veille, entre autres, à ce que les organisations adoptent les mêmes approches en matière de mesures des performances du programme et l'établissement des rapports y afférents. Cette unité a aussi pour tâche de veiller au respect des procédures de l'USAID en matière d'établissement de rapports et d'information. L'Unité rend compte à Africare qui a été désignée par les autres membres du consortium comme principal intermédiaire dans toutes les relations officielles avec l'USAID. Un Comité de direction du programme dit Comité de Pilotage, composé des Représentants résidents des ONGs participantes, est chargé de l’orientation d’ensemble et de la supervision du programme. L’annexe 3 présente l’organigramme d’ensemble du programme ISAN. La valeur totale des ressources externes pour la mise en œuvre du programme est environ 20 millions de dollars américains. Ce chiffre inclut : 6 • 20 650 tonnes de blé fortifié au soja destinées à soutenir des activités à caractère communautaire par le food for work. ; • le montant de la vente de 30 000 tonnes de riz et de 7 500 tonnes d’huile végétale qui rapportera 17.800.439 dollars en monnaie locale. Ces transactions de monétisation sont assurées par CRS au Burkina Faso, et par Africare au Niger. • 2.199.562 dollars en devise américaine sur le compte 202 (e) de l'USAID. En plus de ces ressources externes, la participation physique, matérielle et parfois monétaire des communautés (cas de la construction de certaines infrastructures socio-économiques tels les pistes et les centres de santé ) et de l’Etat( en consentant des taxes sur les produits de la monétisation) représente des ressources non négligeables pour la mise en œuvre du programme. En plus, des cinq secteurs d’activités relatifs aux trois objectifs stratégiques, le Food for work, l’Unité d’urgence, une attention particulière a été accordée aux systèmes de suivi et évaluation (commun et spécifiques) qui constituent le cadre de mesure de la performance du programme et des réajustements de l’intervention. Le progrès du programme vers la réalisation des trois objectifs stratégiques est mesuré à travers la Table de Suivi des Indicateurs de Performance (TSIP). Cette table de suivi est composée de l’ensemble des indicateurs communs présenté dans le cadre logique en annexe 1. En plus de ces indicateurs, chaque district a des indicateurs spécifiques qu’il suit pour prendre en compte les spécificités des différentes zones. Un plan commun de suivi et évaluation permettant la collecte des données pour le calcul des indicateurs retenus, l’analyse du niveau des indicateurs fut conçu et mis en œuvre au niveau du consortium. Ce plan s’est inspiré d’une part des résultats de l’atelier de suivi et évaluation pour le Titre II tenu à Bamako, au Mali du 29 mai au 02 juin 2000 ; d’autre part, des plans de suivi et évaluation mis en œuvre dans des programmes similaires en Afrique de l’Ouest ; et enfin de l’expérience des ONGs participantes en matière de suivi et évaluation des programmes de sécurisation des conditions de vie des ménages. Le système de suivi et évaluation a prévu dans sa mise en œuvre trois grandes évaluations notamment l’évaluation ou étude de base, l’évaluation à mi-parcours et l’évaluation finale. Les études de base ont été organisées entre novembre 2000 et mars 2001 dans les cinq districts du programme. Ces études ont permis de déterminer le niveau de départ des indicateurs communs et spécifiques de performance du programme et de faire des réajustements par rapport à certaines cibles qui sont soit surestimées ou sous-estimées. Elles ont donc permis de détailler la planification avec des situations effectivement appréciées avec les bénéficiaires. L’évaluation à mi-parcours a débuté en septembre 2002 avec la pré évaluation (documentation, étude quantitative, préparation phase qualitative). La phase qualitative a été conduite en mars 2003 avec une équipe de consultants nationaux et internationaux, et les représentants du gouvernement et de l’USAID. Cette phase de mars n’avait pas pris en compte les activités de l’Unité d’Urgence pour laquelle les conditions d’une évaluation n’étaient pas réunies en ce moment. Son évaluation a eu lieu en mai 2003 avec une consultante internationale. 7 Cette évaluation à mi-parcours a permis non seulement de dégager les forces et les faiblesses du programme mais aussi de formuler des recommandations pour une amélioration de l’intervention. Quelques forces du programme • Parfaite cohérence avec les politiques nationales • Grande complémentarité entre ONGs participantes • Grande diversité des stratégies d’intervention, mais forte cohérence entre les programmes sur le terrain • Forte implication des groupes vulnérables • Stratégie d’intervention basée sur les approches participatives • Food for Work a partout contribué à crédibiliser le programme • Implication des autorités coutumières et administratives, des services techniques de l’Etat et de certaines ONG locales dans l’identification des villages bénéficiaires • Chaque CS a mis en place un plan de durabilité • Forte harmonisation dans le système de suivi 2. Quelques faiblesses du programme • Réalisations et impacts du programme peu connus au niveau national, même dans certains cas au niveau régional. • Retard dans la mise en oeuvre des cultures maraîchères • Les AGR ont été peu prises en compte dans la conception du DAP • Faible implication des Agents de suivi et évaluation dans les activités de terrain et vice versa • Faible information sur la monétisation au niveau national 3. Les recommandations de l’évaluation à mi-parcours a. Relatives à l’Objectif Stratégique 1 (OS1) : Renforcer les capacités des communautés à prendre en charge leur sécurité alimentaire et nutritionnelle • Renforcer l’appui et le suivi des COSAN déjà implicite dans la stratégie de ISAN; ajouter un indicateur 3.1 relatif à la performance de ces COSAN. • Harmoniser les termes et concepts clés pour mieux s’intégrer avec les politiques de développement rural et de décentralisation. • Faciliter l’échange des meilleures pratiques pour l’analyse de la participation des femmes et des groupes vulnérables dans les rapports, l’évaluation de FFW ou lors les réunions de coordination. b. Relatives à l’Objectif Stratégique 2 (OS2) : Augmenter la production agricole durable des ménages par la promotion des techniques culturales respectueuses de l'environnement • Elaborer et exécuter un plan d’actions à court terme pour le maraîchage et le marketing de ses produits • Développer des alternatives économiques (AGR) pour les villages n’ayant pas des opportunités de pratiquer le maraîchage 8 c. Relatives à l’Objectif Stratégique 3 (OS3) : Améliorer l’état nutritionnel des membres du ménage, notamment les plus vulnérables (les femmes et les jeunes enfants) • Organiser des échanges entre les experts de santé et nutrition sur les expériences communes et spécifiques à ISAN et qu’ils prennent en charge les recommandations en Santé Nutrition d. Relatives à la gestion du programme • Encourager l’Unité de Coordination du Programme (PCU) à préparer et suivre un plan d’actions des formations pour augmenter la valeur ajoutée des formations au niveau village et des experts du programme • Promouvoir les échanges d’informations au niveau du programme par : Š la création de 3 réseaux ; Š le renforcement du backstopping du PCU (fonds d’assistance technique et de soutien aux activités des réseaux) ; Š la présentation des résultats par objectif et non par ONG participante ; Š le développement d’un canevas de rapport sur le modèle CSR4 pour faciliter la comparaison des rapports au niveau des districts et préparer des annexes par ONG participante • Renforcer les cadres de concertation existants et les mécanismes de rapportages aux ministères, par la formalisation des rencontres annuelles au niveau district • Clarifier les rôles et responsabilités du PCU dans les fonctions de finance, rapportage, communication, suivi et évaluation et coordination pour satisfaire les besoins identifiés du consortium • Renforcer les analyses de monétisation et ses liens avec les tendances de la sécurité alimentaire nationale et les investissements de ISAN dans les régions et partager l’information avec la douane, les clients, les élus et les autorités administratives nationales et régionales e. Relatives au Suivi et Evaluation • Renforcer le suivi communautaire et les auto évaluations en se basant sur le rôle actif joué par les Comité de Sécurité Alimentaire dans la collecte des données et le suivi d’impact du programme • Réviser la table de suivi des indicateurs de performance (IPTT) en : Š désagrégeant les données de Doutchi et Tanout Š intégrant les données de l’étude de base et de l’évaluation à mi-parcours pour les groupes vulnérables dans les indicateurs d’impact 2.1 et 2.2 Š ajoutant des indicateurs du food for work dans la table IPTT • étudiant un échantillon d’au moins un ménage par village et par niveau de vulnérabilité comme outil pour développer des indicateurs d’impact de la production agricole (OS2) plus réalistes et plus fiables f. Relatives au food for work • Simplifier le rapportage annuel du food for work • Faciliter l’analyse comparative des activités de food for work et ajouter des indicateurs y relatifs dans la table IPTT g. Relatives à l’Unité d’Urgence 9 • Intégrer la fiche d’alerte précoce au guide à l’usage des agents de terrain dans un système simple de suivi de vulnérabilité • Analyser les données du système par l’Unité d’Urgence (Gestion de l’Information) pour fournir des rapports réguliers aux partenaires de ISAN. • Analyser les tendances pour identifier les zones à vulnérabilité chronique et les types de problèmes auxquels elles font face régulièrement. • Diffuser le mode opératoire pour l’utilisation des vivres. S’assurer que chaque membre du Consortium prévoit des fonds pour le transport, la distribution et le suivi des activités et que l’ensemble des agents de terrain comprennent parfaitement le processus • Définir clairement les types d’appui que l’Unité d’Urgence peut apporter aux partenaires de ISAN, compte tenu de son staff limité. • Former tous les partenaires de ISAN sur «l’Evaluation des Besoins», incluant l’analyse des capacités et de la vulnérabilité, mais aussi le SCAP-RU. • Les modèles de SCAP RU mis en œuvre et expérimentés par CARE et CRS ont tous montré leur mérite, par conséquent des discussions/évaluations doivent être engagées en vue de l’amélioration du système pour une éventuelle réplication • L’Unité d’Urgence doit continuer sa participation au processus de mise en place du plan d’urgence du gouvernement pour mieux faire comprendre le processus en cours dans la région de Tahoua et faciliter sa prise en compte, mais surtout rechercher la synergie Un plan de mise en œuvre de ces recommandations a été élaboré aussitôt après l’évaluation. Ce plan est également en cours d’exécution et il semble pertinent que l’évaluation finale apprécie la prise en compte de certaines recommandations principales dans la mise en œuvre du programme. III. JUSTIFICATION DE L’EVALUATION FINALE Les activités du programme ISAN ont officiellement démarré à partir du mois d’août 2000. Le programme est donc dans sa quatrième année d’activités. Le document de proposition des activités de développement ou DAP, avait prévu, dans le système de suivi et évaluation du programme, une évaluation de base, une évaluation à mi-parcours au cours du troisième exercice budgétaire (FY03) et une évaluation finale au cours de la cinquième année budgétaire (FY05). Le programme étant actuellement dans la perspective de proposition d’une nouvelle phase du DAP (conformément au calendrier de soumission des propositions à l’USAID) a jugé utile de conduire l’évaluation finale en fin de quatrième année et début cinquième année budgétaire pour tenir compte des leçons tirées de cette évaluation dans la conception de la prochaine phase du DAP. Dans ce cadre, une étude quantitative a été conduite entre février et avril 2004 (période correspondant à celle des études de base) pour évaluer le niveau des indicateurs de performance qui ne sont pas déterminés par le suivi quotidien. Les résultats de cette étude et les rapports d’activités permettront aux équipes d’élaborer un document de pré évaluation qui sera une des sources d’information pour la phase qualitative, objet des présents TDRs. 10 IV. OBJECTIFS DE L’EVALUATION FINALE 4.1 Objectif global • L’évaluation finale est sommative. De façon globale, elle apprécie l’impact du programme sur la sécurité alimentaire et nutritionnelle des populations bénéficiaires. Il s’agit donc d’apprécier les effets et impacts du programme en termes de renforcement des capacités des communautés et des partenaires, d’augmentation de la production agricole, d’amélioration de l’état nutritionnelle des membres des ménages notamment les femmes et les enfants de moins de 5 ans. 4.2. Objectifs spécifiques • Apprécier les réalisations quantitatives en rapport avec les niveaux ciblés pour les indicateurs de performance en se referant à la table IPTT (Indicators Performance Tracking Table). Apprécier qualitativement ces résultats et leurs effets. • Apprécier les effets du processus de renforcement des capacités des communautés et des partenaires en prévention et gestion des crises sur la prise en compte des risques • Apprécier les effets du programme sur la réduction de la vulnérabilité des groupes cibles y compris les femmes, les enfants et les groupes spécifiques de la zone d’intervention du programme. • Apprécier la stratégie globale et opérationnelle de mise de mise en œuvre du programme pour l’atteinte des résultats • Apprécier l’état de mise en œuvre de certaines recommandations parmi les plus importantes de l’évaluation à mi-parcours • Formuler des recommandations et tirer des leçons pour la conception d’une éventuelle nouvelle phase du programme. L’ensemble de ces préoccupations seront analysées à travers les questions clés proposées pour cette évaluation. QUESTIONS CLES Les points contenus dans les questions clés seront analysées en mettant l’accent sur la cohérence, la pertinence, l’efficacité, l’efficience, la durabilité et l’impact. 5.1. Implémentation des activités • Les recommandations de l’évaluation mi-parcours orientées sur les effets, l’impact, les résultats des objectifs stratégiques et sur le Suivi et Evaluation ont-elles été suffisamment prises en compte dans la mise en œuvre du programme ? ont-elles contribué à l’atteinte et à la qualité des résultats souhaités ? 11 • La stratégie opérationnelle du programme a - t’elle été efficace dans l’atteinte des résultas. La stratégie a t’elle favorisé une adhésion et appropriation du processus, des actions et résultats par les bénéficiaires. • Y a t’il eu des changements dans le contexte sociopolitique ? Si oui lesquels ? Comment le programme s’est-il adapté à ces changements ? 5.2. Analyse des effets et impacts du programme • Quels sont les changements notables constatés au niveau des groupes cibles en termes de connaissance, attitudes et pratiques dans les domaines d’intervention du programme (agriculture, élevage, environnement et santé/nutrition) ? • La mise en œuvre du programme a-t-elle permis d’améliorer la disponibilité alimentaire dans les communautés et les ménages ? • La mise en œuvre du programme a-t-elle permis l’améliorer l’accès à la nourriture des femmes et des ménages bénéficiaires quelque soit leur niveau de vulnérabilité? • La mise en œuvre du programme a-t-elle permis d’améliorer l’utilisation de la nourriture au niveau des ménages bénéficiaires ? L’amélioration de l’utilisation, notamment par l’adoption de bonnes pratiques est-t-elle visible dans l’état nutritionnel des enfants de moins de cinq ans et peut-t-elle sembler durable ? • Quels sont les effets induits -- positifs ou négatifs - des activités du programme, dans la vie des communautés ? 5.3. Mise en œuvre des activités d’urgence • Quels sont les effets et impacts des activités de l’Unité d’Urgence : - chez les communautés en termes de renforcement des capacités locales (communautaires) en matière de prévention et de gestion des crises ; - chez les communautés en termes d’opérations d’urgences ; - chez les partenaires (services techniques et ONG) en termes de renforcement de capacités dans le processus d’élaboration des plans d’urgence, de contingence 5.4. Le food for work • La stratégie du food for work est-elle appropriée au contexte de la zone d’intervention ? • Le food for work a-t-il contribué à l’atteinte des résultats ? • Quels sont les effets/impacts de la pratique du FFW sur la durabilité des activités du projet ? 5.5. Durabilité • Les stratégies du programme permettent-elles d’assurer la pérennité des acquis ? Ces stratégies sont-elles pertinentes et efficaces pour maintenir les avantages liés aux activités et à leurs impacts ? 12 • Quels sont les facteurs qui ont favorisé l’adhésion et la participation des populations aux activités du programme ? • Quelles leçons tire-t-on de la mise en œuvre du programme ? • Le partenariat/collaboration avec les services techniques, ONG, etc.) a t- il contribué à la diffusion, réplication et durabilité des actions du programme V. METHODOLOGIE DE L’EVALUATION FINALE L’évaluation finale comprend deux étapes composées de : - Une étude quantitative : qui a déjà eu lieu en avril et mai 2004 et dont les résultats sont disponibles pour servir à la mission d’évaluation ; elle a été dirigée par un consultant national. - Une évaluation qualitative, dont il est question ici. Elle utilisera les résultats de l’étude quantitative et toute la documentation disponible pour son analyse qu’elle complétera avec une série d’entretiens dans les communautés et auprès de tous les acteurs et partenaires du programme. 5.1. L’étude quantitative L’étude quantitative a permis de recueillir des données statistiquement fiables qui permettront de mesurer le niveau des indicateurs d’impact. Le niveau de ces indicateurs à la période de l’évaluation sera comparé au niveau observé au moment des études de base et à la mi-parcours pour apprécier les progrès accomplis le cas échéant, notamment par rapport aux niveaux cibles. Dès lors il faut s’assurer que les données collectées sont comparables aux données de l’étude de base. L’étude quantitative a pour objet d’évaluer la réalisation des objectifs (impacts mesurables) par rapport aux prévisions à travers la mesure des indicateurs communs et spécifiques relatifs à chaque objectif stratégique. La connaissance du niveau des indicateurs nécessite une observation directe des unités participantes (membres des ménages, ménages, communautés, partenaires, etc.). Le rapport de cette évaluation et les documents de pré évaluation des différentes ONGs seront mis à la disposition de l’équipe chargée de l’évaluation qualitative. 4.1. L’évaluation qualitative L’évaluation qualitative doit permettre de recueillir les avis des bénéficiaires et des partenaires sur leur implication dans la mise en œuvre du programme. Elle permettra également d’apprécier dans quelle mesure les activités du programme répondent aux besoins des bénéficiaires. Elle devra aussi permettre d’apprécier l’acceptation de la part des populations des stratégies de mise en œuvre des activités. Enfin elle sera l’occasion pour apprécier le degré de transfert du pouvoir et des compétences aux communautés du point de vue de leur habilitation et de la pérennisation de l’intervention. Elle mettra également l’accent sur les leçons apprises et les recommandations. 13 Elle reposera sur des entretiens communautaires (focus groupes ou informants clés) et des entrevues avec les principaux partenaires (services techniques, autorités administratives et coutumières et société civile). VI. ORGANISATION DE L’EVALUATION FINALE 6.1. Responsable L’évaluation finale sera placée sous la responsabilité de CARE Niger en étroite collaboration avec les autres ONGs participantes (Africare, CRS et HKI). 6.2. Supervision technique et équipement L’évaluation sera menée sous la supervision technique de CARE Niger avec l’appui de l’Unité de Coordination du Programme. Toutefois, en cas de besoin, les autres ONGs participantes peuvent être sollicitées à tout moment par rapport à des questions spécifiques ou à des questions d’ordre général. CARE organisera en collaboration avec l’Unité de Coordination du Programme et les autres ONGs participantes le déroulement de l’évaluation et mettra à la disposition de l’équipe d’évaluateurs tous les moyens matériels (déplacement et hébergement) et humains pour les visites des différents sièges des districts et de leur zone d’intervention. 6.3. L’équipe des Evaluateurs L’évaluation sera conduite par une équipe externe placée sous la supervision de CARE Niger avec l’appui du PCU. La langue de travail étant le français et les rapports soumis à l’USAID devant être rédigés en anglais, les évaluateurs doivent nécessairement avoir une parfaite maîtrise (lecture, rédaction et communication) de ces deux langues. Le Chef de l’équipe des Consultants veillera à ce que le rapport de l’évaluation soit rédigé dans chacune des deux langues. Le rapport doit donc être rédigé dans les deux langues et accepté et approuvé par le consortium avant le paiement final des évaluateurs. L’équipe chargée de l’évaluation sera composée de trois consultants et deux représentants de l’USAID FFP (le bailleurs) et du Gouvernement du Niger: Un Consultant International socio-économiste, Socio-Anthropologue ou agro-économiste (chef d’équipe) avec une expertise confirmée en évaluation des programmes du Titre II de l’USAID. Il est le premier responsable de l’équipe et doit par conséquent disposer d’une sagesse et d'une habileté à diriger une équipe et à assurer la coordination des analyses et la rédaction du rapport. Il est le responsable du rapport final et doit assurer que le document répond aux normes du BHR/FFP de l’USAID. Le chef d’équipe doit avoir une expertise en matière d’évaluation des programmes de développement notamment en ce qui concerne le renforcement des capacités institutionnelles locales, le développement durable, l’analyse de la vulnérabilité et les questions liées au risque et à la gestion des catastrophes. Un Consultant national Agro-économiste, socio-économiste, ou socio anthropologue avec une expertise en évaluation des programmes et projets. Il doit avoir une expertise du 14 développement notamment en ce qui concerne les capacités institutionnelles locales, le développement agricole durable et la nutrition. Il doit également avoir une connaissance sur l’évaluation des programmes/projets de l’USAID. NB : si le chef d’équipe est agroéconomiste, le consultant national doit être socio anthropologue et inversement. L’agroéconomiste doit avoir des compétences techniques en : - évaluation de la faisabilité technique et les analyses coûts-bénéfices des technologies agricoles pluviales et irriguées ; et les techniques de vulgarisation ; - évaluation de l’impact environnemental des innovations agricoles et de la construction des infrastructures socio-économiques. - Il doit aussi être familier avec les projets de l’USAID dans les pays sahéliens. Un Consultant national expert en évaluation des programmes de Santé-Nutrition ou expert en santé publique. Il doit disposer de grandes connaissances en suivi et évaluation des programmes de santé - nutrition basés sur une approche communautaire conformément à l’Initiative de Bamako adoptée par l’Etat nigérien. Il doit donc être suffisamment informé de la politique sectorielle de santé notamment les aspects de participation des communautés dans la prise en charge des problèmes de santé (recouvrement des coûts, consultations prénatales, vaccination de la mère et de l’enfant). En matière de nutrition, l’expert doit maîtriser les outils d’évaluation des approches basées sur la déviance positive, en matière des pratiques d’allaitement et d’alimentation des jeunes enfants (groupe de soutien à l’allaitement, groupe d’appui à l’alimentation, maman lumière) et de récupération nutritionnelle à base communautaire (foyers nutritionnels, suivi nutritionnel à base communautaire). - Un Représentant du gouvernement nigérien. Il est la personne ressource représentant les points de vue de la partie nigérienne. Il doit être averti de la stratégie nationale en matière de sécurité alimentaire et nutritionnelle. Il doit apprécier sur cette base la pertinence et la contribution du programme par rapport à cette stratégie. - Un Représentant du BHR/FFP/USAID. Il est la personne ressource représentant les points de vue du bailleur de fonds. Il doit disposer d’une expérience en matière d’évaluation des programmes du titre II. De part cette expérience, il doit orienter l’équipe d’évaluateurs dans le respect des normes en la matière. - Au besoin, au niveau de chaque district, un représentant des services techniques peut être associé à l’équipe VII. RESULTATS ATTENDUS Le principal résultat attendu de cette évaluation est le rapport d’évaluation axé sur le points suivants : • Les effets et impacts du programme ; • Les leçons apprises ; 15 • Les recommandations concrètes et pertinentes, formulées à partir des forces et faiblesses constatées, • La fonctionnalité de la structure opérationnelle du Programme, c’est-à-dire l’organigramme global (comité de pilotage, PCU, ONGs et Projets) – notamment dans les aspects d’harmonisation, de partage d’expériences et des approches communes – est appréciée et des propositions d’amélioration sont formulées ; • Les recommandations pour : Š la conception d’une nouvelle phase du DAP notamment en terme de choix d’objectifs et d’actions pertinentes et de stratégie dans la zone cible. Š Une meilleure prise en compte du processus de mise en oeuvre du mécanisme de prévention et gestion des crises au sein du consortium VIII. PROJET DE CALENDRIER Le Calendrier suivant est proposé pour la conduite de l’évaluation à mi-parcours. Jours Activités 15 juin 04 Elaboration des TDRs 17 juin 04 Envoi des TDRs aux CS pour feed-back 24 juin 04 Réception des feed-back des CS 25 juin 04 Finalisation des TDRs 7 juillet 04 Avis de consultation 31 juillet 04 Réception des offres consultants 2 Août 04 Choix des consultants 06 août 04 Approbation du consultant chef d’équipe par BHR/FFP 10 août 04 Signature des contrats consultants 01 au 02 septembre 04 Arrivée du chef d’équipe et Briefing à Niamey 03 au 05 septembre 04 Rencontre avec consultants nationaux ; Revue documentaire ; programme de terrain, etc. 06 au 26 septembre Terrain (y compris des restitutions aux districts) 27 septembre 04 Débriefing à Niamey au Steering Committee 28 au 30 septembre 04 Rédaction Rapport en Equipe (Consultants, Superviseurs) 01 au 03 septembre 04 Finalisation Rapport draft (Chef d’Equipe) 04 septembre 04 Restitution des résultats préliminaires au noyau de S/E et coordonnateurs de programme et direction de CARE ANNEX 3 LIST of CONTACTS Liste de Contacts Nom et Prénom Fonction Location Kathy Tilford Directrice, CARE-Niger Niamey Najim Mohamed SG, Ministere du Developpement Communautaire Niamey Pascal Payet Suivi et Evaluation (S/E) ISAN/PCU Hamadou Adamou S/E CRS Idrissa Amadou S/E CARE Rhili Aaboubacar S/E AFRICARE Aklou Sidi Sidi Coordinnateur AFRICARE Agadez Kabwayi Kabongo Chef PCU Baguirbi Issa Coodonnateur ISAN HKI Idrissa Chipkao Coorrdonnateur Adjoint ISAN CRS Edouard Jay Coordonnateur ISAN CRS Ali Bety Coordonnateur ANR CARE Gagara Abdou Communicateur PCU Kwanza Price Agent Finance PCU Alhassana I Outman Representante AFRICARE Ibrahim Idi Issa Programme Manager AFRICARE Fatimata Jules Traductrice Agadez Harouna Hamani Nutritionniste HKI Sayo Amadou Directeur Adjoint CARE Saley Boukari Chef de District ISAN Doutchi Ousseina Sountalma Coord. Adj.Securite Alimentaire ISAN Doutchi Oumarou ELH Omar Coord. Adj. Sante/Nutrition ISAN Doutchi Harouna Mayaou Charge de S/E CRS/HKI, Doutchi Kailou Mallam Gerard CAD Doutchi Michel Zanguina Maidabo CAD Doutchi Alain Maiksoua CAD Doutchi Souley Soumana Lacho Coordinateur de l'equipe, RAIL Doutchi Sanoussi Chaibou Mali Informaticien de l'equipe Doutchi Aziz Ameyagi Superviseur Sante/Nutrition AFRICARE Agadez Kouna Biska Superviseur OS1 AFRICARE Agadez Ounis Ahmen Superviseur Agriculture AFRICARE Agadez Mohamen Alhousseine Superviseur FFW AFRICARE Agadez Yahaya Yan Daka Gouverneur Agadez Mr le Prefet Prefet Tchirozerine Soumana Dogon Yaro Génie Rural Tchirozerine Abdoul Karim Israel Hydraulique Tchirozerine Sani Salaou Inspection Enseignement de base Tchirozerine Mohamed Cheloukan Développement Communautaire Tchirozerine Hassane Moussa Environnement Arlit Mahaman Sani Samailla Ressources Animales Arlit Hamid Mohamed Haïdara Alphabétisation Arlit Oumarou Djatti Préfet Arlit Elhadji Almoustapha Ibrahim Chef de Groupement Arlit Abdoul Aziz Issa Santé Publique Arlit Sanoussi Elisha President CARITAS Zinder Mahamane Laoulai Moussa Chef District, CRS Tanout Marafa Saley Tchiroma Coord. Adj. Survie de l’Enfant, HKI Tanaout Mahaman Souradja Coord. Adj.Sécurité Alimentaire, CRS Tanout Hassane Hamidine S/E CRS/HKI Tanout Adèle Mêley Bienvenu Traductrice Tanout 3 Liste de Contacts, continued Nom et Prenom Fonction Location Yahouza Sabo Prefet Tanout Salou Muonkaila Chef SAP Tanout Hamou Djibrina Chef Service Génie Rural Tanout Ibro Issa Service Environnement Tanout Hamidou Issoufou Développement Agricole Tanout Père Emmanuel Ngora CARITAS Zinder Sœur Dolores Astorga CARITAS Zinder Otchoumaré Haoua CARITAS Zinder Haoua Ali hadi CARITAS Zinder Père Callistus Baalaboore CARITAS Zinder Anatovi Albert Clement CARITAS Zinder Abdoulaye Mati CARITAS Zinder Lamine Oumarou CARITAS Zinder Abdou Garba Coordonnateur ISAN CARE Konni Idrissa Amadou S&E ISAN Konni Alio Namata Chef Unite d’Urgence Konni Sanda Atarka AT Protection Environnement CARE Konni Souley Moussa Gestionnaire des denrees CARE Konni Aichatou Lawali AT CARE Konni Ramatou Alfari AT Sante Nutrition CARE Konni Mamane Sayo AT Initiatives Communautaires CARE Konni Nazir Nakaka Technicien Développement Communautire CARE Konni Mme Manou Saâ AT/PA Konni Ali Oumarou DT AT S&E CARE Konni Aïssa Boubacar Traductrice Konni Abdoulaye Madougou Magasinier des denrées Konni Goumar Alhassane Technicien Pisciculture/Foresterie Konni Sani Mahaman Laminou Chef de Projet Santé Barthe Dotti Chef de Groupement Peulh Konni ELH Abdoulmoumouni Hassane Chef de Canton Konni Lamine Amani Prefet Konni Kané Issaka Environnement Konni Mamane Maïnassara Ressources Animales Konni Boubacar Abdoulaye COFO Konni Chegou Tiza Bacha Développement Agricole Konni Rabiou Ibrahim ONG GYARA Konni Salissou Illiassou, DDDC DDDC Konni Ibrahim Kaza Gaoh Alphabetisation Konni Habibou Moussa Chef, "SIDA en Exode," CARE Konni Ibrahim Amadou Ressources Animales Illela Ibro Kada Développement Agricole Illela Mamane Alassane Environnement Illela Mani Saley COFO Illela Mr. le Adjoint Prefet Adjoint Prefet Illela Mamane Awache Chef Service GR Illela Kalla Lari MMD Illela 4 Liste de Contacts, continued Nom et Prenom Fonction Location Ibrahima Oumarou Inspection Etudes de Base (IEB) Illela Sahidou Ousseini Developpement Social Illela Hamissou Gandasu Alphabetisation Illelat Lamine Hassane CARITAS Tanout Sanoussi Elisha CARITAS Tanout Jennifer Peterson Ambassade des USA Niamey Ali Abdoulaye CRS Nimaye Marie Aughenbaugh AFRICARE Nimaye Marianna Hensley CRS Niamey Sarah BAILEY CRS Niamey Ismaril Ekaney Chimier Coordinateur National, CASPANI￾YARDA Tanout Amadon Laoual Service Sante Dogondoutchi Bako Kabsuri DDDC Dogondoutchi ANNEX 4 INDICATOR PERFORMANCE TRACKING TABLE: Impact Indicators INDICATOR PERFORMANCE TRACKING TABLE, FY04 DRAFT VERSION OS1: AMELIORER LA CAPACITE COMMUNAUTAIRE POUR FAIRE FACE AUX PROBLEMES DE SECURITE ALIMANTAIRE Indicateur d'Impact Base Objectif mi￾parcours Réalisation mi-parcours*13 Réalisation vs. Objectif mi￾parcours Objectif An Fisc 04 Réalisation An Fisc 04 Réalisation vs. Objectif An Fisc 04 Objectif sur la durée du programme Indicateur d’impact 1.1 # de communautés ayant démocratiquement conçu leurs plans de sécurité alimentaire avec genre et équité 0 110/ 142 61/171 55% 200 200 (80%) Africare Agadez 0 12 6 50% 20 (33%) NEC 30 (50%) CARE – Konni/Illela 0 32/40 33 (sur 69) 60% 56 (80%) NEC 56 (80%) CARE – Matameye14 (COSAN) 0 40 9 S 23% CRS/HKI – Dogon Doutchi 0 13 7 S 54% 48 (80%) NEC 48 (80%) CRS/HKI – Tanout 0 13 6 S 46% 48 (80%) NEC 48 (80%) Indicateur d’Impact 1.2. Indice de capacité de sécurité alimentaire 35.0 40.6 50.1 123% Augmentation de 30% Augmentat° de 40% Africare 27.0 31.1 52.5 169% 60 NEC 70 CARE – Konni/Illela 18.016 20.7 45.4 219% 60,1 NEC 62 CARE – Matameye N/A 42.217 35.9 85% CRS/HKI – Dogon Doutchi 52.718 60.619 56.9 94% 69,3 68,7 99,13% 71,3 CRS/HKI – Tanout 42.320 48.721 59.9 123% 60 65 13 Ces données indiquent les réalisations pour sur la durée du programme pour les activités de santé de CARE Matameye achevées en Mars 2003. 14 Pour Matameye, cet indicateur a été calculé pour les COSAN en se basant sur “la fiche d’Evaluation de la Performance des COSAN”. Les évaluateurs à mi-parcours ont inclus les COSAN de Matameye dans les calculs des objectifs et réalisations à mi-parcours. 16 La référence de base a été calculée de manière rétroactive après que le programme ait travaillé dans 20 Unités Interventions. 17 En l’absence de donnée de base, cet objectif correspond à une moyenne de l’indice de capacité communautaire de sécurité alimentaire. 18 Pour les 20 nouveaux villages sélectionnés en année fiscale 03. Ces villages sont situés dans la partie sud de la zone et ont fait montre de plus d’organisation que les villages sélectionnés en année fiscale 01. 19 Pour les 16 villages sélectionnés en année fiscale 01, ont donc bénéficié de deux années d’intervention du programme. 20 Pour les 20 nouveaux villages sélectionnés en année fiscale 03. 21 Cet objectif correspond à une augmentation de 15% du niveau des données de base. OS2: ACCROITRE LA PRODUCTION AGROPASTORALE EN PROTEGEANT L’ENVIRONNEMENT Indicateur d'Impact Base Objectif mi￾parcours Réalisation mi- parcours Réalisation vs. Objectif mi-parcours Objectif An Fisc 04 Réalisation An Fisc 04 Réalisation vs. Objectif An Fisc 04 Objectif sur la durée du programme Indicateur d’Impact 2.1. Nombre de mois de provision alimentaire des producteurs participants37 Moy:6.11 Moy:7 6.2 89% 7,43 6,44 86,67% 8,16 Africare – Agadez 2000 : 6 Moy :7.3 6.0E 82% 6 6,37 106,1% 6,5 Très Vulnérable © 70% 5.27 (55.19 (57,5%) 4,91 Vulnérable (B) 24% 6.62 A7.04 (32,4%) 7,38 Moyennement Vulnérable (A) 6% 10.35 A7I77 (10,1%)I 9,94 CARE – Konni/Illela Moy: 5.3 Moy: 6.7 5.52 82% 7 5,6 80% 7.8 Très Vulnérable (C) 68.1% 4.85 5,07 (66,5%) 4,22 Vulnérable (B) 22.3% 5.90 7,70 (24,6%) 7,14 Moyennement Vulnérable (A) 9.6% 6.62 6,94 (8,9%) 9,17 CRS/HKI – Dogon Doutchi Moy:5.9 Moy: 7 6.1 E 87% 7,25 5,73 79% 8,4 Très Vulnérable (C) 56.0% 5.16 5,80 (62,1%) 5,31 Vulnérable (B) 29.5% 6.66 6,37 (29,1%) 6,72 Moyennement vulnérable (A) 14.5% 7.37 7,51 (8,8%) 7,31 CRS/HKI – Tanout 7.0 Moy:8.67 7.3 84% 9 7,56 84% 9 Très vulnérable (C) 54.1% 3.0 5,64 (64,4%) 7,2 Vulnérable (B) 31.1% 7.0 8,92 (21,2%) 8,2 Moyennement vulnérable (A) 14.8% 11.0 11,9 (14,4%) 9 37 Pendant l’étude de base, les ménages ont été demandés à propos du nombre de mois de couverture de leur besoin alimentaire avec leur propre production agropastorale en 1999 qui était une année à bonne pluviométrie, et l’année 2000, qui était une année à mauvaise pluviométrie. Le staff du projet pense que cet indicateur n’est pas pertinent , sauf si les données des évaluations à mi-parcours et finale sont comparées à des années à pluviométrie similaire. Le but de chaque district est d’augmenter de 2.5 mois la couverture. Le nombre de l’étuide de base à utiliser va dépendre de la pluviométrie des années où les évaluations à mi-parcours et finale ont été conduites – Si l’année est bonne, il est espéré de couvrir une moyenne de 9 mois vers la fin du projet. Si l’année est mauvaise, il est espéré d’atteindre au moins 7 mois de couverture. Le % de ménages qui couvrent leurs besoins, et le % de ceux qui réduisent leur consommation comme stratégie de survie seront suivis pour comparaison et triangulation. Impact Indicateur 2.2. Volume (TM) et valeur (CFA et USD)39 des productions agricoles sélectionnées par les producteurs participants 1,321 kg/HH 92 287 Fcfa 1486,13 kg 1368 kg 92% 1 595,34 Kg 1 394 Kg/HH 165 982 Fcfa 87,39% 1 629,98 Kg/HH Africare –Agadez (les produits à suivre dans le futur comprennent les légumes, le lait, le bétail – les données de base de ces produits seront fournies par les producteurs participants 406 kg. 88,326 CFA 436.45 kg 419.3 kg 96% 466,9 Kg 427 Kg/HH 103 626 Fcfa 91,4% 487 kg Augmentation moyenne de 20% /HH 105 991 CFA Très Vulnérable (C) 70.0% 431 Kg 1216.3 ND Vulnérable (B) 24.0% 403,9 Kg N.A ND Moyennement Vulnérables (A) 6.0% 319,1 Kg 2086.7 ND CARE – Konni/Illela (Mil, sorgho, haricot; les produits à ajouter dans le futur comprennent le riz, le manioc, le poisson – les données de base seront obtenues des producteurs participants)41 1 071 kg, 133 371 F cfa 1 204,88 kg 1 216,3 kg 101% 1 231,65 Kg 1 126 Kg/HH 130 831 Fcfa 91,4% 1285 Kg Augmentation de 20%/HH très Vulnérable (C) 68.1% 855.32 1 032,3 Kg ND Vulnérable (B) 22.3% 1306.03 1 531,2 Kg ND Moyennement Vulnérable(A) 14.5% 2061.79 1 716,2 Kg ND CRS/HKI – Dogon Doutchi (Mil, sorgho, haricot et légumes) 1,558 kg, 202.210 F cfa 1791.7 kg 2 086,7 117% 2134 Kg 1 521Kg/HH 199 208Fcfa 71,2% 2181kg/HH 283.094 F cfa Augmentation de 40% Très Vulnérable (C) 56.0% 1,109.01 1 735,8 Kg ND Vulnérable (B) 29.5% 1,845.32 2 194,6 Kg ND Moyennement Vulnérable(A) 14.5% 2,700.87 4 252 Kg ND CRS/HKI - Tanout (Mil, sorgho, haricot et légumes) 2,270 kg 2,610.5 kg 1,544.8 59% 1622 Kg 1 673 Kg/HH 176 986 Fcfa 103,1% 1622 Kg 39 Les évaluateurs à mi-parcours, en se basant sur les inputs et réactions de FFP/Dakar, ont décidé qu’à cause du nombre des variables qui influencent la valeur des produits agricoles, que seulement les données du volume total doivent être collectées pour cet indicateur. 41 *Les évaluateurs ont proposé de reporter seulement les trois cultures. Très Vulnérable (C) 54.1% 1 396,3 Kg ND Vulnérable (B) 31.1% 1 447,5 Kg ND Moyennement Vulnérable (A) 14.8% 2 352,2 Kg ND SO3: AMELIORER L’ETAT SANITAIRE ET NUTRITIONNEL DES MEMAGES Indicateur d'Impact Base Objectif mi￾parcours Réalisation mi- parcours Réalisation vs. Objectif mi-parcours Objectif An Fisc 04 Réalisation An Fisc 04 Réalisation vs. Objectif An Fisc 04 Objectif sur la durée du programme Indicateur d’Impact 3.1. % d’enfants de 24 to 59 mois en retard de croissance 51.11% 48% 47.8% 100% 44,81% 46,6% 43,54% Africare – Agadez 35.3% F=34.6% H=36% 32% 33.1% 66.7% 31% 38,3% 31% CARE - Konni/Illela (activités de santé ont debuté en Avril 03) 44.1% F=46.2 % H=42.2% 37% 41,1% 36% CARE - Matameye (Matamèye activités finies en March 03) 69.3%47 59.3% 56.4% 129% CRS/HKI – Dogon Doutchi 53.9% F= 55,3% H=52% 50% 44.7% 236% 43% 46% 43%48 CRS/HKI - Tanout 59.7% F=62.5% H=57.4% 56% 67.6% 83% 65% 56,6% 60% 47 Ce chiffre a été corrigé via l’évaluation à mi-parcours qui a remarqué que les chiffres précédents reflétaient le poids/age contrairement à la taille/age. Donc à cause de l’augmentation de la valeur, les prévisions ont été modifiées. La cible dans le DIP Original pour la durée de vie du projet (FY 03) pour Matameye était 64% 48 La cible pour la durée de vie du projet a augmenté sur la base des niveaux de l’indicateur à l’évaluation à mi-parcours. SO3: AMELIORER L’ETAT SANITAIRE ET NUTRITIONNEL DES MEMAGES Indicateur d'Impact Base Objectif mi￾parcours Réalisation mi- parcours Réalisation vs. Objectif mi-parcours Objectif An Fisc 04 Réalisation An Fisc 04 Réalisation vs. Objectif An Fisc 04 Objectif sur la durée du programme Indicateur d’Impact 3.249. % d’enfants mis au sein pendant les 8 heures qui suivent la naissance 30.4% 39.8% 64.7% 163% 70,89% 72,6% 102,4% 73,15% Africare - Agadez 33.3% 40.3% 73.5% 183% 80% 80,5% 100% 85% CARE – Konni/Illela 24.3% 69,8% 60% 85,9% 71,8% CARE - Matameye 39% 50% 54.2% 108% CRS/HKI – Dogon Doutchi 36%50 38.8% 70.2% 181% 73% 73,1% 100% 75% CRS/HKI - Tanout 20%51 31.4% 62.9% 200% 63% 82,8% 131,4% 65% Indicateur d’Impact 3.3. % de villages dont la consommation de la vitamine A est au dessus du niveau de risque 10.22 30% 30%52 100% 32,52% 28,7% 88,25% 36,8% Africare/HKI - Agadez 6% 20% A Déterminer A Déterminer 20% 6,7% 33,5% 25% CARE – Konni/Illela 33%55 45% 36,7% 81,5% 50% CARE - Matameye NE56 NA NA NA CRS/HKI – Dogon Doutchi 9.78%57 30% 33.3% 111% 35% 53,6% 153,1% 40% CRS/HKI - Tanout 6.30%58 30% 26.7% 89% 28 18,5% 66% 30% 49 Pendant les études de base les districts ont considéré différents groupes d’heures. Comme seul le groupe de 0-8 était universel, cet indicateur a été ajusté aux 8 premières heures après la naissance. 50 HKI, page 43 de l’ Evaluation Finale de leur Projet Survie de l’Enfant en 2000. 51 HKI, page 43 de l’ Evaluation Finale de leur Projet Survie de l’Enfant en 2000. 52 Cet indicateur a été calculé pour CRS Doutchi et Tanout seulement. Pour Agadez la taille de l’échantillon était très petite pour assurer la précision des données. 55 Dans de l’ Evaluation Finale du Projet Survie de l’Enfant de HKI en 2000, Le score moyen était de 24, mais il était de 30 dans leur zone d’intervention, et 40 in villages 0 – 5 km du centre de santé. Ainsi, ce niveau est appuyé aussi par des données secondaires. 56 Indicateur non mesuré à l’étude de base et l’évaluation à mi-parcours à Matameye.( NE veut dire Non Evalué) 57 HKI, page 33 de l’ Evaluation Finale de leur Projet Survie de l’Enfant en 2000 58 HKI, page 8 de l’ Evaluation Finale de leur Projet Survie de l’Enfant en 2000