Joint Publication 9 CARE Madagascar Title II Final Evaluation Report of Findings Prepared by Richard Caldwell, TANGO International, Inc. and Sarah Fry, USAID-funded Environmental Health Project January 2003 CARE Madagascar: Title II DAP Final Evaluation Page i Table of Contents Acronyms.......................................................................................................................... iii Executive Summary......................................................................................................... iv I. Introduction......................................................................................... 1 II. Objectives of the Evaluation.............................................................. 2 III. Evaluation Methodology .................................................................... 5 III.A Review and Analysis of Key Documents...............................................5 III.B Key Informant Interviews......................................................................6 III.C Primary Data Collection in the Field ....................................................6 III.D Project Site Selection and Participant Interviews ...............................7 III.E CARE Staff and Counterpart Participation in the Evaluation Process..................................................................................7 IV. Evaluation Findings............................................................................ 8 IV.A Mahavita..................................................................................................8 IV.A.1 Strategic Objective 1 .............................................................................8 Income Generation and Assets Training – Personal Empowerment and Livelihoods Savings Component FFW Activities IV.A.2 Strategic Objective 2 ...........................................................................14 Developing Community-based Structures Fokotany Selection Empowering Structures de Developpement SD Activities and Management IV.A.3 Strategic Objective 3 ...........................................................................18 Infrastructure Development Water/Sanitation/Washbasins Hygiene Promotion Enabling Environment IV.A.4 Strategic Objective 4 ...........................................................................23 Health Sector Malnutrition IV.B CYPREP (Cyclone Preparation on the East Coast) ..........................25 IV.B.1 Intermediate Result 1 ..........................................................................25 Strategic Orientation 1: Strategic Orientation 2: Fokontany Development Plans (PDF) Cyclone Preparedness Plans (PPCs) Strategic Orientation 3: CARE Madagascar: Title II DAP Final Evaluation Page ii Mitigation Activities IV.B.2 Intermediate Result 2 ......................................................................31 Strategic Orientation 4: Strategic Orientation 5: Accomplishment of CVS Tasks IV.B.3 Intermediate Result 3 ......................................................................33 Strategic Orientation 6: The Commune Emergency Management Committee Capacity-building of Social Mobilization Agents IV.C Food Aid Management Systems & Internal Controls .......................35 IV.D Food Distributions in FFW Programs ................................................35 V. Conclusions................................................................................... 36 V.A Mahavita................................................................................................36 Strategic Objective 1: Strategic Objective 2: Strategic Objective 3: Strategic Objective 4: V.B East Coast Programming – CYPREP.................................................39 Intermediate Result 1: Intermediate Result 2: Intermediate Result 3: V.B.1 Other CYPREP Conclusions ...............................................................43 VI. Recommendations for DAP II .................................................... 43 Urban Development/Governance Environmental Health Health Guiding Principles for DAP II Environmental Health activities Enhancing Food and Livelihood Security on the East Coast Annexes Annex A CDD (James Garrett) ....................................................................................50 Annex B Evaluation Terms of Reference ....................................................................59 Annex C List of Persons Contacted .............................................................................65 Annex D Mahavita Fokotanys......................................................................................66 Annex E Qualitative Survey Sites for CYPREP..........................................................67 Annex F Qualitative Research Site Visits ...................................................................68 Annex G Engineering Report (in French) ....................................................................70 CARE Madagascar: Title II DAP Final Evaluation Page iii Glossary and Acronyms ADC Agent de Développement Communautaire AMS Assistant de Mobilisation Sociale (Field/Extension Agent) APIPA Autorité pour la Protection contre les Inondations de la Plaine d’Antananarivo CARE Cooperative for Assistance and Relief Everywhere CCS Conseil Communal de Secours (Commune Emergency Management Committee) CFW cash-for-work CLS Conseil Local de Secours (niveau sous-préfecture) (District Emergency Management Committee) COGES Comite de la Gestion et Suive (Management and Monitoring Committee) CYPREP Cyclone Preparation Project CUA Commune Urbaine de Antananarivo CVS Comite Villageois de Securite (Village Security Committee) DAP Development Assistance Program EHP Environmental Health Project (USAID) FALAFA Famakafakana Lalina ny Fiainana eny Ambanivohitra (Malagasy acronym for HLS analysis) FAMOA Fanadihadiana Miaraka ny Olana sy ny Alaolana FFP Office of Food for Peace FKT Fokotany (the lowest administrative unit in the Government of Madagascar) FMG Francs Malagasy (approximately 6,200 FMG = $1.00 U.S.) Fokontany Malagasy word for village or community FFW food-for-work GOM Government of Madagascar HIF Hygiene Improvement Framework HLS Household Livelihood Security IFPRI International Food Policy Research Institute ISIKI Io no Sehatra Ifanakalozan-Kevitra sy Andrindran’Asa LOP life of project MOH Ministry of Health PAIQ Programme d’Appui aux Initiatives de Quartier PDF Plan de Developpement du Fokotany (Fokotany Development Plan) PDQ Plan de Developpement du Quartier (Neighborhood Development Plan) PL Public Law (480) PPCC Plan de Préparation Cyclonique Communal (Commune Cyclone Preparedness Plan) SCVM Sécurité des Conditions de Vie des Ménages (HLS) SD Structures de Developpement USAID United States Agency for International Development VCT vivre contre travail (food-for-work) CARE Madagascar: Title II DAP Final Evaluation Page iv Executive Summary CARE Madagascar’s Title II DAP began in 1998 and included two sub-programs - Mahavita and CYPREP. Mahavita was operational in 30 of Antananarivo’s poorest neighborhoods and aimed at sustainably improving household food and livelihood security. Its primary delivery mechanism was a well-orchestrated, community-based approach. CYPREP assisted rural communities throughout Tamatave Province on the east coast of Madagascar in risk management through planning and preparing for natural disasters. This evaluation provides findings from the evaluation of both components, as well as conclusions regarding the impact of the program and recommendations for future work in both rural and urban settings. The program set out to facilitate the creation of community-based structures (SDs) empowered to identify, plan, implement, coordinate and sustain activities that would contribute to the well￾being of residents. In this Mahavita made significant progress in empowering communities and the majority completed community plans to guide development over the coming years. Many residents express a feeling of being part of an important process and that their efforts will result in improved living conditions. Mahavita’s efforts to improve income and savings in poor households could have been done more effectively, and as a result there is little evidence that incomes have increased due to new employment opportunities for the urban poor. The FFW component, however, was very successful in targeting poor households and in providing a safety net in the form of cash and food. This was particularly critical during the political crisis of 2002 when food wages made an important economic contribution to households. Most of the community SDs are now struggling with ‘next steps.’ Members still appear highly motivated, and some SDs are now seeking their own funding with limited success, but there has been no coordinated effort on the part of Mahavita to develop alternative strategies and capacity for such action. This will be a key area to reinforce in the follow on DAP. Mahavita also facilitated improvements in the environmental health of Fokotanys (neighborhoods) through training and infrastructure development. Infrastructure development followed community empowerment, and it is more likely infrastructure will be maintained by the community. Where program activities have been active the neighborhoods are noticeably cleaner. Residents speak of noticeable health improvements in areas such as skin disease, children’s diarrhea, and even infant mortality. Despite positive gains in environmental health, more work is needed in water and sanitation, especially as it applies to fecal matter disposal. The CYPREP component was successful in developing networks of local partners for disaster management and response and for mobilizing organizations and communities around important livelihood themes. Communities are now organized to warn residents of impending storms and plan for how they can mitigate the impact of disasters. Communities also for the first time have development plans that not only address disaster response, but also incorporate improvements to livelihoods. In addition, there is much more understanding by community members concerning the impact of cyclones and the action needed to mitigate them. CARE Madagascar: Title II DAP Final Evaluation Page 1 FINAL EVALUATION CARE MADAGASCAR TITLE II DEVELOPMENT ACTIVITY 1998-2003 I. Introduction Households in Madagascar’s urban neighborhoods face critical food and livelihood security problems experienced in many other of the world’s poorest urban environments. Among the challenges household’s face include a changing and often unstable economic climate, rapid population growth, inefficient and poorly functioning municipal services, and intense competition for limited income-generating opportunities. The urban poor have aspirations like any other household – they want to send their children to school, live in a comfortable home, eat preferred foods, and maintain good health. All of these aspirations present challenges to the majority of Antananarivo’s urban poor as they try to achieve stable food and livelihood security. CARE Madagascar uses U.S. Government resources to address food security problems in Antananarivo, Madagascar’s capital. Its current Title II DAP was submitted, off-cycle, in September 1997 and includes two sub-programs: Mahavita1 and CYPREP. Mahavita began in 1998 and is an urban household food and livelihood security program. It assists the populations of 30 Fokontany (FKT), the lowest formal administrative structure in the Government of Madagascar. Each FKT represents 8-10,000 people and is located within Antananarivo’s urban flood zone, an area subject to complete inundation of water for 3-4 months per year and representing the most insalubrious environments in the city. The total population of these thirty FKTs is approximately 200,000 and is comprised mainly poor urban households, some of which are squatters, recent migrants from rural areas, and economically displaced urbanites. Mahavita has evolved as an umbrella program with an overall goal of sustainably improving household food and livelihood security of poor households in Antananarivo’s most vulnerable communities. It is comprised of interventions designed to improve household income and savings, community and personal empowerment, infrastructure and health and hygiene behavior. Its primary delivery mechanism is a well-orchestrated, community-based approach. Projects included under Mahavita include PAIQ (Dec. 1995 – Sept. 1997)2 , TOUCH 2000 (Feb. 1996 to present) and Safe Water System. The main objectives of these projects are included in Table 1. CARE Madagascar’s Title II program’s CYPREP component assists rural communities throughout Tamatave Province on the east coast of Madagascar. This cyclone-prone coastal region is rich in natural resources yet poor households are highly vulnerable to severe storms each year. CYPREP assists communities in risk management through planning and preparing for natural disasters. Through efforts over the past three years, CARE has established strong 1 Mahavita is a Malagasy term meaning “working together to achieve a common goal.” 2 Though this project ended before Mahavita began, PAIQ project efforts were retained and continued to be a large part of Mahavita. CARE Madagascar: Title II DAP Final Evaluation Page 2 relationships with fokotany, district, and provincial governments in its cyclone awareness program. Table 1. Sub-components included in Mahavita. Projects Dates Objectives PAIQ Dec. 1995-Sept. 1997 • Provide technical assistance to local communities in Antananarivo for the submission, funding, implementation, follow-up and post-project management of small urban infrastructure projects. TOUCH 2000 Jan. 1996-September 1999 • Reduce infant and child mortality in poor urban areas of Antananarivo through child survival interventions at public and private health facilities and at community and household levels. Safe Water System Oct. 1999 – Aug 2001 • Improve household water quality and decrease diarrheal disease in the target population by promoting treatment of contaminated water, safe water storage in plastic containers, and behavior change techniques, including social marketing, communication and education. Antananarivo Emergency Program January-July, 2002 • Provided a variety of FFW opportunities for households during the political crisis of 2001 AGETIPA Ongoing • Co-financed infra-structure in Mahavita fokotanys Petit Boulevarde July 2003 - present • Mahavita provides assistance on community awareness and social conflict resolution and supports urban water infrastructure activities. This report will discuss results from both the Mahavita and CYPREP components. It is largely organized around the strategic objectives of each component and includes observations, findings, and ways forward for future urban and East coast programming. II. Objectives of the Evaluation The DAP’s Monitoring and Evaluation System stipulated that: CARE-Madagascar’s DAP activities will be assessed through mid-term and final evaluations, which will cover both project effectiveness and impact. The final evaluation will be conducted in the final year of the project. Unlike the participatory approach used for the mid-term evaluation, the final evaluation will be conducted by an outside agency in compliance with PL-480 Title II DAP guidelines. The primary objective of the evaluation was to determine the effectiveness of the approaches used by Mahavita and CYPREP as well as the effect3 and impact changes that were brought about as the result of activities and outputs achieved throughout the life of project. The evaluation included several phases and components and was conducted by a team of three 3 Here “effect” means the behavioral and systemic changes brought about as a result of achieving project-related outputs. CARE Madagascar: Title II DAP Final Evaluation Page 3 consultants, two international and one national4 . It has also greatly benefited from two other pieces of work. The first was a case study of community development by James Garrett of the International Food Policy Research Institute (IFPRI) entitled “Scaling up Community-Driven Development: Reflections on the Mahavita Experience.” Since this study highlighted many of the processes used by Mahavita it is being used as a key supporting document in this evaluation and is included as Annex A. The second contribution was a commentary by Michaell Drinkwater of CARE on how Mahavita can transition into its next phase of urban programming and is entitled “Building Urban Governance in Antananarivo: Looking to the Future of the Mahavita Program” (CARE 2003). This document is available from CARE Madagascar. For Mahavita, the analytical focus of the evaluation was to assess the following questions: y To what degree have the original goals, objectives, and project outputs been achieved, and what are the main factors that contributed to the level of achievement? y How appropriate was the overall design of the program and of the specific objectives, activities and strategy to address the root causes of vulnerability and food insecurity? In addition, how realistic were the life-of-project and annual targets, the staffing plan and strategy, information systems (including monitoring and evaluation systems as well as geographic information systems), and budgets?. y How effective was program implementation, including the strategy and methodology employed, staffing effectiveness, and the quality of coordination and collaboration with project counterparts. y What was the quality and sustainability of specific interventions, including the project’s success in accomplishing planned results? What was the effectiveness of community mobilization and empowerment, and the effectiveness of Structures de Developpements (SD) in planning and coordinating community-driven activities? y Were there any unintended impacts, including applied mitigation measures, and internal and external factors that affected program implementation and outcomes, either in a positive or a negative manner? The environmental health component determined whether the program had been able to attain its stated objectives of improving environmental health and hygiene behaviors at the community and the household levels. A staff member from USAID’s Environmental Health Project (EHP) was invited to participate in the evaluation and proposed to conduct its part according to the Hygiene Improvement Framework (HIF) model, a program organizing principle for diarrhea prevention advocated by EHP, UNICEF and others. The HIF has three components that need to be addressed together in order to achieve the desired result of disease prevention: 4 The consultants for the Mahavita components were Richard Caldwell (Team Leader, TANGO International, Inc) Sarah Fry (USAID Environmental Health Project), A. Andriamasy (Infrastructure). For CYPREP the main consultants were Rakotomalala Bonaventure (ESA, qualitative research) and Andrianaivo Heritiana (quantitative research). CARE Madagascar: Title II DAP Final Evaluation Page 4 y Hardware (including water/sanitation infrastructure improvement and access to household technologies such as soap, water treatment materials, children’s potties) y Hygiene promotion (IEC, social mobilization, community participation, advocacy) y Strengthening the enabling environment (policy improvement, institutional strengthening, community organization, financing and cost-recovery, cross-sector and public/private partnerships). The evaluation mostly focused on the first two components (hardware/technologies and hygiene promotion) since a review of the enabling environment was a key part of the team leader’s Scope of Work. However, the enabling environment as it relates directly to environmental health was considered, for example, community management and cost-recovery systems established for infrastructure maintenance. Assessment of the quality and sustainability of the infrastructure itself is part of the consultant engineer’s SOW. During the course of the evaluation, the importance of CARE’s original involvement with the Ministry of Health (MOH) became apparent, and some time was devoted to exploring the history and status of relations as well as possible future connections, with MOH representatives at various levels. The infrastructure component involved cash and later food for work to rehabilitate canals and construct walkways, and provide water provisioning through cost-recovery systems. The evaluation examined these infrastructures by tabulating the outputs (# person-days, kilometers of canals rehabilitated, etc.), conducting physical inspections, and by reviewing processes and procedures, both social and technical. The fourth component involves evaluating the outputs of CYPREP and the effectiveness of the methodology used. In addition, CARE would like to use this evaluation as a means of preparing for the start-up of the new DAP. For CYPREP, the analytical focus of the evaluation was to assess: y Accomplishment of the goals, objectives, and project outputs. y Program design, including the appropriateness of the specific objectives, activities and strategy to address the root causes of vulnerability. In addition, annual targets, staffing plan, information systems, and budgets were addressed. y Program implementation, including the strategy and methodology employed, staffing effectiveness, and the quality of coordination and collaboration with project counterparts. y The quality and sustainability of specific interventions, including project success in accomplishing the results, the effectiveness of community mobilization and empowerment, and the effectiveness of community development structures in planning and coordinating community-driven activities. y Unintended impacts, including applied mitigation measures, and internal and external factors that affected program implementation and outcomes. CARE Madagascar: Title II DAP Final Evaluation Page 5 The terms of reference for the CYPREP and Mahavita final evaluation are included as Annex B. III. Evaluation Methodology A variety of activities were included in the evaluation to make it as participatory as possible and to broaden the range of viewpoints to be included. This included group meetings with Mahavita staff., interviews with CARE staff (ADC, IEC, Supervision, Infrastructure, Income Generation), informal talks with CARE and Mahavita management (present and past). A list of persons contacted is provided in Annex C. Evaluation team members held many meetings with health and municipal officials, including representatives at the communal level and the Arrondissement level. There were numerous field visits made to Fokotany’s where group discussions were held with Structures de Developpement (SDs) and FKT representatives. Other less formal discussions were held with staff and residents during neighborhood observational walks and physical “inspection” (technical and non-technical) of infrastructure. III.A. Review and Analysis of Key Documents The evaluation team reviewed several key CARE-Madagascar documents, including: y The original CARE Madagascar DAP, submitted in 1998; y Annual results reports from FY99 – FY2002; y Mahavita ‘Start-up Process Document’; y CARE-Ethiopia’s Long Range Strategic Plan of July 2001 to June 2006; y Mahavita Baseline Survey Document, 2000; y Mahavita Final Survey Document, 2003; y FAMOA – community mobilization and institutional capacity building strategy/methodology document; y HIMO – food for work planning, implementation and monitoring document; y ISIKA – Coordination and capacity building of stakeholders in water and sanitation infrastructure; y Mahavita Annual Evaluation and Planning Exercises; and y FITARIHA – Monitoring System y Food Aid Management Review y CYPREP Annual Reports 2001 and 2002 y CYPREP 2002 Work Plan The evaluation team also reviewed several specific project documents in the Mahavita and CYPREP field offices. These included miscellaneous project documents such as trip reports as well as proposals and progress reports for related activities (e.g. - TOUCH 2000 and Sur Eau). III.B. Key Informant Interviews In order to accumulate multiple perspectives, team members conducted key informant interviews with a wide variety of stakeholders involved in the DAP, including: CARE Madagascar: Title II DAP Final Evaluation Page 6 y CARE Madagascar program staff and management; y USAID Food-for-Peace Officers; y Fokotany officials; y Community members; y Arrondissement and Commune officials; y CYPREP Regional Managers in Tamatave; and y CYPREP Field Managers in Mahanoro. III.C. Primary Data Collection in the Field The most substantial and essential data for the final evaluation was collected during project visits to the field. For Mahavita, these visits were targeted to urban residents, members of SDs, and Fokotany leaders. It was not possible to visit each of the thirty Fokotany’s, but time did allow a significant number and range of sites to be visited, and included areas where both social and physical changes were being made and where current food-for-work activities are taking place. Annex D provides a list of the thirty Fokotanys included in Mahavita and denotes those that were visited. For CYPREP, field discussions were held with project engineers, Communal leaders, Fokotany leaders, members of CVS, and community members. Quantitative data was collected from household members. Annex E provides a list of sample sites from the qualitative survey and Annex F provides details of qualitative research site visits. CARE Madagascar: Title II DAP Final Evaluation Page 7 III.D. Project Site Selection and Participant Interviews In the urban context, a variety of Fokotanys were selected for inclusion in the evaluation study on the basis of: y Age & history of CARE involvement in the Fokotany; y Socio-economic status of the Fokotany (relative poverty or affluence as evidenced by the 2003 final survey); y Types of activities implemented within the Fokotany; and In the rural context of CYPREP, a variety of Fokotanys were selected for inclusion in the evaluation study on the basis of: y Age & history of CARE involvement in the Fokotany; y Level of vulnerability and ability to match with non-CYPREP villages; y Types of activities implemented within the Fokotany; and Each field visit consisted of a combination of: y Key informant interviews with Fokotany administration officials; y Focus group interviews with the SD – the development committee responsible for community-driven development; y Focus group interviews with foor-for-work participants; y Interviews with Fokotany residents; and y A tour of the Fokotany project activity site to assess technical quality and sustainability of specific infrastructure interventions. III.E. CARE Staff and Counterpart Participation in the Evaluation Process The evaluation team spent considerable time interviewing CARE project management and staff, including field agents. Additional time was devoted to interviewing … Finally, the evaluation team spent half a day with CARE staff to present the results of the data collection, discuss the accuracy and relevance of the findings, and together analyzing the findings. These were valuable exercises that brought closure to the evaluation team’s visits to urban and rural sites and added new perspectives to the findings. CARE Madagascar: Title II DAP Final Evaluation Page 8 IV. Evaluation Findings IV.A Mahavita Program Mahavita’s is somewhat unique among CARE’s projects in Southern and Western Africa and among Title II DAPs in general. During the first year of operation it applied a number of theoretical concepts and focused on building organizational structure and establishing common vision and principles. This “inception phase” was a well-crafted strategy to accomplish solid team building, empower staff, and understand communities and the operational environment of Antananarivo well enough so that effective programming could be achieved. The second year, FY 2000, Program Mahavita activities were mainly geared towards designing and testing methodologies for program implementation through a ‘pilot phase’, which also served as a practical training experience – ‘learning-while-designing’ – for the program team. The ‘pilot phase’ proved to be an effective means for achieving quality outputs in a relatively short period of time. It fostered group learning, and emphasized the importance of teamwork. It was also an effective way for the program team to gain ownership of processes. While it did not achieve all it set out to do, Mahavita was a pivotal project in the evolution of Title II programming. It was perhaps a few years ahead of its time in terms of focusing on issues such as empowerment and community mobilization in an urban food security context5 and would fit quite comfortably with the new Office of Food for Peace strategy that was recently articulated. Mahavita operated around three strategic objectives related to improving the food security of poor urban households in Antananarivo. The first addressed food access by focusing on income and savings. The second objective was to empower communities to address constraints affecting their ability to control their own resources and livelihoods. The third objective was to create and maintain healthy and hygienic environments that could support productive livelihoods. The fourth and final objective was to promote healthy behaviors at the household level. The findings within each of these objectives are discussed below. IV.A.1 Strategic Objective 1 Targeted households will be using increased income and savings to provide for their food and livelihood needs. Income Generation and Assets Income generation was recognized in the original project document as important to urban livelihoods. Indeed, income was the focal point of problem analyses conducted during the design of Mahavita and was one of the four objectives of the original TOUCH 2000 project. Improving household incomes was to come from several interventions, including food-for-work 5 Mahavita, although designed five years ago, is fully compatible with the new strategy document put out by the Office of Food for Peace, which has a number of Intermediate Results based on community capacity building, resilience, and human capacity building. CARE Madagascar: Title II DAP Final Evaluation Page 9 activities as a temporary income transfer and skills training that would enable individuals to better compete in a tight employment market. It was also recognized that in addition to training and education, individuals had to develop the confidence and capacities that would allow them to take control over their own livelihoods and manage interrelationships with others. Thus the personal empowerment elements of the project became integral to the strategy of income generation. Food-for-work was part of the overall income generating strategy of Mahavita. Originally, those recruited to work on infrastructure activities would be given 12-18 month contracts and paid in cash or food. Food received as wages would partly substitute for income during the infrastructure work and would improve household food consumption. During their time in the CFW or FFW activities, participants would also be involved in training activities, particularly during the rainy season months when infrastructure work is halted. These training activities were designed to help participants initiate and manage income generating activities more successfully. Thus, a second income generating strategy of Mahavita was to graduate community members out of skills and life training and encourage them to seek their own opportunities. Improving incomes was to include micro-enterprise development as well as skills training for improving individual employment prospects in the formal sector. It was envisioned that part of the population would take advantage of these opportunities and overall incomes would improve. It is unfortunate to address income generation as the first finding of the evaluation, because this was perhaps Mahavita’s major shortcoming in an otherwise successful project. The only focused income-generating strategy that reached a significant percentage of the targeted population was food-for-work. While there is no doubt that participation in FFW relieved households of some of their economic burden and substituted partly for income needs, and thus had a positive outcome on food security, it is questionable whether it had a tangible impact on incomes in the long run for the vast majority of households. It is also debatable whether or not it could have a sustainable positive impact on a household’s ability to access food since in an urban context income is so key to addressing access issues. There is some evidence, however, to suggest that the FFW experience could be effectively parlayed into viable skills training, and that some of the more entrepreneurial members may be able to generate work by offering their labor to service providers. There was one case of employment generated by SD members with skills acquired by FFW and applied in a private sector, contractual approach. The first occurrence was in FY2001 when 8,000 person days of employment was generated. For that particular year this 200 percent more than the target of 3,750 work-days. However, the LOP goal was 33,750 person-days. Thus, the overall achievement in this area was only 24 percent of that planned. The work experience that many gained was a positive and long-term effect, but to generate sustained employment requires a more intensive strategy than was used in Mahavita. In order to measure changes in the economic status of households asset ownership was selected as a proxy indicator, as previous studies in Madagascar and elsewhere had demonstrated the linkage between asset ownership and household wealth. Since direct income measurements were deemed unreliable, the asset status of the household was used to assess economic changes. A CARE Madagascar: Title II DAP Final Evaluation Page 10 standard set of assets was measured for each household. In the baseline survey these were measured for the general population from a random sample of households. In the final survey, these same measurements were made from samples of FFW households, households from Mahavita Fokotanys, and households from non-Mahavita Fokotanys. The results are provided in Table 2. Asset ownership in the baseline survey averaged about 2,200,000 FMG per household. In the final survey, the mean asset ownership for FFW households averaged significantly less at about 1,123,000 FMG. For non-FFW households the value rose about 30 percent to almost 3,000,000 FMG. Comparison household assets valued nearly 3,500,000 FMG. FFW targeted the poorest households, so baseline figures are not directly comparable with final survey figures. It does confirm, however, the FFW households were by and large poorer. It is most likely that asset ownership has not significantly changed for the overall population. Inflation has hovered at 8-10 percent per annum6 during the life of the project, and when factored in to asset value the differences between non-FFW households and baseline households are not significant. There is also no evidence that FFW households are better off than similar households in 2000. In 2000, those households in the same asset category as FFW households averaged about 950,000 FMG in assets. In the final survey they averaged just over 1.1 million FMG in assets. Again, when inflation is factored in, these two values converge, so that FFW households probably have about the same value of assets they did in the beginning of the project. Table 2: Asset ownership, in FMG, by household type. HH Type Electronics Furniture Utensils Appliances Vehicles Total Assets mean 255,990 523,791 308,519 17,626 17,208 1,123,135 FFW s.d. 703,430 1,180,708 1,227,404 60,316 116,666 2,847,151 mean 807,768 1,296,875 431,642 217,621 199,396 2,953,507 Non-FFW s.d. 1,366,443 2,185,848 885,804 759,102 982,000 4,389,986 mean 936,258 1,471,482 281,224 236,541 573,405 3,498,912 Comparison s.d. 1,538,831 2,938,621 561,017 725,714 3,391,136 6,438,718 mean 639,366 629,691 144,145 161,003 649,400 2,223,607 Baseline s.d. 1,551,798 1,046,314 211,835 643,138 3,039,979 5,197,099 This being said, there was a significant and prolonged political crisis during the latter stages of the project, and it is almost certain that FFW acted as an important safety net for FFW households. Employment opportunities during the six month crisis were even fewer than normal while at the same time food prices rapidly rose as a consequence of transportation problems (mainly a lack of fuel) and the blockage of goods from the port in Tamatave. During this time FFW was one of the few alternatives, if not the only one, that most poor households had for employment and for accessing food. The FFW activities provided an important safety net for 6 Azam, Jean-Paul. 2000. Inflation and Macroeconomic Instability in Madagascar. ARQADE and IDEI, University of Toulouse, and Institut Universitaire de France. June 2000. CARE Madagascar: Title II DAP Final Evaluation Page 11 Mahavita’s poorest households, and it may have resulted in significant asset retention for these households or prevented them from having to take out expensive consumption loans to meet their food needs. Qualitative evidence supports this hypothesis, as many of the FFW participants relate that the employment FFW generated was the only thing that ‘saved’ them during this stressful time period. Asset retention in this context was as important as asset accumulation, for if poor households had been forced to sell their meager assets to cope with economic shocks they would have become more vulnerable, and the recovery period of the poorest households could have been quite long. Training – Personal Empowerment and Livelihoods Training was a major activity of Mahavita and an important intervention for poor households. The opportunities to receive formal or non-formal training for adults of poor households are almost non-existent, save for projects like Mahavita. A total of 16,892 individuals benefited from personal empowerment training, which represents 169 percent of Mahavita’s original target and a significant percentage of the adult population in the thirty Fokotanys where Mahavita worked. Personal empowerment training and associated support activities were based on earlier CARE work in Zambia, and were developed to encourage the formation of mutual support and interest groups, and to assist these groups in identifying key livelihoods constraints of an economic and social nature (including gender issues) that inhibited households from engaging in productive activities. The types of activities envisioned included the formation of savings groups and targeted income-generation. Mahavita trainers worked with groups to elicit ideas and strategies for personal improvement, including skills training. Personal empowerment followed, focusing on enhancing people’s confidence and improving their interpersonal, business and decision-making skills to initiate activities. This included working with groups to commence savings activities, to improve existing income generating activities, or to identify and start-up more profitable new activities. It was found that for the poorest community members, improving their democratic voice addressed the issue of their exclusion from decision making processes, but in and of itself was insufficient to impact their employment prospects and other livelihood concerns. Before true gains could be achieved, the project had to build human capital for the poorest groups. Approximately forty percent of project participants have been able to upgrade their skills. For females, the most common income-related training was in canal and dike maintenance and rehabilitation. The training itself was in conjunction with FFW activities. A high percentage of FFW participants have never attended school (49 percent of FFW household heads, for example, have never attended school, compared with 20% of non FFW household heads and 27% of comparison households). Project staff as well as ex-FFW laborers commented that residents have taken greater initiatives to search for jobs within and outside the communities. Project personnel believe that personal empowerment training could be one of the most substantial sustainable effects of the project. However, as noted above, skills improvement has not necessarily transferred to income/asset enhancement due to a lack of employment opportunities CARE Madagascar: Title II DAP Final Evaluation Page 12 outside of FFW. It has, though, contributed to the personal empowerment of individuals, especially females, and provided new avenues of opportunities that perhaps can be pursued over time. Table 3 shows the types of skills training received by community members during the last five years that participated in the final survey. Obviously not all of these people were trained by Mahavita, but it is evident that for the FFW households the opportunities provided by Mahavita were significant (for example, almost 60 percent had received training in various aspects of infrastructure maintenance). These are, however, probably not the skill sets that will result in significant employment opportunities. A majority of laborers and training recipients have been women, enhancing short-term household food security by putting food commodities under the control of women. Women have also been employed in many of the management positions as foremen, timekeepers and commodity issuers, but few have been used in skilled labor positions such masonry. Savings Component The DAP design recognized household savings as key to achieving sustainable improvements in income and consumption, along with training and gender facilitation activities, and the strengthening of community based organizations which address rights and access issues. Savings was to focus on establishing and facilitating a voluntary savings program, modeled largely after CARE Zambia’s urban experiences and other successful savings programs. The savings component never evolved as a major activity in Mahavita. In the early phases of the project, under cash for work activities, there was a mandatory savings of 10% for cash-for-work participants. After their work tenure was completed they then received their savings in a lump￾sum payment. The activity was kept simple and participants were encouraged to use savings to invest in productive activities. For many the savings provided an important safety net, for others it was a cash ‘bonus’ they could use for consumption needs. When Mahavita was forced to start Table 3: Frequency and Percent of Household Members Trained in Various Fields FFW Non FFW Comparison Profession Frequency Valid % Frequency Valid % Frequency Valid % Total % Business 5 2 23 14 21 14 49 9 Industry 36 16 42 26 39 26 117 21 Agricultural Production 2 0.8 4 2 3 2 9 2 Artisanal Skills 18 8 17 10 16 11 51 9 Sewing, cooking, hair styling 24 10 37 23 36 24 97 18 Languages 4 2 12 7 8 5 24 4 Computers 2 0.8 10 6 14 9 26 5 Chauffeur, driver, etc 9 4 16 10 14 9 39 7 Maintenance of canals, dikes, etc. 131 57 2 1 1 1 134 25 Total 231 100 163 100 152 100 546 100 CARE Madagascar: Title II DAP Final Evaluation Page 13 using food instead of cash in their public works activities the savings component was dropped. There had been intentions to revamp it under FFW, but this was never completed. The savings component was, however, integrated into the personal empowerment training. Despite the lack of a focused strategy for promoting savings there is some evidence that the benefits of having savings (briefly covered in personal empowerment trainings) were understood by households. In the baseline survey 36 percent of households reported having some type of savings. This increased significantly in the final survey to about 50 percent of households. Only about 35 percent of those households that were not residing in a project Fokotany and did not receive Mahavita training had savings, which was not any different than the percentages reported in from the baseline survey. FFW participants, however, had a lower savings rate than non-FFW participants. It is also significant to note that over half of those with savings reported having to withdraw funds within the previous year, and this was due primarily to the political crisis that hit the country. FFW Activities A total of 16,892 households benefited from food for work activities during Mahavita, which was 169 percent of that planned. The reasons that the targets were exceeded were due, in part, to the political crisis that hit Madagascar in 2001, and CARE’s ability to leverage funding from other sources in responding to the crisis. Mahavita effectively and successfully selected the poorest and most vulnerable urban residents – one per household – as laborers to work on the food-for-work activities. Targeting success is supported by a number of indicators in the quantitative survey, such as assets, savings, and household expenditures (Figure 1). FFW laborers in those Fokotany’s visited by the evaluation team reported that FFW opportunities resulted in a significant short-term impact on household income, which translated into a positive short-term impact on household food security, providing a safety net for food insecure households for the period that they were in FFW. Residents interviewed during the evaluation agreed that the process for selection of FFW laborers was fair and transparent. Most of the workers heard about the job opportunity through their community Structures de Developpement or through ‘word of mouth.’ FFW had a positive impact on household diets. At the end of Mahavita, only 29 percent of general respondents thought that their diet was improving, but in FFW households, 40 percent felt that their diet was improving, with most noting an increase in meat and fish consumption or daily rice consumption and attributing this to their FFW participation. FFW laborers overwhelmingly expressed dissatisfaction with the content of the food basket, which consisted of maize, corn soy blend and oil. Nearly all FFW participants sold much of their maize to one local miller and used the proceeds to purchase rice. The transaction costs, however, were quite high. Normally they would sell the maize at about 1,000 FMG per kilogram and then immediately buy rice at between 1,800 and 2,000 FMG per kilogram. Although participants did not always understand how their pay rates were calculated, they appeared CARE Madagascar: Title II DAP Final Evaluation Page 14 satisfied that their pay was fair, based on work output, and others were not profiting from their work. FFW workers normally put up with no more than minor scraps and scrapes from stones that occasionally hit people’s legs, arm or even eyes. CARE took out insurance on FFW laborers in case of illness during their participation. Figure 1: Expenditure patterns of FFW and non-FFW households. CARE took considerable precautions to responsibly implement FFW activities. In general, FFW was not used until community development structures were organized well enough to understand its purpose, take some management role, and use it towards sustainable activities. By and large it was used as a short-term solution for long-term development e.g. for initial canal cleaning, while regular canal maintenance would be covered by a community fund. IV.A.2 Strategic Objective 2 Communities will be empowered to address constraints affecting their ability to control their own resources and livelihoods. Developing Community-based Structures It has only been in the last decade that Madagascar has progressed to the point where democratic processes are allowed to freely operate at the community level. While this history is brief, there is a long history of patrimonial rule characterized by tight controls over access to resources and services. CARE took time to introduce processes that were culturally foreign, or at least politically sensitive not that long ago. Mahavita was a project primarily about developing representative structures at the community level, with the aim of raising the voices of the disenfranchised and catalyzing democratic processes within poor urban communities so that development could happen from within. In this respect, Mahavita made significant progress in an operating environment largely untested in these processes. Expenditures (%) - FFW HHs 73% 5% 6% 4% 3% 5% 4% Food Educat ion Housing Clot hin Healt h Savings Ot her Expenditures (%) - General Household Sam ple 61% 9% 7% 4% 4% 9% 6% Food Educat ion Housing Clot hin Healt h Savings Ot her CARE Madagascar: Title II DAP Final Evaluation Page 15 Fokotany Selection CARE Madagascar did an excellent job of targeting needy Fokotanys in Antananarivo. Of the 30 Fokotanys covered by Mahavita, 55 percent were purely urban areas and 45 percent were mixed urban and rice production areas. TOUCH 2000 had worked in 10 Fokotanys and Mahavita continued to work in these areas and then added an additional twenty. The areas were chosen by Mahavita staff according to criteria that were developed collaboratively. The program began by first choosing three areas for pilot activities. Three was believed to be large enough to capture the diversity of the areas, while small enough to conduct a satisfactory study of the performance of program processes and procedures. Needs-based evidence suggests that the Fokotany’s selected by the project were indeed among the worst off. In terms of income, for example, quantitative surveys revealed that the thirty Fokotanys were among the poorest. Anthropometric data also showed that Mahavita fokotanys were worse off. For example, 49 percent of children under five are stunted in the project zone as opposed to 37 percent in comparison zones (other relatively poor Fokotanys that were not selected). Table 5 under Strategic Objective 4 provides further anthropometric evidence. Mahavita did not impose the project on any Fokotanys, opting instead to “invite” participation at the will of the existing local Fokotany administration. For the pilot phase Mahavita chose III G Hangar, Ankasina and Angarangarana because of their relative representativeness of the poor Fokotanys in general. III G Hangar is a completely urban area, and also the best performing Fokotany from the TOUCH 2000 project. Ankasina is a mixed urban and agricultural area and a medium performer of TOUCH 2000. Angarangarana is also a mixed urban and agricultural area, but it is new to the program. Empowering Structures de Developpement One of Mahavita’s objectives was to create effective community-based structures empowered to identify, plan, implement, coordinate and sustain activities that would contribute to the well￾being of residents. These Structures de Developpement (SD) would form the core of the community empowerment process and would complement the personal empowerment processes of the project. The SDs were developed using wholly transparent processes and were based on what became known as FAMOA. “Mahavita worked within the rhythm of the community.” No better complement can be paid to a project that put poor people and the communities they live in at the forefront of its development efforts. It is impossible to overlook the tremendous progress and transformations that have taken place within communities as a result of explicit community empowerment processes fostered and nurtured by CARE and community members. The benefits of the empowerment processes have been realized by both individuals and communities at large. For perhaps the first time in their lives, community members have taken explicit charge of their own development, and have been willing to explore the parameters of community-driven development in partnership with CARE. It cannot be overstated what this means and the potential it holds for future development in these poor urban communities. CARE Madagascar: Title II DAP Final Evaluation Page 16 In its first year, Mahavita worked with three Fokotany’s in a pilot phase that used participatory livelihood assessments as a basis for developing Representative Structures, the original label used for representative community-based organizations. Considerable freedom was allotted to Mahavita staff to explore how to apply participatory tools in engaging communities. Eventually the methods evolved into a process that Mahavita called FAMOA. This was a participatory planning process based on a partnership between communities and Mahavita and was used to facilitate the creation and strengthening of SDs7 throughout the thirty Fokotanys. FAMOA began with an environmental analysis led by the community. It was used to validate and complement information obtained during initial livelihoods analyses. A Social Analysis followed and with information collected through focus groups. Information gained during the environmental and social analyses of the Participatory Livelihood Assessment provided a deep understanding of the livelihoods of different categories of households, their relative levels of livelihood security, and the principal constraints and opportunities to improving their livelihood security status. To date, SDs have been developed in all thirty of the target Fokotanys. The SDs were to include representatives of every organization and association in the Fokotany, as well as represent all of the social classes identified in the social analysis. Members of the SD are voluntary, and often are headed by an influential community member such as the chef de quartier. Because Mahavita ‘followed the rhythm of the communities,’ there is considerable variation in the resulting SDs. There is no fixed membership, so in some SDs such as Ankor Andranomaheny there are seventy active members and thirteen bureau members and in Andavamamba Anjezika II there are twenty-one members. SD Activities and Management A typical SD spent about one year in the FAMOA process and another 12-18 months formulating a Plan de Quartier (PDQ). This plan represents the analysis by the community of their priorities for development, and is the centerpiece of the FAMOA process. The development of the PDQ is also the glue that has held the SDs together and provided a common goal for everybody involved. Many PDQs reflect the communities desires in environmental infrastructure improvements, such as canal cleaning and pathway constructions, but can also include activities in beautification, community mobilization around health issues, and recreation facilities. It is interesting to note that SDs have, in most cases, evolved to being an unofficial arm of the official Fokotany structure. Fokotany officials are appointed by government, but because of the overt activities of the SDs, and perhaps also because of their transparency of operation, the SDs are embraced by the Fokotany officials and used sometimes by officials for a variety of purposes such as accessing food-for-work activities or soliciting assistance from outside agencies. There is absolutely no doubt that the creation of SDs and the empowerment of community members is viewed as an extraordinary and positive benefit to community members. 7 The term Structures de Developpement, or SD, replaced the term Representative Structure after the pilot phase. CARE Madagascar: Title II DAP Final Evaluation Page 17 CARE has been very tactical in its work with Fokotany’s, and has established a rapport with community members unlike any other organization in Madagascar. Responses such as “CARE has been like a good brother” or “We have worked with other organizations and they do not treat us the same way as CARE” were common throughout the evaluation. There is a genuine respect between Mahavita staff and communities. What people point most often to, when asked about the benefits they have received from being involved with their SD, is a change in their own perspective about their community and their personal development. People tend to feel that they are part of an important process and that their efforts will eventually pay off in improved living conditions and opportunities. This outlook is supported by qualitative data gathered during the review and by quantitative data collected during the final survey. Table 4 shows how attitudes have changed with respect to how one’s opinions are perceived. In the baseline, just over 60 percent of community members felt their opinion was valued when it came to important community decisions. This increased to 83 percent for FFW participants and 75 percent for non-FFW participants. Table 4: Perceptions on the value of community member opinions. Your opinion is valued when it comes to important decisions to be made in the community. Household Type Baseline Final FFW Final non-FFW Comparison Agree entirely 60.5 83.5 74.4 70.5 Do not entirely agree 8.3 6.3 12.6 8.8 Disagree 26.4 6.8 9.7 15.9 No opinion 4.8 3.4 3.4 4.9 Figure 2 shows residents’ attitudes toward local infrastructure and perceptions on the quality of the local environment. Over 60 percent of FFW participants felt that infrastructure was improving, compared to about half of non-FFW participants and only 28 percent in non￾Mahavita Fokotanys. Similar trends were found related to perceptions of the local environment. Over 80 percent of FFW households felt the environment was cleaner. Almost 60 percent of non-FFW members felt the environment was cleaner compared to slightly less than 40 percent if comparison households. These data, along with other evidence in the quantitative report, strongly suggest that the work of SDs and infrastructure development are having a positive impact in the community, and that participation in activities such as FFW promote more positive outlooks by those participating. CARE Madagascar: Title II DAP Final Evaluation Page 18 Perceptions on Neighborhood Environment 0 20 40 60 80 100 Cleaner No Changes Dirtier No Opinion % of HHs FFW HHs non-FFW HHs Comparison Opinions on the Status of Local Infrastructure 0 10 20 30 40 50 60 70 Improving Staying the same Declining % of HHs FFW HHs non-FFW HHs Comparison HHs Figure 2: Resident Perceptions of Infrastructure and Environment IV.A.3 Strategic Objective 3 Communities will create and maintain a healthy and hygienic environment within which inhabitants can carry out productive livelihoods. Infrastructure Development Infrastructure provision started relatively late in the program, being preceded by several years of community organizing and development of the PDQs, which are a prerequisite to starting a community construction activity. Infrastructure is separated into “income generating” and “HIMO” or FFW. By far the greatest number of constructions have been in the HIMO, not income generating, category. Mahavita never intended to solve the infrastructure needs of the 30 FKT, but to make reasonable headway in addressing these needs. Canal drainage and construction of pathways and “diguettes” (built-up walkways) have been the main focus of HIMO infrastructure improvement. About 60 kilometers of these pathways have been built, and there is no doubt that this reflects the priority of residents in the majority of the Fokotany’s where Mahavita works – most of which are located in the low-lying floodplain of Antananarivo. Every year the rainy season flooding causes extreme hardship for residents - relocation, resorting to travel via pirogue, exposed to highly contaminated water. Also, this type of construction is an ideal FFW activity. There is clear evidence of improved, cleaner neighborhoods thanks to this operation. Canals that used to be stagnant swamps now flow, with clean pathways linking different settlements. One woman expressed her delight at finally not having to move to the Catholic church every year during the rainy season. Much of the activity, however, has taken place since the last rainy season. The true “livability” of the improved neighborhoods will be tested in the next few months when the rains really start. There were other potential benefits to agriculture accrued by improving canal infrastructure and retention basins. Improved water management should have had a positive impact on agricultural production by improving water flow through canals, preventing prolonged or unplanned CARE Madagascar: Title II DAP Final Evaluation Page 19 inundation, and improving drainage. However, there were no systematic efforts to assess any changes to agricultural production systems. Overall the quality of construction in Mahavita has been high. There have been some isolated problems with quality, such as inferior soil compaction on some dikes and walkways that may cause problems after heavy rains. There are also isolated cases of stagnant water collecting around wash basins and latrines due to poor leveling. These were pointed out to engineering staff and were to be corrected. Water/Sanitation/Washbasins The “income-generating” category of infrastructure consists mostly of “blocs sanitaires,” a public toilet/shower facility usually with 2 of each, a caretaker who collects fees established by the management committee, improved public water taps, and public “basins lavoirs” or laundry facilities. These are all constructed in cooperation with AGETIPA, a government public works agency financed by the World Bank. A total of 37 have been built in 18 (of 30) neighborhoods, meeting construction targets. Of these 26 are functional, and 11 are waiting for water and electricity to be turned on. AGETIPA constructions are generally of high quality, the blocs sanitaires almost startling compared to the poorly built houses of the residents. The only exception is some problems with uneven land leveling around the structures, resulting in some collection of stagnant water. The investment per toilet seat seems very high compared to the need of the community for facilities. Various people estimated that over half the residents in the FKT visited did not have access to latrines or toilets, and small children rarely use any. Both Mahavita technicians and the SD selected the sites. One bloc was within 5 visible household latrines, so apparently only passers-by use it. Blocs sanitaires are the only sanitation or excreta disposal facility improvements carried out by Mahavita, and they were limited in number. Due to the participatory nature of Mahavita, whereby communities developed their own infrastructure priorities, no clear targets were set for sanitation improvements. No household or family-level excreta disposal facilities were built during DAP I despite the fact that only 15-17 percent of households have latrines in the poor neighborhoods, the majority of which are shared with other houses and 90% are outside. Some people have built their own latrines and charge a small fee for others to use them. A family latrine model adapted to the locale was developed and proposed to AGETIPA, but in the end they only built the official Commune model. A sanitation campaign exhorting each family to build a household latrine did not work. During walkabouts in the neighborhoods, latrines were pointed out (although one neighborhood claimed not to have any). In general, the superstructures were of very poor quality, dilapidated, some with large cracks in the slats so the person inside is visible, and often locked. The critical problem mentioned by ADCs and residents alike is the high water table of the zone, making it a technical challenge to dig pit latrines. The AGETIPA blocs sanitaires were constructed above ground to avoid flooding. A review of all PDQs developed by the FKT showed that nearly all CARE Madagascar: Title II DAP Final Evaluation Page 20 proposed to build public “blocs sanitaires” but not family latrines or WCs, indicating either that these are not considered feasible or are not within the SD’s mandate. Water supply is still an issue in these zones, in spite of several new or improved water taps. Over 93 percent of households collect their water from outside their household. This necessitates that public taps be accessible and operational. In one of the newer FKT where Mahavita activities are barely a year old, the evaluation team was told that one unrehabilitated tap served 5,000 people, and only 20 percent of households have access to piped water. Even improved water taps have restricted hours of operation, although at the newer taps no lines were observed and people came at will to collect water during operating hours. All water collection observed was done by buckets, and both the baseline and final surveys showed that about 95 percent of water collection is by bucket (seau). Household water storage was not possible to see, but the follow up to the baseline survey claims that the majority of households (about 72%) use the same receptacle for storing both drinking and cleaning water, while about 27 percent use different containers for storage. In general, water quality in Tana has improved since the cholera scare of 2002. Sur Eau, the household chlorination product promoted by CARE and others, appears to be widely available but not widely affordable in the Mahavita FKT. Almost 98 percent of the respondents in the final survey had heard about sur eau as a product for treating water, which was up from 82 percent reported in the baseline survey. The primary source of information on sur eau was radio broadcasts, but many households also learned about the product and procedures from local organizers (animateur fokontany) and television. Conversations during community visits revealed that the use of sur eau is very low and associated with prevention of cholera, which is not a current threat. The final survey found only 15.2 percent of households making use of sur eau as a water purification method. This is a slight increase from the 13.7 percent of households using the product during the baseline survey. People commented that sur eau was too costly for everyday use and that the water from the taps is clean enough for drinking. However, people also said that they were able to stem the cholera epidemic thanks to Sur Eau and are convinced of its efficacy against the disease. Despite the relatively low usage of sur eau in Mahavita households, it should be noted that it was the initial sur eau activities conducted under the DAP (initiated by CARE in partnership with PSI and CDC) in Antananarivo during the cholera crisis that led to the development of the sur eau model which is now national. Should future cholera or other water-borne disease epidemics emerge in Antananarivo the efforts of CARE under this DAP will likely be noted by surges in the use of sur eau for water treatment. Hygiene Promotion Much has already been said about the remarkable community empowering process carried out by Mahavita. Combined with a thoughtful IEC strategy, results in terms of overall community cleanliness are visible. IEC on health, nutrition, water/sanitation, hygiene, and reproductive health is woven into every activity, and most of all into the HIMO/FFW activity where the majority of workers are women, direct channels to the poorest households. CARE Madagascar: Title II DAP Final Evaluation Page 21 Mahavita’s impact in community-level hygiene improvement is perhaps most evident in the difference between “old” and “new” FKT. Older FKT - where there has been a CARE presence for up to 7 years - appear overall cleaner, with less garbage strewn about, than newer ones where CARE has been working for a year or so. This may be due to a variety of factors (such as the current “clean FKT” IEC campaign and competition), but casual conversation with residents revealed a consciousness of responsibility for the hygiene of the community due to CARE interventions through the SD. One effective hygiene promotion strategy is the training of community-based “animateurs” and “animatrices”, outreach workers who go door-to-door for hygiene promotion and counseling. They take their job seriously and are taken seriously by the residents. Knowledge of good hygiene behavior is very high in the program FKT. Themes, messages and graphics have been well-researched and tested. However, several people mentioned a disconnect between knowledge and practice. Several reasons were given for this. One, the Mahavita IEC strategy for behavior change has tended to focus on themes (i.e. “Nutrition”) rather than on specific behaviors. This was relatively recently identified as a problem within the IEC unit, and the Clean FKT Campaign was developed. It lasted for several months, and culminated in a festival community festival with prizes. A longer-term strategy for addressing non-adoption of hygiene behaviors does not yet exist. A second reason mentioned is poverty. People in the Mahavita zones are so desperately poor that they do not consider behavior change important. Another obstacle to adopting improved hygiene behaviors cited by interviewees has been the frequent problem of the lack of availability of the necessary hardware to carry out the promoted behavior. For example, the pervasive lack of latrines is a clear obstacle to safe disposal of feces. Hygiene promotion continued even when it was clearly not feasible for certain communities to do what was recommended. The problem of community garbage should be mentioned here, as it is an important contributing factor to disease (plague) in Antananarivo. It is a problem far from solved, although advances have been made, especially since the Commune instituted garbage collection from its “bacs”. The FFW efforts have undoubtedly resulted in an increased awareness of garbage as a community problem and a community responsibility. However, a number of people mentioned the problem of the “bacs” being located far from the center of a FKT (near an accessible road, for instance) and the unwillingness of residents to carting their household waste to the “bac”. Much of it thus still ends in the street. Social marketing has centered on promoting the use of Sur Eau. As mentioned before, it was an effective program and mitigated the cholera outbreak, but is not consistently applied. Cost was mentioned as a factor. It did, however, permit Mahavita and others to gain a better understanding of beliefs and behavior around water use. Enabling Environment The solid community foundation created by Mahavita through the FAMOA process and embodied by the Structures de Developpement and their various committees has been discussed CARE Madagascar: Title II DAP Final Evaluation Page 22 and appreciated throughout the evaluation. It has also been pointed out that there is uncertainty among the 30 SD as to how to implement the community projects in their PDQ, and many of these projects are directly linked to the environmental health of the community. The SDs visited were among the most mature, and seemed nearly - but not quite - ready for handling the negotiations needed with the Commune and ministries with whom the PDQ would be executed. It’s doubtful that the SDs with less maturity will be able to manage without continuing capacity building from CARE. The enabling environment for environmental health at the community level should relate to the management of water/sanitation and related systems (basins lavoirs) to assure availability and quality of services. Mahavita’s process for building SD capacity to manage cost recovery fits into its overall community empowerment strategy and involves the creation and training of a management committee who conducts a willingness to pay study of their own community and establishes user fees based on the results. This system appeared to be functioning well, with carefully filled-in ledgers and in a few cases, filled coffers with the prospect of cash on hand for the SD. It also bears mentioning that nine community groups received municipal contracts to clean canals after having cleaned their own. Another enabling factor for improved EH is Mahavita itself and its capacity to assure the availability of required materials and personnel for carrying out a program of infrastructure improvement and capacity building. It appears as though the separate Mahavita sectors responsible for the technical aspects of infrastructure improvement and for community mobilization strategies had some gaps in coordination, resulting in occasional communities mobilized and ready for work, but supplies and materials undelivered. In spite of good faith efforts throughout the Mahavita program to coordinate and collaborate with the Commune of Antananarivo and technical agencies responsible for service provision in the FKT, this proved to be extremely difficult and frustrating. The result is that much of Mahavita’s activities were done independently. The future of the relationship between the SDs and the Commune, and the need to advocate during the next DAP for official recognition by the municipality of the SDs is the subject of the paper on future directions for Mahavita. This is especially critical for assuring the environmental and other health of the residents of the FKT since in a dense urban environment, especially in a flood prone zone crisscrossed by drainage canals, a coordinated effort for environmental improvements is required to assure coverage. Fecal contamination and rats don’t recognize a FKT boundary. CARE Madagascar: Title II DAP Final Evaluation Page 23 IV.A.4 Strategic Objective 4 Households practice healthy behavior and hygiene. Health Sector Mahavita’s health unit focuses its activities mainly on taking care of the health of the FFW workers, support to the IEC division in the development of technically accurate health messages, and is launching a pilot HIV/AIDS prevention activity. It has virtually no connection with the official health sector even though the program has its roots in child health under TOUCH 2000. Meetings with several MOH representatives at various levels were warm and interviewees expressed a strong desire to renew ties with CARE in its urban work. Reasons for the separation between CARE/Mahavita and the health sector are numerous and mainly involve policies and personalities from the previous GOM administration, as well as a conscious move by Mahavita to focus on environmental health as the critical priority of the target FKT and move away from direct health interventions. The MOH officials interviewed all felt that CARE’s strength in community mobilization and the current MOH priority strategies were a good fit, especially in poor urban areas where health service needs are great and success of some MOH interventions depends on community activism. Immunization and TB treatment were cited as examples. Despite the lack of focused, direct health interventions there is information from the monitoring component of Mahavita related to health practices. Illness is common and about half of all households will have had at least one family member ill within the previous two weeks. Of those households with illnesses about one-third are seeking medical treatment. Table 5 provides a breakdown of the different illnesses experienced by households. The most common malady is acute respiratory illness (ARI), accounting for one-third of reported illnesses, followed by localized aches, and flu. Table 5: Incidences of selected illnesses from the Mahavita final survey. Flu Diarrhea ARI Aches Vanishing Skin Other n % n % n % n % n % n % n % General HHs 84 21.7 41 10.6 128 33.1 99 25.5 4 1.0 6 1.6 25 6.5 FFW HHs 29 23.0 22 17.6 40 31.7 24 19.0 1 0.8 1 0.8 9 7.1 Non-FFW HHs 28 21.8 12 9.4 43 33.7 34 26.6 1 0.8 3 2.3 7 5.5 Comparison HHs 27 20.3 7 5.3 45 33.8 41 32.0 2 1.5 2 1.5 8 6.8 In general, knowledge of appropriate health behaviors is high, even among the urban poor. For example, over 85 percent of households with children under two years of age have had their children vaccinated. Knowledge of diarrheal causes is very high, as well as treatment. Malnutrition As shown in Table 6, height for age measurements of children below five years of age suggest serious chronic nutrition problems continue in Antananarivo’s Fokotany’s, resulting in high rates of stunting. Children from Mahavita FFW households had the highest rates of stunting at over CARE Madagascar: Title II DAP Final Evaluation Page 24 55 percent, while non-FFW households were not significantly different, with stunting rates at 48 percent. Comparison households had the lowest stunting rates of 39 percent. These rates are, however, significantly better than those found during the beginning of Mahavita. During the baseline survey (Table 6), almost 64 percent of the urban population of children below five years of age was found to be stunted (Z = < -2.0 standard deviations). No causality was sought in the baseline survey, but its results combined with other studies in Madagascar suggested that diet diversity and general health status of children were two main contributors to stunting. The stunting rates found during the final survey are still high, especially for children residing in the project area. Table 6: Anthropometric data for children under 5. HH Type Measurement n % Below –3 std deviation % Below –2 std deviations Mean Height for Age 151 22.5 55.6 -2.111 FFW Weight for Height 151 0.6 7.3 -0.728 Weight for Age 151 10.6 47.7 -1.842 Height for Age 103 22.3 48.5 -2.067 Non FFW Weight for Height 103 1.0 5.0 -0.511 Weight for Age 103 3.2 36.9 -1.702 Height for Age 89 12.4 39.3 -1.783 Comparison Weight for Height 89 1.1 9.0 -0.842 Weight for Age 89 10.1 40.5 -1.754 Height for Age 186 29.6 63.6 -2.363 Baseline Weight for Height 186 0.5 2.0 -0.072 Weight for Age 186 3.8 35.7 -1.523 CARE Madagascar: Title II DAP Final Evaluation Page 25 IV.B CYPREP (Cyclone Preparation on the East Coast) Special Strategic Objective: To safeguard the livelihood security of Madagascar’s most cyclone-vulnerable communities by enhancing local capacity to plan for and cope with cyclone emergencies. The evaluation of the CYPREP component was conducted against its three intermediate results and including the March 2001 readjustment. For each IR multiple strategic orientations were developed - four were designed to contribute to the achievement of IR1, two to IR2, and one to IR3. The various evaluation themes are organized and presented below with the appropriate strategic orientations. IV.B.1 Intermediate Result 1: Reduced vulnerability in 132 communities on the East Coast of Madagascar. The project met its expectations for this result, working in 133 communities out of a planned 132. The intervention sites were divided as shown in Table 7. Table 7: Geographic location of CYPREP intervention sites Zone Sub-prefecture / District Number of sites % Total by zone % Southern Mahanoro 25 18.8 25 18.8 Northern Fénérive-Est 25 18.8 60 45.1 Vavatenina 17 12.8 Soanierana Ivongo 11 8.3 Tamatave II 7 5.3 Central Tamatave II 24 18.0 48 36.1 Brickaville 24 18.0 TOTAL 133 100.0 133 100.0 In terms of location by zone, the northern zone had the most sites (45.1%), the southern zone had 36.1%, and the southern zone had 18.8% of the intervention sites. Strategic Orientation 1: Involve local authorities throughout the process of site selection. From the outset CYPREP made strong attempts to involve local authorities in the community selection process. Several steps were used to selection sites for the CYPREP component. First, a workshop was held bringing together the sub-prefect, communal authorities and mayors, and project staff. The objectives and strategies of CYPREP were discussed and future plans were developed. Next, mini-workshops were held in each commune selected, following a letter of interest received from communal authorities. In addition to CYPREP orientation, these workshops incorporated an overview of the livelihood and disaster preparedness situation in each village. Village representatives (PCLS – fokontany, representatives of the fokonolona, and village elders) presented the situation in their respective fokontany, and at the end of the meeting the assembly established a prioritized list of potential intervention sites to guide future processes. CARE Madagascar: Title II DAP Final Evaluation Page 26 Finally, the project field agents (AMS) conducted a preliminary site study of likely villages, using the prioritized list prepared by each assembly as a guide. This included collecting baseline information to supplement the presentations prepared by the village representatives, with a special emphasis on evaluating the situation of the most vulnerable elements of the population. Using participatory processes resulted in the following criteria used to select intervention sites: - Dynamism of the community and the local leaders (PCLS, village elders); - Accessibility of the site - in general sites that were not more than 2 hours walk by foot from the nearest motorcycle access point were given higher priority; - Population: fokontanys of more than 1,000 residents were given priority; - Willingness of the community to participate in the planning, management and execution of cyclone preparedness and mitigation activities. The selection process allowed for the active participation of local authorities in the choice of sites, despite some evidence of subjective bias by certain elected officials. From the point of view of the local authorities the site selection process, as well as the criteria used, was appropriate and well adapted to the local context. However, the use of accessibility as criteria handicapped certain vulnerable communities located in more remote areas, despite their high level of motivation and interest. According to certain local leaders, the site selection process also allowed each commune to choose villages that were representative of the situation in the commune as a whole. According to some of the project staff, the project should have also considered the specific conditions prevailing in each zone, notably the access/transport situation, to make the eventual implementation of the project more feasible. The manifestation of interest in the form of a written letter on the part of the community was a very pertinent indicator in the selection process. The system used to present the situation in each fokontany resulted in the development of knowledge, motivation, and capacity for the village representatives. The important role played by the local authorities in promoting and facilitating the project can be linked to the fact that site selection was not a unilateral decision made by project staff, but rather a joint decision between project staff, the local authorities, and the community representatives; CYPREP also had a requirement that each community provide a 20 percent contribution on all mitigation activities with the exception of road rehabilitation. It required communities to collect resources from all households over time. This stipulation may have intimidated certain communities in the beginning, especially in areas where there had recently been cyclones (as was the case for portions of Mahanoro and Brickaville), and it may have excluded the most vulnerable communities that were too poor to meet the criteria. In addition to site selection, local authorities, administrative as well as traditional, were involved in the prioritization of mitigation activities and in determining the mix of community funds and in-kind contributions to be collected. The community members interviewed in the course of the evaluation stated that they were satisfied with this process. Local authorities also participated in CARE Madagascar: Title II DAP Final Evaluation Page 27 the Conseil Villageois de Secours (CVS), the deciding body for planning, analysis, and decision￾making for the implementation of development activities in the village. Strategic Orientation 2: Use the household livelihood security analysis approach (HLS) to design development activities to prevent and mitigate the adverse effects of cyclones on target communities; reinforce the capacity of local communities to identify, plan and design development activities. The FALAFA Process (HLS) Between 1999 and 2000, the project concentrated its efforts on resource inventories and risk evaluations, with a goal of developing cyclone preparedness plans (PPC). This effort was somewhat ad hoc and lacked a strong conceptual framework to guide information collection and analysis. After the strategic reorientation in 2001, Mahavita staff helped CYPREP adopt the FALAFA process to better facilitate the development of village development plans (PDF) in the intervention sites. This was a conceptual framework based on a Household Livelihood Security (HLS) approach and adapted to the local context. Because the two approaches (risk analysis leading to a cyclone preparedness plan and HLS analysis leading to a development plan) were introduced at different points in the evolution of the project, the field agents tended to view them as separate processes, even in sites where both the PDF and the PPC were completed after 2001. The process permitted the following results to occur: - Mitigate the vulnerability of the community; - Give the community a better understanding of cyclones and their effects; - Identify and choose the mitigation activities best corresponding to the situation and capacity of each community; - Invent / conceive new mitigation strategies based on local traditions and resources; - Facilitate the creation of a local body / committee charged with emergency preparedness and relief; - Develop a more comprehensive view of the community; - Measure the degree of motivation and vulnerability in the community; - Increase the motivation and dynamism of communities; - Bring a new spirit of openness and participatory decision-making; - Simplify the identification of problems and risks; - Identify the root causes and links between problems, thus giving the community the possibility to make more informed choices about the solutions to implement; - Put in place a profound and evolving system of community analysis. For the local authorities, the process allowed them to clarify and better identify the needs of the community, especially as related to development. However, the process was sometimes too long for the community, and often due to time constraints different community members would participate in different sessions, sometimes creating a discontinuity in the analysis. CARE Madagascar: Title II DAP Final Evaluation Page 28 The benefits of FALAFA as an information collection and analysis approach were significant. At the top of the list would be the level of community involvement in reflecting on their own livelihoods and its relation to disaster management. This in itself led to improved communication within the villages. The creation of a forum for discussion and reflection (CVS) created a nexus for catalyzing action in the community and improving the social stature and knowledge of CVS members. An increase in participation of women in the development process was also noted, as was improvement in analytic ability and technical skills, especially among younger CVS members. The processes used also resulted in better knowledge of the level of vulnerability of each community, and permitted community members, especially the CVS, to identify the different roadblocks to development in their community and to devise appropriate solutions. Many local authorities noted their satisfaction with the benefits derived from using the FALAFA process and they would like to see it extended to other villages so that they, too, can play a more active role in local development. Fokontany Development Plans (PDF) CYPREP facilitated the development of village development plans (PDF) in each community. The PDF is a summary document that includes an analysis of problems, resources, and opportunities and the proposed solutions and actions to be undertaken for the development of the community. The plan is developed by the community with the assistance of one or more project field agents) using the FALAFA framework. Out of a total of 133 sites where the project worked 89 communities (67 percent) had developed a PDF by the time of the evaluation. The development of a PDF in each site was one of the most interesting and beneficial results of the CYPREP project, both from the point of view of project staff as well as the communities themselves and the communes. The PDF represents an indispensable tool for all development actions to be undertaken in the community, irregardless of subject matter. In general, the CVSs and community members seem satisfied with their PDFs and the processes used to develop them. Many expressed appreciation of the opportunity to participate and the knowledge they gained. Most also feel that the activities outlined in the PDF are realistic and respond directly to the needs identified by their community. Results from the quantitative survey suggest that most community members are aware of the major themes of their PDF and are satisfied with its contents. All those interviewed were unanimous in stating that the PDF is an indispensable tool for the development of the community, which is indicative of a sense of ownership on the part of the local authorities, the members of the CVS, and the community at large. Some communities have already used their PDF to either create or revise PCDs (commune development plans). The existence of the PDFs helps the elected officials at the commune level to identify the needs and wishes of the various fokontany (villages) and to assist them in advocating for the fulfillment of them; Many of the early sites in Fénérive Est, which were targeted by CYPREP between 1999 and 2000, do not have PDFs. However, the abbreviated risk analysis conducted there has permitted CARE Madagascar: Title II DAP Final Evaluation Page 29 many of these communities to proceed to an analysis of problems and risks in the context of emergency preparedness. In the final survey, 54% of the households surveyed in the intervention sites were aware of the existence and general contents of the PDF (almost all the households in the sites where a PDF was completed). This compares with households surveyed in non-intervention sites, where most were unaware of the existence of PDFs or confused with the PCDs (commune level development plans). Cyclone Preparedness Plans (PPCs) The cyclone preparedness plan (PPC) is conceived by the community with the assistance of a project field agent. Based on the local disaster context, its aim is the attenuation of the negative impacts of future cyclones. The PPC contains a list of the actions to be undertaken before, during and after the passage of a cyclone. Of the 133 CYPREP intervention sites, all but one has completed a PPC. Just over 60 percent of the households surveyed were aware of the PPC, compared with 5 percent of the households surveyed in non-intervention sites (whose communities do not have PPCs). The creation of the PPCs in Malagasy appeared to foster a heightened sense of responsibility on the part of community members, reinforced by the creation of sector commissions and the training of the members of the CVS. Many of the households surveyed in intervention villages were also able to recite the contents of the PPC. As a result of CYPREP activities it appears that communities are better prepared to respond to cyclone events. Some examples of important behavior changes includes the use of cyclone flags, warnings broadcast by the dalala8 and goka9 and understood by community members (other types of morse-style conventions are also known and understood by the community); protection of food stocks and clean water in safe locations; reinforcement of buildings, removal of tree branches likely to fall on buildings, etc. In general, there is much more understanding by community members concerning the impact of cyclones and the action needed to mitigate them. In terms of knowledge of households about alert systems, signal systems, and weather alerts, the intervention villages consistently showed higher knowledge compared to non-intervention sites. The difference in level of knowledge ranged from a maximum of 91 percentage points on one indicator to 7 percentage points on the lowest. Three-quarters of the population in the intervention villages (77 percent of households surveyed) know the alert system used in their village, compared with only 10 percent in non-intervention sites). A similar proportion was aware of a coordinating body (i.e., the CVS) that is responsible for cyclone mitigation activities. Almost all the households surveyed in the intervention villages (90%) stated that they had learned about the alert systems from the members of the CVS. Knowledge of the cyclone flag was shown by about three-quarters of the population in intervention villages (77%), compared 8 Dalala: vocal calls 9 Goka: conch shell used as a horn CARE Madagascar: Title II DAP Final Evaluation Page 30 with 28% in non-intervention sites. The difference in knowledge between the intervention and non-intervention sites averaged 74 percentage points in Mahanoro. In the other districts the distinction was less pronounced with an average difference of 44 to 45 percentage points for all indicators. This could be explained in several ways, including the recent passage of cyclones in Mahanoro, allowing the intervention communities ‘real-life’ practice of their new skills and knowledge. Also, project interventions in Mahanoro began later than in other zones, meaning that, on the average, less time had elapsed between the initial training of CVS members and the evaluation. Finally, the baseline level of the non-intervention communities was lower in Mahanoro, perhaps due to the paucity of other development actors in that district. The project produced a surprising level of spread effect in non-intervention villages. With respect to the cyclone flag, one alert system known by a significant percentage of respondents (28%) in non-intervention villages, 56 percent had heard of it from CVS members from a CYPREP site while 40 percent heard about it from the radio. The IEC activities undertaken by the project, through the intermediary of the CVS, had a positive impact, as almost every household in the intervention villages had undertaken cyclone preparation measures outlined in the PPC. From this, one can conclude that the vulnerability of these households has been reduced. As an example, according to the communities of Vohiboazo and Andovoranto, they were able to avoid loss of life from Cylone Manou (May 2003), whereas previous cyclones had always resulted in at least one death. According to the mayor of the commune of Tamatave Sub-Urbaine, some villages, which didn’t work with CYPREP, have also developed PPC (e.g., Vohibolo). This was a product of exchanges between neighboring villages and the CVS members trained by the project. The post-cyclone evaluation of damage and loss (filling out of EIMA forms and sending them to the CCS or CLS) was completed by all CVSs in the CYPREP intervention sites within 48 hours, whereas most non-CYPREP villages had not been able to complete the evaluation in under a week. Besides their timeliness, the information provided by the CVSs was deemed reliable and useful for planning the emergency interventions of the CNS. Strategic Orientation 3: Design cyclone mitigation activities based on HLS analysis and ensure the sustainability of these actions by reinforcing the management capacity of local communities. Mitigation Activities The mitigation activities initiated with the assistance of CYPREP contribute to reducing the negative impacts of cyclones. Major mitigation activities are those that can not be entirely auto￾financed by the community themselves. The actions that can be undertaken by the community, without outside assistance, are auto-mitigation activities: reinforcement of buildings, construction of family latrines, trimming of overhanging tree limbs, etc… The major mitigation activities conducted over the course of the project included: rehabilitation of rural roads, cyclone-resistance community granaries, and cyclone shelters, rehabilitation of CARE Madagascar: Title II DAP Final Evaluation Page 31 drainage and irrigation systems, potable water points, motorized pirogues, and reforestation of watersheds. The levels of effort of these different activities are provided in Table 8. Table 8: CYPREP results with respect to major mitigation activities. Rehabilitation Reforestation(nb) Road Irrigation and Drainage Canals surface Shelter Granary Cloves Litchi Cinnamon Eucalyp Total Boat Potable Water Zone Point (km) (km) (ha) nmbr (m²) nmbr capacity (t) nmbr nmbr nmbr nmbr nmbr nmbr nmbr North 182 0 0 25 1 388 23 93 19 018 5 810 2 746 58 867 86 441 2 14 South 65 26 610 2 0 4 0 7 158 200 0 0 7 358 01 Center 28 3 40 1 40 0 0 0 0 0 0 0 01 TOTAL 274 29 650 28 1 428 27 93 26 176 6 010 2 746 58 867 93 799 2 16 Out of a total of 274 kilometers of rural road rehabilitated, two-thirds were located in the northern zone, 24% in the southern zone, and 10% in the central zone. In terms of reforestation, out of a total of 93,799 saplings, 92 percent were planted in the Fénérive-Est zone and only 8 percent in Mahanoro. Tamatave did not have any reforestation interventions. IV.B.2 Intermediate Result 2: Creating a network of local partners for disaster response. Strategic Orientation 4: Development of a monitoring system to track CYPREP and provide information to the Network of Rural Sentinel Sites and livelihoods analysis. Strategic Orientation 5 was designed to provide timely information to CYPREP staff on the progress of activities and to apply the results of livelihood inquiries into activity programming. The monitoring system, however, was not fully functional until five months before the end of the project, which obviously impeded its impact on activities and programming. The monitoring system was based on periodic reports from each of the three zones. However, there was not a formal monitoring and evaluation unit within CYPREP, and no one individual was charged with collecting, analyzing and reporting on information. Rather, it was the responsibility of all project staff to report on and apply information. The Senior Management Team in Antananarivo had a difficult time following project progress due to the lack of systematization in the monitoring system, and often decisions were made based on less-than￾optimal information. For all it was difficult to apply real-time lessons learned because of the status of the monitoring system. The monitoring system was also intended to feed information into the Network of Rural Sentinel Sites and to assess and affirm CYPREP’s understanding of livelihoods. Its tardiness highlighted above prevented this from being realized. CARE Madagascar: Title II DAP Final Evaluation Page 32 Strategic Orientation 5: Development of more representative community organizations capable of coordinating and managing their development activities. A total of 131 Village Security Committees (CVS) worked with CYPREP, corresponding to a total population of 190,000 inhabitants. Only one site in Mahanoro and another site in Fénérive￾Est do not currently have a CVS or any mitigation activities as a result of CYPREP’s work. Table 9 summarizes CVSs by zone. Table 9: Existence of CVSs by zone. Zone Number of CVS Number of CVS members Total population Fénérive-Est 60 1,534 88,431 Mahanoro 25 854 38,686 Tamatave 48 1,536 62,866 Total 133 3,924 189,983 The CVS is a village-level coordinating body charged with overseeing the planning and implementation of development activities and preparing for and mitigating against natural disasters. Officials from different administrative levels (sous-préfet, mayoral, chef de quartier) as well as community representatives unanimously agree on the usefulness of the CVS for disaster-related matters as well as for community development. The positive benefits that have been noted by participants and as a result of CARE’s establishing and supporting CVSs are many. First and foremost, it is the only community-based structure on the east coast that supports local development based on community representation and participation. CVSs have evolved as the community mobilization force around which shared values and visions are being discussed and acted upon. They have played an important role in making community members aware of their roles and responsibilities related to cyclone preparation and response, and in changing behavior of community members with respect to cyclone preparation. As a result, the arrival of a cyclone is no longer unforeseen by communities and the risks to the general population have been diminished. The CVS also benefits the commune and the CCS by providing a direct linkage to communities, households and individuals. The CVS is known and recognized by approximately 75 percent of the population residing in the CYPREP intervention zone. The activities most well known by community members that are the responsibility of the CVS include cyclone warnings and communications of impending events (recognized as a CVS function by 73 percent of residents polled), the construction of disaster￾related infrastructure (33 %) and livelihoods/asset protection (31 %). After the passage of a cyclone, the CVS is responsible for promoting health messages to communities to prevent illnesses to assure safety of the population. This responsibility is recognized by 34 percent of residents. Fewer recognize the CVS as being responsible for CARE Madagascar: Title II DAP Final Evaluation Page 33 reconstruction (20 %) and other health-related activities (19 %). The vast majority of these activities are not known in villages where CYPREP has not been active and there is no village￾level structure for disaster management. Accomplishment of CVS Tasks The high motivation of CVS members is one of the key factors in making it an important development and disaster-related community body. The In the majority of cases there are no major problems in its functioning. The sole exception noted during the evaluation was the rare event where certain individuals could not accept consensus-based decisions and the general functioning of representative structures. Despite the overall success that has been achieved with regard to CVSs there remains work to be done. The CVS could benefit from becoming a recognized legal structure whose responsibilities are to provide preparedness and mitigation measures related to disaster management and tom promote community-based development. Indeed, all evidence points to endorsing the CVS as legal structure so that it is justly recognized by authorities at all levels. More active engagement in disaster planning by local authorities would ensure better linkages to communities and well as create a more dynamic and integrated disaster preparedness and response capability throughout the east coast of Madagascar. It would also reinforce the motivation of CVS members. One other factor that has helped to keep CVS motivation high is the use of small ‘rewards’ to CVS members such as CVS hats, training certificates, t-shirts, etc. IV.B.3 Intermediate Result 3: Re-enforce the capacity of the National Emergency Preparedness and Management Agency (CNS) to promote the use of preparedness planning and disaster response measures at the local level. The Conseil National de Secours (CNS), the government’s agency responsible for planning and coordinating disaster relief interventions, is part of the Ministry of the Interior. Its historical role in disaster management has been limited due to a lack of resources. In most cases, its structures exist only in the form of lists of government officials who are responsible for responding to an emergency on behalf of their technical Ministry. Both UNICEF and UNDP have worked directly with CNS at the national and regional levels. UNICEF’s strategy in particular has aimed at training regional representatives about their roles and responsibilities in case of emergencies. These efforts are crucial, however there are only six regions in Madagascar, and it has long been evident that regional CNS representatives, even if they are motivated and clear on their purpose, cannot single-handedly manage disaster preparedness and response for the whole of their populations without some assistance. CYPREP’s aim was to fill the gap between the CNS as a coordinating body and communities, which are ultimately the victims of disasters. Until CYPREP, no planned action for disaster management had been undertaken at the village or community level. CYPREP’s assistance in the development of a network of organized village level disaster committees created the means by which the CNS could be both influenced by and support the communities for which they are responsible. CARE has tried to play a key role in enhancing the institutional capacity of CNS in CARE Madagascar: Title II DAP Final Evaluation Page 34 building up its information systems and strengthening its role as national coordinator of disaster response activities. The results have been mixed, primarily due to the difficulties of working with regional bodies and due to the community-based workload undertaken by CYPREP. It is apparent that preparedness and response strategies elaborated by each community will eventually help CNS play its role, but CYPREP has been challenged in doing both local-level development and regional capacity-building at the same time. Timing is better now for focusing on strengthening the CNS now that the communal and community linkages have been established by CCSs and CVSs. Other parallel efforts by CARE to strengthen CNS, however, have been made. Since 1999 CARE Madagascar has orchestrated a system called SIRCAT for the development of an Information System for Risk and Disaster Management within the CNS. This system has succeeded in gaining national consensus on the geography of hazards and vulnerability in Madagascar. Strategic Orientation 6: Support for the development and operations of the Commune Emergency Management Committee. The Commune Emergency Management Committee The Commune Emergency Management Committee, or CCS, was established in all CYPREP communes and some communes outside of the projects focus. All received training in disaster preparedness and mitigation by CNS with CYPREP assistance, but the majority of CCSs are not very functional. Their members are ad hoc, and many still do not know their roles and responsibilities in relation to either the functioning of the CCS or disaster response. It should be noted, however, that CYPREP is only now in a phase of providing assistance to CCSs in, for example, elaborating Communal Plans for Cyclone Preparation (PPCC), and that this has occurred at such a late stage in the DAP that it has not had a chance to be properly evaluated. Capacity-building of Social Mobilization Agents (Assistants de Mobilization Sociale (AMS)) All Social Mobilization Agents (AMS) recruited and used in the project received formal training at the beginning of their service on social mobilization, household livelihood security, and the FALAFA process. This type and level of training was an important experience for AMS agents, all of whom come from rural areas in the CYPREP zone. Despite their limited exposure to the training topics they were able to grasp key concepts and apply them in their work. In each zone it was CYPREP’s policy to put into place a multi-disciplinary team in order to facilitate team work and provide a broad perspective to community development and disaster management. Despite CYPREP’s efforts to provide high-quality training, several AMSs cited areas where they felt their capacity needed more strengthening. Part of the reason for this may be the broad range of issues that they face when working in rural and often remote areas. Many of these CARE Madagascar: Title II DAP Final Evaluation Page 35 communities have had only minimal exposure and interaction with development organizations, and often the AMSs are overwhelmed. Some of the suggestions made were to enhance the training to include communication skills, rights-based approaches, children’s and women’s rights, pedagogy and health issues. Many also expressed in receiving computer training. Other suggestions included having exchanges among AMSs both intra- and inter-zonal in order to reinforce training, exchange experiences, and develop common approaches. All of these capacity-building elements would, they felt, enhance their ability to facilitate the FALAFA process and assist communities in the development of their PDF. IV.C Food Aid Management Systems & Internal Controls A full accounting and evaluation of CARE’s food aid management and practices was not called for in the Scope of Work for the final evaluation of Mahavita. However, some observations are provided here that resulted from the evaluation work. CARE Madagascar has one large warehouse in Tamatave with a capacity of 1,500 tons and two prefab warehouses (e.g., 350 tons in Fenerive) to store the food commodities. Together they have a capacity to store about 2,000 MT. The food was stacked properly and on pallets in clean conditions. CARE Madagascar has occasionally experienced commodity infestation, but was able to work closely with the government to salvage infested commodities. Monthly physical inventory counts were periodically reconciled with stock balances. In all, it appears that proper warehousing management has been practiced in CARE Madagascar. CARE Madagascar has followed the Food Resource Manual Guidelines for commodity management. Designated personnel receive and issue commodities upon authorization of the Project Coordinator. Waybills were properly prepared for receipts and issues of commodities. Inventory reports were prepared monthly and submitted to CARE’s head office for checking and reporting to the donors. In addition, the finance section of the project controlled food transactions. Food Monitors engaged in random and independent checks of the beneficiary lists and activities accomplished. The quality of the food distribution system was monitored and included end-user monitoring. IV.D Food Distributions in FFW Programs CARE Madagascar distributed food for emergency and regular programs based on emergency and FFW ration scales developed within the project. CARE prepared food distribution plans for the whole year and submitted reports by comparing the plan to the actual food deliveries. Some beneficiaries could not tell the team about the quantity and type of food rations they received. However, they believe the food distributions were fair and equitable. FFW participants noted a short-term improvement in household food security. Because the communities controlled the beneficiary selection process, targeting for emergency programs was usually deemed to be fair. CARE Madagascar: Title II DAP Final Evaluation Page 36 V. Conclusions V.A Mahavita Strategic Objective 1: Targeted households will be using increased income and savings to provide for their food and livelihood needs. ƒ The FFW activities in Mahavita were run with the utmost professionalism, and as a result relatively few problems were encountered. The poorest Fokotanys and poorest households within Fokotanys were all well-targeted as evidenced by qualitative and quantitative data. Female participation was also high. Target benchmarks in FFW activities were exceeded by about 70 percent. ƒ Mahavita was able to leverage donor resources during the political crisis and rapidly initiate FFW activities that provided a vital safety net to poor urban households. This rapid response capability was the direct result of working with developing Structures de Developpement and the rapport that Mahavita had built with Fokotany residents and officials. The actions taken during this time period validated CARE’s approach to community development and the impact that creating representative structures can have on mobilizing financial and human resources. ƒ Poor communications among Mahavita, USAID Madagascar and FFP Washington on resolving the CFW-FFW issue created delays in programming and restricted some alternatives that may have been better than those implemented. The amount of time it took to resolve the issue also hindered progress and may stalled momentum that had been building through the CFW activities. ƒ Too little was done to generate employment opportunities for Fokotany residents during the life of the project. Use of cash-for-work and then food-for-work for employment generation was not a sustainable employment generation activity. Rather, it was an important safety net for poor urban households, especially during the political crisis. FFW is not a substitute for employment generation. Mahavita did not have a viable strategy for generating employment opportunities but depended on an early hypothesis that significant demand could be generated for the services of worker groups that gained new skills under FFW programs. It appears that no other alternative strategies were ever employed to transfer skills gained from FFW and personal empowerment training to viable income-generating activities. ƒ An opportunity was lost on carrying the savings component over when CFW was shifted to FFW. The savings component was an explicit objective of the project but was virtually ignored after the cash-for-work activities were transferred to food-for-work, even though there are viable models for establishing savings schemes in FFW activities. CARE Madagascar: Title II DAP Final Evaluation Page 37 Strategic Objective 2: Communities will be empowered to address constraints affecting their ability to control their own resources and livelihoods. ƒ CARE clearly had no well-articulated exit strategy for disengaging with SDs. The development of the PDQ became the de facto end product of the community development process, and Mahavita staff struggled internally with how to “keep the PDQ alive.” It would have been beneficial to pilot strategies to implement the PDQs once the earlier and more mature SDs finished their plans. Some SDs have had their plans finished for over one year without guidance on how to seek funding or how to promote their local plans in other ways. ƒ All of the SDs are struggling with ‘next steps.’ Members still appear highly motivated, but this motivation could quickly wane if the PDQ does not result in tangible outputs and if there are no further activities to keep members engaged. The PDQ should be viewed as being closer to the starting point of the community-development process than to the end point of the process. Some SDs are seeking their own funding with limited success, but there has been no coordinated effort on the part of Mahavita to develop alternative strategies and capacity for such action. ƒ Mahavita’s strategy of obtaining Fokotany approval of the project through an official request and naming the core Fokotany authorities as the initial ad hoc committee was effective. This is a valuable lesson learned for working in urban areas where initially there may be little community cohesion and where it is desirable to avoid the frequent mistake of using infrastructure as an entry point. ƒ In many ways Mahavita became stalled on the success of its FAMOA process. The achievements in developing participatory structures at the community level were so great that staff ignored some of the other vital components of livelihood enhancement strategies, namely income generation and savings. ƒ The political crisis had both positive and negative impacts on the project. On the positive side, as noted above, CARE was able to leverage donor resources during the political crisis and rapidly initiate FFW activities that provided a vital safety net to poor urban households. This was a result of the SDs already being in place and capable of rapidly organizing their communities. CARE was able to respond much quicker than any other NGOs. On the negative side, the crisis diverted attention away from the natural evolution of project activities, and is likely the main reason why more advancement was not made on implementing PDQs. ƒ Often dominated by the chef de quartier or other influential person. Difficult to ascertain how ‘representative’ the SD is over time, as there is little idea of constituency and almost no rotation of those who ‘manage’ the SD. ƒ More cross-fertilization of SD development may have been a good strategy to generate lessons learned and provide SD members with a sense of camaraderie among their neighbors. Also a forum where SD members meet to discuss advances, problems, successes, etc. This CARE Madagascar: Title II DAP Final Evaluation Page 38 forum would have been a good entry point for brainstorming new ideas or building consensus among Fokotany members. ƒ The legal status of the SDs and their positioning in light of recent GOM administrative changes will need to be addressed in order to determine what type of Fokotany structure CARE will continue to support in future urban work. For example, there may be scope for small government subsidies to employers who hire local residents. Strategic Objective 3: Communities will create and maintain a healthy and hygienic environment within which inhabitants can carry out productive livelihoods. ƒ CARE/Mahavita has made some serious positive inroads in the environmental health of an urban zone that is extremely poor and vulnerable. FKT where program activities have been ongoing for a number of years clearly have healthier, more hygienic environments. Residents speak of noticeable health improvements (less skin disease, children’s diarrhea, even infant mortality). These results do not show, however, in newer FKT, and CARE is challenged to continue its work in the original 30 FKT and to add 50 more, all with reduced personnel. ƒ CARE/Mahavita has workable models for community organizing and empowerment, social mobilization, cost recovery, and for linking infrastructure development to food security and poverty reduction. However, most of the infrastructure is too new to yield models for maintenance. ƒ CARE’s strategy of avoiding building infrastructure before the requisite community-driven development process is completed is sound. Past experience in other urban projects has shown that when infrastructure is focused on first then ownership issues arise and maintenance can be a real problem. However, Mahavita probably started too late in the project on infrastructure, with a significant amount being developed during the last six months of the project. This makes it difficult to ascertain the extent to which communities will maintain the infrastructure. Most SDs have maintenance plans and/or committees, but most of the infrastructure is too new to determine how well it will be maintained. ƒ In spite of the fact that CARE/Mahavita attained the majority of its construction targets (the exception being with latrines), water and sanitation coverage of the target FKT is inadequate for the needs of these populations. This is especially the case for sanitation (safe feces disposal), and lacking this, it is difficult to say that communities have completely achieved healthy and hygienic environments. Despite the success of the health and hygiene components in promoting knowledge and behavior change, it is difficult for some households to take full advantage of their knowledge due to access issues associated with latrines. The AGETIPA model does not seem appropriate for meeting the sanitation needs of this population and future interventions in sanitation infrastructure need to reevaluate latrine design and strategy. CARE Madagascar: Title II DAP Final Evaluation Page 39 ƒ Certain themes or community environmental health issues have not been adequately addressed. In particular, food safety in the common gargottes and hotely where the baseline survey found over 50% of the population eats every day, and hand washing with soap. These themes are part of the package of themes and messages, but haven’t been spotlighted. ƒ Decisions concerning environmental health strategies such as HIMO and promotion of Sur Eau were logically made within the political and epidemiological context of certain periods (food emergencies, political upheaval, and cholera). Things have changed and calmed down, and the kind of strategic rethinking required by the new DAP is timely. ƒ CARE’s use of GIS technology was innovative and had a positive impact on its credibility with the communal government. Internal issues on managing the GIS system stalled its usage in latter stages of the project. Strategic Objective 4: Households practice healthy behavior and hygiene. ƒ A systematic household level evaluation was not possible. However, the follow-up to the Baseline has substantial information regarding a range of household hygiene behaviors: water availability transport and storage, latrine availability and use. It also contains information on health status and practices: incidence of diarrhea and ARI, care seeking in case of child illness, children’s nutritional status. ƒ Very unscientific observations and discussions gave the following impressions: many households in older FKT have a new consciousness concerning their responsibility for maintaining their home environment (sweeping, garbage disposal). However, the lack of usable latrines makes it doubtful whether there can really be healthy, hygienic households. While the follow-up to the baseline shows a high percentage of latrines in the zone, it was repeatedly mentioned as a problem during the evaluation. A shift in focus to adequate sanitation coverage is critical. V.B East Coast Programming - CYPREP CYPREP had only one strategic objective, thus the conclusions are organized around its three Intermediate Results. Intermediate Result 1: Reduced vulnerability in 132 communities on the East Coast of Madagascar. ƒ CYPREP would have benefited from prioritizing the rehabilitation and reopening of rural roads earlier on in the intervention process (assuming that in each case it was also a community priority), in order to put the potential intervention sites on more even footing. This would have led to better targeting of the most vulnerable communities. However, it is recognized that road rehabilitation is an ambitious undertaking and the limits imposed by CARE Madagascar: Title II DAP Final Evaluation Page 40 obligatory environmental studies would have made it difficult to entirely eliminate the criteria of accessibility from the site selection process. ƒ While CYPREP staff did conduct site visits and surveys afterwards to validate the information presented in the workshops and provide a more realistic image of the situation, it would have been advantageous to collect this information before the workshops, so that discrepancies between the various information sources could be discussed in the workshop, and allowing the village representatives to benefit from the information collected. One possibility would be to develop a village-level data base for the region. This would allow a more objective discussion of indicators and vulnerability from the start of the process, and would eliminate time spent verifying biased data. What is more, if the local authorities were also implicated in the creation of the data base, they themselves would have better access to objective information about their communities. On the other hand, the time and resources needed to conduct an exhaustive survey of all fokontany would have been prohibitive and possibly delayed the initial project implementation. A third option would be to conduct the fokontany-level survey only in the communes pre-selected in the district workshop – a compromise which would not require such extensive time and resources. ƒ A framework for the presentation of each fokontany was prepared by CYPREP and used by village representatives during the mini-workshops. This should be expanded, to improve the quality of the information presented by the local authorities and to assist those who are handicapped by their low level of education. Some flexibility should be included to account for the varying levels of education of local authorities. This would eventual lead to the development of a standard tool for presenting information at the village level. ƒ CYPREP did not develop any type of consensus-based standards for defining vulnerability. Without some form of standards, and information systems that complement these standards, it is impossible to determine if vulnerability has changed and how. The absence of a strong monitoring component to CYPREP was perhaps its largest shortcoming, and it failed to take full advantage of post-disaster opportunities to critically assess its impact and use reflective practice to inform its programming and strategy. ƒ CYPREP’s requirement that each community contribute 20 percent of the necessary resources/funds for major mitigation activities was too strict. Future community contribution schemes should be more flexible and include different forms of contribution, both monetary and non-monetary. For example, there could be phased contribution to allow work to start with a community commitment to invest. This would be a “mature” community that had participated in other project activities and had demonstrated previous community contributions. A combination of monetary and physical contribution could be used for poorer communities and could be needs-based or applied generally across the community. Or a scaled contribution according to a project-based definition of vulnerability could be used. CARE Madagascar: Title II DAP Final Evaluation Page 41 Intermediate Result 2: Creating a network of local partners for disaster response. ƒ The participation of community members in prioritizing mitigation activities as well as deciding the level of community contributions was one of the strong points of the approach used by CYPREP. This aspect of the approach should be continued and expanded in future projects to increase community ownership. The same is true of the participation of local authorities, which should be continued and reinforced to help them better assume their responsibilities and those of the CVS. ƒ Despite the effort made by the project from the beginning to solicit the participation of the commune-level authorities in the different steps of the process, certain mayors suggested that it would have been even more fruitful to fully integrate the staff from the mayor’s office in the facilitation and training processes, so that they could better support and monitor the activities eventually undertaken by the villages. ƒ The writing of the PDFs in French is useful for the local authorities when dealing with donors and regional authorities. However, it is not useful to the community members at large, few of whom speak French. In recognition of the fact that the community is the end user of the PDF, the document should be developed entirely in Malagasy, with the French version being simply a synthesis of the most important points. This is essential if one wants to speak of full ownership of the document by the community. In this way, more members of the community could read the original document and discuss its contents (rather than relying on the verbal description of the contents given them by community leaders). This would also reinforce the use of the PDF as a tool for negotiation and reference amongst the different community groups. ƒ Because the PDF is an indispensable tool for the creation of a PCD, the active participation of local officials, especially the mayor’s office, in the planning process for each village should be continually reinforced. This will help local officials better understand their development role and will build communication links between them and their communities. ƒ As in Mahavita, CYPREP lacked an exit strategy for disengaging with communities, and the PDF was often viewed as the endpoint. At a minimum, a system for periodically updating the PDF and adding new information should be developed in future programming. This would also serve as an opportunity to retrain CVS members and reinforce their skills through experiential learning. ƒ Given the effectiveness of PPCs in changing practices related to disaster preparedness and mitigation it is important to find a way to develop PPCs in all East Coast communities. Also, the use of the Malagasy language contributes extensively to ownership of these plans by the community. Expanding coverage could be realized through the intermediary of the CCS, which should theoretically pick up the baton and extended the CYPREP activities throughout the commune. The training given to the members of the CCS, in all three intervention zones, conducted by CYPREP in collaboration with CNS and the United Nations Volunteers, will certainly contribute to building the capacity of the communes in this regard. CARE Madagascar: Title II DAP Final Evaluation Page 42 ƒ It is also important to ensure that all villages have alert systems (gokas, flags, etc) and that there is a common code for the different types of cyclone alerts. This could be done at almost no cost, for all the non-intervention villages in the zone. It would perhaps be preferable, compared with providing the village with megaphones, which will eventually need repair or maintenance. Similarly, it will be necessary for CNS to provide the communes with adequate resources so that they can accomplish the tasks assigned to them, and to reinforce the capacity of the members of the CCS. ƒ The choice of mitigation options may not have fully been explored, and was often limited to (or influenced by?) the technical expertise of CARE staff, especially engineers. Future mitigation activities should explore the full range of livelihood options that relate to preparing for and mitigating the impact of disasters. Intermediate Result 3: Re-enforce the capacity of the National Emergency Preparedness and Management Agency (CNS) to promote the use of preparedness planning and disaster response measures at the local level ƒ Community members, local authorities and project staff were all unanimous regarding the usefulness of the CVS in reducing vulnerability of east coast households and in its efforts to mitigate the negative impacts of cyclones in particular and disasters in general. The CVS has evolved into an indispensable structure for all community-level development and in terms of cyclone prevention. In this regard, the development of such CVSs or equivalent structures should be regarded as a key element of any current or future strategy for east coast programming. une telle structure serait indispensable dans toute intervention au niveau du fokontany, car elle constituera la structure de relève lors du retrait du projet. De part son rôle, elle constitue : ƒ Eventually the issue of CVS legality must be addressed more directly. If the CVS is the community-based organization that guides local development and manages disaster preparedness and mitigation then it has to have a more permanent and respected status. This would require that the CVS is recognized as a legal structure by the GOM, that it has certain roles and responsibilities that are commonly and legally defined as well as recognized by the community, and that its active members retain a sense of commitment and purpose. It should be noted here that the same issue exists for SDs in urban programming, and CARE needs to develop a unified strategy for fokotany structure that it can advocate for in the future. ƒ The use of multidisciplinary teams in its programming is a very strong, positive element of CYPREP, and its system merits being copied in other programming. The dynamism within the CYPREP team was high and there was a feeling of empowerment among staff that was facilitated, no doubt, by the team processes that were established and supported (similar to staff feelings in Mahavita). ƒ The FALAFA process contributed to empowerment feelings among staff and AMSs. It would also be useful to supplement the FALAFA experience with training in CARE Madagascar: Title II DAP Final Evaluation Page 43 communications and PRA skills, and with training techniques for adult literacy and andragogy. V.B.1 Other CYPREP Conclusions ƒ The monitoring and evaluation system was put into place too late in the project for it to have a meaningful impact on progress tracking and real-time feedback. Overall there was a lack of structured monitoring and learning. For example, the impact of Cyclone Kesiny (and others) on project activities was never fully researched, and in a disaster-related project often the only opportunity to learn about approaches and their impact on targeted communities is to do intensive post-crisis follow-up. ƒ CYPREP was hampered by not conducting a proper baseline. The original monitoring plan called for assessing temporal change of selected indicators based on their baseline values. Since there was no baseline this was not possible and made it difficult to determine the magnitude of some changes that took place. Of perhaps more importance would have been the experienced and knowledge gained by project personnel participating in the baseline survey. ƒ Future east coast activities should consider using a Management Matrix for monitoring institutional change. The Management Matrix is a tool composed of individual indicators organized around the themes of institutional strengthening and expected performance of institutions. Themes can include such items as governance, financial management, human resources, vision and planning capacity, and implementation. Under each topic is a negotiated set of indicators that the project and each institution (for example, CVSs) agrees upon. The indicators reflect shared but specific criteria for success in capacity-building. Indicators can be qualitative or quantitative and each has negotiated criteria scoring (the same indicator used for different institutions, thus, can have different scoring criteria). Scores can be aggregated within themes and for the entire matrix. There is a maximum obtainable score that can be used to assess overall capacity. ƒ Intensive implementation (multiple activities/interventions in one location) versus spreading out. Is there a “critical mass” for reducing cyclone vulnerability? VI. Recommendations for DAP II This current evaluation, as well as important work by Drinkwater (2003) and Garrett (2003), has identified important issues that need to be considered as CARE continues its urban programming efforts under the new DAP and other funding. The issues and recommendations highlighted will be incorporated into this section for ease of reference since their views are widely shared by the evaluation team. CARE Madagascar: Title II DAP Final Evaluation Page 44 Urban Development/Governance y Seek legal formation of SDs in order that they be recognized as the official development entity of the Fokotany. In doing so, engage Arrondissement and Communal officials in the process so that they recognize from the beginning the logic and value (as many already do) of such action. y Forge a rationale relationship between the newly elected representatives of the Fokotanys and the SDs. Explore with community members what the nature of these relationships should be, along with those of appointed officials. Facilitate the articulation of a more formalized arrangement for SDs. y Develop a clear exit strategy for the next DAP. The exit strategy should include when and how CARE will disengage from their work with specific Fokotanys and what the nature of a broader relationship would be. y Develop a set of alternative strategies for linking communities and their PDQs with service providers and sources of funding. There may not be a “one size fits all” approach to this, and the issue of the number of Fokotanys seeking these linkages should be considered, so that one alternative strategy may include the grouping of Fokotanys according to certain criteria (such as linkages to specific canals). Also, the formal status of the PDQs needs to be clarified, and advocacy is needed to make other institutions aware of its evolution, its linkage to community-based processes and structures, and its role in community-driven development. y Continue to use GIS as a tool for planning infrastructure development and maintenance and as a capacity-building tool for communities and government officials. Mapping of canals in the current DAP was an effective way to gain access to communal government and provided an important service for Antananarivo. y The issues of representativeness and constituency need to be revisited with respect to SD function. Most SDs have had little or no turnover in leadership since their creation. This was a slight advantage to Mahavita during this DAP because it added continuity to a new and sometimes difficult process. In the longer run, however, more democratic processes should be encouraged and could even include periodic elections for SD leaders. The idea of SD leaders or members having constituents should also be explored. Since the SD has emerged as the ‘voice’ of the community there needs to be a reality check to gauge just how representative they are, and who speaks for different social groups that live in the Fokotany. y In the new DAP Mahavita staff will have to develop indirect models to provide support to SD. Using local NGOs is one option. Another is to use mature SDs to help develop capacities in new Fokontanys. It might be feasible for SDs to work with local NGOs, so that the longer term support is provided by the NGOs, and the time demands on the SD members are not too huge. Nevertheless, such a role is when for which the SDs should receive some compensation, perhaps for stages of the process evaluated to be successfully completed. Thus, in the development of a strategy to expand to new Fokontany, existing SDs, as well as CARE Madagascar: Title II DAP Final Evaluation Page 45 relevant local NGOs, provide the major opportunities. The provision of joint training, to ensure SD/NGO partnerships, would be ideal. y In transitioning to the next DAP, Mahavita will need to make strategic decisions on levels and modes of operation with all urban stakeholders, in particular with service government authorities, service providers and donors. y The reduced Mahavita staff will have to forge strengthened relationships with a number of institutional actors for DAP II. Mahavita will need to develop an extensive relationship with the CUA and the Arrondissements and a good working relationship with service providers such as Jirama and Apipa. With still others, such as UNDP and some INGOs, CARE may be a collaborator in promoting urban governance. Additionally, sectoral decisions will need to be made – how much does Mahavita engage with the health sector institutions, for example, from whom a strong willingness to work again with CARE has been expressed to the environmental health consultant on the evaluation team. Since Mahavita cannot simply impose its own agendas, it will have to seek agreements with the institutions themselves, and how Mahavita can support these. In any case, some type of MOU with each organization which specifies joint purposes and respective roles in promoting improved governance and service delivery within the fokontany would be a valuable starting point. y The new DAP will need to facilitate the planning process at the CUA/ arrondissement level, and this will require a coordinating body with active participation. CARE may chose to rehabilitate the ISIKA model. If so, it needs to address the sustainability problems that plagued previous efforts. In the Kuyakana project in Maputo a coordinating and planning body was facilitated by the expedient mechanism of the project paying the salary of a person acting as coordination secretariat in the municipal council. This role subsequently became institutionalized and the person graduated to becoming a full time council employee, once the value of the role had been clearly demonstrated. Initially the role focused on the establishment of regular meetings to coordinate the activities of different agencies responding to the aftermath of floods in Maputo. What is required is a purpose that would be widely accepted by different agencies that would serve as a means of reaching agreement as to how coordination could take place, and how the responsibility would be managed. • Federating SDs, for example, at the arrondissement level. There are far too many SDs – some 192 in the CUA as a whole – for each to be able to negotiate resources for the implementation of their PDQs. There needs to be a much more strategic mechanism whereby the SDs can engage with stakeholders, and this can only happen with mechanisms for grouping the SDs. This could happen in different ways. Perhaps the most obvious would be for the SDs in each arrondissement to develop a federation, so they can work directly with the arrondissement and with other authorities, at that level. Such a structure could also liaise with bodies like Apipa. To begin with though, both the levels of federation of SDs and the mechanisms they could use to reach agreement across fokontany on priorities, would need to be agreed. And this in turn, would require some more formal acceptance of the SDs. • Another generic activity successfully utilized by Kuyakana was to hold periodic stakeholder fora on specific urban themes, many of them related to environmental health. The project CARE Madagascar: Title II DAP Final Evaluation Page 46 would meet with a range of stakeholders beforehand, generate an understanding of their own agendas and challenges, and then use the one day stakeholder workshops as a way of bringing the different stakeholders together, have them share operational issues, and then agree on follow up steps and mechanisms. Some form of stakeholder engagement and collaborative mechanisms will be required for specific thematic issues. • In shifting to the next phase, and an approach that is based more on rights than needs, one critical role for Mahavita is to be able to work at the municipal and, where necessary, national level, to agree service provision entitlements for fokontany residents, particularly in relation to basic service provision. Standards, if they do not exist already, should be agreed for areas such as water provision, sanitation, drainage, and refuse removal, and then more critically, plans for ensuring that these levels of provision are met, should be negotiated. This role may well require advocacy work, and would be best undertaken if Mahavita is part of a broader coalition that agrees to pursue such changes. If necessary, Mahavita should be able to play a leadership role in the establishment of such a coalition. One role of the coalition would be to push for greater resource allocation in areas that are necessary, and where appropriate, the tracking of resources that have been allocated, to ensure they are used for their intended purposes. • One issue for Mahavita in this next phase will be the new staffing configuration and skills that are needed, given that the program is about to change fundamentally the way it works. This applies particularly with respect to the various kinds of partnership work. Present staff will need to upgrade their skills and become more confident working at different levels of government and other organizations. Some new staff may also be required, if certain skills appear obviously lacking. GIS is one area, and advocacy skills may be another, though existing staff should be encouraged to take on this kind of role as a first step. In addition, another area where all staff require retooling, is their ability to shift to more of a rights based approach. This is likely to require external support. • Mahavita – and urban government in Madagascar – remains relatively closed to urban experience elsewhere, and it would help the program and staff to receive wider exposure. The use of judicious cross-visits could be extremely helpful to the program, particularly where urban programs are already working in a more indirect mode. Urban programs that have solid experience of supply side work and facilitating the ‘negotiated development’ role, would be most appropriate. The Shahar program in Bangladesh, or some of the urban programs in Zambia, could be good candidates. Some of the effective experience in working at the level of municipal government has been that of the Kuyakana project in Mozambique, but this project is currently seeking renewed funding and has only a skeletal operation remaining. If cross visits were to occur, they would be most appropriate if they occurred once better operational relations had been developed with the CUA and arrondissements, so that at least one staff person from communal government could also take part. In addition, as wells as potential cross-visits, other kinds of networking and communication should also be instituted that results in the broader exposure of Mahavita and CUA staff to urban governance experience in other relevant contexts. Environmental Health CARE Madagascar: Title II DAP Final Evaluation Page 47 y Focus on achieving better water/sanitation coverage in the target FKT by developing single or multi family latrine models adapted to flood prone areas. Collaborate with the WASH Campaign that is embarking on a program of operations research into such latrine models, but does not currently have a specific urban slum focus. y Develop a hygiene promotion strategy to generate demand for sanitation improvements, and support the Commune technical agencies responsible for providing these services. An overall promotion strategy could be piloted in several Fokotanys and then adapted as a more general approach. y Restart hygiene promotion (IEC) programs with a focus on key household behaviors such as safe excreta disposal, protection of drinking water, and hand-washing with soap. Coordinate closely with the hardware side such as family latrine construction. Even though current evidence suggest that health and hygiene knowledge is high, there is needed improvement in actual practices. y Expand the techniques for hygiene promotion to avoid stagnation and boredom. Link with others in and out of Tana/Mad who are doing similar activities, for exchanges and inspiration. Consider traditional theater, videos etc. y Strengthen the system of community “animateurs” and “animatrices” by increasing their capacity as community health agents through more training that groups different FKT, exposure visits to areas where results are being felt. Consider remuneration (as some already are). y Streamline the FAMOA and PDQ development process without sacrificing its effectiveness. This has been mentioned elsewhere. y CARE needs to be concerned about all aspects of the work it does in coordination with other bodies (e.g., quality of infrastructure, appropriateness of design) because ultimately it also reflects on CARE. For DAP II CARE should establish operating principles or principles of engagement with other donors or agencies to address this issue. Care helped AGITEP with World-Bank financed public latrines but their appropriateness and prospects for cost￾recovery are questionable. Health y Consider reconnecting with the Health Sector at both the Ministry and the Commune levels (not the same) in mutually beneficial activities. In particular, consider collaborating in areas requiring strong community participation such as identifying dropouts from immunization program and from TB treatment program, organizing immunization campaigns, identifying malnourished children for treatment at specialized centers. Mahavita’s community animateurs/trices could play a critical role in these and other areas. CARE Madagascar: Title II DAP Final Evaluation Page 48 y Work with the MOH to advocate for an urban slum policy. This policy has existed for some years in draft form, but the MOH and UNICEF haven’t been able to push it through. The time may be right for such a policy. y Support the official MOH business plan and strategic objectives and avoid parallel health activities. This is one reason there was estrangement between CARE and the MOH in previous years. Guiding Principles for DAP II Environmental Health activities y Whenever and wherever possible, connect environmental health activities to income generation. For example, design a family latrine coverage strategy so that local artisans can make and sell parts, or that a caretaker paid by the families maintains multi-family latrines. Or, help communities organize contracted house-to-house garbage collection in areas where the “bac” is far from the residences. y Conduct all environmental health improvement activities with the cognizant Commune authorities, aiming at increased GOM awareness of the needs of the urban poor, and ultimately to improved urban poor policies and programs. y Seek out partnerships to support various priority programs not covered by the DAP. For instance: - Link with JSI/RTI’s new program to support Communes to develop PDD leading to grants at the CSB level; - Explore collaboration with WASH campaign on a number of fronts – latrine development, hygiene message and methods development, linking with others carrying out similar programs, especially during the upcoming Hygiene Promotion “Braderie”; - Coordinate with other NGOs etc. with programs in areas of relevance to CARE population (housing, agriculture, HIV); and - Develop an effective stakeholder and service provider coordination mechanism such as an Urban Health Alliance or an Arrondissement-level coordination model. CARE can play the role of convener of such a group. Enhancing Food and Livelihood Security on the East Coast y In CYPREP, the involvement of local authorities in site selection contributed to the success and sustainability of the activities undertaken. For this reason, the involvement of authorities, administrative (sub-prefecture, commune, fokontany/village) as well as traditional (tangalamena), throughout the process is indispensable and should be continued in future projects. CARE Madagascar: Title II DAP Final Evaluation Page 49 ƒ As a priority for future interventions, it would be profitable to assist the communities that began working with CYPREP in 1999 and 2000, in Fénérive Est, to also develop PDFs. This would assist future projects working with these communities, and would require very little effort, as the communities, in the course of their collaboration with CYPREP, already completed most of the baseline work. ƒ The communal officials would like to have a more comprehensive packet of training and IEC materials to help them in training villages that did not work with CYPREP. The communes that are in the process of creating their PPCC (communal cyclone preparedness plan) have completed PPCs for all the villages. ƒ It is undeniable that the participation of women in community-based activities brings a new vision to local development. As a result, it is important to consolidate, continue and expand the lessons learned concerning the integration of women in the development process, an issue on which the project has accumulated significant experience, permitting them to contribute more fully to the development of their communities. ƒ The emergency response of CNS, after a cyclone, is based on the report compiled by the CLS, which is in turn based on the EIMA forms arriving from each village. For this reason, the existence of a functional CVS, with members trained to fill out the form and convey it to the proper authorities, is a considerable advantage for CNS and for other actors offering emergency assistance after a cyclone. This CYPREP activity, in particular, merits being expanded throughout the region, to increase the rapidity and effectiveness of future cyclone response efforts. CARE Madagascar: Title II DAP Final Evaluation Page 50 ANNEX A Scaling up Community-Driven Development: Reflections on the Mahavita Experience10 James Garrett International Food Policy Research Institute IFPRI Reference No. 2570-023 This report highlights key issues surrounding Mahavita’s operations and plans to scale up based on my visit to Madagascar from January 28 to February 12, 2003. These reflections are based on wide-ranging conversations with current and former senior and field-level staff, with municipal, prefecture, and fokontany authorities, and with members of the community structure de developpement (Annex 1). Andri Ratsitohara, head of follow up and evaluation for Mahavita, and I spoke with representatives of 10 different communities (out of 30 where CARE now works), 15 field staff, and 5 senior staff using semi-structured interviews, along with municipal and national authorities. (See Annex 2 for study methods, objectives, and interview guides.) This study will form the basis of one of a number of case studies that IFPRI is carrying out in collaboration with the World Bank on scaling up community-driven development. That case study will compare and contrast the Mahavita experience with that of PUSH / PROSPECT, another program initiated by CARE focusing on urban livelihoods, in Zambia. Although this report focuses on factors affecting scaling-up, it will also touch on operational issues. As the prefet de Antananarivo pointed out, “the two are linked. If (questions about operations) are not answered, it is hard to scale up.” Scaling up often refers to expansion in terms of area or numbers (that is, size), but it can also refer to adding new activities, including activities to increase political influence (Dongier et al. 2002, Uvin 1995). Although this report will not focus on these distinctions, they are present in the Mahavita experience. The focus on scaling up is especially relevant given Mahavita’s ambitions to expand from 30 to 80 Fokotany’s in the next DAP. The main expression of community-driven development (CDD) under Mahavita is the FAMOA process – the community-driven preparation of a community development plan, or plan de quartier, a PDQ. CDD does not mean that the community takes on full responsibility for all development activities in the area; but it does mean that the community participates meaningfully, and fundamentally drives, the decisions about development that affect it. In simplified schematic terms, for CDD to be effective, communities must have the capacity to carry out development-related activities and make demands on service providers, primarily 10 A complete report, including annexes, can be requested from IFPRI or CARE Madagascar. CARE Madagascar: Title II DAP Final Evaluation Page 51 government authorities. Government must have the willingness and capacity to respond. At the same time, effective mechanisms that link community and government must be operating in a conducive, enabling environment. It is this conceptual framework that underlies the comments below. Scaling up Interestingly, this study did not find that Mahavita suffered from many of the potential obstacles to scaling up, such as community or staff capacity, perhaps because Mahavita itself avoided many of them. Among the most important facilitating factors: • Mahavita developed a vision for community development that it promoted and, ultimately, had shared by key stakeholders. • Upper-level stakeholders, like CARE’s country director and USAID, provided space for the development of a community-driven approach (despite some serious misgivings and criticisms for not producing more outputs sooner). • Government policy supported decentralization, and few funds were otherwise available for fokontany development. • The project manager strongly supported the idea of community-driven development and pushed for the empowerment of his staff to mirror the empowerment he wished them to transfer to the community. • Mahavita developed processes and tools through pilots (and subsequent reflective learning) before expanding operations. Examination of some other issues in more depth reveals both strengths and weaknesses: Empowering Communities Members of the structure de developpement (SD) and the fokontany authorities were unanimous that Mahavita has benefited the community. A promising sign was that, when asked how they had benefited from Mahavita and the FAMOA process, by far the most common answer was the “change in mentality,” rather than the works as such. The FAMOA process had taught them how to work together as a community. Empowering Staff Mahavita has developed a learning culture, emphasizing learning-by-doing, reflection, and the positive aspects of learning from mistakes. This atmosphere empowers staff to be creative and take charge of their own activities. This sort of flexibility and empowerment, within the framework of principles, is essential for success in the field. Capable field staff who know community conditions are almost always better able than management to handle particular situations that arise at the community-level. Indeed, this kind of confidence in field staff, and a willingness to work closely with them to develop their abilities so they can exercise independent decision-making, is essential to a successful project. Centralizing decisions not only takes time as decisions work their way up and down the authority channel, without wide consultation and understanding this mode of operation also will likely not take adequate account of community conditions. In addition, staff respond to CARE Madagascar: Title II DAP Final Evaluation Page 52 the management example they are given: If staff are disempowered, how can they be expected to work in an empowering way with communities? Establishing Processes Successful scaling up, and successful community-driven development, relies on developing principles for action, rather than a standardized model. Standard structures, mandates and mechanisms may be easily replicated but they violate the very spirit and benefits of development actually driven by the community. CARE did this in its development of a FAMOA process, which is extensively documented. All staff, and SDs and fokontany authorities, are well aware of this process, and can easily outline its steps. This facilitates scaling up because staff and communities now have a blueprint for action. Control of that action, however, remains in the hands of the community. The output of this community-driven planning process is the PDQ. FAMOA and the PDQ are obviously major successes, but Mahavita must now work to bring the process full circle. It must continue to evolve and adapt its approach based on its experience. As of now, although staff and SD do intuitively understand that communities should take the PDQ as a guide for action, the steps they should take post-PDQ are not clear. A number of communities are waiting to receive their “finalized” PDQ back from CARE; others have the PDQ and appear to see its use mostly to show other NGOs who happen to drop by. By default, then, the PDQ is an “end post,” and not a starting point, in the process of promoting community￾driven development. CARE and communities, however, must understand that the PDQ is part of a process. Ultimately it is a tool for community-driven development; it is not an objective. CARE must support the community in implementing its PDQ, probably through various iterations and exercises, but the goal is to enable communities with the capacity to carry out planning exercises, fund-raising, and implementation on their own. CARE must also work to affect the environment in which community-driven development can take place. This means advocating to raise awareness of government authorities of the structures de development and the existence of the PDQs as an initial output of these new community structures, and its usefulness in their own planning activities. As CARE did with the implementation, testing, and documentation of FAMOA, Mahavita should develop a strategy for supporting the priorities that result from the participatory planning process. Staff can employ a reflective process to experiment with approaches and document how to “keep the PDQ alive.” Supporting the Elements of CDD Even as Mahavita must develop its support of the SD and the PDQ further, Mahavita also needs to develop a strategy to strengthen the other components necessary for successful community￾driven development. Specifically, Mahavita now has to turn its attention to developing the capacity and mindset of government authorities to embrace and respond to community-driven needs and demands; and to the mechanisms that link community and governance structures. CARE Madagascar: Title II DAP Final Evaluation Page 53 Although managers note that strengthening these components is a significant activity under the new DAP, more could have been done earlier. Many communities have had their final PDQs for a year or two. Although some CARE staff have worked to help the SDs find funding; and SDs have sometimes actively sought funds themselves, this is not a concerted, coordinated, or strategic effort to develop community capacity and keep the PDQ “alive.” Outcomes in the form of new projects or financing are limited. SDs certainly do not feel they have developed any insight into how to find funding; staff are not sure of how to interact with the SDs once the PDQ is done (many feel it is entirely up to the SD to work out how to implement the PDQ); and staff are not sure of where to turn for advice or information on funding opportunities within Mahavita. At the risk of putting staff and partners through yet another change, CARE should closely examine Mahavita’s organizational structure to ensure it supports these strategic thrusts. CARE must work on the “supply side” of government to support CDD. Government officials must strengthen their understanding of their responsibilities as public servants of all citizens and of their roles and their roles in a decentralized, democratic system (which remains to be clarified). Given limited financial and staff resources, government must also plan strategically to make the best use of them. Right now, CARE’s effort in this important area is weak. Only one person is responsible for institutional strengthening and advocacy beyond the community level. Formal and Informal Capacity Building Building capacity involves not only formal training but informally promoting knowledge exchange and strengthening institutional structures and administration. CARE’s strategy here is not clear to partners or staff. Staff were not sure whether “training” was independent or part of specific interventions; nor could they articulate content or purpose well. Some SD members and FKT authorities did mention training, but most were not sure whether the SD had received any training – and these community leaders should be key targets for institutional strengthening. In any case, training in issues like democracy and good governance as well as project management is clearly not a sustained or strategically targeted effort within Mahavita. Clarifying Roles of the SD and the FKT Authorities The institutional distinctions between the SDs and the FKT authority are not clear to staff and to stakeholders (although they can be clear about the different roles and responsibilities). In the communities themselves, the main distinction that has emerged is that the FKT authority is the “administrative” agency and the SD the “implementing” agency. But whether then SD responds directly to the FKT authority seems to vary by fokontany. In this exploratory exercise, we found that in fokontanies that staff said “worked best” the FKT authorities seemed to believe that the SD was essentially under their control (a concept reinforced by the fact that many of those interviewed held leadership positions in both). Those fokontanies that staff said “had problems” were those with political conflict between the two. Interviews with SD and FKT authority members verified that the two institutions saw themselves as being independent of one another. (Note that conflict could still arise in these fokontanies even if people held positions in both, because the “institutional conflict” could relate more to specific individuals.) Some confusion may have arisen during interviews because it was difficult to clarify exactly what was meant by the FKT authority “controlling” or “regulating” the SD (because, in a broad sense, any government authority would do exactly that over any private or non-governmental CARE Madagascar: Title II DAP Final Evaluation Page 54 entity). But the significant overlap between the leaders of both and the reports of staff of political conflict confirms that this is a genuine issue, if not yet a problem. The exact relationships that currently and should exist need to be investigated further. To now, Mahavita has not tried to clarify the relationship between the SD and the FKT authorities for two main reasons. First, Mahavita likes to see “what develops,” and let communities go their own ways, instead of imposing guidelines. Second, trying to separate FKT authorities from the SDs may have created political problems for Mahavita right from the beginning. So Mahavita has actually encouraged FKT authorities to be part of the SD. Mahavita’s start-up strategy involved not only getting FKT authorities’ approval of the project but essentially naming the core FKT authorities as the initial ad hoc SD. Later, other influentials were added to the SD, but the core members began with (and often remain) the FKT authorities. Although this encourages initial FKT approval, in the longer term it runs a high risk that the SD effectively becomes simply another, probably inert, government agency and those political elites or other influentials will capture the SD for their own party or personal purposes. The focus groups did not look into the issue of whether this is already beginning to happen, or whether the SD is truly a voice of the community, and especially of the poor. This is clearly an important area for future research. The Scope of Responsibility of the SD for Community Planning The relationship of the SD to the planning process and other NGOs is another issue that needs clarification. As noted, most actors currently see the SD as having the responsibility for planning and implementing community development. This implies that activities of other NGOs should also fall under its purview. And FKT authorities and SD members agreed that FKT authorities generally told any NGO that wanted to work in the area to first see the SD. CARE staff have been careful to stress that the SD is a creation of the community, and not of CARE. Still, CARE’s role in setting up the SD is undeniable. So far, however, other organizations do not seem to have objected to the SD’s assertion of this authority over them, even though it is entirely unofficial. This may be for a number of reasons: that this is a theoretical and not actual power and the SD makes no attempt to exert this authority (in fact, no one actually mentioned an instance where the SD was taking on responsibility beyond simply asking to be informed); no NGOs exist in the area, or they are not particularly strong; CARE staff work with other NGOs to help them “understand” why and how they should coordinate with the SD; the NGO has a community contact who is also a member of the SD and who supports central coordination; or the NGO ignores the SD. For now, the SD seems to be playing this role without objection, but that situation is likely to change once more resources are at stake. The SD actually has no official authority to plan or to supervise and coordinate the activities of others. The FKT authorities would. As the SD works to implement its PDQ, how these relationships with the FKT authorities and other development organizations in the community play out become critical to the successful implementation of the PDQ and to the very conception of what the SD is and from where it derives its power. For now its authority rests on (perhaps) perceived competence and filling an important need. Those are CARE Madagascar: Title II DAP Final Evaluation Page 55 fragile bases. Mahavita needs to consider how to develop the SD institutionally or risk later disintegration of the SD. Formalization of the SD One potential model is to give the SD formal stature of some sort in the coordination and planning of community development. Legal statutes could recognize the SD as the CBO within the community that gives voice to residents during the government’s planning process (not that this would preclude other individuals or organizations from speaking up). But the government could formally require planners to give the SD’s point of view “great weight.” In addition, if the SD continues to produce high-quality PDQs, more and more government officials may see the advantages of working with the SD: to legitimize their actions as representing the community and to make their own work more efficient by building on the PDQ in making plans at higher area levels (the arrondissement, the commune). In fact, many SDs are already establishing their organizations officially. Although CARE staff are often encouraging and advising this process, Mahavita has no formal policy or guidelines regarding formalization. As of now, the process and Mahavita’s support of it is ad hoc. Staff are proceeding without any clear idea of what they are doing or how to do it. For such an important issue, CARE needs to consider the pros and cons of various institutional arrangements, assess the “experiments” that are already going on, and decide on an organizational strategy. Confusing Institutional Environment. The governance environment above the level of the fokontany further confuses the situation. Currently Madagascar has two parallel structures for planning at the municipal level (the Prefet and the mayor’s office), limited resources and confusion over who controls those resources, and no policy that outlines responsibilities for administrative structures below level of the municipality. For CDD to work effectively, the community must know who the key decision makers who affect their lives are. The PDQ will remain an effectively dead document despite the best community efforts if the SDs and FKT authorities do not know how it “plugs into” the planning process of upper-level authorities. More importantly, if the process is muddled, those in favor of more robust democratization and good governance will have a difficult time figuring out what regulations and which people are the key ones to target in advocacy efforts. Scaling up reliably and efficiently must take place where actors share understanding of a clear institutional framework and agree on institutional roles. The strategy, of course, must take into account the fact that SD members come and go, and that government authorities, from the FKT level up, change often. Expansion: Other Constraints and Possibilities Under the next DAP, CARE foresees expanding from the 30 fokontanies where it now works to 80. But there are 192 fokontanies in Antananarivo. It also plans to expand to Fort Dauphin. Although this expansion will not happen overnight, and it can proceed with expansion to the other areas using the FAMOA approach in which it has confidence, CARE has not completed “the process” in the first 30’; it must do so to ensure it leaves a sustainable CDD process in place. As emphasized above, it needs to bring its support for CDD full-circle and work on the “supply side” and the institutional environment to ensure that all the elements of successful CDD CARE Madagascar: Title II DAP Final Evaluation Page 56 are in place. In accomplishing its goal, CARE will have to test and document approaches (including an exit strategy), and build capacity among staff and stakeholders. CARE should also think about how to work with communities, other NGOs, and government to expand the reach of CDD within Antananarivo. Does CARE share its approach with other national and international NGOs? Can other approaches to CDD be contained within CARE’s or can they be synthesized? Does the government support the establishment of SDs in other fokontanies? Can SDs and fokontany authorities themselves build networks among themselves, where all share experiences and the strongest and most capable lead the establishment of CDD in other areas? CARE should build on its own knowledge and experience with these communities to maintain its focus on establishing and building the capacity of the SDs and shaping the enabling environment. This is CARE’s comparative advantage. This should not be the responsibility of other international or local NGOs. Their intervention as a “replicator” of FAMOA would largely only add another layer of organizations among which CARE would need to build capacity, as many NGOs have program-specific expertise (such as housing or health). Rather, they should be seen for what they are: additional sources of financing and program expertise (usually revolving around very specific activities such as housing or health) that the SD can tap into to realize its own PDQ. CARE can also continue to build capacity among the SDs, who can themselves, rather than NGOs, replicate the model through sharing their experiences with other FKTs. As CARE scales up, it must seriously consider the impact on staff. The numbers in the DAP are certainly guides, but management must be careful not to expect too much too soon. This is not a lowering of expectations, but a realization that working with SDs and other stakeholders to establish common visions, institute structures, and resolve conflict is not easy and requires a significant investment of time. Supporting true institutional development of the other stakeholders and changing the institutional environment are activities new to CARE. CARE must focus on going slowly and leaning as it goes, to ensure sustainability. Management should carefully monitor staff to see how things go and ensure sustainable structures and processes are in place before moving on. Staff, SDs, and FKT authorities consistently mentioned that one of the most difficult tasks in promoting the FAMOA process was that community residents wanted immediate actions and works. Such changes take time, and now it will not only try to do this in an additional 50 communities but will take on the task of changing the mentalities of authorities in the 30 where it has already worked. If CARE simply sees the next DAP as expanding the current approach, with its focus on the PDQ as its output, and does not spend time to “complete the circle,” it will fail in its efforts to establish sustained community-driven development in any of the 80 FKTs. Part of moving in this way is establishing the appropriate benchmarks for CDD in its role as a facilitator. Certainly CARE will have to respond to the standard operating indicators of donors (although over time I believe a coalition of donors, research organizations, and development agencies, including NGOs, will move donors like USAID to modify these so that, for facilitating organizations, they focus more on outputs related to process than to global development indicators). But this should not limit CARE’s efforts to developing its own indicators that it CARE Madagascar: Title II DAP Final Evaluation Page 57 needs for its own management purposes anyway. CARE has apparently already moved somewhat down this path in its next DAP. CARE can continue to innovate in this area. And again, it makes sense just from its own internal management perspective to have appropriate indicators to measure its performance and the performance of other stakeholders and partners. A significant challenge will be ensuring that Mahavita maintains the rhythm of developing and solidly establishing CDD. CARE is leading this effort. Much of its success depends on being seen as a credible partner by the various stakeholders. If it fails to outline a process and keep its commitments, SDs, government authorities, and residents will lose interest and drift away. CARE has yet to clearly outline its final goal (after the exit) for communities and authorities, and share its strategy to achieve that (strengthening all the elements). Although all communities feel they have developed a sense of solidarity and confidence in their ability to act through the FAMOA process, if CARE does not lead the further development of CDD, it risks losing that goodwill and the building blocks, already constructed within the community, for CDD. Some incidents already raise flags: CARE’s limited support for building institutional capacity and designing strategies to implement the PDQ that leaves the communities rather stuck; some mention among SDs and FKT authorities that in carrying out projects CARE technicians made decisions without consulting the community leaders or explaining their reasoning to them; and others who commented that sometimes, deliveries of construction inputs were late or never arrived. Building community relations is essential to the work of establishing CDD, but it is a fragile process. Experiences elsewhere show that building and maintaining credibility among these vulnerable communities is key. They have had hopes dashed when working with outsiders and experienced the disdain of those who are wealthier and more powerful. CARE promises something different. For CARE to function effectively as a change agent, it must keep that promise. Scaling Up: A Checklist Experiences of scaling-up highlight a number of key issues that affect the nature and success of attempts to scale up (Gillespie 2002). This review of Mahavita points out the importance of some of these issues. Community-driven development often happens as a result of some outside catalyst. Here the need to develop a DAP was a trigger for development of Mahavita. Upon her arrival, Lisa Dean, the CARE country director, found a fairly dysfunctional country office with a crying need for funds. Individuals in authority with vision are often key to taking advantage of this opportunity. This was the case with Mahavita as well. Michael Drinkwater, CARE’s regional adviser, transferred community-driven experiences from Zambia to Madagascar. Chris Dunston joined Mahavita and continued the focus on community and staff empowerment, emphasizing the importance of establishing processes, of flexibility in implementation, and of learning. The political, institutional, cultural, and institutional environment affects the ability to which these individuals can realize their vision. Here the government was relinquishing some control (a legacy of a socialist past) and emphasizing decentralization and community participation. With significant education efforts on the part of CARE, USAID was permitting the project to go CARE Madagascar: Title II DAP Final Evaluation Page 58 forward, albeit often complaining about the need for more concrete outputs. The environment thus gave space to the efforts that CARE was making. Changes in that environment can limit or promote community-driven development, so CARE will need to continue to pay attention to the legal and institutional challenges now found in Madagascar. The capacity of stakeholders is a principal determinant of success. Somewhat delayed, Mahavita is just beginning to confront this constraint. CARE staff themselves need to have a more complete idea of their actions in establishing community-driven development, beyond the PDQ. The community structures must also be strengthened, in concrete and philosophical ways. As many of those interviewed mentioned, changing the mentality of community residents to accept responsibility to be major players in their own development is a major undertaking, and as essential to success as ensuring the structure de development has the technical, administrative, and financial capabilities to function effectively. The building block of scaling up, however, is the development of empowering processes that can be used to guide expansion. CARE has made a good start with the development of FAMOA, but it must complete the circle. Community-driven development is the objective, not the PDQ. And in this context, key questions remain. How will CARE really operationalize the SD, ensure it represents the voice of the community and link it to outside resources and structures? What will it do to develop capacities and shape a stable, enabling environment? The key constraint on scaling up seems to be that CARE still needs to develop and test what processes and strategies will most successfully do these things. References Dongier, P., J. van Domelen, E. Ostrom, A. Ryan, W. Wakeman, A. Bebbington, S. Alkire, T. Esmail, and M. Polski. 2002. Community-Driven Development. World Bank. Mimeo. Gillespie, S. Scaling Up Community Driven Development: An Overview. 2002. IFPRI. Mimeo. Uvin, P. 1995. Fighting Hunger at the Grassroots: Paths to Scaling Up. World Development 23 (6): 927-939 CARE Madagascar: Title II DAP Final Evaluation Page 59 ANNEX B Evaluation Plan / Scope of Work CARE Madagascar’s Title II Development Assistance Program (DAP) 1. Background and Purpose of the Evaluation CARE Madagascar’s DAP 1998-2002 – Transfer Authorization (TA) Award No. FFP-G-00-98- 00060-00 – was granted a ‘no-cost extension’, through the approved FY 2000 DAP Amendment, whereby advancing the official start-date by one year, to FY 1999 (instead of FY 1998), subsequently extending the life-of-activity (LOA) to the end of FY 2003. CARE Madagascar established two programs out of the DAP: Mahavita and CYPREP. Program Mahavita (Urban Household Livelihood Security) is addressing Strategic Objectives 1 and 2. Program CYPREP (Emergency Preparedness) is addressing the Special Program Objective. Program Mahavita Program Mahavita’s actions address food security in urban areas of Antananarivo by sustainably increasing the access of beneficiaries (poor urban households) to food, and by sustainably improving the utilization of food by beneficiaries. Improved access to food is obtained by offering income generation opportunities to project beneficiaries. This will allow them to purchase more food, and to build up assets. Asset buildup will help households overcome periods of difficulties. Improved utilization of food is fostered by protecting people’s health, via the construction of sanitation infrastructure, and the training of beneficiaries in improved sanitation, health and nutrition practices. When the DAP began implementation (October 1998), TOUCH 2000, a USAID funded child survival project had just completed its mid-term evaluation. The evaluation team strongly recommended that the two projects integrate activities and build in the results already achieved by TOUCH’s 2 years of implementation. TOUCH was successful in creating strong community support and motivation towards child survival initiatives. It was clear that the DAP could benefit from TOUCH’s efforts and lessons learned, particularly in terms of community mobilization and institutional capacity building methods. For example, most of the CACs had been very active, planning and carrying out health promotion activities. Also, through a number of participatory workshops, TOUCH was successful in creating a strong partnership with the Ministry of Health, particularly with those supervising and staffing health clinics. On the other hand, TOUCH initiatives would also benefit from DAP’s efforts in environmental health and personal empowerment objectives. Hence, the two projects were combined. CARE Madagascar: Title II DAP Final Evaluation Page 60 2. Program Overview Mahavita’s baseline survey was conducted in March/April 2000. This survey included all the TOUCH results indicators. The final evaluation will include TOUCH activities as well and is planned to begin in February 2003 and conclude in September, involving 3 components. Two strategic objectives, which pertain to Mahavita activities, will be evaluated. There are as follows: Strategic Objective 1: Protect and promote the food and livelihood security of 10,000 urban households. SO 1 has two sub-objectives: 1) Income Generating Activity, and 2) Institutional Capacity Building. Result (impact, effect and monitoring indicators are as follows: 1) Income Generating Activity • Index of key assets owned; • Average number of days a HH worked over previous two weeks; • Number of HHs who participated in FFW programs; • Number of HHs who received Personal Empowerment Training (PET). 2) Institutional Capacity Building • Number of Structure du Development' (SD) operating sustainably; • Proportion of infrastructure maintenance costs covered by public; • Number of paid work-days generated by SD activities; • Number of SD established; • Number of SD with a Community Development Plan (CDP) in place; • Number of SD that have negotiated projects; • Number of SD that have completed projects. Strategic Objective 2: Sustainable improvement in environmental health in 30 Fokontany for 200,000 people in urban Antananarivo. SO 2 has two sub-objectives: 1) Sanitation Infrastructures, and 2) Health and Nutrition Practices. Result (impact, effect and monitoring) indicators are as follows: 1) Sanitation Infrastructures • Improved communication within fokontany; • Proportion of canals rehabilitated; • Number of HHs affected annually by floods; • Percentage of HHs using safe water storage container; • Number of people per water point; CARE Madagascar: Title II DAP Final Evaluation Page 61 • Number of HHs per adequate garbage collection facilities; • Number of HHs per sanitation (toilets) and hygiene (bathing) facilities; • Percentage of HHs using latrines for human waist; • Meters of canal rehabilitated; • Number of public water points established; • Number of public sanitation (toilets) and hygiene (bathing) facilities established; • Number of garbage collection points established; • Number of washing basins established. 2) Health and Nutrition Practices • Percentage of population experiencing diarrhea 2 weeks prior to survey; • Household dietary diversity; • IEC sessions on proper infant feeding practices; • Safe Water System Campaigns; • Number of HHs adopting Safe Water System Practices. From January – August 2002 Madagascar experienced a political crisis due to elections. During this time road blocks were common and fuel and food could not reach the capital Antananarivo. This resulted in increases of food and loss of jobs. Due to the capacity which existed throughout the operational zones of Mahavita, we were able to double our infrastructure efforts and programmed significant more food and cash through safe net programs. The objectives of CYPREP were designed as a Special Project Objective: Safeguard the livelihood security of Madagascar's most cyclone-vulnerable communities by enhancing local capacity to plan for and cope with emergencies. This objective involves two sub-objectives: • Number of villages that have a 'Disaster Preparedness Action Plan' in place – In other words: A 'Disaster Preparedness Action Plan' IN PLACE is defined as a village who has completed the planning process, has a formal 'Disaster Preparedness Committee', a 'Disaster Preparedness Action Plan' and implemented at least one mitigation project e.g. school, grainery, socially marketed products mobilization and sales campaign. • Number of villages that have developed a 'Disaster Preparedness Action Plan' – In other words: A 'Disaster Preparedness Action Plan' is defined as a village that has completed the planning process with an informal committee. It was not necessary to conduct a baseline survey for CYPREP, due to the nature of the activities and the fact that the committees and mitigation strategies didn’t exist before. Due to the crisis, CARE had to decrease some of its activities do to the political problems throughout Tamatave and the lack of fuel. 3. Analytical focus of the evaluation Mahavita's community-drive development approach was designed towards building institutions capacity to diagnose their problems and plan to solve them. The sustainability of the strategies is CARE Madagascar: Title II DAP Final Evaluation Page 62 sought by organizing community development structures that generate employment around the satisfaction of public needs (namely, the construction and maintenance of public sanitation infrastructures). It is expected that the public’s satisfaction with the services provided will create a willingness to pay for the continuation of those services, thus maintaining a demand for the jobs created by the project, and a means to pay for those jobs. The objectives of offering income generation opportunities and of protecting health are thus closely related, and are mutually supportive. Questions: 1. What has been Mahavita’s experience so far in expanding this approach to other communities while phasing out its assistance to the initial communities? 2. What have been the key challenges and elements of success, especially in terms of building community capacity, addressing constraints in the institutional environment, taking an intensive pilot project to larger scale, and developing a strategy around processes and structures rather than projects? 3. In the end, what do these experiences suggest that CARE and like organizations should do to encourage and expand sustainable community-driven development? 4. What is the community members (households, and municipality) level of satisfaction of services performed by the SDs and their willingness to pay for their services. The household livelihood security baseline survey, conducted in March/April 2000, resulted in a quasi-model of urban poverty. Data collected about household asset ownership resulted in the calculation of a new variable representing total household assets. The data supports the hypothesis of asset accumulation serving as a proxy for household wealth. The asset ranking correlates well with variables such as living space (in square meters), number of rooms in the house, average expenditures for food and medicine, and education level. There was surprisingly less correlation with other variables, in particular those associated with health and hygiene behavior, number of days absent from work, and vaccination of children. There was evidence that suggests health behaviors are similar, though not exactly the same, across asset classes. There is also some health evidence to confirm the unhealthy environments cut across social classes, such as the percentage of each class reporting at least one case of cold/flu in the household. As a result, the survey identified several implications for urban programming which should be addressed in the evaluation; what efforts have been made to learn more and validate these programming implications and how activities have effected change. They are as follows: 1. The project should explore opportunities for keeping kids, especially adolescents, in school in order to improve their future chances of gainful employment. 2. The project should conduct focus group interviews to gain more in-depth knowledge about what drives diet diversity. The focus group discussions can target fruit, egg and dairy consumption to determine the consumption dynamics of these food items. The role of street foods in family nutrition should also be explored. CARE Madagascar: Title II DAP Final Evaluation Page 63 3. The project should explore the reason behind high food expenditures. The high consumption of street foods may be a contributing factor, but little is known at present about the relative costs of eating at the home or outside of the home. Little is also known about the nutritional costs and benefits of urban Antananarivo eating habits. 4. Cash for Work (or a mix of Cash for Work/Food for Work) programs should target the unemployed (in this survey 7% of all households) and those occupied in petty trade who live day to day in hopes of selling articles on the street. It should also take into consideration those Fokotanys where unemployment may be highest. 5. According to survey results, 30% of all households in the project area are headed by a woman. Analysis also suggests that these households are among the poorest. Project interventions that target and benefit these households should be considered. 6. Respiratory illness is high among all asset groups, suggesting that the urban environment does not discriminate among wealth rankings with respect to this health concern. However, households do vary in their behavioral responses to illness, with better-off households spending more money on health care and accessing better health services than other households. The environmental health component involves social and technical activities. The evaluation will need to address the following issues: • Engineering aspects (pre-post surveys of canal, walkway rehabilitation) • Materials and equipment management • Food management • Ration size and contribution to household nutrition • Life skills training courses • Income generating activities 4. Proposed Evaluation Methodologies The evaluation will start with the qualitative assessment of the community driven development approach. This will be in the form of focus group discussions, individual interviews and result in a case study as well as a document providing a description of each issue and provide recommendations. The second component of the evaluation will be the post-test of the quantitative household survey conducted in March/April 2000. The sampling frame must be reviewed and designed so that the results will be able to be disaggregates by Fokontany. It will be important to look at the ‘level of effort’ invested by each Fokontany and compare that with the changes in household livelihood security. The questionnaire used before will be reviewed and edited, making adjustments according to the level of significance in baseline monitoring results. The survey will be conducted by a local consulting firm, providing the final data set and preliminary analysis (cleaned data set, completed questionnaires, means, cross tabs, etc.). TANGO International will be responsible for doing the final analysis and report writing. CARE Madagascar: Title II DAP Final Evaluation Page 64 The third component will be to evaluate the Environmental Health activity that involves cash/food for work to rehabilitate canal and walkways, and water provisioning through cost￾recovery systems. This will mainly be a tabulation of monitoring reports to determine the final numbers of food for work participants (man-days), kilometers of canal, walkways, etc., number of trainings, employment days generated, income earned and capacity of SD’s to manage/sustain these services. This will also involve a thorough review of the systems and procedures used in planning, implementation and monitoring activities related to civil engineering, materials and equipment and food management. The effectiveness of the cost-recovery systems will also be evaluating. This will be primarily quantitative; however, it will also include qualitative methods to evaluate the organizational and management aspects of the cost-recovery systems. To determine the effects of the Special Program Objective (CYPREP) an external evaluation team will be recruited to determine whether the results have been achieved. This will primarily involve a quantitative review of the anticipated outputs stated in the original agreement. It will also involve a qualitative sample of Community Cyclone Preparedness Committees to determine the ‘state-of-affairs’ of the plans and outcomes related to mitigating activities. CARE Madagascar: Title II DAP Final Evaluation Page 65 ANNEX C Persons Contacted FRIEDEBERG Tom CARE/Madagascar Country Director DANIEL Jean DAP Coordinator RAMANARIVO Leandre Chef de Projet Mahavita ANDRIAMISANDRATRA Jules Chef de Departement, Institutional Capacity Building Dr. RALAMBOSON Bodosoarivelo Coordinatrice Sante, Projet Mahavita HOLIHASINORO Sabine Coordonnatrice de Zone/Responsible Renforcement de Capacite RATOARIJOANA Avo Assistante IEC RAHERIJAONA Hanitra ADC Joelle ADC Ambilanibe ANDRIAMAHEFARIVO Landy Responsable des Infrastructures Generatrices de Revenue RAKOTOARISOA Naina Responsible des Infrastructures Francia Assistante de Volet Infrastructure RAMANANDRIMALALA Rija Agent de Terrain, Mandrangobato II RAVELOMANANTSOA Solofo ADC Ankasina, Antaniavo, Andrahavo RABEKOTROKO Fidy ADC Tsaramasay RAKOTOLANDRY Lalaine Coordonnateur de Zone/Responsible Renforcement de Capacite Members Structure de Developpement: Tsaramasay Andavamamba Antetezana Centre I Ankasina Dr. RAMANANDRAIBE Lucie Chef de Service, Direction de Soins Medicaux et d’Appui aux Districts Sanitaires Dr. RAVAOMANANA Anne Marie Inspecteur de Sante, Antananarivo Dr. RAKOTOARIMANANA Hanitsoa Medecin Chef, CSB de Isotry Dr. RAHELIMALALA Robertine JSI-RTI Wendy BENAZERGA USAID/HPN Pedro CARILLO USAID Food Security and Disaster Manager Chris DUNSTON former Mahavita DAP Coordinator CARE Madagascar: Title II DAP Final Evaluation Page 66 ANNEX D List of Antananarivo Fokotanys Assisted by CARE’s Title II Program Arrondissement I: Ankasina Antohomadinika FAAMI Antohomadinika Nord Antaniavo Antohomadinika III G Hangar Antohomadinika Centre Antohomadinika Sud Antetezanafovoany I Antetezanafovoany II Manarintsoa Anatihazo Ambalavao Isotry Andavamamba Anatihazo I Andavamamba Anatihazo II Andavamamba Anjezika I Andavamamba Anjezika II Andohatapenaka I Arrondissement III: Ankazomanga Nord Tsaramasay Ankorondrano Ouest Ankorondrano Andranomahery Arrondissement IV: Ivolaniray Andavamamba Ambilanibe Anosibe Ouest II Anosibe Mandrangobato I Anosibe Mandrangobato II Anosibe Angarangarana Madera Namontana Est Anosibe Ambohibarikely Ouest Mananjara Arrondissement VI: Andraharo Ankazomanga Sud CARE Madagascar: Title II DAP Final Evaluation Page 67 ANNEX E Sample Sites from the Qualitative Survey - CYPREP Zone Sous-Préfecture Commune Fokontany Nombre de types d’activités de mitigation Activités de mitigation Tamatave II Tamatave Suburbaine Ranomena 1 Abri anti-cyclone Brickaville Ambodifaho 2 - Aménagement - Adduction d'eau - Andovoranto Andovoranto 0 - TAMATAVE Brickaville Mahatsara Vohiboazo 1 - Pistes Mahanoro Sahabe 2 - Aménagement - Pistes Tsaravinany Taviranambo 2 - Abri anti-cyclone - Pistes Ambodiharina Maromitety 1 - Adduction d'eau MAHANORO Mahanoro Masomeloka Valokianja 0 - Fénérive-Est Ambodimanga II Ampasimpotsy 2 - Reboisement - Pistes Marofinaritra 2 -Grenier - Pistes Vavatenina Mahatera 2 - Adduction d'eau - Pistes Vavatenina Anjahambe Anjahambe 4 - 2 abris anti-cyclone - 2 greniers -reboisement - Pistes Tamatave II Foulpointe Marofaria 0 - FENERIVE￾EST Soanierana Ivongo Soanierana Ivongo Tanambao Matsokely 1 - Abri anti-cyclone CARE Madagascar: Title II DAP Final Evaluation Page 68 ANNEX F Details of Qualitative Research Visits Zone Tamatave Date Lieu Personnes rencontrées Observations Lundi 18 août matinée Tamatave II Sous-Préfet Absent Tamatave Sub-Urbaine Maire Bureau CARE Tamatave AMS après-midi Ranomena CVS Bénéficiaires Mardi 19 août matinée Brickaville Adjoint du Sous-Préfet Maire et Adjoint au Maire Mahatsara Maire et Adjoint au Maire après-midi Vohiboazo CVS Bénéficiaires Mercredi 20 août matinée Andevoranto Adjoint au Maire CVS Bénéficiaires après-midi Ambodifaho CVS Bénéficiaires Jeudi 21 août matinée Bureau CARE Superviseur Tamatave Zone Mahanoro Lundi 25 août matinée Départ Mahanoro Mahanoro Sous-Préfet et Adjoint au Sous-préfet après-midi Sahabe CVS Bénéficiaires Mardi 26 août matinée Tsaravinany Maire Taviranambo CVS Bénéficiaires après-midi Mahanoro Maire AMS Mercredi 27 août matinée Ambodiarina Maire Maromitety CVS Bénéficiaires après-midi Trajet Masomeloka Jeudi 28 août matinée Valokianja CVS Bénéficiaires après-midi Retour Mahanoro Bureau CARE Superviseur Mahanoro Bureau CARE Maire de Masomeloka Vendredi 29 août matinée Bureau CARE Mahanoro Coordinateur Programme Mahanoro Administration après-midi Samedi 30 août matinée Bureau CARE Mahanoro Restitution après-midi Retour Tamatave CARE Madagascar: Title II DAP Final Evaluation Page 69 Zone Fenerive Est Date Lieu Personnes rencontrées Observations Lundi 01 sept matinée Foulpointe Maire Marofaria CVS Bénéficiaires après-midi Fénérive Sous-Préfet Absent Ambodimanga II Maire Fénérive AMS Mardi 02 sept matinée Vavatenina Deux Adjoints du Sous-Préfet Maire Anjahambe Maire après-midi Anjahambe CVS Bénéficiaires Mercredi 03 sept matinée Marofinaritra CVS Bénéficiaires après-midi Mahatera CVS Bénéficiaires Jeudi 04 sept matinée Tanambao Matsokely CVS Bénéficiaires après-midi Soanierana Ivongo Sous-Préfet Adjoint au Maire Vendredi 05 sept matinée Ampasimpotsy CVS Bénéficiaires après-midi Bureau CARE Fénérive Tovo : Ancien superviseur de Tamatave Superviseur Fénérive Samedi 06 sept matinée Bureau CARE Fénérive Restitution après-midi Retour Tamatave Dimanche 07 sept après-midi Tamatave Présentation du déroulement des travaux à Richard Caldwell Lundi 08 sept matinée Tamatave Réunion avec l'équipe de recherche quantitative Lundi 15 sept journée Tamatave Interview du Chef de projet Mardi 16 sept matinée Tamatave Administration Tamatave après-midi Interview Coordinatrice du Programme Est CARE Madagascar: Title II DAP Final Evaluation Page 70 ANNEX G CARE / PROGRAMME MAHAVITA EVALUATION DES TRAVAUX D ' INFRASTRUCTURES SOCIALES REALISES SELON LE PROGRAMME MAHAVITA A. INTRODUCTION - Financement : USAID - Gestion du projet : CARE INTERNATIONAL / PROGRAMME MAHAVITA - Objectifs : Evaluation des travaux d'infrastructures sociales réalisés selon le programme MAHAVITA - Zone d'intervention : Quartiers bas où la majorité de populations est pauvre - Mission : Apprécier le niveau d'atteinte des objectifs relatifs à l'amélioration des infrastructures communautaires - Evaluer les capacités techniques des partenaires Mahavita - Evaluer l'organisation de l'unité de programme Mahavita en charge des aspect d'engineering - Recommandations nécessaires pour la suite du programme Demarche de l'Evaluation - Un programme de visite du 23/09/03 a été réalisé avec les Responsables CARE sur les Fokontany suivants: Fokontany Infrastructures Mandrangobato I Ruelles Mandrangobato II Diguettes Anosibe Ouest II Borne fontaine et bloc sanitaire Ambilanibe Bassin lavoir Anatihazo II Bassin de rétention - Evaluer les ouvrages réalisés suivant les plans fournis par le Responsable des Infrastructures - Les programmes des travaux réalisés par les communautés sont : 1°) Bassin de retention 2°) Aménagement diguettes 3°) Ruelles en béton 4°) Passerelle en bois B. NIVEAU ATTEINT SUR L'AMELIORATION DES INFRASTRUCTURES COMMUNAUTAIRES B1. Bassin de Retention - Il est à signaler que les bassins sont réalisés en collaboration avec L ' APIPA et les techniciens du Génie Rural CARE Madagascar: Title II DAP Final Evaluation Page 71 - Prestation de l 'APIPA avec les communautés : - Fourniture de camion pour le Transport de vivres et évacuation des produits de curage - Fourniture de pompe Bassin de Retention a Anatihazo II - Superficie du bassin réalisée : 60 000 m2 - Objectif des travaux : Stockage des eaux pluviales pour lutter contre l ' inondation - Nombre de fokontany bénéficiaires : 15 dont - Anatihazo I et II - Ambilanibe - Anjezika I et II - Antohamadinika Sud - Manarintsoa Anatihazo - Ivolaniray - Anosibe Ouest II - Ambohibarykely - Angarangarana - Mandrangobato I et II - Madera - Ouest Manjara Qualité des travaux : - Profondeur du bassin : dans le norme - Evacuation des produits de curage : Une partie non réalisée (à la charge de l'APIPA) - Absence des panneaux pour les travaux à réaliser et de signalisation Diguettes côté Nord du bassin du de rétention: - Matériaux utilisés : Produits de curage en tourbe vaseuse plus ou moins argileuse - Compactage : quasiment mal compacté permettant le passage des eaux pluviales à travers la diguette à cause des retraits de l ' argile. - Absence de mottes de gazons sur la pente du talus - Pente des talus non respectée - Quelques buses non encore placées sur leur emplacement prévu - Couche de revêtement instable dans l'ensemble Recommandation 1. L'instabilité de la diguette peut provoquer pendant la période de crue de problèmes sur le bassin ainsi , pour éviter l'émergence de brèche au niveau de la diguette et l'instabilité du talus , il faut implanter au niveau de chaque pente de talus des mottes de gazons 2. Mettre les buses avant la période de pluie sur les emplacements prévus 3. Bien suivre le plan de maintenance CARE Madagascar: Title II DAP Final Evaluation Page 72 Dans un prochain avenir : 1. Apport des matériaux convenables pour avoir un bon compactage (remblai Latéritique). 2. Pente à respecter pour assurer la stabilité. 3. Engazonnement par mottes de gazons avec fixation à l'aide de piquets. 4. Bien maîtriser les ouvrages d'équilibre. 5. Evacuation des déblais de curage. 6. Bourdure du bassin planté en vetiver. B.2 Evaluation Quantitative et Estimative de Bassin de Retention Superficie du bassin (m2) : 60 000 m2 Quantité de déblai à enlever 30 000 m3 Délai d'exécution 50 j Nombre de MO 2600 Rendement 0,23 m3/j Quantite Prevue et Reellement Consommee DESIGNATION UNITE QUANTITE PREVUE QUANTITE CONSOM. Nombre MO u 2 600 1 305 Total Hj Hj 130 000 107 000 Nombre Quartier u 7 15 Outillages Panneaux u 4 0 Petits outillages MO 2600 2600 Matériaux Buse D300 u 28 28 Ciment Sac 24 24 Sable M3 10 10 CARE Madagascar: Title II DAP Final Evaluation Page 73 Cout des Travaux d'Amenegement du Bassin de Retention DESIGNATION UNITE PU QUANTIT E CONS. MONTANT Total Hj Hj 11000 107 000 1 177 000 000 Outillages Panneaux u 250000 0 0 Petits outillages MO 10000 2600 26 000 000 Matériaux Buse D300 u 75000 28 2 100 000 Ciment Sac 60000 24 1 440 000 Sable M3 30000 10 300 000 TOTAL 1 206 840 000