1 Resiliencia- Strengthening Capacities for Disaster Management in Vulnerable Communities in Azua Province, Dominican Republic External Final Evaluation Report USAID/OFDA PLAN DR Santo Domingo January 2012 2 Lourdes Meyreles Consultant Víctor Ruiz Assistant 3 Ubicación del proyecto 4 Table of Contents Página I. Presentation ......................................................................................................................... 6 II. Project Description.................................................................................................................7 III. Participating institutions and organizations......................................................................... 11 IV. Purpose and Context of the Evaluation................................................................................ 12 V. Evaluation Criteria................................................................................................................. 13  intervention methodology and strategic pertinence  efficacy  efficiency  equity  sustainability and replicability  institutional relations VI. Objectives and Scope of the Evaluation...............................................................................14 VII. Evaluation Methodology ............................................................................................ .......16 VIII. Evaluation Results .............................................................................................................20 1. Intervention methodology and strategic pertinence 2. Efficacy and efficiency  Achievement of project objectives and analysis of project results  Risk and development: development agendas, disaster risk reduction and climate change adaptation 3. Equity  Levels of satisfaction , participation and empowerment of specific participating populations: children, youths and Dominican-Haitian and Haitian families. 4. Sustainability and replicability 5. Institutional relations  Risk, Adaptation and Development Tool  Project-community relations IX. Project limitations............................................................................................................ 33 X. Lesson learned and best practices..................................................................................... 34 5 Acronyms AECID Agencia Española de Cooperación Internacional para el Desarrollo Spanish Agency for International Development Cooperation KAP Knowledge, Attitudes and Practices Study CERT Comités Locales de Manejo de Desastres y Brigadas de Rescate Local Disaster Management Comittees and Rescue Brigades CMPMR Comités Municipales de Prevención, Mitigación y Respuesta Municipal Prevention, Mitigation and Response Comittees CNE Comisión Nacional de Emergencia National Emergency Commission ECHO European Commission for Humanitarian Aid and Civil Protection UNPD United Nations Development Program RAD Risk Adaptation and Development Tool DRR Disaster Risk Reduction USAID/OFDA United Sates Agency for International Development/Office of Disaster Assistance 6 I Presentation This Final External Evaluation Report of the Resiliencia : Strengthening Capacities for Disaster Management in Vulnerable Communities Project, carried out in Azua Province, Dominican Republic, is the result of the consultancy process which was convened by the institutions responsible for the project , USAID/OFDA and Plan Dominican Republic. The consultancy has been carried out according to the terms of reference and the evaluation proposal which was presented to the institutions and , revised and approved by them. This document is the revised and approved version of the Final External Evaluation Report. 7 II. Project Description Resiliencia : Strengthening Capacities for Disaster Management in Vulnerable Communities Project , implemented in Azua Province, Dominican Republic was originally programmed to initiate on 30 September, 2009 and has almost two years (23 months) duration period. The total budget for the project was US$447,953.00 with a goal to impact 22,333 individual beneficiaries. The Dominican Republic is vulnerable to storm and flooding hazards. These are the most recurrent hazards the country faces, causing recurrent human and material damage, specifically in the poorest areas of the country where communities lack the necessary capacities to face disaster situations.1 The southwestern region of the country is one of these very vulnerable areas, where the Azua Province is located. Vulnerability - capacity assessments undertaken in this Province prove that there are existing deficiencies for disaster management in responsible sectors, for example, municipal authorities, and there is no integration of disaster management in development plans. This is a key component for developing and securing sustainable disaster risk reduction, DRR, capacities.. The project's main objective was to improve the capacity of rural communities in southwestern Dominican Republic to cope with and recover from storms and flooding and associated cascading hazards2 . The project was implemented in the rural communities which having been most affected in Azua by tropical storm Noel in 2007, incorporated disaster risk management and disaster risk reduction as important needs in their community development agendas which they had been developing with Plan. These were: La Cienaga and El Rosario in the Municipality of Pueblo Viejo; Estebanía, in the Municipality of the same name; Tábara Abajo, in the Municipality of Tábara Arriba; Cañada Cimarrona, in the Municipality of Las Charcas; Los Negros, in the Municipality Azua; and Proyecto C, in the Municipality of Sabana Yegua. The disaster risk reduction activities that the project implemented were modeled in the experiences of similar projects carried out by Plan DR in other provinces located in the same region of the country, Barahona and Independencia. The main goal of the project is the "development, in target communities, of a collaborative capacity to identify, prepare for, and respond to disasters with the resiliency to carry-on regular development activities in the face of disasters".3 1 Proposal Summary Template; Plan International USA; July 14, 2009; page 9 2 Proposal Summary Template; Plan International USA; July 14, 2009; page 3. 3 Ibid; pg. 2 8 The implementing agency, Plan DR, having worked before with the chosen communities, facilitated a needs assessment which was carried out by the local communities. This resulted in the identification of problems related to storms and flooding, along with vulnerabilities to recurrent hazards, risks and also institutional vulnerabilities. Impacts from these on communities were also identified, amongst these, the impact on livelihoods, basically agriculture; the impact on the educational system, on community health and on the most vulnerable populations that live in the communities, such as Haitian migrants and Dominican Haitian population. This population tends to be located in the most dangerous and secluded places, in precarious dwellings and with little integrations to the community and subject to discrimination. The project also proposed to build resilience at the local level integrating disaster management with the community development work that Plan DR carries out in the area. This included: training, and capacity building in needs assessments, first aid, shelter management, the development of early warning systems, the elaboration of community risk maps, awareness raising in disaster risk reduction and the production of Emergency Plans. The project also programmed the development and implementation of the Risk, Adaptation and Development Tool which would integrate disaster risk reduction and climate change adaptation into community development activities. Implementation Strategy and Methodology One key issue in the project's strategic design is the linkage between community development efforts and disaster risk reduction, including adopting sustainable development strategies and climate change adaptation. Another key issue was the creation of what originally were called the Local Disaster Management Comittees and Rescue Brigades (CERTs) linked to supporting the development of the Municipal PMR Comittees, mandated by law. The harmonization process which the DIPECHO partners in the region were implementing, implied unifying criteria and efforts for more efficiency and coherence. This resulted in the CERTs changing their name very early in the project's implementation, to Community Networks for PMR, and were called simply Networks. This responds also to the prevention and DRR perspective that was desired in the project, instead of an emergency response perspective implied in the CERTs. At the community level, the involvement of civil society in the CERTs or Networks was the strategy for the effective implementation of the project. The participation of children, boys and girls, and mothers, which is Plan's specialized area of work, guaranteed the involvement of the home. The participation of Haitian migrant families and Dominican Haitian families guaranteed an inclusive perspective. 9 Members of the Networks representatives of all sectors of the participating communities became multipliers for awareness raising at the community level on disaster risk reduction, and initially it was suggested that they also participate in the training for the Risk, Adaptation and Development Tool. Plan's personnel played a key role also in the process of training multipliers, training of trainers, TOTs, in order to guarantee sustainability of project results. To this end the project planned to involve local businesses and their employees and also reporters from the region to publicize the project and the importance of integrating disaster risk reduction into development activities at the regional and national levels. The existence of a disaster preparedness Cluster working in the southwest region of the Dominican Republic, in coordination with the National Emergency Commission is another important element of the Resiliencia Project's implementation strategy and methodology. This Cluster or Platform is integrated by the following organizations: Dominican and Spanish Red Cross, (Dominican Red Cross had a direct participation in the implementation of the project through their support to TOT's activities in First Aid) Intermon Oxfam, Cooperation Assemblies for Peace ( Asambleas de Cooperaciónn por la Paz); Spanish Agency for Development Cooperation( Agencia Española de Cooperación para el Desarrollo); United Nations Development Program and Civil Defense as permanent members. The Ministry of Health and the Ministry of Education participate regarding specific issues. The project was submitted for revision and approval by this Platform and by the National Emergency Commission. The project's design was submitted to ECHO also for approval. This guaranteed the support of the governing body for disaster risk management in the country for the project, as well as coordination with other development and disaster risk reduction efforts being carried out in the southwestern region and at country level. Another element of support to the project has been the complementarily with other Plan activities, for example those related to the implementation of the DIPECHO projects in disaster risk reduction. This is also a guarantee for coherence and efficiency in methodologies being implemented at the local, regional and national levels as well as regarding project funds and expenses. The Resiliencia Project had support in this respect. The Project initiates at the beginning of 2010. The process of engaging qualified personnel at the local level for project implementation was one of the issues which delayed project initiation, due to the labor market situation in Azua. Another element that should be considered is the fact that in the same communities where the project was to be implemented, another project with similar characteristics was also being implemented and there was a need for coordination. These coordination efforts took time which also delayed the initiation of the Resiliencia project, also affected by the Haitian earthquake and the necessity to respond to that disaster situation in the neighboring country.4 A three month extension was approved to complete all planned activities and project objectives. 4 Catholic Relief Services managed a project funded by USAID/OFDA in the same communities as the Resiliencia project. That fact that this organization does not belong to the regional Cluster made dialogue difficult and so the coordination of activities. 10 Project Objectives 1.- Establishment of community emergency response teams, in seven targeted communities; 2.- Community members (including children) act as DRR trainers of trainers in seven targeted communities, promoting increased awareness about DRR coping capacities, and vulnerabilities associated with natural disasters;. 3.- Development and testing of Risk, Adaptation and Development (RAD) tool and related materials to strengthen DRR awareness and coping capacity in seven targeted communities; 4.- Emergency preparedness and response plans documented and tested in seven targeted communities for enhanced capacity to monitor local hazards, disseminate early warnings and conduct evacuations.5 5 See project logical framework, indicators and activities for details.. 11 III. Participating Institutions Institution Function USAID/OFDA Financing of the Project PLAN Dominican Republic Implementing organization ECHO/DIPECHO Supported with funds Civil Defense Support to the Community PMR Networks Dominican Red Cross First Aid Training Ministry of Education Support to a all project ctivities related to schools Ministry of Environment and Natural Resources Support to the Community PMR Ministry of Public Health Support to the Community PMR Networks 12 IV. Purpose and Context of the Evaluation This final external evaluation of the Resiliencia Project: Strengthening Capacities in disaster management in vulnerable communities, has been carried out at the closing of the project, and covers the complete period in which the project was developed, from it's official starting date in January 2010 until December 2011, a 23 month period. The main purpose of the Evaluation is to analyze project results and systematize good practices and lessons learned. The evaluation analyzes the possibilities for the sustainability and replicability of project results within the framework of other disaster risk reduction processes that are being carried out in the Dominican Republic, specifically in the southwestern region of the country. The analysis of project results will also look into the possibilities of designing similar projects in other countries where cooperating agencies carry out activities. Other purposes are to evaluate the specific work with children, boys and girls, and youths in disaster risk reduction and with Dominican Haitian and Haitian populations in the southwestern region of the Dominican Republic, their inclusion and participation in the project, their empowerment and levels of equity within the project framework and development Also to contribute to the strengthening of disaster management in the Dominican Republic supporting the existing legal and institutional framework. 13 V. Evaluation Criteria The final evaluation is framed by criteria which serve as it's guides, and which are established in the Terms of Reference for the evaluation consultancy. These can be summarized in the following main points: 1.- Pertinence of the project's strategic implementation design and intervention methodology for the achievement of project results, specifically regarding work with children, boys, girls, and youths, in disaster risk reduction. 2.- Project efficacy regarding the achievement of indicators and goals, planned activities, results and products regarding quantity and quality. 3.-Efficiency of project costs versus benefit. 4.- Gender equity. Empowerment and improved relations with Dominican Haitian and Haitian populations in participating communities within project framework. 5.- Sustainability and replicability of project results and strategies to achieve these. Specifically regarding the involvement of community leaders and a DRR agenda in the Azua Province, and relations with local authorities and Plan offices in Azua. 6.- Institutional relations between the organizations and the institutions involved in the project and their contributions to project development. The detailed analysis of these criteria is part of the presentation of project results. 14 VII. Objectives and Scope of the Evaluation The final external evaluation for the Resiliencia- Project: Strengthening Capacities for Disaster Risk Management in Vulnerable Communities in Azua Province, Dominican Republic, seeks to analyze impact quality of the project based on project indicators and the contributions to the achievement of these indicators and project goals. The specific objectives of the final external are: 1. Assess if the project achieved its objectives, goals and expected results based on the project logical framework; 2. Use the results of the knowledge, attitudes and practice surveys (initial and final) indicators to assess project results in the participating communities; 3. Measure satisfaction levels, and scope or access that the children, youths and their families have to project results, and their perception regarding their participation in project design, implementation and evaluation; 4. Assess to what degree community members feel that the project responded to the problems they had identified in their community development agendas and what problems, related to disaster risk reduction and climate change adaptation, they view as remaining unsolved; 5. Assess to what degree community members felt able to shape and change the project as their knowledge of DRR grew – their level of empowerment and control of the project and Plan’s accountability towards them; 6. Identify the degree to which the project has contributed to the protection and empowerment of children and youth (differentiating the analysis for boys and girls and people of Haitian descent); 7. Identify problems or limiting factors that could have impeded project implementation and results achievement, and make recommendations on how they can be avoided in future projects; 8. Make an analysis of lessons learned and best practices that can be disseminated and be used in other relevant disaster risk reduction projects; 9. Analyze the relationship with the organization responsible for result three and the product finally delivered. Make recommendations for the management of such relationships and future development of the “Risk, Adaptation and Development” tool developed under result three and its scale-up and replication in other Plan countries; 10. Assess the appropriateness of the community development agendas as the key tool for 15 identifying future disaster risk reduction programme work (NB this may also reviewing the agendas of the project communities and comparing with the proposal of the existing DIPECHO project, implemented in communities whose agendas did not identify DRR as a priority); 11. In an annex separate to the evaluation report for internal use only, make recommendations to feed in to Plan DR’s disaster management strategy and future DRR project proposals. The final external evaluation of the Resiliencia Project was carried out at the end of the project and field work activities were carried out in the geographical area of project development, in the time established by the convener institution. 16 VII. Evaluation Methodology The proposed methodology implemented for this project evaluation was centered on the use of qualitative research methods and instruments. These include content analysis of project documents and documentation produced as part of project results. Also in depth interviews with key informants, relevant project participants from each of the project communities and with Plan project personnel in Azua. Initially, the methodology proposed included focal groups through which information from children, boys, girls and youths, family representatives, Dominican -Haitians and Haitian population that participated in the project could be raised. In depth interviews originally, would only be applied to local authorities, representatives of first response organizations and Plan project personnel. Due to the lack of time to coordinate focal groups in depth interviews were used to raise information for data construction for all key informants in all participating project communities. Sixty in depth interviews were applied. This implied important changes in field work logistics. The analysis of sixty in depth interviews also implied re organizing that phase of the research. Content Analysis This research tool was used to analyze relevant project documentation: the project presentation document, logical framework, project reports developed throughout the project's implementation, documents that are part of project activities and results, amongst these the RAD Tool, and other documentation which effectively prove that project activities were carried out, such as lists of participants to workshops, certificates of participation in workshops, etc. Analysis of this documentation allows fulfillment of the following evaluation objectives:  Assessing the achievement of project objectives, goals and expected results;  Analyze existing survey results and indicators that assess project results in the participating communities;  Analyze capacity building in emergency management, disaster risk reduction and climate change adaptation, in all involved and specifically in the organizations dedicated to disaster response and community leaders.  Assure that cross cutting issues such as gender perspective and special groups inclusion have been well implemented in the project.  Analyze relations between community organizations and these with Plan project personnel.  Determine possible best practices and lessons learned regarding all phases of the project. 17 In Depth Interviews Seven in depth interview guides were developed one for each of the key informant category.6 These incorporated variables which allowed the raising of information to comply with the following evaluation objectives:  Fulfillment of project objectives, goals and results  Levels of satisfaction with the project  Participation in all project phases  Project impact on participating communities  Project impact on knowledge and information regarding DRR and climate change adaptation  Integration of DRR and climate change adaptation in the communities' development agendas  Integration of the Dominican -Haitian and Haitian populations in project activities  Integration of children, boys and girls and youths in all phases of the project  Relations between the community and Plan project personnel. Key informants are representative of all actors involved in the project: 1) Children, boys and girls, and youths/ including Dominican -Haitians ad Haitians 2) Families/ including Dominican -Haitians and Haitians 3) Representatives of the school system 4) Representatives of first response organizations 5) Local l\authorities 6) PLAN project personnel 7) Community leaders 8) Members of the Community Networks for Prevention, Mitigation and Response 9) Trainers The data which was produced from the interviews allowed the achievement of the following evaluation objectives:  Measure satisfaction levels of specific groups participating in the project, e.g. children, youths and their families, and their levels of participation in all project phases. 6 See these in Annexx ________ 18  Analyze community related consultancy objectives such as the community’s perception of the projects impact on identified problems , specifically those related to disaster risk reduction and climate change adaptation, and community’s involvement and empowerment with the project’s implementation.  Analyze perceptions of relations between community organizations and these with Plan project personnel.  Lessons learned and best practices, specifically related to development and development agendas can also be analyzed with the collection of data through focal groups and in depth interviews regarding the perception of the key informants regarding the project impact on community and development.  Assure that cross cutting issues such as gender perspective and special groups inclusion have been perceived as part of the project’s implementation by key informants.  Allow for the production of data for four human interest stories. Field Work The field work consisted in the application of 62 in-depth interviews to key informants in the seven project communities. Key Informants Qualitative Research Instrument  Boys, girls and youths, including Dominican Haitians and Haitians  Haitian boys and girls  Families and school representatives  Representatives of first response organizations  Local authorities  Community leaders  Members of the Community PMR Networks  Plan- Azua project personnel In-Depth Interviews These were carried out during 3 days in the project communities. The details of the data collection process through the use of the in-depth interviews, are the following: Interviews according to Sex Comunities Total Tábara Proyecto Los Estebanía Cañada El La Azua 19 pertinent actors/informants Abajo C Negros Cimarrona Rosario Ciénaga NNA M 1 1 1 1 1 1 1 0 7 F 1 1 1 1 1 1 1 0 7 Beneficiary Families M 0 1 1 0 1 0 0 0 3 F 2 1 1 2 1 2 1 0 10 Schools M 0 0 0 0 0 0 0 0 0 F 0 0 1 0 0 1 1 0 3 Community Leaders M 1 1 1 3 1 2 3 0 12 F 1 1 1 1 1 0 0 0 5 Mayors/Munici￾pal Committee Directors M 1 1 1 1 0 1 1 0 5 F 0 0 0 0 0 0 0 0 0 Village headman M 0 1 0 0 1 0 0 0 2 F 0 0 0 0 0 0 0 0 2 Haitian population M 0 0 1 0 0 1 0 0 2 F 1 0 0 0 0 0 0 0 1 Dominican Red Cross and Civil Defense Trainers M 0 0 0 0 0 0 0 2 2 F 0 0 0 0 0 0 0 0 0 Plan Personnel M 0 0 0 0 0 0 0 2 2 F 0 0 0 0 0 0 0 0 0 Total 8 8 9 9 7 9 8 2 62 Work Calendar This final external evaluation of the project was carried out following this work calendar: Activities Weeks Preliminary revison of basic project documentation , coordination with the evaluation team and with Plan project team. 1 st Design of data collection instruments 1 st Revision and analysis of project documentation 1 st Field work 2 nd Data analysis 2 nd - 3 rd 20 Writing of the report Lessons learned and best practices Four human interest stories 2 nd - 3 rd 4 th VIII Evaluation Results 1. Intervention methodology and strategy pertinence The project was designed using existing data on hazard exposure in the Dominican Republic, specifically to storms and flooding and data on existing vulnerabilities. The building of resilience is an important issue in the Dominican Republic and the project has this as it's main goal. The impact of tropical storms Noel and Olga in 2007 in vulnerable communities of the southwestern region of the country where Plan carries out projects on disaster risk reduction, specifically in Azua Province, and the resulting inclusion of this problem in the community development agendas, drove Plan to propose a pilot project to be implemented in this region in order to contribute to the strengthening of awareness and preparedness , as well as capacity building in these communities and therefore impact on building resilience in the face of future disasters The original project design took into account the existing disaster management structures by including the Comités Municipales de PMR (Municpal PMR Committees) mandated by law and faced existing weaknesses regarding DRR at the local level by creating community participatory organizations to carry out project activities and to support the official structures. The project's main strategy is the creation of Community PMR Networks which are integrated by all community sectors and associated with the Municipal PMR Committees, with responsibilities to train trainers, raise awareness, disseminate knowledge on disaster risk reduction and disaster management. This is also an important element in project sustainability. The project contributes to the harmonization strategy implemented in the region by joining efforts carried out by cooperation organizations and agencies working in the area, as well as with national authorities that work with disaster management7 . The project's methodology is based on work with children and youths, boys and girls, an area of work in which Plan is experienced. The project effectively incorporates this population and their 7 This has been detailed in another section of this Report. 21 families as well as the schools to which these children and youths belong to. The project's methodology and intervention strategy, in their design, are pertinent and adequate for working with children and youths. This perspective adds value to sustainability of project results, especially regarding the construction of capacity and of resilience. Another important aspect of the intervention strategy is the insertion of disaster risk management in community development efforts. The fact that the Municipal Committees for PMR have not been developed in the project communities, made it impossible to create the desired linkages with the Community PMR Networks. This is a project limitation that will be analyzed in the corresponding section. Other limitations dealing with the implementation of the Risk, Adaptation and Development Tool and with contracting consultancies will be analyzed in corresponding sections of this report. 2. Efficacy and Efficiency The analysis of the achievement of project objectives and results demonstrates project efficacy. Achievement of project's objectives and analysis of results. Verification of achievement of project objectives, as was indicated in the methodology, was carried out through the analysis of project progress reports, in-depth interviews applied to key informants in the seven participating project communities and other project documents pertaining to specific activities.8 Objective 1. Establishment of community emergency response teams (CNPMR), in seven targeted communities. Indicators  One hundred and forty (140) people trained in preparedness, mitigation and disaster management (impact indicators); 8 Reports regarding monitoring visits to project areas; Analysis of Vul and Cap assessments.(VCA) ;Project Progress Reports, Photos, promotion materials(printed and online) Lists of Network members;(representing a cross section of the community including gender, age, work status, etc.); Minutes of meetings; List of meeting participants; TOT workshops (conducted by certified Dominican Red Cross trainers, supported by the Plan project coordinator): Introductory workshops for the Plan project team , including the Area Coordinator for Azua and the Country Office personnel; Focus sessions for Plan personnel to prepare Azua and country office personnel to better assist the communities regarding integration of Disaster Risk Reduction (DRR) in the long term development programs; The ChildMedia PLAN Dominican Republic; Guidelines from the NEC for preparedness plans and risk management; Platform for Disaster Risk Reduction national/regional : Lists of participants in the community training workshops; Mid project evaluations . 22 Ninety eight (98) beneficiaries (70% of all trained persons) members of Networks have retained their knowledge on preparedness, mitigation and disaster management two months after training.( impact indicator)  Seventy percent (70%)of members feel adequately prepared to face the adverse effects of natural disasters (earthquakes, flooding and hurricanes)  Active participation of the municipal authorities in the establishment of the Networks.  At least four Networks show dynamism, with regular meetings, and carry out activities on their own in a continued manner throughout the duration of the project. * The last project report used was the Final Results Report; 1 September 2009 to December 15, 2011. Progress reports for this stage of the project and the final report explain that all necessary processes for the creation of the Networks were carried out with the involvement of all sectors in participating communities. Training workshops were carried out successfully during the project's duration for 136 beneficiaries and in the conditions established by project indicators, with the support of the Dominican Red Cross and Civil Defense. The size of some communities made it unnecessary to integrate the same amount of members in the Networks. The topics of the training workshops included, search and rescue, housing and shelters, evacuation , first aid, early warning systems and other disaster management techniques. In tests that were carried out in the communities, 105 beneficiaries retained knowledge from training workshops 2 months after having participated in them. Documents were produced for the training process, for example, Induction Workshop on risk management (Taller de inducción sobre gestión de riesgos) and Module for Early Warning and Evacuation (Modulo de Sistema de Alerta Temprana y Evacuación). There were evaluations applied to participants to note the retention of knowledge and these tests were successful. The final project report explains the participatory role that the municipal authorities had in the conformation of the Networks and the important role these play as agents for Civil Defense at the municipal level. The passing of Hurricanes Emily and Irene during project implementation allowed for the demonstration of acquired capacities by the members of the Networks and the community, especially regarding a safe evacuation processes. Civil Defense acknowledged the process by certifying Network members. The project reports note that the Networks are dynamic and carry out bi monthly meetings in a regular fashion. The final KAP Survey notes that "Regarding the general knowledge on specific disasters, risks and hazards in the communities, disaster prevention, and search and rescue activities, persons Project Progress Reports * In-Depth Interviews Final KAP Survey Verification Documentation Achieved Achieved Achieved Achieved 23 interviewed consider that they know a lot about these subjects in a greater proportion than what was registered during the first Survey".9 In depth interviews established that this first objective was achieved regarding PMR training and disaster management, also regarding retention of acquired knowledge, the capacity to face disasters and the dynamism demonstrated by community networks which until de the end of the project continued with regular meetings and carrying out activities. One example of knowledge retention after trainings, is the case of the young girl Elizabeth Sánchez, from Estebanía, age 12 who being asked about what she had learned in the project answered that she knows what to do in the face of different hazards such as hurricanes, earthquakes and tsunamis: "I would go to a safe place, the school or the church, and I would warn y family". It should be noted regarding municipal authorities and village headmen that not all had the same involvement in the activities to conform the Networks and maintain relations with these. In the cases of the District Boards of Tábara Abajo and El Rosario where their participation was very active and dynamic. In the case of Cañada Cimarrona the acting village headman had a very dynamic participation at the beginning of the project and in the conformation of the Networks. But it cannot be said that other municipal and local authorities were as involved as the afore mentioned, without it being implied that they were not at all involved. The fact that the CMPMR do not exist in the communities added to the size of most of these communities, makes it difficult for bottom up demand efforts to be met, and thus the lack of involvement of the municipal and local authorities in these issues. Objective 2 . Community members (including children) act as DRR trainers of trainers in seven targeted communities, promoting increased awareness about DRR coping capacities, and vulnerabilities associated with natural disasters. Indicators  One hundred and forty (140) persons trained in preparedness, mitigation and disaster management. (Impact indicator);  One hundred and seventy five (175) members of the community (including children, boys and girls, and youths, and adults) as trainers in awareness raising of community members after receiving training;  One hundred and twenty (120) beneficiaries (69% of persons trained) member of Networks retain knowledge on preparedness, mitigation and disaster management two months after the trainings (impact indicator); 9 Results Report: Final Survey on Knowledge, Attitudes and Practices regarding Disasters in Six Communities of Azua Province , Dominican Republic; Dec. 2011. Reported by Alejandro Moliné. Pg. 68. 24  Twenty (20) members of Plan personnel are aware of disaster risk reduction (DRR) methods along knowledge on hurricanes, flooding in the project communities;  Seventy percent (70%) of the children and youths aware of living in disaster prone areas feel adequately prepared to face adverse effects of flooding and hurricanes;  Sixty percent (60%) of the adults aware of living in disaster prone areas feel adequately prepared to face the adverse effects of flooding and hurricanes;  Fifty (50) journalists and radio broadcasters aware of risks and vulnerabilities and willing to further engage in raising awareness using the media to target a broader audience;  Active involvement of municipal authorities with CNPMRs, and TOT events;  Four private businesses exposed to DRR messages . The indicators were achieved according to the project progress reports and final report, exceeding proposed goals. The project successfully selected and trained 154 multipliers, male and female, amongst these, 35 children, boys and girls, and 8 Haitians, men and women. Each multiplier had 20 families under his/her charge. The inclusion of the Haitian population confronted difficulties due to the existing discrimination in the country and in the project area, which increased during the cholera outbreak. These multipliers recieved training which allowed them to carry out awareness raising activities in the communities, in Spanish and Creole. Materials were produced and distributed as planned: brochures, posters (in Spanish and Creole), family calendars and the Risk Management Community Guide. A strategy to work with children, boys and girls, was implemented, and the children participated enthusiastically. A lot of enthusiasm was also shown by school authorities. Four hundred and eighteen (418) students from the 5th and 6th grades were trained and received the School Security Guidelines booklet, which was prepared within the framework of the DIPECHO project in DRR, which is being carried out in the region. In the schools the project worked with the game Riesgolandia which Plan personnel adapted for the better use of the communities involved. ( Versions of this game were made in Spanish and Creole). Communication radios were installed in the seven project communities which facilitate communication with Civil Defense. Also risk maps were made of the communities. Three small community private businesses and their employees were trained and the final project report notes that 48 journalists working in the project zone participated in awareness raising sessions in disaster risk reduction and it's integration into development efforts. Project Progress Reports In-Depth Interviews Final KAP Survey Verification Documentation Achieved Achieved in general Achieved Achieved 25 The results of the KAP Survey state that the indictors for Objective 2 were met. It indicates that in the project evaluation based on the data from the survey and focus groups that:  local structures were created for risk management;  Knowledge was transmitted to the communities;  Capacities were developed and attitudes and practice have improved;  Capacities were developed within the project framework which will be useful throughout the lives of the people;  The Networks effectively trained the population.10 . In the process of applying in depth interviews, the existence of the Networks was made evident with the participation of different local actors as members, whom participated in all project activities. This includes multipliers, youths, boys and girls, community leaders, women and men and persons of Haitian and Dominican Haitian descent. Objective 3. Development and testing of Risk, Adaptation and Development tool and related materials to strengthen DRR awareness and coping capacity in seven targeted communities. Indicators  One hundred and forty (140) persons trained in use of Risk and Development tool for DRR (impact indicator);  Ninety eight (98) Network volunteers (70% of those tranined) retaining and applying Risk and Development tool methodologies for DRR two months after training (impact indicator);  Risk and Development Tool sued for Disaster Risk Reduction , DRR, in the seven project communities (results indicator);  Development and sharing of Risk and Development tool, other DRR training materials and tools;  Active participation of municipal authorities in the development of the Risk and Development Tool. This Tool is an innovative aspect of the Resiliencia project. There was a keen interest in the project design that DRR and local development be integrated and that climate change 10 Ibid pg. 72-76 Project Progress Reports In Depth Interviews Final KAP Survey Verification Documentation Achieved Not completely achieved Does not Apply There is a Risk and Development Tool document 26 adaptation, an important new situation for the Dominican Republic, also be included. Project progress reports show that the process starts with the search for qualified personnel at the local level to develop and implement the Tool. This personnel was hired and the Tool was produced. Project progress reports note that there was a pilot trial of the Tool in it's first phase , with local authorities in two project pilot communities: La Ciénaga and Estebanía. The Tool was also presented to experts and authorities on disaster risk management for dissemination and evaluation. The continuation of the second phase of the process was delayed due to the activities implemented after the Haiti earthquake and the cholera outbreak which came after. It is also important to note that administrative aspects of the development of this Objective presented a challenge to Plan personnel at the local level, which also contributed to the delay in implementation of the second phase. Institutional relations related to the development of the Tool also contributed to the delay. This will be analyzed in the corresponding section of this report. During that phase, members of the Networks, including boys , girls, and youths were trained. Despite the difficulties mentioned, Plan Azua personnel continued the training process successfully and completed training for 105 Network members. In depth interviews leave evidence that a significant part of the people interviewed knew of the Risk and Development Tool. Objetctive 4 . Emergency preparedness and response plans documented and tested in seven targeted communities for enhanced capacity to monitor local hazards, disseminate early warnings, and conduct evacuations. Indicators  Eighty (80) personas trained in development and monitoring of community emergency plans (impact indicator);  Seven Networks trained to collaborate with community members in design and development of emergency plans;  Emergency simulation exercise carried out and analyzed by CNPMRs, community members and Red Cross;  Forty nine (49) 70% of all trained) retaining and applying methods used to develop and monitor community emergency plans two months after initial simulation exercise (impact indicator)  Seven target communities successfully drawing up an emergency response plan and integrating it into their community development activities (result indicator);  Sharing of community emergency plans with local authorities;  Three simulations carried out by the end of the project;  Active involvement of municipal authorities in simulation exercises, development and regular updating of community emergency plans;  Rain and/or river gauges installed in each community;  Number of communities in which VHF radios and necessary energy systems are installed;  A written protocol for the use of VHF communication systems in each community is established with the Civil Defense; 27  Where communities share a VHF system, a written protocol is established facilitating access to the VHF system for key representative of the communities . Project progress reports note the complete achievement of this objective and all the indicators, surpassing expected goals in some cases. Community emergency plans were made but at the closing of the project, these had not been turned into the municipal authorities. The project worked with a consultant related to the National Institute for Hydraulic Resources (INDHRI) for the installation and support regarding early warning systems. Rain gauges were bought to be installed in the rivers that run through the project communities The work carried out by the consultant, due to OFDA regulations, was funded by Plan This administrative situation complicated and delayed the buying and installment of the rain gauges, which were installed at the end of the project with the full participation of the project coordinator.11 .  Risk and Development: Development agendas, disaster risk reduction and adaptation to climate change. The project was designed on the basis of the demands regarding disaster risk reduction and management made by the communities and included in the community development agendas. These agendas also included demands dealing with social and economic problems which communities face, as well as environmental demands and the topic of climate change which the project design refers to. The Risk, Adaptation and Development tool is a response to these community issues. Opinion of families, community leaders and Plan, Azua, personnel During the application of the in depth interviews, community leaders, families and Plan personnel from Azua were asked about the development agendas and the integration of DRR and climate change adaptation within the agendas and this had varied results. In the case of beneficiary families, half of the interviewees did not know of the development agendas. The 11 Telephone conversation with MR. Welhouse , January 21, 2012. Project Progress Reports In Depth Interviews Fina KAP Survey Verification Documentation Achieved Achieved Achieved Achieved Project Progress Reports* In Depth Inteviews Final KAP Survey Verification Documentation Does not apply Achieved in part Does not apply Does not apply 28 majority of the community leaders did know of these and Plan Azua personnel noted the existence of the development agendas and the use that they were given in the communities. The fact that half of the families interviewed and various community leaders did not know of these agendas leaves doubts about the project's relation with these. The people interviewed who did know of them, identified the DRR topic in these and understand that the project contributed to effectively facing related DRR problems mentioned in the agendas. Some of the problems identified by the communities, such as deforestation and the necessity to build small mitigation works were not solved by project activities since they were not project objectives and due to scarce budget. Project design did include awareness raising on environmental problems, an aspect of the project on which there is no information outside of the RAD Tool. Some municipal authorities interviewed assured that during their time in office, they have invested resources for DRR. Some people in the project communities corroborated this news, although the majority of the interviewees denied that their respective municipalities were effectively carrying out this investment. Regarding climate change the project had inconveniences. Under Objective 3 the project planned awareness raising on climate change adaptation and it's relation to disaster risk reduction. The means to achieve this was the production of the Risk, Adaptation and Development Tool. The project Proposal Format and progress reports portray and idea of the RAD Tool which emphasizes DRR as an important aspect of community development. The Tool effectively includes the relation between development processes, disaster risk and DRR, and stresses the importance of planning and of developing technical elements in order to be able to effectively carry it out and the need for a participatory methodology designed for all community actors. The climate change aspect needs to be reinforced in technical terms in order to make it more useful to other Plan projects in the Dominican Republic as well as internationally. 3. Equity  Levels of satisfaction and participation and levels of empowerment of specific populations: children, boys and girls, youths and families, Dominican Haitians and Haitians. Project Progress Repots In Depth Interviews Final KAP Survey Verification Documentation Achieved Achieved Does not apply Achieved 29 In the section of this report on "Achievement of project objectives and indicators" it was noted that children and their families had access to project results: they were active members of the Networks, training workshops, as multipliers and participated in simulations and games, among other project activities. In depth interviews show the veracity of these results, making it evident that these results were obtained with gender equity. It should be noted , though, that the participation of children and families of Dominican Haitians and Haitians was less active. In a general sense, children, boys and girls and youths that were interviewed answered that they felt integrated and part of the project. This being mentioned along with the information given in other sections of this report we can rightfully conclude that children, boys / girls, and youths and their families had an active participation in the different phases of the project, although they did not participate in project design. Also, the project contributed to the protection of children, including Dominican - Haitians and Haitians, through all DRR disaster management related activities. Acquired knowledge related to their security regarding hazards and the conditions of vulnerability that characterize their communities, can be considered an important contribution to the protection of children in these communities in the case of emergencies and disasters. The work on risk reduction is a contribution to the future security of these children. The impact of the project on the empowerment of children, including Dominican-Haitians and Haitians can be seen in their participation and leadership roles they acquired in the activities related to project objectives. A sample of this empowerment is the fact that the children stated that the knowledge acquired within the project framework was shared by them with their families and friends. Another example is how the children led the process of producing family emergency plans, after the training they received in the schools. The project successfully implemented a gender equity and gender inclusive perspective. There is no evidence that the girls and women who participated in the project did not have the same access to project results and the same participation in all activities. The results of the KAP survey show that in the project there was more participation of women than men. Reviewing the participants list for multipliers, women exceeded men. From a total of 154 multipliers, 81 were women. The women had leading positions and participated in decision making in different project activities, more so in some communities than in others. Interviews with Plan personnel show that girls also assumed leadership positions, specifically in transmitting knowledge to their families. 4. Sustainability and Replicability In the Proposal Format , the project is described as a pilot project, with elements to guarantee it's sustainability and the possibility of replicating in another context. Various elements in the project design take into account it's sustainability: 30  the TOTs methodology impedes the possible loss of acquired knowledge in Plan personnel as well as in the community.  working with children supports awareness raising in regards to the importance of disaster risk reduction, prevention and management. Children will give continuity throughout their lives to this, a fact which will have an important impact on future generations.  the relations that Plan has established with formal community organizations, such as the Civil Defense, the National Emergency Commission, and the Cooperation Platform for DRR in the region and their participation in the project, and probably with future CMPMRs.  the relations established within the project framework with community private business.  the training of journalists from project communities and from the rest of the country on information and disaster risk reduction. Everyone interviewed was asked for their opinion on the sustainability of project results. On this topic very few people disagreed regarding project sustainability. The majority agreed that project results are sustainable, although opinions differed on how to achieve this sustainability. We can summarize the most important opinions in the following way:  the will of the community  the use of the networks as a team , a project legacy and linked to Civil Defense.  The knowledge acquired within the project framework seen as a community strength towards the construction of resilience.  through the school system  continuing with training  due to the training and capacity of community leaders, men and women who participated in the project.  creating strong community ties based on the knowledge acquired in the project. Despite all these opinions there are few people interviewed that mentioned the existence of concrete plans and strategies to give sustainability to project results. It is important to note here that there were important linkages created between Civil Defense and the Community PMR Networks, which strengthens sustainability of project results. The management of evacuations by the communities during the passing of Hurricanes Thomas, Emily and Irene is evidence that capacity building is another element that upholds sustainability of results. It is also important to note regarding sustainability of the follow up to the climate change adaptation topic, that this was included in the Management Committee in all the communities were Plan is working. 31 5. Institutional Relations  Risk, Adaptation and Development Tool Institutional relations linked to the implementation of the Project can be analyzed on two levels: 1. Project-community relations 2. consultancies Project-community Relations Project progress reports refer to good relations between Plan Azua with other entities participating in the project at the local level and at the national level. These relations were considered by those involved as relations of equity and cooperation. This can be valued as a basic element for the sustainability of project results. In depth interviews show diverging opinions regarding relations with Plan Azua. A majority of community leaders manifested that they ignored how these relations had been conducted. In regards to the relations of Plan Azua with the communities, interviewed families and the teachers that were interviewed coincided by pointing out that these relations were good. Consultancies Risk, Adaptation and Development Tool: As was mentioned in the section on Objective 3 of this Report, the process for finding qualified personnel at the local level for the development of the RAD Tool took longer than expected, and the hiring process was delayed. Despite this, the consulting firm Raíces y Alas, carried out the first phase of the consultancy in the pilot communities. Other processes external to the project delayed the implementation of the second phase of the process. Interviews which were carried out with Plan RD personnel to this respect allows for an analysis of the relations between the consulting firm and it's personnel and Plan Azua personnel from which we can conclude that there was not a total appropriation of the activities related to RAD Tool development on the part of the personnel in Plan Azua. Specifics related to the management and administrative logistics for the development of the activities at the local level regarding the Tool became complicated for the Plan Azua office. Project Progress Reports* In-Depth Interviews Final KAP Survey Verification Documentation Achieved Achieved in part Does not Apply Does not apply 32 In order to carry out the activities under Objective 4 of the Project related to early warning systems, it was decided that the project could support the improvement of these systems in the country, specifically those related to flooding. The project established relations with the National Institute for Hydraulic Resources ( Instituto Nacional de Recursos Hidráulicos, INDRHI) since this is the responsible institution at the national level for early warning systems for flooding. . As the section on Objective 4 of this report states, again in this aspect of project development the hiring of personnel for this consultancy delayed the project process. OFDA norms, which do not allow the hiring of governmental personnel in projects which they fund , resulted in Plan having to finance this activity, carried out by an external consultant for the identification of locations for the water gauges, their selection and installment. These procedures, new to Plan RD, created difficulty in the development of the activities related to the buying and installing of these early warning systems. 33 IX Project Limitations There were no problems identified, or limiting factors that could have impeded project development. In other sections, this report has mentioned that the project initiation was delayed, and that a three month extension was allowed, therefore the initial delay had no impact on project results. Regarding the achievement of project results the following limitations should be mentioned: 1) Full participation of the Haitian population , due to varying factors. Among these the following are most important:  The language of the group:  Continuous migratory activity due to their status as migrant workers;  Existing discrimination in the country against this population group, Within the project framework, this discrimination became evident after the cholera outbreak, and caused the exit of part of the Haitian men and women who were integrated in project activities. Despite the fact that this cause delays in project implementation, other Haitian citizens were able to be integrated into the project to substitute those who had left. 2) Full incorporation of the climate change topic 3) Impossibility of incorporating big local businesses located in the areas of project implementation. This was due to different factors:  Some businesses already had their own emergency and security plans in place;  Existing contradictions between big businesses and the communities;  Lacking or deficient linkages between businesses and the communities, specifically regarding employees; 5) Since the CMPMRs were not set up in the project communities,, it was not possible to integrate the community networks with these. 6) Institutional relations and project timeline were affected in the cases of the hiring of consultancies due to the type of norms established by USAID/OFDA and the lack 34 of experience in the implementation of Plan regulations to this respect, specifically at the local level. 7) The geographical dispersion of selected project communities is a factor which influenced some of the above mentioned limitations, for example, the resulting impossibility to impact in the conformation of the CMPMRs and possibly in the integration of big local businesses in the project. Also regarding the timeframe for some project activities. The intervention strategy should be revised in this sense, and working in communities of a single municipality was suggested during interviews with Plan personnel. It was considered that this could create a more dynamic situation at the community level, with more capacity for bottom up demands and with more local impact. X Lecciones Aprendidas y Buenas Prácticas Lessons learned and best practices presented here stem from project related documentation and the in depth interviews carried out for this external evaluation. From the data gathered from focal groups carried out during the final KAP survey, lessons learned from project implementation where identified. We have reorganized these as best practices and lessons learned for this report considering that lessons learned in the KAP survey actually portray both: Best practices According to the KAP final survey, we can mention the following best practices stemming from project implementation: collective perspective supports disaster risk management and strengthens the National PMR System.  Early warning systems, for example radios and water gauges are of great importance for disaster risk management at the local level.  Awareness raising and education in disaster risk management is important to persons and impacts on diverse aspects of their lives. In the in depth interviews, many answers stressed best practices regarding the following aspects of the project: 35 Community empowerment regarding risk management. Persons interviewed recognize they have been empowered by the project regarding the following:  They recognize and can distinguish hazards.  The recognize the use of shelters.  They know how to apply first aid.  They know early warning systems.  They understand and value the importance of the Community PMR Networks  The ability of youths, boys and girls to protect their community  Most communities were well prepared and ready for action.  The importance of training people to know how to act before, during and after a natural phenomenon.  They value being prepared for a storm. Achieving community support for the security of those with the least resources  Recognizing the importance of giving first aid during a storm to the most exposed areas  The importance of community preparedness  They value having knowledge for effective communication with others  They know the value of knowing how to defend their community without outside help  They know how to avoid danger  They know how to evacuate dangerous areas Project Design Regarding project design, the men and women, boys and girls interviewed considered it a best practice that the design took the following elements into account  The value of using multipliers to train other community members 36  Working with children in all project phases  Taking into consideration that disaster risk management strengthens community leadership  The importance of acquiring the confidence of those who live in exposed areas  Having an inclusive perspective for selecting community multipliers. Community Training  Recognizing the existence of people in the communities which are trained to help  Training of children  The training of persons so that they know what to do before, during and after a disaster Early Warning Systems  Being able to recognize the importance of radio communication during an emergency situation and giving this activity importance within the project.  Having achieved that the community recognize the importance of carrying out simulation exercises.  Having equipped the Community PMR Networks Vulnarable Groups  Having trained persons who live in exposed areas. 37 To these best practices that project participants have identified, we would like to add the following: Regarding project design:  Existing systematized information on the Dominican Republic, regarding hazards and risk, was used for project design, specifically data pertaining to the area of project implementation, as well as on the experience in disaster risk management in the country, the legal and institutional framework, other sectors involved and their actions. This resulted in a good project design and effective implementation.  Establishing the linkages between development processes and disaster risk management, as well as relating this to climate change adaptation, and this through the use of community development agendas, is a strategy which should be continued and replicated in other projects in this area  Integrating a gender perspective in all project phases. Regarding project implementation , we could mention the following aspects as good practices which should be replicated:  The coordination and harmonization efforts with other institutions, international cooperation institutions as well as national ones, and with their projects, has benefitted effective project implementation and the achievement of project results.  The work with children , Dominican-Haitians and Haitians , which is an important part of project design, achieved the effective inclusion of these groups in all aspects of the project and is an input to the efforts being made to include vulnerable populations in disaster risk management in the country; a best practice to be replicated also.  The involvement of the educational system as a strategy to reach community children. This is a guarantee for sustainabililty.  The use of high quality training materials and some in Spanish and Creole is an important contribution.  The involvement of community private business, also part of project design, is another important element in the sustainability of results. 38 Lessons Learned The KAP Survey states the following:  The political context has to be taken into account for an effective involvement of the authorities, at the local and national levels. There has to be awareness raising at all levels and it is important to coordinate with the political agenda.  Community work is more effective when roles are well defined and clear. The selection of adequate personnel is basic for project effectiveness.  Disaster risk management is given more importance in the face of hazards and when practical results can be made apparent during dangerous situations.  It is important to include all community sectors in disaster risk management activities.  A culture of response prevails in communities in the face of disasters.  The need for small infrastructure, mitigation works, at the community level should be considered and included in projects of this type. The following lessons learned were mentioned by the persons who participated in the in- depth interviews carried out for the external project evaluation: Project Design  Producing training materials takes a lot of time The production of training materials for the project took more time than was foreseen. and this delayed project implementation. Making use of existing training materials could have eased the training process. Vulnerable Groups There are difficulties related to the work with the Haitian population in Azua. 39 The language barrier and the segregation in which the Haitian population lives along with their high migration rate make it difficult to work in a stable and sustained manner with this very vulnerable population group. 40