1 Risk Management/Community Preparedness El Salvador, Guatemala, Honduras and Nicaragua OFDA Agreement # AOT-G-00-01-00040-00 23 January 2001 – 30 April 2003 Final Impact Evaluation Report American Implemented in Partnership between The American Red Cross (ARC), Salvadoran Red Cross (SRC), Guatemalan Red Cross (GRC), Honduran Red Cross (HRC) and Nicaraguan Red Cross (NRC) July 2003. 2 Acknowledgements __________________________________________________________________________ The results of these two surveys are dedicated to all CAMI staff and volunteers who worked so hard over the past two years to make the CAMI Project a success. Throughout the life of the project they displayed unfailing professionalism, persistence, dedication to their work; the Red Cross Principles -- and to the people of Central America whom we are all committed to serving. Particularly, the American Red Cross would like to acknowledge the 11 Red Cross Chapters and 54 Community Emergency Committees who have taken on the responsibility for ensuring the safety of their communities in the event of a future disaster in their area. Most of all, this report is dedicated to the communities who gave of their time and their energy during this evaluation so that we might better understand that which they already know so well. It is our hope that the results of this survey will lead to improved quality of Red Cross Disaster Preparedness programs which serve them 3 Red Cross Central America Mitigation Initiative Final Evaluation Report TABLE OF CONTENTS TABLE OF CONTENTS 3 THE FUNDAMENTAL PRINCIPLES 6 LIST OF ACRONYMS 7 EXECUTIVE SUMMARY 8 TABLE A. TOTAL NUMBER OF TARGETED AND REACHED BENEFICIARIES 9 DESCRIPTION OF THE EVALUATION 17 EVALUATION PURPOSE 18 EVALUATION METHODS 18 CAMI FINAL EVALUATION TEAM MEMBERS 18 DRAFT AMERICAN RED CROSS EVALUATION GUIDELINES 18 OFDA RESULTS REPORTING GUIDELINES. 19 POTENTIAL UTILIZATION OF CAMI EVALUATION RESULTS 19 BACKGROUND AND PROJECT HISTORY 20 THE CAMI PROJECT HYPOTHESIS: 22 NEGOTIATION OF AGREEMENTS WITH NATIONAL SOCIETIES. 23 COUNTRY BACKGROUND AND OPERATING ENVIRONMENT 23 CAMI PROJECT GOALS AND OBJECTIVES: 28 PLANNED VERSUS ACTUAL OBJECTIVES AND INDICATORS: 28 4 PLANNED VERSUS ACTUAL GEOGRAPHIC TARGETING. 28 DESCRIPTION OF INTERVENTIONS 31 TABLE B. PLANNED VERSUS ACTUAL TARGETING OF BENEFICIARIES 32 OBJECTIVE #1 33 OBJECTIVE #2. 34 QUANTITATIVE 38 DATA COLLECTION METHODS 38 LIMITATIONS OF THE DATA 40 TABLE B. SUMMARY OF RESULTS: KEY PROJECT INDICATORS 42 TABLE C. TOTAL NUMBER OF TARGETED AND REACHED BENEFICIARIES 45 OBJECTIVE #1 45 TABLE D. SURVEYED COMMUNIDADES Y CASERIOS BY COUNTRY IN BASELINE/FINAL EVALUATION. 45 TABLE E. PROPORTION OF CASERIO/COMMUNIDADES WITH BRIGADES 48 TABLE F. PROPORTION OF COMMUNIDADES WITH COMMUNICATION CAPABILITY WITH RED CROSS CHAPTER 49 OBJECTIVE #2 51 TABLE G. PROPORTION OF SCHOOLS EVALUATED BY COUNTRY 51 TABLE H. ACHIEVEMENT OF ATTENDANCE IN TARGET SCHOOLS. 53 OBJECTIVE #3 55 TABLE I. QUALITATIVE REACTIONS OF RESPONDENTS TO DRILL SCENARIO 58 5 OBJECTIVE #4 59 COST ANALYSIS 61 TABLE J. COST PER BENEFICIARY ANALYSIS 64 PROGRAMMATIC 65 PROGRAMMATIC 66 MANAGEMENT OF REGIONAL DPP PROJECTS 72 MONITORING AND EVALUATION OF DPP PROJECTS 79 APPENDICES 82 (A) EVALUATION TERMS OF REFERENCE, INCLUDING LIST OF DOCUMENTS REVIEWED AND INTERVIEWS CONDUCTED 83 (B) 6 The Fundamental Principles of the International Red Cross and Red Crescent Movement Humanity The International Red Cross and Red Crescent Movement, born of a desire to bring assistance without discrimination to the wounded on the battlefield, endeavors, in its international and national capacity, to prevent and alleviate human suffering wherever it may be found. Its purpose is to protect life and health and to ensure respect for the human being. It promotes mutual understanding, friendship, cooperation, and lasting peace among all peoples. Impartiality It makes no discrimination as to nationality, race, religious beliefs, class, or political opinions. It endeavors to relieve the suffering of individuals, being guided solely by their needs, and to give priority to the most urgent cases of distress. Neutrality In order to continue to enjoy the confidence of all, the Movement may not take sides in hostilities or engage at any time in controversies of a political, racial, religious, or ideological nature. Independence The Movement is independent. The National Societies, while auxiliaries in the humanitarian services of their governments and subject to the laws of their respective countries, must always maintain their autonomy so that they may be able at all times to act in accordance with the principles of the Movement. Voluntary Service It is a voluntary relief movement not prompted in any manner by desire for gain. Unity There can be only one Red Cross or Red Crescent Society in any one country. It must be open to all. It must carry on its humanitarian work throughout its territory. Universality The International Red Cross and Red Crescent Movement, in which all Societies have equal status and share equal responsibilities and duties in helping each other, is worldwide. 7 List of Acronyms APS Annual Program Statement ARC American Red Cross BVAs Budget versus Actual CI Confidence Interval DIP Detailed Implementation Plan DK Don’t Know DP Disaster Preparedness DR Disaster Response EOC Emergency Operations Committee FEMA Federal Emergency Management Agency HOD ARC Head of Delegation HoRD ARC Head of Regional Delegation IDRU International Disaster Response Unit IFRC International Federation of Red Cross and Red Crescent Societies HRC Honduran Red Cross GRC Guatemalan Red Cross LOP Life of Project MOU Memorandum of Understanding NHQ American Red Cross National Headquarters NRC Nicaraguan Red Cross OFDA Office of US Foreign Disaster Assistance PAHO Pan American Health Organization REF Refused SD Standard Deviation SRC Salvadoran Red Cross TAPE Technical Assistance, Planning and Evaluation Unit of ARC/NHQ TBD To be determined USD US Dollar 8 Executive Summary ______________________________________________________________________ Background: In October 1998, Hurricane Mitch swept through Central America with sustained winds of 112 kilometers per hour, causing devastating floods and mudslides, massive infrastructure and property destruction, and significant population displacement. Up to 10,000 people were killed, an estimated 3.6 million people were affected, and nearly 100,000 homes were completely destroyed. In February 2000, as part of the $630 million U.S. Government response, USAID/OFDA announced a three-year, $11 million Central America Mitigation Initiative (CAMI) for the region, with preference to the most severely affected countries of El Salvador, Guatemala, Honduras, and Nicaragua. CAMI's goal was to reduce or negate the impact of natural disasters in Central America by financing activities that increased the capability of regional, national, and community authorities and organizations to forecast, respond to, and prevent disasters. A number of causal factors that led to excess mortality, morbidity, and missing persons. One factor was poor risk management (preparedness, readiness, and response capabilities) by emergency personnel and an over centralization of disaster services at the capital level. Further, there was poor community knowledge of how to prepare for and respond to disasters, as well as cultural beliefs in myths about the causes of disasters that may have prohibited communities from taking action. The American Red Cross was awarded a three-year CAMI grant in the amount of $1.2 million to build the capacity of four Central American National Societies to monitor and respond to disasters. The CAMI project aimed to reduce excess morbidity, mortality, and property loss by: 1) Training chapter staff and local emergency committees to develop, become trained in, and institutionalize standard disaster response procedures; 2) Training households and schools in disaster preparedness/response; 3) Training a corps of Red Cross volunteers, who will be community-based, to provide disaster preparedness training and follow up in their communities. The CAMI evaluation of this project is a summative evaluation, which analyzes the outcomes of the program at the completion of project activities. As such, this evaluation’s focus is to assess whether or not the CAMI project interventions achieved their intended outcomes in an efficient, effective, and sustainable manner. Based on these findings, conclusions and recommendations will be made to other ARC and Red Cross/Red Crescent Movement Partners.1 1 This evaluation will ensure our compliance with our grant obligations per ARC’s signed agreement with OFDA. As such, the evaluation report will be written to ensure that relevant portions of the overall report can be inserted into the final report per OFDA Guidelines for Grant Proposals and Reporting. 9 Achievement of Beneficiary Targets: The CAMI Project exceeded beneficiary targets with establishment of school emergency committees and school brigades. Over 26,000 teachers and students benefited from CAMI interventions in their schools. Due to extensive efforts of the project in training volunteers, more than 7,600 home visits were made to implement disaster preparedness education interventions which, given an average family size in the Americas of 5.5 indicates that the project covered 42,080 individuals through home visiting efforts. Some of these individuals also benefited from participation in Disaster Preparedness Workshops, simulations and/or other school or community based project activities. While all Red Cross Chapters ended the project with either a part-time or full-time emergency committee, targets were not achieved as originally envisioned with community emergency committee’s where 2 of the original 12 became operation 24/7 and 54 of the original targeted communities established part-time local emergency committees. The target number of Chapters also was adjusted from 12 to 11 when it was realized that one of the target Chapters in Honduras (Choluteca) functionally ended up utilizing the Tegucigalpa National Headquarters' Table A. Summary of Targeted and Reached Beneficiaries Targeted Reached Red Cross National Headquarters Red Cross Emergency Committees Established (any) Red Cross Chapter 24/7 Operational Emergency Committees Established 4 12 12 4 11 2 Community Emergency Committees Established Beneficiaries covered by an Emergency Committee and a Disaster Plan Number of Community Brigades Schools Students in Target Schools2 Teachers3 Number of School Brigades Students Trained as Brigade members or other specialized training Number of Disaster Preparedness Workshops conducted Number of Beneficiaries participating in Disaster Preparedness workshops 60 123,175 225 65 NA NA 270 NA NA NA 54 115,857 206 87 24,563 1,952 502 3,311 481 4,641 Number of Volunteers Trained Number of Educational Home Visits performed by Volunteers Individuals covered through Household Visits 705 NA NA 1,057 7,651 42,080 Achievement of Project Results: The first objective of the project was to improve the timeliness and appropriateness of mitigation disaster responses of Red Cross national Societies and communities through developing sustainable protocols and training personnel in operation centers and emergency committees (APS, Objective 1, Activities 2 and 3). By project’s end, 11 Red Cross Chapters had established an Emergency Operations Committee and had established an emergency plan. Most of these emergency committees function on varying part-time schedules, whereas 2 of the Chapters function 24 hours a day/7 days a week (1 in El Salvador and 1 in Nicaragua). This was due to the fact that some communities had other emergency committees in place such as CONRED in Guatemala. As such, the need for a Red Cross Chapter to be operational 24/7 would have been a duplication of roles and responsibilities with other agencies. The project therefore focused its efforts on establishing p/t committees as adjuncts to these other committees and ‘filling gaps’ in existing 2 Benefited from total CUSE intervention including implementation mitigation projects. 3 Ibid 10 response capacities, such as communication capabilities with the Capital City. About 2/3’s of committees reported holding an exercise or practice in the 6 months prior to the final evaluation and 82% could pass messages to their National HQs within pre-established time limits during a simulation. However, only about a 1/3 of Chapters verified the source and content of emergency messages during these drills. Key Results - Red Cross Chapters: • 100 percent of target Red Cross Chapters ended the project with an emergency committee (increased from 0 percent at baseline). 18 percent (2/11) were ‘fully operational’ by project’s end (i.e., functioning 24/7); • 100 percent of target Red Cross Chapters ended the project with an emergency plan (increased from 0% at baseline) • 64 percent of target Red Cross Chapters reported holding an exercise or practice for a disaster in the 6 months prior to the survey (increased from 17 percent at baseline). • 82 percent of target Red Cross Chapters who can pass emergency messages up and down to next ‘higher’ administrative level within 30 minutes during a simulation exercise (increased from 50 percent at baseline). • 64% of National HQ’s verified the source and content of disaster messages sent to/from Red Cross Chapters at project’s end (increased from 50%) o 36% of Chapters verified the source and content of disaster messages sent to/from Red Cross National HQ’s at project’s end (increased from 17%) Summary of Key Results: CAMI Project Objective One - Red Cross Chapter Emergency Operation Committees 0 10 20 30 40 50 60 70 80 90 100 Red Cross EOCs EOC Disaster Exercises EOC Messages Passed Timely Source/Content Verified (National to/from Chapter) Source/Content Verified (Chapter to/from National) CAMI Indicator Percent of EOCs 2001 2003 11 The second objective was to increase knowledge, self-efficacy, and skills in Disaster Preparedness and response of individuals and households through incorporating risk management concepts in formal and informal education methods (APS, Objective 3, Activities 2 and 3). Again, the following percentages indicate the project’s success in achieving this objective. Achievement of preparedness practices such as newly formed committees conducting disaster drills or meetings on roles and responsibilities were more significant that quality control during communication simulations. While the majority of Community Emergency Committees could pass messages within time limits during a simulation by project’s end, only about a third were observed to verify the source and content of these messages. Key Results - Community Emergency Committees • 90 percent of target communities (n=60) had formed an emergency committee by project’s end (increased from 18 percent at baseline) • 90 percent of target communities (n=60) had formed emergency ‘brigades’ by project’s end (increased from 6 percent at baseline) o 94 percent of these established community emergency committees (n=54) reported holding any kind of disaster drills OR meetings on roles and responsibilities OR other activities to prepare for a disaster (increased from 14 percent at baseline). o 71 percent of established community emergency committees (n=54) could pass emergency messages up and down to next ‘lower’ administrative level within 60 minutes during a simulation exercise (increased from 17 percent at baseline). o 32% of Red Cross Chapters (n=11) verified the source and content of disaster messages sent to/from established Community Emergency Committees at project’s end (increased from 7%) o 32% of Community Emergency Committees (n=54) verified the source and content of disaster messages sent to/from Red Cross Chapters at project’s end (increased from 12%) Summary of Key Results: CAMI Project Objective Two - Community Emergency Committees 0 10 20 30 40 50 60 70 80 90 100 Community EOCs Comm. Disaster Practice Community Brigades Comm. Messages Passed Timely Source/Content Verified (Chap to/from Comm) Source/Content Verified (Comm to/from Chap) CAMI Indicators Percent of Community Emergency Committees 2001 2003 12 The CAMI Project achieved noteworthy results during the life of the project, particularly with school-based interventions where gains were rapid and significant. This was particularly true in achievement of 10 or more ‘safe conditions’ in schools. Safe conditions included the establishment of risk maps, evacuation maps, visible security and restricted zones on school property, exits from schools and classrooms being clear of obstructions, heavy furniture being secured against walls, desks facing away from windows, etc. Schools also achieved successful results in achievement of safe disaster response practices during a simulated drill. These safe responses were broken down into administrative preparation of the school (e.g., a written emergency plan could be observed at the time of the evaluation, evacuation routes and risk maps were observed, etc.). Key Results -- Schools: • 93 percent of schools had an emergency committee (increased from 39 percent at baseline). • 97 percent of schools had an emergency plan (increased from 23 percent at baseline) • 98 percent of schools had at least one brigade (increased from 39 percent at baseline) • 89 percent of schools have at least 10 CAMI recommended ‘safe conditions’ present in their schools (increased from 13 percent at baseline) • 90 percent of schools demonstrated at least 10 safe disaster response practices during a simulated drill (increased from 53 percent at baseline) Summary of Key Results: CAMI Project Objective Two - School Preparedness and Response 0 10 20 30 40 50 60 70 80 90 100 School EOC School Plan School Brigades 10 Safe Conditions Safe Disaster Response Practices CAMI Indicators Percent of Schools 2001 2003 13 Households4 : Achievements at the household level were steady throughout the project, but slower to achieve. For example, significant gains were made in knowledge that a family should evacuate with family identification papers, but gains were slower in achieving this as actual practice during a simulated drill. Disaster preparedness practices were defined as actions families could take to prepare themselves for a disaster that did not include any physical changes to be made to their households such as having a family disaster plan. Safe conditions in households were defined as actions households could take that involved some non-structural changes to be made to their household environment such as securing dangerous objects on shelves. Findings were tabulated from a sampled survey of households in the four countries, versus Schools and Emergency Committees where census sampling was implemented. • 72 percent (CI: 69.23, 74.76) achieved at least 7 safe ‘practices and conditions’ ((significantly increased from 58 percent (CI: 55.10, 60.89) at baseline)) • 60 percent (CI: 57.11, 62.89) of heads of households correctly stated you apply ‘pressure above a wound’ ((significantly increased from 46% (CI: 41.2, 50.80) at baseline)) • 35 percent (CI: 32.11, 37.89) of heads of households correctly stated they have a pre-existing plan to meet family members in the event of an emergency ((significantly increased from 26% (CI: 23.10, 28.89)) • 29 percent (CI: 24.82, 33.18) achieved at least 10 safe ‘practices and conditions’ in their households ((significantly increased from 17 percent (CI: 14.53, 19.46) at baseline)) • 24 percent (CI: 21.11, 26.89) of households demonstrated correct response to a disaster scenario in their household ((significantly increased from 13 percent (CI: 9.75, 16.25) at baseline)) • 15 percent of heads of households correctly stated that you should evacuate with family identification papers ((significantly increased from 9 percent (CI: 6.20, 11.80)) Summary of Key Results: CAMI Project Objective Two - Household Preparedness and Response 0 10 20 30 40 50 60 70 80 90 100 7 Safe Practices and Conditions 10 Safe Practices and Conditions Apply Pressure Above Wound Pre-Existing Plan to Meet Family Correct Responses During Disaster Drill Evacutate with Family Papers CAMI Indicator Percent of Households 2001 2003 4 As the household survey was a clustered, sampled survey; significance of change was evaluated. See methods. 14 The third objective was to increase technical capabilities of four Central American National Societies to implement community based education programs for disaster preparedness and response. • 1,057 volunteers performed 7, 651 behavior change visits in beneficiary homes. o The resulting coverage rate of target households was 34 percent. • 481 disaster preparedness workshops were conducted, with 4,641 attendees. Cost Analysis: 1) The average cost per person to be covered by an emergency committee and an emergency plan was $5.71 per person, or $12, 250 per community. 2) The average cost per student or teacher to be covered by a school emergency committee and a school preparedness plan was $21.48, with an increased cost of $5.15 to complete a small mitigation project in each school. The average cost per student or teacher covered by both a plan, emergency committee and to have a small mitigation project implemented was $26.63. 3) The key driving factor of proportionate cost by country was the number of beneficiaries served versus number of communities or schools served. a. For example, under Objective #1 El Salvador covered 11 communities, the lowest of all countries. Their subsequent cost to cover a community with a plan and an emergency committee was therefore also the highest at $14,799 per community. However, El Salvador also double the number of beneficiaries of the 4 countries at the community level (n=41,373). At $3.93 the cost-per-beneficiary was significantly lower than the other countries which ranged from $5.33 to $7.43. b. This same pattern held with the school interventions, Guatemala covered 39 schools, approximately twice as many as the other countries. Their cost per school was the lowest at $3,776 whereas all other schools cost between $8,700 and $8,900. However, Guatemala also covered small, rural schools and therefore had the second lowest number of teachers and students covered (5,428). The cost-per￾beneficiary of the school intervention was therefore the second highest at $27.13 whereas in Nicaragua where the project covered urban schools, the cost-per beneficiary was $15.17 with coverage of 8,633 teachers and students. Lessons Learned and Recommendations The evaluation revealed lessons learned and recommendations that can be applied specifically to future regional disaster projects. The project areas selected for CAMI remained one of the most challenging issues throughout the life of the project. Some chapters were chosen for political reasons, and the actual capacities and interests of chapters and communities were not always properly assessed (e.g., if there other designated disaster responders in the area). o Criteria for targeting future disaster preparedness projects should include not only disaster vulnerability, but also poverty indicators, capacity, and interest of the local chapter to respond as well as assessing if other responders are in the area. In addition, the operational EOC’s role should be based on an assessment of needs in the specific area, and the roles and responsibilities of each chapter should be defined in conjunction with other disaster responders in the area. o In future projects, selection of participating Chapters could occur through an application process whereby Red Cross chapters apply to their National Headquarters to participate in a DPP project during which they will develop an emergency committee. This type of competitive process would define the minimum profile for participation (chapter board approval, existence of volunteers, and the ability to sustain this new cadre of volunteers), as well as the required logistical capacity to implement an emergency operations committee). . 15 However, despite these challenges all Chapter level emergency operations centers experienced noteworthy improvements in capacity as demonstrated throughout this evaluation report. Some of the project’s biggest success’s occured in the schools. In short, the CUSE curriculum worked. Further, CAMI teams (and the evaluation team) felt tthere is an on-going, sustainable role for Central American National Societies with the Ministry of Education in implementing school-based preparedness activities. o The Ministry of Education should assume long-term training of teachers in CUSE with Red Cross assistance in the short-term . o In the long–term, future projects should consider negotiating a Red Cross role as a permanent adjunct with the Ministry of Education. Responsibilities could include the training of school brigades and ensuring that instructors follow up on disaster plans. At the community and school levels, drills and simulations worked well. Children responded to drills more attentively than the adults and this might be attributed to the relative lack of practice found in communities and households versus schools. Therefore, community based trainings should focus on the most important aspects of disaster preparedness defined tentatively as: o Safe meeting place if family is not together during a disaster event o Knowledge of what to do about children (where to meet) if in school at time of disaster o Knowledge of evacuation routes o First aid o The need to evacuate with family identification papers during an emergency o Safe zones (house, community) o From whom do you get -- and give -- information Certain interventions were adapted from American Red Cross domestic disaster experience, and some proved not to be appropriate outside the United States (e.g., adopting ‘doors and windows’ opening to the outside). o The parameters of non-structural mitigation need to be better defined in future projects in terms of what defines a safe school or household and/or a secure school or household o Disaster Preparedness projects less than three years in duration should replicate the successes that the CAMI Project demonstrated could be achieved in relatively short periods of time (e.g., in schools and establishment of function disaster response committees), rather than interventions targeting behavior change at the household level which generally takes significantly longer to achieve. o Incremental project activities (e.g., adding CUSE to existing school activities or adding a COE to an existing Red Cross Chapter) are easier to sustain than completely new structures (e.g., training and maintenance of new community brigades). It is crucial to begin planning for sustainability in the project design. A step-down design may be helpful – especially for sustainability of chapter and community emergency committees; where positions funded by the project are gradually shifted financially to the National Society (when appropriate). In general terms, when project participants were queried regarding their satisfaction with their participation in the project many expressed that strong points of the CAMI Project were the trainings offered, the simulations (i.e., disaster drills), and the quality of CAMI project personnel. Mentioned constraints were too little time in the project (especially at the community level), the material lack of support (e.g., lack of distribution of materials and supplies), as well as the fact that not all the schools of participating municipalities were covered. The CAMI Project utilized a training of trainer methodology that tended to be expensive in the initial phases. However, a somewhat longer project time frame would facilitate cost analysis for projects using this sort of methodology. For example, volunteers from San Vicente Chapter in El Salvador had begun replicating community work in 5 additional communities shortly before the project ended, however, these populations were not counted as beneficiaries of the CAMI project, nor were any costs incurred to the project. Had the project been longer, a formal replication phase by non-CAMI staff could have been included and provided a more reliable cost analysis base (and 16 indeed a more reliable indicator of long-term impact). Nonetheless, cost per beneficiary, when allied with evaluation results, provides a useful unit of comparison for similar DPP programs. Staffing needs for disaster operations are different from those required for managing long-term development projects. Technical delegates brought in to a disaster-struck area to implement emergency programs are not always the best suited to manage multi-year, disaster preparedness interventions. Future projects should plan on an appraisal of staff in the post-disaster recovery period to assess the skills and experiences of disaster response staff – and the management needs of long-term development projects. To improve cost analysis in the future ARC should: o Improve beneficiary counting tools for projects through quarterly reporting mechanism o Improve project level financial management to integrate with quarterly reporting programmatic requirements, as well as financial and compliance requirements. A standardized minimum data set of indicators for evaluation of safe conditions in schools and households needs to be further developed and tested as additional guidance becomes available as to what constitutes non-structural mitigation interventions in an international context (i.e., what achievements can a Red Cross Project realistically expect to achieve without major construction activities). Disaster preparedness methods involve use of observation (e.g., observation of evacuation practices). Observation methods are sometimes more complex to implement than standard question/answer evaluation methods; and the pilot methods utilized in the CAMI Project require further validation for validity and reliability. The CAMI Project can feel proud that 79 percent of rural community emergency committees can now get emergency messages back and forth to their 'headquarters' within 60 minutes of a simulated incident (increased from 17 percent at baseline). However, based on current industry knowledge of the efficacy of disaster preparedness activities, it cannot be estimated how many injuries/deaths/loss of livelihood that MAY prevent in the future5 . 5 Email dialogue with WHO/Office of Emergency and Humanitarian Assistance regarding efficacy and efficiency of disaster preparedness activities. 17 Description of the Evaluation 18 I Description of the Evaluation A. Evaluation Purpose. The evaluation to be undertaken is a summative evaluation. (A summative evaluation analyzes the outcomes of a program at the completion of project activities.) As such, this evaluation’s focus is to assess whether or not the CAMI project interventions achieved their intended outcomes in an efficient, effective, and sustainable manner. Based on these findings -- conclusions and recommendations will be made for other ARC and Red Cross/Red Crescent Movement Partners. This evaluation will ensure that our compliance with our grant obligations per ARC’s signed agreement with OFDA is met. As such, the evaluation report will written to ensure that relevant portions of the overall report can be a part of the final report per OFDA Guidelines for Grant Proposals and Reporting. See Appendix A and B for Evaluation TOR and Agenda of Evaluation Team Review Meeting. B. Evaluation Methods. The overall design of the baseline and final evaluation of project outcomes is a before and after study within the CAMI project area. Baseline results are enabling the CAMI project to establish what is already working and what can be improved in Red Cross, school and household preparedness and response. 1. To evaluate disaster preparedness indicators, face-to-face interviews were conducted with key informants (e.g., heads of households, principals). These interviews, which recorded self-reported data, were supplemented by systematic, non-participant observation using structured instruments and guidelines about who and what to observe. 2. To evaluate disaster response indicators, disaster drill scenarios were conducted, using systematic observation as the primary review method. Here we also used systematic, non-participant observation using structured instruments and guidelines about who and what to observe. Key indicators (pre-determined behaviors) were established for CAMI and trained observers used observation checklists during the drills to record the presence or absence of an element, whether a particular event did or did not occur, and/or frequency of occurrences of events. 3. To evaluate overall implementation of the project (process evaluation). Key informant interviews, reviews and analysis of staff interviews and document reviews of CAMI project reports were used to assess the implementation side of the project. C. CAMI Final Evaluation Team Members. 1. Julia Guzman, Nicaragua Red Cross (NRC) 2. Wilfredo Rosario, CAMI Regional Coordinator (ARC) 3. Patricia McLaughlin, Deputy Head of Regional Delegation (ARC) 4. Diana Benitez, OD Regional Delegate, The Federation 5. Christine Leonardo, Manager, Disaster Planning and Preparedness (ARC) 6. Dalia Castaneda, CAMI Coordinator, Guatemala (ARC) 7. Carol Puzone, Technical Assistance, Planning and Evaluation (Evaluation Team Leader, ARC) D. DRAFT American Red Cross Evaluation Guidelines. 1. All projects must report on achievement of expected results 2. Projects of 36+ months must plan for a midterm review as well as a final one. 3. Projects of 36+ months must have an external evaluator as part of the evaluation team, preferably as team leader for the final evaluation 19 4. Projects of $2,000,000 or more, regardless of timeframe must have an external evaluator as part of the evaluation team E. OFDA Results Reporting Guidelines. Results reports are required to be submitted 90 days after the end of the program or annually if the program is extended beyond one year. They emphasize quantitative as well as qualitative data and measure impact using indicators. This is not negotiable. The starting point for results reporting should be the performance baseline established in the first program update and should provide the following for each objective: • A description of assessments and surveillance data used to measure results; • Total number of targeted and reached beneficiaries; • Quantitative and qualitative data that reflect results. Indicators should be used to express this information; • An explanation of successes achieved, constraints encountered, and adjustments made for achieving each objective; • A discussion of the overall performance of the project, including details of any discrepancies between expected and actual results and any recommendations for improving the design of the program; • Success stories; • Overall cost effectiveness should be addressed with particular attention paid to cost savings and/or cost overruns. Other significant cost impacts such as significant exchange rate fluctuations or other types of inflation should be detailed. F. Potential Utilization of CAMI Evaluation Results: • OFDA: Can use lessons learned for future PVO solicitations; • Other CAMI NGOs: Can use lessons learned in future DPP projects; • National Societies: National Societies can design and apply for their own grants utilizing lessons learned from CAMI; • IDRU: The evaluation report can assist in defining standard indicators for community DPP and methodologies for things that work and don’t work; • American Red Cross: Can demonstrate its capacity to implement quality disaster preparedness programs in future proposals 20 Background and Project History 21 II Background and Project History A. Problem or Opportunity Addressed: Hurricane Mitch In October 1998, Hurricane Mitch swept through Central America with sustained winds of 112 kilometers per hour, causing devastating floods and mudslides, massive infrastructure and property destruction, and significant population displacement. Up to 10,000 people were killed, an estimated 3.6 million people were affected, and nearly 100,000 homes were completely destroyed. A number of causal factors led to excess mortality, morbidity and missing persons. In Central America the high poverty levels, lack of access to education, and poor housing and living conditions severely limit the population’s ability to develop. Those living in poverty are generally excluded from the benefits of economic growth and prosperity. These conditions contribute greatly to the vulnerability of natural disasters; hence, it is through education, training programs, and the elimination of poverty that risk is reduced. Other causal factors are poor risk management (preparedness, readiness, and response capabilities) by emergency personnel and an over centralization of disaster services at the capital level. Further, there was poor community knowledge of how to prepare for and respond to disasters, as well as cultural beliefs in myths about the causes of disasters which may have prohibited communities from taking action. B. USAID/OFDA Response In February 2000, as part of the $630 million U.S. Government response, USAID/OFDA announced a three￾year, $11 million Central America Mitigation Initiative (CAMI) for the region, with preference to the most severely affected countries of El Salvador, Guatemala, Honduras, and Nicaragua. CAMI's goal was to reduce or negate the impact of natural disasters in Central America by financing activities that increased the capability of regional, national, and community authorities and organizations to forecast, respond to, and prevent disasters. Disaster preparedness and mitigation activities improve management skills, expand knowledge of how to respond to disasters and what tools are needed to do so, and provide coordination and networking among organizations that can pool their resources and expertise to respond effectively. To promote comprehensive disaster management, OFDA/CAMI financed several critical areas. A major CAMI activity was to be strengthening national emergency systems to enhance their coordinating and operational roles before and during disasters. This included national emergency offices and their municipal counterparts as well as NGOs and other key first responders. CAMI projects also strengthened natural hazards monitoring institutions and early warning systems and promoted the use of standardized methodologies for collecting and analyzing data, with particular attention to providing meaningful information to meet the needs of key national and community decision makers. Technical assistance would be provided to strengthen national hazard monitoring agencies and efforts are made to standardize data collection, analysis, and interpretation methods. C. American Red Cross Response Prior to the issuance of the CAMI APS, ARC had organized and paid for a conference in Honduras with representatives of the four Mitch countries, Belize, Costa Rica and Panama. The Federation was invited and brought a group of delegates serving in these countries. OFDA and PAHO also made short presentations about their organizations. Each national society presented their DDP program and future plans. A book came out of that Conference entitled: "Plan para la Preparacion para desastres en Centro America". At this point, a regional DP committee with representation of the Mitch countries, the IFRC, ARC and SRC were formed. 22 This DP committee met monthly in El Salvador and became active in drafting a paper that defined the role of the Red Cross in each country as an active component of the National Emergency Management Structure in each country. After a lengthy development process, the American Red Cross submitted a proposal and were awarded a 2- year CAMI grant for $1.2 million dollars? The American Red Cross proposed the following project hypothesis: To address causal factors of excess morbidity and mortality, the CAMI Project will build the capacity of four Central American National Societies to develop a decentralized emergency response system that will include trained emergency personnel in emergency operations centers (EOCs) at national, chapter and community levels. To address the communities’ poor risk management, the project theorizes that by increasing community knowledge and reducing beliefs in mythical causes of disasters, the communities’ perceived ‘self efficacy’ to prepare for and respond to disasters will be improved, thereby enabling positive changes in preparedness and response behaviors. The Red Cross project will achieve these results through training of school personnel and school children within their classroom environments, as well as using Red Cross volunteers to train community members within their home and community environments. The project will also train emergency community committees including community brigades in selected Neighborhoods in each country. These combined interventions, will assist CAMI communities to achieve an improved state of preparedness, readiness, and response which will, in turn, contribute to decreasing excess morbidity and mortality during the event of an actual natural disaster in the future. D. Project Timeline Agreement Signed: 15 February 2001 Start Date: 15 February – June 2001. Administrative project start-up; staff hiring. July/August 2001. Detailed implementation plan (DIP) and baseline design, including targeting and staff training. Baseline: Sep 2001 Project baseline data collection and community interventions started. No cost For period Extension6 : 1 Feb-30 April 2003 Granted by OFDA Final Eval: March/April 2003 Final Evaluation Data Collection End Date: 30 April 2003 End of project, draft evaluation report. Duration of Community Activities: 20 Months 6 See NCE Letter of Acknowledgement in Project Files. 23 E. Negotiation of Agreements with National Societies As reported in the August 2002 CAMI Annual Report, after working closely with each ONS and agreeing on the selected Red Cross Chapters and geographical areas where the CAMI project would be implemented, a series of Memorandums of Understanding (MOUs) were needed. Each ONS appointed a CAMI counterpart to coordinate the CAMI project on behalf of the ONS - in El Salvador, Miguel Vega; in Guatemala, Ovidio García; in Honduras, Oscar Fernández, and in Nicaragua, María del Socorro Orozco. An MOU with each ONS was completed. This followed with MOUs in each selected chapter, which included the following: El Salvador (Santiago Maria, San Vicente, and Zacatecoluca); Guatemala (Chiquimula, Escuintla, and Jalapa); Honduras (Choluteca, San Lorenzo, and Tegucigalpa); Nicaragua (Masaya, Rivas, and Tipitapa). And finally, MOUs or letters of agreement were finalized with municipalities, minister of education and others, as deemed appropriate. This process took considerably more time than originally expected due to administrative delays, school strikes, and availability of responsible officials. F. Background and Operating Environments of Selected Countries In Central America the high poverty levels, lack of access to education, and poor housing and living conditions severely limit the population’s ability to develop. Those living in poverty are generally excluded from the benefits of economic growth and prosperity. These conditions contribute greatly to the vulnerability of natural disasters; hence, it is through education, training programs, and the elimination of poverty that risk is reduced. 1. El Salvador Country Background El Salvador is the smallest of the seven Central American nations with a land area of 21,040 km2 , a population of 5.9 million people, and a population density of 261 people per km2 (1999 estimates). Approximately 46 percent of the population lives in the metropolitan area of San Salvador. With the highest population density and the smallest land area in the region, El Salvador presents a higher intrinsic vulnerability to the effects of natural disasters than the other countries in the region. The principal hazards that El Salvador faces result from earthquakes and volcanic eruptions. The effects of tropical storms and hurricanes from the Caribbean Sea and landmass movements pose less of a threat. Fifty-one percent of the country’s population lives in urban areas, 30 percent of which are not connected to potable water. Fifty percent of urban dwellers do not have any type of sewer service, 15 percent do not have any type of sanitation facilities, and 15 percent still have only dirt floors. Lack of land-use and urban development policies have led to deterioration in the quality of life, the environment, and civil security and greater vulnerability to the effects of natural disasters. Basic services programs are disparate in focus, and efforts are not coordinated among the various agencies providing services. There is no underlying development strategy unifying the actions. Institutional and political interests take precedence over provision of services. In some cases there is duplication of effort. 24 2. Operating Environment of the El Salvador Component The project’s initial start met some challenges in the selection of participating chapters. Zacatecoluca, one of the chapters selected for the project, was considered by the National Society (NS) as not having the capacity to engage in such a project. This proved to be a correct analysis as the chapter lacked leadership, volunteers, and other resources. This resulted in falling short of the expected outcomes in four of the five communities within the Zacatecoluca area. Although, the work in schools was good, lack of coordination with the Ministry of Education was a problem at the start of the project. Changes in selected neighborhoods for the final evaluation had to be made due to some periodic violence occurring in some communities in the area of Santa Maria. Elections and primaries had some minor affect as trips to the field needed to be reprogrammed. In late February 2002, the area of San Vicente suffered through some serious flooding, which halted the progress of the project for a few weeks. As a result of the capacity building efforts of CAMI, trained CAMI volunteers responded immediately to this flooding with damage assessment and community evacuations. 3.Guatemala Country Background Guatemala is a multilingual, multiethnic, and multicultural society with an estimated population of 11 million inhabitants in 1999. Thirty five percent of the population resided in urban areas and 65 percent in rural areas. Fifty eight percent of the population lives in extreme poverty. The illiteracy rate for persons over fifteen is 36 percent, but as high as 82 percent in some rural areas (PAHO, 1999). The 1960s saw the beginning of an armed conflict that lasted more than thirty-six years. The most intense fighting occurred between 1980 and 1984. For the affected indigenous population, community wide traumatic and stressful experiences included wholesale destruction of villages, internal displacement, asylum in refugee camps in Mexico, and repatriation from refugee camps back to Guatemala. The indigenous population was the most affected in terms of health, education, loss of traditional values, and community relations. The area of Guatemala severely affected by Hurricane Mitch was the coast, mainly in the departments of Izabal, Zacapa and Peten, and the Pacific Coast region. The National Disaster Reduction Commission (CONRED, 1999) estimated that more than 100,000 persons were evacuated and 268 died. Thousands of homes were damaged, and crops were destroyed. 4. Operating Environment of the Guatemala Component A key operational factor in operating the CAMI Project in Guatemala was the lack of support by the NS in Guatemala, during most of the project, which created a few problems for the CAMI staff. This was complicated by the frequent turnover of key personnel and the lack of volunteers in the selected chapters. The area most affected was the establishment of Centers of Emergency Operations (EOCs). In all of the chapters, EOCs were equipped but overall did not meet operational capabilities. Contributing to this was the lack of established emergency protocols and procedures at the headquarters, as found in some of the other countries--with the exception of El Salvador. Appropriate materials were not reviewed, validated, or approved for publication. This created additional work for the CAMI staff and had an overall negative affect on the project. The work in the schools went extremely well but, during the final evaluation, a teachers’ strike forced the delay and incomplete evaluation of CAMI schools in the selected areas. 25 5. Honduras Country Background Honduras has an area of 112,088 km2 with a population of almost 6,000,000 people. The majority of its territory is mountainous, with over 75 percent of its land area on slopes. Honduras’s principal environmental risk comes from the effects of the abundant rains that occur from June to November, as well as its direct exposure to hurricanes that form in the Caribbean Sea. Hurricane tracking records indicate that Honduras is hit by a major hurricane more than once every decade. Historically, flooding has greatly affected the three major valley systems, mentioned above, where most of Honduras's agricultural production occurs. Landslides constitute the other major natural hazard in Honduras. Due to its steep topography and the deteriorated condition of its natural resources, landslides and mudflows often occur as a result of prolonged rains. The recent experience of Hurricane Mitch shows the reality of this threat for Tegucigalpa, the capital of Honduras, which was affected by landslides and mudflows. When Hurricane Mitch hit Honduras in November 1998, it dramatically changed the country, already one of the poorest countries of the region. PAHO/WHO and the Honduran government reported an estimate of 6,600 people killed and 8,052 persons missing. There were also 11,998 persons reported injured. Approximately 1,375 shelters had to be established to accommodate 427,138 persons affected by this disaster. These shelters were closed in December 2002. 6. Operating Environment of the Honduras Component Although, the operating environment in Honduras was relatively quiet, there were some initial problems with a school strike that had some affect in the delay of the baseline study. Work with the NS went rather well but, again, the lack of established emergency protocols and procedures at the headquarters affected the operational readiness of EOCs at the chapter level. The target number of Chapters was adjusted from 12 to 11 when it was realized that one of the target Chapters in Honduras (Choluteca) functionally ended up utilizing the Tegucigalpa National Headquarters'. The lack of interest in some designated communities in pursing some of the project’s objectives such as the establishment of emergency committees, also had some negative impact. The CAMI coordinator in Honduras was replaced in March 2002. Even though, this had some effect on the project since the position was vacant for approximately a month, the remaining staff worked diligently until a replacement was named. 7. Nicaragua Country Background Nicaragua has a population of 4.5 million people of which 50 percent are living in extreme poverty. Illiteracy rate is 34 percent. Trailing Haiti, the gross national product per capita is the lowest in the Americas (PAHO, 1999). The government elected in 1990 inherited a country recovering from nearly a decade of civil war. The current government, elected in 1996, leads a divided and polarized society. The results of the Hurricane Mitch affected 867,752 people directly or indirectly. The official statistics are 3,045 dead. 50,000 homes were completely destroyed and 94,000 homes were partially damaged. Nicaragua's central bank estimates the losses at $1.5 billion, not including losses in the agricultural sector or environmental impact (PAHO, 1999). The northern region of the country was most affected. In Posoltega losses were particularly severe. A total of 2000 people were crushed and drowned by a mudslide down the slopes of the volcano, Las Casitas (Casa de Cultura, 1998). 26 8. Operating Environment of the Nicaragua Component The operating environment in Nicaragua, was similar to the other countries regarding NS support. Due to board and presidential elections in January 2002, two of the three participating chapters, Masaya and Rivas, were somewhat disrupted by a reduction in leadership volunteers, which ultimately affected the project. After a few months, the Rivas chapter was back on target. However, changes had to be made to target communities within the Chapter of Masaya See Table xx for list of final communities. As in other countries, the lack of established emergency protocols and procedures at the headquarters affected the operational readiness of the EOCs. As in El Salvador, torrential rains in late February 2002 caused severe flooding in the areas of Managua and Tipitaca. This caused some minor delays in the progress of the project but as in El Salvador, the CAMI volunteers responded to the challenge. Government elections or labor strikes have some minor impact as trips to the field needed to be reprogrammed but, overall, the working environment was stable. 27 III CAMI Components SAFE Households + SAFE Schools = SAFE Communities Weather Service Monitoring Notification to President’s Office Office of the President Monitors weather service reports Red Cross monitors weather service reports National Level Government Emergency Operations Committee And Ministry of Health/Education National Level Red Cross Emergency Operations Committee in Liaison with Ministry of Health and Ministry of Education Gov’t Mun. EOC+MoE RC Chapter EOC RC Chapter EOC RC Chapter EOC Gov’t Mun. EOC+MoE Gov’t Mun. EOC+MoE Households without School Children Households with School Children Neighborhood Schools RC RC The Red Cross Volunteers & are part of the community brigades, as well as do home visiting In Neighborhoods Community Brigades Where Community (Aldeas, Canton, Comarcas) Emergency Committees exist, five will be recruited to participate in CAMI. A trained RC volunteer will be integrated into the committee. The RC rep will provide direction and supervision of Red Cross activities at the Neighborhood level. Where Community (Aldeas, Canton, Comarcas) Emergency Committees do not exist, five Communities will be established for each participating chapter. This level will provide direction supervision of Red Cross activities at the Neighborhood level. Community Workshops Obj. 1,2 and 3 link from RC National Society down to Neighborhood Level 28 A. CAMI Project Goals and Objectives: 1. Project Goal The project’s goal is to decrease excess morbidity and mortality caused by natural disasters by increasing the capacity of four Red Cross National Societies to monitor and respond to such disasters. 2. Objectives a) The first objective is to improve the timeliness and appropriateness of mitigation responses of Red Cross national societies and communities to disaster through developing sustainable protocols and training personnel in operation centers and emergency committees (APS, Objective 1, Activities 2 and 3). b) The second objective is to increase knowledge, self-efficacy, and skills in disaster preparedness and response of individuals and households through incorporating risk management concepts in formal and informal education methods (APS, Objective 3, Activities 2 & 3). c) Increased technical capabilities of four Central American National Societies to implement community-based education programs for disaster preparedness and response. 3. Planned Versus Actual Objectives and Indicators The first two objectives remained the same throughout the life of the project; from proposal submission through to the final evaluation: The third objective was revised. In the original proposal submission, the community stress mitigation objective was listed as “Decrease adverse psycho-social effects of disaster for the existing vulnerable in two countries through use of stress (psycho-social) reduction activities (APS, Objective 6)”. During the development of the detailed project implementation plan for the project, ARC added a new third objective for its own internal purposes: Increased technical capabilities of four Central American National Societies to implement community-based education programs for disaster preparedness and response. 4. Proposed versus Actual Indicators The measurement of disaster preparedness and planning projects is an emerging field. As such, there was significant fluctuation in the design of indicators for the project before implementation of the baseline assessment. These revisions are reflected in Appendix xx. The CAMI final indicators are listed in the attached table. B. Targeting 1. Planned versus Actual Geographic Targeting. In the original proposal, Managua, Rivas, and Chinandega were chosen as the three implementing locations in Nicaragura. In Guatemala Guatemala, Retalhuleu, and Tecun Uman were chosen. In Honduras the communities were Tegucigalpa, Choluteca, and San Lorenz and in El Salvador, San Salvador, San Vicente, and Sonsonate were chosen. 29 This targeting changed significantly from the time the proposal was written to the time of actual project implementation. The target population coverage increased from 80,000 in the proposal to 123, 175 in new geographic areas. 2. Criteria for Selection of Target Areas (Communidads) a) Experienced disasters on a frequent basis—high vulnerability; b) Did not receive direct service from Emergency Management Agency in their country; c) Had previous relationship with the Red Cross; d) Had “ buy in” from the community. 3, Criteria for Selection of the Schools a) Schools with more than 15 teachers b) Schools with three sessions per school (morning, afternoon, and evening) 4. Criteria for Selection of the Neighborhoods and Households a) Selected Neighborhoods around the schools 5. Final Choices a) Guatemala: Jalapa, Chiquimula, Escuintla Total Target Population: 33,612 in 15 Communities. The average sized Community had a population of 2,241 (range: 680, 4,279) with a median population of 2,050 b) Honduras: Tegucigalpa, Choluteca, San Lorenzo Total Target Population: 24,601 in 15 Communities. The average sized Community had a population of 1,462 (range: 370, 4,500) with a median population of 1,160 c) Nicaragua: Masaya, Rivas, Tipitapa Total Target Population: 42,692 in 15 Communities. The average sized Community had a population of 1,941 (range: 105, 7601) with a median population of 1,190 d) El Salvador: Santiago de Maria, San Vicente, Zacatecoluca7 Total Target Population: 22,270 in 15 Communities. The average sized Community had a population of 1,485 (range: 625, 3,000) with a median population of 1,250 6. CAMI Project Area as a Whole: Total Target Population: 123,175 in 60 Communities. The average-sized Community had a population of 1,794 (range: 105, 7,601) with a median population of 1,324 7 Communities targeted within these Chapter areas were revised in Zacatecoluca after the implementation of the baseline survey. The revised Communities. See Appendix xx. 30 Table B. CAMI Programme Area PAÍSES SECCIONALES/FILIALES/DELEGACIONES Nombre de Communities Jalapa 1) Sabanetas 2) Las Flores 3) San Carlos Alzatate. 4) Tabacal 5) Pino Zapotón Chiquimula 1) Aldea Shororaguá 2) Aldea el Matazanos 3) Aldea El Pinalito 4) Aldea Santa Bárbara 5) Aldea Maraxcó GUATEMALA Escuintla 1) Aldea Llanitos 2) Aldea Obero 3) Aldea Las Guacas 4) Aldea Cuyuta 5) Aldea Guardianía Santiago de María 1) El Modelo 2) Barrio El Centro 3) Monte Bello I 4) El Nao 5) El Cerrito (Las Playitas) San Vicente 1) Caserío San José 2) Barrio Ceoncepció 3) Agua Caliente 4) San Francisco Tehuacán 5) San Antonio Caminos EL SALVADOR Zacatecoluca 1) San Luis Talpa 2) San Rafael 3) Santa Lucía 4) Tepechame 5) El Pajal Tegucigalpa 1) Barrio El Chile 2) Colonia Venezuela 3) Barrio El Reparto 4) Colonia 19 de Septiembre 5) Colonia Canaan Choluteca 1) El Palenque 2) Playas de Iztoca 3) Las Arenas 4) Fray Lazaro 5) Copal Arriba HONDURAS San Lorenzo 1) San Jeronimo 2) La Criba 3) Agua Sarca 4) Laure Abajo 5) La Puente Masaya Monimbo Valle de la Laguna de Apoyo Pacayita Tisma Urbano Los 24 Rivas El Estillero Las Salinas Nancimi La Chocolata Veracruz NICARAGUA Tipitapa San Francisco Libre Las Maderas Banderas Noel Morales Yuri Ordoñes 31 Relevance of Interventions to Achievements of Project Results* *Data Sources: Staff Survey conducted by Project Manager; Project Manager; Evaluation Team Leader, Four CAMI National Societies 32 III Relevance of CAMI Technical Interventions to Achievement of Project Results. A, Changes in Original Focus and Targeting of Beneficiaries During the design phase of the project, coordination and communication took place with the NS and Ministry of Education; however, all respondents felt that the project assessments were incomplete, and were not truly participatory with the national societies. For example, project targeting and criteria for participation remained a challenge throughout the life of the project; particularly under activities for Objective #1. The first section focuses on changes in what the project was targeting to achieve. In addition to the geographic areas of focus being modified, planned versus actual targeting of beneficiaries sometimes changed as well. For example, the target number of chapters shifted from 12 to 11 when it was realized that Tegucigalpa was a National HQ, with chapter responsibilities. Although the target number of communities remained at 60 from the time of the proposal to the end of project, the actual communities being targeted changed. This caused the revised target population of 127, 915 to decrease to 123, 175 at the end of the project. See Table B. Table B. Planned Versus Actual Targeting of Beneficiaries Proposal As of Nov 2000 DIP As of Aug 2001 End of Project Through 30 April 2003 Target RC National Societies (National Level) 4 4 4 Target RC Chapters (branches) 12 12 11 Target Communities 60 60 60 Target Community Emergency Committees NA 85 60 Target # of Schools 40 65 65 Target # of teachers NA NA NA Target # of students NA 24,471 NA Target Population 80,000 127, 915 123, 175 The CAMI project aimed to reduce excess morbidity, mortality, and property loss by: 1) Training chapter staff and local emergency committees to develop, become trained in, and institutionalize standard disaster response procedures. 2) Training households and schools in disaster preparedness/response. 3) Training a corps of community-based Red Cross volunteers to provide disaster preparedness training and follow up in their communities. Although the projects all adapted methods to their unique environments, the following represents the broad categories of interventions that ALL countries implemented. 33 B. Objective #1 -- Key Interventions: Specific Interventions under Objective #1 (improve the timeliness and appropriateness of mitigation disaster responses of Red Cross national Societies and communities through developing sustainable protocols and training personnel in operation centers and emergency committees )were as follows: 1. Establishment, training and equipping of 12 emergency committees of Red Cross Chapters. New training materials included courses in psychological first aid, ‘Safe community and Safe Homes’. Training materials adapted from existing courses developed originally by OFDA included “Damage Assessment and Needs Analysis” (Evaluacion de Danos y Analisis de Necesidades/EDAN and “Damage Assessment and Needs Analayis – tools for Decision Makers”: (Evaluacion de Danos y Analisis de Necesidades - Toma de Decisiones/EDAN-TD). The project also developed and adapted evaluation instruments for community drills. 2. Development of Emergency Plans for Red Cross Chapter Emergency Committees: These plans included general plans of assistance and response, the plan of operations and procedures for the chapter emergency committees. Disaster plans were written, based on each country’s national disaster plans which, in turn, were based on specific laws. For example, in Nicaragua the plan was based on the Law 337. Therefore, there was no standardized format for EOC plans from country to country. Incorporation of 24/7 EOCs worked well in El Salvador (3 out of 3) and not at all in Guatemala (0 out of 3). There were many reasons for this. a) Some communities had other emergency committees in place such as CONRED in Guatemala. As such, the need for a Red Cross Chapter to be operational 24/7 would have been a duplication of roles and responsibilities with other agencies. The question should have been: If a EOC is necessary in the area, how will it integrate with other organizations? b) Support of the national level had significant implications for the success, preparedness, and response capabilities of chapters. In some chapters, the lack of a functional body of leadership volunteers (Boards) detracted from the CAMI team’s ability to make important decisions affecting the project. Also in Nicaragua, in areas such as Tipitapa and Rivas the chapter had little capacity such as board leadership. 3. Training of National Level Emergency Committees of the Red Cross Societies. To initiate ”Protocols and Procedures” for EOC at the chapter level, much work was needed at the national level. This was not originally planned for in the original project timeframe, thus some EOCs could not be raised to the fully operational level (24 x 7) during the life of the project. The national level of the participating national society was gradually re-integrated back into the project through trainings in EDAN (see above), CUSE (see description below), and shelter/site planning. 4. Development of Emergency Plans at the National Level Emergency Committees: Templates used to develop these plans were adopted from the national response plans of each country. 5. Simulations and Drills. Drills and simulations were developed specifically around the issues of information and decision making with chapter and community emergency committees. In addition, in Nicaragua the newly formed Chapter Emergency Committees participated in a National Level Full Scale Exercise in December, 2002. This activity was part of a national effort to enhance preparedness and response in Nicaragua. 34 6. Equipping Chapter EOCs and National Level Emergency Committees: Equipment was provided to chapter and national levels to enhance their communication/information sharing capacity. See Appendix C for complete list of donated equipment. a. National Level: Donations to national-level emergency committees varied. The most donations were made in Guatemala including a computer and related accessories and the least in El Salvador, where no donations were made. b. Chapter Level: Donations included fax machines, maps, file cabinets, fans, computers, cameras, radios, and t-shirts. 7. Implementation of micro-projects in chapters (equipping volunteers): Equipment for use by the Brigades and other volunteers included protection helmets, rescue lines, radios, first aid kits, rigid stretchers, extinguishers, childbirthing kit, boots, writing-desks, and mobiliario(silla), batteries for radio bases as well as the rehabilitation of an office to become the ‘situation room’. The main changes from the original concept to implementation revolved around the establishment of the emergency committees at both the Red Cross chapter and community levels. An example of this was the project concept of developing 12 chapter emergency operations committees to be emergency responders 24 hours per day, 7 days per week. Appropriate targeting and criteria for selection of emergency committees remained a constant challenge throughout the project and the selection of chapters changed multiple times before the final decisions were made in September 2001. The concept of emergency committees at the Neighborhood level was also modified during the life of the project. In places where Communities covered multiple Neighborhoods, the Neighborhoods were merged into forming one communidad committee to cover all those Neighborhoods. As a result, instead of having 96 Neighborhood-level emergency committees, the project targeted 60 Communities. Some of these Communities were single Neighborhoods, and some were clusters of Neighborhoods. Clustering mainly occurred in El Salvador and Nicaragua where larger, urban areas were targeted. B. Objective #2 (Increase knowledge, self-efficacy, and skills in disaster preparedness and response of individuals and households through incorporating risk management concepts in formal and informal education methods). 1. School Interventions a) Implementation of “Curso de Seguridad Escolar/CUSE”: One of CAMI’s main interventions with the national society counterparts was implementing the CUSE course in target schools. CUSE is a course directed at school administrators and teachers that aims to provide techniques in how to teach effective disaster preparedness to students. It is hoped that disaster preparedness becomes part of the school plan and students and teachers can protect themselves in case of a disaster in the schools (fire, flood, earthquake, and hurricane). The course is conducted in collaboration with the Ministries of Education. CUSE is NOT a curriculum for teachers to teach in their classrooms. Rather, it is a course directed at teachers to produce specific ‘outputs’ for a school – that will make the school safer in the event of an actual disaster (e.g., development of risk maps, school emergency plans, brigades, performance on evacuation drills, etc.). CUSE was originally developed in Costa Rica by Mr. Manuel Ramirez, an OFDA consultant. The CAMI team modified it during the life of the project to include the addition of ‘psychosocial’ brigades. Brigades are groups of students, coordinated by teachers, who are trained and given specific tasks (e.g., first aid, prevention and control of fire, etc.) in the event 35 of a disasters. CAMI also modified the CUSE monitoring instruments to include an evaluation of the functioning of the school brigades during a disaster. b) Implementation of Micro-projects. Small mitigation projects were funded by CAMI in the last quarter of the project. Examples included installation of tubes for drainage, fencing the perimeter of the school, etc. In addition to the CAMI project, micro-projects were also put into place based on school emergency plans that were developed during the project. As noted in the financial section of this report, the budget allocated to salaries was reduced, and the savings were put towards expanding the small mitigation projects with. The project established criteria that the community emergency committee and the school emergency committees used to evaluate their risks/vulnerabilities and to come up with a project to propose for funding from their disaster plan. CAMI ended up funding 15 mitigation projects per country, each with a budget ranging from 1,500 – 2,000 USD. See Appendix D for complete list of Mitigation Projects. c) Linking Department to Municipal Level. The project added a component of training departmental and municipal technicians to link to the Ministry of Education. d) Equipping of target schools. Selected schools were equipped with extinguishers, first aid kits, signals/signs for evacuation routes and danger, stationery stores, and miscellaneous training materials including project-developed brigade training manuals. e) T-shirts were donated to members of the School Brigades. Development of Emergency Plans including risk maps and plans of action . As with the emergency committee plans of the chapter emergency committees, disaster plans were developed based on each country’s national disaster plans which were, in turn, based on specific laws. For example, in Nicaragua it was based on the Law 337. Therefore, there was no standardized format for EOC Plans from country￾to-country. 2. Communities and Households. a) Implementation of a ’Safe Home’ curriculum which was used as the educational instruction guide during home visits and community workshops. b) Community Workshops: Community trainings were implemented for emergency committees, community brigades, and community members who were interested in attending. Community Brigades were a group of trained volunteers (who may also be Red Cross volunteers) who live in the community. Once an emergency alert is activated, the brigades would be activated to perform specific functions such as: drills, search and rescue, first aid, shelter, food preparation, etc. As such, community trainings were taught on topics of: first aid, evacuation, damage assessment, and shelter/site planning. Instruments for evaluation simulations in communities and households were also developed during the project. T-shirts, training materials, and first aid kits were donated to community brigades. c) Behavior Change Communication with Households: Volunteers made house visits to train household members in first aid, teach the content of their community’s emergency plan , and develop family emergency plans. d) Community Simulations and disaster drills. Community emergency committees and households in the target areas performed drills. 36 3. Community Trainers Trainers of communities varied from country to country, and changed over the life of the project. Some countries used a training of trainers model for volunteers--who then implemented the community and household trainings (e.g., El Salvador). In other countries, CAMI staff were involved in direct implementation of trainings (e.g., Guatemala). See Appendix E for complete list of community training courses. 4. Non-implemented Interventions Two community level interventions were originally planned but did not take place. The implementation of billboard, radio, and television messages was eliminated due to financial limitations of the project, and coverage of these types of communication by other projects. Also, interventions targeting changes in myths and self-efficacy were not implemented as baseline data indicated these were not major problems. C. Objective #3 (Increase technical capabilities of four Central American National Societies to implement community based education programs for disaster preparedness and response.) 1. Volunteer Training Originally, there was no intent to actively recruit and train Red Cross volunteers. However, once the project began implementation, the shortage of volunteers at the chapter levels became apparent and thus, plans to recruit and train volunteers to support the project’s community education activities were added. This led to the addition of a third objective that focused on achievements in capacity building from a volunteer base. The addition of this component was also added for sustainability with the National Society. Specific roles, responsibilities, job descriptions, and recruitment/maintenance policies for the volunteers training during CAMI were not developed during the project. 37 Quantitative Evaluation Results * *Data Sources: Baseline / Final Evaluation Community Based Data Collection 38 IV Quantitative Evaluation Results A. Objectives of Quantitative Baseline Study and Final Evaluation. 1. To establish pre-post intervention results of preparedness, readiness and response behaviors of the existing Red Cross emergency response system (EOC) in target areas 2. To establish pre-post intervention results of preparedness, readiness and response behaviors of a sample of participating schools in target areas: 3. To establish pre-post intervention results of preparedness, readiness and response behaviors of a sample of participating households in target areas; B. Lessons Learned from baseline study to Final Evaluation 1. What worked well in the emergency response structure, schools and households; 2. What can be improved in the emergency response structure, schools and households; 3. What external factors might influence preparedness, readiness and response that have not previously been considered. C. Key Definitions 1. Household: A group of people living within the same structure and sleeping under the same roof8 . 2. Head of Household: The person who self-reported themselves as head of household during baseline and final evaluations. 3. Rural versus. Urban: Communities with a population greater than 20,000 are considered to be urban areas. All other populations are considered ‘rural’. 4. Children: Individuals less than 16 years of age. D. Data collection Methods. The baseline and final evaluation was made up of the three following survey tools:. 1. Emergency Committee Census Survey (Objective #1): Preparedness and response questionnaire composed of self-reported information about the committee, followed by observation of key disaster preparedness practices and implementation of a timed, communications drill. 2. School Census Survey (Objective #2): The methodology has been adapted from the CUSE curriculum and uses similar methods as described above. 3. Sampled Household Surveys (Objective #3): Individual face-to-face interviews were completed with a sample of households in the target area. Interviews were supplemented by two direct observations. One observation examined the physical preparedness of households (safe conditions); the second was observation of response practices during a disaster “scenario” at the end of the interview. E. Sampling. 8 Note the slight variation standard definitions such as ‘sharing the same purse’ or ‘eating from the same pot’ for a disaster preparedness project. Here, we emphasized ‘sleeping under one roof’ … a more important factor in DPP. 39 1. Red Cross-Community EOCs. Census of all participating Red Cross chapter EOCs and community-level emergency committees. There was a 100 percent response rate. 2. Schools. Census of all participating schools (65 participating schools with 64 matched pairs analyzed in the final evaluation). Although the participating schools may have more than one session or school per day, these sessions are generally taught and managed by the same teachers and staff. Thus, each school was only counted ‘once’ in the sample. There was 100 percent response rate. 3. Households. Sample Size was determined as: (3.841*23257*0.5*0.5)/(((0.05*0.05)*23257)+(3.841*0.5*0.5))*2*10 percent for a total of 840 households (including 10 percent oversample). Probability Proportionate to Size (PPS) Cluster sampling was used. This method entails first selecting a sample of Neighborhoods and subsequently a sample of households to interview within each of those Neighborhoods. PPS means that larger clusters or Neighborhoods were given a greater chance of selection in the sample than smaller Neighborhoods. Household selection utilized “segmentation”. When combined with the sampling method described above, our sample had the least amount of bias and was self-weighted making analysis of the data much simpler. Completed interviews were located within 42 clusters of the four countries, in segments of 20 households each (approximate). F. Analysis Methods and Calculation of Confidence Limits for Household Survey 1. Confidence Level Analysis of all baseline data was conducted in SPSS Version 10.0 for Windows. To ensure ability of the CAMI team to easily replicate findings in other programs, confidence limits (for household survey only) were calculated manually in an Excel Spreadsheet using the following formula: P = p + z √(pq/n′), where n′ = the effective sample size of the sample or sub-sample. When we indicate that we are 95 percent confident that the opinions of our survey respondents do not differ from those of the target population, this indicates our confidence level. The following tables describe the results of the CAMI survey, and our “margin of error” as +/- 4 percent, for example, this refers to the confidence interval. For example, within a community survey, we may find that 64 percent of respondents indicated that another adult cares for their children. Based on the design of the CAMI evaluation, we have a 95 percent confidence level that across all members of the CAMI target population (i.e., all residents of the Neighborhoods), the percentage of people that have an adult caring for children is between 59 percent and 69 percent (64 percent +/- 4 percent), which is our confidence interval. Survey Response Rates: Baseline and Final Evaluation 40 Baseline Sep 2001 Final Evaluation Mar/Apr 2003 Households 93 percent 85 percent Schools (census sampling) N=65 N=70 Chapters (census sampling) N=12 N=11 Community Emergency Committees N=85 (Neighborhood Level) N=56 (Community Level) Households Contacted and Completed Interviews September 2001 March 2003 Frequency Percent Frequency Percent Completed Interviews 824 92.8 827 84.6 Partly Completed 1 0.1 0 0 Not at Home 48 5.4 58 5.9 Moved Away 11 1.2 15 1.5 House Abandoned 11 1.2 20 2.0 Bad Address 0 0 1 0.1 Refused 5 0.5 48 4.9 Other 0 0 6 0.6 Total 889 100.0 977 100.0 2. Data Limitations Several limitations of the survey methodology and findings require discussion. (See lessons learned in Monitoring and Evaluation). a) First, many household interview questions relied on “self-reported” information from respondent households. Response bias by respondents (e.g., telling the interviewer what you think he/she wants to hear) is a commonly reported limitation of household surveys. b) Second, although the assessment methodology added the observation component to ensure more objective data beyond that of self-reported data, observer presence may have altered what took place during the baseline assessment. Also, due to a variety of reasons, observations of safe conditions, especially at the household level, should be interpreted with care as not all observations may have been made following instructions. c) Third, data collectors and supervisors were of varying skills. Some were experienced interviewers while others were first-time surveyors. Although there were no particular biases in where these interviewers were placed, this inequity of interviewing experience may affect the comparability of some aspects of the data. Also, the interviewers changed from the time of the baseline study to the time of the final evaluation. For example, in the baseline phase, it was determined that university students would be used as data collectors for the household survey but in the final evaluation some of 41 the trained volunteers from the CAMI project were used. These volunteers had less training, and their level of education was lower than the original interviewers participating in the baseline study. This affected the data quality in the final evaluation. d) e) Fourth, the pilot test prior to implementation of the baseline survey attempted to calculate inter-rater reliability of observation questions/sections in each of the household and school questionnaires. However, due to the fact that the pilot test was also the ‘practical’ training component, not all interviewers recorded their cover sheets correctly, thereby making it impossible to match households for the inter-rater comparison. However, lessons learned from the pairing of observers during the pilot test yielded significant findings to improve overall reliability. Further, the internal consistency of the final baseline findings is also indicative that both inter-country and inter-observer reliability was high. f) Fifth, this baseline and final survey resulted in somewhat more-than-usual missing data and “don’t know” responses. This may reflect the lack of experience of both the Red Cross and community members in asking and answering questions related to disaster preparedness and response. g) Sixth, during the baseline implementation, the team had to change five of the Neighborhoods (villages) where the project will actually be implemented. In the affected Neighborhoods (where data was collected; but are no longer part of the project); the sample contained two clusters with a total of three segments. To mitigate any effects on comparative findings, , we have agreed, in consultation with Tulane University, that in the follow-up survey, we will choose a sample of three segments from the five replacement Neighborhoods following the same sampling procedure used in the baseline survey. The data from these Neighborhoods will provide follow-up survey measures to be compared with the data from the three segments in the "lost" Neighborhoods. Given that the number of affected Neighborhoods is only a small percentage, and the replacement Neighborhoods are comparable in characteristics to the lost Neighborhoods, the resulting estimates of impact are quite defensible. As a sensitivity test, we may rerun the impact analyses excluding the lost Neighborhoods from the baseline survey data and the replacement Neighborhoods from the follow-up survey data and see what, if any, difference this makes in the estimates of impact. If none or small, this will add to our confidence level that the loss of the five Neighborhoods did not affect our comparative findings from 2001 to 2003 to any significant degree. The following tables display the summaries of results. 42 Table B. Summary of Results: Key Project Indicators: : MAY 20039 Objective Indicator Baseline Value Final Value Change from Baseline Origin Targ Met #1 Improve the timeliness and appropriateness of mitigation responses of Red Cross national Societies and communities to disaster through developing sustainable protocols and training personnel in operation centers and emergency committees (APS, Objective 1, Activities 2 and 3). 1. 85 percent CR Chapters reporting holding any kind of exercise or practice for a disaster in past 6 months. 2. 85 percent Communities reporting holding any kind of disaster drills OR meetings on roles and responsibilities OR other activities to prepare for a disaster. 3. 85 percent CR Chapters who can pass Emergency Messages up and down to next ‘higher’ administrative level within 30 minutes during a simulation exercise. 4. 85 percent Community Emergency Committees who can pass Emergency Messages up and down to next ‘lower’ administrative level within 60 minutes during a simulation exercise. 17 percent 14 percent 50 percent 17 percent 64 percent 94 percent 84 percent 71 percent 276 percent 571 percent 68 percent 318 percent No NA No No #2 Increase knowledge, self-efficacy and skills in Disaster Preparedness and response of individuals and households through incorporating risk management concepts in formal and informal education methods (APS, Objective 3, Activities 2 and 3). Preparedness/Readiness Households: 1. 80 % of households have at least 5 CAMI recommended ‘safe conditions’ present in their household. 80 percent of households have at least 10 CAMI recommended ‘safe conditions’ present in their households 2. At least 50 % improvement in Perceived Disaster Self-Efficacy of Heads of households from baseline 3. At least 80 % of households report children < 16 are not left home alone OR a neighbor is informed that the children are alone in the house Schools 1. 80 % of schools have at least 5 CAMI recommended ‘safe conditions’ present in their schools 80 % of schools have at least 10 CAMI recommended safe conditions. 72 % 17 % 6 % 64 % 89 % 13 % 83 % 29 % 6 % 60 % 98 % 89 % 15 % 71 % … - 6 % 10 % 585 % Yes NA No No Yes NA 9 See Appendix xx. Questions comprising the indicators were modified as some questions were eliminated from the final analysis due to validity errors. BOTH Baseline and final evaluation data were re-analyzed without the invalid questions, and are presented here. As the survey instruments were not changed baseline to final, and both sets of data were re￾analyzed, the comparison baseline to final evaluation was not affected. 43 Objective Indicator Baseline Value Final Value Change from Baseline Origin Targ Met 1. 80 % of schools have student, teacher, staff attendance taken every day. 80 % of schools have # of students present posted in classroom Response Households: 1. At least 75 % of households demonstrate correct response to a disaster scenario in their household School 1. 80 % of schools demonstrate 20 correct responses to a disaster scenario in their school 80 % of schools demonstrate at least 10 correct responses to a disaster 95 % (stu) 94 % (staff) 19 % 13 % 0 % 53 % 98 % 98 % 63 % 24 % 13 % 90 % 3 % 4 % 232 % 85 % … Yes Yes NA No No NA Objective #3. Increased technical capabilities of four Central American National Societies to implement community based education programs for disaster preparedness and response. 1. Community volunteers performing on average xx home visits per volunteer per month - Average number of home visits per volunteer,LOP - % coverage of Population with Home Visits 2. Red Cross Volunteers implementing at least xx DP workshops in each Community per quarter Total # of DPP workshops conducted, LOP 3. Client Satisfaction of Beneficiaries 0 0 0 0 0 NA NA 1.4 34 % NA 481 NA10 NA NA NA NA NA NA 10 Data on this indicator is reported qualitatively in the Evaluation Results Section. The quantitative indicator could not be calculated due to reliability errors with the data. See Evaluation Results Section for more information. 44 Table C. Targeted versus Reached Beneficiaries The CAMI Project exceeded beneficiary targets with establishment of school emergency committees and school brigades. Over 26,000 teachers and students benefited from CAMI interventions in their schools. Due to extensive efforts of the project in training volunteers, more than 7,600 home visits were made to implement disaster preparedness education interventions covering 42,080 individuals. Some of these individuals also benefited from participation in Disaster Preparedness Workshops and/or participation in a community or school brigade. While all Red Cross Chapters ended the project with either a part-time or full-time emergency committee, targets were not achieved as originally envisioned with community emergency committee’s where 2 of the original 12 became operation 24/7 and 54 of the original targeted communities established part-time local emergency committees. The target number of Chapters also was adjusted from 12 to 11 when it was realized that one of the target Chapters in Honduras (Choluteca) functionally ended up utilizing the Tegucigalpa National Headquarters' Table A. Summary of Targeted and Reached Beneficiaries Targeted Reached Red Cross National Headquarters Red Cross Emergency Committees Established (any) Red Cross Chapter 24/7 Operational Emergency Committees Established 4 12 12 4 11 2 Community Emergency Committees Established Beneficiaries covered by an Emergency Committee and a Disaster Plan Number of Community Brigades Schools Students in Target Schools11 Teachers12 Number of School Brigades Students Trained as Brigade members or other specialized training Number of Disaster Preparedness Workshops conducted Number of Beneficiaries participating in Disaster Preparedness workshops 60 123,175 225 65 NA NA 270 NA NA NA 54 115,857 206 87 24,563 1,952 502 3,311 481 4.641 Number of Volunteers Trained Number of Educational Home Visits performed by Volunteers Individuals reached through Household Visits 705 NA NA 1,057 7,651 42,080 11 Benefited from total CUSE intervention including implementation mitigation projects. 12 Ibid 45 Objective #1 Improve the timeliness and appropriateness of mitigation responses of Red Cross national societies and communities to disaster through developing sustainable protocols and training personnel in operation centers and emergency committees (APS, Objective 1, Activities 2 and 3). G. Findings Concerning Objective 1 After the baseline was conducted, the organization of community- level emergency committees was reorganized. There were 96 Neighborhoods included in the 60 target Communities. For the baseline, a few Neighborhoods were clustered by administrative oversight divisions into 85 Neighborhoods where the project staff thought emergency committees would be formed. During the project implementation, in Honduras and Nicaragua, 25 Neighborhoods (12 in Honduras and 13 in Nicaragua) underwent mergers with other Neighborhoods participating in the project as the actual emergency committees formed. This left a total of 60 target Communities for the project – which is what was originally written in the proposal. Therefore, the denominator used for the baseline is ‘85’ and ’60’ for the final evaluation. Of the 60 target Communities, 56 established emergency committees were established. El Salvador did NOT assess four of their Communities who did not establish committees. All other countries assessed all original Communities whether a committee was established or not. Therefore, in the four El Salvador Communities, a response of ‘0’ was used for these Communities to calculate final evaluation results. Table D. Communities y Neighborhoods included in Baseline/Final Evaluation. Sep/Oct 2001 (N) Mar/Apr 2003 (N) Guatemala Communities Neighborhoods 15 15 15 15 Honduras Communities Neighborhoods 15 27 15 33 Nicaragua Communities Neighborhoods 15 28 15 24 El Salvador Communities Neighborhoods 15 15 11 11 Total Communities Surveyed Total Neighborhoods Covered Communities Neighborhoods 60 85 56/60 H. Red Cross Chapter Response to Disasters: Pre-CAMI Intervention. One hundred percent of Red Cross Chapters reported responding to a disaster in the past six months before the start of the project. El Salvador Chapters responded in 2001, (1 missing date); in Nicaragua, two Chapters responded in 1998, and one Chapter responded in 2000, Honduras responded in 1998 and 2000, and Guatemala responded in 1976 and 1998 for Hurricane Mitch. 46 I Proportion of Chapters with Emergency Committees. In 2001, zero (0) of the 12 participating chapters had emergency committees. By the project’s end in 2003, 100 percent of 11 chapters had some form of an emergency committee that all had member directories. However, only two of these chapters were established as fully operational emergency committees that functioned 24 hours a day, 7 days a week. Proportion of Filials with Emergency Plans 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Guatemala Honduras Nicaragua El Salvador Total Countries Proportion reporting/observed to have Plans 2001 (Reported Having Plan) 2001 (Observed to Have Plan) 2003 (Reported Having Plan) 2003 Observed to Have Plan Proportion of Chapters with Emergency Plans Of the 12 Chapters assessed in 2001, 83 % reported having an emergency plan (n=4), with only 17 % of Chapters producing plans that could be observed (n=2). In 2003, of the 11 Chapters evaluated, 100 % reported having an emergency plan in 2003 (n=11), with 100 % producing plans for observation during the final evaluation (n=11). Disaster Preparedness Practices of Chapters As a whole, there was improvement in preparedness practices of participating Chapters (see Appendix xx for details). The only targeted preparedness practice achieved by 100 % of Chapter EOCs was holding meetings on roles and responsibilities (increased from 25 % in 2001). Conducting vulnerability assessments was only achieved in 36 % of Chapters (increased from 8 % in 2001). There was a great amount of country variability in which preparedness practices were achieved. Proportion of Filials With Disaster Preparedness Practices at End of Project By Country 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 2003 Drill 2003 R&R Mtg 2003 COE Procedures 2003 Contingency Plan 2003 Sit Rep 2003 Risk Map 2003 Vul. Assess Disaster Preparedness Practices % of Filials with Practice, by Country Guatemala Honduras Nicaragua El Salvador Total 47 J. Proportion of Communities with Emergency Committees. In 2001, 26 percent of Communities reported having some form of an emergency committee (22/85). By the project’s end, this had increased to 90 percent (54/60) with most being able to produce a directory of their members to CAMI evaluators (91 %, n=49). Proportion of Chapters with Emergency Plans In 2001, 18 % of Communities (15/85) reported having Emergency Plans, with 10 of those committees having plans that could be observed by CAMI evaluators. El Salvador was the only country where reported plans were not observed. By the project’s end in 2003, 90 % of Communities had some form of an emergency plan (54/60), with 48 committees producing copies for observation during the evaluation. Proportion of Communidades with Emergency Committees, by Country 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Guatemala Honduras Nicaragua El Salvador Total Countries Proportion of Communidades with COEs, by Country 2001 (Reported Having Plan) 2001 (Observed to Have Plan) 2003 (Reported Having Plan) 2003 Observed to Have Plan Proportion of Communidades with Disaster Preparedness Practices at End of Project, By Country 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 2003 Drill 2003 R&R Mtg 2003 Other Dis. Prep Activities 2003 Sit Rep 2003 Risk Map 2003 Vul. Assess Country Proportion of Communidades, by Country Guatemala Honduras Nicaragua El Salvador Total Disaster Preparedness Practices of Communities As with Chapters, there was general across-the-board improvement in terms of preparedness practices. Biggest gains were achieved in conducting disaster drills (an increase from 1 % to 62 %) and possessing risk maps (up from 4 % to 70 %). The exception was the development of updated situation reports for the Community Emergency Committee. At the end of the project, only three Community had these, up from two at the time of the baseline study. 48 K. Proportion of Neighborhoods/Communities with Brigades. Brigades are defined as a group of trained Neighborhood/Community volunteers who may also be Red Cross volunteers who live in the area. The Brigades are activated only when the alert is activated in a Neighborhood or Community. There may be one or more Brigades per Neighborhood with functions similar to: drills, search and rescue, first aid, shelter, food preparation, etc. During drills and actual emergencies the Brigades have assigned responsibilities and functions within the Neighborhoods’ overall emergency plan. See Annex A. for detailed definitions of ‘Brigades’. Table E. Proportion of Neighborhoods/Communities with Brigades SEP/OCT 2001 (NUMBER) SEP/OCT 2001 (TYPE) MAR/APR 2003 (NUMBER)13 MAR/APR 2003 (TYPE) Guatemala 0 % 0/15 (0 directories observed) NA 100 % 15/15 (7 directories observed) 12 - Manejo de Albergues 12 - EDAN 8 - Búsqueda y Rescate 13 - Prevención y Control de Incendios 15 - Primeros Auxilios 13 - Evacuación 4 -- Salud Mental Honduras 7 % 2/27 (0 directories observed) 1-- Manejo de Albergues 100 % 15/15 (10 directories observed) 8 -- Comunicación 14 - Logística 9 - Manejo de Albergues 7 - EDAN 12 - Búsqueda y Rescate 1 - Prevención y Control de Incendios 9 - Primeros Auxilios 10 - Evacuación 14 - Seguridad 11 - Salud Mental Nicaragua 11 % 3/28 (3 directories observed) 1-- Logística 1 -- Manejo de Albergues 3 -- Búsqueda y Rescate 2 -- Prevención y Control de Incendios 1 -- Primeros Auxilios 2—Evacuación 1—salud mental 1-- other 87 % 13/15 (8 directories observed) 7-- Manejo de Albergues 7-- EDAN 1--Búsqueda y Rescate 2-Prevención y Control de Incendios 11-Primeros Auxilios 7--Evacuación 3--Seguridad 3--Salud Mental 4—Other El Salvador 0 % 0/15 (1 directory observed) 1-- Evacuación 73 % 11/15 (7 directories observed) 1-Comunicación 4-Logística 0-Manejo de Albergues 5-EDAN 8-Búsqueda y Rescate 5-Prevención y Control de Incendios 11-Primeros Auxilios 10-Evacuación 5—Seguridad 2--Salud Mental 1--other Total 6 % (5/85) 90 % 54/60 13 Question 16: ‘There is at least one brigade observed’. Baseline was 4 which matched number of observed directories. Final, response was 48 (10-Guatemala; 15-Honduras; 12-Nicaragua; 11-El Salvador). 49 L. Communications and Performance on Drills. In 2001, 92 percent of chapters reported having capabilities of communicating with their national society headquarters and the Communities within their jurisdiction. The most common forms of communication were telephones (landlines), cellular phones, radios, fax machines, or physically traveling to the destination. Communities require more creativity when considering how to communicate with their chapters. Guatemala and El Salvador had the biggest gains in capability, with Guatemala increasing communication capacity in their mostly rural Communities from 7 percent to 80 percent at the end of the project and El Salvador increased capacity from 13 percent to 73 percent. Both Honduras and Nicaragua achieved communication capabilities in more than 90 percent of their Communities, however, the baseline in Nicaragua was 86 percent from the beginning of the project due to the more urban nature of their Communities. As evidenced in Table E, communication tools were upgraded and standardized during the life of the project. Table F. Proportion of Communities with Communication Capability with Red Cross Chapter SEP/OCT 2001 (NUMBER) SEP/OCT 2001 COMM CAP. MAR/APR 2003 (NUMBER) MAR/APR 2003 COMM CAPABILITIES Guatemala 7 % (1/15) 80 % (12/15) 6--Telefono (no cellular) 10--Cellular 0--Radio 1--Fax 5--Fisicamente ir hasta alli 0--Otro Honduras 55 % (15/27) 93 % (14/15) 10--Telefono (no cellular) 12--Cellular 2--Radio 0--Fax 14--Fisicamente ir hasta alli 0--Otro Nicaragua 86 % (24/28) 93 % (14/15) 7--Telefono (no cellular) 10--Cellular 0--Radio 0--Fax 5--Fisicamente ir hasta alli 1--Otro El Salvador 13 % (2/15) 73 % (11/15) 8--Telefono (no cellular) 6--Cellular 0--Radio 0--Fax 10--Fisicamente ir hasta alli 0--Otro Total 49 % (42/85) Of Neighborhoods reporting having communication capabilities, all but one had access to a phone of some type. Other means of secondary communication (to phones) included bicycles, radios, buses, health centers, horse, on foot, , taxis, telegrams, collective transport, and community vehicles. 85 % (51/60) This capacity to communicate was then tested in communications drills with pre-scripted messages. The drills also tested quality indicators such as whether the source and content of messages was verified. Findings were as follows: 50 Proportion of Filials Observed to Pass Disaster Messages within 30 Minutes, Baseline and Final 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Guatemala Honduras Nicaragua El Salvador Total County Proportion of Filials, by Country 2001 Passed (National) 2003 Passed (National) Messaging Completed within Limits Chapter – National – Chapter At baseline, messages were successfully passed within time limits Chapter-National￾Chapter 50 % of the time, increasing to 82 % by projects end. Honduras made the most improvement (0 % - 100 %), whereas Nicaragua’s capacity went down from 100 % at baseline to 67 % at the final evaluation. Verification of Messages: At baseline, the source and content of messages between National to Chapter were verified in 50 % of message attempts (6 out of 12); and 17 % of message attempts Chapter to National (2 out of 12 times). At final, the source and content of message between National to Chapter were verified in 64 % of message attempts (7 out of 11 cases); and in 36 % of message attempts (4 out of 11 times) Chapter to National levels. Messaging Completed Within Limits Community – Chapter – Community At baseline, messages were successfully passed within time limits Chapter￾National-Chapter 17 % of the time, increasing to 71 % by projects end. Guatemala made the most improvement (0 % - 80 %). Honduras also achieved 80 % success at the final evaluation (increased from 7 % at baseline). Verification of Messages: At baseline, the source and content of message between Community and Chapter were verified in 7% of message attempts (6 out of 85 attempts); and also 12 % of message attempts (10 out of 85attempts) Chapter to Community. At final, the source and content of message between Community and Chapter were verified in 32 % of message attempts (18 out of 56 attempts); and also 32 % of message attempts (18 out of 56 attempts) Chapter to Community. Proportion of Communidades Observed to Pass Disaster Messages wtihin 60 Minutes, Baseline and Final 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Guatemala Honduras Nicaragua El Salvador Total Country Proportion of Communidades, by Country 2001 Passed (Communidades) 2003 Passed (Communidades) 51 Objective #2 Increase knowledge, self-efficacy, and skills in disaster preparedness and response of individuals and households through incorporating risk management concepts in formal and informal education methods (APS, Objective 3, Activities 2 and 3) M. Target versus Actual Schools. Work was implemented in a total of 87 schools, a 33 percent increase from the target of 65. All countries implemented projects in 15 schools as originally targeted. However, in Guatemala the CAMI team targeted 20 schools – and achieved implementation in 39. Of these 39, the first original 20 were considered the primary schools for full CAMI intervention. Table G. Proportion of Schools Evaluated by Country Sep/Oct 2001 (Number) Mar/Apr 2003 (Number) Guatemala 20 24 a) 4 extra schools deleted from final analysis b) 1 unmatched school deleted from baseline Honduras 15 15 No Change Nicaragua 15 15 No change El Salvador 15 16 1 extra school deleted from final analysis Total 65 70 64 Matched Pairs Analyzed in Final Evaluation Note: there was a school strike in Guatemala up until March 2003. Schools that were not ‘primary’ CAMI Were not targeted for evaluation due to limited time. 1. Socio-demographic information of participating schools. Of the schools included in the final evaluation, 72 percent were in rural areas, and the remaining 28 percent were in urban areas. Guatemala worked in rural schools exclusively whereas Nicaragua and El Salvador had a more urban focus (10/15) and (9/15) respectively. Honduras was almost evenly split (8/15 were rural). Urban schools were defined as schools at the municipal level. Rural schools were below the municipal level (e.g., Neighborhoods or Aldea level). In 2001, 42 percent of schools reporting having at least one student with a disability (n=27/64). In 2003, 38 percent of schools (n=24/64) reported the same finding represented by a decrease of three schools in El Salvador reporting students with disabilities. It is unknown why three schools in El Salvador experienced this change. 52 Proportion of Schools with Emergency Committees 3/19 Obs 0/15 Obs 2/15 Obs 2/15 Obs 7/64 Obs 14/15 Obs 56/64 Obs 18/19 Obs 12/15 Obs 12/15 Obs 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Guatemala Honduras Nicaragua El Salvador Total CAMI Countries Proportion of Schools 2001 2003 Schools with Emergency Committees Of the 64 schools assessed, 39 % reported having emergency committees, with 7 of those schools having committee directories available for observation in 2001. 93 % reported having emergency committees in 2003, with 56 committee directories being available for observation. El Salvador had the highest baseline values with 87 % of schools starting with committees, while Honduras had the lowest number of schools with committees at baseline; 13 %. Schools with Emergency Plans By project’s end, 97 % (62/64) of CAMI schools had some kind of emergency plan that could be observed during the final evaluation, whereas only 23 % (15/64)began the project with a plan. Again, El Salvador had the highest baseline values (87 % 13/15)) while in Guatemala and Honduras, none (0) of the schools had plans at the start-up of the project. Proportion of Schools With Observed Emergency Plans 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Guatemala Honduras Nicaragua El Salvador Total CAMI Countries Proportion of Schools 2001 2003 Proportion of Schools with Brigades 4/64 Obs 1/15 Obs 1/19 Obs 0/15 Obs 2/15 Obs 56/64 Obs 14/15 Obs 12/15 Obs 18/19 Obs 12/15 Obs 0% 20% 40% 60% 80% 100% 120% Guatemala Honduras Nicaragua El Salvador Total CAMI Countries Proportion 2001 2003 Schools with Brigades At baseline, 39 % of schools had at least one Brigade, 4 of whom were observed to have written directories of those brigades. In 2003, 98 % of schools reported having at least one brigade, 56 of whom were observed to have written brigades. For a detailed breakdown of the types of brigades formed in each of the CAMI countries, please see table xx. 53 2. Proportion of Schools with Observed Disaster Preparedness Practices. The evaluator assessed the achievement of disaster preparedness practices according to 19 key safe conditions.14 The safe conditions were broken down into administrative preparation of the school (e.g., a written emergency plan could be observed at the time of the evaluation; evacuation routes and risk maps were observed, etc.) and physical dangers of school buildings (e.g., ensuring exits from classrooms were not blocked). The original target project had been to have schools achieve five or more safe conditions over the life of the project. At baseline, 89 percent of schools achieved or surpassed that target (98 percent at the final evaluation). Therefore, baseline and final evaluation data were re-analyzed against a new target of 10 demonstrating that at baseline 13 percent of schools had 10 or more safe conditions and at final evaluation that proportion had increased to 89 percent. Of particular interest was whether attendance was being taken every day (one of the 19 safe conditions. Findings indicate that although attendance was routinely being taken, the number of students present at any given time was not. Presence of that safe condition increased from 19 percent to 63 percent over the life of the project. See Table G. Table H. Achievement of Attendance in Target Schools. Baseline Final Evaluation % of schools where student, teacher, staff attendance is being taken every day 95 % (student) 94 % (staff) 98 % 98 % % of schools where number of students present is posted in classroom 19 % 63 % 3. Schools with Gas and Electric Hazards. For schools with gas tanks on school grounds, there was no change in the proportion who had the gas separated from the flame (71 percent did NOT have gas separated from flames in the kitchen). However, for schools with electricity, there was change in the proportion who had a maintenance directory that could be observed (2 percent at baseline and 22 percent at the final evaluation (n=51) at baseline and final). Increases occurred in ALL countries. 4. Proportion of Schools with Observed Disaster Response Practices. The evaluator assessed the achievement of disaster response practices according to 20 key responses15. Whereas the safe condition targets were set too low, initially targets for disaster response were set too high (schools would achieve all 20 key responses. These responses were broken down into administrative preparation of the school (e.g., a written emergency plan could be observed at the time of the evaluation, evacuation routes and risk maps were observed, etc.) and physical dangers of school buildings (e.g., ensuring exits from classrooms were not blocked). As such, baseline and final evaluation data was re￾analyzed to explore changes in 10 key responses AND 20 key responses. 14 At baseline, the number of safe conditions was 21. Three conditions were eliminated from the final analysis due to reliability errors in data collection (School questionnaire, #21, 24 and 29). 15 At baseline, the number of safe conditions was 21. Three conditions were eliminated from the final analysis due to reliability errors in data collection (School questionnaire, #21, 24 and 29). 54 Findings indicate that 13 percent of CAMI schools achieved all 20 response practices (increased from 0 percent at baseline) – whereas 90 percent of schools achieved 10 or more correct responses to a disaster scenario (increased from 53 percent at baseline). The country with the greatest gain was Guatemala (26 percent to 95 percent with 10 or more correct responses) and the country with the highest baseline and the least gain was Honduras (80 percent to 93 percent). Proportion of Schools with Correct Disaster Response Practices Baseline and Final Evaluation 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Guatemala Honduras Nicaragua El Salvador Total Country Proportion of Schools 2001: 20 Correct Disaster Responses 2003: 20 Correct Disaster Responses 2001: 10 or more Correct Disaster Responses 2003: 10 or more Correct Disaster Responses 55 Objective #2 Increase knowledge, self-efficacy and skills in disaster preparedness and response of individuals and households through incorporating risk management concepts in formal and informal education methods (APS, Objective 3, Activities 2 and 3). N. Households Disaster Preparedness 1. Exposure to Disaster Preparedness Messages The CAMI Project was interested in tracking a key indicator-- prior exposure to disaster preparedness messages before CAMI interventions--and to continue tracking this indicator as a measure of the coverage of CAMI interventions. Results are presented in Table xx. Findings indicate that before CAMI interventions, there was a high degree of exposure of the population to disaster preparedness messages via radio or TV (68 percent). In the baseline report, it was speculated that the recent El Salvador Earthquakes were a major reason for this coverage by non￾CAMI interventions. The ARC/CAMI project did not implement any radio or TV messages. This coverage dropped in the final evaluation. However, there were significant increases in coverage via home visiting (30 percent), and participation in a community event (16 percent) or simulation (18 percent). Although overall coverage of the population with these types of outreach remained low, these coverage rates (as reported directly by respondents) are consistent with ARC project activity monitoring reports. Exposure of Household Members to Disaster Preparedness Messages (reported by Head of Household) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Rec'd Home Visit Rec'd Pamphlet or brochures Heard Radio or TV message Saw Billboards or Signs Participated in Comm. Prep Participated in Disaster Drill Organized Comm. Prep Type of Education Exposure Proportion of Target Population Exposed 2001 2003 56 2. Disaster Preparedness Knowledge (of Head of Household): Knowledge of disaster preparedness was assessed based on key messages the CAMI team felt they would give to communities at the start of project interventions. These included first aid knowledge, most common causes of illnesses after a disaster, safe evacuation, care for underage children16 and the contents of their community disaster plans. Results of knowledge acquisition were as follows: Acquisition of Disaster Preparedness Knowledge, Baseline to Final (**) Indicates Significant Change 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Another Adult Cares for Children First Aid (Where to Apply Pressure)** Most Common Cause of Illness Evacuate with Bleach Evacuate with Family Papers** Evacuation Orders come from COE Correctly State 3 things in emergency plan** Knowledge Acquired 2001 2003 3. Disaster Preparedness Practices and Safe Conditions of Households. Just as with schools, the CAMI project aimed to increase safe conditions within households in the event of an emergency. Disaster preparedness practices were defined as actions families could take to prepare themselves for a disaster that did not include any physical changes to be made to their households such as having a family disaster plan. Safe conditions in households were defined as actions households could take that involved some non-structural changes to be made to their household environment such as securing dangerous objects on shelves. The largest gains is in safe preparedness practices by beneficiary households (i.e., significance at p<.05) The greatest gains (i.e., significant change) were achieved in windows being able to be opened from inside the house (19 percent increase) dangerous objects of 5 kilograms or more on top of furniture being secured (a 26 percent increase), and reduction of holes being visible in roofs from inside the house (20 percent decrease). No gains were achieved in exits and pathways being cleared within the house or in the proportion of households who had family emergency plans. 16 While all countries felt it was important to train caretakers not to leave young children alone unattended, the four country teams had a difficult time agreeing on what exact age was ‘too young’. A compromise was reached at age 16, an age that many teams remained dissatisfied with up to the end of the project 57 In total, there were 15 practices and safe conditions assessed in target households. As the project staff had little experience with setting these types of targets before project implementation, baseline and final evaluation data were re-analyzed according to multiple thresholds to see where the greatest gains occurred. The greatest gains (i.e., significant change) were achieved in households having 7 to 10 of the selected practices and safe conditions (gains of 58 to 72 percent and 17 to 29 percent respectively). See lessons learned in monitoring and evaluation for more detailed information about this section. Proportion of Households with Safe Practices and Safe Conditions Baseline and Final Evaluation 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Less than 5 5 or More 7 or More 10 or More All 15 Number of Practices and Conditions Proportion of Households 2001 2003 4. Households with Gas Hazards. As with schools, there was no significant change in behavior in households with a gas stove that keep a combustible liquid inside the house (approximately half of respondents with a gas stove keep a combustible inside their house17). 17 N=334 at baseline and 329 at final evaluation. 58 5. Indicators of Household’s Skill and Practice in Implementing a Disaster Drill. As with schools, the CAMI project staff was interested in assessing response practices in a simulated disaster. Although the proportion of households who completely implemented a disaster simulation, increased, coverage remained slightly below 25 percent of households. 0% 20% 40% 60% 80% 100% Proportion of Households who Implemented Household a Simulated Disaster Drill Baseline and Final Evaluation Implemented Household Disaster Drill 13% 24% 2001 2003 Of the approximate 25 percent of households who implemented the drill, there were significant changes in responses in the following areas: • Someone verbally prepared the household for the evacuation; • Someone counted members of he family before leaving the household; • Someone verified that there was no one left in the house after the evacuation. Subsequently, there was a significant decrease in the proportion of households where the only activity during the drill was ‘running outside the house’. There was, however, no significant change in the proportion of households who exited the house with their family papers. See Appendix xx for detailed data tables and lessons learned in monitoring and evaluating disaster simulation drills at the household level. Of those households who did NOT complete the drill scenario, the following qualitative reactions were recorded: Table I. Qualitative Reactions of Respondents Who Did not Implement Drill Scenario. Sep/Oct 2001 (n=715) Mar/Apr 2003 (n=630) % Change * =Significant Change Partly implemented after 1 prompt 3 % (CI: 1.35, 4.65) 2 % (CI: -0.89, 4.89) 33 % Partly completed after 2 prompts 2 % (CI: 0.65, 3.35) 1 % (CI: -1.89, 3.89) 50 % Did not implement drill at all; looked confused or worried 36 % (CI: 31.37, 40.63) 30 % (CI: 27.10, 32.89) 17 % Did not implement drill at all; respondents laughing, thought exercise was ‘crazy’ 25 % (CI: 20.82, 29.18) 11 % (CI: 8.11, 13.89) * 56 % Other 6 % (CI: 3.70, 8.29) 7 % (CI: 4.11, 9.89) 17 % No Reason 11 % (CI: 7.98, 19.02) 0 % 59 O. Indicators of Disaster Beliefs/Myths and Disaster Self-Efficacy Two key indicators that the project developed and implemented were designed around disaster myths (e.g., “It is necessary for foreign countries to provide medicines, foods and other things during a disaster”) and disaster self￾efficacy (e.g., “I am able to help victims during a disaster”). Baseline findings indicated that the belief in myths was not as great as anticipated; possibly due to the relatively high exposure to previous disaster education messages; especially via radio. Interventions and measures for ‘disaster self-efficacy’ and changing beliefs such as ‘that households conducting drills was dangerous’ were controversial and not uniformly implemented. As such, the project did not implement interventions designed around decreasing myths and/or defining and increasing disaster self-efficacy. As such, unsurprisingly, there were no significant changes in these indicators from the baseline study to the final evaluation. The rate of ‘Disaster Self-Efficacy’ as defined and measured for the CAMI baseline remained at 6 percent and there were no significant changes in opinion about disaster myths or beliefs. From baseline study to the final evaluation, approximately one-third of heads of households believed that conducting disaster drills in schools was dangerous. Approximately 90 percent believed that it was necessary for foreign countries to provide medicines, foods, or other items during a disaster and three-fourths continued to believe that it was necessary to keep survivors in shelters as long as possible. More complex and culturally difficult myths to assess and measure also did not change. Approximately one-half of respondents believed that living or dying was chance, slightly less than one-third believed that without decent burials, spirits would roam freely and cause harm and about three-fourth’s believed that only God decides who lives or dies. Objective #4: Increase the technical capacity of National Red Cross Societies to implement community-based disaster preparedness and response projects. Neighborhood volunteers performing on average xx home visits per volunteer per month Sep/Oct 2001 Mar/Apr 2003 Avg. Home Visits Per volunteer/per month Mar/Apr 2003 Avg. Home Visits Per volunteer Guatemala NA … 2.4 Honduras NA ... 0.45 Nicaragua NA ... 0.85 El Salvador NA ... 1.75 Total NA ... 1.4 NA=Not Applicable 60 P. Number of Community Workshops Implemented (Life of Project). The CAMI project implemented a total of 481 educational community workshops about disaster preparedness and planning that served approximately 4,641 participants. These people may also have received a visit from a CAMI representative in their home. As demonstrated by the average class size, workshops were small with workshops averaging 10 participants per session. # of Workshops # of Participants Average Class Size of Workshops Guatemala 247 1,053 4 Honduras 80 1,215 7 Nicaragua 67 1,056 6 El Salvador 87 1,317 7 Total 481 4,641 10 Coverage of the Target Population with Household Visits (according to CAMI records): Guatemala Honduras Nicaragua El Salvador Total Coverage 0% 20% 40% 60% 80% 100% Country Coverage of CAMI Population with Household Behavior Change Visits (as of March 2003) There were a total of 7,651 household visits made by volunteers in the CAMI Project, achieving a coverage rate of 34 % of the estimated 22,395 households in the project area. This is equal to approximately “covering” 42,080 people in the target area of 123, 175. Coverage ranged from 9 % in Guatemala to 88 % in El Salvador. Q. Meeting Beneficiary Expectations and Client Satisfaction. Implementation of measures to assess indicators of client satisfaction (Indicator #3, Objective #3) were implemented inconsistently across countries. As such, only general qualitative impressions can be made and the client satisfaction indicator will not be reported on. In general terms, participants felt that the strong points of the program were the training, the simulations, and the quality of project personnel. Mentioned constraints were too little time in the project (especially at the community level), the material lack of support (e.g., lack of distribution of materials and supplies), and not all the schools of participating municipalities were covered. 61 Cost Analysis 62 V Cost Analysis: Cost analysis allows one to measure costs and calculate the average or incremental cost, Implementing a full cost￾effectiveness evaluation was not built into the original evaluation design . As such, the regional finance delegate, as part of the final evaluation, implemented a forensic accounting exercise whereby cost-per-beneficiary and cost￾per-community was estimated as follows. A. CAMI Cost-per-Beneficiary Analysis 1. All cost information is based on payments made in Central America and recorded in the Quicken System at five offices. 2. The worksheet titled Summary by Month & Quarter, begins with USG costs for DPP objective 1 and objective 2 regarding education (school) programs for each country. Added to these costs are any cost￾share amounts for that country, and an allocation to each objective for each country of 12.5 percent of all regional costs charged to the grant or to cost share. 3. The CAMI cost-per-beneficiary analysis was done retrospectively. As such, the ‘forensic accounting techniques used may underestimate costs of the project as not all NHQ costs have been, or will be able to be, calculated into the spreadsheet (e.g. proportioning out the cost of NHQ staff time). 4. The CAMI project is able to estimate fairly accurately the number of beneficiaries covered by an emergency plan and an emergency committee. However, estimating the number of beneficiaries actually in direct contact with a staff or volunteer of the project proved more difficult. As such, a conservative estimate was used-- the proportion of the target population directly reached through the CAMI project based on coverage achieved in each country with household visits and adding in an additional 10 percent other coverage for contact through other means. B. Findings 4) The average cost per person to be covered by an emergency committee and an emergency plan was $5.71 per person, or $12, 250 per community. 5) The average cost per student or teacher to be covered by a school emergency committee and a school preparedness plan was $21.48, with an increased cost of $5.15 to complete a small mitigation project in each school. The average cost per student or teacher covered by both a plan, emergency committee and to have a small mitigation project implemented was $26.63. 6) The key driving factor of proportionate cost by country was the number of beneficiaries served versus number of communities or schools served. a. For example, under Objective #1 El Salvador covered 11 communities, the lowest of all countries. Their subsequent cost to cover a community with a plan and an emergency committee was therefore also the highest at $14,799 per community. However, El Salvador also double the number of beneficiaries of the 4 countries at the community level (n=41,373). At $3.93 the cost-per￾beneficiary was significantly lower than the other countries which ranged from $5.33 to $7.43. b. This same pattern held with the school interventions, Guatemala covered 39 schools, approximately twice as many as the other countries. Their cost per school was the lowest at $3,776 whereas all other schools cost between $8,700 and $8,900. However, Guatemala also covered small, rural schools and therefore had the second lowest number of teachers and students covered (5,428). The cost-per-beneficiary of the school intervention was therefore the second highest at $27.13 whereas in Nicaragua where the project covered urban schools, the cost-per beneficiary was $15.17 with a coverage of 8,633 teachers and students. 63 7) The CAMI Project utilized a training of trainer methodology that tended to be expensive in the initial phases. However, a somewhat longer project time frame would facilitate cost analysis for projects using this sort of methodology. For example, volunteers from San Vicente Chapter in El Salvador had begun replicating community work in 5 additional communities shortly before the project ended, however, these populations were not counted as beneficiaries of the CAMI project, nor were any costs incurred to the project. Had the project been longer, a formal replication phase by non-CAMI staff could have been included and provided a more reliable cost analysis base (and indeed a more reliable indicator of long-term impact). Nonetheless, cost per beneficiary, when allied with evaluation results, provides a useful unit of comparison for similar DPP programs. 64 Table J. Cost-per-Beneficiary Analysis Description El Salvador Guatemala Honduras Nicaragua Total DPP Activities Expenses Total expense without mitigation projects $162,797.89 $188,736.19 $126,707.54 $183,275.42 $661,517.04 Beneficiaries of DPP Activities Total Number of Communities 11 15 15 13 54 Total Population living in CAMI Communities 41,373 25,402 23,779 25,303 115,857 Cost per Beneficiary (to be covered by a plan) $3.93 $7.43 $5.33 $7.24 $5.71 Cost per Community (covered by a plan) $14,799.81 $12,582.41 $8,447.17 $14,098.11 $12,250.32 Note: The CAMI Consolidated Indicators 'double counts' community members contacted. For example, a person receiving a household visit, may be a volunteer, who attended a DPP Workshop. A school teacher in a CAMI school may have attended a DPP Workshop be a Red Cross volunteer and/or have received a household visit. Therefore, the number of direct recipients (i.e.; had direct inter-facewith CAMI staff or volunteers could not be determined. Educational Activities Expenses Total expense without mitigation projects $148,355.24 $147,273.60 $142,846.28 $131,000.16 $569,475.28 Beneficiaries of Educational Activities # of teachers in target schools 435 332 579 606 1,952 # of students in target schools 7,023 5,096 4,417 8,027 24,563 Total Beneficiaries 7,458 5,428 4,996 8,633 26,515 Total Number of Schools 17 39 16 15 87 Cost per beneficiary to be in a school w/ plan $19.89 $27.13 $28.59 $15.17 $21.48 Cost per School (covered by a plan) $8,726.78 $3,776.25 $8,927.89 $8,733.34 $6,545.69 Mitigation Projects Total Costs $19,996.81 $20,557.74 $44,908.07 $51,040.98 $136,503.60 Total Beneficiaries (teachers and students in schools) 7,458 5,428 4,996 8,633 26,515 Cost per Beneficiary $2.68 $3.79 $8.99 $5.91 $5.15 Note: Due to retrospective, forensic accounting methods used to calculate cost-per-beneficiary, point estimates are likely not exactly correct. However, proportionate and relative costs have a higher accuracy when interpreting the data. 65 Programmatic Recommendations and Lessons Learned * * (Sources: CAMI Country Reports, Interviews with Regional and NHQ Staff, CAMI Evaluation Team) 66 VI Programmatic Recommendations and Lessons Learned A. Participatory Design and Project Start-Up 1. Findings a. Coverage and sustainability of change could have been improved had primary stakeholders been more involved in design process and re-involved in planning as soon as funding approved. Few people on the project had the full picture required of the project as a whole as it evolved from proposal to practical implementation. National Societies and CAMI staff felt strongly that this was in part attributable to the project not being developed and/or presented to stakeholders, all at the same time in the beginning, so that they could debate the issues. Wasn’t presented in a standard manner to all people in the same way, so not everybody had the same understanding of the project until July 2001. b. Impact and sustainability of the project was impacted when original selection criteria for targeting Chapters were not respected. National Societies and CAMI staff felt there was a lack of a systematic, participatory project design process with all partners; including HoPs in the proposed/awarded countries. c. Throughout the project there remained a lack of clarity – and consensus – between project staff and National Societies on what an “operational EOC” meant, and what was “important” in risk reduction plans for each school once it was clear original definitions were not always applicable (e.g., not every community needed a Red Cross Chapter to be a 24/7 responder as there was already another agency assigned that role). d. The overall consensus from all project staff, national societies and NHQ management was there were delays in getting the project started, which rushed the implementation phase. This was due in part to major response in El Salvador which coincided with the receipt of the grant which preoccupied NHQ. Implementation in El Salvador was set back several months due to that, as well as at NHQ. e. Strategic planning was taking place at the same time, which also caused delays. f. Regional programs are a challenge because certain elements have to be standardized, but other elements need to be flexible in each region or country. Not everyone will be happy at all times with the compromises reached during project planning. 2. Lessons Learned and Recommendations a. Involve stakeholders from the beginning, and keep them involved throughout the planning process. b. Set and respect selection criteria. c. Comprehensive training for staff in methodologies and most importantly, the rationale for the overall project design including strengths, limitations, and where differences of opinion had to be negotiated. Document the process. d. Have pre-established contingency plans for how to continue on-going projects in the event of an isolated disaster. However, if it is clear the disaster will significantly alter the pace and type of interventions that will be implemented, ensure paperwork such as ‘no-cost’ extensions, etc. are filed as soon as possible. e. ARC/NHQ needs to provide clear prioritization of tasks for the field; and respect donor f. Regional projects require a careful balance of standardization and customization at project start-up. Regional managers need to be clear from the beginning where and what the standardization issues are in order to give clear guidance to staff at the country level. Developing an environment of compromise and transparent negotiation is critical to a regional project. 67 B. Beneficiary Targeting 1. Findings: National Societies remained concerned until the end of the project that the original criteria for selection of Communities were not applied. Not all of the Chapters where the project took place were their first choice. a. Original targeting: 1) Disaster Vulnerability (experienced disasters on a frequent basis), 2) Not receiving direct service from Emergency Management Agency in their country, 3) Previous Relationship with the Red Cross, 4) “ Buy in” from the community. b. What Actually Happened in the Project: 1) Some sites ended up being politically chosen (e.g., The choice of Jalapa in Guatemala was not based on any of the original targeting criteria), 2) The chapter’s capacity was not considered in its totality (e.g., was there any type of volunteer base in existence to support the training efforts of the project). 3) Interest of chapters and communities was not properly assessed at the project’s beginning during the site selection. 4) In urban programming, the evaluation team felt it was more difficult to achieve results. 2. Lessons Learned and Recommendations. a. Disaster preparedness projects need to target not only on disaster vulnerability criteria, but also poverty indicators, and the national society’s capacity and interest of at the local level to respond. b. Chapters with NO capacity versus Chapters with SOME response capabilities may be two different projects . c. Rural/urban differences should be carefully considered when selecting sites. C. Education of Beneficiaries 1. Findings a. National Societies and CAMI staff all referenced initial skeptism on the part of communities, but they felt that skepticism was overcome in most places. However, this skeptism caused some delays in ‘uptake’ of project interventions by communities. b. Training/Materials supervisor was originally part of the project. Project started by developing and testing new materials; however, many of these materials already existed. The position was eliminated because a lot of work was going into reinventing the wheel. Each one of the NS has their own first aid courses, etc. so – this position probably wasn’t required. c. There were no uniform materials used in the project’s four countries. Country-specific materials were used and certain materials were heavily adopted from existing documents(e.g., UNICEF materials were used.) Each country’s national society and country coordinators need to validate new materials. d. The CAMI project focused on training disaster preparedness volunteers. However, home visits were burdensome for the staff and only 30 percent coverage of the target population was achieved during the life of the project. 2. Lessons Learned and Recommendations 68 a. Future projects should explore integrating DPP education into other “non-DPP” interventions e.g., integrating health-related disaster messages into routine IMCI health messaging to streamline community-based education efforts. b. Training materials should be assessed at the start of a project to minimize duplication of effort. D. Sustainability 1. Findings a. The plan for CAMI staff is that all CAMI staff will be hired into the National Societies either with Federation funding or from other PNSs to keep the DPP staff capacity that has been built within the movement. b. At the household level, it is likely that sustainability will depend on the strength of the EOCs and community emergency committees that have been developed. c. Although the training in each country was well received by the Red Cross, communities, and schools, the need to provide transportation, food and, in some cases, per diem for outside instructors (firemen) was continuously necessary. Thus, the question is raised about what happens to sustainability when there are no funds for transportation, food, or per diem instructors in the future d. CAMI partners, including staff and National Societies, agree that the most sustainable intervention across all countries appears to be the CUSE training in the schools. MOEs are discussing adding CUSE as part of the curriculums. As such, the Project was reinforcing the Governments’ plans to have Brigades in Schools. The CUSE captured much attention among teachers, school, local, and municipal officials. Thus, this project segment blossomed beyond project expectations and the need to program more activities in this area grew while activities associated with EOCs, Chapter involvement, and volunteers lessened. e. The EOCs and community emergency committees have variable likelihood of being sustained and replicated. Communities brigades were set up to be able to respond and improvise—using their local materials—in order to encourage sustainability of the disaster plans they had developed. 2. Lessons Learned and Recommendations a) Begin planning for sustainability of staff in the movement early in a project. A ‘step-down’ plan may be helpful; where positions are gradually shifted financially to the National Society (when appropriate). b) When developing community based training projects, determine from the beginning which capacities and costs will be ‘recurrent’ and c) Strong support by local/government counterparts increases sustainability d) Incremental project activities (e.g., adding CUSE to existing school activities or adding a COE to an existing Red Cross Chapter) are easier to sustain than completely new structures (e.g., training and maintenance of community brigades). E. Project Time Frame 1. Findings a) The three-month extension was critical. In the two-year project, the full two years was needed. b) In some respects, since most of the objectives were achieved, time was sufficient. It must be noted that all the staff consistently worked many hours of overtime. c) National Societies and CAMI staff and managers all cited time shortages as problematic. d) The CAMI Coordinator stated that all projects should have three phases: start-up, implementation, and an exit phase. It was felt that CAMI had a half start-up period due to the mismanagement of American Red Cross workload 69 (integrated planning workshop/EOAs, etc being done in June) plus the project was already starting late, and consequently rushed the implementation phase and the crash exit phase. e) The change in communities was very rushed in July due to the late start-up. f) To achieve objectives, CAMI staff began prioritizing certain interventions over others. Extra time went to schools and communities (DPP community meetings took priority over household visits). EOCs were given less time in some areas where no progress was being made. g) Country teams including national society staff and community members specifically referenced how rushed the mitigation projects were. 2. Lessons Learned a) Project staff need to take the time required to assess and design projects in a participatory manner. It is important to ensure that all implementers/partners have a full understanding of the project’s goals and objectives before implementation starts to avoid delays. b) Ensure that key stakeholders are consulted in the design phase, which is the time that changes can be made without delaying implementation c) New regional projects will likely take longer to build consensus and agreement due to geographical differences and distances. Plan to take additional time once a proposal is submitted to ensure that everyone involved has a full understanding of what was agreed upon during what may have been a rush period to get a proposal in by a deadline. F. Cost per Beneficiary 1. Findings a) The number of beneficiaries is a key component to achieving a low cost per beneficiary. b) Cost analysis was extremely difficult to implement in CAMI due to difficulties in assessing unique beneficiary numbers and difficulties in breaking down financial expenditures in a manner that matched the needs of cost analysis and cost effectiveness analysis. 2. Lessons Learned and Recommendations a) Program, M&E, and financial personnel need to create a unified budget and quarterly statistical and financial reports that can easily calculate a cost-per-beneficiary number. For example all cost information can contain an alphanumeric sort code for indicators. b) Statistical reports with beneficiary information could be matched to the cost information for a month or a quarter. c) Improve beneficiary counting tools for projects through quarterly reporting mechanism d) Improve project level financial management to integrate with quarterly reporting programmatic requirements, as well as financial and compliance requirements. Detailed Findings for Specific Interventions Findings Lessons Learned and Recommendations School Interventions Lack of active participation of MoE in design led to delays in implementation, reduced potential impact and sustainability. Involve MoE as a partner -- not as the beneficiary. Train MoE to be facilitators of process. There wasn’t time to train all the brigades in some countries. Start CUSE at the beginning of the school cycle through to end. 70 Findings Lessons Learned and Recommendations We covered more schools than proposed in countries where schools were relatively close together. There should be 10 to 15 schools per dedicated staff person, depending on logistics. Inter-institutional coordination worked well and the project awakened the interest of MoE Start coordination with partners from the assessment and design phase. Good results, especially in preparation for response, more variable with relation to conditions. A secure school depends on specific environment and needs. CUSE has two parts--mitigation and response plans--and we have been more successful at implementing the second “Safe” school indicators need rethinking--possibly to depend on the specific plan as defined by the school. Quality checklist for mitigation and emergency plans should be implemented, based on purpose of tools Risks exist within and outside the project’s reach. Policy on referral/information on situational risks outside project’s reach There is an on-going, sustainable role for Central American National Societies with the Ministry of Education and school-based preparedness. MoE should assume long-term training of teachers in CUSE with CR assistance in short-term . Long-term CR role would be training brigades and ensuring that instructors (at Mined) follow up on plans. EOC Chapter and Community-Level Emergency Committees Impact reduced when selection criteria not respected For example, the Guatemala delegations after two years are still very weak, and replication is fairly unlikely whereas in El Salvador the EOC concept is already being replicating in 5 other chapters. Involve stakeholders in establishing selection criteria through the filter of needs/interests/resources/capacity, specifically the highly vulnerable (risk plus poverty). Chapters should apply for project and be approved by their HQ. Define minimum profile (Chapter) for participation (chapter board approval, existence volunteers and ability to sustain, also logistical capacity) Some ONS’s left the project not fully understanding how the CAMI project was a way to achieve Strategy 2010 Regional and CAMI coordinators should ensure that all staff of projects understand Strategy 2010, and the applicability of ARC/NS projects to achievement of that strategy. Little agreement on what an operation EOC is, 24/7 communication? 24/7 response? Ability to implement plan? Define exactly what aiming at, based on the role of Chapter in conjunction with other local actors. From the start of the project, develop clear quality checklists for mitigation and emergency plans, based on a clear stated purpose of how the tools were to be used Not enough time to train all community level brigades The project needed the full 2 years implementation phase Capacity of EOCs and L-EOC to communicate improved significantly CR should disseminate its new capacities widely in communities EOC methodologies and preparation improved over time based on lessons learned during The CAMI methodology for preparing a Red Cross Branch for response works effectively and can be replicated 71 Findings Lessons Learned and Recommendations implementation. elsewhere. Participatory methodology for L-EOC worked (communication, plan, trained people) with proviso on project reach. Some L-EOC didn’t cover all the community. Explicitly deal with coverage as a task of plan. Use community volunteers who are responsible for a given area. Households Greater exposure to DPP activities but still short of goal More time needed if aiming at household change. More time is needed for HH change, at least 2 years implementation (3-year project). If less time is available, concentrate on L-EOC and brigade training at community level if starting at zero While there was only certain limited change at the household level -- that limited change reflected the actual interventions of the project At the community and school levels, drills worked well. Children responded to drills more attentively than the adults and this might, again, be attributed to the cultures and the lack of practice found in communities and households versus schools. Focus Prepared Families on reduced, most important aspects defined tentatively as: o Safe meeting place o Knowledge of what to do about children (where to meet) if in school at time of disaster o Knowledge evacuation route o First aid o Bring ID papers o Safe zones (house, community) o From whom do you get and give info Perhaps include use of audio visuals such as showing a disaster and how to respond may be helpful in some communities. Poor acceptance of individual drills at baseline and final. Suggestions: 1. Table tops as part of household education so drill at evaluation fits in 2. Group households get together to perform drills Typical ARC Domestic Disaster Preparedness activities at the household level were not always appropriate to the general cultures found in each country. For example, households didn´t have disaster household plans, kits, or stored items of food, water, etc… This didn´t detract from the fact that in most households there existed a mindset of Disaster Preparedness and what needed to be done during emergencies. New Projects need to implement participatory assessments to determine what preparedness activities are most appropriate in each international context. Only then, should the question be asked: Do the ARC materials meet these needs? ARC domestic disaster capacity does not always translate into international disaster capacity. In proposals, ARC domestic capacity should be supplemented with emerging data of ARC’s international successes in disaster preparedness and planning (e.g., Turkey, the Carribean and CAMI). 72 Lessons Learned and Recommendations in the Management of Regional DPP Projects * * (Sources: CAMI Country Reports, Interviews with Regional and NHQ Staff, Findings of CAMI Evaluation Team) 73 VII Lessons Learned and Recommendations in the Management of Regional DPP Projects A. Coordination of Partners FINDINGS LESSONS LEARNED RECOMMENDATION Each country has a national coordinating body. For example, CONRED is the National Coordinating body in Guatemala. Each National Society and ARC coordinated very well with the National Coordinating Bodies as the National Society were part of the Coordinating body in each country. As with CAMI, in a new DPP projects the National Society should be officially part of the National Coordinating body; this should be a first priority of the project (e.g., build it into the proposal). Management with National Societies was handled by Country Staff. Coordination and communication took on its own form in each country including MOE and the Chapters. The CAMI country staff started to coordinate more. Coordination, reporting, communication and buy-in continually needs to be reinforced throughout the life of a project. Not all partners were considered and consulted from the beginning e.g., A big problem was the lack of coordination with the MOE in the original planning phase.. For sustainability, formal agreements on the National Society roles and responsibilities with other actors in a country need to be established BEFORE project implementation starts. OFDA’s coordinating role was viewed by all staff (NHQ and field) to be ‘impressive’ (e.g., MACOE, EDAN). Staff particularly cited the OFDA/CTO (Tim Callaghan) for doing a good job of coordinating the 11 grantees (total) participating in the CAMI projects funded by OFDA. ARC/CAMI staff felt that the OFDA quarterly country meetings were valuable. The country and/or regional coordinator attended all meetings. This enabled different awardees under the OFDA regional grant to coordinate activities and work together. For example, one CAMI grantee in a country would take on MACOE at the national level; and another NGO would take on CUSE. Donors should replicate similar model in future projects/other countries. OFDA should consider evaluating the impact of the coordinated interventions by ALL CAMI grantees in a country Limited interaction of Regional Delegation with other units except for limited scope issues with DPP manager, TAPE. Regarding budget matters, for example, the Regional Project Coordinator spoke with the Regional Finance Delegate who spoke with WDC. The creation of ARC/NHQ and field planning teams should ensure coordination and integration of NHQ support to the field. The issue of country staff/delegates approaching support units directly (e.g., TAPE and/or Business Ops) remains an issue for greater clarity. 74 B. Marketing of Project Successes FINDINGS LESSONS LEARNED RECOMMENDATION CAMI was a central grant awarded by OFDA in Washington and monitored by the OFDA regional office for Latin America and the Carribean. While some countries, such as Honduras, had active involvement with USAID/Missions from the start of the project, others only started regular communications during the second half of the project. We need to do a better job of keeping the local USAID missions in the loop so they are fully aware of the nature and activities of the program. General Management of a Regional Project FINDINGS LESSONS LEARNED AND RECOMMENDATION Initial management structure was a mismatch between a regional project structure with the regional project management making decisions and hiring without coordinating or involving HoPs. It took a long time for HoPs to come on board with the project, even though they were responsible for line management on CAMI country activities. Finally, in Jan/Feb 2002 the project came under ‘normal’ management systems and everybody’s comfort levels were increased. Today, CAMI Coordinators report to CAMI PM on technical project issues. The HoPs managed all other administrative issues including reporting. This worked for CAMI, and is recommended for other similar projects Define management structures and systems before developing job descriptions, roles and responsibilities, and reporting lines before project implementation. Staff began working and planning together as a team when quarterly meetings were implemented with the program coordinators. This helped to create an environment of collaboration and cooperation with the exchange of viewpoints and knowledge. Budget accordingly for quarterly meetings of regional teams in proposals. 75 C. Staffing, Job Descriptions and Salary Issues 1. Findings a. Planned Versus Actual Staffing Project staffing changed significantly over the life of the project from a proposed staff of 21in the original proposal to an actual staff size of 35. The major change occurred in the area of country specialists (reduced from 11 to 7), to the addition of 8 temporary hires as well as the addition of 5 part-time drivers. The financial savings from this professional workforce reduction were utilized in the development and implementation of small mitigation projects. Proposal As of Nov 2000 DIP As of Aug 2001 End of Project Program Manager 1 1 1 Assistants to PM 1 1 1 Coordinators 4 4 4 Specialists 11 8 7 Administrative Assistants 0 0 3 Accountants 1 0 0 Drivers 1 0 5, part-time Liaisons, temporary hires 0 0 8 Total 21 14 35 b. Lessons Learned and Recommendations FINDINGS LESSONS LEARNED AND RECOMMENDATIONS Initial CAMI project staff members were technical disaster delegates with specific skill sets in specific areas. Disaster preparedness and planning projects should be considered the same as any other development project. Strong management skills are required at the regional level--more than technical skills. Hire regional staff with management and supervision skills. Hire country-level staff with more technical skills. Write better job descriptions outlining roles and responsibilities between disaster operations and disaster preparedness projects There were no standard CAMI job descriptions with a pay structure set up at the beginning of the project. This was mainly due to a lack of regional experience with managing long-term development projects. CAMI management structures were set up nationally. So, for example, Honduras staff was The perspectives of NHQ, Regional and Country Management differed most greatly when it came to the issue of salaries. A compromise recommendation is: * When there are bad feelings over salary structure, try to motivate staff in other ways; such as those offered in CAMI (e.g., opportunities for 76 FINDINGS LESSONS LEARNED AND RECOMMENDATIONS hired at NS Honduras rates, not at “CAMI” rates. Both the field and NHQ acknowledge that in some countries ARC/CAMI staff were paid well above established National Society rates – raising issues about sustainability once external OFDA funding ceased. American Red Cross local staff members who have been hired into other projects from CAMI have found their salaries cut to be more in line with National Society salaries. Despite salary challenges, CAMI staff members were motivated by being able to attend different courses, such as English classes. National societies were even not entirely happy with CAMI pay scales; which created challenges and pressures for ARC field staff not necessarily felt at NHQ ARC/NHQ felt that “job descriptions aren’t pay structures” – and that you cannot pay one person the same rate in four different countries. Examples used were the president of a National Society in one country not making as much money as a president in another. continuing education) * Before a regional project starts, establish job descriptions for regional- and country-level positions. THEN, reach agreement with National Societies on pay scales. * When recruiting, publicize the position and pay ranges by country so that differences are transparent from the beginning. Work plans were not developed for staff until 4 to 5 months after they were hired. The work plans were developed after a change in CAMI management took place. Develop work plans in the first month of a project. NHQ felt that CAMI was a “management experiment” for the Americas region. It was the first multi-country USG grant that the Americas region had managed. Most HOPs had little or no experience with grant management. There is an art to this kind of management, and not having had in-house capacity in the region slowed our ability to recognize and anticipate problems and situations. “We became a ‘USAID/PVO overnight.” was one observation. When existing staff are hired into new positions requiring different skills than they have utilized before, give adequate briefing and training to make sure there is an understanding of roles, responsibilities, and expectations. D. Project Budgeting and Financial Reporting 1. Planned Versus Actual Budgeting a. Cost-share:. The original award document from OFDA did not reference the cost-share commitment that ARC had put forth in our proposal. This wasn’t caught until several months into the project, at which point there was some back and forth between the desk and compliance staff regarding the need to report on 77 cost-share. The compliance officer argued that because it was not included in the grant, as awarded, we need not report on it. In the end, although we tracked our cost-share throughout the project, it was decided, in consultation with the DC OFDA grants officer, not to report on it. b. December 2001 Budget Modifications (1) Personnel Salary - Local - Decreased from $446,355.00 to $366,355.00. Note:$80,000.00 Decrease will be added to Small Mitigation Program (2) Travel, Meetings, Workshops - Increased from $10,675.00 to $23,525.00 Note: $17,850.00 Increase will be taken from contractual consultant fees and expenses (3) Contractual Consultant Fees – Decrease from $32,850.00 to $20,000.00 Note: $17,850.00 Decrease will be added to Travel (4) Item VI. Equipment Vehicle Purchase -Decrease from $20,000 to $0.00. Note $20,000 Decrease will be added to Other Direct Costs (New item-Rental Vehicle) (5) Item V. Supplies – Increase from $190,050.00 to $270,050.00 (for small mitigation projects). Note $80,000.00 Increase will be taken from personnel salaries (6) Item VII. Other Direct Costs – Increase from $189,840.00 to $209,840.00. Note Increase will be taken from equipment vehicle purchase FINDINGS LESSONS LEARNED AND RECOMMENDATION Original project over-budgeted salaries initially and did not allocate enough funds for field time for all trainings, including quarterly workshop or community DPP trainings including travel. This has consistently been a problem for ARC including all the recent proposals in the Americas. (This includes Washington cost shares.) Washington costs also seemed to be consistently over-budgeted. The Regional Delegation Office stated the opinion that ARC generally does not do “good budgeting”. During the life of the CAMI grant, ARC has developed capacity for better USG budgeting for long-term programming versus emergency programming. A lot of our financial systems were originally set up for disaster accounting and CAMI inherited domestic disaster systems which are slowly being adjusted over time. Country coordinator @ 100 % Regional coordinator @ 100 % if grant is large Budgeted NHQ overhead will depend on the size of grant and level of activity. However, proposal writers should probably budget for 10 to 30 percent involvement from NHQ. There was no budgeting in the original grant that would allow for financial sustainability of the region as a whole. The grant was a ‘loss leader’ whereby ARC contributed large amounts of cash to the project without any cost recovery mechanism. This led to some confusion with competing issues of delegation expenses for CAMI and Hurricane Mitch being ‘mixed’. Grant funded projects need to be separated from ARC projects, especially when monies are running low. The original award document from OFDA did not reference the cost-share commitment that ARC had put forth in our proposal. This wasn’t caught until several months into the project, at which point there was some back and forth between the desk officer and the compliance officer regarding the need to report on cost-share. The issue was not ARC has implemented new systems. All new grants are required to have separate account codes for billable items and the cost-share. No new recommendation. 78 FINDINGS LESSONS LEARNED AND RECOMMENDATION fully resolved until 15 April when, in consultation with OFDA, it was determined that ARC would not report on it in the final report to OFDA. NHQ didn’t always book all expenditures. Regional delegation did not get copies of financial reports that went to OFDA, except for some email copies. NHQ was often missing information from shared services in Virginia. Challenges with timeliness of budget versus actual expenditure reports (BVAs) throughout the life of the project led to over-expenditure of funds that ARC will have to cover ARC financial systems that use ‘Quicken’ at the field level remain problematic as a tool for country level financial management because the system does not allow the kind of reporting and tracking that is necessary for accurate and timely accounting. New quarterly reporting should improve the situation. During the CAMI grant, NHQ hired a regional finance delegate who has helped to streamline matters with the Shared Services, the finance department in Charlotte. No new recommendation. A review needs to be made of the Quicken system as an adequate country management tool. Until that time, regions and countries should plan to adapt their own financial systems to accommodate their/donor financial tracking/reporting needs. c. Project Reporting Findings and LESSONS LEARNED RECOMMENDATION Project reporting was done in OFDA formats – which were different than may have been preferred by the CAMI staff. Generally the system of narrative reporting from the field and comments sent from NHQ went fairly smoothly. Project staff should create reporting tools to help with management and indicator tables. There needs to be formats for micro-projects, and a reporting format for the field staff to report to regional staff. Use reporting formats that are flexible to meet demands of multiple users and conform to donor standards 79 Conclusions and Lessons Learned Monitoring and Evaluation of DPP Projects * * (Sources: CAMI Country Reports, Interviews with Regional and NHQ Staff, CAMI Evaluation Team) 80 VIII Monitoring Conclusions and Lessons Learned A. Baseline/Final Evaluation: 1. Revision of Logframes needs to keep pace with revisions in data collection instruments. 2. Many of the indicators, data collection questions, and methods were pilot tests. These pilot initiatives had to be developed, tested, and finalized during a two-week period in July 2001, after one week was spent clarifying the design and definitions of the project. Consensus was not reached on every point among the four countries. For example, having all four countries agree on one simulation scenario exercise for school drills that could be applicable in all 60 project areas was extremely difficult. The implementation of drills at the household level remained controversial until the end of the project. 3. At the time of the design of the baseline, elements of project implementation were still being worked out (e.g., we knew there would be community DPP trainings, but not exactly the focus and content of those trainings). As such, in some cases too much or extraneous information was collected about the implemented interventions. 4. Supervision of data collection is critical. Supervision was better in the baseline than during the final evaluation, resulting in better baseline data. ,Observation of ‘safe conditions’ in households was also difficult to define and, like household drills, remains controversial as to what the project should have been targeting for change given the fact that community-level, small mitigation projects were not in the budget. 5. The quality of some of the observations made by the interviewers is difficult to assess. The pilot testing phase of data collection was not successful in calculating inter-rate reliability. 6. CAMI staff mentioned that the short time of the project did not allow the project to see full changes in communities. 7. Culturally, in Central America documenting things in writing remains a challenge. As such, it worked to the advantage of the evaluation that people were asked verbally what they had, as well as whether or not it was documented. The DHoRD gave examples that included written directories of Brigades and emergency committee members. Just because directories were not available did not mean that the Community or school did not have the Brigades. 8. Many areas of measurement were new (e.g., disaster self-efficacy). The measurements were adapted from standard indicators of cross-cultural ‘health related self-efficacy’ that have been tested and validated for DPP project and/or in Spanish/Spanish cultures. However, these revised scales have not yet been validated and may not reflect ‘true’ disaster-related self-efficacy in Central America. 9. Observations may need instruments that distinguish conditions in rural/urban environments; type of hazard in the area – and types of changes we are looking for in plans (risk maps). This requires proper assessment of the schools. Instruments need to more clearly distinguish item not available/not applicable to change denominator. 10. Indicators and measurements were stronger on ‘response’ than ‘preparedness’ in both schools and households since the definitions of what is meant by “preparedness” are more clear. CUSE for example is much clearer in understanding the meaning of response versus preparedness and what the Red Cross is going to do about it. 11. Observations of existence of dangerous, heavy objects in inappropriate locations may be difficult to interpret. This is because many rural households and schools do not use these items, and a “not applicable” response was not available to allow appropriate adjustment of the denominator (i.e., percentages may be underestimates due to an inflated denominator). There is no reason to believe the denominator changed baseline to final; so while the point estimates baseline and final may be underestimate, observed changes are likely to be accurate be real. 81 12. The concept of ‘doors and windows’ opening to the outside were adapted from ARC materials and may not have been appropriate to Central America. 13. There was an original plan to run full scale, integrated drills in target Communities to supplement the segmented drills. However this only happened in El Salvador. This kind of supplementation is recommended in future projects. We lost the impact and the interaction of the brigades without this. 14. Methodology of the baseline can be adapted to include information about whether there is (1) a EOC that is being upgraded and whether (2) there is no EOC and it is being set up. 15. Estimating and/or imputing impact-level results in disaster preparedness is an emerging field. For example, in measles, you can estimate/impute the number of lives saved by calculating the number of vaccines delivered, vaccine efficacy, etc. Disaster preparedness is not at that stage yet, and operations research is recommended to enable these types of estimations to be calculated. 16. As an example, the CAMI Project can feel proud that 79 percent of rural community emergency committees can now get emergency messages back and forth to their 'headquarters' within 60 minutes of a simulated incident (increased from 17 percent at baseline). However, it cannot be estimated how many injuries/deaths/loss of livelihood that MAY prevent in the future. B. Monitoring 1. CUSE is currently an output-based program that assesses outputs produced such as the number of emergency plans. As such, although part of the original logframe, some of the initial educational indicators assessing knowledge and skills acquired during these trainings were not assessed. Further, knowledge gained by students was also not assessed. 2. A lot of substantive monitoring data was collected and utilized at the country level, (e.g., the results of community simulations). However, at the regional level, few summaries were made of the results of these exercises, and are therefore are not reported on in this final evaluation report. Quarterly reporting within ARC should now mitigate this from happening in future projects. 3. National Societies expressed that they would have preferred better integration of monitoring systems with National Society Systems. 4. National Societies stated that the project could have been enhanced by greater monitoring of the quality of the volunteers trained as trainers. 5. Not every person who is a good project manager can be a good data manager. 6. Quality verification checklists of assessments/risk maps and resulting disaster plans need to be developed for future projects. 82 Appendices 83 Appendix A: Evaluation Terms of Reference Terms of Reference (ToR) for completion of CAMI Final Project Evaluation and Final Report Outline April 2003 _________________________________________________________________________ 1) Background and Purpose of the Evaluation o The evaluation to be undertaken is a summative evaluation. Summative evaluation analyzes the outcomes of a program at the completion of project activities. As such, the focus of this evaluation is to assess whether or not the CAMI project interventions achieved their intended outcomes in an efficient, effective and sustainable manner. Based on these findings -- conclusions and recommendations will be made for other ARC and Red Cross/Red Crescent Movement Partners. o This evaluation will ensure our compliance with our grant obligations per ARC’s signed agreement with OFDA. As such, the evaluation report will written to ensure that relevant portions of the overall report can be input into the Final Report per OFDA Guidelines for Grant Proposals and Reporting. 2) Composition of the Evaluation Team (Stakeholder Participation) Team Composition. o 1 member from TAPE M&E as Evaluation Team Leader o Regional DHoRD o Regional Coordinator o 1 rep from one of the participating National Societies o 1 rep from the Federation/Regional DPP o DPP Manager from IDRU o 1 rep from CAMI Country Teams Commentors on Draft Report. o Miguel Vega, Salvadoran Red Cross o Corey Michaud o Ricardo Caivano, ARC country rep from one of the participating delegations 3) Dissemination and Utilization of Evaluation Findings. The evaluation report will be designed to ensure utilization by key stakeholders as follows: o OFDA: Can utilize lessons learned in future RFAs o Other CAMI NGOs: Can utilize lessons learned in future DPP projects o National Societies: National Societies can design and apply for their own grants utilizing lessons learned from CAMI. o IDRU: The evaluation report can assist in defining standard indicators for community DPP and methodologies for things that work and don’t work. 84 4) List of Documents Reviewed o Original Proposal/Amendments o Donor Agreements o List of Training Courses o List of Mitigation Projects Completed o List of DPP Workshops Implemented o Letters to OFDA o Regional CAMI Coordinator Review in each country o Baseline Survey o Final Evaluation Survey o CAMI monthly reports o COEs; Full-scale drill evaluation reports by CAMI staff with ‘Cepredemac’ e.g., Nicaragua, Honduras o Quarterly CAMI Project Reports (Apr – Jun 2001; Jul-Sep 2001; Oct-Dec 2001; Jan-Mar 2002; Apr-June 2002; Jul-Sep 2002; Oct-Dec 2002; Jan-Mar 2003; Final Report (as written) o Annual CAMI Progress Report (March 2001 – July 2002) o National Society Volunteer Information o Incident Reports o CAMI Schools Instructor training course records o End of Project Review with National Societies and Country Teams18 o End of project Review with CAMI Staff conducted by Regional CAMI Coordinator 5) Key Informant Interviews with CAMI Staff • Deputy Head of Regional Delegation, the Americas • Regional CAMI Coordinator • Americas Associate responsible for CAMI at ARC/NHQ • Manager of the Americas Region, ARC/NHQ • CAMI staff in the four countries answered an extensive end of project review questionnaire administered by the Regional CAMI Coordinator. These written responses were reviewed for summarization in the final evaluation report 6) Key Evaluation Questions, Evaluation Process and Methods. Consistent with the Federation Framework for evaluation, and OFDA donor requirements, the final evaluation report will present findings, conclusions, recommendations and lessons learned with regards to each of the following key questions. Table 1. Summary of Data Collection and Analysis Methods being used in Final Evaluation of CAMI General Evaluation Questions Detailed Evaluation Questions Source of Evaluation Data Methods Effectiveness of interventions Changes in preparedness Changes in response capacity Baseline/Final quantitative Assessment Baseline/Final quantitative Baseline/Final Assessment Document analysis and 18 Guatemala Report not available 85 General Evaluation Questions Detailed Evaluation Questions Source of Evaluation Data Methods Changes in responses (during life of project) assessment. Project Monitoring Data, (E.g., Drill Performance Evaluation Reports, School course evaluation forms) Incident Reports CAMI reports, key informant interviews with CAMI staff CAMI reports, key informant interviews with CAMI staff “” “” group assessment Document analysis and group assessment & key informant interviews Document review, analysis and discussion & key informant interviews “” “” Relevance: which interventions worked ‘the best’ Changes in Project Design from proposal submission to end of project Potential for replication of interventions Overall, what worked in the ‘design’; what didn’t – and why? How much of the design was necessary as part of RFA? What interventions were added, that were not originally intended, and why? What interventions were dropped, and why? Key informant interviews with CAMI project staff; CAMI project reports, National Society Volunteer Information; Emergency Committee Records, Volunteer Brigade Records, School Brigade Records; CAMI project reports Document analysis and group assessment; key informant interviews with CAMI staff Efficiency of interventions Cost per beneficiary analysis Cost per intervention analysis Person hours per intervention (for NS and ARC) Project Financial expenditure data Project Consolidated Indicator Table Key Informant Interviews Cost per beneficiary, by project intervention Key informant interviews and group assessment 86 General Evaluation Questions Detailed Evaluation Questions Source of Evaluation Data Methods Sustainability (proxied through capacity built) What was intended to be sustained? What is the likelihood that these elements WILL be sustained What was not intended to be sustained. What capacity was built to support the COEs and community DPP? Was it the right capacity? What else could have been done? What capacities do project participants think will still exist two years from now The impact of ‘legal status’ issues on the project. Key informant interviews with CAMI project staff; CAMI project reports, National Society Volunteer Information; Emergency Committee Records, Volunteer Brigade Records, School Brigade Records; CAMI project reports “” Client satisfaction assessment, key informant interviews with CAMI staff Key informant interviews, CAMI project reports Key Informant Interviews with project participants and “” Key informant interviews; focus groups “” Satisfaction Were expectations of project participants met? Strenths and limitations of participation in the project Overall satisfaction of participants with the project Client Satisfaction Assessment Client Satisfaction Assessment Key informant interviews with project participants Lessons learned in DPP evaluation Which measure worked, and didn’t work. What are recommended DPP indicators for future projects. Key Informant interviews with CAMI staff; baseline final assessment results. Key Informant interviews; document analysis and group assessment. Lessons Learned in Management What worked, and what could have been better, in multi-country project management Project timeframe. Too much, or not enough? Linkages between countries￾regional office-NHQ. What worked, what could be improved in future projects? Quarterly Reports Correspondence with OFDA; Key informant interviews with CAMI staff Document analysis and group assessment; Key Informant Interviews 87 7) Evaluation Time Frame and Resource Requirements January CAMI Coordinators meeting re: final evaluation February Last two weeks of February, data collection begins March First two weeks of March, data collection continues Last two weeks of March; data entry. Team compiles master file of all secondary project documents. April 7-18 April. Analysis and report writing; draft report started. Key informant interviews with CAMI staff via telephone. 20-23: Carol in Guatemala filling in gaps with documentation with written documents down in Guatemala. Completion of any key informant interviews missing. 24-25: Evaluation workshop. Review of draft report with key stakeholders. 30 May. Final report completed in English19. 19 First Draft Completed 25 April, 2nd Draft Completed ~15 June, Final Draft Finalized for Donor Submission 31 July 88 Appendix B: CAMI Final Evaluation Review Meeting 28 and 29 April Guatemala City, Guatemala 9:00 – 5:00 ________________________________________________________________________ Purpose of the Meeting: a) Review draft findings from the final evaluation b) To discuss interpretations of quantitative data c) To discuss if there are gaps in our understanding of effectiveness of interventions. Do we have the information to ‘complete the picture’ ? If yes, how do we ‘fill the gaps’. d) To discussion conclusions, recommendations and lessons learned e) To discuss presentation, dissemination and utilization of the evaluation report f) Next steps Evaluation Review Team. 1) Julia Guzman, Nicaragua Red Cross 2) Freddy Rosario, CAMI Regional Coordinator 3) Patricia McLaughlin, Deputy HORD 4) Diana Benitez, Federation OD Regional 5) Christine Leonardo, ARC DPP Manager 6) Dalia Castaneda, Guatemala CAMI Country Coordinator 7) Carol Puzone, ARC Technical Assistance (Evaluation Team Leader) Documents Reviewed. • CAMI Staff Surveys • Lessons learned by countries • Quarter Reports, Annual Report. • Baseline/Final Evaluation Questionnaires (in Spanish) • CAMI Budget as of 28 February (English) • Country End of Project Reports, including final budget figures. The discussion of the evaluation team around lessons learned and recommendations was organized in the following manner, for the three components of the project. Documents used as reference were the draft data findings of the baseline/final evaluation, results of lessons learned meetings held in each country with each of the 4 national societies and CAMI staff, and the CAMI staff survey. Guiding Questions ‘Cross-Cutting’ Issues What was Project Impact: Assessment/design Targeting Proposal writing/project start up Partnership Dpp interventions/cost effectiveness Potential for replicability/scale up a) did we have impact b) what did we learn, what do we recommend c) would rec’s improve impact or sustainability d) role of ns 89 AGENDA Day One Day Two 9:00 – 9:30 * Greetings and Introductions * Review of Meeting Purpose * Review of the CAMI Evaluation TOR. Where are we? Review of Day One 9:30 – 12:00 (with break) Review of Prelim Findings *Overview of Report Sections * Detailed review of the findings of the effectiveness and relevance of interventions; baseline/final evaluation data. Do we understand the findings based on CAMI interventions? Conclusions, Recommendations, Lessons Learned * Project Assessment and Design * “Beneficiary” targeting * Proposal Writing and Project Start-Up * Partnership with National Societies 12:00-1:00 Lunch Lunch 1:00-3:00 Review of Prelim Findings * Overview of Efficiency, Sustainability and Satisfaction with the Project Conclusions, Recommendations, Lessons Learned con’t * DPP Interventions * Potential for scale up/replication? * Management of Regional Projects * Monitoring, evaluation and reporting of DPP projects (e.g., recommended indicators) 3:00-5:00 (with break) * Con’t overview of Efficiency, Sustainability and Satisfaction with the Project * Start of Conclusions, recommendations and lessons learned – time permitting. * Presentation, dissemination and utilization of the evaluation report * Next Steps. Final report due 30 May. 7:00 Free Time Team Dinner 90 Appendix C: Equipment donated to CAMI Chapter COE’s El Salvador Chapter Donated Equipment Amount Computer, (CPU, Monitor, Keyboard and Mouse) 1 Printer 1 UPS 1 Phone/fax 1 Desk 1 Secretarial Chair 1 Department Map with acrylic protection 1 Municipality Map (of each location correspondingly), with acrylic protection 1 Topographic Document (of the corresponding Municipality) 1 1.85 x 1.20 M Acrylic Board 1 San Vicente Santiago de María and Zacatecoluca 1.85 x 1.20 Meters Acrylic/cork Board 1 Honduras Chapter Donated Equipment Amount FM/AM SONY Radio installed in vehicle No. 633 1 Air conditioning unit installed in vehicle No. 633 1 Headquarters and Chapters 4 x 4- meter Canvas tents, with floor, for eight people. 15 Computer Desktop - UPS 1 Desktop Monitor 1 Mouse 1 Key Board – 1 UPS Tripplelite 500VA OmniPro 1 Laserjet 1200 Printer - 1 4-drawer File Cabinet 1 Stand Fan 2 Ceiling Fan 2 Standard Desk (Arena) 1 Swivel Chair without arms 1 Comfort Star Air Conditioning Unit 1 Municipality Map and Honduras Map, laminated 5 San Lorenzo Cushioned folding chair 20 91 Metal/ imitation leather stackable Chair 6 Conference table 96-8-10 2 Panasonic TV set - 010 1 Panasonic Fax 1 Hitachi 4-head Video recorder 1 Overhead projector 2000 AG 3M 1 60 x 60 Tripod Screen 1 Set of Rods (5 sizes) 1 Motorola Portable Radio Slow Charger 2 Computer Desktop - UPS 1 Desktop Monitor 1 Mouse 1 Key Board – and Multimedia Kit (with Speakers) 1 UPS Tripplelite 500VA OmniPro 1 Laserjet 1200 desktop Printer 1 4-drawer File Cabinet 1 Desk telephone set COBY For Emergencies (195) 1 SONY Telephone -grey 1 30 x 48 Standard Desk (Arena) 1 Arm chair - Swivel Chair (without Arms) 2 Frigidaire Air Conditioning Unit 1 Municipality Map and Honduras Map, laminated 5 Metal/material upholstered Pile-up Chair 15 Four-module Conference (with two small extensions) 4 Panasonic TV set - 010 1 Hitachi 4- head Video recorder 1 Overhead projector 2000 AG 3M 1 60 x 60 Tripod Screen 1 Set of Rods (5 sizes) 1 Choluteca Motorola Portable Radio Slow Charger 2 92 Guatemala Chapter Donated Equipment Amount Compaq Multimedia Computer (CPU SPS), mouse, Keyboard Model 6512- WA Acer, monitor S700 1 PCStar 500RAT UPS 1 Screen Protector 1 Work Table (for 6) 1 Upholstered folding chairs 6 Two-drawer file Cabinet 1 Cupboard with lock 1 Desk 1 Secretarial Chair 1 Metal bookshelf 1 Telecraft telephone set 1 Acrylic Board 1 Cork Bulletin board 1 Astor Floor fan 3 Lakewood Fan 3 Chiquimula Konica Pop Z Camera with flash 1 Multimedia Computer: LG52X (CDROM drive), LG Studio Works Monitor, Mouse-Browser, Acer Keyboard, Starmicro SP-693 Multimedia Speakers 1 Screen Protector (neutral) 1 PC Star 500RAT UPS 1 Lexmark Color Printer 1 Work table for 6 1 Upholstered folding chairs (for work table) 6 2-drawer File Cabinet 1 Cupboard with lock 1 Desk 1 Secretarial Chair 1 Metal bookshelf 1 Telecraft Telephone Set 1 Acrylic Board 1 Cork Bulletin board 1 Wooden-easel paperboard 1 Astor Floor fan 1 Konica Pop Z Camera with flash 1 9 Jalapa COMPAQ CPU (black), V570 Compaq Monitor, Compaq Mouse, Keyboard Model SK-1688 1 Screen Protector 1 VAC 500 UPS 1 10 Escuint la Lexmark Color Printer 1 93 Sharp Fax Model UX510 1 Work table for 6 1 Upholstered folding chairs (for work table) 6 2-drawer File Cabinet 1 2-door cupboard with lock 1 Desk 1 Secretarial Chair 1 Metal bookshelf 1 Telecraft Telephone Set 1 Acrylic Board 1 Cork Bulletin board 1 Astor Floor fan 2 Konica Pop Z Camera with flash 1 COMPAQ CPU (black), Compaq Mouse, Compaq Keyboard S/C, multimedia Speaker System, 136BM Monitor 1 Screen Protector 1 Liebert UPS 1 Hewlett Packard Deskset Color Printer 1 4-drawer File Cabinet 1 Two-door cupboard with lock 2 Metal bookshelf 1 Acrylic Board 1 Cork bulletin board 1 Wooden-easel paperboard (broken) 1 Desk 1 Secretarial Chair 1 Sharp El-1801C Adding Machine 1 Exacto 15X14” Wooden Paper Cutter 1 Telecraft Telephone Set 1 Portable Radios with battery and charger Motorola 1 Headquar￾ters COE Konica Pop Z Camera with flash 1 94 Appendix C: List of CAMI Mitigation Projects Nicaraguan Red Cross / American Red Cross Initiative of Mitigation Project on Disaster for Central America Proposals of Masaya Micro-Projects Objective Goal Activity Place Provide a better use of the study area and use Hall as Auditorium. Build the walls of the School Auditorium using Plycem Repair the new Auditorium and repair electrical power system. Purchase construction materials. School of Concepción de María, Pacayita Masaya Students and teachers have better hygienic practices using latrines. Improve education when repair the roof of the study center. Repair the electrical power system in the directorate of the School. Construction of new latrines. Repair electrical power system Repair the roof of the other side of wing Construction of 4 latrines dug by hand. Repair roof using Corrugated zinc and 2x2 lumber. Repair the wiring system of the directorate in the electrical system. School, Santos Díaz Rivera Los 24, Tisma, Masaya Provide a large capacity to the school for the vital liquid consumption. (water). Repair Electrical System Installation of a 4,100 Liters Tank and its accessories. Repair entire Electrical System. 4,100 liters tank overhung. Installation of a new water tank. Repair Electrical System since it is 80% damaged in its structure. 11 School Esmeralda Gutiérrez, Monimbo Arriba, Masaya Improve the emergency exit with the construction of an additional gate. Improve consumption and storage of water. Rehabilitation of Electrical System. Construction o a gate at the right side of this center. Construction of a 4,100 liter Rotoplast tank. Repair 50% of the electrical system. Construction and installation of a metal gate, which shall be used as an exit in cases of emergencies. Installation and storage of drinkable water for consumption of children at the school. Repair 50% of the electrical system. Autonomous Center Augusto Flores Zúñiga, Valle de la Laguna, Masaya Improve education after fixing the roof of the study center Repair the electrical power system in the Principal’s office in the School. Roof frame using Corrugated zinc and 2x2 lumber. Repair the electrical power system in the directorate of the School. Repair the roof using Corrugated zinc and 2x2 lumber. Take off old Nicalit from the school. Repair electrical power system in the directorate of the school. School Benjamin Zeledón Norte. Tisma Urbano, Masaya 95 Nicaraguan Red Cross / American Red Cross Initiative of Mitigation Project on Disasters for Central America Proposals of Escuelas de Rivas Micro-Projects. Objective Goal Activity Place Improve the physical conditions of the school, and provide better sanitary hygienic conditions to prevent spreading of infectious diseases. Improve the electrical system installations so that it may contribute to reduce risks of fire caused by a short circuit Reduce risks of accidents among children by providing protection to drainage canals at the school. 1.Construction of two latrines 2. Electrical system in the school rehabilitated. 3. School drainage canales protected with an iron grill. 1. Construction of two latrines in the school dug by hand. 2. Repair some existent defects in the electrical system of the School 3. Construction of iron grills to protect drainage. School Salinas de Nagualapa Provide the minimum conditions for water supply, which shall satisfy the need of water for the students in the School Repair the electrical system of the School. 1,100-liter water supply tank already installed and operative. Electrical system of the school rehabilitated Installation of the tank, including construction of its foundation and the tower. Make repairs to the electrical system of the school School, San Jerónimo de la Chocolata, Rivas Reduce vulnerability of the school in the event of floods from ravines near the school. Improve the electrical system in order to reduce the risk of fire caused by a short circuit. Make the well, which supplies water to the school, works by the purchase of a hand rope pump. Construction of a retention wall Repair the entire Electrical System. Well pump already installed and working. Retention wall already built. Repair Electrical System, since its structure is 80% damaged. Purchase and installation of hand rope pump to supply the school. Purchase of extinguishers for the school. Purchase of a first-aid kit. School Simón Bolivar of the Community of Rio Grande. Rivas Improve the emergency exit when constructing an additional gate. Rehabilitate the Electrical System. Reduce vulnerability of schools and community when facing floods and also contribute to the population’s well being. Construction of a gate at the right side of the school. Repair 50% of electrical system. Access bridge built. 130 square meters Construction and installation of a metal gate, which shall be used as exit in case of emergency. Repair 50% of electrical system of the center. Construction of a ramp crossing the community so that it shall deviate any flooding. Purchase of first-aid kits and fire extinguishers School Salomón Ibarra Mayorga del Astillero, Rivas Reduce vulnerability of the school before the threat of a flood. Wall Built. Construction of a wall surrounding the school, Purchase of first-aid kits and extinguishers School Andrés Castro of the Community Nancimí, Rivas. 96 Nicaraguan Red Cross / American Red Cross Initiative of Mitigation Project on Disasters for Central America Proposals of Tipitapa Micro-Projects Objectives Goals Activities Place Improve the conditions for an appropriate and rapid evacuation of educational community. Reduce risks of fire by fixing the entire electrical system of the School. Construction of a gate located west of the school as an alternate exit for a faster evacuation during an emergency. Reduce risks of short circuit, repair electrical System. Start-up of gate construction. Demolition of concrete foundations of the wall in order to begin construction. Materials budgeted for the new gate including 7 square meters of starting￾up works. General repair electrical system. Autonomous School “José de la Cruz Mena”, Tipitapa. Elementary School and High-School Reduce vulnerability levels of children because of those open drains. Reduce risks of fire by repairing the entire electrical system of the school. Fence for drain built General repair electrical system. Fencing the whole drain of the school which represents a high risk for both children and teachers. General repair electrical system, in the elementary school building Autonomous Center of Las Maderas, Tipitapa. Elementary School and High-School Improve physical structure of the school, so that it may provide better conditions for children. Reduce risks of fire by repairing the entire electrical system of the School. Repair roof one wing which is the first priority of damages. Entire electrical system. Roofing with 1/8” lumber for a better durability in future years. Replace damaged electrical circuits for a better effectiveness of use and handling breakers and switches. Autonomous Center Concepción de María, San Francisco Libre, Tipitapa. Elementary School and High-School Improve the hygienic conditions of the school through the constructions of two latrines. Improve the physical structure of the school. Reduce risks of fires by repairing the entire electrical system of the School. Construction of 2 new latrines. Repair roof in one wing of the school. Repair electrical system Construction of 2 latrines dug by hand. Repair roof with Corrugated zinc and 2x2 lumber. General repair electrical system. School José Dolores Estrada Las Banderas, Tipitapa. Elementary School and High-School Reduce risks of fires by the repair entire electrical system of the School. Electrical system repaired and working General repair electrical system in high school Autonomous Center Salomón Ibarra Mayorga, Tipitapa. Only High-School 97 Salvadoran Red Cross / American Red Cross Initiative of the Mitigation Project on Disasters for Central America Activities Place Construction of gutters, for runoff and clear rubble from different streets, as well as the access to the school of the community. Community Montebello I, Santiago de María. Construction of a water collector tank and gutters to collect rain water, and an elevated tank for potable water. Catholic School Obispo Castro Ramírez, Santiago de María. Construction of catwalks and access steps to different buildings of the school, protection or handrail in existing stairways School Dolores de Jesús Montoya, Santiago de María. Construction of containment walls in the school, and gutters. Installation of lighting at El Modelo community. School Santiago de María and Community El Modelo. Santiago de María. Construction of an elevated tank for collecting potable water, in order to help the nearby communities since this water service is deficient in the event of an emergency. Salvadoran Red Cross, Sectional Santiago de María Construction of an elevated tank of potable water, and connection of other accessories for sanitary facilities and tubs. School San Antonio Caminos, San Vicente Construction of elevated tank for potable water and connections to toilets and wash tubs School Antón Flores, San Vicente Remodel Emergency Operations Center– Central, and electrical installations of the same area. Remodel Center of Emergency Operations (CRS). Remodel information area and construction of windows in that area. Arrangements in Information installations of COE￾CRS Remodel of external area of the Operational Center, in the front desk area and dispatch of emergencies, and repair electrical installations. Restoration of external area of COE- CRS. Guatemalan Red Cross / American Red Cross Initiative of Mitigation Project on Disaster for Central America DELEGATION COMMUNITY SCHOOL MICRO PROJECT DESCRIPTION Chiquimula Maraxcó Rural State Schools for Boys and Girls Plan del Jocote Water and Sanitation: • Installation of 1 water deposit of 1,000 liters • Installation of 1 sink with 3 faucets • Chiquimula Maraxcó Rural State Schools for Boys and Girls, Kindergarten and High-School (Telesecundaria) Water and Sanitation: • Installation of 1 deposit of water 1,000 liters • Rehabilitation of toilets • Installation of 1 tub Chiquimula Maraxcó Rural State Schools for Boys and Girls Héctor Manuel Vásquez Water and Sanitation: • Installation of 2 water deposits of 2,000 liters • Rehabilitation of toilets • Chiquimula Pinalito Rural State Schools for Boys and Girls, kindergarten El Pinalito Water and Sanitation: • Installation of 2 water deposits of 2,000 liters • Rehabilitations of 2 toilettes • Installation of 1 tub Escuintla Rural State Schools for Boys and Girls Santa Luisa Water and Sanitation: • Introduction of drainage piping • Place 1 deposit of 1,000 lts. • Installation of 1 pump Escuintla Llanitos Rural State Schools for Boys and Girls Llanitos Construction of perimetral wall Escuintla Las Guacas Rural State Schools for Boys and Girls Cuyuta No. 2 Repair warehouse roof Jalapa Tabacal Rural State Schools for Boys and Girls El Tabacal Construction of stair-like storm drains Jalapa El Pino Zapotón Rural State Schools for Boys and Girls Pino Zapotón Construction of a hanging bridge in the river of the community Jalapa Las Flores Rural State Schools for Boys and Girls Las Flores First Stage of Circulation of school yard: 65.5 meters perimeter of the school Jalapa Sabanetas Rural State Schools for Boys and Girls Sabanetas Construction of retention wall 98 Honduran Red Cross / American Red Cross Project How to Mitigate Disasters Objectives Goals Activities Place Improve physical conditions of the school to improve the educational well being and of the community. Construction of curbs. Repair electrical installation and water system of the school. Construction of external sidewalk of school. Construction of sidewalks in main street and next to the school. Repair water system of the school. Construction of sidewalk outside the school. School and streets of Colonia 19 de Septiembre Tegucigalpa Reduce vulnerability of 30 houses of Laure Sur Construction of 140 lineal meters per 2 mt. high. Construction of retention wall of 140 meters large and 2 meters High in the south sector of Laure river. South Sector of Laure Abajo, San Lorenzo, V. Reduce vulnerability School Augusto C. Coello before the threat of collapsing of the building Demolition and construction of performance center for the school and community Demolish old school module Construct an auditorium School Augusto Coello, village La Puente, San Lorenzo, Valle Reduce vulnerability of school before disasters and contribute to well being and protection of the students . Installation of 4 doors and 1 gate of emergency in the school Negotiation and approval of project. Negotiation and coordination of resources. Economic activities. Execution of installation project of doors and gate. Eschool Gerardo Medina, San Lorenzo, Valle Reduce vulnerability and protect the life of student population of the Elementary School Center. Reconstruction of roof of 4 classrooms. Reconstruction of roof of 4 classrooms. Strengthening of walls. Permanent maintenance Elementary Center Terencio Sierra, La Criba, San Lorenzo, Valle Prevent risk of being in￾communicated and the loss of human lives caused by floods. Reconstruction of access ramp (only one way access and evacuation). Reconstruction of Ramp. Repair section. Maintenance of ravine. School Gerardo Medina, Agua Zarca, San Zarca, San Lorenzo, Valle. Prevent deterioration of access way and in-communication of students and adult population of two communities Construction of retention wall 25 wide meters X 4 meters high on main access street to the community Selection of labor. Transportation of materials. Construction of retention wall Village Fray Lázaro, Choluteca, Choluteca Reduce vulnerability of school and risk of accidents for the students . Repair Module ( 2 classrooms) of Educational Center and Repair 328 meters Floor of Educational Center Plaster and internal and external painting of module of 2 classrooms. Repair module roof. Continue the project actions. School Oscar Alvarez Dubón, Palenque No1, Choluteca, Choluteca Provide safety to the student population of the school (located in a very dangerous zone due to heavy traffic). Construction of an 80- meter fence around school yard Making a Diagnosis. Organization of construction committee. Call to collective participation. Select labor. Construct /Install 80-meter wire fence School Ricardo Soriano, Las Arenas, Choluteca, Choluteca Reduce the risk toward the student population due to the poor condition of the physical structure of the school. Repair doors, windows Construction of sidewalks (68 linear meters) Installation of wire fence in the windows. Paint. Construction of 68 lineal meters of sidewalks. Change doors. Install wire fence in windows. Paint of walls. Repair roof. School José Cecilio del Valle, Copal Arriba, Choluteca, Choluteca 99 Honduran Red Cross / American Red Cross Project How to Mitigate Disasters Objectives Goals Activities Place Prevent landslide with obstruction and accidents of the students . Facilitate the access toward the school and community Construction of retention wall and stairway inside and outside the school. Socialization meetings of the project. Awareness of community. Hire qualified personnel. Construction. School Enmanuel colonia Cannaán, Tegucigalpa. Reduce risks for student population in case of an emergency or disaster. Provide the school with equipment needed for Emergencies and maintenance of curbs. Purchase equipped first-aid kits. Purchase maintenance/cleaning equipment and warehouse rearrangement Purchase and install tank/water Purchase of alarm megaphones. School República de Chile, Barrio El Reparto, Tegucigalpa. 100 Appendix D: List of CAMI Training Courses Name of the Course Materials/ Workshops Description of Contents For Whom are these Courses Duration Language Designed by or Adapted by Countries where ARC has performed the course Available in E-mail Course for Instructors (CPI) The purpose of this course is to train persons in order that they train others in different courses. This course includes: Instructional methods, how to draft objectives, development of lecture plans, review and assessment, effective use of equipment and visual help. Persons experienced in training other personnel and who work with Disaster Agencies and or are Chapter volunteers 5 days Available Spanish English USAID/OFDA Personnel El Salvador Guatemala Nicaragua Honduras Yes Damage Assessment and Analysis of Needs (EDAN) (Course) Acquire the necessary knowledge and skills in order to perform, in the field, a first assessment of the damage in health, vital lines, housing and infrastructure. Produce and perform a Needs Analysis and suggest priority actions. Volunteers and Technicians with leadership qualities, and due to their position, have the capacity to perform initial diagnostic of damages, analyze needs and suggest actions. 3 days Spanish USAID/OFDA Personnel El Salvador Guatemala Nicaragua Honduras Yes Damage Assessment and Analysis of Needs Decision Making (EDAN – TD) (Course) Provide with the knowledge and skills necessary to make decisions before an event that might generate adverse effects, based on the damages assessment and analysis of needs reports. Volunteers and technicians responsible for disaster response, are members of emergency committees, and are part of governmental and non governmental entities, at the local, regional and national level. 3 days Spanish USAID/OFDA Personnel El Salvador Guatemala Nicaragua Honduras No. Administration of Operating Centers (APD) (Workshop) Provide the necessary knowledge to form the Operating Centers in their different stages, information handling, and the process in the decision making, development of protocols and response procedures Volunteer personnel from the National Societies, such as Technicians of the Emergency Operations Center. 5 days Spanish USAID/OFDA Personnel El Salvador Nicaragua Honduras Yes Training Courses Provide all volunteer Chapter and 2 days Spanish RC No. 101 for Facilitators in Community Education (Workshop) personnel the training tools so that they can prepare the community in the handling of different methods of teaching. community Volunteers responsible for training others at the community, school and Chapter level. Federation’s Methodology oriented to CAMI objectives. Personnel CARE El Salvador Guatemala Basics of Evacuation (Workshop) This is an adaptation process based on practical experiences oriented to the development of evacuation basic evacuation fundamentals, that can be applied to communities, schools and other places as required. Chapter and community Volunteers interested in basic evacuation fundamentals on how they are used. 1 day Spanish CAMI personnel. Own materials and materials adapted to CAMI. El Salvador Honduras Guatemala Nicaragua No. Handling of Community Training Materials (Workshop) Provide the necessary tools for the use of the materials made in the CAMI project, to train the communities and homes in subjects such as the preparedness for disaster, Safety Home and Safety Community. Chapter and communities volunteers capable of being trained in the use of different tools and materials developed in the CAMI project so that they can train others.. 2 days Spanish CAMI personnel. CAMI materials. El Salvador Yes. There is a support in Format PDF. In a disc Community Rescue Course Training in the area of community basic rescue so that an immediate response can be given with the local resources available, according to the risks found in the communities.. For community personnel, Chapter volunteers, school personnel and students. 2 days Spanish CAMI personnel and from the Salvadorenian RC. Basic thematic adapted and taken from a Salvadorenian Red Cross manual. El Salvador Yes Workshop on Prevention and Fire extinguishing. Theory-Practice Training, designed to provide basic theory of priority needs for first responders at the chapter level. For First Responders and Volunteer personnel at the Chapters level. 1 day Spanish CAMI personnel and Firemen Manuals adapted and taken from the rescue manual of the Salvadorenian Red Cross El Salvador Guatemala Honduras Nicaragua No Course of Psychosocial Support in Emergencies. (Teachers) Train teachers so they can provide first emotional aid in group and individual sessions for children in Teachers and support personnel that work in different schools. 1 day Spanish CAMI personnel, Advisors and personnel of the National El Salvador Guatemala Honduras Nicaragua Yes 102 emergency situations. Societies Course on Psychosocial Support in Emergency. (Volunteers). Provide work tools to assists different populations in first aid psycho-social, either individual or group sessions; at community, schools, and Chapter level. Chapter and community Volunteers and Teachers. 2 days Spanish CAMI personnel, Advisors and personnel of the National Societies El Salvador Guatemala Honduras Nicaragua Yes Handling of Facilitators´ Manual on the Psychosocial Support in Emergencies. (Workshop) Training of Methodology on the emotional support services by school brigades; emotional support before and during an emergency and psychological first aid. Chapter Volunteers, school teachers and members of psycho-social emergency brigades. 2 days Spanish CAMI personnel, Advisors and personnel of the National Societies El Salvador Guatemala Honduras Nicaragua Yes Seminar. Workshop on Preventive Activities, and Mental Emotional/Health Support Activities in Disasters Intervention tools in mental health activities in disaster, to the different populations, besides affected participants; and the creation of their action plans. . Technicians of the National Mental Health Council of the Family National Secretariat and linking personnel of the Disaster Commission of the respective country. 2 days Spanish CAMI personnel, Advisors and personnel of the National Societies El Salvador Yes First Aid and Evacuation community Courses Provide community personnel with the basic knowledge on community first aid and evacuation, as well as the handling of different assistance techniques. Development of evacuation plans Persons of the communities responsible for first aid brigades and evacuation, as well as personnel responsible for community actions and decisions making.. 2 days Spanish CAMI personnel, Advisors and personnel of the National Societies El Salvador Guatemala Honduras Nicaragua Yes Error! Bookmark not defined.