Final Performance Evaluation of Partnership for Capacity Building in Disaster Management Program: National Incident Management System (NIMS) (Final Report) United States Agency for International Development (USAID) USAID Contract Number: AID-663-O-14-00022 Prepared by Dadimos Development Consultants Mulugeta Tefera (Team Leader) Agez Hidaru Guluma Soboksa Abdurehman Eid June 2014 Addis Ababa, ii Contents Acronyms..................................................................................................................................................... iii Acknowledgement ....................................................................................................................................... iv Executive Summary...................................................................................................................................... v 1. Introduction...........................................................................................................................................1 2. Background...........................................................................................................................................2 3. Scope and Methodology........................................................................................................................3 4. Findings.................................................................................................................................................4 4.1 Contextualization and Integration of NIMS Components ............................................................4 4.1.1 Contextualization ..................................................................................................................4 4.1.2 Integration ............................................................................................................................5 4.2 Functionality and Efficacy of NIMS ...............................................................................................8 4.2.1 Multi-Agency Coordination...................................................................................................8 4.2.2 Emergency Operation Centers............................................................................................12 4.2.3 Incident Command System.................................................................................................13 4.2.3 Strengths and challenges in the integration and functionality of NIMS.............................15 4.3 Process and Outcomes NIMS Trainings ......................................................................................17 4.4 Study Tours.................................................................................................................................21 4 4 1 Processes and key outcomes of study tours.......................................................................21 4.4.2 Recommendations from the study tour participants.........................................................22 4.5 Program Design and Management..............................................................................................24 4.5.1 Relevance of Program Design and Management...............................................................24 4.5.2 Coordination and Collaboration among Stakeholders ........................................................26 5. Sustainability.......................................................................................................................................28 6. Summary of findings and conclusions................................................................................................32 7. Lessons Learned..................................................................................................................................37 8. Recommendations...............................................................................................................................38 Annexes.......................................................................................................................................................42 Annex 1: Evaluation Protocols................................................................................................................42 Key References ........................................................................................................................................51 iii Acronyms AWSA: Addis Ababa City Water and Sanitation Authority BoA: Bureau of Agriculture CEPRP: Comprehensive Emergency Preparedness and Response Planning DRMFSS: Disaster Risk Management Food Security Sector DRMTWG: Disaster Risk Management Technical Working Group EOC: Emergency Operation Centres EW-ECC: Early Warning and Emergency Coordination Centre EWRD: Early Warning and Response Directorate FEPRA: Fire Emergency Preparedness Rescue Agency FGD: Focus Group discussion FMAC: Federal Multi Agency Coordination GoE: Government of Ethiopia HR: Human Resources HRD: Humanitarian Requirement Document ICPs: Incident Command Post ICS: Incident Command System IMT: Incident Management Team IRC: International Rescue Committee LIU: Livelihoods Integrated United MoA: Ministry of Agriculture MoARD: Ministry of Agriculture and Rural Development MoFA: Ministry of Foreign Affairs MoH: Ministry of Health MoWR: Ministry of Water Resource NGOs: Non-Governmental Organizations NIMS: National Incident Management System TWG-NIMS: NIMS Technical Working Group NPS-DRM: National Policy and Strategy- Disaster Risk Management DPPC: Disaster Prevention and Preparedness Commission QDA: Quantitative Data Analysis RRFA: Regional Reserve Fund Administration SC: Steering Committee SCI: Save the Children International SMAC: Strategic Multi Agency Coordination SNNP: Southern Nations and Nationalities Peoples Somali DPPB: Disaster Prevention and Preparedness Bureau TDY: Temporary Duty Assignment UNOCHA: United Nations Organization for Coordination and Humanitarian Affairs UN: United Nations USAID: United States Agency for International Development USDA: United States Department of Agriculture USFS: United States Forest Services USG : United States Government iv Acknowledgement The team from Dadimos Development Consultants wishes to express its thanks to David E. Horton III, Food for Peace Officer who is directly in charge of the evaluation, Demeke Eshete, Sarah McNiece from USAID for their critical comments on the evaluation methodology and as well as flexibility of the report submission. We are also greatly indebted to Muluneh W/Mariam EWRD Director, Adane Tesfaye from USFS, Beletu Tefera Federal EW-ECC Coordinator and NIMS Co-focal person, Rahel Asfaw Co- focal person from DRMFSS, Hassan Aden Deputy head of Somali DPPB, Hessen Barud NIMS program alternate focal person of Somali DPPB, Mahamed Reshid NIMS program alternate focal person from Somali DPPB, Commander Amauel Reda Deputy Director for Operation, Nesibu Yasin FEPRA Head of Training Institute, Beto Dima Deputy Director for Disaster Risk management from FEPRA and Mulatu Neguma Oromia DPPC Head of EWRPO, Dawud Mussa NIMS program Focal person, and Harun Husen ORF Unit Head from Oromia. We also express our gratitude to Markos Budusa from BDU for his conspicuous support in providing his views on NIMS program. The cooperation of woredas level cascade trainees and other humanitarian actors such as UN-OCHA field Office, and SCI field office, is gratefully acknowledged. v Executive Summary This report presents the findings of the Final Performance Evaluation of the Partnership for Capacity Building in Disaster Management Program: National Incident Management Systems (NIMS). NIMS is a $ 5 million and four year pilot program, technically and financially supported by US Government to strengthen the GoE disaster risk management system. The pilot program is still being implemented. However, this evaluation covered only the time up until September 2013 in which major pilot activities took place at federal level by DRMFSS, in Oromiya Region by DPPC, in Somali Region by DPPB and in Addis Ababa City by FEPRA with the technical support from USFS. Contextualization of NIMS Components Initially, the GoE identified four components of NIMS including MAC, EOC, ICS and CEPRP to integrate into the country’s DRM system. In the course of implementation the first three components focusing on emergency command and management system were found to be timely relevant to be integrated in the DRM system by the GoE. The three components have been contextualized, training manuals prepared and operational guides drafted during the pilot phase. Integration and Functionality of NIMS Components At federal level major attempts have been made by DRMFSS to integrate MAC, EOC and ICS within the existing DRM structures. From these SMAC has been formed by expanding the role of the earlier ad-hoc group called Emergency Coordination Forum (ECF). In the last two years the SMAC played a role in mobilizing resources and prioritizing emergency responses based on the request of the GoE through its HRD. TMAC was activated from DRMTWG to coordinate specific emergency responses in coordination of with ICPs. Currently, DRMFSS plays a leading role for both SMAC and TMAC, whose members are from key ministries, donors, UN agencies and NGOs. In the past three years, DRMFSS in collaboration with line ministries, Oromiya and Somali regions have activated ICPs to manage humanitarian incidences related to conflicts, return of migrant workers from Saudi Arabia and floods. According to evaluation informants from DRMFSS, UN agencies and regions the ICPs played important roles in speeding up and coordinating emergency responses by bringing different actors together and facilitating information flow from emergency places to DRMFSS which makes decisions over resources. In Oromiya Region existing structures have expanded their roles to assume new tasks and responsibilities introduced with NIMS to enhance multi-agency and multi-hazard approach. Based on this SMAC roles are being played by Disaster Preparedness and Response Committee while TMAC roles are being carried out by Regional DRMTWG. In Addis Ababa City SMAC was established and functional with the leadership role of FEPRA. However, TMAC is not yet activated or established. vi In Somali Region neither SMAC nor TMAC were established or activated during the pilot phase due to delayed NIMS familiarization to bureau heads and other higher officials as well as existence of two parallel humanitarian structures in the region. However, recently the regional DPPB drafted MAC operation guide and conducted a familiarization workshop towards the end of the pilot program. EOCs have been formed under DRMFSS and Oromiya DPPC. Both centers are currently working on compiling and sharing emergency situation updates. FEPRA has allocated offices space and planning to establish EOC soon. However, Somali DPPB has no plan or readiness to establish EOC. In this course some of the key challenges of integration of NIMS have been lack of linkage between EOCs and ICPs, low decision making role of the ICPs, absence of assigning NIMS trained personnel in the ICPs, shortage of logistic facilities for the ICPs on scene, limited professional diversity within EOC vis-a-vis the nature incidences in the county, and absence of approved NIMS operation guides that harmonize structures suggested by NIMS with the existing practices. NIMS Study Tours and Trainings The study tours organized by USFS were eye openers and motivators for the Ethiopian senior and middle-level officials, and experts to develop a good perception and attitude towards NIMS in the US and bring it to Ethiopia. The tours helped the GoE to identify and develop a strategy to integrate NIMS components in the country’s DRM system. A total of 41 (five women) Ethiopians participated in the study tours from different line ministries, regional bureaus and agencies. However, visa restrictions to some of the proposed study tour participants have created a dreadful sentiment, especially among NIMS Program implementers in Somali Region. Moreover, repeated participation of some people was indicated to reduce the chance to involve others from key line ministries in the study tours. NIMS trainings on MAC, EOC and ICS were instrumental to enhance human resources capacity in emergency resource management of the pilot regions and agencies. These trainings also served as a forum for contextualization of NIMS into the country’s DRM system. So far 59 and 601 people participated in the ToTs and cascade trainings respectively. From these only about 24% of the ToT participants have attended all the three trainings while about 19% of the cascade trainees did the same. No competency and facilitation skill test was given for master trainers before they involve in NIMS cascade trainings. This occasionally have resulted in a few incompetent master trainers facilitating NIMS trainings Sustainability of NIMS In general there is a strong desire and commitment from federal to regional level to pursue with NIMS components integrated with the DRM system of Ethiopia. The following are positive signs and challenges for NIMS Program sustainability. vii The positive contributing factors for sustainability of NIMS:  DRM policy and institutional commitments of the GoE,  Contextual relevance of NIMS to Ethiopia,  Preparation of contextualized NIMS training manuals and operation guides,  Existence of training institute under FEPRA dedicated for human resources development,  Interests of UN agencies and NGOs to participate in NIMS initiatives (e.g. ICPs) and funding the establishment of EOCs,  Availability of resources from some GoE to operationalize EOCs. Challenges and areas improvement for NIMS sustainability:  Absence of institutions dedicated to DRM human resource capacity building at federal or regional level,  Limited budget allocation for institutional capacity building outside of training from NIMS program,  Absence of a specific responsible structure within the DRM system to promote the integration of NIMS,  Lack of a clear scaling up and phase out strategy and commitment,  Existence of parallel DRM structures in Somali Region,  Use of non-standard/uniform terms and structures within DRM system,  Delayed approval and application of NIMS operation guides,  The NIMS pilot woredas and zones have not yet proactively put NIMS into practice, and  Absence of a binding memorandum of understanding among MAC group members at federal level. Key Lessons from the NIMS Pilot Program A range of lessons in the area of program implementation and management as well as policy processes have been documented by this evaluation in a way they are useful for ensuring sustainability and scaling up of NIMS. The key lessons are:  The NIMS study tours have played decisive roles in the institutionalization/integration of NIMS components into the Ethiopia’s DRM system.  Early familiarization of NIMS to key decision makers can play an advocacy and enlightening role for the integration of NIMS into the DRM system at federal and regional level.  The time overlap in the introduction of NIMS with DRM policy development process and institutional reform creates a higher propensity for acceptance and integration of NIMS components.  The strong strategic and technical leadership responsibilities of the DRMFSS in the NIMS program have helped the country to explore different options for the integration of NIMS. viii  Past experiences from DRMFSS and Oromiya DPPC indicate that there are possibilities of obtaining external resources for financing NIMS initiatives beyond the development of DRM system, trainings and study tours.  Standardized classification of emergency response resources can help for appropriate, coordinated and rapid responses to incidences. This was typically seen in FEPRA that classified resources by type and kind appropriate for incidence management based on the lessons from the study tours and technical supports. Recommendations Based on the findings of the evaluation study, the following recommendations have been put forward for consideration in strengthening and scaling up the NIMS program pilot initiatives in more coordinated and sustainable ways by considering the GoE DRM policy framework. 1. The challenges of of the full integration and functionality of NIMS components into the DRM system identified through this evaluation and others should be broadly discussed and a action plan part of the next phase on the NIMS program. 2. The next phase of NIMS program, regions should encourage and make woredas accountable to put NIMS components into practice. 3. Further standardization of DRM structures (such as committees and work groups), terms and emergency resources has to be promoted to enahance coordinated and effective communication and actions in the course of NIMS integration and incidence management. 4. Future NIMS Program should have a clear and transparent results framework that indicates expected outcomes, outputs and activities linked with budget. 5. Depending on budget availability, DRMFSS and regions (both piloted and scale up) should deploy fulltime or part time staff for coordination and monitoring of NIMS activities. 6. UN agencies and NGOs should be part of the NIMS steering committee to speed-up the integration and functionality of NIMS. 7. The next phase of NIMS program should allocate budget or come up with a strategy for obtaining other matching funds from existing complementary programs to ensure effective functionality of all NIMS components. 8. NIMS should consider resources and technical supports to build EOCWeb with components broadly capturing and sharing early warning information. Such system should enable EOCs to expand the existing use of and access to early warning and climate information from international data suppliers and NMA. 9. Future study tours should be organized in other African or Asian countries that practice emergency response management system similar to NIMS to overcome the problem of US visa restriction. 10. Priority should be given to ToT trainings to have adequate number of master trainers who have attended all the NIMS trainings at regional level. 11. NIMS operation guides should be further reviewed to align the contents with the existing DRM practices, structures and terms, and approved before scaling up of NIMS. 12. Develop, resource and apply a NIMS Program visibility strategy to demonstrate values of NIMS components for the promotion of multi-agency and multi-hazard approach through a decentralized DRM system as per the new DRM policy of Ethiopia. 1 1. Introduction This report presents the findings of the Final Performance Evaluation of the Partnership for Capacity Building in Disaster Management Program: National Incident Management Systems (NIMS). The objectives of the evaluation is to gain an independent view of the performance of the Program in order to help guide USAID, partners and other stakeholders to learn from what has been undertaken for future scale-up interventions, management and implementation. The evaluation exercise assessed the effectiveness of the NIMS components in helping the GoE to respond to disasters, the functionality and effectiveness of the NIMS established coordination mechanisms and the capacity of DRMFSS and pilot partners to support increased program scale￾up. The evaluation also identified and documented lessons from the program implementation in order to inform the subsequent program scale-up. The key questions the evaluation attempted to answer were program performance in terms of achievement of its objectives and results (outcomes and goal), the sustainability of these results and the overall management of the program. 2 2. Background Ethiopia is amongst the developing countries most vulnerable to natural and human-made hazards. The most profound hazards include drought, flood, forest fire, human and livestock diseases and crop pests. Drought induced hunger and famine, flood and civil conflicts are the most catastrophic, causing suffering to communities and millions of dollar worth of property destructions. Although the system of disaster management in Ethiopia seems to benefit from a paradigm shift, there are still salient problems that DRM has to reckon with. A consultative meeting and a brainstorming workshop between the USFS of United States Department of Agriculture (USDA) and the Early Warning and Response Department (EWRD) within the Ministry of Agriculture and Rural Development (MOARD) was held in February 2008 and October 2009 respectively for a potential partnership program between the U.S. and Ethiopian Governments to build Ethiopia's disaster management capacity. These events found the relevance and applicability of the U.S. Incident Command System (ICS) in command and management, preparedness, resource management areas in Ethiopia. In June of 2009, USAID commissioned USFS expert to develop a proposal and to hold further discussions with the federal Early Warning and Response Department (EWRD), regions and other stakeholders on the viability of a disaster risk management partnership program. Following this, ALT brought USFS disaster management technical experts to Ethiopia to introduce components of the U.S. National Incident Management System (NIMS) to GoE counterparts. Then a four year NIMS program fully integrated into the GoE disaster risk management system with estimated budget of $ 5 million program was designed with the GoE. The program is believed to significantly contribute to operationalize the GoE's National Disaster Policy, further mitigating and preventing disasters, while improving preparedness to respond to emergencies. It intends to strengthen the capacities for disaster risk management capacity of the GoE integrating the relevant components of the NIMS into the Ethiopian DRM system. The program also aims to provide technical assistance to further professionalize the DRM capacities of the partner institutions. The selected priority NIMS components include, Multi-Agency Coordination (MAC), Emergency Operations Centre (EOC), Incident Command System (ICS), and Comprehensive Emergency Preparedness and Response Planning. Partners to implement the program in the pilot phase are the Federal Government through the DRMFSS, Oromiya Region through Disaster Prevention and Preparedness Commission, Somali Region through Disaster Prevention and Preparedness Bureau and Addis Ababa City Government through Fire and Emergency Prevention and Rescue Authority. 3 3. Scope and Methodology The Final Performance Evaluation of the Partnership for Capacity Building in Disaster Management: NIMS Program is conducted using secondary data collected from documents and primary data attained from key informant interviews. Pilot regions and agencies of the program were the main source of information for this evaluation. Secondary Data Different relevant document about NIMS Program were reviewed at the start of the assignment. The team carefully studied the reports previously compiled by USAID, USFS and DRMFSS. Accordingly, all relevant documents including quarter reports, monitoring reports, field visit reports, stakeholders workshop reports, study tour reports, steering committee meeting reports, assessment reports and program manuals and guidelines were sources of secondary data for this evaluation. Primary Data Ten set of qualitative instruments to collect primary data from pilot regions and agencies were developed and applied to assess the performance, design, management arrangement, sustainability and impacts of the program. The instruments were composed of a set of structured questions for key informant interviews and key guiding questions for focus group discussions. The data collection was done from May 5 to 20, 2014. Key Informant Interviews (KIIs): Structured interviews were conducted with the staff of Federal Disaster Risk Management and Food Security Sector (DRMFSS), Oromiya Disaster Preparedness Commission (DPPC), City Government of Addis Ababa Fire and Emergency Prevention Rescue Authority (FEPRA) and Somali Disaster Prevention and Preparedness Bureau (DPPB) and Bahir Dar University (BDU) and Study tour participants. In addition, key informant interviews were conducted with select UN Agencies (UNOCHA) and NGOs (SCI and IRC). Data Analysis The qualitative data was analyzed using a qualitative data analysis (QDA) procedure. The data was transcribed from the notes and recorded voices collected from each respondent group and individual informants. Then cross-section content analysis under each question was carried out and crucial findings were summarized. 4 4. Findings 4.1 Contextualization and Integration of NIMS Components 4.1.1 Contextualization Since 2010 the Federal Government of Ethiopia (GoE) has been adopting the relevant components of National Incidence Management System (NIMS) in order to strengthen the country’s disaster management. The government started the process by incorporating the relevant components of US NIMS into the existing disaster risk management system through the funding and technical partnership (the NIMS Program) with the Government of the United States. The technical support is being given by United States Forest Service (USFS). After two rounds of study tours to the US (in February and August 2010), senior officials from DRMFSS, line ministries and regional bureaus identified four components of NIMS to adapt and integrate into Ethiopia’s Disaster Risk Management (DRM) system. These components are:  Multi-agency Coordination (MAC);  Emergency Operation Centre (EOC);  Incidence Management System (ICS); and  Comprehensive Emergency Preparedness and Response Plans (CEPRP). In the middle of the two study trours, an assessment was conducted on the DRM capacity of the GoE at federal and regional levels July 2010. Based on the outcomes of the assessment, NIMS program pilot regions and agencies were selected. Since then, the GoE, with different contextualization efforts, has been integrating the NIMS components and associated functions in the pilot regions and agencies. A two-round NIMS familiarization workshops were undertaken for regional bureaus in Oromiya (July 2012) and in Somali Region (September 2013). Similar workshops were given on time (November 2012) to people selected from Addis Ababa City Government key bureaus and agencies, civil society organization as well as Addis Ababa City Fire and Emergency Prevention and Rescue Authority (FEPRA). NIMS familiarization workshop in Somali Region was delayed because of the preoccupation of the regional bureau heads with their routine activities. Despite the huge effort exerted by the regional Disaster Preparedness and Prevention Bureau (DPPB) to hold NIMS familiarization workshop to the regional bureau heads as per the schedule, it was postponed three times before it was finally conducted in September 2013. Contextualization of NIMS into the Ethiopia’s DRM system was started during initial trainings on NIMS components identified for the country. The United States Forest Service (USFS) and DRMFSS organized the first MAC training of trainers (ToT) with participants from DRMFSS, key line ministries, pilot regions (Oromiya and Somali) and FEPRA, in January 2011, using the original training manuals prepared for US experts. Subsequent to this, in March 2011 a small group of the first cohort of master trainers (representing DRMFSS, MOH, MOWR, Pilots and BDU) were brought together to work on the contextualization of MAC training manual. At that 5 juncture, the MAC training manual was adapted to Ethiopia’s DRM system, jurisdictional contexts and disaster scenarios with a facilitation support from USFS. For instance, the tasks assigned to the different administrative structures and agencies in the US were changed to the context of Ethiopia. Moreover, hypothetical disaster response planning scenarios were also converted to Ethiopia’s disaster risk management scenarios in the contextualized MAC training manual. In this way the first Ethiopian MAC training manual was developed. The EOC and ICS training manuals were contextualized by DRMFSS, with the technical support from USFS, in the same way. The DRMFSS and USFS continually improved the MAC, EOC and ICS training manuals to reflect the Ethiopian realities based on lessons obtained from training events and pilot experiences. DRMFSS believes in the need for continuous upgrading of the training manuals along with further contextualization of NIMS based on future experiences and changing policy contexts. With regard to the fourth NIMS components, CPERP, trainings were given at the federal level for experts from DRMFSS, pilot regions and FEPRA. Yet, contextualization of CEPRP in the DRM system was not made at all. The main reason for this is that DRMFSS is having an ongoing woreda risk profiling initiative which works on emergency and response contingency planning. However, alike the other components, the training has built the disaster response planning capacity of the government staff and created the opportunity to share comprehensive preparedness and response planning guide template for woreda risk profiling. 4.1.2 Integration i. Federal-level Integration of NIMS The DRMFSS, with the technical support of the USFS, has developed operation guides for MAC, EOC and ICS to integrate NIMS in the Ethiopia’s DRM system. The process of developing operational guides involved the participation of DRMFSS senior expert team; key line ministries (MOH, MOWEI); NIMS focal persons from FEPRA, Oromiya DPPC and Somali DPPB; USFS and BDU. Finally, the draft operation guides for MAC, EOC and ICS were distributed to selected donors, UN agencies and NGOs to solicit their feedback. The operation guides set standards and procedures for the integration of the NIMS components into the existing DRM system of Ethiopia. Moreover, the guides describe as to how the NIMS command and management structures are activated and deactivated as well as their functions at federal, regional, zonal and woreda levels. The operational guides are awaiting the endorsement of federal authorities for their full-scale adoption at federal, regional, zonal and woreda levels as well as by other state and non-state actors. At the federal level, the NIMS components are being integrated into the DRM system and structures. MAC is one of the components being included in the DRM architecture. MAC group members are from relevant ministries (MoH, MoE, MoWEI, MoE and MoD), donors, UN agencies and NGOs under the leadership of DRMFSS. SMAC is chaired by the MoA State Minister from DRMFSS side, and co-chaired by UN agencies Resident Humanitarian Coordinator with the secretarial role of UNOCHA. The executives and decision-makers on emergency response resources from the above stated government organizations and non-state humanitarian actors are members of SMAC. 6 SMAC has taken over the roles of the earlier Emergency Coordination Forum (ECF). In the past the ECF was formed on ad-hoc basis where there were needs for humanitarian appeals. Members of ECF varied from time to time depending on the type of disaster. Currently, permanent SMAC members are known and whenever there is a need additional institutional members are pulled together to join the permanent SMAC members. Based on DRMFSS informants the SMAC group receives DRM situational updates from the DRM Technical Working Group (DRMTWG)1 chaired by the Director of EWRD, with technical and operational members from the abovementioned organizations. Based on the current practice, DRMTWG activates TMAC and ICPs when emergency situations are happening or looming to happen. The TMAC is led by Director of EWRD. Its members selected from DRMTWG based on the relation between the nature of the emergency and the mandates of the organizations they represent. DRMTWG meets on a monthly basis to review DRM situations in the country and TMAC meets on more frequently to provide guidance to ICPs and review emergency response situations on the scene. This technical working group has been there before the introduction of NIMS and technically lead the preparation of HRD and regularly discuss on multi-hazard issues. Currently TMAC is activated from DRMTWG when emergency situations are occurring or likely to occur. By DRMFSS, TMAC is seen as a subset of DRMTWG. In the past TMAC was activated during different times to technically coordinate emergency responses. DRMFSS and other DRM actors at federal level give strong focus on the importance of DRMTWG in the country’s DRM system. However, the draft MAC operational guide has not specified how the DRMTWG integrates in the DRM system. On the other hand the DRMFSS has the intension to maintain SMAC, DRMTWG and TMAC in the DRM architecture as multi￾agency forums for multi-hazard management. As an integral component of the NIMS, DRMFSS established federal-level EOC in March 2013 under the Early Warning and Response Directorate of DRMFSS. The centre is blended with early warning functions and termed as Early Warning and Emergency Coordination Centre (EW￾ECC) in the contextualization of NIMS by DRMFSS. The coordinator leads the ECC for Early Warning and Emergency Coordination Centre who is also NIMS Program co-focal person. The team members were given NIMS training after they assign as federal-level ECC staff. Their professional backgrounds are agro-meteorology and crop production. Nonetheless, experts competent in livestock, conflict management and food market information system for early warning and decision-making purposes seem to be lacking. ii. Integration of NIMS in Oromiya and Somali Regions The Oromiya Regional DPPC has assumed the responsibility of integrating NIMS components in the regional DRM system. Lessons from study tours and NIMS training as well as engagements in federal forums promoting NIMS program have helped in this regard. In the past both SMAC 1 It was also operational within the DRM system before the introduction of NIMS. 7 and TMAC groups have been ‘formed’ despite there is no MoU that binds the participant organizations. The Oromiya Regional DPPC is waiting for the endorsement of Federal NIMS guide to develop its own operational guide for MAC groups. The DPPB in Somali Region is responsible for the integration and application of NIMS components in the regional DRM system. Despite the initiative of drafting regionally contextualized MAC operational, based on the federal level draft guide, by the DPPB and the participation of regional and pilot woreda staff members in NIMS training, both SMAC and TMAC groups are not integrated in the regional DRM system. This delay in integration is associated to the lag in NIMS familiarization workshop for regional DRM actors, which was behind schedule (September 2013). In addition absence of participation of the region in the NIMS study tours has reduced the motivation of officials to quickly run the integration work in parallel with the other pilot region and agency. According to a Somali DPPB informant, the familiarization workshop for the regional senior officials was postponed three times due to the unavailability of the regional bureau heads. In addition, the presence of two parallel and separate DRM forums that lead and coordinate humanitarian issues in the region has made the integration of MAC difficult. The two forums are the Humanitarian Coordination Forum led by BoFED, and the DRM Technical Working Group led by DPPB. Interestingly, BoFED has a mandate on the emergency resources while DPPB has a judiciary mandate on DRM with control over resources. From the pilot regions, only Oromiya Region dedicated facilities and established a regional EOC within the regional DPPC. The establishment of EOC started with the allocation of a room and staff by DPPC. Based on the experiences obtained from the US study tours and capacity building efforts of NIMS program, the Commission developed a program proposal and solicited funds from donors, NGOs and UN agency to equip the EOC with information communications facilities and operational procedures. As a result, so far it was able to equip the centre with the necessary facilities with financial support from Save the Children International Ethiopia. ICS is used to organize on-scene operations for a broad spectrum of emergencies from small to complex incidents, both natural and human made. The GoE and humanitarian actors in the country have proven ICS to be an important part of command and management component of NIMS as a means for strengthening on ground coordination of emergency responses. Some of the ICS functions were not new to Ethiopia as emergency response team (ART) was assigned and dispatched for on the scene disaster management before the NIMS program. However, the current Incidence Command Posts (ICPs) replacing the ERT started to engage multi-agency with multi-hazard coordination concepts during disaster responses. The assimilation of these concepts with the leadership of DRMFSS has been demonstrated to a limited extent in Addis Ababa, Oromiya and Somali regions (see the functionality section of this report). As indicated by DRMFSS there were a number of humanitarian agencies that showed the interest to be part of ICPs. However, in the past it only allowed agencies with readily available emergency response resources and presence in the disaster areas to be the members of ICPs. However, limited NIMS trained ICP participants and lack of adherence to approved ICS 8 guideline and tools has been indicated to be the limitation for the full and harmonized integration of ICS. iii. Integration of NIMS in Addis Ababa City, FEPRA The City Council of Addis Ababa and the Mayor are providing support to FEPRA and other DRM actors in the city for a full-scale integration and application of NIMS components being adopted by the country. So far, FEPRA, in consultation with DRM partners in the city and USFS, has developed MAC and ICS operational guides for the Addis Ababa City Government DRM. These operational guides took urban disaster and risk scenarios and the recently (2013) revised mandate and structure of FEPRA into account. The MAC guide is approved by the City Council, and the ICS guide is being commented upon by MAC member organizations. FEPRA has translated the MAC guide into Amharic. The MAC operational guide indicates the existence of two permanent structures at City level, including S-MAC and T-MAC and other two MAC structures City-MAC (CMAC) and Sub-city MAC (SCMAC) to be activated during emergency times. The document provides guidance on the roles of the different MAC structures and the activation and deactivation of CMAC and SCMAC at the city and sub-city levels, respectively. FEPRA has dedicated office space for EOC at its headquarters. The Authority is committed to operationalize EOC by assigning the required human resource and office facilities soon. The EOC operation envisages guarantying the activation of CMAC, SCMAC and unified command centres during incidences by providing early warning and response information to DRM actors. 4.2 Functionality and Efficacy of NIMS The functionality and efficacy of NIMS within the DRM system of Ethiopia (pilot regions and agencies) depends on the level of integration of NIMS components which is facilitated or constrained by different factors. Participation of leaders, existence of harmonized DRM structure, contextualization of NIMS, favourable policy and strategies and existing human resource capacity are the main factors. Based on these factors, the efficacy and the extent of functionality of NIMS components vary by pilot region and agency. 4.2.1 Multi-Agency Coordination During the pilot phase, three out of four NIMS components (MAC, ICS and EOC) mainly focussing on command and management in the DRM system, were functional at different levels of efficacy in Ethiopia. The fourth component, CEPRP, which was supposed to focus on building capacity of planning disaster responses, has not been functional as part of NIMS program. Thus, this section of the evaluation examines the functionality of the first three NIMS components including MAC, ICS and EOC. 9 Federal Multi-Agency Coordination DRMFSS has the authority to lead the coordinate disaster risk management (DRM) activities in the country. Therefore, it is functioning as a pivotal body for the MAC at federal level in times of disaster. At the same time, it is spearheading integration and functioning of the NIMS components into the DRM system throughout the country. With the leadership responsibility of DRMFSS, it is possible to claim that MAC, EOC and ICS have started to function at federal level. The SMAC group functions as a continuation and expansion of the roles of the earlier Federal Emergency Coordination Forum (FECF) at federal level. The SMAC group is led by the MOA State Minister. The different feature of SMAC from FECF is that it has got multiple actors from government ministries (MoWIE, MoA, MoA, MoFA and other ministries as appropriate), donors, UN agencies and NGOs. Senior officials from these organizations are also members of SMAC groups. The group members meet as required during disaster time to discuss and pass decision over resource mobilization, coordination and prioritization for emergency responses as per the Humanitarian Requirement Documents (HRD) presented to them by the GoE. In the past, DRM actors often responded to disasters unilaterally and in less coordinated ways during emergency time. Thus, the functionality of SMAC and incorporation of multi-agency and multi-hazard approaches, have enabled efficient resource use and coordination of the DRM system. The DRM actors discuss over a report from DRM technical working group and make emergency response decisions in the SMAC meetings as per the HRD. In the meeting, the SMAC group members agree on roles and responsibilities of the different actors in relation with the HRD. In addition, the SMAC group members meet to review the progress and discuss over outstanding issues of ongoing emergency responses. Based on this, HRD of the country were resourced jointly by SMAC group members twice in 2013 FY and once in 2014 FY. However, some donors and non-state humanitarian actors consider that the HRD preparation process to be less consultative and participatory. According to DRMFSS, the TMAC group has started reviewing ICPs reports from emergency scene and providing ICPs with necessary supports, by convening more frequently compared to DRMTWG. This practice is an indication of promoting multiagency participation and direct information flow from ICPs at the emergency scene to MAC member institutions. However, the practice is not always the case as the system is immature. “Previously DRMFSS, MoA, MoH and others were working independently NIMS has created coordination of emergency response and management efforts. This has substantially reduced duplication of resources delivered by the different agencies to the same needy people. Now we started to jointly assess and coordinate shared responsibilities by working together through MAC.” [DRMFSS key informant]. Although there is a progress on a functional integration of MAC in the DRM system at federal level, there is no binding memorandum of understanding among its member ministries and organizations. This approach is unbinding to both governmental and humanitarian actors to take part in MAC activities on regular basis or during disaster times. According to DRMFSS this 10 problem will be addressed after the legislation of the new National Policy and Strategy on Disaster Response and Management (NPS-DRM). Oromiya Regional Multi-Agency Coordination In Oromiya, the regional Disaster Prevention and Preparedness Commission (DPPC) is a house for multi-agency coordination (MAC) forums. The MAC system as one of the piloted components of NIMS are being executed by the existing disaster risk management structures of the region called Disaster Prevention and Response Committee, whose members are heads of regional bureaus, including BOWIE, BoH, BoE, Livestock Development Agency and Cooperative Promotion Agency. The roles of TMAC are also being played by regional DRM technical working group whose members are drawn from the directors, process owners and experts of the above regional institutions. According to the regional officials, there was no need to change the existing names of DRM coordination structures to SMAC or TMAC thus far. However, the regional DPPC informants use the new terms (SMAC and TMAC) interchangeably with the name of the existing structures. This is mainly associated with the absence of NIMS operational guides to apply in the course of integration of NIMS. Activation of MAC group was not required in the past as there were no major disaster events in the region. However, both SMAC and TMAC groups are meeting to discuss on DRM issues. The region is waiting for the approval of the federal MAC operational guide to develop its own contextualized version of the guide. “An inter-sectoral coordination and collaboration on disaster management has highly improved. A few years back it was not possible to get representative/ leaders of any sector bureaus for DRM meetings without the circulation of formal letters. This made the decision-making time long. Now, it is possible to hold regional SMAC members meetings by simply making telephone calls by the DPPC Commissioner or Vice Commissioner. In the same way, members of TMAC can be invited for meetings through telephone calls of the technical MAC coordinator in the commission.” [Key informants from Oromiya Regional DPPC]. The coordination and collaboration level of the existing structures have been strengthened due to series of capacity building interventions such as study tours, seminars and NIMS training. In the past, calling committee members that played roles of SMAC had not been easy. DPPC had to write a letter and wait for days to hold meetings. These committee members can now immediately come together to discuss and pass decisions over DRM issues. “We can say one thing; the NIMS program has contributed toward enhancing the disaster risk management of our region.” [Key informants from Oromiya Regional DPPC]. Somali Regional Multi-Agency Coordination Multi-agency coordination in the DRM system has not been functional in Somali Region until the time of this evaluation. As indicated in the NIMS integration section above, the NIMS 11 familiarization workshop for regional DRM actors was conducted lately. In addition, the existence of two parallel regional DRM structures in the region has hampered the integration of NIMS within the existing structures. Lack of exposure of regional DRM authorities and experts to international study tours has also played negative role in dragging the operationalization of MAC structures in the region. With these constraints, the regional DPPB informants also acknowledged that there is an effort to establish the MAC structure and develop operational guide at regional level. However, operations of two parallel structures on DRM have to be resolved in order to ensure the integration and functioning of MAC and other NIMS components in the regional DRM system effective. “MAC is not established because there is an operational guide that must be finalized first. We have almost finished the operational guide and now will move to establish MAC. The guide is adapted from the federal level draft MAC guide.” [Regional DPPB key informants]. Addis Ababa City Multi-Agency Coordination In 2013, FEPRA was restructured and legislated by the Addis Ababa City Council with the mission of informing the city residents about urban risk of hazards, and preventing and controlling disasters in the city. Previously, FEPRA solely focused on a fire fighting approach. In line with the restructuring and the new legislation, however, FEPRA together with civil society organizations and city-level government stakeholders is moving towards multi-hazard and multi￾agency coordination approach. The approved city-level MAC operational guide and experiences learned from NIMS program are contributing to this move. FEPRA serves as focal body for multi-agency coordination (MAC) at city-level in times of disaster. FEPRA, Addis Ababa Water and Sewerage Authority (AWSA), Bureau of Labor and Social Affairs, Bureau of Women and Youths Affaires, Bureau of Health, Addis Ababa Police Commission and Federal Police Commission, Addis Ababa Electric Utility, Addis Ababa Ethio-telecom Branch and Addis Ababa Red Cross Society are permanent members of MAC. The role of each agency varies based on the nature and scope of the disaster. “NIMS program is contributing to the realization of FEPRA’s Vision and Mission. So far, MAC, EOC and ICS are found appropriate for this purpose. We are also coordinating our efforts with others. Through MAC, we have realized the existence of resources within the city that can be pooled together during disasters.” [Key informant interviews with FEPRA authorities.] According to the deputy director and NIMS focal person at FEPRA, the formation of MAC system along with NIMS training and awareness raising activities have created wider understanding among the city government, and different bureaus as well as agencies. For this reason, the different DRM actors in the city are regularly meeting and discussing prevention and control of disasters with the coordination of FEPRA. So far, only SMAC is functional from the NIMS architecture. Although the role of TMAC is described in the City MAC guide, the body is 12 not yet formed. However, FEPRA is taking part in the federal NIMS technical working group led by DRMFSS, thus acquiring experience which would enable it to form a city level TMAC. 4.2.2 Emergency Operation Centers The Federal EW-ECC, which performs EOC functions, has three functional rooms: situations update room, operation room and decision-makers (SMAC) room. The situations update room has four permanent staff members that work data collection, analyze and dispatch early warning information for decision-making by federal authorities and for public consumption on regular basis. Although the necessary facilities and office spaces are provided for each of the EW-ECC rooms, the operations and SMAC rooms have not been active as there were no major disaster events triggered full-scale activation of the ECC. However, all the rooms are fitted with networked and internet connected computer system as well as large situation update display screens acquired through financial support of UNDP. Given the existing setting, the ECC has huge potential to assist agencies and organizations to fully integrate the components of NIMS in the DRM system. The situation update team is regularly obtaining meteorological data from international sources and Ethiopia Meteorological Agency (EMA) as well as early warning information from regions and different agencies/departments under MoA indicating incidents and situations at local and regional levels. With the introduction of NIMS and the establishment of EW-ECC, Early Warning and Disaster Response Directorate has improved its facility and system to capture and disseminate early warning information. However, logistics and budget shortages at woreda and zonal levels are still the constraints that hamper federal EW-ECC from obtaining ground-level early warning data on time. In this regard, the EW-ECC hopes for the timely accession of early warning data from woredas with the completion of Woreda NET connectivity system that is underway. With regard to EOC, Oromiya Region has already established a funding scheme with Save the Children International (SCI) Ethiopia for the purchase of fixed items and budget. Based on this, the regional DPPC operationalize EOC with three staff and furnished with the minimum required equipment and facilities. The EOC is awaiting for the start Woreda Net system to establish a wide area computer network system with woreda DPP office for automated data collection. Large display screen, disaster profile maps and computer systems are available in a large single room dedicated for EOC. The regional DPPC has allocated a running budget for the EOC. Somali DPPB has developed a proposal to request funds from UNDP and WFP to establish EOC. DPPB plans to establish and operationalize the EOC as soon as the funds are secured. FEPRA is planning to establish EOC base at its headquarter. EOC operational guide has not yet been prepared. The MoH has agreed with FEPRA to assign three doctors and an ambulance which will work at the EOC for 24 hours a day. In addition, FEPRA is searching for funds to equip the EOC and make a real time linkage (online watch system) between the EOC and emergency scenes during incidences. 13 4.2.3 Incident Command System Since the launching of NIMS program, incidence command system (ICS) has started to function in the pilot agencies and regions by activating incident command posts (ICP) to coordinate emergency responses at the scene. These include:  ICP activated by DRMFSS to coordinate reception and dispatch of Saudi migrant worker returnees in 2014;  ICP activated by FEPRA to coordinate the control of fire and rescue lives and properties during the Merkato area fire incidence in November, 2013;  ICPs activated by DRMFSS and Somali Regional DPPB to coordinate emergency responses in Godey and Jigjiga; and  ICPs established by DRMFSS and Oromiya Regional DPPC to coordinate emergency responses in Koni, Chiro and Yabelo. To facilitate the reception of Saudi migrant worker returnees in 2014 DRMFSS made attempts to activated ICPs from the relevant ministries and agencies by assigning commanders from its staff. According to DRMFSS informants the ICPs formed were not as per the NIMS principles. The people assigned were not structure as per requirement for the different posts in ICP and have no adequate skills and knowledge to apply. Both NIMS trained and untrained ICP personnel were assigned at the command posts within Addis Ababa. Nevertheless, in those command posts the ICPs played important roles in psycho-social support, organization of temporary shelters, food and cash handing, and providing transportation. The ICP members who took the ICS training have provided better service under the post they assumed at the time. Participants of these ICPs, by representing DRMFSS, indicated better response to crisis in relative terms due to the introduction of NIMS. For instance some of the NIMS trained participants had applied ICS monitoring forms to communicate with the area commander which was responsible to coordinate the different ICPs. However, this was not uniformly done across all the ICPs. The ICP activated by FEPRA in Addis Ababa had helped a lot during the fire disaster in Merkato on November 11, 2013. During this incidence, FEPRA activated a unilateral and on-scene incident command post (ICP) with a command system that only involved its staff. Information sharing to the different agencies involved in the rescue operation was done through FEPRA to facilitate coordinated responses during the incidence. Currently, FEPRA and other city-level DRM actors are working to create a system that would help to activate a unified command post during emergency operations which require multi-agency responses. ICPs were established in Godey and Jigjiga of Somali Region to respond to flood damages and conflict induced crisis, respectively. Similarly, ICPs were activated in Yabelo of Oromiya Region to respond to crisis which emanated from migration of Kenyan pastoralists to Ethiopia as well as in Kumbi to respond to crisis due ethnic conflicts between Oromos and Somalis. In both region, ICP members came from DRMFSS, regions, zones, NGOs, woredas and UN agencies (UNOCHA, UNICEF and WFP). In both regions, the activated ICPs were led by commanders from DRMFSS. The involvement of federal government in the formation and leadership of ICPs 14 was mainly required because decisions over resources for disaster response are made centrally by the DRMFSS, as per the existing policy. In addition inter ethnic conflicts and cross-country nature of the incidents demanded the involvement of the federal government in the ICPs. According to the federal and regional informants, in the past, disaster response teams were deployed on emergency scene without proper skill and knowledge of managing disasters in unorganized manner. As a result, the information required by federal authorities to decide on emergency resource dispatching and utilization had to go through the administrative chain, including woredas, zones and regions. This was one of the causes for the relative delayed emergency responses. Thus, the incorporation of ICS in the DRM structure broke the long chain of information flow and created direct linkage between ICPs on scene and DRMFSS. In this manner ICP enhanced relatively effective, better coordinated, transparent, appropriate and timely emergency responses. For instance, the Godey and Jigjiga ICPs coordinators were instrumental in creating relatively effective link and harmonious communication among UN agencies and NGOs, and with regional and federal government. The ICP at Godey worked closely with UN agencies and NGOs in the multi-agency flood disaster assessment and helped DRMFSS to clearly understand the situation and allocate 100% of the resources requested. Based on the information from Oromiya DPPC, in addition to information linkage and coordination, the ICP at Koni provided supports for the woreda level concerned structures in conflict resolution and management. Specifically the ICPs activated in Somali Region encounter shortage of office, communication and transport facilities. On top of this, the person assigned to lead the ICP in Godey did not go through NIMS training. “ICP is the most important ICS facilities that helped to strengthen on-ground coordination of emergency responses from assessment to delivery of assistance”. [A key informant] FEPRA recognizes the relevance of ICS as it suits urban areas, mostly those affected by fast onset disaster. As a result, it has been applying different lessons learned from the US study tours in its institutional system. Classifying human resources and machinery by type and kind according to the response needs during incidences is one of the actions being taken for the application of ICS. Based on these lessons, the post of fire-fighter, which was collective in the previous FEPRA human resources structure, is now structured at three-levels as fire-fighter 1, fire-fighter 2 and fire-fighter 3. Similarly fire trucks are grouped under two types, based on their capacity. The federal ECC and ICPs are less coordinated to feed into MAC and make effective command and management structures in the DRM system. This is mainly due to the fact that the incorporation of NIMS in the Ethiopian DRM system is at its infancy stage. There are no operational guides, individuals not trained in NIMS are sometimes assigned as ICP commanders, and as indicated above commanders have no adequate power to make decisions over resources. Due to local capacity gaps and long-standing practices, centralized emergency resources 15 management and decentralized decision-making on emergency responses have not yet been in place. Therefore, ICPs cannot make decisions on resources while they are at emergency scenes. The implementation of the new National Policy and Strategy on Disaster Risk Management (NPS-DRM) issued in July 2013 is expected to promote decentralized disaster risk management system. The enforcement of this policy requires government proclamation to realign the policy with other policies and strategies of the country, create disaster risk management coordination structures, and improve physical, financial and human resources capacity for disaster risk management from local to federal levels. “The country is following decentralized disaster risk and management system. Thus future focus with regard to adopting NIMS components would help realize its goal.” [KII from a humanitarian agency] 4.2.3 Strengths and challenges in the integration and functionality of NIMS Since 2010 major achievements have been gained in the integration and functionality of NIMS with the Ethiopia’s DRM system in piloting NIMS components. This evaluation has come across the following key areas of strength and challenges of the piloting process that have to be taken into account as lessons and areas of improvement in the course of improving efficacy of NIMS components in the pilot areas and agencies as well as nationwide scaling up the system. i. Multi-agency coordination system a. Strengths of MAC 1. SMAC has helped to expanded multi-agency and multi-hazard approach in DRM by bringing decision makers from relevant federal ministries, donors, UN agencies and NGOs. 2. TMAC activated from DRMTWG at federal level has started to provide support to IPCs on emergency scene. 3. NIMS introduction into the DRM system of FEPRA during the its restructuring and legislation process from agency to authority level supported FEPRA to mainstream multi￾agency and multi-hazard approach. 4. The preparation and approval of Addis Ababa City MAC Operational Guide has promoted clear understanding and brought binding engagement of DRM actors in the city with leadership responsibility of FEPRA. 5. The multi-agency coordination approach followed by the Disaster Prevention and Response Committee, after the integration of NIMS, has improved collaboration and coordination of DRM actors in Oromiya Regions on emergency responses. 6. Participation of NIMS focal persons from regional and pilot agencies in federal level NIMS Technical Working Group meetings is helping them to acquire experiences to establish their own MAC. b. Challenges of MAC 16 1. Both at regional and federal levels there are unclear functional overlaps between DRM structures existing before the introduction of NIMS and structures to be reformed or created after NIMS. These perplexities are observed during DRM practices and in the draft operational guides expected to be endorsed in the near future. 2. Addis Ababa City level TMAC is not yet formed and could not be activated during occurrence of incidents. This has limited the formation of multi-agency ICPs at the time of responding to incidents. 3. Dalliance of regional NIMS familiarization events, existence of computing dual humanitarian response structures, and lack of senior officials exposure to study tours have contributed to the retarded integration of NIMS within the DRM structures in Somali Region. ii. Emergency Operation Centers a. Strengths of EOC 1. The interest to integrate and functionalize EOC has been seen at all levels. 2. Pilot regions, DRMFSS and EFPRA have dedicated resources in different forms including staff, office space and facilities. 3. Some funding agencies have shown the interest to finance the establishment of EOC. 4. Federal EW-ECC and Oromiya Regional EOC have started to access early warning information from regional, national and international sources b. Challenges of EOC 1. EOCs are not yet linked with ICPs to provide their comprehensive supports throughout the DRM cycle. 2. EOCs lack some critically required professionals to provide decision making support for different types of incidences. 3. EOCs are not yet established in Somali Region and FEPRA though there are strong efforts to do this in both cases. 4. Zonal and woreda offices lack budget for collection and transferring of early warning information to the higher level. 5. Delay in WoredaNet impended timely early warning data transfer to regions and federal EOCs. iii. Incidence Command System - Incidence Command Posts a. Strengths of ICS/ICP 1. ICS enhanced communication from disaster scene to DRMFSS which resulted in relatively timely emergency response actions. 2. The same way information from incidence scenes to FEPRA created coordinated emergency response by DRM actors in Addis Ababa. 3. ICP created an opportunity for humanitarian actors to speak common language and collaborate on emergency responses. 17 4. The participation of different actors at different levels with pre-defined roles and responsibilities in the ICPs contributed for the promotion of transparency in disaster response. 5. With minimum orientation about specific disaster, ICPs can quickly be activated and functional by involving NIMS trained members. However, readily getting trained people for ICPs is still a challenge. 6. ICPs contribute to DRM professionalism during emergency responses among participants among different profession by representing different sectors. 7. Beyond responding to tangible needs (such as food, medical and cash support) of disaster affected people, ICPs have shown their potential for fulfilling the emotional needs of the target population through psychosocial and providing advises to local administrators on conflict management. b. Challenges of ICS/ICP 1. ICPs have no decision making power over emergency responses due to centralized emergency response management system. 2. Mix of trained and untrained ICP members are assigned together, sometimes this hinders the effectiveness collaboration and reduces DRM professionalism 3. Shortage of facilities and logistics for ICPs on scene reduces effective coordination of emergency responses 4. Segregation of roles and responsibilities of ICP members are not made based on NIMS principles mainly due to the absence of commonly accepted ICS operational guideline. 5. Lack of effective linkage with EOC limits holistic and systematic integration of NIMS within the DRM structures. 4.3 Process and Outcomes NIMS Trainings Once the NIMS components to be integrated within the DRM system of Ethiopia were known after the initial US study tours and consultative meetings as well as NIMS familiarization events, the next major action was building the capacity of DRM actors through two levels of training. The first-level was Trainers of Training (ToT) to develop NIMS master trainers, and the second￾level was cascading NIMS training in the pilot regions, woredas and agencies. Overall, the NIMS program had achieved 90% of its training plans, and about 6.8% of the trainees were female. As indicated in Table 1, below, 59 trainees attended master trainers training (ToT) on at least one of the NIMS components in two rounds during the pilot phase of NIMS Program. From these participants only 23.7% (14) of them completed the trainings in the three components of NIMS can be regarded as fully equipped mater trainers. Over three quarters of ToT participants have attended one or two of the NIMS component trainings. According to the program records (Table 2), 601 individuals have participated in at least one of the three NIMS cascaded trainings, up until September 2013... From these about, 15.1%, 35.4%, 17.6% and 31.9% of the participants were from DRMFSS, Oromiya, FEPRA and Somali respectively. Likewise about 70% attended only and 18.6% participated in two of the trainings. 18 From the cascaded NIMS training participants only 11.8% attended all the three trainings. On top of the trainings on NIMS components, additional skills training were provided to DRMFSS, regional and FEPRA staff in National Comprehensive Emergency Response and Preparedness (CEPRP), All Hazard Incidence Management, Facilitation Skills and Exercise Design. Overall, the NIMS program had achieved 90% of its training plans, and about 6.8% of the trainees were female. Obviously holistic knowledge of and making connections between the NIMS components are fundamental, both during the facilitation of trainings by master trainers and emergency response management by cascade trainees. Specifically attending trainings on the three components of NIMS is very vital for master trainers, emergency response planners and EOC coordinators. However, until the program time covered by this evaluation, from the total number participants of the ToT and cascade trainings about 23.7% and 18.6%, respectively, took part in MAC, EOC and ICS trainings. Table 1: Master trainers participated in ToT by NIMS componets Trainees MAC only EOC only ICS only MAC and EOC only MAC and ICS only EOC and ICS only All MAC EOC and ICS At least or more of the trainings Number 9 22 7 1 6 14 59 Percentage 15.3% 0.0% 37.3% 11.9% 1.7% 10.2% 23.7% 100% Table 2: Number of people participated in NIMS Cascade Trainings Pilot Period Trainees from MAC only EOC only ICS only MAC and EOC only MAC and ICS only EOC and ICS only All MAC EOC and ICS At least or more of the trainings Federal level 27 23 23 7 2 5 4 91 (15.1%) Oromiya 61 55 46 15 9 4 23 213 (35.4%) FEPRA 13 10 32 14 2 5 29 105 (17.6) Somali 58 38 32 16 4 29 15 192 (31.9%) Total 159 126 133 52 17 43 71 601 (100%) Table 3: Percentage of cascade trainings participants by type of training Trainees from Only one training Only two trainings All MAC EOC and ICS MAC only EOC only ICS only Total MAC and EOC only MAC and ICS only EOC and ICS only Total Federal level 29.7% 25.3% 25.3% 80.2% 7.7% 2.2% 5.5% 15.4% 4.4% Oromiya 28.6% 25.8% 21.6% 76.1% 7.0% 4.2% 1.9% 13.1% 10.8% FEPRA 12.4% 9.5% 30.5% 52.4% 13.3% 1.9% 4.8% 20.0% 27.6% Somali 30.2% 19.8% 16.7% 66.7% 8.3% 2.1% 15.1% 25.5% 7.8% Total 26.5% 21.0% 22.1% 69.6% 8.7% 2.8% 7.2% 18.6% 11.8% MAC master trainers training was provided as training of trainers (ToT) to 32 (originally planned to be 30) people from DRMFSS, line ministries, UN agencies, I/NGOs and USFS focal persons by USFS staff in January 2011. The ToT was not only about training on NIMS, but also laying foundation for the creation of cadres for contextualizing NIMS into the Ethiopia’s DRM system and situations. Following the first round training and contextualization of the MAC training 19 manual, the second round master trainers training was given to another group of 24 people (originally planned to be 30) in March 2011 by Ethiopians who attended the first training with support from USFS. This training was given to regional experts drawn from the concerned line bureaus (DPPB/DPPC), federal stakeholders (MoWRIE, MoH and MoA), and FEPRA. The second ToT given by master trainers was conducted to test whether the first master trainers were capable of properly delivering the NIMS training contents to the cascade trainees as well as creating another set of master trainers. Similarly, ToT was given to master trainers on EOC and ICS, each in two rounds. Facilitation skills training were next given to master trainers to enhance local capacity at DRMFSS and pilot regions for cascading NIMS training. Four to five days were allocated for each NIMS component. The trainings were participatory and master trainees from the federal and pilot regions presented their contexts developing their local and hazard specific scenarios which later on were used as inputs in adapting the NIMS training manuals. The ToT training in the NIMS components (MAC, EOC and ICS ) were provided by the trainers from USFS who met the expectations of the master trainers. The master trainers highly appreciated the quality of presentations and skills of the USFS facilitators in delivering training contents with clear and understandable English. They also acknowledged that the four to five days allotted per training were enough. In addition, the training manuals provided both on CDs and paper copies helped the trainees to continually refer about NIMS components both during and after the training. The master trainers from the DRMFSS, regional offices and FEPRA have further cascaded the NIMS components (MAC, EOC and ICS) into the pilot woredas, FEPRA, and UN agencies (in Somali) and NGOs operating in the pilot woredas with financial and technical supports from NIMS. Pilot regions, zones, woredas and FEPRA staff were also trained on all Hazard Incident Management Team (IMT) to capacitate them in managing disasters/incidents in their respective jurisdictions. The cascade training participants from pilot regions, woredas and agencies have recognized the quality and relevance of the training they had. However, cascade trainees in Oromiya and Somali regions indicated that the duration of the training (3 days for MAC, 4 days for EOC, and 5 days for ICS) were very short as compared to the volume of the training manuals and extensiveness of the topics. Some of the participants suggested that the training manuals be translated to local languages. As opposed to this, the participants from pilot woredas of Oromiya and Somali regions told the evaluation team that there was lack of follow-up and support from the regions to institutionalize NIMS components after the cascade trainings. However, they indicated that the training have helped them enhance their understanding and practices in coordination and multi-hazard approach to DRM. The Oromiya DPPC indicated that providing specific follow-up and support to the two pilot woredas out of 246 total woredas in the region was unrealistic. DRM at large, and Early Warning (EW) and emergency response in particular, are cross￾sectional issues that critically need the involvement of all concerned actors. The participation of the sector offices in NIMS cascaded training in DRMFSS in Oromiya region and Addis Ababa 20 (FEPRA) was good but very limited in Somali region. As opposed to this, the regional sector offices that have master trainers were reluctant to collaborate in the process of cascading the NIMS training to regional and woreda level offices, together with DPPB. Delayed familiarization of the NIMS program to the regional officials had its own negative impact in reducing such collaboration among regional bureaus. After the training, both the NIMS master and cascade trainees are technically backing their respective offices in times of incident management in preparing disaster preparedness planning and coordination of efforts. In this course the trainees at federal and regional levels and FEPRA played pivotal roles in the integration and institutionalization of NIMS components in the DRM system. For instance, people leading and writing funding proposals for the establishment of EOC, preparation operational guides, and acting as commanders and members of ICP are those participated in master or cascade trainings. The fourth NIMS component, i.e. the comprehensive emergency preparedness and response planning (CEPRP) preparation training was also given only to master trainees of the DRMFSS, Addis Ababa (FEPRA) and pilot regions. It has not yet reached woreda level experts in both Oromiya and Somali regions. Strength of the NIMS trainings 1. The NIMS trainings were not only skill transfer events but also they served as platform for contextualizing NIMS and later for the preparation of NIMS training manuals and operational guides. 2. Trainings were conducted in a participatory manner to make a multi-way experience sharing and attractive for trainees. 3. Cascade trainees were selected from multiple DRM actors from federal, regional zonal and woreda as well as UN agencies and NGOs having DRM roles in the pilot regions and federal level. This has helped the integration NIMS at different levels. 4. Scenario based exercises during the NIMS training in the context of incidences prevailing or expected to prevail in the pilot area helped participants to link NIMS training with their actual work. Challenges and areas of improvement on NIMS trainings 1. The same participants are not always sent for different rounds and components of NIMS ToT to produce master trainers having attended all the required trainings. 2. Provision of all the trainings for all participants of cascade trainings was difficult to achieve during the pilot phase. However, selection participants for the specific NIMS training based on their current mandate and DRM roles of their institutions have not been taken into account so far. 3. NIMS trainings have built participants capacity. However, when it goes down to woreda level the culture of maintaining institutional memories by documenting training materials has been found to be weak. 4. Trainees from pilot woredas have not proactively brought what they leaned into practice to the required level. So far there was no system that encourages or makes the trainees or their institutions accountable to integrate NIMS in the local DRM practices. 21 5. There is no a follow-up and monitoring mechanism to ensure the application NIMS at woreda or zonal levels. Both federal and regional level DRM actors have no the capacity and the resources to this. 6. The training materials given to the cascade trainees are prepared in English. This limits the readability and application of the manuals during on the job DRM practices. 7. So far there is no formally designed and accepted after training test to check the uptake of NIMS knowledge by both ToT and cascaded training participants. This system would have helped to examine the effectiveness of the trainings and the readiness of the trainees to engage in DRM activities with good understanding on NIMS within DRM system. 4.4 Study Tours 4 4 1 Processes and key outcomes of study tours With the funding from the NIMS program, four rounds of study tours were conducted by senior and middle-level officials and experts from federal ministries, regional bureaus and agencies responsible for leading and coordinating emergency response operations with the GoE. The main purpose of the tours was to get exposure to the application different NIMS components in the US, understand the importance of interagency coordination in disaster management, observe on￾scene operations of ICPs and exchange ideas with US colleagues on disaster management. In the study tours a total of 41 (five women) Ethiopians from MoA, MoH, and MoWRIE; MoFA, Disaster Prevention and Preparedness Food Security program coordination Office of Amhara Region; DPPB of Somali Region; BoA and DPPC of Oromiya Region; and FEPRA from the City of Addis Ababa as well as BoA from Tigray and SNNP regions. In the 2nd, 3rd and 4th round study tours, the participants were drawn only from pilot regions and institutions, including DRMFSS, Oromiya DPPC and FEPRA. In the first round of training, federal state ministers, regional bureau heads and directors were the main participants. In the last two rounds of the tours were dominated by case team leaders and experts from DRMFSS, Oromiya DPPC and FEBRA. The individual participants and the institutions they represented were selected based on the importance of the roles they play in the national DRM system. Due to this nature of the study tours arrangement, the first study tour participants played a greater role in making strategic decisions in terms of identifying NIMS components that are relevant to be adapted to Ethiopia’s DRM context, and in selecting regions and institutions appropriate for piloting NIMS. The study tours were instrumental for the integration of NIMS components in the DRM system with its effect at federal level in general and in pilot agencies and regions in particular. The tours enhanced the knowledge, skills and attitudes of the participants that are contributing for the improvement of the national and regional DRM system at different levels. Participants also recognize the motivational effect the study tours on the participants to improve the national DRM system after seeing new ways doing things in the US. Apart from the repeated participation of certain individuals, limited participants from federal ministries and repeated denial of visas for participants from Somali regions were reported as shortcomings of the study tours. 22 4.4.2 Recommendations from the study tour participants Based on the lessons learned and after each tour the participants collectively made recommendations, with the facilitation support from USFS, on the actions to be taken to enhance the national DRM system. Some of the recommendation was taken as part of the bi-annual NIMS and DRMFSS’s regular implementation plans. These recommendations are grouped into six clusters indicated below. The numbers in the parentheses and in front of the recommendations indicate the rounds of the study tours. The statements in the italics font also describe the status of the implementation of the recommendations organized based on the evaluation results. 1. DRM Policy and Strategy  Refine the draft DRM policy on the basis of lessons learned from the tour to U.S. (1)  Legislation of DRM policy (3) The National Policy and Strategy on Disaster Response Management was endorsed by July 2013. The policy has benefited from NIMS Program in incorporating different principles including multi-agency coordination and multi-hazard focus. However, the document is not legislated yet to institutionalize and develop its implementation guideline. 2. NIMS Program Management  Establish and operationalize NIMS Steering Committee comprising State Ministers and/or Directors of key line ministries, with a clear TOR. (1)  Assign persons as NIMS program coordinator/advisor to be based and work for DRMFSS. This person was expected to provide technical assistance in disaster risk profiling as well) The NIMS Steering Committee was established and acting as oversight body of the program. The committee meets every six months to discuss its progress and provide directives for the next six months period. The placement program coordinator/advisor basing at and working for DRMFSS was not executed. Rather two co-focal persons were assigned from the DRMFSS and a liaison office from USFS. These focal persons have played important roles in planning and implementation of the NIMS Program togther with USFS. 3. Integration and contextualization of NIMS components  Review existing DRM system in Ethiopia and identify and agree which specific and component/components of NIMS can be incorporated in to it. (1)  Further contextualization and customization of MAC, EOC, ICS processes. (3)  Familiarization and Training on MAC, EOC and ICS. 3  Further revitalize the DRMTWG so that it could serve as champion for NIMS as well as working group. (1)  Awareness creation on NIMS components has to be further enhanced to government and private agencies. (4) 23 The DRM system, programs and capacity of Ethiopia was assessed with the support of USFS. The findings of the review contributed for developing the NIMS Program implementation strategies and identification of specific components of NIMS to be incorporated in the national DRM system. Customized NIMS training manuals and operation guides have been developed after contextualization and familiarization efforts. Training manuals and federal level operation guides developed on MAC, EOC and ICS. The training manuals are being widely applied in pilot areas before September 2013, the reference period for this evaluation. The federal level MAC, EOC and ICS operation guides are waiting the approval of DRMFSS. Familiarization of NIMS components have been carried out for head of pilot institutions, regions and woredas, federal ministries as well as member Federal Council of Representatives. This has helped to promote NIMS program components at different levels. However, delayed familiarization action taken in Somali Regions has contributed for dragged integration of the program in the region. Awareness creation efforts have not been widely promoted among private agencies on NIMS components. However, FEPRA has started to provide advice letters on suspected disaster risks and precautions to private agencies in Addis Ababa. 4. Establish and functionalize NIMS  Establish federal EOC with minimum required facility (3)  The two pilot regions and FEPRA should establish EOC/ECC as soon as possible for timely response and sharing of information during an incident(4)  Strengthen relationship between strategic MAC and technical MAC (3)  Study and analyze the possibility of WebEOC (3)  Enhancing the Agency’s involvement in the NIMS component is very crucial (4) DRMFSS and Oromiya DPPC have established and staffed their EOC with minimum required facilities. These facilities are providing situation update services in the DRM system under their jurisdiction. Somali DPPB and FEPRA, as pilot agencies, are making preparations and soliciting funding to establish EOC. SMAC and TMAC relations have shown improvement at federal level and in Oromiya Region. The functions of MAC were mainstreamed in the pre-existing DRM coordinating bodies, especially at federal and in Oromiya at regional level. WebEOC option was not examined so far. However, DRMFSS and Oromiya DPPC are looking for the finalization the Woreda Net System to integrate early warning systems with woredas to timely obtain information. 5. DRM Capacity building  Explore improvement of communication systems in DRMFSS. (3) 24  Strengthening the capacity of DRMFSS in terms of mandate, human resources and structure enables the organization to carry out its responsibility of coordinating and managing disasters in the country (4)  The idea of domestically financed DRM should be operationalized(4) The DRMFSS ECC has been linked to the internet to obtain online meteorological data from international sources. In addition, it obtains electronic information from regions and NMA. At the same time it disseminates regular situation updates electronically to users at different level. FEPRA also has a radio communication system with its frontline workers. The tendency of domestically financing DRM is largely seen in FEPRA by doubling its human resources and dramatically increasing machinery and ambulances in the last two years. Oromiya DPPC is developing a guideline on raising and administering disaster reserve fund. The reserve fund will be established shifting the previous annual contingency budget. 6. Scaling UP and Sustainability of NIMS  Develop a NIMS scale-up plan, including timeframe, methodology, and resource requirement (1)  Develop institutionalization strategy to ensure sustainability and effectiveness including identifying higher learning or research institution for providing training and on-going research-based refinement (1)  Developing Mutual Aid Agreement (3) The NIMS Program has prepared a scale-up plan based on repeated consultation meetings and workshop outputs. However, consolidated scaling-up plan with defined goals, indicators, resource plans and program management structure was not developed and endorsed so far. NIMS program considered institutionalization of NIMS in the BDU training curriculum as one of the strategies. Bahir Dar University is waiting its curriculum review season to do this. 4.5 Program Design and Management 4.5.1 Relevance of Program Design and Management i. NIMS Program Management The highest management body of NIMS program is a steering committee with members from DRMFSS, USFS, USAID, OFDA, FEPRA, Somali DPPB, Oromiya DPPC, MoH, MoWIE and NMA. The steering committee is an oversight body and conducts semi-annual meetings to discuss on program progress and approve next six-month plans. The NIMS steering committee is fed by NIMS-TWG which is a technical body that consists of the Co- focal persons of DRMFSS and focal and alternate focal persons of pilot regions, up to four persons from USFS (including a liaison officer), and directors and case team workers from DRMFSS. This TWG conducts planning and review of NIMS Program activities and presents the outcomes to the steering committee. 25 The main strength of the steering committee is that it provides a platform for the donor and key GoE’s DRM actors joint planning and reviewing of the Program. In addition, the program is not both strategically and technically led by a coordinator external to DRMFSS. This made the GoE and its institutions (DRMS, FEPRA, DPPC and DPPB) feel responsible as owners in achieving the program objectives. This also has helped all the program outputs as a system and tools to be contextualized start functioning within the capacity of the owners of the program. Despite these strengths, the following are found to be the limitations in the program management:  The attendance of State Ministers of federal ministries having stake on NIMS program has been replaced by their delegates during the steering committee meetings. This has an implication to reduce the effort to enforce strategic decisions made by the committee members that would help the integration of NIMS components.  NIMS program detailed implementation plan with associated budget was not clear to the NIMS management body. This has limited the management body to decide on effective allocation of program resources.  Key stakeholders of the DRM system in the country, including UN agencies and NGOs (both local and international), have not been included in the steering committee membership while they are members of the different DRM forums in which NIMS components are being integrated in one way or another.  The NIMS Program monitoring is based on six-month reviews meetings of the steering committee and technical working group. The NIMS Program has continuous activities, but it has no continuous and systematic performance monitoring and follow-up system.  Participation of Somali Region in the steering committee meeting is also lower. As a result, it has the implication to lower the rate of the integration of NIMS into the regional DRM system.  The absence of full or part-time staff dedicated to the implementation and monitoring of NIMS activities across the pilot regions and DRMFSS, has reduced the effectiveness of program implementation. Specifically, this problem will be magnified in the future as NIMS Program is scaled-up to other regions and woredas. ii. NIMS Program Design NIMS Program was designed to be flexible and incorporate lessons obtained in the course of piloting. The design of the program was flexible, starting from the choice of NIMS components and pilot regions/institutions. In addition, the conceptual framework of the program envisioned in four phases was appropriate to promote collaborative partnership based on flexible, responsive and adaptive piloting and integration of NIMS components in the country’s DRM system. The selection of four pilot organizations helped to promote NIMS principles and components at policy and strategic level in different livelihood systems, in urban-rural settings within varied 26 institutional capacity and multiple hazard contexts. These situations have yielded varied lessons to scale out NIMS across regions. The limitations in the design of the program as observed by the evaluation team include:  Lack of clearly defined NIMS monitoring and evaluation framework. The Pilot program lacked program monitoring and evaluation framework with defined indicators to systematically measure and report on outcomes and impacts as well as continuously document lessons. As well, there are no definite structures within the pilot agencies assuming M&E roles of the program. As a result, obtaining organized data on the pilot program was a bit difficult for this evaluation team.  Limited Number of Pilot Woredas. The Program incorporated only two woredas from each pilot region, Oromiya and Somali. As results obtained from this pilot regions pointed out it was very difficult for regions to give due attention to the system being piloted only in two woredas while all the others were being under a different system which was there before the introduction of NIMS. This has also further led to reduction in the resource use efficiency of the Program by holding training only from small number of woredas, while more can be included by adding pilot woredas with marginal increase in the program cost.  Lack of clear program scale up and phase out strategy Though there were discussions on scaling up of NIMS Program to more regions and woredas, there is no clear strategy as to how that would be done. Moreover, there is no clear phase out strategy that would inform the relevant actors in terms of planning for sustainability of the program outcomes. 4.5.2 Coordination and Collaboration among Stakeholders The efficiency in coordination and collaboration of NIMS Program stakeholders is examined by DRMFSS with USFS, and DRMFSS with key federal ministries and regions. The findings of the evaluation with this respect are summarized as follows. DRMFSS with USFS: Better line of consistent coordination and coordination has been observed between DRMFSS and USFS in the implementation NIMS Program. The NIMS focal persons in DRMFSS and the USFS liaison officers jointly prepare program plans and reports and present them to the NIMS Program Steering Committee and Technical Working Group. DRMFSS and regional master trainers are responsible to facilitate training while the USFS officer acts as a backstop. USFS also has been a facilitator for the successfully completed four round of study tours. However, lack of a dedicated office space for NIMS Program and USFS liaison officer in DRMFSS has affected the performance of the link. Usually the focal persons are busy with their regular work and for them it is difficult to perform what they intend to do for NIMS Program with NIMS liaison officer. DRMFSS with Federal Ministries: Federal ministries are part of the NIMS Steering Committee as an oversight body for the Program and members of SMAC group. These ministries were 27 usually represented by their State Ministers in the bi-annual meetings organized to review and approve plans and accomplishments of the NIMS Program. Gradually, due to competing responsibilities, the State Ministers designated directors at the levels. These representatives have limited decision-making power in resource mobilization and strategic decisions in relation to DRM. This situation has definitely reduced the role of NIMS in influencing the coordination and collaboration role of senior government actors in the program implementation. DRMFSS – Pilot Regions and FEPRA: Pilot regions DRM actors (DPPC, DPPB and FEPRA) are part of the NIMS Steering Committee and NIMS TWG, both led by DRMFSS. The regional DRM actors present and share their cases during these two forums meetings. They also take from the federal level forums what they do at home when they return to regions and agencies. This is actually a periodic engagement between federal and regional actors. In the case of Somali region, senior DPPC officials do not usually come to the federal forums formed to manage NIMS program. Apart from this, the coordination and collaboration between DRMFSS and pilot regions has gone well with smooth communication mainly during provision of NIMS training and study tours. Especial the level of coordination and collaboration of DRMFSS with Somali DPPB, Oromiya DPPC and FEPRA in master training and cascade NIMS training went with satisfactory results. However, the regions are not receiving further follow-up and technical supports after the training events from DRMFSS. This loose coordination in monitoring and technical support is mainly due to the absence of full time staff for NIMS both at DRMFSS and in the pilot regions. Secondly, DRMFSS mainly focuses on rural disasters when compared to urban areas, and its coordination with FEPRA is very limited. 28 5. Sustainability The sustainability of NIMS is about the extent to which outcomes and impacts would continue after NIMS Program assistance has come to an end. Assessing sustainability involves evaluating the extent to which there is contextual and policy relevance, availability of sufficient capacity (in human, financial, and institutional terms) to maintain, manage and ensure development results in future by the different DRM stakeholders. The positive contributing factors for sustainability of NIMS:  DRM policy and institutional commitments of the GoE,  Contextual relevance of NIMS to Ethiopia,  Preparation of contextualized NIMS training manuals and operation guides,  Existence of training institute under FEPRA dedicated for human resources development,  Interests of UN agencies and NGOs to participate in NIMS initiatives (e.g. ICPs) and funding the establishment of EOCs,  Availability of resources from some funding agencies and GoE to operationalize EOCs. Challenges and areas improvement for NIMS sustainability:  Absence of institutions dedicated to DRM human resource capacity building at federal or regional level in relation with on the job trainings,  Limited budget allocation for institutional capacity building outside of training from NIMS program,  Absence of fulltime of part time staff responsible within the DRM system at federal and regional level to promote the integration of NIMS,  Lack of a clear scaling up and phase out strategy and commitment,  Existence of parallel DRM structures in Somali Region,  Use of non-standard/uniform terms and structures within DRM system at different levels,  Delayed approval and application of NIMS operation guides, and  Absence of a binding memorandum of understanding among MAC group members. Key Lessons from the NIMS Pilot Program A range of lessons in the area of program implementation and management as well as policy processes have been documented by this evaluation in a way they are useful for ensuring sustainability and scaling up of NIMS. The key lessons are:  The NIMS study tours have played decisive roles in the institutionalization/integration of NIMS components into the DRM system.  Early familiarization of NIMS to key decision makers can play an advocacy and enlightening role for the integration of NIMS into the DRM system at federal and regional level.  The time overlap in the introduction of NIMS with DRM policy development process and institutional reform creates a higher propensity for acceptance and integration of NIMS components. 29  The strong strategic and technical leadership responsibilities of the DRMFSS in the NIMS program have helped the country to explore different options for the integration of NIMS.  Past experiences from DRMFSS and Oromiya DPPC indicate that there are possibilities of obtaining external resources for financing NIMS initiatives beyond the development of DRM system, trainings and visits. Some of these key sustainability factors are described below. DRM Policy Commitment One of the important fertile grounds for the sustainability of NIMS within the Ethiopia’s DRM is that the integration of NIMS components are supported by the newly ratified National Policy and Strategy on Disaster Risk Management. NIMS components can be considered as a best way for bringing most of the aspects of the new DRM policy such as comprehensive disaster risk management system, decentralized DRM, and multi-agency approach for mainstreaming DRM into social and crosscutting issues, mobilizing resources, and information management through EOC. However, the legislation, development of directives and institutionalization of the new policy is the cornerstone to sustain NIMS in the Ethiopia’s DRM system. NIMS has a potential to sustain in Addis Ababa, as the city has a goal to make itself safe from disasters by 2020. In recognition to this FEPRA has incorporated the NIMS principle of multi￾agency and multi-hazard approach in its new legislation and organizational structure. Institutional Commitment NIMS activities such as capacity building, contextualization, recommendations of study tours, bi-annual check-in results and establishment of NIMS structures (such as EOC) are incorporated in the DRMFSS, Oromiya DPPC, Somali DPPB and FEPRA annual operational plans. These are further divided into directorates, case teams and individual staff plans. Based on these plans, the reporting and performance evaluations at different levels are being carried out. This a good sign of the commitment of pilot regions and agencies to sustain NIMS Program. Integration of NIMS Components into the DRM System Currently MAC, EOC and ICS are being integrated in the DRM system at federal level. These components have started to function. Multi-agency and multi-hazard approach in DRM is practiced through MAC structure and activation of ICPs for on scene management of incidences. The different DRM actors are participating in SMAC, TMAC and ICPs. There is also a strong desire among DRM actors including GoE at federal and regional level, UN agencies and NGOs to work through systems and structures functioning using NIMS principles. Contextual relevance of NIMS From the pilot program, NIMS has proven to be working under different DRM contexts including causes of disasters (Human made or natural), socio-economic settings (urban or rural), 30 livelihood systems (farming or pastoral), and size and type of disaster. In this respect, all the pilot regions and agencies with different DRM contexts have shown great interest and commitment to instill NIMS approach in their systems. To sustain NIMS, the manuals for the adapted components have been contextualized to Ethiopia’s DRM system. Human Resource Capacity Having dedicated, skilled and trained human resource is vital for the sustainability of NIMS. In the country human resources fly from one job to the other and from office to office so frequently. Therefore, building dedication and skills as well as retaining staff in the DRM positions are required. So far, i.e. up until September 2013, only about 59 people from the pilot agencies have been trained on at least one of components of NIMS as master trainers. From these only less than a quarter (23.7%) attended the full set of NIMS trainings. This was achieved in three years time through the pilot program. This number of master trainees might be enough for the pilot regions and woredas. However, this could not be enough to scale up the program to new regions and woredas (even within the pilot regions). When it comes to institutionalization of HR capacity building, FEPRA has a training institute that can serve this purpose. Thus, FEPRA can use this facility and its current master trainers to produce new trained personnel and provide refresher training on NIMS as required with reasonable budget. NIMS can be sustainable and scaled up to urban areas if other cities and towns have been further familiarized to NIMS through a federal structure that has strong functional responsibility on urban development. Apart from this, federally or regionally there are no such facilities for HR capacity development on DRM, including NIMS. Thus, this will continue to be a challenge to sustain NIMS Program outcomes unless actions are taken by establishing new or strengthening existing DRM capacity building centers such as Bahir Dar University (BDU). The NIMS Program expected BDU to institutionalize NIMS in its curriculum. This action has not yet been taken by BDU as it is currently waiting for the coming curriculum review period. However, the University has shown the interest to include NIMS in the next cycle of curriculum review as an independent course or chapters of one or more courses. This will somehow guarantee future availability of DRM professional graduates having NIMS training. Scaling-up NIMS The scaling up of NIMS Program to other woredas of pilot and non-pilot regions is vital to sustain NIMS. Unless there is technical and budgetary support to expand and uniformly apply NIMS in the rest of the woredas and regions of the country, it will be less likely to continue to work with NIMS only in the pilot areas. Currently, NIMS Steering Committee and NIMS Technical working groups have the desire to scale-up NIMS in pilot and non-pilot regions and woredas. So far, the focus of the NIMS scaling-up strategy has been identified. However, the plan has not been approved by the GoE and USG. Thus, this will be a threat for the sustainability of NIMS. 31 Financial resources NIMS program came with resources to build human capacity without having other funds required for other institutional capacity building needs. In the past, a few donor agencies and NGOs showed an interest to support the establishment of EOC as part NIMS components. Specifically, DRMFSS and Oromiya DPPC also allocated budget to run the operations of NIMS. This budget allocation by the GoE and donors indicates the desire for sustaining the system. Institutional Support In the past NIMS initiatives have been supported by focal persons having other fulltime responsibilities at federal and regional levels. Although the focal persons have technical competencies, they have no decision making role with regard to the program, including writing official letters to partners. These have limited their roles to regularly monitor the program, including providing supports to regions and woredas. In this way, it was difficult for them to make effective linkages about NIMS at different levels. Absence of binding memorandum of understanding among MAC group members The lead DRM actors which are members of MAC have no binding memorandum of understand and approved terms of reference for their operations. DRMFSS considers the legislation of the new NPS-DRM as a future tool for this. Even after the legislation of this policy there should be a clear terms of reference for MAC groups embedded in the MAC operation guide which is still expected to be approved by DRMFSS. Use of Non-Uniform Terms and DRM Structures In general NIMS is about standardized disaster response management. However, currently, DRM structures that are supposed to mainstream NIMS are being named differently at different level. For instance in Oromiya Region DPP committee is understood as SMAC while DRMTWG is referred as TMAC. The regional DPPC is not yet ready to replace the earlier names with the names that came with the introduction of NIMS. At DRMFSS level TMAC is considered and activated as a sub-set of DRMTWG. On top of this the draft federal MAC operation guide which is expedited to be widely applied has not mentioned about DRMTWG in its suggested DRM architecture. This would make effective communication and application of NIMS very difficult within DRM system. 32 6. Summary of findings and conclusions General The NIMS program has contributed to DRM capacity building through trainings and introduction an effective system for emergency response and management. There are positive improvements in the DRM system of the country both at federal and regional levels where NIMS pilot program reached. Now leaders of DRMFSS, Oromiya DPPC, Somali DPPB and FEPRA have developed a mindset through which they involve all other key stakeholders in the DRM system. Institutional collaborations on DRM issues have been improved due to the creation MAC group especially at federal, Oromiya and Addis Ababa levels. Integration and functionality of NIMS components MAC has been included in the existing DRM architecture at federal level with the participation of relevant ministries, UN agencies and NGOs, and the leadership role of DRMFSS. SMAC group members are State Ministers and executives of UN agencies and NGOs that have decision￾making roles over resources. The role of TMAC is fully assumed by the existing DRM Technical Working Group. TMAC is activated from DRMTWG during emergency times. SMAC made decisions on HRD three times in the past two years based on the reports organized by DRMTWG. The DRMFSS commitment to strengthen the linkage between SMAC and DRTWG the draft federal MAC operation guide has not mentioned the role of this group in the DRM system. In Oromiya Region, the SMAC and TMAC functions are incorporated in the existing DRM structures. With the lead role or DPPC, both groups are meeting on regular basis and discussing emergency issues. In Somali Region, it was not possible to establish MAC groups until the time frame covered by the evaluation, September 2013, due to the delay in the familiarization of NIMS among bureau heads and other senior officials, and unaddressed issue of the parallel DRM structures at regional level. Addis Ababa City has formed SMAC groups under the leadership role of FEPRA. The integration and functionality of NIMS was relatively straightforward because the NIMS introduction coincided with the restructuring and legislation of FEPRA. EOCs have been formed in DRMFSS and Oromiya DPPC. Both centers have started their situation update duties as a regular responsibility. The DRMFSS’s EOC named by EW-ECC has separate three rooms: situation update, coordination and decision-making. The Oromiya EOC has a single room, but people assuming different responsibilities during emergency time have been identified, trained on NIMS and informed. FEPRA has dedicated space for EOC and soliciting funding for its establishment. Apart from this lack of communication facilities and budget for telephone service are hindering a smooth flow of data from regions and woredas to ECO/ECC. In relation with ICS facilities, ICPs were formed to manage on scene emergency responses in different times over the last three years. These include 6 ICPs sites in Addis Ababa for Saudi 33 migrant returnees, Jigjiga and Godey ICPS in Somali, and Yabelo and Koni ICPs in Oromiya. The IPCs have contributed for coordination of emergency responses, provision of psycho-social and conflict management supports and speedy delivery of emergency resources to affected population. It was also found to be instrumental for promoting transparency and accountability within the DRM system, and rapid flow of information between the emergency scene and DRMFSS. These in turn have contributed to effectively save lives and livelihoods of emergency affected population. However, as the system is at the infancy stage there is no appropriate coordination of information for decision-making through EOC facilities. In some cases members and commanders not trained on NIMS were assigned in the ICPs. Moreover, centralized resources management system and limited decision-making power of ICPs are found to be the limiting features of the current DRM system to fully exercise NIMS principles. The integration NIMS at woreda level is not yet started. There is no woreda follow-up and monitoring system at both federal and regional levels. After training, woreda staff raise about NIMS only if ICPs are activated in their areas by DRMFSS and regions to coordinate emergency responses. NIMS/DRM Standardization Over time regions and federal institutions have been using different terms and conceptual formulations for similar issues, topics or groups/structures involved in DRM. Even after the introduction of NIMS the same structures or committees are termed differently in different institutions and times. This practice indeed blurred the chance to exchange message and support one another in times of disaster and capacity building. Finally, it could cause confusion among DRM actors. This situation also negatively affects the sustainability and scalability of NIMS in the DRM system of Ethiopia. In the case of FEPRA, standardization of the DRM system in terms of classifying emergency response machineries and human resources is being practiced. This is helping the organization to ease communication during emergency responses and classify which resources to be used for what size and type of incidence. NIMS Program Design and Management NIMS Program initiated to build DRM capacity of Ethiopia is led by the GoE with its US partners (USAID and USFS). This has created ownership of the Ethiopian institutions at federal and regional levels, both on strategic and technical aspects of the program coordinated by NIMS steering committee and TWG. Nevertheless, lack of clear information on planned program activities and associated budget was indicated as challenge by management body from GoE’s side. The day-to-day activities of the NIMS Program were orchestrated mainly by USFS through its liaison officer from Ethiopia and focal persons assigned by DRMFSS, Oromiya Regional DPPC, Somali Regional DPPC and Addis Ababa City FEPRA. However, since the focal persons have a 34 fulltime job on top of NIMS Program responsibilities, their limited availability and mandates restricted only to technical issues have affected the program performance. In addition, the M&E responsibilities of the program are not fully handled in the DRMFSS to track the records of work progress based on preset plans and indicators. The absence of UN agencies and NGOs, from NIMS Program management structures, while having decisive roles in SMAC and humanitarian actions in the country, is a missed opportunity. These institutions have important roles in the country’s DRM system. They could have played more roles by participating in the NIMS program management. Roles of the Study Tours The study tours organized by USFS were eye openers and motivators for the Ethiopian senior and middle-level officials and experts to develop a good perception and attitude towards NIMS program in the US and bring it to Ethiopia. The tours helped the GoE to identify and develop a strategy to integrate NIMS components in the country’s DRM system. At the end of every study tour, the participants prepared lessons and recommendations to enhance the DRM system in the country. Afterwards the NIMS management system, point persons and USFS exerted their efforts to implement and follow-up issues of recommendations. A total of 41 (five women) Ethiopians participated in the study tours from different line ministries, regional bureaus and agencies. A few of the participants have taken part in two three round of the tours. On the other hand, visa restrictions to some of the proposed study tour participants have created a dreadful sentiment, especially among NIMS Program implementers in Somali Region. Moreover, repeated participation of some people was indicated to reduce the involvement of key line ministries in the study tours. However, the development of criteria for selection study tour participants in the later stage has helped to promote fair distribution in the participation of officials and experts and agencies. NIMS Training NIMS training were organized for two levels of trainees: ToT for master trainers and cascade trainees. The first cohort of master trainers from DRMFSS, key line ministries (MOH, MOWIE), Oromiya DPPC, Somali DPPB, FEPRA, UN agencies and NGOs were trained by USFS. This group of master trainers in their turn trained second cohort of master trainers (Master trainers by master trainers exercises) with a technical support from USFS. These two training occasions and another forum formed from master trainers effectively worked in contextualizing NIMS as per the Ethiopian DRM system and local context scenarios. Based on this, the master trainees produced the Ethiopian version of MAC, EOC and ICS training manual with the support of USFS. The master trainers also cascaded NIMS training to pilot regions, staff and agencies, using the manuals prepared for this purpose. The trainings have has contributed in enhancing DRM capacity of pilot regions and agencies staff. The pilot regions and agencies have master trainers with a capacity to cascade NIMS training independently with minimum technical support. Pilot woreda staff that attended NIMS 35 have also increased their understanding of the NIMS components and their places within the DRM system. However, there were limited a proactive engagement of woreda staff in applying the skills obtained from the trainings. In addition, there is not practice of checking the facilitation capacity of master trainers before they are allowed to train others. So far 59 and 601 people participate in the ToTs and cascade trainings respectively. From these only about 24% of the ToT participants have attended all the three trainings while about 19% of the cascade trainees did the same. Gender and NIMS Program The NIMS program has directly or indirectly promoted gender issues in DRM system. NIMS has created a command and management approach into the existing DRM system which helps to promote the accessibility of vulnerable groups, mainly women and children. On top of this, about 6.8% of participants of NIMS training were women. Of course this could be a good start in a male dominated civil service system. Sustainability of NIMS The policy environment both at federal and regional levels are conducive to continue the DRM initiatives started by NIMS Program. The lessons from NIMS program have contributed for the preparation of the newly ratified DRM policy. The policy is a favorable ground to promote NIMS components in the country. In addition, the recent legislation of FEPRA perfectly goes with NIMS principles of multi-agency and multi-hazard approach. NIMS Pilot Program was able to create limited number of master trainers. BDU is still waiting time for the institutionalization of NIMS into its curriculum. Except FEPRA, other federal and regional DRM agencies have no training centers to incorporate NIMS and provide on the job training for government, humanitarian and private agencies. On top of human resources issue, delayed approval of NIMS operation guides, lack of clear scaling up and phase out strategies, use of non-standard terms and structures, absence of specific structure at federal and regional levels for coordinating NIMS program, and shortage of budget outside of training are the key challenges to sustain NIMS initiatives at all levels. NIMS Visibility Program visibilities can be vital to leave messages and memories in the minds of stakeholders in many programs like NIMS focusing on policy, systems and capacity building. NIM was visible in the past through familiarization workshops and a single brochure which was developed by Federal ECC. We believe NIMS sustainability to be reduced due to inadequate visibility efforts by the programs. 36 Piloting in Somali Region Due to different reasons, the piloting program in Somali Region has not gone along with other regions and DRMFSS. Some of the reasons include late familiarization of NIMS to regional authorities, existence of two parallel humanitarian coordination structures and limited involvement of decision-makers from regions in the NIMS Program Steering Committee meetings as well as low participation of regional representatives in the US study tour. On the other hand, the region showed great interest towards NIMS Program and the staff who attended the training have appreciations to the applicability of the NIMS components in their region. 37 7. Lessons Learned  The NIMS study tours were sources of inspiration for senior leaders and experts to play decisive roles in the institutionalization/integration of NIMS components in the DRM system.  Early familiarization of NIMS to key decision makers can pay an advocacy role for integration of NIMS at federal and regional levels. Pilot agencies started NIMS promotion with familiarization events such as meetings and workshops, and exposure visits to existing outcomes of NIMS have found to be successful in bringing leadership commitment and integration of NIMS within the DRM system.  The time overlap in the introduction of NIMS with DRM policy development process and institutional reform creates a higher propensity for acceptance and integration of NIMS components in a new system. This can be explained by wider integration of NIMS components in FEPRA’s new legislation and organizational structure as well as in the newly ratified National Policy and Strategy on Disaster Response Management (NPS￾DRM).  The strong strategic and technical leadership responsibility of the DRMFSS in the NIMS program has helped the country to explore different options for the integration of NIMS, including in planning and monitoring of NIMS program activities, policy formulation, and institutional restructuring.  NIMs as a system is flexible to be contextualized and applied under wider context where there is decentralized DRM system.  Past experiences from DRMFSS and Oromiya DPPC indicate that there are possibilities of obtaining external resources by drawing partners into NIMS program for financing NIMS initiatives outside of DRM system development, trainings and visits.  Application NIMS lessons for classification and standardization of emergency resources can help for appropriate, coordinated and rapid responses to incidences. This has been seen from the standardized classification of fire fighter personnel and machineries in FEPRA. 38 8. Recommendations The evaluation has captured findings on effectiveness and sustainability of and lessons from NIMS pilot program. Based on this the following recommendations have been put forward for consideration by NIMS program and GoE for strengthening and scaling up the pilot initiative in more coordinated sustainable ways by taking NIMS principles and the GoE DRM policy framework. 1. Integration and functionality of NIMS components within the DRM system 1.1 In all the pilot regions and agencies all the components of NIMS have not been fully integrated and functional due to different circumstance as indicated in this evaluation. Thus, with the participation of senior officials at federal and regional levels and USFS possible solutions should draw for the key challenges based on the findings this studies and existing institutional realities. The outcomes of this process should to apply and integrate them in the next NIMS scaling up plan. 1.2 NIMS components have not been proactively practiced in the pilot woredas for emergency planning and response management. The full integration of NIMS can be realized when the system is adopted and able function up to the lower DRM structures, at woreda level. The next phase of NIMS program should encourage and make woredas accountable to put NIMS components in to practice. 1.3 Further standardization of DRM structures, terms and resources has to be promoted to enhance coordinated and effective communication and actions in the course of NIMS integration and during incidences. 2. NIMS Program Design and Management 2.1 Future NIMS Program should have a clear results framework that indicates expected outcomes, outputs and activities. This framework should have indicators, tagged resources and responsible actors to promote both effectiveness and accountability during the scaling up phase of the program. 2.2 Steering committee and technical working group in the management of NIMS system can continue to be a good oversight structures. However, these cannot support day-to￾day activities of the program together with DRMFSS and regional focal persons as well as USFS liaison officer. Therefore, program coordinator and M&E personnel reporting to both DRMFS and USFS should be embedded in the EWRD of DRMFSS. Depending on budget availability, regions (both piloted and scale up) should have a fulltime or part time staff for making linkages from federal to region and from region to woredas. The regional staff can be embedded in the regional EOC mainly responsible for program M&E responsibility. 2.3 UN agencies and NGOs should be part of the NIMS steering committee to speed-up the NIMS integration and functionality. Their participation in this forum can help bringing resource and knowledge gaps that cannot be supported by NIMS program. 2.4 Different strategies should be followed by the NIMS Program to encourage the participation of state ministers in the NIMS steering committee meetings. For this reason the state ministers should be consulted for their absence from the meetings and possible encouraging and binging solutions must be sought. At the same time 39 important NIMS program issues should be discussed during SMAC meetings in which these senior officials are participating. 2.5 NIMS integration and functionality cannot be fully realized only through systems development and human resources capacity building of the current practice. The next phase of NIMS should take this into account and allocate budget or come up as with a strategy and activities for obtaining other matching funds required for the effective functionality of NIMS components. 2.6 Alternatively, it is recommended to map complementarily of existing programs with NIMS program during the scaling up to match funds that would help the full integration of NIMS. 2.7 EOCWeb was one of the important recommendations highlighted after one of the sturdy tours. The next phase of NIMS should come with resources and technical support to build EOCWeb with the components of capturing and sharing early warning information. This system should be able to expand the existing use of and access to early warning and climate information from international data suppliers and NMA. The EOCWeb should be planned in such a way that it can be linked to the forth coming Woreda Net system. On top of this the EOCWeb should help in storing and sharing institutional memories and lessons of NIMS program implementation. 3. Study tours 3.1 Study tours should be organized in other African or Asian countries that practice emergency response management system to overcome the problem of US visa restrictions. 3.2 The program should further refine its selection criteria for study tour participants to reduce repeated visits of a few individuals and provide a chance for others like from key line ministries. 3.3 The implementation of recommendations after every study tours in the future should be part of NIMS program as well as participant institutions responsibility that are reflected in their operational plans. The achievements towards these recommendations should be monitored by NIMS steering and technical committees. 4. NIMS Trainings 4.1 Priority should be given to ToT trainings to have adequate number of master trainers who have attended all the NIMS trainings at regional level. 4.2 Master trainers should be given a certificate of proof of their capacity to facilitate cascade trainings. For this reason the candidates should be given a test by DRMFSS and USFS after the ToT for all the NIMS trainings. The certificate can be qualifying the candidate for one or more NIMS trainings based on their test scores. 5. Sustainability of NIMS 5.1 NIMS operation guides should be further reviewed to align them with the existing DRM practices, structures and terms. For instance DRMTWG is playing decisive roles in DRMFSS and Oromiya DPPC. DRMFSS still determined to continue with this structure. However, the current version of MAC operation guide does not mention about it. 40 5.2 Promoting visibility of NIMS program will have strong contribution for its effectiveness and sustainability. Mainly the next phase should develop, resource and apply NIMS Program visibility strategy to demonstrate values of NIMS components for the promotion of multi-agency and multi-hazard approach through a decentralized DRM system as per the new DRM policy. Therefore, visibility has to be one of the next key activities of NIMS. 6. Recommendations to DRMFSS and regions 6.1 Clearly, DRMFSS is working hard to legislate the NPS-DRM. However, we would like stress the advantage of finalizing legislation and start putting the NPS-DRM while NIMS Program is scaled up into other regions and woredas. 6.2 Directorates, processes and case teams and staff members of DRMFSS have started to put NIMS components and related activities in their operational plans and reports. To ensure sustainability NIMS these structures and staff members should have incorporated such tasks in their regular terms of references and job description. 6.3 EOCs, specially, at federal level should have professional staff that can support in early warning and emergency response including food market information system, conflict management and livestock. 6.4 Somali Region is a little behind in terms of NIMS integration into regional DRM system. Before putting further time and resources, the region should harmonize the DRM arrangement in the region, based on the legislative power of the regional DPPB. 6.5 Continues human resource capacity development through on the job trainings is vital to effectively undertake DRM activities including the application of NIMS components. In the latter case, FEPRA has a training institute. DRMFSS requires having a facility that can lead DRM system and HR capacity building and provide supports for federal, regional and private agencies in DRM. Alternatively, national universities or training institutions should assume this role with a strong support from the GoE. 6.6 Bahir Dar University should be supported and encouraged by DRMFSS and USFS to incorporate NIMS in its curriculum in the near future. 6.7 Alongside the formation of EOC, FEPRA should work on establishing unified ICPs during emergency situations like the case of Merkato where fire happened in 2013. 41 42 Annexes Annex 1: Evaluation Protocols Protocol 1: KI with State Minster of DRMFSS Federal Level Informants: State Minister of DRMFSS General 1) What are the improvements in DRM system and capacity in preparedness and emergency relief provision of the GoE at federal and regional levels? 2) What are the key factors contributed for these improvements? 3) What was the role of NIMS project in this respect? 4) Have the NIMS pilots works? At what level? 5) What skills and knowledge has been gained from the study tours to further enhance Ethiopian DRM system? Which study tour was better in gaining valuable DRM experience to you and other participants from Federal level? 6) How was the collaboration of DRMFSS with USFS in the project implementation? What would have been differently? 7) What are the positive and negative lessons learned from NIMS to be drawn up on to inform future DRM approach? 8) Would the outcomes and impacts of NIMS sustain after the NIMS project? What are the signs and commitments from GoE’s side? Protocol 2: Federal MAC Strategic (KII) Federal Level Informants: People from DRMFSS (EWRD and EW-ECC) General 1) What are the improvements in DRM system and capacity in preparedness and emergency relief provision of the GoE at regional levels? 2) What are the key factors contributed for this improvements? 43 3) What was the role of NIMS project in this respect? Performance Questions 4) How (and which) NIMS components (MAC strategic, EOC, ICS, and C-EPRP) coherently interface with the existing DRM structures, NGO and GO sectoral task forces and DRM working group in the region? 5) How DRMFSS is coordinating the different activities and stakeholders within MAC groups? 6) At what levels strategic and technical MAC working groups including EOC and ICPs established? At what level and where are they functional or less functional? Why? 7) What skills and knowledge has been gained from the study tours to further enhance Ethiopian DRM system? Which study tour was better in gaining valuable DRM experience to you and other participants from Federal level? 8) How was the study participants identified from the deferent government offices during the different cycles of the tours? 9) What overall views do you have on the achievements of NIMS project vis-a-vis the project objectives and goals? 10) Would the outcomes and impacts of NIMS sustain after the NIMS project? What are the signs and commitments from GoE’s side? Project Design and Management 11) How federal and regional level structures (MoA, DRMFSS, EWRD, DPPC/B) and USFS effectively cooperated and coordinated to harmoniously execute NIMS project components? What aspect of the project went well? What aspects the project went poor? (Relate these questions with NIMS project implementation and management system.) 12) How DRMFSS, DPPC/B and USFS are effectively and harmoniously cooperating and coordinating DRM efforts in the pilot region? (take the concept of multi-hazard in to this questions) Impacts 13) What are the key impacts of the NIMS project in improving the quality and effectiveness of the regional DRM system in the following areas: a. Proactive disaster preparedness and response plan 44 b. Application standard mechanisms in prioritization of critical resources and information coordination c. Focus given to multi-hazard d. Improved emergency responses e. Decentralized decision making f. Resource mobilization g. Information flow (vertical and horizontal) h. Improved gender integration Sustainability 14) What evidences suggest NIMS would sustain in the regional DRM system? (From government, donors and CSO perspectives) ? What is there any concrete plan and commitment to scale up NIMS initiative throughout the region? (Budget, institutional structure, HR) Lessons learned 15) What are the positive and negative lessons learned from NIMS to be drawn up on to inform future DRM approach? 16) Are there any lessons documented for future use on NIMS? Please share them with us. Protocol 3: Regional MAC Strategic (KII) Regional Level Informants: Oromia/Somali: DPPC Commissioner/Bureau Head, V/DPPC Commissioner/ Deputy Bureau Head and EWR Process Owner 1. What are the improvements in DRM system and capacity in preparedness and emergency relief provision of the GoE at regional levels? 2. What are the key factors contributed for this improvements? 3. What was the role of NIMS project in this respect? Performance Questions 4) How DPPC/B is coordinating the different activities and stakeholders within MAC groups? 5) What overall views do you have on the achievements of NIMS project vis-a-vis the project objectives and goals? 45 6) At what levels strategic and technical MAC working groups including EOC and ICPs established? At what level and where are they functional or less functional? Why? Project Design and Management 7) How DPPC/B coordinated with NIMS project, and DRMFSS? How do you judge the effectiveness of this coordination in planning, implementation and monitoring of NIMS project? What would have been differently? 8) In which woredas NIMS project was piloted including training provisioning and establishment of NIMS structure? Which woredas were able to effectively integrate NIMS in their DRM system? Which were not? Why? Impacts 9) What are the key impacts of the NIMS project in improving the quality and effectiveness of the regional DRM system in the following areas: a. Proactive disaster preparedness and response plan b. Application standard mechanisms in prioritization of critical resources and information coordination c. Focus given to multi-hazard d. Improved emergency responses e. Decentralized decision making f. Resource mobilization g. Information flow (vertical and horizontal) h. Improved gender integration Sustainability 10) What evidences suggest NIMS would sustain in the regional DRM system? (from government, donors and CSO perspectives) ? 11) What are the concrete plans and commitments to scale up NIMS initiative throughout the region? (Budget, institutional structure, HR) Lessons learned 12) What are the positive and negative lessons learned from NIMS to be drawn up on to inform future DRM approach? 13) Are there any lessons documented for future use on NIMS? Please share them with us. 46 Protocol 4: Addis Ababa City Administration MAC Strategic (KIIs) Informants: City Administration and AA FEPRA 1) What are the improvements in DRM system and capacity in preparedness and emergency response of the FEPRA? 2) What are the key factors contributed for this improvements? 3) What was the role of NIMS project in this respect? Performance Questions 4) Would you share us what you benefited from the training in relation to enhancing your urban DM? 5) What overall views do you have on the achievements of NIMS project vis-a-vis the project objectives and goals? Project Management 6) How federal and regional level and USFS effectively cooperated and coordinated to harmoniously execute NIMS project components? 7) What aspect of the project went well in the City? What aspects the project went poor? (Relate these questions with NIMS project implementation and management system.) 8) What do you think about what was done by USFS with you and other stakeholders to enahnce urban DM? Impacts 9) What are the key impacts of the NIMS project in improving the quality and effectiveness of the regional DRM system in the following areas: a. Proactive disaster preparedness and response plan b. Application of standard mechanisms in prioritization of critical resources and information coordination c. Focus given to multi-hazard d. Improved emergency responses e. Decentralized decision making f. Resource mobilization g. Information flow (vertical and horizontal) h. Improved gender integration 47 Sustainability 10) What evidences suggest NIMS would sustain in the Administration and FEPRA DRM system? (From government, donors and CSO perspectives)? 11) Is there any concrete plan and commitment to scale up NIMS initiative throughout the City Administration and FEPRA? (Budget, institutional structure, HR) Lessons learned 12) What are the positive and negative lessons learned from NIMS to be drawn up on to inform future DRM approach? 13) Are there any lessons documented for future use or scale up to other cities on NIMS? Protocol 5: (FGD) with study tour participants 1) In how many study tours have you participated? 2) To what extent the study tours conducted in two rounds helped in transferring NIMS skills to the regional DRM at strategic and technical level? What aspects of the DRM operations enhanced after the study tours? 3) To what extent the recommendations from the study tours followed-up and implemented on the ground? 4) Which recommendations were implemented and which were not and at what level? Why? 5) Was the study necessary and important? How? 6) In the study tours you participated which one was more educative and enlighten? 7) Was the skill and knowledge provided in the different study tours you participated in varied in their nature? Protocol 6: Master Trainers (KII) at DRMFSS and USFS level 1 To what level do you feel that the the NIMS training including the training manuals, content covered, duration, approach, and language of the NIMS training were to the needs and standard of the trainers and target offices? 48 2 How were the effectiveness cascaded the trainings to grass roots level? (Discuss within the context or the readion) 3 What were the criteria by which NIMS master trainees selected? 4 What are the mechanisms in place to monitor whether or not the trainings are cascaded to the level required by the project to the woredas level DRM implementers? 5 To what level the training helped promote NIMS in the regional and local Incident management efforts? 6 What opportunities you observe for the NIMS project with regard to the master trainees for it to be cascaded to the grass roots level? 7 What were the constraints you face (problems of whatever kind) that hampered you not to properly deliver NIMS components to Master trainees? 8 Do you feel that this training approach should sustain in DRM capacity building efforts in the future? Protocol 7: FGD for NIMS training participants at woreda level 1. In how many rounds of NIMS trainings have you participated? Would you please describe? 2. What was the criterion for you to be selected for the NIMS training? Do you think you are the appropriate person for the NIMS training? Why? 3. How adequate were the trainings to operationalize ICPs, EOC, and C-EPRP (duration, capacity of the trainers? Do you need extra refresher training to effectively implement NIMS? 4. How relevant were the quality, adequacy and appropriateness of the NIMS manuals (relevance to the operational context, presentation style, language, training setting, approach) 5. What changes has the training brought on your awareness and skill on incident management. 6. What are the positive and negative lessons learned from NIMS to be drawn up on to inform future DRM approach? 49 7. What do you think is the contribution of NIMS project to the woreda in its incident management efforts? Protocol 8: Post training evaluation for training participants at woreda level Please rate by ticking the cell your level of agreement as per the rating scale below (Likert). Behaviors’ evaluation to measure the extent to which the trainees applied the learning. 1 stands for the very low and 5 the very high rate. Rate 1 2 3 4 5 1) Training participants were selected as per the established criteria 2) The trainees have the required level of skill and awareness of NIMS to use it in times of incident management 3) Trainees performance at work place in incident management has improved 4) There is better coordination, collaboration and partnership of woredas level DRM structures with humanitarian agencies working in incident management after NIMS 5) Woredas level LCPs, EOC and C-EPRP structures are established with clearly stated duties and responsibilities to carry out 6) Woredas level LCPs, EOC and C-EPRP structures are properly functioning to the desired level 7) The trainings conducted were adequate enough to operationalize ICPs, EOC, and C-EPRP (duration, capacity of the trainers)? 8) The quality and Relevance of Trainings of NIMS provided (in terms of Topics, selection of participants from relevant positions, language, approach, and manuals prepared) was to the required level. General remark: What part of the training has been applied at the job place? What have improved in your work after the training by on the skills you reveived? 1 Strongly Disagree 2 Disagree 3 Neither Disagree or Agree 4 Agree 5 Strongly Agree 50 Protocol 9: KII with UNOCHA, WFP and NGOs 1. What are the improvements in DRM system and capacity in preparedness planning and emergency response of the GoE at regional/FEPRA levels? 2. What are the key factors contributed for this improvements? 3. What was the role of NIMS project in this respect? 4. What do you know about NIMS project? 5. Has your organization been involved in any of NIMS structures? If yes what was your involvement in the NIMS components and at what level (federal/regional/zonal/woreda)? Lessons learned 6. What are the positive and negative lessons learned from NIMS to be drawn up on to inform future DRM approach? Protocol 10: Bahir Dar University - Disaster Risk Management and Sustainable Development Department (DRMSD) 1) Please tell us some the background about BDU-DRMSD and its current status and contributions in the Federal and regions DRM system? 2) What aspects or NIMS institutionalized in the BDU-DRMSD curriculum? 3) What was the contribution of NIMS project to capacitate BDU-DRMSD in order to strengthen and sustain institutionalization of NIMS curriculum? (Number staff obtained long/short term trainings, technology transfer, logistics support, etc) 4) What values have been added to the Ethiopian DRM system due to the institutionalization of NIMS in to the BDU-DRMSD curriculum? 5) To what extent BDU-DRMSD is effectively coordination with DRM HR (long and short term) development in collaboration with DRMFSS and NIMS groups? 6) How effective was of NIMS project design, management and collaboration to institutionalize NIMS in the BDU-DRMSD curriculum? 7) How is the trend and commitment of BDU to sustain DRMSD masters/graduate training program. 51 Key References Disaster Risk Management and Food Security Sector (DRMFSS). (September 2012). Federal Level Operational Guide for Multi-Agency Coordination. Addis Ababa: Federal Democratic Republic of Ethiopia Ministry of Agriculture. Addis Ababa. Disaster Risk Management and Food Security Sector (DRMFSS). (December 2012). Emergency Operation Centre (EOC) Operational Guide. Addis Ababa: Federal Democratic Republic of Ethiopia Ministry of Agriculture. Addis Ababa. Disaster Risk Management and Food Security Sector (DRMFSS) (December 2013). Draft ICS Operational Guide. Addis Ababa: Federal Democratic Republic of Ethiopia Ministry of Agriculture. Addis Ababa. Addis Ababa City Government Fire and Emergency Prevention and Rescue Agency (FEPRA) (March, 2013). City Level Operational Guide for Multi-Agency Coordination System. Addis Ababa. Addis Ababa City Government Fire and Emergency Prevention and Rescue Agency (FEPRA) (October 2013). Draft ICS Operational Guide . Addis Ababa. Federal Democratic Repoblis of Ethiopia. (July 2013). National Policy and Strategy on Disaster Risk Management. Addis Ababa. Disaster Risk Management and Food Security Sector Partnership of GOE with US government for Capacity Building, Report of the 3rd Study Tour Program to the USA, August 18, 2011. Addis Ababa.