USAID’s Strengthening Earthquake Resilience in Bangladesh Program (SERB) Performance Evaluation Final Report November 2017 This publication was produced independently by Mr. Golam Kabir for Asian Disaster Preparedness Center (ADPC). SERB PERFORMANCE EVALUATION REPORT 1 | P a g e Table of Contents Acronyms 3 Executive Summary 5 I. Background and Context 13 I.A. Seismic Risk in Bangladesh 13 I.B. SERB Goal, Objective and Tasks 13 I.C. SERB Nodal Agencies 14 I.D. SERB Activity Link to USAID/Bangladesh‘s CDCS 14 I.E. SERB Activity Development Hypothesis 15 I.F. Critical Assumptions and Potential Risks 15 II. SERB Components and Tasks 15 III. Evaluation Purpose and Objectives 17 IV. Evaluation Methodology 18 V. SERB Targets and Achievements 18 V. A. SERB Componentwise Targets and Achievements 19 V. A.a. Targets and Achievements – HOPE Component 19 V.A.b. Targets and Achievements – FSCD Component 22 VI. Findings and Recommendations 24 VI.A. SERB Geographic and Institution Targeting Context 24 VI.B. Accomplishments Analysis 26 VI.C. Relevance of SERB Tasks 27 VI.D. Effectiveness of SERB Tasks 35 VI.E. Replicability of Lessons Learned and Best Practices 39 VI.F. Gender Responsiveness in SERB 42 VI.G. Coordination and Collaboration 44 VI.H. SERB Training Institutionalization 45 VI.I. Pool of HOPE/CBDRR Graduates/Instructors (2003-2017) 46 VI.J. Activities Synergistic to SERB 46 VI.K. SERB Outcomes and Impact 48 VI.L. Toward Sustainability 49 VII. SERB Performance Evaluation Recommendations 49 VIII. The Way Forward - Future SERB 51 IX. Conclusion 51 SERB PERFORMANCE EVALUATION REPORT 2 | P a g e Table of Contents X. Annexes and Attachments Annex A Earthquake Zone Map of Bangladesh 53 Annex B Methodology of SERB Performance Evaluation 54 Attachment 1: TOR of Evaluation 58 Attachment 2: List of Documents Reviewed and Information Source 64 Attachment 3: Set of Evaluation Questionnaires 65 Attachment 4: List of Places/Organizations/Respondents Visited/Interviewed 74 Annex C SERB LOA Targets and Achievements by Component 78 Annex D Distribution of Trainees by Hospitals/Institutions 79 Annex E HRA Summary Findings and Recommendations 80 Annex F Distribution of Participants by FSCD Fire Stations 85 Annex G Distribution of the FSCD Volunteers by Cities 86 Annex H Result of Responses 87 Annex I Study Visit Action Plans 92 Annex J List and Short Briefs of Activities Synergistic to SERB 93 Annex K FSCD Tools and Equipment Maintenance Guideline 97 XI. List of Figures Figure 1: Map Showing SERB Location 16 Figure 2: Training Courses Held Quarterly from June 2013 to May 2017 19 XII. List of Tables Table 1: List of Tasks of SERB by Components by Phases 17 Table 2: Summary of Achievements of SERB Tasks (As of 31st May 2017) 19 Table 3: Summary of Achievements of HOPE Component (As of 31st May 2017) 20 Table 4: Distribution of Graduates/Participants by Tasks (As of 31st May 2017) 21 Table 5: Summary of Achievements of FSCD Component (As of 31st May 2017) 23 Table 6: Distribution of FSCD Volunteer Participants by Tasks (As of 31st May 2017) 23 Table 7: SERB Phase-wise LOA Targets of Public Hospitals and Fire Stations 25 Table 8: Results of Quantifiable Questions on Relevance of Tasks under HOPE Component 28 Table 9: Results of Quantifiable Questions on Relevance of Tasks under FSCD Component 34 Table 10: Results of Quantifiable Questions on Effectiveness of Tasks under HOPE Component 36 Table 11: Results of Quantifiable Questions on Effectiveness of Tasks under FSCD Component 38 Table12: Results of Questions on Replicability of Training Courses of HOPE & FSCD Components 39 Table 13: Statistics of Female Graduates, Volunteers, Trainers and Instructors 42 Table 14: Pool of USAID-Supported HOPE/CBDRR Graduates/Instructors in Bangladesh (2003 -2017) 46 Table 15: SERB Performance Evaluation Recommendations 49 Cover Page Photographs (From Left) Group Exercise of Participants of HOPE Course Held in Mymensingh Simulation Scene of CSSR/FF/FA Course Held in Siddique Bazar Fire Station, Dhaka SERB PERFORMANCE EVALUATION REPORT 3 | P a g e Acronyms ADPC Asian Disaster Preparedness Center AOR Agreement Officer‘s Representative BDRCS Bangladesh Red Crescent Society BNBC Bangladesh National Building Code BSMMU Bangabandhu Sheikh Mujib Medical University CADRE Community Action for Disaster Response CDCS Country Development Cooperation Strategy (CDCS) CDMP Comprehensive Disaster Management Program CMCH Chittagong Medical College Hospital (CMCH) COP Chief of Party CSSR Collapsed Structure Search and Rescue DDM Department of Disaster Management DG ECHO Director General for Humanitarian Aid of European Commission DGHS Directorate General of Health Services DIPECHO Disaster Preparedness DG ECHO DNCC Dhaka North City Corporation DRM Disaster Risk Management DSCC Dhaka South City Corporation EMT Emergency Medical Team EOC Emergency Operation Center ERP Emergency Response Plan EWMC East West Medical College, Uttara, Dhaka FA First Aid FDHA USAID/Bangladesh Office of Food, Disaster & Humanitarian Assistance FF Fire Fighting FSCD HQ Fire Service and Civil Defence Headquarters FSCD Fire Service and Civil Defence Directorate GCC Global Climate Change GOB Government of Bangladesh HCPEPR Health Sector Contingency Plan for Earthquake Preparedness and Response HEPRP Hospital Emergency Preparedness and Response Plan HICS Hospital Incident Command System HOPE Hospital Preparedness for Emergencies HOPE-SA HOPE South Asia Project HRA Hospital Risk Assessment HSI Hospital Safety Index IMS Incident Management System IPH Institute of Public Health JICA Japan International Cooperation Agency KGH Kurmitola General Hospital LOA Life-of-Activity MCI Mass Casualty Incidents MCM Mass Casualty Management MODMR Ministry of Disaster Management and Relief MMCH Mymensingh Medical College Hospital MOHFW Ministry of Health and Family Welfare SERB PERFORMANCE EVALUATION REPORT 4 | P a g e Acronyms NCDC Non Communicable Disease Control (NCDC), DGHS NICRH National Institute of Cancer Research and Hospital NIDCH National Institute of Diseases of the Chest and Hospital NIPSOM National Institute for Preventive and Social Medicine NITOR National Institute of Traumatology and Orthopedic Rehabilitation PAHO Pan American Health Organization PEER USAID Program for Enhancement of Emergency Response PICE Potential Injury Creating Event PWD Public Works Department SAR Search and Rescue SCC Sylhet City Corporation SERB USAID‘s Strengthening Earthquake Resilience in Bangladesh SDC Sapporo Dental College, Dhaka ShSMCH Shaheed Suhrawardy Medical College Hospital SKH Sarkari Karmachari Hospital SOMCH Sylhet MAG Osmani Medical College Hospital STAMCH Shaheed Tajuddin Ahmed Medical College Hospital TGH Tangail General Hospital URP Urban Resilience Project WHO World Health Organization SERB PERFORMANCE EVALUATION REPORT 5 | P a g e Executive Summary Introduction Bangladesh manages well in dealing with recurring natural disasters like floods, cyclones and droughts. The country, however, is yet to achieve sufficient preparedness and mitigation systems for responding to earthquakes and related seismic vulnerabilities. Hence, USAID‘s Strengthening Earthquake Resilience in Bangladesh (SERB) activity (June 2013 – May 2018) is designed to address the challenge of insufficient earthquake preparedness and response capacity. SERB is implemented by the Asian Disaster Preparedness Center (ADPC) in partnership agreements with the National Institute for Preventive and Social Medicine (NIPSOM) and Fire Service and Civil Defence Directorate (FSCD) in collaboration with the Directorate General of Health Services (DGHS), the Department of Disaster Management (DDM), participating hospitals, FSCD fire stations and urban volunteers. The overarching objective of SERB is to increase the Government of Bangladesh (GOB) and its communities' disaster management skills and earthquake resilience. The activity comprising of two components, Hospital Preparedness for Emergencies (HOPE) and FSCD, covers male and female health professionals responsible for hospital operations at the 21 selected public hospitals and the urban community volunteers of 21 FSCD fire stations, located in the cities of Chittagong, Dhaka, Gazipur, Manikganj, Mymensingh, Narayanganj, Rangpur, Sylhet and Tangail. Performance Evaluation ADPC commissioned a performance evaluation of SERB by an independent development professional. The evaluation is focused on the performance of SERB from its beginning until May 2017 by examining its implementation, inputs, outputs and outcomes. To assess the relevance, effectiveness and replicability of SERB tasks including training courses, simulations, search and rescue (SAR) tools and equipment, hospital risk assessments, emergency response plans, gender responsiveness and coordination in responding to unplanned events, specifically earthquakes and also fires, building collapse, road accidents, and other emergencies. SERB Accomplishments SERB is on track in meeting its targets and deliverables planned for the last four years from June 2013 to May 2017. ADPC, with the support and cooperation of the GOB implementing and collaborating partners, participating institutions and concerned stakeholders, has successfully implemented almost all tasks which included: basic and specialized training courses on health facility risk management, and SAR; incident management system (IMS); conduct hospital risk assessments (HRAs); development of hospital emergency response plans (ERPs); study visit to regional country; SAR simulation exercises; and delivery of light SAR equipment to FSCD. Highlights of accomplishments are as below:  Over the last four years (June 2013 to May 2017) SERB has trained a pool of 567 medical and support staffs, representing 39% female and 61% male, from 16 public hospitals and 1,850 FSCD urban volunteers, representing 23% female and 77% male, from 17 fire stations selected from the six target cities of SERB.  SERB conducted HRAs in 16 hospitals in coordination with DGHS, NIPSOM and participating hospitals. The HRAs were preceded by the HRA planning workshops participated by the hospital management SERB PERFORMANCE EVALUATION REPORT 6 | P a g e and staff. In the planning workshops, the Hospital Safety Index (HSI) comprising of 151 indicators, developed by the World Health Organization and Pan-American Health Organization (WHO–PAHO)1, was introduced to the participants in order to enable them to conduct their own hospital‘s risk assessment. During the workshop, participants attended group works to better understand the structural, non-structural, and emergency and disaster management components of the HSI.  Findings and recommendations of HRAs have been presented by SERB at the national dissemination meeting held in May 2017. The meeting was attended by the SERB funding agency, nodal agencies, implementing partners, hospital management and graduates, bilateral donors and SERB stakeholders including the MoHFW State Minister and Secretary, DGHS Director General and senior officials, USAID/Bangladesh Office of Food, Disaster and Humanitarian Assistance Director and senior officials, ADPC Executive Director and senior staff.  As part of disaster preparedness of the hospitals, SERB developed all 7 ERPs planned under Phases 1 and 2 and finalized them through validation exercises and meetings participated by the hospital management, HOPE graduates and SERB staff. In developing the ERPs, SERB has drawn on many sources for information, including the GoB Disaster Management Act 2012, Standing Orders on Disasters, DGHS‘s Post Earthquake Comprehensive Health Care Action Plan for Urban Cities and the findings of HRAs conducted by SERB.  A team of SERB staff, HOPE instructors and graduates, FSCD instructors and NIPSOM staff made a study visit to Kathmandu, Nepal in March 2017. The aim of the visit was to build participants‘ capacities on hospital preparedness and response through observation of related best practices in Nepal. It also aimed to foster regional cooperation among South Asian Association for Regional Cooperation (SAARC) countries. The study visit included meetings and consultations with various government agencies, hospitals/health facility personnel and other stakeholders related to managing emergency health facilities, rescue operations, and allocating resources in disaster affected areas.  A two-day national level introductory workshop on IMS which aims at a common approach for managing a disaster response or incidents was held in March 2017. This was a high-level workshop with participation of the Minister for Disaster Management and Relief, Secretary of Ministry of Disaster Management & Relief (MOMDR), Ambassador of US Mission to Bangladesh, Director General, Department of Disaster Management (DDM), Director General, FSCD, other high level government officials, representatives from USAID/OFDA regional office, United States Forest Service (USFS), USAID Bangladesh, UN agencies, diplomatic offices, Bangladesh Armed Forces Division (AFD), Police, FSCD, international and national NGOs, Bangladesh Boys Scout, Girls Guide and other agencies.  Achievements of 13 simulations attended by 650 volunteers exceeded the target of 12 SAR simulation exercises for a total of 600 volunteers under the FSCD component. SERB facilitated these simulations with the assistance of FSCD fire fighters and instructors for the urban community volunteers to enable them to respond effectively and efficiently to victims‘ needs during major natural disasters. About 80% of the total simulation participants have completed the combined training course in collapsed structure search and rescue, fire-fighting and first aid (CSSR/FF/FA) conducted by SERB. 1 Source: Hospital Safety Index Guide for Evaluators, Second Edition, World Health Organization, 2015 SERB PERFORMANCE EVALUATION REPORT 7 | P a g e  SERB delivered 171 sets of light SAR tools and equipment to the FSCD fire stations in the six cities under Phases 1 and 2. The SAR tools and equipment provided by SERB are for light duty and complement the stock of heavy SAR equipment received from the government and other donors.  Of the total staff of the 21 SERB target hospitals under SERB, an estimated 40% are female. The HOPE￾B, HOPE-T and HICS graduates, HRA participants, and study visit team members represent 30% female. The total 20,762 FSCD urban community volunteers in the 21 SERB target fire stations have 24% female and the total female volunteers trained in CSSR/FF/FA training and SAR simulations have 23% female.  SERB is making progress in addressing the challenges and constraints associated with gender responsiveness which include: gender differentiation; vulnerability of gender; gender marginalized in emergency response; and gender inequity in disaster preparedness and response with respect to MCI. Evaluation Findings Findings of the evaluation show that the tasks of SERB are highly relevant and effective in MCM and therefore have potential for replication. SERB is dedicated for the public institutions and has successfully established strong working relationships with them at the field and national levels. The field level collaboration is found to be effective in organizing the training and simulation events fruitfully and disseminating the tasks of SERB widely. Highlights of evaluation findings are as below.  SERB and its GOB implementing partners, NIPSOM and FSCD, have collaboratively been able to manage the potential challenges such as reluctance of medical professionals to participate in training courses. SERB did a daunting task in maintaining the training schedules uniform throughout the activity period. This has been accomplished because of timely preparation, proper planning, and effective implementation. All these accounts for skills and experiences of the staff involved despite the challenges of getting confirmation of the proposed schedules with the participants such as the hospitals and concerned government authorities.  SERB is implemented in the selected seismic zones and focused on earthquake preparedness and response capacities of selected public health facilities and FSCD fire stations. Almost two-thirds of the public hospitals and fire stations (42 out of 65) have been selected by SERB in the nine program cities. There are areas beyond the current nine cities including Dinajpur, Bogra, Bandarban, Khagrachari, Rangamati and Cox‘s Bazar affected by one or more of the natural and human-made hazards, such as, cyclone, storm surge, tornadoes, flash floods, landslides, cold wave, road accidents, and fire. Most recently the regions of Chittagong Hill Tracts, Chittagong and Sylhet frequently devastated by landslides. This is becoming a growing concern of the host government and international communities.  Notably, the positive feedback on the relevance and effectiveness of the SERB-conducted training courses have good prospect for adoption and replication in the country. Finally, the SERB-conducted training courses seem to have worked well. The adoption and replication of both HOPE and CSSR training can be done in the disaster prone areas of the country to include types of hazards beyond earthquake. In the event that there will be no more donor funding, NIPSOM needs to plan in consultation with ADPC on how to integrate the course into their existing system. SERB PERFORMANCE EVALUATION REPORT 8 | P a g e  As compared to the scale of disasters faced, the progress and achievements so far have been not enough quantitatively in order to effectively reduce the risk of the population of the cities significantly and to trigger a self-sustained process. So, there is still the need to strengthen self-sustained process of high standard and high quality instruction and development by consolidating the foundation of SERB process and to ingrain it with the national program and efforts.  SERB‘s training in HOPE/HICS and CSSR/FF/FA has generated considerable interest in the private sector - hospitals, educational institutions, industries and factories. It is, therefore, worth exploring private sector investments in this humanitarian effort as part of their Corporate Social Responsibilities (CSR).  Results of HRAs about the overall safety index calculated for all three components of the 16 hospitals covered by SERB show that 6 hospitals are potentially at risk and 10 hospitals are unlikely to function during and after emergencies and disasters. The former category of hospitals need short term intervention measures while the latter category of hospitals need urgent intervention measures for the safety of patients and hospital staff, and the hospitals‘ ability to function during and after emergencies or disasters. In addition, HRAs are the basis of developing ERPs which aims at emergency operation plans of the hospitals.  ERPs developed By SERB for hospitals are found to be useful in the activation of hospital incident command system in responding to incidents such as, gas tank explosions in a ship breaking factory in Chittagong, bomb explosions in Sylhet, road accidents on the airport road at Dhaka. The hospitals involved in managing the incidents are Chittagong Medical College Hospital (CMCH), Sylhet MAG Osmani Medical College Hospital (SOMCH) and Kurmitola General Hospital (KGH). The evaluation gathered about the instances that some hospitals responded to unplanned events such as, fires, building collapse or road accidents but those were not documented by the responders nor by the hospitals.  SERB-organized study visit to Nepal was not only the learning opportunity to understand the practical application of hospital preparedness and hospital incident command system but prompted development of action plans by the participants for their own organizations. It also gave participants the chance to establish a network between hospital administrators, which will help foster future regional cooperation between the SAARC member states, Bangladesh and Nepal. Thus, cross visits to countries, organizations and communities have potential for replication.  Both HOPE-B and HICS courses trained a total of 515 (39% female and 61% male) graduates including doctors, nurses, technicians and support staff. The distribution of these 515 graduates by professional categories show that 66% (340) are doctors, 26% (132) are nurses and 8% (43) are support and technical staff. The latter two respondent categories found the courses tough to grasp due to their technical contents. They need the courses that can be easily understood at their levels. There are also desire from the high level officials of DGHS, DDM and NIPSOM and heads of the hospitals to arrange short courses for them, preferably a shorter version of HOPE-B. These high level officials and hospital heads and senior administration usually cannot manage time to attend the week-long courses. The evaluation finds it important for the high level officials of the decision making organizations and the hospital heads and senior administration to have a practical knowledge of the dynamics of MCM.  IMS aims a systematic, proactive approach to guide all levels of government departments and agencies, nongovernmental organizations, critical infrastructure owners and operators, private sector and all other organizations. All these organizations assume a role in emergency management to work together SERB PERFORMANCE EVALUATION REPORT 9 | P a g e seamlessly and manage incidents involving all threats and hazards—regardless of cause, size, location, or complexity—in order to reduce loss of life, property and harm to the environment. The two-day introductory national workshop on IMS held in March 2017 has successfully drawn on the expertise of the GOB senior level decision makers, head of diplomatic missions, UN agencies and key stakeholders.  Of the total 20,762 FSCD volunteers in 21 target fire stations CSSR/FF/FA course covered 1,200 volunteers (about 6%) under Phases 1 and 2 and additional 160 volunteers are planned under Phase 3. The simulation exercises covered 650 volunteers (less than 3%) of the total volunteers. Realizing the importance of the trainings and simulations offered by SERB, almost all respondents of the evaluation have expressed their interest for additional refreshers and simulation exercises to be conducted annually.  SERB delivered 171 sets of light SAR tools and equipment to the fire stations in these cities. The SAR tools and equipment provided by SERB are for light duty and complement the stock of heavy SAR equipment received from the government and other donors. There is a desire from volunteers and a need for more equipment for fire stations.  An important rationale mentioned by most respondents of the evaluation about the relevance of the SAR training and simulation conducted by SERB was that it covers the MCM which is a prime and massive responsibility of the FSCD and its community volunteers. The interviewees gave the highest rating to the question on relevance of the CSSR/FF/FA training course topics, materials, presentations, etc. The FSCD volunteers endorsed all 16 CSSR/FF/FA lessons as being important but they need to be updated and practiced on a regular basis, maybe twice in a year. The training course and simulation are a direct benefit to the participating fire stations and community volunteers in responding to the MCIs.  In urban settings in Bangladesh the pool of trained FSCD community volunteers is invariably a valuable resource in emergency response. Yet the volunteers provide these services voluntarily, there was no direct financial incentive for them. These youths have developed a self-esteem and recognition in the community which they value significantly. Interestingly, most of the fire stations have no system of tracking the drop out volunteers. Some stations lack up-to-date register of volunteers. This will be a serious problem for the concerned fire stations as they would not know the exact number of volunteers should there be a need for response. However, there are promising solutions as presented by the volunteers. Examples of self-initiation by the volunteers in humanitarian assistance provide the government and community a serious learning include: volunteers‘ engagement in community development; volunteers maintaining list of volunteers; and volunteers keeping regular contact with other volunteers.  Yet the number of female trainees/participants of the hospitals and fire stations are close to their respective institutions‘ overall levels of female members; the ratio between female and male HOPE/HICS graduates and FSCD volunteers are not the right balance. SERB has made it a point in advocating and encouraging more female participation in all its tasks. The target of 30%, with exception of 50% in a few cases set by the project for the training courses and simulation exercises, seems low. The rationale being, there is still a need to sensitize decision-makers regarding the importance of empowering female staff and encourage them to participate in emergency response programs. The respondents suggested that the target be set at a higher level (50% - 60%).  As far as the sustainability of the tasks of SERB is concerned, it depends on the tasks‘ relevance, effectiveness, replicability and institutionalization. A deeper analysis of the relevance, effectiveness and replicability parameters show a positive result that SERB will eventually affect overall sustainability. Key SERB PERFORMANCE EVALUATION REPORT 10 | P a g e contributing factors to the sustainability of SERB are: institutionalization of SERB training courses; capitalization of resource pool of HOPE/HICS graduates and FSCD urban community volunteers; proper maintenance and utilization of SAR tools and equipment; GoB endorsement of HRA findings and recommendations; GoB provision of funding support to continue the tasks of SERB (i.e., HOPE/HICS courses, HRA/ERP and FSCD components including scaling up and scaling out to other areas in the country.  SERB, under its HOPE component, targets to build the MCM capacities of the hospitals through integration of training, risk assessment, and emergency response planning. This integrated approach shows positive results as hospitals reporting that they have gained knowledge and skills and established systems to respond to emergencies, especially MCIs. Likewise, the package of CSSR/FF/FA training, SAR simulation and light SAR tools and equipment are found to be essential to an emergency and its potential humanitarian impact. These packaging of interventions that are much needed have potential for replication. Evaluation Recommendations A total of 26 recommendations have been formulated by the evaluation for the remaining life of the SERB as well as way forward when SERB ends. The evaluation recommends for advocacy for scaling up and institutionalization of SERB training programs by the public and private sectors, establishment of network of trainees, increased female participation in MCM training program and enhanced role of female in MCM. The areas of recommendations are grouped as Adoption and Replication, Follow-up Actions, Gender Responsiveness, Institutionalization and Sustainability under two categories of recommendations: Recommendations for implementation within the remaining life of SERB (Category A); and Recommendations for any follow-up activities (Category B). Key recommendations by categories include: Recommendation Groups Recommendations Recommendation Categories Adoption and Replication If there to be a future activity like SERB it is recommended that, new cities, namely, Dinajpur, Bogra, Bandarban, Khagrachari, Rangamati and Cox’s Bazar be brought under the activity as they are affected by one or more of the natural and human-made hazards, such as, cyclone, storm surge, tornadoes, flash floods, landslides, cold wave, road accidents, and fire. The focus of the activity shall stretch beyond earthquake but multi-hazard depending on the vulnerability of the cities to the types of hazards for mass casualty management. B Given the growing concern on the impact of landslides, it is recommended that the three hill districts - Bandarban, Khagrachari and Rangamati be selected on a priority basis for MCM adopting the SERB model. Both A & B The HOPE/HICS graduates being employees of the government health facilities are subject to move from one health facility to another. So, to reduce potential loss of skilled staffs in case of an emergency, a database and network shall be established by NIPSOM to track the HOPE/HICS graduates who leave their posts. The network shall be fully functional in tracking the HOPE/HICS graduates in MCM and there shall be understanding among DGHS, NIPSOM and participating hospitals that the rotated staffs are brought back to their previous positions for emergency response. Both A & B Compared to the need for training for the FSCD urban community volunteers, SAR refreshers shall be offered at least once a year and simulations shall be conducted at least in every six months to the volunteers. Both A & B SERB PERFORMANCE EVALUATION REPORT 11 | P a g e Recommendation Groups Recommendations Recommendation Categories The training course shall be based on the target participants’ skills, instincts and abilities to understand and adapt and their availability to attend. The suggested training course for the nurses and support staff shall be a revised version of the HOPE-B course, less technical and according to their level, and shorter courses and orientation on HOPE/HICS for the high level decision makers and hospital heads. Both A & B For the safety of patients and hospital staff, and the hospitals’ ability to function during and after emergencies or disasters, the concerned hospitals shall explore funding for implementation of the HRA recommended actions pertaining to the structural, nonstructural and functional elements. (a) Hospitals shall:  Improve, enhance and automate the nonstructural elements which include systems of: electricity and lighting in critical areas; telecommunications; water supply; fire protection; waste management; fuel reserve; medical gases; HVAC; and shelving of furnishings and equipment; safety and security of staff and patient; and medical and laboratory equipment and supplies used for diagnosis and treatment..  Develop and implement contingency plans that are required for certain critical events and essential services pertaining to maintenance and inspection, fire safety and fire emergency, flood management, chemical spills management, power failure, hospital evacuation, psychosocial support, hospital business continuity, and personal health and critical incident stress.  Strengthening of the functional capacity of the hospital as a priority action of the hospital in the immediate time for effective responses to the emergencies or disasters. (b) DGHS shall:  Undertake proper retrofitting and remodeling of the structures as early as possible, especially those hospital buildings’ are of concrete frame structures and the overall safety level of structure was found to be average.  Conduct detailed structural assessments of those hospitals assessed to be of low safety level by HRA and take mitigation measures as per assessments.  Establish a National Emergency Medical Team (EMT) based on WHO Classification and Minimum Standards for National Medical Teams in sudden onset disasters. Both A & B If there to be a future MCM activity it is recommended that the SAR training shall include lessons, like, gender sensitiveness in disaster response, incident command system, and maintenance of SAR tools and equipment be taught to develop a pool of SAR trainees. B FSCD shall: (i) advocate for establishment of linkages between urban community volunteers and the national cadet core, ansar troops (para militia), village defense police (VDP), etc.; (ii) organize SAR refreshers and simulations for the volunteers; and (iii) provide light SAR tools and first aid kit to the volunteers. Both A & B All hospitals shall have ERPs developed and for that they shall first conduct HRA. The ERP shall serve as a living document containing reference tool for hospital incident management system for responding to mass casualty incidents (MCI). Both A & B SERB PERFORMANCE EVALUATION REPORT 12 | P a g e Recommendation Groups Recommendations Recommendation Categories FSCD shall streamline the self-initiation community services of the volunteers as they are found to be innovative and have the potential for adoption by the fire stations. Both A & B In any future MCM program the evaluation recommends adoption of the integrated intervention approach of SERB which includes development of human resources and provision of tools and equipment to respond to MCI. B Follow-up Actions Hospitals shall keep the ERP up-to-date and fully functional. Hospitals shall organize orientation on the ERP for the new committee members and staff so that they are aware of and ready for responding to mass casualty incidents (MCI). Both A and B SERB shall conduct in-depth analysis of the usefulness and benefit of study visits of the program staff, HOPE instructors and graduates, FSCD instructors and NIPSOM staff to the health facilities and emergency MCM activities within and outside the country in the region. Since executing study visits are expensive, before advocating for them, the program shall determine the feasibility including expected result of the proposed study visit. B Given the importance of systematically maintaining the information on the effectiveness in responding to unplanned events on a regular basis, the hospital management, specifically the casualty department, shall establish systematic collection of information and documentation. Both A and B Gender Responsiveness SERB shall initiate advocacy and awareness programs to motivate more female members to participate. Since this is a cultural matter in the country – a policy supporting women to take more active participation in activities like SERB needs to be promoted widely. All efforts need to be made to ensure that a sufficient gender balance is maintained. Both A & B SERB shall coordinate with hospitals to engage women members in active dialogue with hospital management and staff in hospital emergency response plans. Hospitals with the focus on the gender equity shall assign appropriate number of women officials and staff in the hospital incident management team (IMT) referred to in the ERPs. Both A & B Since it is very important having more female participation in the program to cater for the needs of female disaster survivors, women, girls and boys shall be empowered and capacitated so in the event of emergencies or disasters, they will be prepared and ready to provide the needed assistance. Concerned authorities of the hospitals and FSCD shall engage male staffs in the discussion of addressing gender violence during emergency response. Both A & B Institutionalization Both NIPSOM and FSCD shall prioritize to adopt the training courses they have implemented under the external project assistance into their respective institutional framework. Encourage and influence both public and private sectors to sponsor these training courses to develop disaster resilient hospitals. For example, adopt course modules of SERB training, as appropriate, in the post graduate academic curriculum. Both A & B FSCD shall drive for achieving development of 100% of the target 62,000 volunteers. If required, FSCD shall explore international donor funding opportunities for development of the volunteers. B Sustainability Based on the factors of sustainability, discussed above, following are sustainability related recommendations for SERB.  Advocacy for scaling up and institutionalization of SERB training programs such as basic and refreshers of HOPE, HICS, and Both A and B SERB PERFORMANCE EVALUATION REPORT 13 | P a g e Recommendation Groups Recommendations Recommendation Categories CSSR/FF/FA courses by the public and private sectors for MCM.  Recognition of volunteers by the government for their contribution in humanitarian assistance service to inspire them to stay in volunteerism. The concerned organizations, thus, shall require institutionalization of the training received from SERB. The HOPE/HICS graduates, for example, can be absorbed in the proposed EMT. The volunteers, for example, can be absorbed as Fire & Rescue volunteers by the FSCD.  Maintenance of the current level of quality and routine maintenance of SAR equipment with engagement of the trained volunteers.  Identification of an appropriate GOB institution in consultation with the DGHS to establish and own a fully functional database and networking of HOPE graduates and instructors.  GoB provision of funding support to continue SERB tasks (i.e., HOPE/HICS courses, HRA/ERP and FSCD components) including scaling up and scaling out to other areas in the country. Conclusion Resilient health systems are a popular point today, which means that the health system can sustain during any type of emergencies, such as outbreaks of Ebola and Zika viruses, earthquakes, floods, etc. If the hospitals are not prepared to save people affected by a disaster, if the hospitals are not prepared to provide treatment and give people the right care at the right time, then they will die and this is a disaster in itself. This is of critical value that people and the government needs to consider. As far as possible, ensure that the works of SERB is sustainable – this could be continuation of services or lasting changes in lives of the trained personnel in hospitals and FSCD volunteers. As far as possible, ensure that expertise and momentum for change in the country is not lost. Finally, respondents of the evaluation reported that the benefits derived from the interventions of SERB have far exceeded their expectation and they are, however, under the impression that there still remain opportunities for expansion and replication and room for improvement. The potential area of replication is in the pre- and post-emergency context and in areas vulnerable to earthquake, fires, road traffic accidents, landslides and building collapse which are subject to mass casualties. SERB PERFORMANCE EVALUATION REPORT 14 | P a g e Detailed Report I. Background and Context I.A. Seismic Risk in Bangladesh Bangladesh is situated in a seismically active area. Historically, earthquakes in the magnitude 6.0 - 7.0 range have been experienced in Bangladesh. Bangladesh is divided into four seismic zones in terms of intensity of earthquakes: Zone 4 – Very High Intensity Area; Zone 3 – High Intensity Areas; Zone 2 – Moderate Intensity Area and Zone 1 – Low Intensity Areas. Refer to the Map on Seismic/Earthquake Zones of Bangladesh provided at Annex A. Key facts about earthquake risk in Bangladesh are as below.  Earthquake risk in Bangladesh is very high because the country is at the nexus of two major tectonic elements of the ongoing Indian-Eurasian plate collision.  Several major active faults also inferred in the country which are able to generate large earthquakes, even over 8.0 on Richter scale2.  The Great Indian Earthquake in 1897 with the magnitude of 8.4 on the Richter scale caused severe damages to life and properties in Bangladesh.  Since 2001, around 516 earthquakes shook the country, and according to the seismologist these minor tremors are indicators of the possibility of much more earthquake in Bangladesh.  The April 2015 earthquake in Nepal felt in Bangladesh resulted in damages and injuries, although the epicenter was more than 400 kilometers away.  The powerful 6.9 magnitude earthquake in April 2016 in Myanmar, about 420 kilometers east of Dhaka city, left more than 60 injured in Bangladesh and thousands in shock as they rushed out of buildings in panic.  A study3 in 2009 shows that in the event of an earthquake over 7.5 Richter scale on the Madhupur fault line, colossal damage will occur, with an estimated 72,000 commercial and residential buildings likely to collapse.  Earthquakes experienced in Bangladesh typically caused collapse of buildings, injuries, may cause fires, and may produce floods from collapsed dams and land or mud slides. I.B. SERB Goal, Objective and Tasks USAID‘s Strengthening Earthquake Resilience in Bangladesh (SERB) activity (June 2013 – May 2018) aims at improved GOB and Bangladeshi communities‘ disaster response skills while simultaneously increasing earthquake resilience through the adaptation of technologies and enhancement of the capacity of hospital personnel, fire service, civil defense authority, and related institutions. SERB is implemented under a grant agreement of approximately $1.75 million in development assistance with the Asian Disaster Preparedness (ADPC), Thailand through partnership agreements with the Government of Bangladesh (GOB) National 2 Source: CDMP, 2009 3 CDMP commissioned study conducted by ADPC in 2009 SERB PERFORMANCE EVALUATION REPORT 15 | P a g e Institute for Preventive and Social Medicine (NIPSOM) and the Fire Service and Civil Defense Directorate (FSCD). SERB comprises of two components: Hospital Preparedness for Emergencies (HOPE Component); and Capacity Building of FSCD and its volunteers (FSCD Component) with the objective to increase the capacity of the Government of Bangladesh to respond to and manage natural disasters. The geographic coverage of the activity includes nine cities - Chittagong, Dhaka, Gazipur, Manikganj, Mymensingh, Narayanganj, Rangpur, Sylhet and Tangail. Out of the nine target cities Mymensingh and Sylhet are from Zone 4 – Very High Intensity Area, Chittagong and Rangpur are from Zone 3 – High Intensity Area and Dhaka, Gazipur, Manikganj, Narayanganj and Tangail are from Zone 2 – Moderate Intensity Area. SERB under its hospital preparedness for emergencies (HOPE) component implements: basic and specialized training courses on health facility risk management; training for instructors; capacity building of hospital command teams to command, coordinate and control the emergency incidents including assessment of hospital risks and safety levels; and development of hospital emergency response plans (ERPs) including incident management system (IMS) introductory workshop for policy makers; and IMS resource mapping. Tasks of the FSCD component of SERB includes: delivery of light search and rescue (SAR) equipment and tools for FSCD volunteers for use in the aftermath of an earthquake; training to the urban community volunteers on Collapsed Structure Search and Rescue (CSSR), Fire Fighting (FF) and First Aid (FA); and carry out SAR simulations for the volunteers. I.C. SERB Nodal Agencies ADPC works through two implementing partners, NIPSOM and FSCD, and collaborates with the Directorate General of Health Services (DGHS) of the Ministry of Health and Family Welfare (MOHFW) and Department of Disaster Management (DDM) of the Ministry of Disaster Management and Relief. ADPC works closely with these agencies to get their support, guidance in the implementation of tasks and expertise. To provide guidance, give strategic direction and review project implementation, SERB has a Steering Committee, comprising of the DDM, USAID, DGHS, Focal Agencies of HOPE and FSCD (NIPSOM and FSCD), ShSMCH and SERB COP and Training Manager. The Chair of the committee is the Director General of DDM, the nodal agency focal point. The committee meets twice in a year and reviews the project implementation status, challenges and issues, takes decisions and follows up recommended actions. I.D. SERB Activity Link to USAID/Bangladesh’s CDCS The Development Objective 4 (DO4) of USAID/Bangladesh‘s Country Development Cooperation Strategy (CDCS) results framework for 2011‐2016: ―Responsiveness to Climate Change Improved‖, includes Global Climate Change (GCC) and PL480 Title II Multi-Year Assistance Program resources to improve Bangladesh‘s ability to respond to climate change and to mitigate the effects of climate change on the country‘s most vulnerable populations. SERB PERFORMANCE EVALUATION REPORT 16 | P a g e The aim of SERB is to improve GOB and Bangladeshi communities‘ disaster response skills while simultaneously increasing earthquake resilience through the adaptation of technologies and enhancement of the capacity of hospital personnel, fire service, civil defense authority, and related institutions. The tasks of SERB are therefore linked to the achievement of DO4 Sub IR 4.2.2: Increased Climate Resiliency through Technologies Adopted, and other Public Law (PL) 480 Title II activities that promote disaster risk mitigation technologies and work with vulnerable communities to help them recover from shocks. I.E. SERB Activity Development Hypothesis The Development Hypothesis for SERB is that a reduction in the devastating effects of natural disasters, such as major earthquakes which causes number of casualties and associated economic losses. This reduction in the devastating effects of natural disasters can be achieved by providing Bangladeshi public hospital staffs the core skills necessary to sustain life, prevent further injury and reduce pain at the scene of a natural disaster, and by providing the FSCD‘s SAR teams and community volunteers‘ appropriate tools and equipment to respond more effectively and efficiently. This will hold true as long as the GOB continues to support hospital and community based disaster response efforts. The development hypothesis lays out a causal pathway to achieve the activity‘s intermediate result, “Enhanced capacity of public health facility personnel and urban community volunteers to better respond to victims of earthquakes”. I.F. Critical Assumptions and Potential Risks SERB identified the following critical assumptions that are beyond the activity‘s control but associated with its successful achievements.  National response to natural disasters and environmental risks remains manageable;  GOB commitment to climate change remains strong; and  International commitment to global climate change remains strong. There are three potential risks that SERB has identified include:  Political unrest;  Medical professionals‘ reluctance in the training courses; and  Irregular maintenance of tools and equipment. II. SERB Components and Tasks SERB is implemented in the cities of Chittagong, Dhaka, Gazipur, Manikganj, Mymensingh, Narayanganj, Rangpur, Sylhet and Tangail. The rationale for choosing these locations are because these areas are within the high seismic risk zone according to the Bangladesh earthquake risk map including hydromet hazards such as floods and cyclones. The geographic coverage of SERB is shown in the map (Figure1). SERB PERFORMANCE EVALUATION REPORT 17 | P a g e SERB, under its HOPE component aims to enhance public hospital medical personnel disaster response capacity. Main tasks under the component are the basic and specialized training courses on health facility risk management, training for instructors and capacity building of the hospital command team to command, coordinate and control an emergency incident including hospital risk assessment (HRA), development of new emergency response plans (ERPs) and/or improvement of existing ERPs, study visit, and HRA dissemination workshop, incident management system (IMS) introductory workshop for policy makers; and IMS resource mapping. Tasks of the FSCD component of SERB include: combined course of CSSR/FF/FA to the urban community volunteers; simulation exercises for these volunteers on search and rescue (SAR); and procurement of light SAR tools and equipment for use primarily by the community volunteers in MCI response of the FSCD fire SERB PERFORMANCE EVALUATION REPORT 18 | P a g e stations. SERB lately added one new task of documentation of best practices and case studies of the FSCD community volunteers and published them in booklet format for distribution to the FSCD fire stations, urban volunteers, participating hospitals, HOPE/HICS graduates, SERB implementing agencies and stakeholders. Table - 1 provides the componentwise list of tasks, their LOA targets and achievements. Table 1: List of Tasks of SERB by Components by Phases III. Evaluation Purpose and Objectives The evaluation is focused on the performance of SERB by examining its implementation, inputs, outputs and outcomes. Evaluation objectives of specific interest to SERB are to:  Review the progress towards achievement of stated SERB activity objective and intermediate and sub￾intermediate results.  Assess the relevance of the training materials used in SERB for HOPE-B, TFI, HICS and CSSR, Fire Fighting and First Aid training courses.  Determine the effectiveness of SERB interventions to handle mass causalities by the public health facilities and FSCD.  Assess the effectiveness of SERB interventions in contributing to the USAID/Bangladesh DO4: Responsiveness to Climate Change Improved.  Identify the success factors and constraints of the activity including institutionalization of the training courses by the implementing agencies.  Assess the effectiveness and sustainability of gender integration toward increased gender responsiveness in mass casualty management. Task Types Targets Achievements (As of 31st Phase I Phase 2 Phase 3 LOA Total May 2017) Tasks of HOPE Component HOPE-Basic Course 9 8 4 21 17 HOPE-TFI Course 2 1 1 4 3 HICS Course 3 1 2 6 4 ERP Development 3 4 3 10 7 HRA 12 4 3 19 16 HRA Findings Dissemination Meetings 12 4 3 19 15 National HRA Dissemination Workshop - 1 - 1 1 Study Visit - 1 - 1 1 IMS Introductory Workshop - 1 - 1 1 IMS Resource Mapping - 1 - 1 - Total: All Tasks – HOPE Component 41 26 16 83 65 Tasks of FSCD Component CSSR/FF/FA Training - 30 4 34 30 SAR Simulation - 12 - 12 13 SAR Tools and Equipment 91 80 - 171 171 Case Study Booklet - 1 - 1 1 Total: All Tasks – FSCD Component 91 123 4 218 215 SERB PERFORMANCE EVALUATION REPORT 19 | P a g e  Assess the effectiveness of partnership, coordination and collaboration with NIPSOM, FSCD, DDM, DGHS and other stakeholders as well as the systems established to support activity implementation.  Based on the overall findings of the evaluation, highlight key lessons learned and best practices which can be replicated.  Make specific recommendations on areas and/or interventions that will require further improvements in the future for activities similar to SERB. IV. Evaluation Methodology A mixed methodology of qualitative and quantitative data gathering and analysis has been the approach for this performance evaluation. Annex B of the report has the detailed evaluation methodology, evaluation questionnaires, list of places visited and interviews conducted during the evaluation. The evaluation process includes the review of secondary data sources, focus group discussions (FGDs) and key informant interviews (KIIs), activity site and community visits, and ongoing training programs observation. The questionnaires included a range of structured questions pertaining to these evaluation criteria and scoring questions required for collecting subjective responses. In addition, appropriate open ended questions were asked to capture the views and perspectives of the respondents, especially those of SERB participants, implementers and stakeholders. For data collection, the evaluator made specific site visits and face-to-face respondent interviews and telephone calls. Respondents included ADPC senior management and staff, fire stations, hospital management and HOPE/HICS graduates, FSCD urban volunteers and communities. The information also include feedback of USAID, ADPC senior management and key SERB activity staffs, former Agreement Officer‘s Representatives (AOR) of SERB/USAID, and former Chief of Party (COP) of SERB/ADPC. The findings and recommendations are based on the review of relevance, effectiveness, replicability (adoption, continuity) coordination, gender responsiveness, and sustainability of the interventions of SERB. The ADPC senior management and staff at Bangkok, Thailand have been interviewed over phone and personnels at the hospitals and fire stations and FSCD volunteers have been interviewed on the spot. The scope of this performance evaluation is limited to examination of the implementation, inputs, outputs and outcomes of SERB. FGDs and KIIs were conducted with over 200 HOPE graduates and instructors, FSCD urban community volunteers and fire station officers, hospital and FSCD senior management and staff of 8 out of 16 health facilities and 8 out of 17 fire stations in the six program cities of Phases 1 and 2. The numbers of hospitals and fire stations visited represent nearly 50% of the total target. Refer to Section III of Annex A for detailed information about determination of number of interview sites. V. SERB Targets and Achievements Overall, progress of implementation of SERB activity is 93% of the LOA target with accomplishments of 78% of the HOPE component and 99% of the FSCD component. The summary of achievements of SERB is presented in Table 2. Annex C of the report contains the component-wise detailed statistics of the targets and achievements of tasks from the beginning of SERB in June 2013 until the completion of this evaluation in May 2017. SERB PERFORMANCE EVALUATION REPORT 20 | P a g e 0 1 2 3 4 5 6 7 Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12 Q13 Q14 Q15 Q16 FIGURE 2: TRAINING COURSES HELD QUARTERLY FROM JUNE 2013 TO MAY 2017 (Q1 refers to June – August 2013 ..............Q16 refers to March – May 2017) HOPE-B HOPE-T HICS CSSR/FF/FA SIMULATION NUMBER OF EVENTS Table 2: Summary of Achievements of SERB Activity (As of 31st May 2017) The highlight of accomplishments is the progress of almost all of training and simulation tasks planned under both the components, HOPE and FSCD in Phases 1 and 2. SERB and its GOB implementing partners, NIPSOM and FSCD, have collaboratively been able to manage the potential challenges such as reluctance of medical professionals to participate in training courses. Figure 2 shows the schedules of training courses and SAR simulation exercises undertaken quarterly by SERB under both HOPE and FSCD components. It is evident from the information collected that SERB did a daunting task in maintaining the training schedules uniform throughout the activity period. This has been accomplished because of timely preparation, proper planning, and effective implementation. All these accounts for skills and experiences of the staff involved despite the challenges of getting confirmation of the proposed schedules with the participants such as the hospitals and concerned government authorities. V.A. SERB Componentwise Targets and Achievements V.A.a. Targets and Achievements – HOPE Component Overall accomplishments of HOPE component is 78% of the LOA target. Apart from a few tasks planned for Phases 1 and 2, most others have been completed. The outstanding tasks include: HRA findings dissemination meeting with the National Institute of Traumatology and Orthopedic Rehabilitation (NITOR); SERB Components LOA Targets Achievements (As of 31st May 2017) Achievements as % of Targets Remarks HOPE Component 83 65 78 Refer to Table 4 for detailed statistics of HOPE Component FSCD Component 218 215 99 Refer to Table 6 for detailed statistics of FSCD Component Total - Both Components 301 280 93 SERB PERFORMANCE EVALUATION REPORT SERB PERFORMANCE EVALUATION REPORT SERB PERFORMANCE EVALUATION REPORT 23 | P a g e Director and senior officials, ADPC Executive Director and senior staff, senior management and health professionals of the SERB and non-SERB hospitals, and FSCD senior administration and instructors. iii. Emergency Response Plans (ERP) As part of disaster preparedness of the hospitals, SERB developed all 7 ERPs planned in Phases 1 and 2 and finalized them through validation exercises and meetings participated by the hospital management, HOPE graduates and SERB staff. In developing the ERPs, SERB has drawn on many sources for information, including the GoB Disaster Management Act 2012, Standing Orders on Disasters, DGHS‘s Post Earthquake Comprehensive Health Care Action Plan for Urban Cities and findings of Hospital Risk Assessment (HRA) conducted by SERB using the tools and guidance on Hospital Safety Index issued by WHO-PAHO. The ERP is a living document containing reference tool for hospital incident management system for responding to mass casualty incidents (MCI), especially in the event of earthquakes. In summary, the ERPs cover both preparedness and response aspects of MCM. iv. Study Visit SERB organized a study visit for a team of 15 (6 female and 9 male) from the program staff, HOPE instructors and graduates, FSCD instructors and NIPSOM staff to Kathmandu, Nepal, from 19 – 23 March 2017. The aim of the visit was to build participants‘ capacities on hospital preparedness and response tasks through observation of related best practices in Nepal. The study visit included meetings and consultations with various government agencies, hospitals/health facility personnel and other stakeholders related to managing emergency health facilities, rescue operations, and allocating resources in disaster affected areas. v. IMS National Introductory Workshop IMS is a systematic, proactive approach to guide all levels of government departments and agencies, nongovernmental organizations, critical infrastructure owners and operators, private sector and all other organizations who assume a role in emergency management to work together seamlessly and manage incidents involving all threats and hazards— regardless of cause, size, location, or complexity—in order to reduce loss of life, property and harm to the environment. IMS aims at a common approach for managing a disaster response or incidents. A two-day national level introductory workshop of IMS was held in March 2017 in Dhaka. This was a high-level workshop with participation from the Minister for Disaster Management and Relief, Secretary of Ministry of Disaster Management & Relief (MOMDR), Ambassador of US Mission to Bangladesh, Director General, Department of Disaster Management (DDM), Director General, FSCD, other high level government officials, representatives from USAID/OFDHA regional office, United States Forest Service (USFS), USAID Bangladesh, UN agencies, diplomatic offices, Bangladesh Armed Forces Division (AFD), Bangladesh Police, international and national NGOs, Bangladesh Study Visit to Nepal IMS Workshop SERB PERFORMANCE EVALUATION REPORT 24 | P a g e Boys Scout, Girls Guide and other agencies. The two-day introductory workshop on IMS has successfully drawn the GOB senior level decision makers, head of the diplomatic mission and key stakeholders. V.A.b. Targets and Achievements – FSCD Component SERB is right on track in implementing all planned tasks of Phases 1 and 2 under the FSCD component with one task well ahead of the target. By 31st May 2017, the component achieved 30 out the LOA target of 34 batches of training in CSSR/FF/FA and 13 against the LOA target of 12 simulation exercises. The remaining 4 batches of training are planned for Phase 3 of SERB. The CSSR/FF/FA trainees were selected from 17 fire stations located in the six SERB cities and the simulation participants were selected from 10 fire stations in three6 out of the six SERB cities. The target number of 171 set of light SAR tools and equipment under the component have been fully delivered to 17 fire stations in the six SERB cities. Table - 5 provides the summary of achievements of the FSCD component as of 31st May 2017. Table 5: Summary of Achievements of FSCD Component (As of 31st May 2017) Task Types LOA Targets Achievements (As of 31st May, 2017) Achievements as Percentage of Targets CSSR/FF/FA Training 34 30 88 SAR Simulation 12 13 108 SAR Tools and Equipment 171 171 100 Case Study Booklet 1 1 100 Total: All Tasks 218 215 99 i. Advance/Refreshers Training (CSSR/FF/FA) Overall achievement is 94% of the LOA target. The target number of batches (30) and participants (1,200 volunteers) of the CSSR/FF/FA course of the FSCD component under Phases 1 and 2 have been fully achieved. Table 6 presents the distribution of participants by task. Distribution of trainees by locations is available at Annex F. Table 6: Distribution of FSCD Volunteer Participants by Task (As of 31st May 2017) Task Types LOA Targets Achievements (As of 31st May 2017) Achievements as Percentage of Targets Female Male Total CSSR/FF/FA 1,360 323 877 1,200 88 Simulation 600 106 544 6507 108 Total: All Tasks 1,960 429 1,421 1,850 94 ii. SAR Simulations The achievements (13 simulations for a total of 650 participants) exceeded target of 12 SAR simulations for a total 600 participants (50 per event) under the FSCD component. SERB facilitated these simulations with the assistance of FSCD fire fighters and instructors for the urban community volunteers 6 Three cities are Dhaka, Chittagong and Sylhet. 7 Includes one additional simulation participated by 50 volunteers. Community Volunteers Simulation Exercise SERB PERFORMANCE EVALUATION REPORT SERB PERFORMANCE EVALUATION REPORT 26 | P a g e and Dhaka city, the total number of public hospitals and fire stations increased to 21 in each case. Table 7 has more detailed statistics on geographic targeting followed by a map showing the SERB target cities. Table 7: SERB Phase-wise LOA Targets of Public Hospitals and Fire Stations 10 Source: Health Bulletin 2016, DGHS, MoHFW A. Target of Public Hospitals Cities Total Public Hospitals10 SERB Target Hospitals SERB Total Target - All Phases % of SERB Coverage Phase 1 June 2013 to May 2015 Phase 2 June 2015 to May 2017 Phase 3 June 2017 to May 2018 Chittagong 5 3 - - 3 60 Dhaka 18 7 - 1 8 44 Sylhet 2 2 - - 2 100 Gazipur 1 - 1 - 1 100 Mymensingh 2 - 2 - 2 100 Tangail 1 - 1 - 1 100 Manikganj 1 - - 1 1 100 Narayanganj 2 - - 2 1 100 Rangpur 1 - - 1 1 100 Total – All Cities 33 12 4 5 21 64 B. Target of FSCD Fire Stations Cities Total Fire Stations SERB Target Fire Stations SERB Total Target - All Phases % of SERB Coverage Phase 1 June 2013 to May 2015 Phase 2 June 2015 to May 2017 Phase 3 June 2017 to May 2018 Chittagong 8 2 - - 2 25 Dhaka 14 8 2 2 12 86 Sylhet 2 1 - - 1 50 Gazipur 2 - 2 - 2 100 Mymensingh 1 - 1 - 1 100 Tangail 2 - 1 - 1 50 Manikganj 1 - - 1 1 100 Narayanganj 1 - - 1 1 100 Rangpur 1 - - - - 0 SERB PERFORMANCE EVALUATION REPORT 27 | P a g e SERB is implemented in the selected seismic zones and focused on earthquake preparedness and response capacities. Almost two-thirds of the public hospitals and fire stations (42 out of 65) have been selected by SERB in the nine program cities. As compared to the scale of disasters faced, the progress and achievements so far have not been enough quantitatively in order to effectively reduce the risk of the population of the cities significantly and to trigger a self-sustained process. So, there is still need to strengthen self-sustained process of high standard and high quality instruction and development by consolidating the foundation of SERB process and to ingrain it with the national program and efforts. Findings of the evaluation show that the SERB tasks are highly relevant (Refer to detailed discussion in Section VI.C) and effective (Refer to detailed discussion in Section VI.D) in mass casualty management (MCM) and therefore have potential for replication. There are areas beyond the current nine cities including Dinajpur, Bogra, Bandarban, Khagrachari, Rangamati and Cox‘s Bazar affected by one or more of the natural and human-made hazards, such as, cyclone, storm surge, tornadoes, flash floods, landslides, cold wave, road accidents, and fire. Most recently the regions of Chittagong Hill Tracts, Chittagong and Sylhet have frequently been devastated by landslides. This is becoming a growing concern of the host government and international communities. Recommendations No. 1: Consideration should be given, if funding is available, to include the remainder of the public hospitals and fire stations in the cities of Dhaka and Chittagong in the SERB activity. No. 2: If there to be a future activity like SERB it is recommended that, new cities, namely, Dinajpur, Bogra, Bandarban, Khagrachari, Rangamati and Cox’s Bazar be brought under the activity as they are affected by one or more of the natural and human-made hazards, such as, cyclone, storm surge, tornadoes, flash floods, landslides, cold wave, road accidents, and fire. The focus of the activity shall stretch beyond earthquake but multi-hazard depending on the vulnerability of the cities to the types of hazards for mass casualty management. No. 3: Given the growing concern on the impact of landslides, it is recommended that the three hill districts - Bandarban, Khagrachari and Rangamati be selected on a priority basis for MCM adopting the SERB model. VI.B. Accomplishments Analysis VI.B.a. HOPE Component - Training Courses i. HOPE-B, HOPE-T and HICS Findings Overall accomplishments of the HOPE-B, HOPE-T and HICS training courses is 77% of the LOA target 1,223 participants. The HOPE-B training and HRA have achieved 82% and 85% of their respective targets. Three HOPE-T courses were held under Phases 1 and 2 with a total of 52 participants attended against the target of 72 participants in these courses. These HOPE-T courses are thus short of achieving their target number of trainees under Phases 1 and 2 by 28%. The reasons behind the low achievement being the non￾availability of required number of eligible participants who are subject to meet the selection criteria such as: Total – All Cities 32 11 6 4 21 66 SERB PERFORMANCE EVALUATION REPORT 28 | P a g e first, a candidate must have completed the HOPE-B course with a high score (60%+); then the result of his/her extra curriculum qualities evaluated by the course instructors; and finally, must meet the required threshold level in the selection test, usually pre- and post-tests. The 4 HICS courses planned in Phases 1 and 2 have already exceeded their target number of 96 participants. The remaining 2 HICS courses are planned in Phase 3. About one-fifth of the pool of 567 graduates (52 HOPE instructors, 415 HOPE and 100 HICS graduates), representing 39% female and 61% male, from the 16 hospitals and other different organizations has moved to different hospitals and locations such as Upazila Health Complex within 6 to 12 months of training completion. Yet this is good in the sense that spreading the benefit of the training to a greater range of organizations the intended purpose of SERB is affected. It will be interesting to see whether these graduates will be able to influence their hospitals and health care facilities in their areas. SERB also created a database of HOPE/HICS graduates and instructors. The process of turning over the database of Phases 1 and 2 to NIPSOM and DGHS is underway. As part of the process SERB will orient the concerned staff at the NIPSOM and DGHS before the database is turned over to these organizations. Recommendation No. 4: The HOPE/HICS graduates being employees of the government health facilities are subject to move from one health facility to another. So, to reduce potential loss of skilled staffs in case of an emergency, a database and network shall be established by NIPSOM to track the HOPE/HICS graduates who leave their posts. The network shall be fully functional in tracking the HOPE/HICS graduates in MCM and there shall be understanding among DGHS, NIPSOM and participating hospitals that the rotated staffs are brought back to their previous positions for emergency response. VI.B.b. FSCD Component – Training and Simulation i. Advance/Refreshers Training (CSSR/FF/FA) and Simulation Exercises Of the total FSCD volunteers of 20,762 (24% female and 76% male) in 21 target fire stations the CSSR/FF/FA course covered 1,200 volunteers (about 6%) under Phases 1 and 2 and additional 160 volunteers are planned under Phase 3. The simulation exercises covered 650 volunteers (less than 3%) of the total volunteers. Realizing the importance of the trainings and simulations offered by SERB, almost all respondents of the evaluation have expressed their interest for additional refreshers and simulation exercises to be conducted annually. Annex G contains the distribution of the FSCD volunteers by cities. Recommendation No. 5: Compared to the need for training for the FSCD urban community volunteers, SAR refreshers shall be offered at least once a year and simulations shall be conducted at least in every six months to the volunteers. VI.C. Relevance The evaluation collected information surrounding the appropriateness of the design and delivery of training courses with respect to the cultural and administrative contexts and needs of the target institutions, relevance of tasks conforms to the priority the SERB nodal agencies (DDM and DGHS) and need of the implementing partners (NIPSOM and FSCD) in respect to preparedness in and response to MCI. The question on the respondents‘ opinion about the level of relevance included a scoring question rated between 0 to 4 where 0 indicated not-relevant, 1 indicated slightly relevant, 2 indicated moderately relevant, 3 indicated fully relevant and 4 indicated highly relevant. VI.C.a. Relevance of Interventions of HOPE Component SERB PERFORMANCE EVALUATION REPORT 29 | P a g e i. Training Courses on HOPE-B, HOPE-T and HICS The rationale mentioned by most respondents about the relevance of training courses offered under the HOPE component was that it covers mass casualty management which is a prime and massive responsibility of health professionals and medical institutions. The respondents endorsed all 23 HOPE lessons as being important but these lessons need to be updated and practiced on a regular basis, maybe once a year. The training courses in HOPE and HICS are a direct benefit to the participating hospitals‘ incident command system (ICS). Out of 72 interviewees in 8 hospitals covered by the evaluation, 51 interviewees (71%) gave the highest rating to the question on relevance of the training courses (topics, materials, presentations) under the HOPE component. The findings presented below support these figures. A summary of results are presented in Table 8 below followed by brief descriptions. The full results are contained in Annex H. Table 8: Results of Quantifiable Questions on Relevance of Tasks under HOPE Component Tasks Total Respondents11 (F=Female M=Male) Number of Respondents Gave Highest Rating (F=Female M=Male) Percentage Notes HOPE-B, HOPE-T and HICS Training 72 (F39+M33) 51 (F27+M22) 71 72 respondents from 8 hospitals were covered by the field interviews in 8 out of 16 SERB hospitals Hospital Risk Assessments 72 (F39+M33) 70 (F37+M33) 97 72 respondents from 8 hospitals were covered by the field interviews in 8 out of 16 SERB hospitals Hospital Emergency Response Plans 56 (F28+M28) 56 (F28+M28) 100 56 respondents from 6 hospitals; 2 hospitals (BITID and SKH) are not targeted by SERB for ERP development Study Visit 9 (F4+M5) 9 (F4+M5) 100 Question on relevance of study visit covered in STAMCH only; Other 7 hospitals were covered prior to the study visit. Recalling of participants‘ experience, citing of practical examples, and interaction with the participants in a positive manner by the instructors were observed in an ongoing HOPE-B training course. The participants stated that the topics and materials covered in the training courses were useful. The majority of responses were very supportive of the training courses wanting more of them, wanting refresher and advanced training. They showed keen interest in the courses. There were problems associated with the availability of doctors in some of the HOPE-B courses. However, the hospitals, nodal agencies, implementing partners want more of these HOPE/HICS courses. The participants like to have refreshers and simulations at least once a year. The participants have expressed a divergence of views regarding the usefulness of the various lessons with the majority of the lessons/modules being most popular and some being less popular. The triage, HICs and MCI are found to be the most popular HOPE-B lessons while structural/nonstructural lesson is less popular 11 F=Female; M=Male SERB PERFORMANCE EVALUATION REPORT 30 | P a g e as it is too technical (engineering oriented) which is not easily understood by those who are nontechnical and unfamiliar with the subject. Recommendation No. 6: The training shall arrange separate session for the groups of participants who require special attention for particular topics or lessons. Both HOPE-B and HICS courses trained a total of 515 (39% female and 61% male) graduates including doctors, nurses, technicians and support staff. The distribution of these 515 graduates by professional categories show that 66% (340) are doctors, 26% (132) are nurses and 8% (43) are support and technical staff. The respondents from the latter two categories found the courses tough to grasp due to their technical contents. They need the courses that can be easily understood at their levels. There is also desire from the high level officials of DGHS, DDM and NIPSOM and heads of the hospitals to arrange short courses for them, preferably a shorter version of HOPE-B. These high level officials and hospital heads and senior administration usually cannot manage time to attend the week-long courses. The evaluation finds it important for the high level officials of the decision making organizations and the hospital heads and senior administration to have a practical knowledge of the dynamics of MCM. Recommendation No. 7: The training course shall be based on the target participants’ skills, instincts and abilities to understand and adapt and their availability to attend. The suggested training course for the nurses and support staff shall be a revised version of the HOPE-B course, less technical and according to their level, and shorter courses and orientation on HOPE/HICS for the high level decision makers and hospital heads. ii. Hospital Risk Assessments (HRA) Hospitals play a vital role in ensuring healthcare delivery during disasters, assessing hospital safety is of utmost importance. This is one of the components of disaster risk reduction strategies employed in the health sector in assessing the safety of health care facilities, identifying their vulnerabilities, as well as setting the priorities in dealing with existing gaps and capacity to respond to emergencies. The HRAs have been conducted by SERB in coordination with the nodal and implementing agencies using the HSI comprising 151 indicators developed by WHO–PAHO. The HRA is useful as a rapid and low-cost diagnostic tool for assessing the probability that a hospital will remain operational in emergencies and disasters. The tool enables the hospital management to have a better understanding of risk factors in their facility and utilize the result of the assessment in developing corrective action plans to improve hospital safety from disasters. The HRA provides useful information about a hospital‘s strengths and weaknesses and points to the actions required to improve the safety and emergency and disaster management capacities of the hospital. Almost all 70 (97%) out of 72 (Female 39 and Male 33) respondents reached during the field interviews rated the HRA as highly relevant. The rationale cited by the respondents was that the hospitals have developed skills to conduct their own hospital‘s risk assessment. The group works enabled them to better understand the structural, non-structural, and emergency and disaster management components of the HSI. Results of HRAs show that out of 16 hospitals the overall safety index, calculated for the structural, nonstructural and functional components of the hospitals by the 151 HSI indicators, 6 hospitals were placed SERB PERFORMANCE EVALUATION REPORT 31 | P a g e in ‗B‘ category and the other 10 were placed in ‗C‘ category 12. In case of Category B hospitals, the intervention measures are needed in the short term as the hospitals‘ current levels of safety and emergency and disaster management are such that the safety of patients and hospital staff, and the hospitals‘ ability to function during and after emergencies and disasters, are potentially at risk. For Category C hospitals, the urgent intervention measures are needed when the hospitals are unlikely to function during and after emergencies and disasters, and the current levels of safety and emergency and disaster management are inadequate to protect the lives of patients and hospital staff during and after emergencies or disasters. Annex E provides the HRA findings and recommendations followed by the details of HSI calculations. Following is a summary of HRA results:  The hospitals that are potentially at risk, also known as Category B hospitals, are required to take short term intervention measures and the hospitals that are unlikely to function during and after emergencies and disasters, also known as Category C hospitals, are required to take urgent intervention measures for the safety of patients and hospital staff, and the hospitals‘ ability to function during and after emergencies or disasters.  The hospitals at present have many of their nonstructural elements between low and average levels of safety. Specifically, they are the systems of: electricity and lighting in critical areas; telecommunications; water supply; fire protection; waste management; fuel reserve; medical gases; HVAC; and shelving of furnishings and equipment; safety and security of staff and patient; and medical and laboratory equipment and supplies used for diagnosis and treatment. Most of these elements are inadequate, too. The hospitals, therefore, need improvement, enhancement and automation of these nonstructural elements. The hospitals also are to conduct repair and regular cleaning and maintenance of these nonstructural elements.  Hospitals may not always require full activation of ERPs for the internal events such as fire, floods, chemical spills and utility failures if they have contingency plans and procedures established for these internal events to ensure a safe hospital. So, the hospitals are required to develop and implement contingency plans and procedures that are required for certain critical events and essential services. Suggested plans are:  Maintenance and Inspection Contingency Plan 12 HOSPITAL SAFETY GROUPS BY SAFETY INDEX SCORES AND GENERAL RECOMMENDATIONS: THE SAFETY INDEX HAS A MAXIMUM VALUE OF 1 (ONE) AND A MINIMUM OF 0 (ZERO). Hospital Safety Index Safety Group Description 0.66 – 1.0 A It is likely that the hospital will function in emergencies and disasters. It is recommended, however, to continue measures to improve emergency and disaster management capacity and to carry out measures in the medium- and long-term to improve the safety level in case of emergencies and disasters. 0.36 – 0.65 B Intervention measures are needed in the short term. The hospital‘s current levels of safety and emergency and disaster management are such that the safety of patients and hospital staff, and the hospital‘s ability to function during and after emergencies and disasters, are potentially at risk. 0 – 0.35 C Urgent intervention measures are needed. The hospital is unlikely to function during and after emergencies and disasters, and the current levels of safety and emergency and disaster management are inadequate to protect the lives of patients and hospital staff during and after emergencies or disasters. SERB PERFORMANCE EVALUATION REPORT 32 | P a g e  Fire Safety and Fire Emergency Plan  Flood Management Contingency Plan  Chemical Spills Management Contingency Plan  Power Failure Contingency Plan  Hospital Evacuation Contingency Plan  Psychosocial Support Plan  Hospital Business Continuity Plan  Personal Health and Critical Incident Stress Plan  Most of the old hospital buildings‘ are concrete frame structures and subject to ageing. So the overall safety level of structure was found to be average. Since all but one13 SERB target hospitals are located at the moderate seismic intensity area and very close to the Madhupur fault, in case of moderate to severe earthquake the hospitals are subject to heavy damage. In addition, among other possible hazards that pose threats to the safety of the hospital include river floods, tornadoes, local storms, and mass gathering as well as biological and technological accidents. The hospitals are the frontline medical facility in case there are emergencies in their immediate surrounding especially during working days and hours. So, the concerned hospitals are required to undertake proper retrofitting and remodeling of the structures as early as possible.  Most SERB target hospitals‘ level of preparedness with regards to their emergency and disaster management capacity in responding to mass casualty incidents is low. This functional capacity of the hospital, therefore, needs strengthening as a priority action of the hospital in the immediate time for effective responses to the emergencies or disasters. The preparedness include:  Formation of a fully functional Hospital Disaster Management Committee with roles and responsibilities for each member.  Conducting simulation exercises/drills for capacity building purpose with the involvement of various sections/departments of the hospital.  Establishment and application of Hospital Incident Command System (HICS) to increase the effectiveness and efficiency of response.  Establishment of fully functional network between local health facilities, and district level disaster management programs and committees implemented by the Ministry of Disaster Management and Relief, Fire Service and Civil Defence, Red Crescent Society, etc.  Equip Emergency Operations Centre (EOC) with equipment and supplies to be readily available to set up the EOC for communications, information management including digitalization of patient record system, identification, security, and well-being of EOC staff. The EOC should be backed up by an information management system that supports emergency operations and that can link to data from the hospital‘s information management system.  Establishment of all-hazards hospital recovery plan that defines actions to be taken to recover normal functions of the hospital after an emergency or disaster. The recovery plan should provide continuity of recovery and rehabilitation of patient services, the recovery needs of personnel, the replenishment of supplies and replacement of equipment, and procedures for determining priorities for assessment and rehabilitation of the hospital‘s structural and nonstructural elements which may have been damaged. 13 SOMCH at Sylhet is within the high seismic intensity area. SERB PERFORMANCE EVALUATION REPORT 33 | P a g e  Most old buildings of the hospitals were constructed well before the introduction of the modern standard and safety regulations such as the Bangladesh National Building Code (BNBC). Availability of engineers was a challenge in case of assessment of the structural component. Only one or two civil engineers from the Public Works Department (PWD) of the government were present in each of the HRAs. But the assessment required a group of engineers from PWD (civil, electrical, etc.) to fully participate in the assessment. Thus assessment of the structural component of the hospital was observation based and understanding of the assessment team. In most instances in which major structural deficiencies were found, it is important to take note that these are not the result of a detailed engineering assessment but rather limited to visual observations and feedback from interviews conducted during the assessment with relevant hospital staff.  There is a need for establishing a National Emergency Medical Team (EMT) based on WHO Classification and Minimum Standards for National Medical Teams in sudden onset disasters.  Hospitals shall arrange separate training on Hospital Emergency Response Plan (ERP) for the hospital personnel and allocate budget for implementation of the ERP. Hospitals shall conduct simulation exercises/drills for capacity building purpose and refreshers trainings for all hospital personnel regularly. Recommendation No. 8: For the safety of patients and hospital staff, and the hospitals’ ability to function during and after emergencies or disasters, the concerned hospitals shall explore funding for implementation of the HRA recommended actions pertaining to the structural, nonstructural and functional elements. (a) Hospitals shall:  Improve, enhance and automate the nonstructural elements which include systems of: electricity and lighting in critical areas; telecommunications; water supply; fire protection; waste management; fuel reserve; medical gases; HVAC; and shelving of furnishings and equipment; safety and security of staff and patient; and medical and laboratory equipment and supplies used for diagnosis and treatment..  Develop and implement contingency plans that are required for certain critical events and essential services pertaining to maintenance and inspection, fire safety and fire emergency, flood management, chemical spills management, power failure, hospital evacuation, psychosocial support, hospital business continuity, and personal health and critical incident stress.  Strengthening of the functional capacity of the hospital as a priority action of the hospital in the immediate time for effective responses to the emergencies or disasters. (b) DGHS shall:  Undertake proper retrofitting and remodeling of the structures as early as possible, especially those hospital buildings’ are of concrete frame structures and the overall safety level of structure was found to be average.  Conduct detailed structural assessments of those hospitals assessed to be of low safety level by HRA and take mitigation measures as per assessments.  Establish a National Emergency Medical Team (EMT) based on WHO Classification and Minimum Standards for National Medical Teams in sudden onset disasters. iii. Emergency Response Plans (ERP) An ERP is a living document containing reference tool for hospital incident management system for responding to mass casualty incidents (MCI), especially in the event of earthquakes. The document contains information on both general and specific responsibilities for the Incident Management Team and hospital departments, sections and units; formats and reference material for assessing and reporting on populations at risk; descriptions of external and internal disaster preparedness and management; general information SERB PERFORMANCE EVALUATION REPORT 34 | P a g e related to disaster interventions; and information on working with the disaster committees, authorities and other stakeholders and different government departments in the district. The overall goal of ERP is to mainstream disaster preparedness and response interventions of hospitals to be fully functional immediately after disaster strike and are able to respond without any delay to the medical requirements of the affected community. Developed by following the WHO Hospital Emergency Checklist the response operation procedures (ROP) is the core of the ERP with main focus on phases and procedures of emergency response in the context of management of mass casualties by sudden onset of disasters. The ROP covers the standing operating procedures to carry out the emergency response operation from the phase of preparation in pre-disaster time to the phase of recovery in the post-disaster time. The guiding policies and procedure include: GoB Disaster Management Act 2012 and Standing Orders on Disasters; and findings of HRA conducted by SERB using the tools and guidance on Hospital Safety Index Risk issued by WHO-PAHO. The ERPs cover both preparedness and response aspects of mass casualty management. The respondents (100%) rated ERP as highly relevant. The rationale cited by the respondents was that the hospitals have developed skills to conduct their own hospital‘s risk assessment. Hospitals were oriented on appropriate steps and measures to take in the event of MCIs. The ERPs of the hospitals have been drafted by SERB and shared them with the respective hospitals for validation. None of the 16 SERB hospitals that have HRA conducted has ERP in place. Recommendation: No. 9: Hospitals shall keep the ERP up-to-date and fully functional. Hospitals shall organize orientation on the ERP for the new committee members and staff so that they are aware of and ready for responding to mass casualty incidents (MCI). iv. Study Visit The study visit to Nepal brought outstanding alumni from the SERB‘s HOPE/HICS courses and government representatives from Bangladesh in March 2017. The study team reported that the visit was a great opportunity for discussions, exchanges of hospital preparedness practices and learning about topics of common interest between hospital administrators from Nepal and Bangladesh. It also gave participants the chance to establish a network between hospital administrators, which will help foster future cooperation between Bangladesh and Nepal. There are demands for more of this type of visit. The participants including the health professionals from different public hospitals of the study visit to Nepal reported that the visit provided them with a great learning opportunity to understand the practical application of hospital preparedness and hospital incident command system knowledge in case of a real disaster such as earthquake. They learned practically the importance of a hospital emergency response plan and its implementation from the visit. During the consultations with different agencies, participants had the opportunity to learn from high ranking officials and hospital authorities who shared their experiences in the management of response and recovery of the Nepal earthquake in April 2015. These first-hand accounts gave participants valuable lessons on how to manage chaotic situations during a crisis and how to provide healthcare, food, shelter, water, and even psychosocial support to disaster survivors. The respondents (100%) rated the study visit highly relevant. There has been no outbriefs after the visit. Recommendations No. 10: SERB shall conduct in-depth analysis of the usefulness and benefit of study visits of the program staff, HOPE instructors and graduates, FSCD instructors and NIPSOM staff to the health facilities and emergency MCM SERB PERFORMANCE EVALUATION REPORT 35 | P a g e activities within and outside the country in the region. Since executing study visits are expensive, before advocating for them, the program shall determine the feasibility including expected result of the proposed study visit. No. 11: Hospital shall organize outbriefs of the study visit to disseminate the experience gained and lessons learned from the study visit and shall review and implement the action plan developed by the participants as feasible. VI.C.b. Relevance of Interventions of FSCD Component An important rationale mentioned by most respondents about the relevance of the SAR training and simulation conducted under the FSCD component was that it covers mass casualty management which is a prime and massive responsibility of the FSCD and its community volunteers. All of 136 (Female 40 and Male 96) interviewees in 8 fire stations covered by the evaluation gave the highest rating to the question on relevance of the CSSR/FF/FA training course (topics, materials, presentations) under the FSCD component. The findings presented below support these figures. The full results are contained in Annex H. A summary of results is presented in Table 9 followed by brief descriptions. Table 9: Results of Quantifiable Questions on Relevance of Tasks under FSCD Component Tasks Total Respondents (F=Female M=Male) Number of Respondents Gave Highest Rating (F=Female M=Male) Percentage Notes CSSR/FF/FA Training Course 136 (F40+M96) 136 (F40+M96) 100 136 respondents from 8 fire stations covered by the field interviews in 8 out of 17 SERB fire stations Simulation/Drill 81 (F24+M47) 77 (F24+M53) 95 81 respondents from 5 out 8 fire stations. No simulation conducted at the 3 SERB target fire stations of Gazipur, Mymensingh and Tangail cities. SAR Tools and Equipment 136 (F40+M96) 136 (F40+M96) 100 136 respondents from 8 fire stations covered by the field interviews in 8 out of 17 SERB fire stations i. Advanced/Refreshers Training The volunteers endorsed all 16 CCSR/FF/FA lessons as being important but they need to be updated and practiced on a regular basis, maybe twice a year. The training course and simulation are a direct benefit to the participating fire stations and community volunteers in responding to the MCIs. This SERB-funded CSSR/FF/FA course is a modified version of the regular CSSR course offered by the FSCD. The course offered by SERB has three lessons including gender sensitiveness in disaster response, incident command system, and maintenance of SAR tools and equipment in addition to those of the regular CSSR course. According to the FSCD volunteers, the most popular lessons taught in the CSSR/FF/FA course were SAR (practical), earthquake response, and equipment use and maintenance. They found the first aid lesson to be essential but strenuous. Recommendation SERB PERFORMANCE EVALUATION REPORT 36 | P a g e No. 12: If there to be a future MCM activity it is recommended that the SAR training shall include lessons, like, gender sensitiveness in disaster response, incident command system, and maintenance of SAR tools and equipment be taught to develop a pool of SAR trainees. ii. SAR Simulation The FSCD urban community volunteers participated in the SAR simulations as part of awareness raising, interpretation of different levels of ICS, prepare for and time to move to the MCI sites upon receipt of message from the fire stations, techniques to protect lives and assets, and orientation on the operation of SAR tools and equipment. The respondents reported that all these topics help improve their skills and capacities to prepare for natural and human made disasters. All of 69 interviewees in 5 fire stations covered by the evaluation gave the highest rating to the question on relevance of the SAR simulation exercises under the FSCD component. As the volunteers provide these services voluntarily, there was no direct financial incentive for them. However, these youths have developed a self-esteem and recognition in the community which they value significantly. In the long run, the volunteers may not work together and some may migrate to other places but the knowledge and information remain with them and will pass on to other community people. This group offered two recommendations: first, stronger linkages to the national cadet core, ansar troops (para militia), village defense police (VDP), etc. and, second, additional training and equipment, including a first aid kit. Recommendation No. 13: FSCD shall: (i) advocate for establishment of linkages between urban community volunteers and the national cadet core, ansar troops (para militia), village defense police (VDP), etc.; (ii) organize SAR refreshers and simulations for the volunteers; and (iii) provide light SAR tools and first aid kit to the volunteers. iii. SAR Equipment and Tools The SAR tools and equipment (191 sets) provided by SERB are for light duty and complements the stock of heavy SAR equipment received from the government and other donors. There is a desire from volunteers and a need for more equipment for fire stations. The light SAR tools and equipment delivered by SERB to these fire stations seems inadequate against their need for use in emergency response. The FSCD authority reported the need for more equipment for fire stations. The SAR tools and equipment provided by SERB are for light duty and complements the stock of heavy SAR equipment received from the government and other donors. There is a desire from volunteers and a need for more equipment for fire stations. The fire stations mentioned about some essential tools and equipment such as hydraulic cutter, door opener, air lifting bag, ram jack, pipe squeezer, big size electric wire cutter which were not provided in the equipment package from SERB. All the interviewees at fire stations covered by the evaluation gave the highest rating to the question on relevance of the SAR tools and equipment provided by SERB to FSCD fire stations. SERB PERFORMANCE EVALUATION REPORT 37 | P a g e Recommendation: No. 14: Compared to the need of SAR tools and equipment, such as, hydraulic cutter, door opener, air lifting bag, ram jack, pipe squeezer, big size electric wire cutter for the FSCD fire stations SERB should continue distribution of tools and equipment. The provision of tools and equipment shall be included if there to be a future MCM activity. VI.D. Effectiveness The evaluation collected information about effectiveness of tasks of SERB (training courses, simulations, SAR tools and equipment, HRAs, ERPs, etc.) in responding to unplanned events (earthquakes, fire, building collapse, road accidents, disasters, etc.). The evaluation findings followed by recommendations are provided in this section. In addition, this section presents findings of the effectiveness of SERB interventions in contributing to the USAID/Bangladesh DO4: Responsiveness to Climate Change Improved. VI.D.a. Effectiveness of Interventions of HOPE Component i. HOPE-B, HOPE-T and HICS Training Out of 72 interviewees in 8 hospitals covered by the evaluation 57 (79%) gave the highest rating to the question on effectiveness of the training courses (topics, materials, presentations) under the HOPE component. Table 10 provides a summary of results. The full results are contained in Annex H. Table 10: Results of Quantifiable Questions on Effectiveness of Tasks under HOPE Component Tasks Total Respondents (F=Female M=Male) Number of Respondents Gave Highest Rating (F=Female M=Male) Percentage Notes HOPE-B, HOPE-T and HICS Courses 72 (F39+M33) 57 (F32+M25) 79 72 respondents from 8 hospitals were covered by the field interviews in 8 out of 16 SERB hospitals Hospital Risk Assessments 72 (F39+M33) 72 (F39+M33) 100 72 respondents from 8 hospitals were covered by the field interviews in 8 out of 16 SERB hospitals Hospital Emergency Response Plans 56 (F28+M28) 56 (F28+M28) 100 56 respondents from 6 hospitals; 2 hospitals (BITID and SKH) are not targeted by SERB for ERP development Study Visit 9 (F4+F5) 9 (F4+F5) 100 Question on effectiveness of study visit covered in STAMCH only; Other 7 hospitals were covered prior to the study visit. The findings presented below support these figures. This shows that all groups agreed that training courses were highly effective because of their practicability. The majority of responses to the evaluation interviews and meetings were very supportive of the training courses, wanting more of them, wanting refresher and advanced training. These indicate that the courses are interesting, useful and effective. The pool of HOPE and HICS graduates produced by SERB is found to be skilled and confident in responding to MCI. The graduates reportedly responded well during their hospitals‘ management of SERB PERFORMANCE EVALUATION REPORT 38 | P a g e casualties such as the gas explosion in a ship breaking factory in Chittagong, bomb explosions in Sylhet, road accidents on the airport road at Dhaka. These are discussed in the lessons learned section of this report. The hospitals involved in managing the incidents are the Chittagong Medical College Hospital (CMCH), Sylhet MAG Osmani Medical College Hospital (SOMCH), and Kurmitola General Hospital (KGH). There is another example of effectiveness of the HOPE/HICS training and ERP development tasks of SERB. ShSMCH at Dhaka separated the casualty service from the emergency department to allow them to efficiently handle MCI. The evaluation gathered about the instances that some hospitals responded to unplanned events such as, fires, building collapse or road accidents but those were not documented by the responders nor by the hospitals. Recommendation No. 15: Given the importance of systematically maintaining the information on the effectiveness in responding to unplanned events on a regular basis, the hospital management, specifically the casualty department, shall establish systematic collection of information and documentation. ii. Hospital Risk Assessments (HRA) and Emergency Response Plans (ERP) The goal of the HRA is to enable the management and other hospital personnel to have a better understanding of risk factors in their facility and utilize the result of the assessment in developing corrective action plans to improve hospital safety from disasters. The HRA provided a preliminary diagnosis of the hospital‘s safety and capacity to provide services in the event of emergencies and disasters. Every interviewee, 72 out of 72, in 8 hospitals covered by the evaluation gave the highest rating to the question on effectiveness of the HRAs. The full results are contained in Annex H and the summary results are presented in Table 10 above. HRAs are the basis of developing emergency operation plans, and of establishing practical command and control mechanisms that will lead the hospitals response efforts during an incident where effective utilization of limited hospital resources is critical. These actions are part of a continuous process of consultation and dialogue with the hospital management, the hospital disaster committee and the DGHS, with the conduct of simulation exercises to validate revised hospital operational plans and newly developed response procedures. The ERPs among other chapters include: mass casualty preparedness and response system; response operation procedure; resources, services and facilities available at the hospital; functions of hospital incident management system; roles and responsibilities of hospital incident command team; contingency plans and their purposes; emergency response plan validation; and key policy and guidance references. Most importantly the response operation procedure (ROP) section of the ERP, developed by following the WHO Hospital Emergency Checklist, focuses on phases and procedures of emergency response in the context of management of mass casualties by sudden onset disasters. The ROP covers the standing operating procedures to carry out the emergency response operation from the phase of preparation in pre-disaster time to the phase of recovery in the post-disaster time. The ShSMCH, CMCH and SOMCH have found their respective hospital‘s ERP useful in the activation of hospital incident command system in responding to incidents. SERB received requests from the Sarkari Karmachari Hospital (SKH) at Dhaka for assisting the hospital in developing their ERP. These findings support the opinion of all 80 interviewees who gave the highest rating to the question on effectiveness of the HRAs (Refer to Table 10). SERB PERFORMANCE EVALUATION REPORT 39 | P a g e SERB PERFORMANCE EVALUATION REPORT 40 | P a g e Recommendation No. 16: All hospitals shall have ERPs developed and for that they shall first conduct HRA. The ERP shall serve as a living document containing reference tool for hospital incident management system for responding to mass casualty incidents (MCI). iii. Study Visit According to the study visit team, the visit provide not only learning opportunity to understand the practical application of hospital preparedness and hospital incident command system, but it prompted the team to develop action plans. Question on effectiveness of study visit as per responses of 11 out of 12 interviewees at STAMCH is fully effective (Refer to Table 10). The respondents reported that the study visit to Nepal was a great learning opportunity to understand the practical application of hospital preparedness and hospital incident command system (HICS) knowledge in case of a real disaster. Refer to Annex G for full results. Based on lessons learned and experiences gained in the study visit, the team members have developed action plans on 12 tasks. Among these, the most common plan is proposed by 10 out of 15 members on proper storage of supplies for emergency followed by two equal number (9 each) of action plans on development of ERP and establishment of disaster management plan. Other proposed action plans include: training of hospital staff for disaster awareness and capacity building (7), develop and conduct MCM simulation exercises (7), preparation of contingency plan involving budget allocation for equipment, drugs, food and water for incident victims and hospital staff (7), and establishment of digital communication network of key disaster response agencies (6). The Action Plan list is provided at Annex I. Recommendation No. 17: Hospitals are recommended to conduct the feasibility of actions proposed in the action plan and implement those that are feasible. VI.D.b. Effectiveness of Interventions of FSCD Component The training course in CSSR/FF/FA, SAR simulations and SAR tools and equipment provided by SERB are found to be effective according to the FSCD staff and urban community volunteers while responding to hazards and accidents including earthquake shocks, building fires, road traffic accidents and building collapses. Respondents gave the highest rating to the question on effectiveness of the tasks of FSCD component. The summary results are shown in Table 11 below and the full set of statistics are provided at Annex H. Table 11: Results of Quantifiable Questions on Effectiveness of Tasks under FSCD Component Tasks Total Respondents (F=Female M=Male) Number of Respondents Gave Highest Rating (F=Female M=Male) Percentage Notes CSSR/FF/FA Training Course 136 (F40+M96) 136 (F40+M96) 100 136 respondents from 8 fire stations covered by the field interviews in 8 out of 17 SERB fire stations Simulation/Drill 81 (F24+M47) 81 (F24+M47) 100 81 respondents from 5 out 8 fire stations. No simulation conducted at the 3 SERB target fire stations of Gazipur, Mymensingh and Tangail cities. SAR Tools and 136 136 100 136 respondents from 8 fire stations SERB PERFORMANCE EVALUATION REPORT 41 | P a g e i. Advanced/Refreshers Training The evaluation finds that the FSCD volunteers trained under SERB have responded within and outside their communities to a number of disasters, such as earthquake shocks, building fires, road traffic accidents and building collapses. The volunteers moved on to volunteering with the fire service teams during emergencies. As because of their skill and knowledge gained from the training and simulation and access to the SAR tools and equipment, the volunteers have made their response missions successful. The case study booklet prepared by SERB has more examples with details of these response actions of the volunteers. The trainings, drills and simulations were all reportedly successful and geared towards the needs of the intended beneficiaries in developing and strengthening their institutions‘ capacities including that of the individual. All these trainings and simulations are monitored carefully for their quality and set standards. The teams are found to be prepared and when the call is needed for them to respond to an emergency or any unplanned events, they will be able to do the right thing as per their trainings. ii. SAR Simulation and Drill Simulations and drills are found to be effective methods for training and for evaluating or validating preparedness and response efforts in a variety of areas. Individual trainees were taught to use and operate the right equipment in different situations presented in the different practical exercises. These have been extremely well received by all parties covered by the evaluation. The effectiveness was assessed to see how the community volunteers were able to respond to unplanned events, i.e., in response to fire, building collapse, etc. Simulations have been one of the major success aspects of SERB. Most organizations and individuals requested more of these SAR simulations. iii. SAR Equipment and Tools Having seen the usefulness of the SAR tools and equipment during different responses by the FSCD fire stations and community volunteers, discussed in the foregoing section, there are growing demands for the tools and equipment for emergency response. One of the female volunteers said, ―We feel proud that search and rescue equipment has been allocated specially for us. We are using this equipment during emergencies, and our confidence level has increased unimaginably.‖ (Source: SERB Case Study Booklet) VI.D.c. Effectiveness of Contributions of SERB interventions to the USAID/Bangladesh Development Objective SERB aims to improve GOB and Bangladeshi communities‘ disaster response skills while simultaneously increasing earthquake resilience through adaptation of technologies and enhancement of the capacity of hospital personnel, fire service, and urban community volunteers. These tasks of SERB are linked to the USAID/Bangladesh‘s DO4 Sub IR 4.2.2: Increased Climate Resiliency through Technologies Adopted, and other PL Law 480 Title II activities that promote disaster risk mitigation technologies and work with vulnerable communities to help them recover from shocks associated with experiencing a natural disaster. The successes of SERB are thus found to be a valuable input to the achievement of USAID/Bangladesh‘s DO4 Sub IR 4.2.2. VI.E. Replicability of Lessons Learnt and Best Practices Equipment (F40+M96) (F40+M96) covered by the field interviews in 8 out of 17 SERB fire stations SERB PERFORMANCE EVALUATION REPORT 42 | P a g e As part of evaluating the replicability of tasks of SERB, specifically training courses, development of ERPs, study visit, SAR simulations, and SAR light tools and equipment those were found to be highly relevant and effective, the factors of willingness and interest of the individuals, institutions and policy makers to continue after SERB concludes its tasks have been analyzed. The respondents reported that the benefits derived from the interventions have far exceeded their expectation and they are however under the impression that there still remains rooms for improvement. The potential area of replication is in the pre- and post￾emergency context and in areas vulnerable to earthquake, fire, road traffic accidents, and building collapse which are subject to mass casualties. Following are, based on the findings of relevance and effectiveness of the tasks of SERB discussed in the forgoing sections, key lessons and best practices which have high promise for replication. Respondents gave the highest rating to the question on replicability the training and simulation tasks implemented under both HOPE and FSCD components. The full results are contained in Annex H. The summary of results is presented in Table 12 followed by brief descriptions. Table12: Results of Questions on Replicability of Training Courses of HOPE & FSCD Components Tasks Total Respondents (F=Female M=Male) Number of Respondents Gave Highest Rating (F=Female M=Male) Percentage Notes HOPE-B, HOPE-T and HICS Courses 72 (F39+M33) 63 (F32+M31) 88 72 respondents from 8 hospitals were covered by the field interviews in 8 out of 16 SERB hospitals CSSR/FF/FA Training Course 136 (F40+M96) 136 (F40+M96) 100 136 respondents from 8 fire stations covered by the field interviews in 8 out of 17 SERB fire stations Simulation/Drill 81 (F24+M47) 81 (F24+M47) 100 81 respondents from 5 out 8 fire stations. No simulation conducted at the 3 SERB target fire stations of Gazipur, Mymensingh and Tangail cities. Study Visit 9 (F4+M5) 9 (F4+M5) 100 Question on relevance of study visit covered in STAMCH only; Other 7 hospitals were covered prior to the study visit. The findings presented below support the statistics presented at Table 12 on the replicability of HOPE￾B/HOPE-T/HICS training and study visit under the HOPE component and Advanced CSSR and SAR simulation exercises under the FSCD component of SERB. a. Greater impact of integrated interventions SERB under its HOPE component targets to build the MCM capacities of the hospitals through integration of training, risk assessment, and emergency response planning. This integrated approach shows positive results as hospitals reporting that they have gained knowledge and skills and established systems to respond to emergencies, especially MCIs. Likewise, the package of CSSR/FF/FA training, SAR simulation and light SAR tools and equipment are found to be essential to an emergency and its potential humanitarian impact. These packaging of interventions that are much needed have potential for replication. SERB PERFORMANCE EVALUATION REPORT 43 | P a g e b. Structured solution of nonstructural problem The HOPE focal point of one of the SERB target hospitals reports, "After receiving Hospital Preparedness for Emergencies (HOPE) and Hospital Incident Command System (HICS) training, we discussed them with our hospital director along with other HOPE graduates. We established a committee to formulate a way forward to address some issues of hospital preparedness. It was followed by conducting hospital risk assessment with technical support from Asian Disaster Preparedness Center (ADPC). The first thing we did to manage mass casualty was to make sure the hospital’s corridors are clear. As trained doctors, we have to consider how we manage mass casualty. We have now separated the casualty service from the emergency department because casualty patients hamper emergency services. For this reason, the emergency department has been moved to the front of the hospital building. While carrying out the risk assessment, we also noticed how water logging affects hospital services. There was a lavender garden, which was turned into a lake garden by developing proper drainage system. The water logging problem has been resolved”. This is found to be a best practice. It has potential for replication. c. Study visits promote instant actions The study visit to Nepal was not only the learning opportunity to understand the practical application of hospital preparedness and hospital incident command system but prompted development of action plan. Soon after their return to the posts, the participants developed action plans for their own organizations. One of the participants already announced a training course on major disasters, especially earthquake, for workers of diverse health organizations. Thus cross visits to countries, organizations and communities have potential for replication. d. Practical training makes a big difference Simulations are found to be a fast and practical method for training and for evaluating or validating preparedness and response efforts. Individual trainees are taught to use and operate the right equipment in right situation. This practical learning exercise is very popular among the people and organizations and merits its replicability. e. Self-Initiation to Humanitarian Assistance The pool of trained FSCD urban community volunteers is invariably a valuable resource in emergency response in the country. Yet the volunteers provide these services voluntarily, there was no direct financial incentive for them. These youths have developed a self-esteem and recognition in the community which they value significantly. Interestingly, most of the fire stations have no system of tracking the drop out of volunteers. Some stations lack up-to-date register of volunteers. This will be a serious problem for the concerned fire stations as they would not know the exact number of volunteers should there be a need for response. However, there are promising solutions as presented by the volunteers. Following examples of self-initiation by the volunteers in humanitarian assistance provide the government and community a serious learning. i. Community Volunteerism by volunteers: A group of volunteers registered with the Taltola fire stations at Sylhet has been doing social works in addition to their SAR service in their community. The volunteer group trains the students of a number of schools in basic lessons on fire fighting, self-safety, etc. ii. Volunteer tracking volunteers: A small group of volunteers at the Taltola fire station of Sylhet maintain the list and contacts of the volunteers. The volunteers keep routine contacts with the volunteers and convey messages to them through the mobile phone text message and face book group set by them. SERB PERFORMANCE EVALUATION REPORT 44 | P a g e This group of volunteers also established a database of all (5,238) volunteers registered with the two fire stations in Sylhet. These actions of volunteers tracking volunteers are found to be self-motivated and innovative and have the potential for adoption by other fire stations. Recommendation No. 18: FSCD shall streamline the self-initiation community services of the volunteers as they are found to be innovative and have the potential for adoption by the fire stations. f. HOPE instructor leads emergency response One of the HOPE instructors at the Chittagong Medical College Hospital (CMCH) relays a situation on the adaptation of guidelines taught in the HOPE training supported a response in her ward. “There was a gas tank explosion in a ship breaking factory; twenty people were expected to come to the ward at one time with multiple injuries.” The HOPE instructor explains, “I was able to use the plan that had been developed and practiced in the HOPE training”. The Director informed all staff via the designated SMS about a mass-casualty event and there was one and a half hour to prepare. This HOPE instructor (doctor) led the response for her ward. “All the equipment and medicines were prepared, designated a separate area in the ward where triage and marking would take place. All were systematically prepared”, she described and also added, “My professors came to support and advise but I was leading the ward, I felt proud of my work”. She adds, “Before the HOPE training, I was not aware of how to respond in these situations”. This HOPE instructor is part of the strong resource pool of qualified trainers in Bangladesh. g. Adoption of HOPE/HICS knowledge and skills in managing MCI The resources of HOPE/B and HICS graduates and ERPs developed by SERB at the Sylhet MAG Osmani Medical College Hospital (SOMCH) have been utilized by the hospital as part of the hospital’s MCM when bomb explosions ripped through a crowd killing six people and injuring around 50 others in Bangladesh's north-eastern city of Sylhet on 25th March 2017. It was a dynamic controlled regional event and Level-II Potential Injury Creating Event (PICE)14. SOMCH is about five kilometers away from the incident site. It is a tertiary level government hospital in the Sylhet division. An onsite medical team of SOMCH was the medical first responder of the incident. They started field triage immediately after the blast and sent severely injured people to hospital. The experienced and trained multi-disciplinary team of doctors, nurses, technicians, infrastructure and equipment handled this MCI effectively and efficiently. VI.F. Gender Responsiveness in SERB SERB, based on the coordinated efforts of ADPC, NIPSOM, participating hospitals and FSCD, has achieved the targets of women as trainees and trainers in the training courses, simulation exercises, HRAs and study visit. Of the total staff of the 21 hospitals under SERB, an estimated 40% are female. The HOPE-B, HOPE-T and HICS graduates, HRA participants and study visit team members represent 30% female. The total 20,762 FSCD urban community volunteers in the 21 fire stations covered by SERB have 24% female and the total female volunteers trained in CSSR/FF/FA training and SAR simulations have 23% female. 14 Potential Injury Creating Event (PICE): The nomenclature devised to simplify the understanding of responses needed during emergency situations. The characteristics of PICE for Mass Casualty Incident (MCI) include: Potential for additional casualties; Extent of local resource disruption; Geographic boundaries of involvement; and Stage (Level) - Likelihood of needing outside assistance. SERB PERFORMANCE EVALUATION REPORT 45 | P a g e Nearly one-third of a total of 290 (31% Female and 69% Male) course trainers, instructors and resource persons engaged in HOPE-B, HOPE-T, HICS and CSSR/FF/FA training courses is female. The engagement in HOPE component (36%) is ahead of FSCD component (25%). Table 13 has detailed statistics. Yet the number of female trainees/participants of the hospitals and fire stations are close to their respective institutions‘ overall levels of female members; the ratio between female and male HOPE/HICS graduates and FSCD volunteers are not the right balance. So is the performance in case of trainers, instructors and resource persons. SERB has made it a point in advocating and encouraging more female participation in all its tasks. The target of 30%, with exception of 50% in a few cases set by the project for the training courses and simulation exercises, seems low. The rationale being, there is still a need to sensitize decision-makers regarding the importance of empowering female staff and encourage them to participate in emergency response programs. The respondents suggested that the target be set at a higher level (50% - 60%). Recommendation No. 19: SERB shall initiate advocacy and awareness programs to motivate more female members to participate. Since this is a cultural matter in the country – a policy supporting women to take more active participation in activities like SERB needs to be promoted widely. All efforts need to be made to ensure that a sufficient gender balance is maintained. Gender Differentiation 15 About 13% of total (705) staff of SOMCH and 70% of Total (301) staff of TGH are female. Table 13: Statistics of Female Graduates, Volunteers, Trainers and Instructors Components Total Female Female as a % of Total A. HOPE Component - 21 Hospitals (Staff) N/A N/A 40% (estimate)15 1. HOPE-B/HOPE-T/HICS Training (Graduates) 567 219 39 2. Hospital Risk Assessment (Participants) 386 69 18 3. Study Visit (Participants) 15 6 40 Sub-Total: HOPE Component (1+2+3) 968 294 30 B. FSCD Component – 21 Fire Stations (Volunteers) 20,762 4,889 24 1. CSSR/FF/FA Training (Volunteers) 1,200 323 27 2. Simulation Exercises (Volunteers) 650 106 16 Sub-Total – FSCD Component (1+2) 1,850 429 23 Course Trainers/Instructors/Resource Persons (HOPE/HICS Courses) 170 61 36 Course Trainers/Instructors/Resource Persons (CSSR/FF/FA) 120 30 25 Sub-Total – Course Trainers/Instructors 290 91 31 Total (HOPE and FSCD Components) 3,108 814 26 SERB PERFORMANCE EVALUATION REPORT 46 | P a g e The gender integration tasks in addressing the gender differentiation issues include the addition of separate training lessons/modules – ‗Gender Equity Intervention in Hospital Response‘ in the HOPE-B course for the hospital personnel and ‗Gender Sensitiveness During Response‘ in the CSSR/FF/FA courses for the urban community volunteers. Also, SERB has developed a gender-based community mapping of the locations of FSCD fire stations indicating the male and female number of volunteers, male – boys/adults, female – women/girls, in each of the communities. These training modules and gender-based mapping were developed considering the needs of elderly, disabled, women and girls. Vulnerability of Gender: The ERPs developed by SERB among the processes of MCM contain the plan and procedure of sorting injured people into groups based on their gender and severity of conditions. In developing the triage process in the ERPs, SERB integrated gender-related challenges for women, children, elderly and disabled and prioritized handling of patients from these groups of people to reduce gender inequality gaps in accessing assistance and services during disasters. Gender Inequity: As a step to promote equity across all funded tasks SERB facilitated the training courses with emphasis on cultural sensitivities which were in-grained in all levels. Women are encouraged to participate in all tasks both inside the classrooms and during outdoor drills/simulations. The senior hospital management has been sensitized to facilitate and support women staff to participate in the training. The training instructors have been sensitized to be unbiased on the delivery of training lessons. Recommendation No. 20: SERB shall coordinate with hospitals to engage women members in active dialogue with hospital management and staff in hospital emergency response plans. Hospitals with the focus on the gender equity shall assign appropriate number of women officials and staff in the hospital incident management team (IMT) referred to in the ERPs. Gender Marginalized in Emergency Response: Female participation in the overall program implementation is still low in terms of coverage and numbers yet the coordinated efforts of SERB, NIPSOM, participating hospitals and FSCD help achieve the project targets of women as trainees and trainers in the training courses, simulation exercises, HRAs and study visit. The SERB stakeholders suggested that the activity addressed gender violence, sexual harassment, etc. that are commonly faced by women during disasters. Recommendation No. 21: Since it is very important having more female participation in the program to cater for the needs of female disaster survivors, women, girls and boys shall be empowered and capacitated so in the event of emergencies or disasters, they will be prepared and ready to provide the needed assistance. Concerned authorities of the hospitals and FSCD shall engage male staffs in the discussion of addressing gender violence during emergency response. VI.G. Coordination and Collaboration SERB is implemented in partnership agreements with NIPSOM responsible for HOPE component and with FSCD responsible for FSCD component. Effective coordination among program implementing partners has been assessed by examining the working relationship and collaboration amongst them. For SERB, NIPSOM works under the guidance of DGHS of MOHFW which deals with health and family welfare related services. The selection of HOPE-B/HOPE-T/HICS training participants are finalized in coordination with SERB PERFORMANCE EVALUATION REPORT 47 | P a g e NIPSOM, DGHS and participating hospitals. The trainees under FSCD component are selected in coordination with FSCD HQ at Dhaka and participating fire stations. These arrangements of selection of trainees are reportedly working well. Besides NIPSOM, FSCD and DGHS, the ADPC SERB team has sound collaboration at the national level with the DDM of the Ministry of Disaster Management and Relief (MOMDR) and the Bangladesh Red Crescent Society (BDRCS). At the field level, SERB maintains close collaboration with the hospitals, fire stations, government officials and private stakeholders including NGOs. This local level collaboration is found to be effective in organizing the training and simulation events fruitfully and disseminating the tasks of SERB widely. The project is dedicated for the public institutions and has successfully established strong working relationship with them at the field and national levels. To provide guidance, give strategic direction and review project implementation, SERB has a Steering Committee, comprising of the DDM, USAID, DGHS, Focal Agencies of HOPE and FSCD (NIPSOM and FSCD), ShSMCH and SERB COP and Training Manager. The Chair of the committee is the Director General of DDM, the nodal agency focal point. The committee meets twice in a year and reviews the project implementation status, challenges and issues, takes decisions and follows up recommended actions. The evaluation finds that these factors are key to efficient coordination for augmenting smooth project implementation. The steering committee reportedly suffers from a lack of required level of representation of the committee member organizations. The committee is found to be represented by a few female officials. However, the committee will operate until the end of SERB. Recommendation No. 22: The evaluation suggests that the steering committee member organizations shall ensure appropriate level of representation and gender-balanced participation in the meetings for augmenting smooth project implementation. VI.H. SERB Training Institutionalization Notably, the positive feedback on the relevance and effectiveness of the SERB-conducted training courses (Refer to Section VI.C –Relevance, Section V.D – Effectiveness and V.E – Replicability) have good prospect for adoption and replication in the country. Finally, the SERB-conducted training courses seem to have worked well. The adoption and replication of both HOPE and CSSR training can be done in other areas of the country to include other types of hazards with focus on different DRR elements. In the event that there will be no more donor funding, NIPSOM needs to plan in consultation with ADPC on how to integrate the course into their existing system. FSCD is a service oriented first responding government organization. FSCD on its own initiative conducts CSSR training for its urban community volunteers, is a part of the GOB commitment to develop 62,000 urban community volunteers16. To date, FSCD has over 35,000 (56%) registered volunteers in about 60 fire stations in the country. These volunteers had to complete the basic SAR training which was the prerequisite to qualify for registration as volunteers. Out of the 35,000 registered volunteers 30,000 received SAR under CDMP and the remaining 5,000 were trained by FSCD. There are still 27,000 volunteers to be developed out of the total commitment of 62,000 volunteers. 16 Source: file:///E:/SERB%20INTERNAL%20EVALUATION%202/SERB%20EVALUATION%20REPORT/Urban%20Community%20Vol unteers%20ready%20to%20tackle%20urban%20disasters%20_%20Bangladesh%20_%20Save%20the%20Children.html SERB PERFORMANCE EVALUATION REPORT 48 | P a g e Most recently FSCD under a contract with Save the Children in Bangladesh has trained 300 urban community volunteers residing in Dhaka and Narayanganj on firefighting so that they can provide firsthand support to the affected people even before the arrival of the professionals. These training programs have the potential for replication by the public educational institutions and industrial installations. SERB‘s training in HOPE/HICS and CSSR/FF/FA has generated considerable interest in the private sector - hospitals, educational institutions, industries and factories. It is, therefore, worth exploring private sector investments in this humanitarian effort as part of their Corporate Social Responsibilities (CSR). Recommendations No. 23: Both NIPSOM and FSCD shall prioritize to adopt the training courses they have implemented under the external project assistance into their respective institutional framework. Encourage and influence both public and private sectors to sponsor these training courses to develop disaster resilient hospitals. For example, adopt course modules of SERB training, as appropriate, in the post graduate academic curriculum. No. 24: FSCD shall drive for achieving development of 100% of the target 62,000 volunteers. If required, FSCD shall explore international donor funding opportunities for development of the volunteers. VI.I. Pool of HOPE/CBDRR Graduates/Instructors (2003-2017) Bangladesh‘s history of integrated basic, refreshers and advanced training in capacity building of hospital preparedness for emergencies (HOPE) and community based disaster risk reduction (CBDRR) goes back to 2003 when the USAID-funded Program for Enhancement of Emergency Response stage two (PEER II) began in the six countries in the South-Asia region aimed at development of HOPE, CSSR and MFR (Medical First Responder) graduates and instructors. Under PEER II (2003 – 2009) a pool of over 300 graduates and instructors in HOPE; MFR and CSSR were developed in the country. This pool of humanitarian assistance resource was further increased to exceeded 1,000 including HOPE, MFR, CSSR and a new course CADRE (Community Action for Disaster Response) under stage three of PEER (2009-2014). To date, Bangladesh has over 2,700 graduates and instructors developed by these past and ongoing USAID programs including SERB (2013 -2017), PEER IV (2014 -2019) and HOPE South Asia (HOPE-SA) (2016 – 2018). Table 14 has more detailed statistics of these achievements. Table 14: Pool of USAID-Supported HOPE/CBDRR Graduates/Instructors in Bangladesh (2003 -2017) SERB PERFORMANCE EVALUATION REPORT 49 | P a g e VI.J. Activities Synergistic to SERB Interviews with the government, non-government and international organizations and through review of secondary sources, the evaluation found a substantial number of works on disaster preparedness and response, skills development, and capacity building of the public health facilities in Bangladesh. Refer to Attachment 2 of Annex B for a list of secondary sources. These include development of the first ever Hospital Emergency Preparedness and Response Plan (HEPRP) for hospital based health managers by the DGHS with technical assistance from WHO in 2011. In the same year, the DGHS with the technical support of WHO published for the first time a guidebook for emergency health personnels entitled, ―Health Sector Contingency Plan for Earthquake Preparedness and Response (HCPEPR)‖. During the same period the DGHS issued a Guideline for Hospital Nonstructural Vulnerability Assessment under the joint GOB, bilateral and multilateral funded Comprehensive Disaster Management Program (CDMP-2). Prior to publication of the HEPRP and HCPEPR, ActionAid Bangladesh, in cooperation with DGHS under funding support of the European Commission Directorate General for Humanitarian Aid (DG ECHO), developed a training manual for mass casualty management by hospitals and a guideline for hospital contingency plan in 2010. The development of these two and many other MCM related documents for building hospitals‘ capacity in preparedness for and responding to MCI was based on the lessons learned in preparing the first ever contingency plan of the country in 2006 by ActionAid under its DIPECHO project. The 2006 pilot was carried out in three private hospitals in Chittagong. Later in 2007, ActionAid assisted the DMCH and CMCH to prepare their own contingency plans. Annex J contains a list and short briefs of the activities described above along with other related works. Among the most recent and ongoing projects related to MCM is the Urban Resilience Project (URP) funded Japan International Cooperation Agency (JICA) for the period of 2015 – 2020 involving three pillars considered for disaster resilience in urban settings. The pillars are: effective emergency management; reinforce existing infrastructure; and ensure resilient construction. Specific activities in three target city corporations of Dhaka North City Corporation (DNCC), Dhaka South City Corporation (DSCC) and Sylhet City Corporation (SCC) include: 17 Source: NSET PEER News Bulletin No. 12, March 2009 18 USAID/OFDA funded ADPC PEER3 Final Results Report 19 SERB statistics is exclusive of 78 HICS graduates. 20 Trainees of Advance/Refreshers CSSR Course 21 Information not available till the time evaluation report preparation. PROGRAM HOPE TRAINING CBDRR TRAINING ALL TRNG Basic TFI HOPE TOTAL TOTAL MFR CADRE CSSR CBDRR TOTAL Basic IW Basic IW Basic IW PEER II17 (2003-2009) 90 9 99 89 13 - - 72 34 208 307 PEER III18 (2009 – 2014) 162 60 222 56 60 233 71 48 40 508 730 SERB19 (2013 – 2017) 415 52 467 - - - - 1,20020 - 1,200 1,667 PEER IV21 (2014 – 2019) - - - - - - - - - - - HOPE-SA (2016 - 2018 22 - 22 - - - - - - - 22 ALL PROGRAMS 689 121 810 145 73 233 71 1,320 74 1,916 2,726 SERB PERFORMANCE EVALUATION REPORT 50 | P a g e  Build disaster risk management and emergency response capability by establishing EOC, satellite control rooms, and DRM office Zone level.  Enhance emergency management planning and response capability of FSCD by establishing fixed and mobile emergency control and command rooms.  Increase institutional capacity to manage and respond emergencies through exercises, drills, etc. Comprising of five interrelated components, the Urban Resilience Project of Bangladesh (2015 – 2020), funded by the World Bank, has an objective to strengthen the capacity of GoB agencies to respond to emergency events and to strengthen systems to reduce the vulnerability of future building construction to disasters in Dhaka and Sylhet. The five components are:  Component A - Reinforcement of an emergency management system which will be put in place and will mobilize the resources at all levels and assign roles and responsibilities more efficiently.  Component B - Development of the consensus-driven analytical foundation required for longer-term investments to reduce risk in the built environment of Dhaka, Sylhet and other cities in Bangladesh.  Component C – Establishment of the institutional infrastructure and competency to reduce long￾term disaster vulnerability in Dhaka.  Component D - Provision of necessary funding for project coordination, monitoring and evaluation.  Component E – Reallocation of project funds and designate them as Immediate Response Mechanism funds to be engaged to partially cover emergency response and recovery costs. With regards to the works of the FSCD related to MCM the key programs include development of urban community volunteers who are trained in search and rescue, firefighting and first aid (CSSR). The FSCD runs several training programs under government and project funding and on payment basis. These include CADRE, occupational safety and fire safety manager course. The latter is a six-moth training course and offered on payment of around Tk. 25,000 per participant. Simulations and drills on SAR, Fire Fighting are other interventions the department and its volunteers implements for hospitals, factories, educational institutions. VI.K. SERB Outcomes and Impact The overarching objective of SERB is to increase GOB and communities disaster management skills and earthquake resilience. As such, SERB will enhance the capacity of hospital personnel, FSCD, and related institutions and target urban communities to enable them to respond effectively and efficiently to the survivors‘ needs during major natural disasters, specifically MCI, such as, earthquake. SERB activity targets the male and female health professionals responsible for hospital operations at selected public hospitals, located in the cities/municipalities of Dhaka, Chittagong, Sylhet, Gazipur, Mymensingh and Tangail and urban community volunteers currently being trained by the FSCD in these cities/municipalities. SERB under its HOPE component targets to build the MCM capacities of the hospitals through integration of training, risk assessment, and emergency response planning. This integrated approach shows positive results as hospitals reporting that they have gained knowledge and skills and established systems to respond to emergencies, especially MCIs. Likewise, the package of CSSR/FF/FA training, SAR simulation and light SAR tools and equipment are found to be essential to an emergency and its potential humanitarian impact. These packaging of interventions that are much needed have potential for replication. Recommendation SERB PERFORMANCE EVALUATION REPORT 51 | P a g e No. 25: In any future MCM program the evaluation recommends adoption of the integrated intervention approach of SERB which includes development of human resources and provision of tools and equipment to respond to MCI. Yet the impact of the tasks implemented by SERB in last four years is still not visible. However, the analysis of the outcomes as a result of managing different emergencies with the application of knowledge and skills imparted by SERB show a positive sign of impact. Also, it is too early to see the impact of the tasks such as the HRA findings dissemination and study visit that have recently been completed as well as the HOPE-B and CSSR/FF/FA trainees of the courses held in the final year of the program. The following matrix portrays the impact expected from the tasks implemented by SERB. Plan (Input) Progress (Output) Outcomes (Results) Indicators of Measurement of Outcomes Expected Impact  Training and capacity building in emergency preparedness to the medical and non-medical staff of the hospitals for development of skills in mass casualty management  Training to the urban community volunteers on use of S&R tools and equipment and their maintenance.  Deliver SAR equipment to the FSCD fire stations for readiness in SAR operations in emergencies  Produced a pool of 545 graduates in HOPE-B, HOPE-T and HICS courses  Conducted 16 Hospital Risk Assessments (HRAs)  Produced 7 Hospital Emergency Response Plans (H-ERPs)  Trained 1,200 FSCD urban volunteers in advanced CSSR  Delivered 151 set of light SAR tools and equipment to 17 target FSCD fire stations  Hospital staff and FSCD urban community volunteers developed skill to manage mass casualty incidents  Hospitals and FSCD capacity to and performance in responding to mass casualty incidents  FSCD fire stations and urban volunteers access to search and rescue tools and equipment Performance of trained hospital personnel in handling casualties of localized disasters such as fire at garment factories, road transport accidents. Performance of trained FSCD volunteers and their access to SAR equipment in SAR in localized incidents. Routine and proper Maintenance of SAR tools and equipment received under SERB Responsiveness to the HRA recommendation such as activation of contingency plans, patient management. Responsiveness to ERP guidance and procedure to make the hospital disaster health committee fully functional. The expected outcomes are to contribute towards improved emergency preparedness and response capacities of the participating hospitals and FSCD fire stations in managing mass casualty incidents. The assumption is that completion of training of hospital personnel and community volunteers, and delivery of tools/equipment to FSCD will contribute to the achievements of activity objective is expected to hold true. The above analysis shows a clear evidence of improved preparedness by the SERB target institutions in responding to MCIs. The DRR professionals say, “Better prepared Better response”. VI.L. Toward Sustainability SERB PERFORMANCE EVALUATION REPORT 52 | P a g e As far as sustainability of the SERB activity is concerned, the outcomes mentioned in the matrix under foregoing section shall be sustainable. The contributing factors towards sustainability of these outcomes are found to be institutionalization of training courses, capitalization of resource pool, maintenance and utilization of SAR tools and equipment, endorsement of HRA findings and recommendations, GoB buy-in and ownership of tasks of SERB. These are discussed below. i. Institutionalization of SERB Training Courses: With respect to the contribution of institutionalization of SERB training, pool of HOPE and FSCD graduates and instructors, the evaluation finds a strong need of advocacy to the public and private sectors to provide financial and technical support and commitment of the policy makers to integrate the training courses to the national curriculum. ii. Capitalization of Resource Pool of HOPE/HICS Graduates and FSCD Urban Community Volunteers: Bangladesh has built up a resource pool of HOPE/HICS graduates, from 307 in 2003 to 2,716 in 2017, which is inclusive of 545 graduates developed by SERB. This is a positive factor of sustainability of tasks of SERB. Apart from these stock of graduates/instructors, completed and ongoing activities of the DGHS with financial support from DG ECHO and technical assistance from WHO, MODMR‘s CDMP-II, DNCC‘s URP funded by JICA, USAID‘s PEER IV and HOPE-SA programs have synergistic effect to the SERB activity. The pool of trained FSCD urban community volunteers is invariably a valuable resource in emergency response in the country. But institutionalization of the volunteers is found to be progressing slowly. The most important factor is that institutionalization of trainees depends on links and networks and the training courses require to forge links, raise awareness, maintain and build networks with all participating institutions, implementing partners, nodal agencies and relevant stakeholders. This is an important factor for the sustainability of the government‘s volunteer development program being implemented since 2008. iii. Proper Maintenance and Utilization of SAR Tools and Equipment: Efficient storage, regular maintenance and proper disposal of equipment and tools help smooth and efficient equipment operation and avoid/limit health hazards. To ensure these maintenance standards, SERB developed a guideline for use by those FSCD fire stations received the SAR tools and equipment. Refer to Annex K – FSCD Tools and Maintenance Guideline. In addition, the volunteers of these fire stations received practical training on SAR equipment preventive and routine maintenance which is one of the lessons of the advanced/refreshers CSSR training provided by SERB. The volunteers are reportedly enjoying their engagement in the SAR equipment maintenance work at their reporting fire stations. iv. GoB Endorsement of HRA Findings and Recommendations: The endorsement of the HRA findings and recommendations by the senior officials of the MOHFW and DGHS is encouraging. However, the turnover of trained personnels from hospitals is found to be a negative factor to the MCM. The turnover is unavoidable as far as public employment is concerned. However, for the HOPE graduates and instructors, the key is to track the turnover information to ensure that the volunteers are not lost and can be contacted to join emergency response. These information need to be tied to the database and networking. v. GoB Buy-in/Ownership of SERB Tasks: Besides endorsement of HRA findings and recommendations, GoB provision of funding support to continue SERB tasks (i.e., HOPE/HICS courses, HRA/ERP and FSCD components) including scaling up and scaling out to other areas in the country is needed. Recommendation No. 26: Based on the factors of sustainability, discussed above, following are sustainability related recommendations for SERB. SERB PERFORMANCE EVALUATION REPORT 53 | P a g e  Advocacy for scaling up and institutionalization of SERB training programs such as basic and refreshers of HOPE, HICS, and CSSR/FF/FA courses by the public and private sectors for MCM.  Recognition of volunteers by the government for their contribution in humanitarian assistance service to inspire them to stay in volunteerism. The concerned organizations, thus, shall require institutionalization of the training received from SERB. The HOPE/HICS graduates, for example, can be absorbed in the proposed EMT. The volunteers, for example, can be absorbed as Fire & Rescue volunteers by the FSCD.  Maintenance of the current level of quality and routine maintenance of SAR equipment with engagement of the trained volunteers.  Identification of an appropriate GOB institution in consultation with the DGHS to establish and own a fully functional database and networking of HOPE graduates and instructors.  GoB provision of funding support to continue SERB tasks (i.e., HOPE/HICS courses, HRA/ERP and FSCD components) including scaling up and scaling out to other areas in the country. VII. SERB Performance Evaluation Recommendations A total of 26 recommendations have been formulated by the evaluation for the remaining life of the SERB as well as way forward when SERB ends. The evaluation recommends for advocacy for scaling up and institutionalization of SERB training programs by the public and private sectors, establishment of network of trainees, increased female participation in MCM training program and enhanced role of female in MCM. The areas of recommendations are grouped as Adoption and Replication, Follow-up Actions, Gender Responsiveness, Institutionalization and Sustainability under two categories of recommendations: Recommendations for implementation within the remaining life of SERB (Category A); and Recommendations for any follow-up activities (Category B). Table 15 has a complete list of recommendations indicating by categories of recommendations under each recommendation group. Table 15: List of SERB Performance Evaluation Recommendations Recommendation Groups Recommendations Recommendation Categories Adoption and Replication No. 1: Consideration should be given, if funding is available, to include the remainder of the public hospitals and fire stations in the cities of Dhaka and Chittagong in the SERB activity. A No. 2: If there to be a future activity like SERB it is recommended that, new cities, namely, Dinajpur, Bogra, Bandarban, Khagrachari, Rangamati and Cox’s Bazar be brought under the activity as they are affected by one or more of the natural and human-made hazards, such as, cyclone, storm surge, tornadoes, flash floods, landslides, cold wave, road accidents, and fire. The focus of the activity shall stretch beyond earthquake but multi-hazard depending on the vulnerability of the cities to the types of hazards for mass casualty management. B No. 3: Given the growing concern on the impact of landslides, it is recommended that the three hill districts - Bandarban, Khagrachari and Rangamati be selected on a priority basis for MCM adopting the SERB model. Both A & B No. 4: The HOPE/HICS graduates being employees of the government health facilities are subject to move from one health facility to another. So, to reduce potential loss of skilled staffs in case of an emergency, a database and network shall be established by NIPSOM to track the HOPE/HICS graduates who leave their posts. The network shall be fully functional in tracking the HOPE/HICS graduates in MCM and there shall be understanding among DGHS, NIPSOM and Both A & B SERB PERFORMANCE EVALUATION REPORT 54 | P a g e Recommendation Groups Recommendations Recommendation Categories participating hospitals that the rotated staffs are brought back to their previous positions for emergency response. No. 5: Compared to the need for training for the FSCD urban community volunteers, SAR refreshers shall be offered at least once a year and simulations shall be conducted at least in every six months to the volunteers. Both A & B No. 6: The training shall arrange separate session for the groups of participants who require special attention for particular topics or lessons. Both A & B No. 7: The training course shall be based on the target participants’ skills, instincts and abilities to understand and adapt and their availability to attend. The suggested training course for the nurses and support staff shall be a revised version of the HOPE-B course, less technical and according to their level, and shorter courses and orientation on HOPE/HICS for the high level decision makers and hospital heads. Both A & B No. 8: For the safety of patients and hospital staff, and the hospitals’ ability to function during and after emergencies or disasters, the concerned hospitals shall explore funding for implementation of the HRA recommended actions pertaining to the structural, nonstructural and functional elements. (c) Hospitals shall:  Improve, enhance and automate the nonstructural elements which include systems of: electricity and lighting in critical areas; telecommunications; water supply; fire protection; waste management; fuel reserve; medical gases; HVAC; and shelving of furnishings and equipment; safety and security of staff and patient; and medical and laboratory equipment and supplies used for diagnosis and treatment..  Develop and implement contingency plans that are required for certain critical events and essential services pertaining to maintenance and inspection, fire safety and fire emergency, flood management, chemical spills management, power failure, hospital evacuation, psychosocial support, hospital business continuity, and personal health and critical incident stress.  Strengthening of the functional capacity of the hospital as a priority action of the hospital in the immediate time for effective responses to the emergencies or disasters. (d) DGHS shall:  Undertake proper retrofitting and remodeling of the structures as early as possible, especially those hospital buildings’ are of concrete frame structures and the overall safety level of structure was found to be average.  Conduct detailed structural assessments of those hospitals assessed to be of low safety level by HRA and take mitigation measures as per assessments.  Establish a National Emergency Medical Team (EMT) based on WHO Classification and Minimum Standards for National Medical Teams in sudden onset disasters. Both A & B No. 12: If there to be a future MCM activity it is recommended that the SAR training shall include lessons, like, gender sensitiveness in B SERB PERFORMANCE EVALUATION REPORT 55 | P a g e Recommendation Groups Recommendations Recommendation Categories disaster response, incident command system, and maintenance of SAR tools and equipment be taught to develop a pool of SAR trainees. No. 13: FSCD shall: (i) advocate for establishment of linkages between urban community volunteers and the national cadet core, ansar troops (para militia), village defense police (VDP), etc.; (ii) organize SAR refreshers and simulations for the volunteers; and (iii) provide light SAR tools and first aid kit to the volunteers. Both A & B No. 14: Compared to the need of SAR tools and equipment, such as, hydraulic cutter, door opener, air lifting bag, ram jack, pipe squeezer, big size electric wire cutter for the FSCD fire stations SERB should continue distribution of tools and equipment. The provision of tools and equipment shall be included if there to be a future MCM activity. Both A & B No. 15: Given the importance of systematically maintaining the information on the effectiveness in responding to unplanned events on a regular basis, the hospital management, specifically the casualty department, shall establish systematic collection of information and documentation. Both A & B No. 16: All hospitals shall have ERPs developed and for that they shall first conduct HRA. The ERP shall serve as a living document containing reference tool for hospital incident management system for responding to mass casualty incidents (MCI). Both A & B No. 18: FSCD shall streamline the self-initiation community services of the volunteers as they are found to be innovative and have the potential for adoption by the fire stations. Both A & B No. 25: In any future MCM program the evaluation recommends adoption of the integrated intervention approach of SERB which includes development of human resources and provision of tools and equipment to respond to MCI. B Follow-up Actions No. 9: Hospitals shall keep the ERP up-to-date and fully functional. Hospitals shall organize orientation on the ERP for the new committee members and staff so that they are aware of and ready for responding to mass casualty incidents (MCI) Both A and B No. 10: SERB shall conduct in-depth analysis of the usefulness and benefit of study visits of the program staff, HOPE instructors and graduates, FSCD instructors and NIPSOM staff to the health facilities and emergency MCM activities within and outside the country in the region. Since executing study visits are expensive, before advocating for them, the program shall determine the feasibility including expected result of the proposed study visit. B No. 11: Hospital shall organize outbriefs of the study visit to disseminate the experience gained and lessons learned from the study visit and shall review and implement the action plan developed by the participants as feasible. A No. 17: Hospitals are recommended to conduct the feasibility of actions proposed in the action plan and implement those that are feasible. A Gender Responsiveness No. 19: SERB shall initiate advocacy and awareness programs to motivate more female members to participate. Since this is a cultural matter in the country – a policy supporting women to take more active participation in activities like SERB needs to be promoted widely. All efforts need to be made to ensure that a sufficient gender balance is maintained. Both A & B No. 20: SERB shall coordinate with hospitals to engage women Both A & B SERB PERFORMANCE EVALUATION REPORT 56 | P a g e Recommendation Groups Recommendations Recommendation Categories members in active dialogue with hospital management and staff in hospital emergency response plans. Hospitals with the focus on the gender equity shall assign appropriate number of women officials and staff in the hospital incident management team (IMT) referred to in the ERPs. No. 21: Since it is very important having more female participation in the program to cater for the needs of female disaster survivors, women, girls and boys shall be empowered and capacitated so in the event of emergencies or disasters, they will be prepared and ready to provide the needed assistance. Concerned authorities of the hospitals and FSCD shall engage male staffs in the discussion of addressing gender violence during emergency response. Both A & B No. 22: The evaluation suggests that the steering committee member organizations shall ensure appropriate level of representation and gender-balanced participation in the meetings for augmenting smooth project implementation. A Institutionalization No. 23: Both NIPSOM and FSCD shall prioritize to adopt the training courses they have implemented under the external project assistance into their respective institutional framework. Encourage and influence both public and private sectors to sponsor these training courses to develop disaster resilient hospitals. For example, adopt course modules of SERB training, as appropriate, in the post graduate academic curriculum. Both A & B No. 24: FSCD shall drive for achieving development of 100% of the target 62,000 volunteers. If required, FSCD shall explore international donor funding opportunities for development of the volunteers. B Sustainability No. 26: Based on the factors of sustainability, discussed above, following are sustainability related recommendations for SERB.  Advocacy for scaling up and institutionalization of SERB training programs such as basic and refreshers of HOPE, HICS, and CSSR/FF/FA courses by the public and private sectors for MCM.  Recognition of volunteers by the government for their contribution in humanitarian assistance service to inspire them to stay in volunteerism. The concerned organizations, thus, shall require institutionalization of the training received from SERB. The HOPE/HICS graduates, for example, can be absorbed in the proposed EMT. The volunteers, for example, can be absorbed as Fire & Rescue volunteers by the FSCD.  Maintenance of the current level of quality and routine maintenance of SAR equipment with engagement of the trained volunteers.  Identification of an appropriate GOB institution in consultation with the DGHS to establish and own a fully functional database and networking of HOPE graduates and instructors.  GoB provision of funding support to continue SERB tasks (i.e., HOPE/HICS courses, HRA/ERP and FSCD components) including scaling up and scaling out to other areas in the country. Both A and B SERB PERFORMANCE EVALUATION REPORT 57 | P a g e VIII. The Way Forward - Future SERB SERB is implemented in the selected seismic zones and focused on earthquake preparedness and response capacities. The strategy and approach that ADPC finds worth for future activities are: Expansion of geographic coverage: New areas beyond the current six cities may include the regions of Chittagong, Chittagong Hill tracts, Comilla, Bogra and Dinajpur. These areas are affected by one or more of the natural and human-made hazards, such as, cyclone, storm surge, tornadoes, flash floods, cold wave, road accidents, and fire. Inclusion of multi-hazard interventions: Earthquake is the focus of SERB. The hazards such as cyclone, storm surge, tornadoes, flash floods, cold wave, road accidents, and fire may be included for MCM. Introduction of specialized training courses: Develop training course such as Incident Management for both health professionals and urban volunteers. This shall be in addition to the courses (HOPE-B, HOPE-T, HICS and advanced/refreshers CSSR/FF/FA being offered now under SERB. Complementing ongoing regional programs: Future activities will be complementary to the USAID/OFDHA‘s HOPE-SA project‘s target of four HOPE-B courses in Bangladesh in 2017. In this regard, through SERB project extension, ADPC can support the participating hospitals in their HRAs, HICS and in the development of their ERPs. Continuation of SERB tasks: Under future programs, a full package of the HOPE component will be implemented (HOPE-B, HOPE-T, HICS, HRAs, ERPs). FSCD will be provided with SAR equipment and training on use of the equipment as part of the package just like in the current phase of SERB. Exploration of Funding Opportunity: Government investment and/or exploration of funding opportunities for developing disaster resilient health facilities including hospitals, primary health centers, and other health facilities are essential because these institutions are central to sustainable recovery from disaster, and to health-driven Millennium Development Goals. IX. Conclusion Resilient health systems are a popular point today, which means that the health system can sustain during any type of emergencies, such as outbreaks of Ebola and Zika viruses, earthquake, flood, etc. If the hospitals are not prepared to save people affected by a disaster, if the hospitals are not prepared to provide treatment and provide people with the right care, then they will die and this is a disaster in itself. This is of critical value that people and the government needs to consider. People working in hospitals should be made aware that their hospitals are being made safe for all people. If the people working at hospitals understand what they should do, and how they can make their hospitals safer, they can do more to play an effective role inside the hospital to help keep it operational. As far as possible, ensure that the works of SERB is sustainable – this could be continuation of services or lasting changes in lives of the trained personnel in hospitals and FSCD volunteers. As far as possible, ensure that expertise and momentum for change in the country is not lost. SERB PERFORMANCE EVALUATION REPORT 58 | P a g e Finally, the respondents of the evaluation reported that the benefits derived from the interventions of SERB have far exceeded their expectation and they are, however, under the impression that there still remains opportunities for expansion and replication and rooms for improvement. The potential area of replication is in the pre- and post-emergency context and in areas vulnerable to earthquakes, fires, road traffic accidents, landslides and building collapse which are subject to mass casualties. SERB PERFORMANCE EVALUATION REPORT 59 | P a g e ANNEX A SERB PERFORMANCE EVALUATION REPORT 60 | P a g e ANNEX B METHODOLOGY OF SERB PERFORMANCE EVALUATION I. Evaluation Purpose and Objectives In February 2017 ADPC commissioned an independent evaluation of performance of SERB tasks undertaken since its beginning in June 2013. The evaluation is focused on the performance of SERB by examining its implementation, inputs, outputs and outcomes according to the Terms of Reference (TOR), provided at Attachment 1 to this Annex A, the evaluation examined the following evaluation objectives.  Review the progress towards achievement of stated SERB activity objective and intermediate and sub￾intermediate results.  Assess the relevance of the training materials used in SERB for HOPE-B, TFI, HICS and CSSR, Fire Fighting and First Aid training courses.  Determine the effectiveness of SERB interventions to handle mass causalities by the public health facilities and FSCD.  Assess the effectiveness of SERB interventions in contributing to the USAID/Bangladesh DO4: Responsiveness to Climate Change Improved.  Identify the success factors and constraints of the activity including institutionalization of the training courses by the implementing agencies.  Assess the effectiveness and sustainability of gender integration toward increased gender responsiveness in mass casualty management.  Assess the effectiveness of partnership, coordination and collaboration with NIPSOM FSCD, DDM, DGHS and other stakeholders as well as the systems established to support activity implementation.  Based on the overall findings of the evaluation, highlight key lessons and best practices which can be replicated.  Make specific recommendations on areas and/or interventions that will require further improvements in the future for activities similar to SERB. II. Evaluation Methodology A mixed methodology of qualitative and quantitative information gathering and analysis has been the approach for this performance evaluation. The evaluation process included the review of secondary sources, focus group discussions (FGD) and key informant interviews (KII), activity site and community visits, and ongoing training programs observation. The questionnaires included a range of structured questions pertaining to these evaluation criteria and scoring questions required for collecting subjective responses. In addition, appropriate open ended questions were asked to capture the views and perspectives of the respondents, especially those of SERB participants, implementers and stakeholders. For information collection the evaluator made specific site visits and face-to-face respondent interviews and telephone calls to the ADPC senior management and staff at Bangkok, Thailand, fire stations, hospital management and HOPE/HICS graduates, FSCD urban volunteers and communities. The information also include the feedback of USAID, ADPC senior management and key SERB activity staff, former Agreement Officer‘s Representatives (AOR) of SERB/USAID and former Chief of Party (COP) of SERB/ADPC. The SERB PERFORMANCE EVALUATION REPORT 61 | P a g e findings and recommendations are based on the review of relevance, effectiveness, replicability (adoption, continuity) coordination, gender responsiveness, and sustainability of the tasks of SERB. III. Evaluation Visit Locations and Respondents A total of 240 interviews were conducted during the evaluation. The numbers of hospitals (8 out of 16) and fire stations (8 out of 17) visited in the six SERB program cities represent nearly 50% of the Phases 1 and 2 targets. The table below shows the names of cities, and numbers of hospitals, fire stations, FGD and KII selected for discussions, interviews and observation. The process/steps of selection are as below. - Selected all six cities of SERB - Selected at least one hospital and one fire station from each of the six cities. The selection criteria included seven hospitals that received support for ERP development - Conducted at least one FGD (8-10 participants per FGD) in each of the 8 selected hospitals and 8 selected fire stations - Target of KII conducted during the visits to the hospitals and fire stations includes the hospitals senior management (Director, Deputy Director, Assistant Director and Superintendent) and fire station officers (Assistant Director, Senior Station Officer) and fire fighters. Locations Organizations FGD/KII Hospitals Fire Stations Hospitals Fire Stations Total Evaluation Sites Total Evaluation Sites FGD KII FGD KII Dhaka 7 2 10 2 2 2 2 2 Chittagong 3 2 2 2 2 2 2 2 Gazipur 1 1 2 1 1 1 1 1 Mymensingh 2 1 1 1 1 1 1 1 Sylhet 2 1 1 1 1 1 1 1 Tangail 1 1 1 1 1 1 1 1 Total 16 8 17 8 8 8 8 8 In addition, KII at national level (Dhaka) was conducted of NIPSOM, FSCD, DG/DDM, NCDC/DGHS, Offices of FDHA and Program, USAID/Bangladesh, ActionAid, DNCC/URP, WHO, ED/ADPC, SERB/ADPC staff, Former COP of SERB, Former AOR of SERB/USAID. Other tasks of information gathering included observation of an ongoing HOPE-Basic course at NIPSOM, visit to the Bandhan Volunteer Group at Taltola Community of Sylhet and ShSMCH campus, and participation in the National HRA Dissemination Meeting held in Dhaka. III. Evaluation Tasks Performed a. Desk Review: Reviewed the documents and reports available from ADPC/SERB, hospitals, fire stations and organizations visited. Another source of information is the documents relevant to the activity and evaluation available online. Attachment 2 of this Annex A provides the list of documents reviewed. SERB PERFORMANCE EVALUATION REPORT 62 | P a g e b. Tools Development: Table 2 of TOR (Attachment 1) included a range of questions pertaining to effectiveness, relevance, replicability and sustainability, gender responsiveness of the tasks and coordination and collaboration with the stakeholders to evaluate the performance of SERB. Based on these questions specific structured questionnaire by respondent type have been prepared. The questionnaires included a range of structured questions pertaining to these evaluation criteria and scoring questions required for collecting subjective responses. The questions on the respondents‘ opinion about the level of relevance/effectiveness/replicability included scoring questions rated between 0 to 4 where 0 for not, 1 for slightly, 2 for moderately, 3 for fully and 4 for highly relevant/effective/replicable. Attachment 3 contains a set of questionnaires. In addition, checklist or open ended questions was used to capture the views and perspectives of the respondents, especially those of stakeholders. c. Consultation and Interview: Held consultations and interviews to gather information with respect to the criteria of effectiveness, relevance, replicability, sustainability, gender responsiveness, coordination and collaboration with the stakeholders. The respondents included HOPE/HICS graduates and instructors and senior management and focal point of the hospitals, FSCD volunteers and fire station officers and staff, SERB implementing partners and nodal agencies, USAID/Bangladesh Mission, ADPC senior management and SERB officials, ActionAid, DNCC/URP, WHO, Former COP of SERB, Former AOR of SERB/USAID. Attachment 4 includes a list of these respondents. d. Analysis and Reporting: Produced preliminary report based on analysis of information collected through desk review of secondary source of information, field interviews and consultations with the respondents mentioned in the foregoing section, and observation of and participation in ongoing tasks of SERB. Also shared a preliminary draft with ADPC/SERB team. IV. Evaluation Team ADPC engaged an independent development professional, Golam Kabir, to conduct the evaluation. Mr. Kabir has nearly 40 years of experience in the design, management, evaluation and research of food security and disaster management programs in and outside of Bangladesh. Mr. Kabir served as the Senior Food Aid Manager and Principal Advisor to USAID Bangladesh for about 29 years in different positions: Food Security Project Development and Management Specialist, USAID Mission Disaster Management Specialist, USAID Mission Alternate Disaster Relief Officer, and USAID Mission Coordinator for USG-funded food aid and development assistance projects of multilateral agencies and NGOs in Bangladesh. Mr. Kabir has vast experience in short- and long-term consultancy including Evaluation Consultant with USAID/Bangladesh, CARE/Bangladesh, Fintrac Inc., FHI360, TANGO International, Moneval Solutions (UK), Asian Disaster Preparedness Center (Thailand), and Catholic Relief Services (Malawi); as Grants and Contracts Management Trainer with CARE and Save the Children; as Program Design Consultant with Save the Children, International Relief and Development, and Land O‘Lakes; and as Program Advisor to Save the Children and ACDI/VOCA. Mr. Kabir‘s thematic expertise includes livelihoods development, gender equity and empowerment, coalition building, food aid commodity monetization and management, environmental compliance, and disaster risk reduction. Mr. Kabir has a BSc-Honors and a MSc in Statistics from the University of Dhaka. IV. Evaluation Deliverables Three deliverables are envisioned in the TOR of the evaluation. They are: SERB PERFORMANCE EVALUATION REPORT 63 | P a g e a. Preliminary Report b. Draft Report c. Final Report V. Attachments to the Annex B Attachment 1: TOR of Evaluation Attachment 2: List of Documents Reviewed Attachment 3: Set of Evaluation Questionnaires Attachment 4: List of Places/Organizations/Respondents Visited/Interviewed SERB PERFORMANCE EVALUATION REPORT 64 | P a g e Attachment 1 to Annex B Terms of Reference – SERB Performance Evaluation Dated February 2017 I. Background and Context I.A. SERB Activity USAID/Bangladesh‘s Strengthening Earthquake Resilience in Bangladesh Activity (SERB), from June 2013 to May 2017, under a grant agreement of nearly $1.6 million in development assistance is implemented by ADPC, Thailand in partnership agreements with the National Institute for Preventive and Social Medicine (NIPSOM) and the Fire Service and Civil Defense Directorate (FSCD). SERB comprises two components: Hospital Preparedness for Emergencies (HOPE Component); and Capacity Building of FSCD and its volunteers (FSCD Component) with the objective to increase the capacity of the Government of Bangladesh to respond and manage natural disasters. I.B. SERB Implementing Agencies Besides the two implementing partners, NIPSOM and FSCD, ADPC collaborates with the Directorate General of Health Services (DGHS) and Department of Disaster Management (DDM). ADPC work closely with these agencies to get their support and expertise. I.C. SERB Key Outputs and Achievements As of January 2017 SERB under its HOPE component has conducted 22 out of 23, the LOA target of training courses in HOPE-Basic, Training for Instructors (TFI) and Hospital Incident Command System (HICS). These courses trained 519 medical and support staff (female – 203 and male – 316) from 16 public hospitals selected from the six target districts22 of SERB. Please refer to Table1 1 for status of achievements of tasks as of January 2017. During the same period the FSCD component achieved its LOA target of training to 1,200 urban community volunteers (female – 323 and male – 877) in Collapsed Structure Search and Rescue, Fire Fighting and First Aid (CSSR/FF/FA) and conducted 13 simulation exercises against the LOA target of 12 for 650 (female – 106 and male - 544) urban community volunteers to improve their skills in using search and rescue equipment and tools. The CSSR/FF/FA trainees were selected from 17 fire stations located in the six SERB districts and the simulation participants were selected from 10 fire stations in the three23 out of the six SERB districts. Under the FSCD component SERB delivered 171 sets of light search and rescue equipment and tools for FSCD volunteers. II. Evaluation Purposes and Objectives The evaluation will focus on the performance of SERB and examine its implementation, inputs, outputs, outcomes and objective. Specifically, the evaluation will: (a) review the effectiveness, relevance, replicability 22 Six target districts are: Dhaka, Chittagong, Sylhet, Mymensingh, Gazipur and Tangail. 23 Three districts are Dhaka, Chittagong and Sylhet SERB PERFORMANCE EVALUATION REPORT 65 | P a g e and sustainability of the SERB tasks; and (b) assess the effectiveness and sustainability of gender integration in SERB tasks and coordination and collaboration with the SERB implementing partners and concerned GoB agencies. Evaluation objectives of specific interest to SERB are to:  Review the progress towards achievement of stated SERB activity objective and intermediate and sub￾intermediate results.  Identify the activity approaches and interventions that contributed to or impeded the achievement of intended impact of activity interventions.  Determine the effectiveness of SERB interventions to handle mass causalities by the public health facilities and FSCD.  Assess the relevance of the training materials used in SERB for HOPE-B, TFI, HICS and CSSR, Fire Fighting and First Aid training courses.  Assess the effectiveness of SERB interventions in contributing to the USAID/Bangladesh DO4: Responsiveness to Climate Change Improved.  Identify the success factors and constraints of the activity including institutionalization of the training courses by the implementing agencies.  Assess the effectiveness and sustainability of gender integration toward increased gender responsiveness in mass casualty management.  Assess the effectiveness of partnership, coordination and collaboration with the NIPSOM FSCD, DDM, DGHS and other stakeholders as well as the systems established to support activity implementation.  Based on the overall findings of the evaluation, highlight key lessons and best practices which can be replicated.  Make specific recommendations on areas and/or interventions that will require further improvements in the future for activities similar to SERB. III. Evaluation Questions Table 1 presents a range of questions pertaining to effectiveness, relevance, replicability and sustainability of the tasks and coordination and collaboration with the stakeholders to evaluate the performance of SERB. Based on these questions specific structured questionnaire by respondent type will be prepared. In addition, appropriate open ended questions will be asked to ensure that all views and perspectives are taken into consideration, especially those of project participants, implementers and stakeholders. Table 1: Evaluation Questions Criteria Evaluation Questions Main Question Sub-Questions Effectiveness (E) How well were the interventions of SERB activity planned and implemented? E1: Were SERB interventions implemented as planned? E2: Were there any key programmatic challenges during implementation and how were they addressed? E3: How well did the SERB activity respond to unplanned events? E4: How effective were the modules and courses? E5: How effective were the simulations to enhance skills of the participants? SERB PERFORMANCE EVALUATION REPORT 66 | P a g e Criteria Evaluation Questions Main Question Sub-Questions Relevance (R) Assess the relevance of the training materials used in SERB for HOPE￾B, TFI, HICS and CSSR, Fire Fighting and First Aid training courses. R1: How relevant were the training courses and S&R tools and equipment in mass casualty management? What are communities‘ experiences about these interventions? R2: Were the training courses culturally appropriate to the communities and health professionals? R3: Which SERB tasks/outputs seem most relevant now in terms of having impact on the lives of targeted impact groups? Why are these relevant? Replicability (RP) Are SERB approach and interventions replicable in other settings (rapid and sudden onset of disasters like floods and cyclone) within Bangladesh? RP1: Are the public/private agencies interested to scale up SERB interventions? If so, what extra support they may need? RP2: What are community recommendations for future training courses? RP3: What are the recommendations from doctor/nurses/admin staff/ FSCD volunteers to improve training courses? RP4: Based on the overall findings of the evaluation, highlight key lessons and best practices which can be replicated. Sustainability (S) How sustainable were project outcomes? S1: What are the main factors that affect, either positively or negatively, the sustainability of project outcomes? S2: What exit strategies were incorporated into project design? Were such strategies implemented and to what extent did they contribute to sustainability? S3: Do hospitals and FSCD now have the capacity to effectively respond to disaster? S4: How the capacities of the Hospitals/FSCD enhanced due to HOPE /FSCD component initiatives? Are the program results sustainable, i.e., participants are likely to continue with the lessons learned? S5: Which outcomes of SERB are likely or unlikely to be sustainable, and why? What can be done to increase the sustainability? S6: What is the extent of SERB‘s collaboration with the relevant government agencies? S7: Are beneficiaries able to receive follow-up technical support from their respective agencies/organizations? S8: Are the community organizations and institutions and their tasks sustainable after the project exit? SERB PERFORMANCE EVALUATION REPORT 67 | P a g e Criteria Evaluation Questions Main Question Sub-Questions Gender Integration (GI) How effective is gender integration in mass casualty management? GI1: How SERB addressed gender issues while developing training module for the hospital staff, especially the needs of elderly persons, disabled person, women and girls? GI2: How SERB ensured that hospital response plan proactively integrates the gender-related challenges for women and children especially girls? GI3: What were the special consideration made during the overall planning of the hospital, to reduce gender inequality gaps in accessing assistance and services during disasters? GI3: How effective was the coordination of SERB with hospitals and FSCD on specific measures to include women as trainees and trainers? GI4: Whether SERB included gender consideration in the training module, so that the participants are aware about the needs of women, girls and elderly people? GI5: How effective was the step taken by SERB to promote equity across all funded tasks, i.e., Course facilitation was culturally￾sensitive and pro-actively focused on overcoming barriers to the meaningful participation of women staff? Coordination & Collaboration (CC) How well did SERB coordinate with its implementing partners and stakeholders? CC1: How coordination was established among agencies involved? How far is that effective? What are the best lessons? What to be strengthened for future projects? CC2: What were the challenges and successes of the relationships among implementing partners, stakeholders? IV. Evaluation Methodology and Tools Based on review of secondary documents and discussions with SERB team the evaluation methodology will include selection of number of hospitals, fire stations, communities and respondents for visit and interview. The collection of information will be done through: desk review of relevant SERB documentations; focus group discussions (FGD); key informant interviews (KII); conference calls with stakeholders; group interviews; and visits to SERB partners, sites, communities and stakeholders. The tools include the structured questionnaire and checklist for conducting face-to-face interviews and conference calls. More details of the evaluation approach and methodology are as below. IV.A. Information Gathering Information gathering will include collection of data from SERB reports for quantitative data and interviews of focus groups, and key informants for qualitative information. Field visits will cover SERB‘s target hospitals, FSCD fire stations and urban volunteers and communities. To gain a perspective from communities, stakeholders, implementing partners, HOPE-B, TFI and HICS graduates, FSCD (CSSR/FF/FA trainees and non-trained) volunteers and participating hospitals and fire stations. Below is list of key secondary documents. 1. SERB Extension Proposal to USAID 2. SERB M&E Plan 3. SERB RDQA Operation Manual SERB PERFORMANCE EVALUATION REPORT 68 | P a g e 4. SERB RDQA Reports 5. SERB Training Course Manuals 6. Training Course Evaluation (Pre- and Post-test) Records 7. USAID Evaluation Policy, TIPS 8. Semi-Annual Reports 9. Annual Work Plans 10. FSCD Tools and Equipment Maintenance Plan 11. Inventory of S&R tools and equipment 13. S&R Equipment Maintenance Plan and Record 14. SERB Gender Integration and Equity Plan IV.B. Field Visits and Interviews Field visits will cover all six SERB districts to gain a perspective from the hospitals, HOPE graduates, fire stations, FSCD volunteers. Attachment 4 indicates the different types of interviewees in each of the sites visited and number of interviewees reached in these six districts. IV.C. Preliminary Findings Debriefing/Presentation Upon completion of desk reviews, information gathering and site visits a debriefing/presentation of preliminary findings will be made to the SERB team. The feedback and any new information received will be incorporated into the evaluation report. IV.D. Draft Report Draft report will be prepared by analyzing the findings from desk reviews, interviews and incorporating SERB/ADPC comments and feedback in debriefing of preliminary findings. The draft report will present a thorough analysis supported with data, and clearly specifying the objective and outcomes will be assessed. IV.E. Final Report The main report will include a complete analysis, incorporating feedback received on the draft report. The final draft of the full evaluation report will be submitted to SERB/ADPC for feedback before finalizing. V. Evaluation Tasks and Deliverables Key tasks of the evaluation are: Preparation for evaluation; data collection and interviews; analysis of findings; preparation of draft and final reports. These tasks will produce three deliverables which are submission of preliminary findings, draft and final evaluation reports. Tasks End Product/Deliverables Schedule of Tasks Desk review of key documents, finalization of questionnaires and preparation for interviews and field visits Desk review and Evaluation plan submitted 12 – 16 February 2017 SERB PERFORMANCE EVALUATION REPORT 69 | P a g e Tasks End Product/Deliverables Schedule of Tasks Conduct interviews of key informants, meetings with stakeholders at the selected districts and site visits Data gathering carried out 18 February – 2 March 2017 Sharing of preliminary findings with SERB/ADPC officials Preliminary findings presented No later than 12 March 2017 Preparation of draft evaluation document and presentation and discussion of draft report with ADPC, partners and other key stakeholders Draft evaluation report submitted No later than 23 Mach 2017 Evaluation report finalised and submitted to ADPC Final evaluation report submitted No Later than 5 working days after receipt of feedback/comments on draft report VI. Evaluation Report Outline The main text of the evaluation report must be between 30 and 40 pages (excluding the appendices), in Word, single-spaced in 11-point font size, Calibri type and include the following sections:  Executive summary (no more than 3 pages)  Introduction  List of acronyms  Context (description of SERB)  Objectives of the evaluation and criteria used  Methodology  Analysis and findings  Activity impacts  Activity sustainability  Lessons learned and best practices  Constraints and opportunities impacting the project  Recommendations  Conclusions  Appendices: SOW, list of organizations and key people interviewed and schedule, questionnaires, interview guides SERB PERFORMANCE EVALUATION REPORT 70 | P a g e Attachment 2 of Annex B: List of Documents Reviewed 1. A Guidebook on Earthquake Preparedness and Fire Safety Drill in Garment Factories, ActionAid 2. ADPC Program for Enhancement of Emergency Response (PEER) 3 Final Results Report 3. Guideline for Hospital Non Structural Vulnerability Assessment, CDMP-2 4. Guidelines for Hospital Contingency Plan, DGHS, September 2010 5. Guidelines for Mass Casualty Management Drill 6. Health Bulletin 2016, DGHS, MoHFW 7. Hospital Emergency Preparedness and Response Plan, DGHS and WHO, September 2011 8. Hospital Safety Index Guide for Evaluators, WHO, 2015 9. NSET PEER News Bulletin No. 12, March 2009 10. Reference Book on Mass Casualty Management 11. Reference Book on Mass Casualty Management, ActionAid 12. SERB Annual Work Plans 13. SERB Extension Proposal to USAID 14. SERB FSCD Tools and Equipment Maintenance Plan 15. SERB Gender Integration and Equity Plan 16. SERB Hospital Emergency Response Plans 17. SERB Hospital Risk Assessment Reports 18. SERB RDQA Operation Manual 19. SERB RDQA Reports 20. SERB Revised Monitoring and Evaluation Plan 21. SERB Search and Rescue Equipment Maintenance Plan 22. SERB Semi-Annual Reports 23. SERB Training Course Evaluation (Pre- and Post-test) Records 24. SERB Training Course Manuals 25. Standing Orders on Disaster, MODMR, April 2010 26. Urban Disaster Resilience of Dhaka North City Corporation, SEEDS Asia, February 2017 27. USAID Evaluation Policy 2011 SERB PERFORMANCE EVALUATION REPORT 71 | P a g e Attachment 3 of Annex B: Set of FGD/KII QUESTIONNAIRES HOPE Basic, HOPE TFI & HICS Graduates Interviews Location: Please indicate the name of your working location (Hospital) 1. INTRODUCTION & BACKGROUND [Associated with SERB] 1.1 Which course did you take: HOPE-B/HOPE-TFI/HICS)? 1.2 When did you take the course(s)? 1.3 What is your position? 1.4 Academic background? 1.5 How/why have you joined training? 2. OUTCOMES/IMPACTS [Program’s Goal Achievement] 2.1 Has there been any mass casualty incidents (since you have trained in MCM)? 2.2 (If yes) Have you or your hospital been involved in any emergency response? Please explain your experience in handling the response operation. 2.3 Does your hospital now has the capacity to respond to mass casualty management (MCM)? 3. EFFECTIVENESS [SERB Responses to Disaster Emergencies] How well were the interventions of SERB activity planned and implemented? 3.1 How well did the program respond to unplanned events (mass casualties)? 3.2 Please indicate your rating and explain the rationale of your rating of the effectiveness of HOPE training courses? Hospital Risk Assessments? Emergency Response Plans? 0 1 2 3 4 Not Effective Slightly Effective Moderately Effective Fully Effective Highly Effective 4. RELEVANCE [Appropriateness of SERB Tasks] How appropriate (relevant) was the SERB activity design in the changed circumstances? 4.1 Were the courses administratively appropriate to the hospitals? 4.2 How do the HOPE tasks fit your national government‘s (e.g. MoDMR) priority in respect to disaster preparedness? SERB PERFORMANCE EVALUATION REPORT 72 | P a g e 4.3 Please indicate your rating and explain the rationale of your rating about your general opinion on the HOPE course curriculum? Hospital Risk Assessments? Emergency Response Plans? 0 1 2 3 4 Not Relevant Slightly Relevant Moderately Relevant Fully Relevant Highly Relevant 4.4 Which training sessions were most effective/relevant? 4.5 Which training sessions were least effective/irrelevant? 5. REPLICABILITY [Adaptation of SERB Tasks] Are SERB approach and interventions replicable in other settings (rapid and sudden onset of disasters like flash floods, tornadoes and cyclone) within Bangladesh? 5.1 Were there any recommendations by the hospitals for future training courses? If yes, please mention the training type and subject of their interests. 5.2 Are your national government agencies (e.g. MoDMR/MoHFW) interested to scale up the HOPE tasks? If so, what extra support they may need? 5.3 Have you used your HOPE skills in any emergencies or disaster or any other usages? 5.4 Please indicate your rating and explain the rationale of your rating of the replicability of course materials (workbook, presentations, training materials, training modules (lessons), etc.)? 0 1 2 3 4 Not Replicable Slightly Replicable Moderately Replicable Fully Replicable Highly Replicable 6. SUSTAINABILITY [Exit Strategies and Sustainability of Program Outcomes] How sustainable were project outcomes? 6.1 When SERB no longer supports you, will you be able to continue to support these training programs as a resource person/ an instructor? 7. Gender Integration How well did SERB coordinate with its implementing partners and stakeholders? 7.1 Is there an appropriate gender balance in the HOPE/HICS graduates? If not, what are the challenges to the gender balance in the HOPE/HICS? 7.2 Are you aware about the needs of women, girls and elderly people in the program? SERB PERFORMANCE EVALUATION REPORT 73 | P a g e 7.3 Could you tell us the rationale for having women, girls, etc. in the program? 7.4 What is your community views about women and girls participation in the program? 7.5 How was the course facilitation with respect to culturally-sensitiveness and pro-activeness to overcome barriers towards meaningful participation of women/girls? Questions for Female Graduate 7.6 Why the female participation is low in your hospital? 7.7 How to increase/motivate participation of female members in the program? 8. Coordination and Collaboration [Working Relationship among Program Implementing Agencies] How well did SERB coordinate with its implementing partners and stakeholders? 8.1 How do you coordinate your works with the administration and graduates in your hospital, etc.? 8.2 How do the hospital value your skills? How do the hospital cooperate/support with you and your engagement in emergencies? 8.3 Do you have any coordination with FSCD? If yes, what are they? How they are done? 9. LESSONS LEARNED [Best Practices, Lessons Learned] 9.1 What are the key lessons learned from the program that would help future programs in other regions of Bangladesh and elsewhere? 9.2 What were the best practices of the programs? 10. RECOMMENDATIONS/SUGGESTIONS 10.1 What follow-up training should be required for HOPE/HICS Graduates? 10.2 In your opinion what could have been improved or done differently? 10.3 What has been the most useful? 10.4 What has been least useful? 11. CONCLUDING REMARKS 11. Please let me know of your remarks, if any, on the subjects we have discussed. SERB PERFORMANCE EVALUATION REPORT 74 | P a g e Attachment 3 of Annex B: Set of FGD/KII QUESTIONNAIRES FSCD URBAN VOLUNTEERS/FIRE STATION OFFICERS/STAFF Field Interviews Location: Please indicate the name of your reporting fire station 1. INTRODUCTION & BACKGROUND [Associated with SERB] 1.1 Since when you have been involved in the FSCD –UV program? 1.2 What is your occupation? 1.3 Academic background? 1.4 How/why have you joined FSCD-UV program? 2. OUTCOMES/IMPACTS [Program’s Goal Achievement] 2.1 Has there been any mass casualty incidents (since you have trained in Search and rescue (SAR), mass casualty management? 2.2 (If yes) Have you or your fire station been involved in any emergency response? Please explain your experience in handling the response operation. 2.3 Do you or your fire station now have the capacity to respond to SAR, mass casualty management? 3. EFFECTIVENESS [SERB Responses to Disaster Emergencies] How well were the interventions of SERB activity planned and implemented? 3.1 What are the training courses you have taken from SERB / other than SERB? 3.2 Were SERB tasks (S&R tools/equipment, Advance/refreshers training/Simulations & Drills) appropriately designed to address identified needs in targeted communities/fire stations? 3.3 Which sessions of the training were most effective/relevant? 3.4 Which sessions were least effective/irrelevant? 3.5 Please indicate your rating and explain the rationale of your rating of the effectiveness of (a) training courses, (b) simulation exercises and (c) SAR tools and equipment in responding to unplanned events (fire, building collapse, etc.) mass casualty management? 0 1 2 3 4 Not Slightly Moderately Fully Highly SERB PERFORMANCE EVALUATION REPORT 75 | P a g e Effective Effective Effective Effective Effective 4. RELEVANCE [Appropriateness of SERB Tasks] How appropriate (relevant) was the SERB activity design in the changed circumstances? 4.1 Were SERB tasks (S&R tools/equipment, Advance/refreshers training/Simulations & Drills) appropriately designed to address identified needs in targeted volunteers)/fire stations? 4.2 Please indicate your rating and explain the rationale of your rating about your general opinion on the CSSR/FF/FA course curriculum, workbook, presentations, training materials, etc.? 0 1 2 3 4 Not Relevant Slightly Relevant Moderately Relevant Fully Relevant Highly Relevant 5. REPLICABILITY [Adaptation of SERB Tasks] Are SERB approach and interventions replicable in other settings (rapid and sudden onset of disasters like flash floods, tornadoes and cyclone) within Bangladesh? 5.1 Do you have any suggestions/choices for future training courses? If yes, please mention the training type and subject of your interests. 5.2 Are there any community recommendations for future training courses? If yes, please mention the training type and subject of their interests. 5.3 Are your communities interested to scale up the SERB volunteer task? If so, what extra support they may need? 5.4 Please indicate your rating and explain the rationale of your rating of the replicability of course materials (workbook, presentations, training materials, etc.)? 0 1 2 3 4 Not Replicable Slightly Relevant Moderately Relevant Fully Relevant Highly Relevant 6. SUSTAINABILITY [Exit Strategies and Sustainability of Program Outcomes] How sustainable were project outcomes? 6.1 When SERB no longer supports you, will you be able to continue to support these training programs as a resource person/ an instructor? SERB PERFORMANCE EVALUATION REPORT 76 | P a g e 7. Gender Integration How well did SERB coordinate with its implementing partners and stakeholders? 7.1 Is there an appropriate gender balance in the FSCD-UVs? If not, what are the challenges to the gender balance in the UVs? 7.2 Are you aware about the needs of women, girls and elderly people in the program? 7.3 Could you tell us the rationale for having women, girls, etc. in the program? 7.4 What is your community views about women and girls participation in the program? 7.5 How was the course facilitation with respect to culturally-sensitiveness and pro-activeness to overcome barriers towards meaningful participation of women/girls? Questions for Female Volunteers 7.7 Why the female participation is low in your community? 7.8 How to motivate participation of female members in the FSCD urban volunteer program? 8. Coordination and Collaboration [Working Relationship among Program Implementer Agencies] How well did SERB coordinate with its implementing partners and stakeholders? 8.1 How do you coordinate your works with the Ward Counselor, your community leaders, etc.? 8.2 Do the Ward Counselor, your community leaders, etc. accept/endorse your engagement in FSCD volunteer program? 8.3 How do the Ward Counselor, your community leaders, etc. cooperate/support with you and your engagement in emergencies? 8.4 Do you have any coordination with HOPE/HICS Graduates? If yes, what kind of coordination? How they are done? 9. LESSONS LEARNED [Best Practices, Lessons Learned] 9.3 What are the key lessons learned from the program that would help future programs in your ward and elsewhere in the city? 9.4 What were the best practices of the SERB-FSCD tasks you would like to share with us? 10. RECOMMENDATIONS/SUGGESTIONS SERB PERFORMANCE EVALUATION REPORT 77 | P a g e 10.1What follow-up training should be required for FSCD Volunteers? 10.2In your opinion what could have been improved or done differently? 10.3 What has been the most useful? What has been least useful? 11. CONCLUDING REMARKS 11. Please let me know of your remarks, if any, on the subjects we have discussed. SERB PERFORMANCE EVALUATION REPORT 78 | P a g e Attachment 3 of Annex B: Set of KII QUESTIONNAIRES NODAL AGENCY FOCAL POINT (DGHS/DDM) IMPLEMENTING PARTNERS (NIPSOM/FSCD) DONOR AGENCY 1. OUTCOMES/IMPACTS [Program’s Goal Achievement] 1.1 Please tell me your overall impression of SERB tasks with respect to achieving its objective to increase the capacity of the Government of Bangladesh to respond and manage natural disasters (specifically earthquake). 1.2 How well the SERB tasks benefit the emergency preparedness in the country? 2. EFFECTIVENESS [SERB Responses to Disaster Emergencies] How well were the interventions of SERB activity planned and implemented? 2.1 Please explain the effectiveness of SERB tasks (training courses, simulations, search and rescue tools and equipment, hospital risk assessments, ERPs, etc.) in responding to unplanned events (earthquakes, fire, building collapse, road accidents, disasters, etc.)? 2.2 Do the SERB tasks (FSCD community volunteers/ HOPE hospital graduates/instructors) now have the capacity to effectively respond to disasters? If so, please explain how you would measure the effectiveness of the SERB tasks? 3. RELEVANCE [Appropriateness of SERB Tasks] How appropriate (relevant) was the SERB activity design in the changed circumstances? 3.1 Please explain the relevance of SERB tasks (training courses, simulations, search and rescue tools and equipment, hospital risk assessments, ERPs, etc.) in responding to unplanned events (earthquakes, fire, building collapse, road accidents, disasters, etc.)? 3.2 Were there any issues raised by program participants and/or partners concerning their level of satisfaction with SERB? If yes, what were the issues of concern? 4. REPLICABILITY [Adaptation of PEER Tasks] Are SERB approach and interventions replicable in other settings (rapid and sudden onset of disasters like flash floods, tornadoes and cyclone) within Bangladesh? 4.1 What is your view on the replicability of SERB tasks? (Training, Simulation) 4.2 Are there scope to replicate the SERB training modules in the national curriculum? 5. MCM ACTIVITIES OTHER THAN SERB 5.1 What are the Mass Casualty Management related works supported by the GoB, Donors and other agencies in Bangladesh (past, present and upcoming)? 5.2 How do they compare/complement the HOPE/SERB tasks? SERB PERFORMANCE EVALUATION REPORT 79 | P a g e 6. SUSTAINABILITY [Exit Strategies and Sustainability of SERB Outcomes] How sustainable were project outcomes? 6.1 What are the main factors that affect, either positively or negatively, the sustainability of SERB outcomes? 6.2 What is your suggestions to make the SERB tasks sustainable in Bangladesh? 7. GENDER INTEGRATION How well did SERB coordinate with its implementing partners and stakeholders? 7.1 Is there an appropriate gender balance in the HOPE/HICS graduates and FSCD Volunteers? If not, what are the challenges to the gender balance in the HOPE/HICS/FSCD Volunteers? 7.2 Are you aware about the needs of women, girls and elderly people in the program? 7.3 Could you tell us the rationale for having women, girls, etc. in the program? 7.4 How was the course facilitation with respect to culturally-sensitiveness and pro-activeness to overcome barriers towards meaningful participation of women/girls? 7.5 How to increase/motivate participation of female members in the program? 8. COORDINATION & COLLABOTATION 8.1 How was coordination established among SERB stakeholders? FSCD and HOPE partners? How effective was it? What are the best features of the coordination? What needs to be strengthened for any future projects? 8.2 What is your comment on the effectiveness of the integration/coordination approach (such as planning and networking meetings) in emergency response and preparedness by the community, national and target (hospitals/fire stations) organizations? 8.3 How effective was the SERB Steering Committee? What was the SC contribution to the coordination and collaboration among SERB partners and stakeholders? 9. RECOMMENDATIONS/SUGGESTIONS 9.1 What follow-up tasks should be required for FSCD volunteers and HOPE graduates? In your opinion what could have been improved or done differently in the program? 7.1 What has been the most useful in the program? 7.2 What has been least useful in the program? 10. CONCLUDING REMARKS Please let us know of your remarks, if any, on the subjects we have discussed. SERB PERFORMANCE EVALUATION REPORT 80 | P a g e Attachment 4 of Annex B: List of Interviewees by Organizations and Locations Organizations Designation Location Methodology Tools ActionAid Programme Manager – DRR &CJ Dhaka Interview Checklist ADPC Executive Director Bangkok Conference Call Checklist Acting COP SERB Bangkok Interview/ Conference Call Checklist SERB Training Manager Dhaka Interview Checklist SERB Program Coordinator Dhaka Interview Checklist Human Resources and Admin Officer Dhaka Interview Checklist Former COP, SERB Dhaka Interview Checklist DGHS Line Director -Non Communicable Disease Control (NCDC) Dhaka Interview Checklist Deputy Program Manager, NCDC Program Dhaka Interview Checklist DDM Director General Dhaka Interview Checklist DNCC Project Director, URP- DNCC-Part Dhaka Interview Checklist FSCD Director, Training, Planning and Development Dhaka Interview Checklist Deputy Assistant Director, South Surma Fire Station Sylhet Interview Checklist Urban Community Volunteers, Taltola Fire Station Sylhet FGD Questionnaire Senior Station Officer, Agrabad Fire Station Chittagong Interview Questionnaire Urban Community Volunteers, Agrabad Fire Station Chittagong FGD Questionnaire Senior Station Officer, Chandanpura Fire Station Chittagong Interview Questionnaire SERB PERFORMANCE EVALUATION REPORT 81 | P a g e Organizations Designation Location Methodology Tools Senior Station Officer and Staff, Siddique Bazar Fire Station, Dhaka Interview Questionnaire Urban Community Volunteers, Siddique Bazar Fire Station Dhaka FGD Questionnaire Senior Station Officer and Staff, Mirpur Fire Station, Dhaka Interview Questionnaire Deputy Assistant Director and Staff, Mymensingh Fire Station Mymensingh Interview Questionnaire Urban Community Volunteers, Mymensingh Fire Station Mymensingh FGD Questionnaire Deputy Assistant Director and Staff, Tangail Fire Station Tangail Interview Questionnaire Urban Community Volunteers, Tangail Fire Station Tangail FGD Questionnaire Senior Station Officer and Staff, Tongi Fire Station Tongi Interview Questionnaire Urban Community Volunteers, Tongi Fire Station Tangail FGD Questionnaire Hospitals Director, CMCH Chittagong Interview Checklist Deputy Director, CMCH Chittagong Interview Checklist HOPE/HICS Graduates and HRA Participants, CMCH Chittagong FGD Questionnaire Deputy Director, BITID Chittagong Interview Checklist HOPE Graduates and HRA Participants, BITID Chittagong FGD Questionnaire Director, SOMCH Sylhet Interview Questionnaire Resident Surgeon (General), SOMCH Sylhet Interview Checklist Resident Surgeon (General Surgery) SOMCH Sylhet Interview Checklist HOPE/HICS Graduates and HRA Participants, SOMCH Sylhet FGD Questionnaire SERB PERFORMANCE EVALUATION REPORT 82 | P a g e Organizations Designation Location Methodology Tools Superintendent, SKH Dhaka Interview Questionnaire HOPE Graduates and HRA Participants, SKH Dhaka FGD Questionnaire Superintendent, TGH Tangail Interview Questionnaire HOPE Graduates and HRA Participants, TGH Tangail FGD Questionnaire Assistant Director, MMCH Mymensingh Interview Questionnaire HOPE Graduates and HRA Participants, Mymensingh Mymensingh FGD Questionnaire Deputy Director, STAMCH Gazipur Interview Questionnaire HOPE Graduates and HRA Participants, STAMCH Gazipur FGD Questionnaire NIPSOM Director / SERB Focal Point – HOPE Component Dhaka Interview Checklist HOPE Training Monitor/Instructor Dhaka Interview Checklist SERB Steering Committee Chairperson Dhaka Interview Checklist USAID Director, OFDHA Dhaka Interview Checklist Deputy Director, OFDHA Dhaka Interview Checklist Senior Advisor, OFDHA Dhaka Interview Checklist Senior Mission Engineer and AOR/SERB, OFDHA Dhaka Interview Checklist Project Management Specialist, OFDHA Dhaka Interview Checklist Mission Gender Specialist, Program Office Dhaka Interview Checklist Senior Monitoring and Evaluation Specialist, Program Office Dhaka Interview Checklist Senior Monitoring and Evaluation Specialist, Program Office Dhaka Interview Checklist SERB PERFORMANCE EVALUATION REPORT 83 | P a g e Organizations Designation Location Methodology Tools Senior Monitoring and Evaluation Specialist, Program Office Dhaka Interview Checklist WHO National Professional Officer, EHA Unit Dhaka Interview Checklist SERB PERFORMANCE EVALUATION REPORT 84 | P a g e ANNEX C SERB LOA TARGETS AND ACHIEVEMENTS (June 2013 to May 2017) Components Tasks Target (LOA -June 2013 to May 2018) Achievements as of May 2017 # of Units # of Participants HOPE Component HOPE Basic Course 21 Training Courses 17 Courses 415 (male - 253 female – 162) HOPE TFI Course 4 Training Courses 3 Courses 52 (male 36 female 16) Hospital Risk Assessment (HRA) 19 HRAs 16 HRAs 386 (male 317 female 69) HRA Findings Dissemination 19 Meetings 15 Meetings N/A Emergency Response Plan (ERP) Development and validation 10 plans 7 Plans N/A Hospital Incident Command System (HICS) Training 6 Training Courses 4 Courses 100 (male 59 female 41) HRA Findings Dissemination Meeting at National level 1 Meeting 1 Meeting 56 (male 46 female 10) IMS National Introductory Workshop 1 Workshop 1 Workshop 76 (male 65 female 11) IMS Resource Mapping Study 1 Study 1 Study N/A FSCD Component Procurement of Search Tools 71 sets 71 sets N/A Procurement of Rescue Equipment/Tools 31 sets 31 sets N/A Procurement of First Aid Kits 48 kits 48 kits N/A Procurement of containers 21 containers 21 containers N/A Advance/Refreshers‘ training for FSCD Volunteers 34 Batches 30 Batches 1, 200 (male 877 female 323) Simulation for FSCD Volunteers 12 Simulations 13 Simulations 650 (male 544 female 106) Case Study Booklet Production 1 1 2,500 copies Both HOPE and FSCD Components Nepal Study visit 1 visit 1 Visit 15 (male 9 female 6) SERB Performance Evaluation 1 Evaluation 1 Evaluation Draft report done SERB PERFORMANCE EVALUATION REPORT 85 | P a g e ANNEX D SERB PERFORMANCE EVALUATION REPORT HOPE Component – Distribution of Participants by Hospitals/Institutions Phases 1 and 2 (June 2013 to May 2017) Hospitals Number of Graduates HOPE-B HOPE-T HICS All Training F M T F M T F M T F M T BSMMU 19 16 35 2 2 4 3 6 9 24 24 48 DMCH 12 29 41 1 4 5 2 6 8 15 39 54 ShSMCH 16 23 39 3 4 7 3 7 10 22 34 56 SKH 10 10 20 - 2 2 1 4 5 11 16 27 KGH 6 5 11 1 2 3 1 3 4 8 10 18 NITOR 3 14 17 - - - 3 5 8 6 19 25 SSMCMH 3 4 7 1 2 3 1 1 2 5 7 12 CGH 5 8 13 - 1 1 5 4 9 10 13 23 CMCH 4 18 22 1 3 4 5 8 13 10 29 39 BITID 5 5 10 - - - 2 3 5 7 8 15 SOMCH 14 33 47 2 5 7 8 9 17 24 47 71 SSADH 11 0 11 0 5 1 6 16 1 17 MMCH 16 42 58 - 3 3 - - - 16 45 61 SH - - - - - - - - - - - - STAMCH 15 14 29 2 2 4 2 1 3 19 17 36 TGH 10 9 19 - - - - - - 10 9 19 NIPSOM 6 4 10 1 1 2 - - - 7 5 12 FSCD - 10 10 - - - - - - - 10 10 OTHERS 7 9 16 2 5 7 - 1 1 10 14 24 TOTAL 162 253 415 16 36 52 41 59 100 219 348 567 OTHER ORGANIZATIONS INCLUDE: ADPC, BGMEA, DGHS, EWMC, FSCD, IPH, NICRH, NIDCH & SDC. SERB PERFORMANCE EVALUATION REPORT 86 | P a g e ANNEX E SERB PERFORMANCE EVALUATION REPORT SUMMARY OF FINDINGS & RECOMMENDATIONS HOSPTAL RISK ASSESSMENTS Short-Medium Term Availability, Repair and Maintenance of Nonstructural Elements: The hospitals at present have many of their nonstructural elements between low and average levels of safety. Specifically, they are the systems of: electricity and lighting in critical areas; telecommunications; water supply; fire protection; waste management; fuel reserve; medical gases; HVAC; and shelving of furnishings and equipment; safety and security of staff and patient; and medical and laboratory equipment and supplies used for diagnosis and treatment. Most of these elements are inadequate, too. The hospitals are therefore recommended for improvement, enhancement and automation of these nonstructural elements. The hospitals also shall conduct repair and regular cleaning and maintenance of these nonstructural elements. Development and implementation of Contingency Plans: Hospitals may not always require full activation of ERPs for the internal events such as fire, floods, chemical spills and utility failures if they have contingency plans and procedures established for these internal events to ensure a safe hospital. It is therefore recommended that the hospitals develop and implement contingency plans and procedures that are required for certain critical events and essential services. Suggested plans are:  Maintenance and Inspection Contingency Plan  Fire Safety and Fire Emergency Plan  Flood Management Contingency Plan  Chemical Spills Management Contingency Plan  Power Failure Contingency Plan  Hospital Evacuation Contingency Plan  Psychosocial Support Plan  Hospital Business Continuity Plan  Personal Health and Critical Incident Stress Plan Long Term Conduct Detailed Structural Assessment: Most old buildings of the hospitals were constructed well before the introduction of the modern standard and safety regulations such as the BNBC. Availability of engineers was a challenge in case of assessment of the structural component. Only one or two civil engineers from the Public Works Department (PWD) of the government were present in each of the HRAs. But the assessment required a group of engineers from PWD (civil, electrical, etc.) to fully participate in the assessment. Thus the assessment of the structural component of the hospital was observation based and understanding of the assessment team. In most instances in which major structural deficiencies were found, it is important to take note that these are not the result of a detailed engineering assessment but rather limited to visual observations and feedback from interviews conducted during the assessment with relevant hospital staff. So the concerned hospitals are recommended to conduct further detailed structural assessment by engineering professionals to know the safety level of structural component of the hospital. SERB PERFORMANCE EVALUATION REPORT 87 | P a g e Retrofitting and Remodeling of Structure: Most of the old hospital buildings‘ are concrete frame structures and subject to ageing. So the overall safety level of structure was found to be average. Since the SERB target hospitals are located at the moderate seismic intensity area and very close to the Madhupur fault, in case of moderate to severe earthquake the hospitals are subject to heavy damage. In addition, among other possible hazards that pose threats to the safety of the hospitals include river floods, tornadoes, local storms, and mass gathering as well as biological and technological accidents. The hospitals are the frontline medical facility in case there are emergencies in their immediate surrounding especially during working days and hours. So, the concerned hospitals shall undertake proper retrofitting and remodeling of the structures as early as possible. Establishment of Emergency Preparedness and Response System: Most SERB target hospitals‘ level of preparedness with regards to their emergency and disaster management capacity in responding to mass casualty incidents is low. This functional capacity of the hospital is therefore needs strengthening as a priority action of the hospital in the immediate time for effective responses to the emergencies or disasters. The preparedness shall include:  Formation of a fully functional Hospital Disaster Management Committee with roles and responsibilities for each member.  Conducting simulation exercises/drills for capacity building purpose with the involvement of various sections/departments of the hospital.  Establishment and application of Hospital Incident Command System (HICS) to increase the effectiveness and efficiency of response.  Establishment of fully functional network between local health facilities, and district level disaster management programs and committees implemented by the Ministry of Disaster Management and Relief, Fire Service and Civil Defence, Red Crescent Society, etc.  Equip Emergency Operations Centre (EOC) with equipment and supplies to be readily available to set up the EOC for communications, information management including digitalization of patient record system, identification, security, and well-being of EOC staff. The EOC should be backed up by an information management system that supports emergency operations and that can link to data from the hospital‘s information management system.  Establishment of all-hazards hospital recovery plan that defines actions to be taken to recover normal functions of the hospital after an emergency or disaster. The recovery plan should provide continuity of recovery and rehabilitation of patient services, the recovery needs of personnel, the replenishment of supplies and replacement of equipment, and procedures for determining priorities for assessment and rehabilitation of the hospital‘s structural and nonstructural elements which may have been damaged. The recovery plan, as well as the response plan, should also be linked to the business continuity plan for the hospital and reviewed/updated at least annually. Continuation of Emergency and Disaster Management Training and Simulation: Hospitals shall arrange separate training on Hospital Emergency Response Plan (ERP) for the hospital personnel and allocate budget for implementation of the ERP. Hospitals shall conduct simulation exercises/drills for capacity building purpose and refreshers trainings for all hospital personnel regularly. SERB PERFORMANCE EVALUATION REPORT 88 | P a g e Hospital Safety Index (HSI) Score Explanation on Safety Index Scores and Recommendations Data on the assessment levels (high, average, or low) for each individual parameter is processed by making use of the ―Safety Index Calculator‖ – a software application specifically designed for the tool, which is automatically computes a safety index and subsequently the vulnerability index for the hospital being assessed. There are in total 12 steps of calculation of module-specific and overall hospital safety index. The HSI represents a value expression of a health care facility‘s ability to operate during emergencies, and can range from 0 to 1.0. It is worth mentioning that various sections have a different weight under the methodology used to compute the Index as follows: the ―Structural Safety‖ section accounts for 50 percent of the Index value, the ―Nonstructural Safety‖ section accounts for 30 percent of it, whereas the ―Emergency and Disaster Management Capacity‖ section contributes to 20 percent of its value. The rationale behind this weighting percentage is that the area where the hospital is located has a high risk of earthquake and/or cyclones as a primary concern and consideration thus structural component is assigned the highest weight. Responses by Module: The results of the safety of STAMCH, evaluated against the parameters in Modules 2, 3 and 4, with reference to both the hazards identified in Module 1 and the maximum capacity of the hospital for emergencies and disasters identified in Form 1 (General Information about the Hospital), showed that the Nonstructural and Emergency and Disaster Management components of STAMCH are at the average safety levels and the structural component is at a low safety level. That means, the nonstructural and emergency management components are likely to function during a major incident while the structural component is unlikely to function because of its low level of safety. An example of results of responses by module is depicted in the table below. Results of Safety Level of a Hospital by Safety Components Hospital Safety Component Average weights of hospital safety levels (%) Low Safety Level Average Safety Level High Safety Level Structural 53.00 37.25 9.75 Nonstructural 36.72 63.03 0.25 Emergency & Disaster Management 32.30 52.80 14.90 Safety and Vulnerability Index by Module: The vulnerability is an important factor as to become more safe the vulnerability shall be reduced. The safety index and vulnerability index have been calculated by module in order to arrive at the overall safety index which ultimately determines the overall health facility status. An illustration of results of safety index and vulnerability index by module is as below. SERB PERFORMANCE EVALUATION REPORT 89 | P a g e Results of Safety and Vulnerability Index by Module Module Safety Index Vulnerability Index Structural (Module 2) 0.01 -0.78 Nonstructural (module 3) 0.14 -0.91 Emergency & Disaster Management (module 4) 0.00 -0.78 The sum of the weighted results of the three modules gives a hospital safety rating expressed as the probability (percentage) that a facility will be able to function in an emergency or disaster situation. When formulas are applied to the data from the checklist, the calculator will assign weighted values to each item, section, sub module and module. The formulas calculate a specific value and index for each of the structural, nonstructural, and emergency and disaster management modules, and calculate an overall hospital safety index. The checklist results are entered as number 1 in the corresponding cells in the software and the calculation page automatically applies a series of formulas to carry out its computations that calculates and charts the relative safety and the module-specific safety index for each module automatically classifies the module￾specific index using small letters ―a‖, ―b‖ or ―c‖. The result score can also be as an overall safety index with an assigned classification of ―A‖, ―B‖ or ―C:‖ in capital letters. HOSPITAL SAFETY GROUPS BY SAFETY INDEX SCORES AND GENERAL RECOMMENDATIONS Hospital Safety Index Safety Group Description 0.66 – 1.0 A It is likely that the hospital will function in emergencies and disasters. It is recommended, however, to continue measures to improve emergency and disaster management capacity and to carry out measures in the medium- and long-term to improve the safety level in case of emergencies and disasters. 0.36 – 0.65 B Intervention measures are needed in the short term. The hospital‘s current levels of safety and emergency and disaster management are such that the safety of patients and hospital staff, and the hospital‘s ability to function during and after emergencies and disasters, are potentially at risk. 0 – 0.35 C Urgent intervention measures are needed. The hospital is unlikely to function during and after emergencies and disasters, and the current levels of safety and emergency and disaster management are inadequate to protect the lives of patients and hospital staff during and after emergencies or disasters. SERB PERFORMANCE EVALUATION REPORT 90 | P a g e Overall Safety Index of Hospital: After the quantitative assessment using the scores from the HSI assessment tool, the calculation of the hospital‘s status is determined as Category A or B or C. SERB PERFORMANCE EVALUATION REPORT 91 | P a g e ANNEX F Distribution of Participants by FSCD Fire Stations Phases 1 and 2 (June 2013 to May 2017) Training Location Training/Simulation Events CSSR/FF/FA Simulation All Events F M T F M T F M T Mirpur 53 107 160 16 84 100 69 191 260 TC Mirpur 26 54 80 18 82 100 44 136 180 Mohammadpur 17 63 80 6 44 50 23 107 130 Postogola 12 28 40 12 28 40 Kurmitola 5 35 40 8 42 50 13 77 90 Siddique Bazaar 39 121 160 8 42 50 47 163 210 Baridhara 4 36 40 9 41 50 13 77 90 Sadarghat 15 25 40 8 42 50 23 67 90 Tejgaon 18 62 80 8 42 50 26 104 130 South Surma/Taltola 32 48 80 7 43 50 39 91 130 Agrabad 12 28 40 8 42 50 20 70 90 Chnadanpura 15 25 40 10 40 50 25 65 90 Hajaribag 14 66 80 14 66 80 Khilgaon 26 54 80 26 54 80 Tongi 12 68 80 12 68 80 Mymensingh 13 27 40 13 27 40 Joydebpur 10 30 40 10 30 40 Tangail 10 30 40 10 30 40 All Locations 323 877 1200 106 544 650 439 1411 1850 SERB PERFORMANCE EVALUATION REPORT 92 | P a g e ANNEX G SERB PERFORMANCE EVALUATION REPORT CITY-WISE DISTRIBUTION OF FSCD URBAN COMMUNITY VOLUNTEERS SERB LOA (June 2013 – May 2018) SERB CITIES TOTAL FSCD VOLUNTEERS IN 32 FIRE STATIONS IN 9 CITIES FSCD VOLUNTEERS IN 21 SERB TARGET FIRE STATIONS IN 9 CITIES TOTAL MALE FEMALE TOTAL MALE FEMALE Dhaka 15,796 12,018 3,778 15,710 11,973 3,737 Chittagong 7,722 5,892 1,830 2,775 2,260 515 Sylhet 3,796 3,006 790 949 717 232 Mymensingh 200 153 47 200 153 47 Gazipur 396 274 122 396 274 122 Tangail 200 146 54 200 146 54 Manikganj 150 100 50 150 100 50 Narayanganj 382 250 132 382 250 132 Rangpur - - - - - - TOTAL 28,642 21,839 6,803 20,762 15,873 4,889 % of Total - 76.25 23.75 - 76.45 23.55 (Source: FSCD HQ) ANNEX H – SUMMARY TABLES OF RESPONSES TO QUESTIONS WITH SCORES TABLE A: HOPE-B/HOPE-T/HICS TRAINING PARTICIPANTS & SENIOR MANAGEMENT / FOCAL POINT OF HOSPTALS [FIELD INTERVIEWS] Review Criteria Hospitals Visited →→ CMCH BITID SOMCH SKH MMCH TGH ShSMCH STAMCH Admin. of 8 Hospitals All Hospitals Comments Date Visited→→ 23/2/17 23/2/17 2/3/17 13/3/17 18/3/17 20/3/17 23/3/17 13/4/17 23/2/17 -13/4/17 Number Persons Interviewed→→ T8 (F3+M5) T7 (F5+M2) T9 (F5+M4) T9 (F6+M3) T8 (F6+M2) T6 (F3+M3) T9 (F5+M4)) T=8 (F4+M4) T=8 (F2+M6) T=72 (F39+M33) Tasks ↓ Score ↓ Score ↓ Score ↓ Score ↓ Score ↓ Score ↓ Score ↓ Score ↓ Score ↓ Score ↓ Relevance HOPE-B/HOPE￾T/HICS H=5 F=3 T=8 H=5 F=2 T=7 H=7 M=2 T=9 H=7 F=2 T=9 H=6 F=2 T=8 H=5 F=1 T=6 H=5 F=3 M=1 T=9 H=5 F=2 M=1 T=8 H=6 F=2 T=8 H=51 (71%) F=17 (24%) M=4 (5%) T=72 Score: H=Highly F=Fully M=Moderately S=Slightly HRAs H=8 H=7 H=9 H=9 H=8 H=6 H=9 H=8 H=6 F=2 T=8 H=70 (97%) F=2 (3#) T=72 ERPs H=8 N/A H=9 N/A H=8 H=6 H=9 H=8 H=8 H=56 (100%) T=56 N/A = BITID and SKH are not targeted by SERB for ERP development; Question on relevance does not apply Study Visit N/A N/A N/A N/A N/A N/A N/A H=8 T=8 H=1 T=1 H=9 (100%) T=9 N/A = Hospitals visited prior to the study visit; Question on relevance of study visit does not apply except STAMCH. Draft SERB Performance Evaluation Report Review Criteria Hospitals Visited →→ CMCH BITID SOMCH SKH MMCH TGH ShSMCH STAMCH Admin. of 8 Hospitals All Hospitals Comments Date Visited→→ 23/2/17 23/2/17 2/3/17 13/3/17 18/3/17 20/3/17 23/3/17 13/4/17 23/2/17 -13/4/17 Number Persons Interviewed→→ T8 (F3+M5) T7 (F5+M2) T9 (F5+M4) T9 (F6+M3) T8 (F6+M2) T6 (F3+M3) T9 (F5+M4)) T=8 (F4+M4) T=8 (F2+M6) T=72 (F39+M33) Tasks ↓ Score ↓ Score ↓ Score ↓ Score ↓ Score ↓ Score ↓ Score ↓ Score ↓ Score ↓ Score ↓ - One HOPE trainee from STAMCH included in the study visit Effectiveness HOPE-B/HOPE￾T/HICS H=7 F=1 T=8 H=6 F=1 T=7 H=8 M=1 T=9 H=7 F=2 T=9 H=5 F=3 T=8 H=5 F=1 T=6 H=7 F=2 T=9 H=6 M=2 T=12 H=6 F=2 T=2 H=57 (79%) F=12 (17%) M=3 (4%) T=72 HRAs H=8 H=7 H=9 H=9 H=8 H=6 H=9 H=8 H=8 H=72 (100%) T=72 ERPs H=8 N/A H=9 N/A H=8 H=6 H=9 H=8 H=8 H=56 (100%) T=56 N/A = BITID and SKH are not targeted by SERB for ERP development; Question on effectiveness of ERP does not apply. Study Visit N/A N/A N/A N/A N/A N/A N/A H=8 H=1 T=1 H=9 (100%) T=9 N/A = Hospitals visited prior to the study visit; Question on effectiveness of study visit does not apply except STAMCH. Replicability HOPE-B/HOPE- H=6 H=6 H=8 H=8 H=7 H=5 H=9 H=6 H=8 H=63 (88%) Draft SERB Performance Evaluation Report Review Criteria Hospitals Visited →→ CMCH BITID SOMCH SKH MMCH TGH ShSMCH STAMCH Admin. of 8 Hospitals All Hospitals Comments Date Visited→→ 23/2/17 23/2/17 2/3/17 13/3/17 18/3/17 20/3/17 23/3/17 13/4/17 23/2/17 -13/4/17 Number Persons Interviewed→→ T8 (F3+M5) T7 (F5+M2) T9 (F5+M4) T9 (F6+M3) T8 (F6+M2) T6 (F3+M3) T9 (F5+M4)) T=8 (F4+M4) T=8 (F2+M6) T=72 (F39+M33) Tasks ↓ Score ↓ Score ↓ Score ↓ Score ↓ Score ↓ Score ↓ Score ↓ Score ↓ Score ↓ Score ↓ T/HICS F=2 T=8 F=1 T=7 F=1 T=9 F=1 T=9 F=1 T=8 F=1 T=6 T=9 F=2 T=8 T=8 F=9 (12%) T=72 Study Visit N/A N/A N/A N/A N/A N/A N/A H=8 T=8 H=1 T=1 H=9 (100%) T=9 N/A = Hospitals visited prior to the study visit; Question on replicability of study visit does not apply except STAMCH. Scale: 0=Not Relevant/Effective/Replicable; 1=Slightly Relevant/Effective/Replicable; 2=Moderately Relevant/Effective/Replicable; 3= Fully; 4= Highly Relevant/Effective/Replicable Figures in parenthesis are number of female (FX), male (MY) and total (T=FM+MY) interviewees. Draft SERB Performance Evaluation Report ANNEX H – SUMMARY TABLES OF RESPONSES TO QUESTIONS WITH SCORES TABLE B: FSCD URBAN COMMUNITY VOLUNTEERS & FIRE STATION STAFF [FIELD INTERVIEWS] Review Criteria Fire Stations Visited →→ Agrabad Chandan￾pura Taltola Siddik￾bazar Mirpur Mymen -singh Tangail Gazipur Staff of 8 Fire Stations All Fire Stations Comments Date Visited→→ 22/2/17 22/2/17 2/3/17 13/3/17 18/3/17 20/3/17 23/3/17 13/4/17 23/2/17 - 13/4/17 Number Persons Interviewed→→ T15 (F3+ M12) T18 (F5+ M13) T15 (F5+ M10) T9 (F3+ M6) T12 (F8+ M4) T18 (F5+ M13) T17 (F7+ M10) T=20 (F4+ M16) T=12 (F0+ M12) T=136 (F40+ M96) Tasks ↓ Score ↓ Score ↓ Score ↓ Score ↓ Score ↓ Score ↓ Score ↓ Score ↓ Score ↓ Score ↓ Relevance CSSR/FF/FA Training H=15 T=15 H=18 T=18 H=15 T15 H=9 T=9 H=12 T=12 H=18 T18 H=17 T=17 H=20 T=20 H=12 T=12 H=136 (100%) T=136 136 respondents from 8 fire stations covered by the field interviews in 8 out of 17 SERB fire stations Simulation Exercise H=15 T=15 H=18 T=18 H=15 T15 H=9 T=9 H=12 T=12 N/A N/A N/A H=8 F=4 T=12 H=77 (95%) T=81 No simulation conducted at the 3 SERB target fire stations of Gazipur, Mymensingh and Tangail districts. SAR Tools/Equipment H=15 T=15 H=18 T=18 H=15 T15 H=9 T=9 H=12 T=12 H=18 T18 H=17 T=17 H=20 T=20 H=12 T=12 H=136 (100%) T=136 Effectiveness CSSR/FF/FA Training H=15 T=15 H=18 T=18 H=15 T15 H=9 T=9 H=12 T=12 H=18 T18 H=17 T=17 H=20 T=20 H=12 T=12 H=136 (100%) T=136 Simulation Exercise H=15 T=15 H=18 T=18 H=15 T15 H=9 T=9 H=12 T=12 N/A N/A N/A H=12 T=12 H=81 (100%) T=81 See comment above SAR Tools/Equipment H=15 T=15 H=18 T=18 H=15 T15 H=9 T=9 H=12 T=12 H=18 T18 H=17 T=17 H=20 T=20 H=12 T=12 H=136 (100%) T=136 Replicability CSSR/FF/FA Training H=15 T=15 H=18 T=18 H=15 T15 H=9 T=9 H=12 T=12 H=18 T18 H=17 T=17 H=20 T=20 H=12 T=12 H=136 (100%) T=136 Draft SERB Performance Evaluation Report Review Criteria Fire Stations Visited →→ Agrabad Chandan￾pura Taltola Siddik￾bazar Mirpur Mymen -singh Tangail Gazipur Staff of 8 Fire Stations All Fire Stations Comments Date Visited→→ 22/2/17 22/2/17 2/3/17 13/3/17 18/3/17 20/3/17 23/3/17 13/4/17 23/2/17 - 13/4/17 Number Persons Interviewed→→ T15 (F3+ M12) T18 (F5+ M13) T15 (F5+ M10) T9 (F3+ M6) T12 (F8+ M4) T18 (F5+ M13) T17 (F7+ M10) T=20 (F4+ M16) T=12 (F0+ M12) T=136 (F40+ M96) Tasks ↓ Score ↓ Score ↓ Score ↓ Score ↓ Score ↓ Score ↓ Score ↓ Score ↓ Score ↓ Score ↓ Simulation Exercise H=15 T=15 H=18 T=18 H=15 T15 H=9 T=9 H=12 T=12 N/A N/A N/A H=12 T=12 H=81 (100%) T=81 See comment above Scale: 0=Not Relevant/Effective/Replicable; 1=Low; 2=Moderately; 3= Fully Relevant/Effective/Replicable; 4= Highly Relevant/Effective/Replicable Score: H=High; F=Full; M=Moderate; S=Slight Figures in parenthesis are number of female (FX), male (MY) and total (T=FM+MY) interviewees. ANNEX I SERB PERFORMANCE EVALUATION REPORT ACTION PLANS DEVELOPED BY NEPAL STAUDY VISIT PARTICIPANTS Tasks Number of Team Members Thematic Areas Establishing a Disaster Management Plan. 9 Disaster Management Plan Training of all levels of hospital staffs. 7 Capacity Building Developing a multi hazard emergency response plan. 9 ERP Develop training program me and conduct regular simulation exercise for hospital staff and volunteers. 7 Simulation Exercise Establishment of definite plan for sanitation, food supply and emergency energy backup 10 Proper storage of supply for emergency Awareness and capacity building of all staffs including people of admin, support services, 7 Awareness building Establish good digital communication with other agencies like fire service 6 Digital communication Development of disaster management network involving multiple agencies like police, BGB, Armed forces, public and private agencies. 4 Disaster Coordination Make emergency contingency plan, allocate budget for equipment, drugs, food and water for victims and hospital staffs. 7 Allocation of budget Declaration of Open Space of Each Hospital. 4 Identification of Open space Improve Hospitals Cleaning & Sanitation facilities, Repair of Structural Damage / Cracks of in the Buildings. 5 Structural & Non￾structural improvements Establish disaster management floor mapping of the hospital with orientation to all hospital staffs regarding the map and different action station with contact 6 Hospital Mapping Revised Draft SERB Performance Evaluation Report ANNEX J SERB PERFORMANCE EVALUATION List and Short Briefs of Activities/Tasks Synergistic to SERB A. DGHS Title of the activity: Strengthen Health Emergency Preparedness and Response Program of Bangladesh (Jointly Implemented by EPR Program of NCDC, DGHS and WHO) Tasks undertaken: Task 1: Assess country capacity by using 12 SEARO EHA Benchmarks  Three (3) one-day long workshops at Sylhet, Rangamati & Rajshahi on 12, 15 and 23 November respectively and one (1) 3-day long consultative national level workshop held at Dhaka on 17, 18 & 19 December 2016 in order to assess the country Emergency Risk Management capacity by using 12 SEARO EAH benchmarks. Multi-sectoral representation was ensured in the all workshops for better assessment, relevant and effective suggestions and recommendations. Professionals/Representative from different Organization/Institutions/NGOs including various level health facilities, Fire Service and Civil Defense Office, Disaster Management Office, District Police Office, UN agencies and NGOs participated in the workshops. The participants were grouped into 6 sub-groups and each group worked on 2 benchmarks. Each group presented their group findings (achievements, gaps and recommendations) against each benchmarks. The group mainly recommended on timely allocation of budget & resources, capacity building training for field staff, development of EPR Plan, preparation of database of community level volunteers and NGOs, Strengthening of multi-sectoral coordination and conduction of health facility assessment. Task 2: Health Cluster Consultative workshop to finalize Barisal and Sylhet Division Health Emergency Preparedness and Response Plan  Health Cluster Consultative Workshops were held at Barisal and Sylhet on 02 & 30 August respectively and Finalized Health Sector Emergency Preparedness and Response Plan of Barisal and Sylhet Division. Before conducting the workshop, a team consisting of DGHS & WHO officials visited various health facilities of Barisal & Sylhet Division and met key health officials several times and drafted plans which were shared with local level health cluster partners. The participants of the workshop ranked the local level disasters by using Risk Ranking Matrix. The participants were grouped into 3 sub-groups for analyzing on minimum preparedness actions (MPA) for the plan. Participants for the workshop include all the Civil Surgeons, Some UHFPOs, Representatives from National and International NGOs, Fire Service & Civil Defense, Police, Anser & VDP Office and Department of Disaster Management. High Officials of DGHS and Office of Divisional Directors (Barisal and Sylhet), NIPSOM Faculty Member and NPO-EHA facilitated both the workshops. Task 3: Conduct training workshop on ERM for health and disaster management officials Revised Draft SERB Performance Evaluation Report  Two training workshops were conducted at Barisal and Chittagong on 12-13 April and 24-25 April 2016 respectively. The resource persons from DGHS, NIPSOM, and WHO conducted the sessions of the workshop and they were highly qualified and technically sound. The ‗training manual on Health Emergency Preparedness and Response for Health Professionals‘, which was newly developed, used in the workshop. The sessions were appreciated by all the participants and they quickly recognized the session objectives. Participants from different health facilities, local UN agencies, National and International NGOs participated actively in the workshop. The deliberation of the resource persons followed by interactive session by the participants opened up opportunity for understanding the issues and stimulated the participants greatly. Visual training on earthquake management, snakebite management and drowning management were shown to the participants. The participants understood their role in an emergency situation and also achieved more updated knowledge and skills regarding comprehensive disaster management. Task 4: Advocacy workshop for the functionality of health facilities  Advocacy workshops for the functionality of Health Facilities were held at Sylhet & Rangpur on 10 and 30 October respectively. The objective of the workshop was to orient hospital staff about the functionality of health facilities. Participants for the workshop include selected health officials and other disaster management officials of Sylhet and Rangpur Division. The participants emphasized to take appropriate action in order to develop disaster-resilience health facilities as well as mobilize field hospitals. They also mentioned that overcoming the devastating situation due to earthquake still is a great challenge. Appropriate health facilities resilience is essential for earthquake preparedness and response. Task 5: Development of communication strategy and plan of action in ERM in all hazards  Communication Strategy for emergency health risk management was developed by arranging 2 preparatory workshops and 1 final workshop in participation of relevant Health Cluster members. The 2 preparatory workshops were held on 19 and 30 May 2016 and final workshop was held on 28 July 2016. High level national officials of DGHS, NIPSOM, IEDCR, including Humanitarian Affairs Advisor of UNRC office, other UN agencies, Divisional Directors, Civil Surgeons were present as a resource person and contributed to develop this strategy. Major achievements & outcome: Based on 12 EHA benchmarks assessment, a comprehensive report has been produced with findings and recommendations, which will help to develop policy strategies and plans of action to strengthen the EPR program of health sector. Barisal and Sylhet Division Health Sector Emergency Preparedness and Response Plan developed for reducing avoidable mortality, morbidity and disability, and ensure the delivery of essential preventive and curative health care as quickly as possible in disaster situations. Trained Health Professionals were developed for health emergency management. This training workshop helped better coordination among the participants to identify and address the critical gaps for priority interventions. Revised Draft SERB Performance Evaluation Report Health professional were oriented about safe hospital in terms of structural, non-structural and functional aspects. Communication strategy and plan of action in ERM in all hazards developed for effective communication with disaster affected people, stakeholders, media and donor agencies. B. ActionAid Reference Book on Mass Casualty Management Key word: Hospital Safety, Mass Casualty Management Due to geographical location Bangladesh has been facing several devastating natural disasters and is prone to cyclones, tornadoes, flood, tidal-waves, and even earthquakes. In the recent years, Bangladesh also has experienced number of devastating disasters, such as landslide, building collapse, fire outbreak, terrorist attack etc. Some disasters may overwhelm the capacity of the hospital and its resources. Hospitals have a significant role to play in emergencies, disasters and mass casualty incident. This "Reference Book on Mass Casualty Management" will help to enhance the knowledge in managing mass casualties aiming with reducing the number of death, disability and sufferings of the affected people. The Reference Book also gives a guideline on Mass Casualty Incident and describes its goal from initial management perspectives. This Book has been developed and published by the ―Technical Committee‖ which has been formed by the Directorate Health Services, Bangladesh Government in collaboration with ActionAid Bangladesh and European Commission Directorate General for Humanitarian Aid (DG ECHO).  MCM ENglish.pdf Guidelines for Hospital Contingency Plan Key words: Hospital Safety, Mass Casualty Management, Hospital Contingency Plan Hospitals play a leading role in managing the injured during any kind of disasters. But it is also very likely that the hospitals, its staffs can also be affected in the disaster. The hospital building may even completely or partially collapse and this may break down the utility connections like water, electricity, gas supply etc. of the hospital. Communications may be disrupted among the staffs and the authorities of the hospital. Thus to ensure proper health care facilities of the affected population a Hospital Contingency Plan is very much essential during any disaster. This Guidelines for Hospital Contingency Plan has been developed and published by the ―Technical Committee‖ which has been formed by the Directorate Health Services, Bangladesh Government in collaboration with ActionAid Bangladesh and European Commission Directorate General for Humanitarian Aid (DG ECHO). It has been contextualized and developed based on the WHO Hospital Contingency Plan.  Hospital Contingency Plan Englsish.pdf Revised Draft SERB Performance Evaluation Report  Hospital Contingency Plan Bangla.pdf Guidelines for Mass Casualty Management Drill Key words: Hospital Safety, Mass Casualty Management, Hospital Drill, Guidelines for Hospital Drill Hospitals always are to play remarkable role in reducing the impacts and risks from various disasters on the lives of people. Thus, it is undoubtedly felt that Hospital based disaster preparedness plan and mass casualty management drills are very effective tools to ensure proper running of the hospitals to provide maximum support to the victims and hereby reducing the impact of any disaster. Mass Casualty Management Drill Guideline has thus been produced to assist the hospitals in conducting the drills by themselves. This Guidelines for Hospital Contingency Plan has been developed and published by the ―Technical Committee‖ which has been formed by the Directorate Health Services, Bangladesh Government in collaboration with ActionAid Bangladesh and European Commission Directorate General for Humanitarian Aid (DG ECHO).  MCM Drill English.pdf  MCM Drill Bangla.pdf A Guidebook on Earthquake Preparedness and Fire Safety Drill in Garment Factories. Key words: Earthquake Preparedness and Fire Safety Drill in Garment Factories, Safety of Garment Factory The capital city Dhaka is considered as one of the most vulnerable cities given increasing rates of unplanned urbanization and rampant industrialization. One of the sectors that have persistently exacerbated the threat to people in Dhaka is the Ready-made Garment factory. At present Dhaka houses thousands of garment factories. The workers, owing to their rural upbringing, are heedless to adequate safety standards. Regular mock drills, however, is drawing more popularity among garments workers and are one of the most effective tools to create awareness among garments workers in a very short span of time. Sensing this opportunity, ActionAid Bangladesh under the DIPECHO V project funded by the European Commission‘ Humanitarian Aid & Civil Protection Department in collaboration with the Fire Service and Civil Defence Department of Bangladesh and the Safety Cell of Bangladesh Garments Manufacturers and Exporters Association formulated a Guidebook on Earthquake Preparedness and Fire Safety Drill in Garment Factories. Apart from imparting the concepts of fire and earthquake drills, the guideline encompasses various technical and operational issues pertaining to the topic. This guidebook will benefit factory managements, laborers and others striving to achieve safety in garment factories. The apex body of garment sector, Bangladesh Garment Manufacturers and Exporters Association (BGMEA) and the Garment factory will be able to conduct the drills following this guidebook.  Guidebook on Earthquake Preparedness and Fire Safety Drill in Garment Factory.pdf Mass Casualty Management Revised Draft SERB Performance Evaluation Report Promoting Mass Casualty Incident Management and Hospital Safety Key words: Mass Casualty Management, Hospital Disaster Preparedness, Hospital Safety Some natural disasters like cyclones, floods and volcanoes hospitals are likely to receive advance warning and be able to activate their disaster plan before the event. For other natural disasters, such as earthquakes and landslides, and many man-made disasters such as chemical plant explosions, industrial accidents, building collapses and acts of terrorism do not provide advance warning. Quick response is one of the key objectives and a significant yardstick of success of effective mass casualty management during pre and hospital phase of disaster management. This requires special attention due to the vital functions hospitals and emergency responders perform. The hospital preparedness for disasters is a dynamic process and plays crucial role in easy mobilization of the staff, bed, equipment and supplies in a safe environment during any mass casualty or mass gathering incidents. Therefore, strengthening the capacity of casualty medical officers, hospital managers and surgeons on basics of mass casualty management is essential for any hospital‘s better disaster preparedness and effective response to the patients to avoid the situation of a secondary disaster. ActionAid Bangladesh has developed a comprehensive approach to promote Mass Casualty Incident Management and Hospital Safety. So far, AAB played the lead role to develop mass casualty management system and hospital contingency plan in Bangladesh. Since, DIPECHO started in 2007; AAB has implemented a series of initiatives to make hospital‘s better disaster preparedness and more effective in response. School Safety Plan Culture of Safety in Schools Key words: Culture of School Safety, Disaster Risk Reduction through Schools School safety and preparedness should become mandatory for each school which will allow students, teachers and management committee to bring Culture of Safety in schools. At the national level, disaster management and education sector can play greater role in ensuring mainstreaming disaster risk reduction orientation into educational programs and projects. Development partners and donors along with policy makers can emphasize the importance of school safety through allocating budget for preparedness and mitigation. This research work is an attempt to define culture of safety in schools and develop framework for that to highlight the linkage between child‘s right and disaster risk reduction. It also brings in to the investigation to understand the global initiatives on school safety and way forward for Bangladesh. The research work also develops the indicators of culture of safety in terms of safety audit, safer construction and education in emergencies.  Culture of Safety in Schools.pdf Revised Draft SERB Performance Evaluation Report ANNEX K SERB PERFORMANCE EVALUATION USAID’s Strengthening Earthquake Resilience in Bangladesh FSCD Tools and Equipment Maintenance Guideline Section I: Background USAID/Bangladesh’s SERB activity (June 2013 – May 2017) is implemented by ADPC, Thailand through partnership agreements with the National Institute for Preventive and Social Medicine (NIPSOM) and the Bangladesh Fire Service and Civil Defense Directorate (FSCD). One of the SERB tasks is provision of search and rescue (SAR) tools and equipment in 18 locations (17 fire service stations and 1 Training center) in the cities of Dhaka, Chittagong, Sylhet, Mymensingh, Tangail and Gazipur to strengthen the capacity of FSCD for effective search and rescue operations in mass casualty management especially in the event of earthquake. Below is the breakdown of the search and rescue items (Phase 1 and Phase 2):  Search Tools (71 Set) such as safety vest, fire extinguisher, K12 wrench, oil/fuel can, water spray bottle;  Rescue Equipment (31Set) such as chipping hammer, rotary rescue saw, reciprocating saw, rotary hammer drill, chain saw, hydraulic jack with lever, generator (diesel operated), sledge hammer, hammer, shovel, lock cutter, bolt cutter and key hole saw;  First Aid Kit (48 Set) containing medical kit with all equipment, long back board, splint, cervical collar and blanket; and  Storage Container (21 Set) made of steel. The list of tools and equipment SERB delivered as planned to the target fire stations included: a) Search Tools (71 Set) such as safety vest, fire extinguisher, K12 wrench, oil/fuel can, water spray bottle; b) Rescue Equipment (31 Set) such as chipping hammer, rotary rescue saw, reciprocating saw, rotary hammer drill, chain saw, hydraulic jack with lever, generator (Gasoline operated), sledge hammer, hammer, shovel, lock cutter, bolt cutter and key hole saw; c) First Aid Kit (48 Set) containing medical kit with all equipment, long back board, splint, cervical collar and blanket; and d) Storage Container (21 Set) made of steel. The FSCD volunteers are the main users of the tools and equipment provided by SERB. Section II: Responsibilities of FSCD Efficient storage, regular maintenance and proper disposal of equipment and tools help smooth and efficient equipment operation and avoid and limit health hazards. Lack of these measures will otherwise result in short shelf life of the equipment and will affect the Revised Draft SERB Performance Evaluation Report environment as well. So responsibilities of FSCD that are most important to effectively and efficiently operate and use of the S&R equipment and tools at their disposal include the following:  Ensure adequate spaces, safe locations and facilities for temporary/permanent storage of tools and equipment and their handling;  Orient FSCD Community volunteers on safe locations using the tools and equipment;  Conduct routine/periodical review of equipment components to verify that they meet the original design criteria for efficient operations  Conduct exercise plans for FSCD and its volunteers to test actual operations using the tools and equipment; maintenance of the equipment and tools; and  Disposal of equipment and tools at the end of their useful life in accordance with the plan to be developed jointly by ADPC and FSCD. Section III. S&R Equipment and Tools Maintenance Plan What is a Maintenance Plan? What are the types of maintenance? Why these are needed? What should a Maintenance Plan contain? How to implement the Maintenance Plan? - All these are addressed in the following sections. III.A. What is a Maintenance Plan? The Equipment Maintenance Plan, EMP as it is commonly called, is a document used when developing the tasks needed to properly maintain facility, plant or process equipment. The EMP helps lead the person or persons developing the required maintenance tasks by ensuring that the development is done consistently for all equipment. The FSCD Standing Operation Procedure on Drill Order among the various pre-field operation duties of the Senior Station Officers articulates that they shall ensure effectiveness and right use of firefighting equipment (Refer to Attachment B). The FSCD stations’ daily routine includes practical training/exercise on specialized equipment and their maintenance (Refer to Attachment C). III.B. What are the types of maintenance? Generally speaking, there are three types of maintenance in use: Preventive Maintenance (PM); Operational Maintenance (OM); and Corrective Maintenance (CM). PM is the most important - prevention is better than cure. Preventive maintenance, where equipment is maintained before break down occurs. Preventive maintenance is maintenance performed in an attempt to avoid failures, unnecessary production loss and safety violations. Recent studies have shown that Preventive maintenance is effective in preventing age related failures of the equipment. For random failure patterns which amount to 80% of the failure patterns, condition monitoring proves to be effective. So SERB gives importance on PM. Operational maintenance, where equipment is maintained in using. This is usually performed during routine exercises to test actual operations using tools and equipment. Revised Draft SERB Performance Evaluation Report Corrective maintenance, where equipment is maintained after break down. This maintenance is often most expensive because worn equipment can damage other parts and cause multiple damages. Corrective maintenance is probably the most commonly used approach, but it is easy to see its limitations. When equipment fails, it often leads to downtime in production. In most cases, this is costly business. Also, if the equipment needs to be replaced, the cost of replacing it alone can be substantial. It is also important to consider health, safety and environment issues related to malfunctioning equipment. Corrective maintenance is carried out on all items where the consequences of failure or wearing out are not significant and the cost of this maintenance is much greater than preventive maintenance. III.C. Why are these tasks needed? Conducting search and rescue operation after a major earthquake incident is a complex task that requires performing the right procedures from alert and mobilization of rescue teams to a quick assessment of needs, rescuing lightly trapped victims, performing search operations for the missing, extrication of heavily trapped victims using the appropriate tools and equipment, stabilization of condition of rescued victims and eventually transporting to a medical facility for emergency treatment and turn over for advance care, if needed. All of these steps require well précised actions considering safety not only for the victims but most specially for the rescuers themselves. III.D. What should a Maintenance Plan contain? A big part of safety consideration during rescue operations is the use of appropriate tools and equipment that will affect the safety during operations. Proper maintenance practices for safe deployment and being mission ready, proper training of rescuers in operating the tools and equipment and having a safety operations plan are key to having a safe search rescue operations. III.E. How to implement the Maintenance Plan? Assign individual or group to ensure the continued operation and maintenance of an equipment item. Each individual or group may be assigned one or more maintenance tasks out of the three types of tasks (PM, CM and OM) described above. The table below provides an illustration of maintenance tasks assigned to an individual or a group of FSCD station staff and volunteers. About 50% of the volunteers will be selected from those have received the refreshers training conducted by SERB. Table: FSCD Station Staff Assignment of Maintenance Tasks Assigned Task / Work Group Maintenance Task Preventive Maintenance (PM) Estimated Time to Perform Task Operational Maintenance (OM) Estimated Time to Perform Task Corrective Maintenance (CM) Estimated Time to Perform Task Individual/Group 1 X - X Individual/Group 2 - X X Individual/Group 3 X - - Revised Draft SERB Performance Evaluation Report Individual/Group 4 - X - Each of the fire stations will follow the following tasks under this S&R Tools and Equipment Maintenance Plan for SERB S&R items:  Program approved Equipment and Tools Maintenance Plan in place.  A descriptive title for each maintenance task to be performed individually or by groups including estimated time to perform the task (Refer to the Table above).  Identify the maintenance in-charge.  Maintenance of EMP records and reports.  Maintenance or Servicing Manual those are supplied by the vendor.  Availability of in-house service facilities and may be outsourcing (if needed).  Follow the set protocols for equipment and materials maintenance to control contamination by grease, oil and fuels (mechanized equipment, such as Generator).  Handle, store, use and process branded materials in accordance with manufacturer’s instructions and recommendations.  Follow the maintenance schedule for inspection and reporting performance conditions. A frequency assigned for performing of each task. Refer to the FSCD Daily Maintenance Schedule (Attachment 1).  Availability of special tools, materials and equipment required to perform the task.  Periodic review system components to verify that they meet the original design criteria for efficient operations.  Encourage warehouse staff to use or refer to the warehouse management manual while on duty.  Availability of the equipment and materials management manual as reference tool in the warehouse.  Follow the disposal plan and procedures. [Disposal of equipment and materials especially those made of metal and non-biodegradable substances may pose environmental and human health risks. ADPC in consultation with FSCD will develop the plan for disposal of equipment and materials in an environmentally sound and safe manner consistent with best management practices according to USG, European Union or equivalent standards.] Section IV: Maintenance Plan Progress Reporting FSCD will assign a focal point at each of the 18 locations (17 Stations and 1 Training Complex) for monitoring and reporting of the maintenance plan implementation. The focal point person is responsible for maintenance work record keeping. FSCD through ADPC Dhaka office will contact the vendors for repair and/or replacement of equipment/tools or its spare parts as per guarantee within the warranty period. FSCD will keep ADPC posted of any such cases. Revised Draft SERB Performance Evaluation Report Revised Draft SERB Performance Evaluation Report ATTACHMENT 1 TO ANNEX K Maintenance Schedule for SERB-Equipment Fire Service and Civil Defense Directorate Sunday Monday Tuesday Wednesday Thursday  Chipping Hammer  Rotary Rescue Saw  Reciprocating Saw with Blade  Rotary Hammer Drill with Bit  Sledge hammer  Mega phone with Battery Charger  Work lamp  Safety Cone  Measuring Tape  Cribs/Shim  Scene Tape  Safety Vest  Spray Paint Orange  Tarp  Electric Extension Cord  Bucket  Water Jar  Oil Cane  K12 Wrench  Water Spray  Chain Saw with Chain  Generator  Hack Saw with Blade  Lock Cutter  Mega phone with Battery Charger  Safety Cone  Measuring  Scene Tape  Safety Vest  Electric Extension Code  Water Jar  Plywood  K12 Wrench  Water Spray  Shovel  Safety Cone  Safety Vest  Water Jar  K12 Wrench  Water Spray  Medical kits with all equipment  Long back board  Splint  Cervical Collar  Blanket  Lock Cutter/Bolt Cutter  Pry Bar  Crow Bar  Chisel  Tin Snip  Key hole Saw(Full set)  Hand saw  Safety Cone  Safety Vest  Fire Extinguisher  Water Jar  K12 Wrench  Water Spray Fuel type equipment (Octane/Petrol) Oiling (Graze)  Generator  Chipping Hammer  Rotary Rescue Saw  Reciprocating Saw with Blade  Rotary Hammer drill with Bit  Chain Saw with Chain  Lock Cutter/Bolt Cutter Maintenance Day : All equipment will be cleaned / made oiling on this day. Note: Kindly note that as per above mentioned plan, equipment should be used during training session / practical session for proper utilization and preventive measures