SUMMATIVE PERFORMANCE EVALUATION OF USAID/OFDA’S DISASTER RISK REDUCTION (DRR) PROGRAMMING IN NEPAL Contract No. AID-OAA-I-15-00028; Task Order: AID-OAA-TO-16-00039 Evaluation Report July 27, 2017 This publication was produced at the request of the United States Agency for International Development. It was prepared by The Mitchell Group, Inc. (TMG) with the assistance of IMC Worldwide and the Center of Resilience Development (CoRD) in Nepal. . SUMMATIVE PERFORMANCE EVALUATION OF USAID/OFDA’S DISASTER RISK REDUCTION (DRR) PROGRAMMING IN NEPAL Evaluation Report Contract No. AID-OAA-I-15-00028; Task Order: AID-OAA-TO-16-00039 July 27, 2017 DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION I TABLE OF CONTENTS EXECUTIVE SUMMARY................................................................................................ 1 1. PROGRAM BACKGROUND .................................................................................... 4 1.1. PEER Program ...............................................................................................................................................4 1.2. BCIPN Program .............................................................................................................................................6 1.3. Theory of Change............................................................................................................................................8 2. PURPOSE, USE AND QUESTIONS....................................................................... 10 2.1 Evaluation Purpose.......................................................................................................................................10 2.2 Evaluation Use ..............................................................................................................................................10 2.3 Evaluation Questions ...................................................................................................................................10 3. METHODS AND LIMITATIONS .......................................................................... 12 3.1 Evaluation Methods......................................................................................................................................12 3.2 Data Limitations ...........................................................................................................................................13 4. FINDINGS & CONCLUSIONS .............................................................................. 15 4.1. Findings & Conclusions for PEER Evaluation Questions ........................................................................15 4.2. Findings & Conclusions for BCIPN Evaluation Questions ......................................................................29 4.3 GESI Distributions Among Training Participants .....................................................................................36 5. RECOMMENDATIONS......................................................................................... 39 5.1 OFDA-WIDE recommendations .................................................................................................................39 5.2 Nepal-specific recommendations ................................................................................................................39 NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION II List of Tables Table 1 Summary of PEER Programs ............................................................................................................................ 5 Table 2 Numbers trained under PEER 3 and 4 through October 2016 ................................................................... 5 Table 3 Numbers trained by BCIPN through December 2016 .................................................................................. 7 Table 4 Comparative Scores of PEER Training Participants....................................................................................16 Table 5 Re-tests of PEER Participants by Year of Training .....................................................................................17 Table 6 Rank Ordering of Skills Retention by MFR Training Participants.............................................................17 Table 7 Rank Ordering of Skills Retention by CSSR Training Participants............................................................18 Table 8 Rank Ordering of Skills Retention by CADRE Training Participants ......................................................18 Table 9 Rank Ordering of PEER Skills Usefulness during and after Gorkha Earthquake ..................................19 Table 10 Types of PEER Skills Ranked by PEER Trainees as Highest in Saving Lives after the Gorkha Earthquake.........................................................................................................................................................................20 Table 11 Rank Ordering by MFR training participants of course usefulness during and after Gorkha Earthquake.........................................................................................................................................................................20 Table 12 Rank Ordering by CSSR training participants of course usefulness during and after Gorkha Earthquake.........................................................................................................................................................................20 Table 13 Rank Ordering by HOPE training participants of course usefulness during and after Gorkha Earthquake.........................................................................................................................................................................21 Table 14 Rank Ordering by CADRE training participants of course usefulness during and after Gorkha Earthquake.........................................................................................................................................................................21 Table 15 CSSR trainee responses to open-ended questions in FGD and KIIs......................................................23 Table 16 CADRE trainee responses to open-ended questions in FGD and KIIs.................................................23 Table 17 HOPE trainee responses to open-ended questions in FGD and KIIs..................................................24 Table 18 MFR trainee responses to open-ended questions in FGD and KIIs .....................................................24 Table 19 Comparative Scores of Level 0 and Level 1 PEER Training Participants.............................................25 Table 20 Pre-, Post- and Re-tests Scores of Masons and Engineers benefiting from BCIPN ...........................30 Table 21 Knowledge Gained from BCIPN Trainings (municipal officials and social mobilizers) ....................31 Table 22 Municipal Systems Facilitating Compliance with NBC in BCIPN-targeted Municipalities.................32 Table 23 Adherence of Drawings to NBC and Buildings to Drawings in BCIPN-targeted Municipalities......33 Table 24 Sex of PEER training participants re-tested and surveyed by the evaluation team .............................37 Table 25 Social groupings of PEER training participants re-tested and surveyed by the evaluation team.......37 Table 26 Sex of BCIPN training participants re-tested and surveyed by the evaluation team ...........................37 Table 27 Social groupings of BCIPN training participants re-tested and surveyed by the evaluation team.....38 Table 28 Nepal-Specific Conclusions, Implications and Recommendations.........................................................40 NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION III ANNEXES (UNDER SEPARATE COVER) ANNEX A: STATEMENT OF WORK ANNEX B: EXPERTISE, ROLES AND RESPONSIBILITIES OF THE EVALUATION TEAM ANNEX C: UPDATED EVALUATION AND IMPLEMENTATION SCHEDULES ANNEX D: SOURCES OF INFORMATION • D.1 Bibliography of documents reviewed • D.2 List of persons interviewed • D.3 Structured questionnaire results – Tabular format • D.4 Database of participant re-tests, post-tests and pre-tests – Tabular format NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION IV ACRONYMS ADPC Asian Disaster Preparedness Center ADS ARC Automated Directives System, USAID’s Operational Policy American Red Cross BCIPN Building Code Implementation Program in Municipalities of Nepal CADRE Community Action for Disaster Response CPR CSSR Cardio Pulmonary Resuscitation Collapsed Structure Search and Rescue DRR Disaster Risk Reduction DUDBC EQs Department of Urban Development and Building Construction Evaluation Questions FGD FY Focus Group Discussion Fiscal Year GESI Gender Equality and Social Inclusion GON HOPE Government of Nepal Hospital Preparedness for Emergencies IOM IP JICA KIIs MFR Institute of Medicine Implementing partner Japan International Cooperation Agency Key Informant Interviews Medical First Responders MoFALD NRCS Ministry of Federal Affairs and Local Development Nepalese Red Cross Society NSET National Society for Earthquake Technology NBC NGO National Building Code Non-Governmental Organization NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION V OFDA Office of U.S. Foreign Disaster Assistance PEER Program for Enhancement of Emergency Response SOW Statement of Work SWR Swift Water Rescue TFI Training for Instructors TOT Training of Trainers UNDP USAID WHO United Nations Development Program U.S. Agency for International Development World Health Organization NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 1 EXECUTIVE SUMMARY This is a summative performance evaluation of disaster risk reduction (DRR) programming in Nepal funded by USAID’s Office of U.S. Foreign Disaster Assistance (OFDA) between 2009 and 2016. It is focused on the performance of two programs: The Program for Enhancement of Emergency Response (PEER), Stages 3 and 4, and the Building Code Implementation Program in Municipalities of Nepal (BCIPN). The purpose of the evaluation is to improve OFDA’s understanding of the performance of its DRR activities in Nepal. To the extent possible, the evaluation has utilized the Gorkha Earthquake of 2015 and its powerful aftershocks as one means to assess the degree to which those activities were successful in achieving their desired outcomes. Methods and Limitations The evaluation utilized a mixed-methods design that included written re-tests of hundreds of training participants on the skills they learned through OFDA-funded DRR capacity building activities, utilizing the same tests that were administered immediately after they were trained. Statistical tests were then used to determine the degree to which the skills they had acquired were retained over time. This was supplemented by structured interviews of the training participants that consisted of both multiple choice and open-ended questions; focus group discussions (FGDs); and key informant interviews (KIIs). The methodology also included comparison of evidence obtained through site visits to OFDA-targeted and non-OFDA-targeted municipalities to see whether building code compliance appeared to be better in the former. Pre- and post-tests were not administered by the implementing partners to all categories of training participants, and so in these cases the evaluation had to rely only on qualitative evidence. Another limitation was that, for the PEER program trainees, it was not possible to observe practical skills in application. This was ameliorated by asking participants to self-rank their practical skills retention. The lack of information on trainees ‘two levels removed from OFDA’s Implementing Partner (IP)’ meant that it was not possible to assess the acquisition or retention of skills by these trainees. And the fact that several other donor entities had been working throughout Nepal to improve municipal adherence to the National Building Code (NBC) contributed to making it difficult to document differences between OFDA-targeted and non-OFDA￾targeted municipalities on this measure. Data Collection For the PEER program, the evaluation team administered re-tests, followed by structured interviews, of 43 Medical First Responder (MFR), 96 Collapsed Structure Search and Rescue (CSSR) and 44 Hospital Preparedness for Emergencies (HOPE) professionals trained by OFDA. They also completed structured interviews of 41 Community Action for Disaster Response (CADRE) trainees, 29 key informant interviews and four focus group discussions. For the BCIPN program, the evaluation team administered re-tests followed by structured interviews of 244 masons and 88 engineers trained by OFDA. They also completed structured interviews of 86 municipal officials/social organizers who participated in OFDA-sponsored orientations, 25 key informant interviews and 20 focus group discussions in 10 municipalities. Structural engineers reviewed 300 building permits for adherence to the NBC, made site visits to 60 of the permitted buildings, and conducted a rapid assessment walkabout in each of the 10 municipalities to review new construction compliance. NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 2 PEER: Evaluation Question A: To what extent has the training provided to project beneficiaries resulted in improved skills for responding to disasters in their specific respective roles? Findings. Direct re-tests of hundreds of MFR, CSSR and HOPE professionals in Nepal tended to confirm that the skills they acquired from OFDA-funded training to help them respond to disasters were retained. The retention of skills acquired was significant whether they were trained directly by foreign OFDA instructors, or by OFDA-trained Nepali instructors. There was no evidence to support the hypothesis that there was a pattern of progressive skills decline over time. In structured interviews, 88% of CSSR, 94% of HOPE and 93% of CADRE training participants stated that they actively participated in responding to the Gorkha earthquake, and they said that the training increased their preparedness to handle the challenges of that catastrophe. While earthquakes are rare events, accidents, fires, and other more frequent disasters were also reported to be regularly responded to by these professionals; particularly by the CSSR and HOPE training participants. Responding to these disasters contributed to reinforcing the skills they had learned in PEER training. CADRE trainees also tended to confirm that they were better able to retain the skills that they regularly practiced. The evaluation team also found that, during a period of nearly two decades of PEER training activities in Nepal, there has been significant integration of the training curricula and materials into the training programs of Nepali institutions. Nonetheless, although regional standards were maintained, adaptation of the PEER course content to local context and institutional capacities has been relatively limited. Beyond this, training participants and key informants reported that emergency response organizations and health facilities at lower administrative levels, which are often the closest to disaster sites, cannot sufficiently benefit from DRR training without a sustainable institutional budget for training. Lack of sufficient funding by the Government of Nepal (GON) of search and rescue equipment and medical materials purchase and maintenance remain key constraints. Conclusions. The principal conclusion of the evaluation team was that the skills OFDA imparted through PEER training have been retained over time, and this was true whether the training was performed directly by expatriate OFDA instructors, or by OFDA-trained Nepali instructors. Both of these program outcomes represent key elements of impact sustainability; in other words, evidence that the project’s impact will survive once project funding ends. Since emergency response professionals and volunteers attend to a wide range of emergencies, including road accidents and fires, it is reasonable to conclude that where those skills acquired through PEER training are exercised, they tend to be better retained. It was also the case that the PEER training curricula and materials have influenced the training programs of relevant institutions (e.g., Nepal Police, Armed Police, Army, public and private hospitals) and NGOs (e.g., Nepal Red Cross Society). Nonetheless, without sustainable funding for the GON to provide DRR training to lower administrative levels, and some minimal sustainable budget for the purchase and maintenance of search and rescue equipment and medical materials, it will be difficult to put the PEER skills training to good use in future emergencies, reducing the program’s potential impact. BCIPN: Evaluation Question B: To what extent has the project strengthened the capacity of local governments to effectively administer building code regulations? Findings. Direct re-tests of hundreds of masons and engineers tended to confirm that the skills they acquired from OFDA-funded training have increased their knowledge of earthquake-resistant building techniques adhering to the NBC. While no information on the knowledge of municipal government officials and social mobilizers prior to BCIPN training could be recovered from the implementing partner, in KIIs and FGDs, these key stakeholders confirmed that BCIPN orientation sessions had enhanced their knowledge of disaster preparedness, the cause of earthquakes and challenges in implementing the NBC. The evaluation team was unable to confirm, however, that those municipalities targeted by the BCIPN program performed better on adherence to the NBC. This may have been the case, in part, because many other donor activities had provided capacity building similar to that funded by OFDA, in non-targeted NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 3 municipalities, especially following the Gorkha earthquake. Key stakeholders cited a variety of factors that contributed to their ability to adhere to the NBC, including higher public demand following the earthquake and the orientation programs provided by donors. Factors inhibiting adherence to the NBC included higher costs of construction, continued lack of house owner awareness, and lack of municipal enforcement. Finally, in regard to benefits to their livelihood, 69% of masons interviewed said that their wages had increased following the BCIPN training and 90% said that BCIPN training helped them to gain employment. Conclusions. The main conclusion of the evaluation team was that OFDA-funded BCIPN training had increased the knowledge of engineers and masons about earthquake-resistant building skills, as embodied in the NBC. Further, this knowledge had been retained over time, a key element of impact sustainability. The structured interviews, KIIs and FGDs confirmed that on the whole, training participants, including municipal officials, highly valued and put to good use the skills they acquired from BCIPN concerning adherence to the NBC. Nonetheless, while NBC adherence greatly improved following the Gorkha earthquake, municipal enforcement remains a major constraint. Unfortunately, the capacity of municipalities to enforce adherence to the NBC is as much an issue of resources and political will as it is of technical knowledge, issues that go beyond capacity building. Recommendations OFDA-wide recommendations. While OFDA does require performance reporting within the context of its DRR awards, it is not mandated to conform to USAID’s ADS 201. Nonetheless, if the purpose is to improve OFDA’s programming decisions for future DRR activities, the Office might consider requiring delivery of Implementation Plans that articulate clear theories of change; performance indicators and targets for achievement of program objectives; and exit strategies for eventual program closure with sustainable impact. Second, as a disaster response institution, OFDA’s own theory of change must implicitly be to coordinate with development institutions (including USAID Missions), and host-country authorities in the countries in which it sponsors DRR activities to foster work on the longer-term development tasks required to achieve impact sustainability and scalability, and so it is recommended that this aspect be made explicit in its DRR theory of change and that of its implementing partners. PEER Program. OFDA should continue to progressively hand off MFR, CSSR and HOPE capacity building activities in disaster risk reduction to GON entities in accordance with an established exit strategy. This effort should include integration of the DRR training programs and materials into GON institutions and support for their dissemination to decentralized levels, as well as the establishment of mechanisms for periodic updates of the training materials and for ensuring that DRR skills are refreshed. The effort should also address involvement of senior levels of relevant Nepalese Ministries in DRR activities to help facilitate better management and coordination, as well as ownership. Finally, it should address how to inventory and ensure adequate distribution to critical locations, as well as maintenance, of medical and search & rescue equipment and materials. BCIPN Program. OFDA should adhere to a role of piloting, standardizing and then progressively devolving training programs for masons, engineers and municipal staff in building code compliance to Nepalese agencies and development institutions. To follow up on the Knowledge, Attitudes, and Practices (KAP) survey, a strategy should be developed on how best to encourage bottom-up demand by house owners for more earthquake resistant buildings. OFDA should also consider facilitating one or more studies designed to frame the way forward in terms of the policy and institutional reforms needed to encourage building code compliance in Nepal. NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 4 1. PROGRAM BACKGROUND This summative performance evaluation covers disaster risk reduction (DRR) programming in Nepal funded by USAID’s Office of U.S. Foreign Disaster Assistance (OFDA) between 2009 and 2016. The evaluation was carried out from October 11, 2016 through July 31, 2017. It is focused on the performance of two DRR programs supported by OFDA: The Program for Enhancement of Emergency Response (PEER), Stages 3 and 4, and the Building Code Implementation Program in Municipalities of Nepal (BCIPN). OFDA has supported natural disaster preparedness and resilience in South Asia for decades. For Nepal specifically, approximately $17 million has been invested in DRR since 1998.1 These investments have primarily taken the form of grants and cooperative agreements with non-governmental organizations to implement DRR programs in close coordination with both national and sub-national authorities of the Government of Nepal. 1.1. PEER PROGRAM The PEER program's aim is to reduce mortality during disasters and increase the survival rates of disaster victims by enhancing professionalism and community-based disaster preparedness, as well as the response capacity of public, private, academic, social und non-governmental organizations. The program aims, in each of the countries in which it works, to reduce mortality in mass casualty events and increase the survival rates of disaster victims, through the following activities: 1) Spread knowledge of Medical First Responders (MFR), Collapsed Structure Search and Rescue (CSSR), Hospital Preparedness for Emergencies (HOPE) and Community Action for Disaster Response (CADRE) and develop the capacity of key actors in disaster-response in these topics; 2) Create a functioning and sustainable network or professional association that is able to implement and coordinate the courses on its own; and 3) Initiate a knowledge and skills certification procedure for MFR and CSSR professional first responders proficiency, and for CADRE community volunteers. The PEER program has been implemented in fourdistinctstages by different implementing partners.The first stagewas implemented from 1998 to 2003 by theAsia Disaster Preparedness Center(ADPC). The second stage was implemented from 2003 to 2009 by the National Society forEarthquakeTechnology (NSET).The third stagewasimplemented from 2009 to 2014 by bothNSETand ADPCjointly,withNSETfocusing on the MFR and CSSR trainings andADPC focusing on the CADRE and HOPE trainings. The fourth stage was awarded to NSET in October of 2014 and is expected to run through 2019. 2 PEER is a regional program that aims to enhance local and regional disaster preparedness and response capacities of vulnerable countries within the Asia region -- including Bangladesh, Cambodia, India, Indonesia, Laos, Nepal, Pakistan, the Philippines, Thailand, and Vietnam -- through the institutionalization of sustainable disaster preparedness training programs and emergency response systems. Standardized training covers four core areas of medical first response (MFR), collapsed structure search and rescue (CSSR), community action for disaster response (CADRE), and hospital preparedness for emergency (HOPE) following a disaster. The MFR course provides individuals with the first response knowledge and skills necessary to assess, treat, 1 Source: Task Order for the Evaluation. 2 The total budget for PEER 3 was $4.7 million (Source: 4th Modification of Assistance Award - February 2012), with nearly $3.7 million awarded by OFDA, $50,000 of in-kind contributions and $1 million (from other sources). The total budget for the entire PEER 4 is $4.7 million (Source: 1st Modification of Assistance Award - August 2015). NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 5 and transport sick or injured patients following an emergency or disaster. The course is delivered over a 13- day period. The CSSR course prepares emergency first responders to search for, stabilize, and extricate victims trapped by collapsed structures. The MFR course is a pre-requisite to the CSSR course; however, not all MFR graduates proceed to qualify for or undergo the CSSR course. HOPE is a training course for hospital staff and healthcare personnel, both medical and non-medical, to prepare healthcare facilities to respond effectively to emergencies involving large numbers of casualties. In Nepal, most PEER courses, including all HOPE courses, ran under the Partial Funding Assistance Program, in which country partners submitted counterpart resources to conduct the courses. The CADRE training is a course that can teach anyone how to prepare for disasters in their communities. The training incorporates simple elements of medical first response and collapsed structure search and rescue into a modular course (3 days under PEER 3, extended to 4 days under PEER 4). In Nepal, community action in the event of a disaster was essential due to the remoteness of many at-risk locations. Table 1, below, summarizes PEER 3 and 4 programs, during the period under review by the evaluation. Table 1 Summary of PEER Programs PEER PROGRAM DESCRIPTION PEER 3 (2009 to June 2014) Capitalizing on the successes of PEER 1 and 2, the objective was to establish a system for enhanced community-level first responder capacity in disaster-prone communities through CADRE with supplemental funding support by the American Red Cross (ARC) and support by the Nepalese Red Cross Society (NRCS); improve the capacity of hospitals and medical facilities to manage mass casualty incidents through HOPE; improve medical response and search and rescue capacities of the armed forces and the Nepal Police through MFR and CSSR; and train in-country instructors for long-term sustainability through instructor training workshops PEER 4 (October 2014- September 2019) To capitalize on the successes of Stages 1 to 3, the objective of this stage is to further improve the capacity of hospitals and medical facilities by expanding HOPE and enhancement of the capacities of the Nepal Police, Army and Armed Police Force in MFR and CSSR. During this stage CADRE continued as a separate program outside of PEER 4 with additional funding by OFDA directly to ARC for implementation by NRCS. PEER 4 also seeks to encourage local ownership of disaster preparedness and response capacity building efforts by strengthening engagement with national, public, and private institutions. Training for Instructors and Training for Master Trainer Instructors Workshops were undertaken by ADPC and NSET to create a cadre of trained staff within the national partner organizations to conduct their respective training activities and support others. In total, there are up to nine steps of training, including training for the course monitors and coordinators. Table 2, below, summarizes the numbers of trainees participating in the PEER 3 and 4 program through October 2016. Table 2 Numbers trained under PEER 3 and 4 through October 20163 End users Instructors Coordinators/monitors CSSR 274 87 MFR 311 113 HOPE 340 67 CADRE 169 47 Total 1,094 314 27 3Data provided to the evaluation team by NSET, November 2016. NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 6 PEER 4 is implemented by a number of different implementing partners. This evaluation does not examine the entirety of the PEER 4 program. It only examines the portion of PEER 4 implemented by NSET in Nepal from 2009 to 2017. 1.2. BCIPN PROGRAM The majority of buildings in Nepal are poorly constructed and extremely vulnerable to earthquakes due to unplanned growth, poor implementation and enforcement of building codes, and a lack of awareness by the public and building professionals of seismically safe construction practices. While legislation has in recent years made compliance with building codes mandatory, municipalities lack appropriate mechanisms and capacities for building code implementation and enforcement. The BCIPN program focuses on assisting municipal governments in Nepal to enhance their capacity to develop and properly administer the building permits and control system in compliance with the National Building Code (NBC), to ensure improved seismic performance of all new building construction in those urban and urbanizing areas of Nepal where compliance has been made mandatory by law. This approach entails a number of interconnected activities, including: • Assisting municipalities to develop mechanisms for building code implementation through certification and registration of trained masons, municipal policy, and a field monitoring system. • Providing training to masons on earthquake resistant construction. • Providing training to engineers and architects on earthquake resistant construction and design. • Conducting a home owners’ orientation program. • Preparing earthquake scenarios and earthquake risk management action plans. • Conducting exposure visits for municipal officials to other municipalities. • Organizing lessons-learned and sharing workshops on earthquake resistant construction. BCIPN was designed in collaboration with the Ministry of Federal Affairs and Local Development (MoFALD) and the Department of Urban Development & Building Construction (DUDBC), and implemented in collaboration with municipalities. The program was awarded to the National Society for Earthquake Technology – Nepal (NSET) with a budget of $1.89 million, 4 extending from June 2012 – June 2017. BCIPN aims to improve earthquake disaster resilience and enhance earthquake resilience of urban settlements in Nepal by raising public awareness of building safety regulations in seismically active areas, developing capacity of municipal professionals to effectively implement the NBC as well as developing policy recommendations to improve residential building codes. Initially covering 24 municipalities, the BCIPN is now implemented in 30 municipalities5 throughout Nepal.6 The training courses for masons were delivered over five days (subsequently expanded to seven days) and dedicated to the construction of earthquake resistant buildings. The main theoretical and practical sessions included the following: • Effects and causes of earthquakes • Quality and loadbearing capacity testing of slab, StoneCrete blocks, and concrete bricks 4 The total cost of the program is $1,893,438 (Out of the total amount, $1,721,307 was awarded by OFDA and $172,131 is generated by NSET as in-kind contributions) – Source: 2nd Modification of Assistance Award (signed in August 2014). 5 The municipalities were selected by taking into account the following criteria: population of municipality (10%), number of building construction permits per year (15%), Minimum Conditions & Performance Measure (MCPM) ranking (15%), NBC adherence level (15%), level of interest and enthusiasm towards NBC implementation (20%), technical manpower (10%), accessibility by road (5%), proximity to other municipalities (5%) and availability of Geographic Information Systems (GIS) data (5%). 6 In a baseline survey conducted in 2015 on ‘Knowledge, Attitudes and Practices (KAP),’ NSET polled 39,874 respondents in 23 of the municipalities supported by BCIPN concerning their perceptions and awareness of earthquake risk. NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 7 • Layouts and configurations of earthquake resistant buildings • Loadbearing masonry buildings • Reinforced concrete building construction • Construction technologies with timber and stone • Quality of construction materials • Repairing minor damages • Non-structural mitigation measures and earthquake preparedness • Testing of slab bar for loadbearing capacity and position of reinforcement bars • Role of masons in convincing house owners about the benefits of constructing earthquake resistant buildings The training courses for engineers and sub-engineers focused on earthquake resistant design and construction of buildings. The main sessions of these three-day or four-day trainings consisted of the following: • Key features of the NBC • Earthquake resistant construction of masonry buildings and earthquake resistant construction of reinforced concrete buildings • Group exercises on appropriate configurations of earthquake resistant buildings • Repairing and strengthening existing buildings • Quality of materials and workmanship The awareness raising sessions for the house owners presented the causes and effects of earthquakes, made them aware about critical construction stages and demonstrated the importance of implementing the NBC. The awareness raising sessions in the municipalities for local social mobilizers, community leaders and municipality staff consisted of orientation sessions on earthquake risk reduction measures and the importance of implementing the NBC. Table 3, below, summarizes the numbers of Masons, Engineers, House Owners, and Municipality Staff, Social Mobilizers and Community Leaders and trained by the BCIPN program through December 2016. Table 3 Numbers trained by BCIPN through December 20167 Numbers trained Males Females Total Masons 4,204 48 4,252 Engineers 919 124 1,043 House Owners 1,218 350 1,568 Social Mobilizers, Community Leaders and Municipality Staff8 2,462 2,142 4,604 7 Data provided to the evaluation team by NSET, December 2016. 8 Participation in awareness and orientation program activities from FY2013 onwards. NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 8 1.3. THEORY OF CHANGE A document prepared in 2011, ‘U.S. Disaster Risk Reduction Framework – Nepal, FY2012–FY2016,’9 provided general guidance for United States Government engagement in DRR in Nepal. It presented summary information from an interagency assessment and outlined illustrative activities and outcomes to be considered under the framework. The strategic framework outlined the U.S. Government’s DRR theory of change in Nepal, in three parts, as follows: Objective 1: Prepare to Respond If the response capacity (human and systemic) in Nepal is enhanced, then preparedness for, and response to, natural disasters will be effective, efficient and timely. Objective 2: Advocacy & Awareness If awareness and advocacy campaigns make DRR relevant to people’s lives, then the society as a whole will take responsibility for reducing its risks from natural disasters. Objective 3: Mitigation If Nepal incorporates DRR into all sectors, then investments and systems will be better protected and more resilient. OFDA’s direct contributions to achieving these three objectives through PEER and BCIPN are its training activities targeting relevant stakeholders, as well as in the case of BCIPN, broader community awareness strengthening activities and some technical support to municipal authorities. Critical assumptions of the strategic framework include continued inter-agency coordination of personnel and resources for development projects/programs, and the motivation and investment of the GON to improve and maintain its DRR capacities. The strategic framework also anticipated that an inter-agency Disaster Working Group at Post would define specific outcomes and activities when they formulated annual work plans for disaster risk reduction in Nepal, and also would establish a monitoring and evaluation plan for the inter-agency DRR work. There does not appear to have been any follow through on the inter-agency M&E Plan, which might have been expected to contain explicit exit strategies for the DRR activities being implemented by OFDA and other inter-agency partners. Nonetheless, it is clear that exit strategies that would ensure the sustainability of OFDA’s DRR programs were very much under discussion during the design of both the PEER 3 and 4 programs and the BCIPN program. References to a sustainable exit strategy within the PEER 3 program appear mainly relating to American Red Cross (ARC) activities (ADPC proposal, August 2008) and a separate ARC proposal in September 2011. But NSET’s proposal for the PEER 4 program, dated July 2014, refers explicitly to “…built-in strategies to establish a self-sustaining system and process which should ultimately be driven perpetually by the country’s institutions. The emphasis is on institutionalization and transfer of ownership…" In regard to the BCIPN program, NSET’s proposals (dated September 2012 and August 2014) contained explicit sections on transition and exit strategy, stating that “…NSET’s strategy will be to continue engagement in areas that show success and fulfillment of its objectives, developing new methods and helping the stakeholders to implement new initiatives.” It also states that “…NSET is determined to taper its involvement, especially the financial contribution for hiring the additional technical manpower” in targeted municipalities. 9 Prepared by the USAID/Nepal Disaster Risk Reduction Office with support from the U.S. Embassy in Nepal, USAID/DCHA/OFDA, USAID/DCHA/CMM, USAID/DCHA/OCR, and the Center for Excellence in Disaster Management & Humanitarian Assistance. NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 9 The issue that is implicitly addressed by these exit strategies is that, since OFDA is a disaster response institution, and not a development institution, it relies on coordination with other entities to foster institutional sustainability of the results of its capacity building activities. Accordingly, one underlying assumption concerning OFDA’s own theory of change is that it will introduce high standards to the capacity building it supports as part of the inter-agency DRR framework for Nepal, and then follow through to coordinate with development institutions (including the USAID Mission) and Nepalese agencies to foster work on the longer-term development activities required sustainably to achieve the inter-agency team’s DRR objectives. NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 10 2. PURPOSE, USE AND QUESTIONS 2.1 EVALUATION PURPOSE The purpose of the evaluation is to improve OFDA’s understanding of the performance of its disaster risk reduction (DRR) activities in Nepal, within both the PEER program and the BCIPN program. It is a summative evaluation focused on the performance of key activities funded by OFDA and the outcomes they achieved. To the extent possible, the evaluation has utilized the Gorkha Earthquake of 2015 and its powerful aftershocks as one means to assess the degree to which OFDA’s DRR activities in Nepal were successful in achieving their desired outcomes. 2.2 EVALUATION USE The primary user of the evaluation will be OFDA itself, which wants to develop a clear understanding of the performance of its DRR approach in Nepal. The evaluation will, in turn, inform future OFDA disaster risk reduction programming in Nepal in particular, but also more broadly the Office’s efforts to promote more disaster-resilient societies worldwide. 2.3 EVALUATION QUESTIONS As detailed in the Statement of Work for the Nepal DRR Summative Evaluation (Annex A), the evaluation addresses the following questions and sub-questions concerning the performance of each of the two OFDA-funded DRR programs: PEER Program A: To what extent has the training provided to project beneficiaries resulted in improved skills for responding to disasters in their specific respective roles? • A1. How many skills did project beneficiaries retain from the trainings they received at key intervals of time after the training (6 months, 1 year, 2 years, 3 years)? • A2. To what extent were project beneficiaries able to use the skills passed on in the training during the 2015 Gorkha Earthquake and aftershocks? • A3. What barriers, if any, did they experience that prevented them from using the skills passed on in the training? • A4. Was there any difference in the level of retention of beneficiaries from different iterations of the trainings - original training provided by OFDA’s IP / trainings one level removed from OFDA’s IP / trainings two levels removed from OFDA’s IP? • A5: To what extent have aspects of the curriculum of PEER trainings been integrated into the work of Nepali disaster response institutions, both governmental and non-governmental? • A6. To what extent have OFDA’s IPs and other Nepal-based institutions changed the original content of the trainings supported by the PEER project? BCIPN Program B: To what extent has the project strengthened the capacity of local governments to effectively administer building code regulations? • B1. Did the project improve knowledge of the building code among the project’s key stakeholders, including engineers, housing construction contractors, masons and municipal government officials/social mobilizers? NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 11 • B2. Did the project improve the quality of adherence to the building code in the project’s targeted municipalities? • B3. What do key stakeholders in the construction process think is most important reason they did or did not adhere to the building code? • B4. What was the effect of participation in the BCIPN program on the livelihoods of skilled construction labor in the project’s targeted municipalities? The scope of these questions and sub-questions focuses principally on obtaining evidence of the specific outcomes of OFDA’s direct contribution to the U.S. Government’s DRR theory of change in Nepal, and not on other questions typical of performance evaluations, such as the extent to which program targets and objectives were met, whether any adjustments were needed to overcome challenges encountered, or evidence of the sustainability of the results achieved. This, in turn, has limited the degree to which the evaluation team is able to inform future OFDA-funded DRR programming. NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 12 3. METHODS AND LIMITATIONS 3.1 EVALUATION METHODS This performance evaluation utilizes mixed analytical methods to address the above questions and sub￾questions. In particular, the evaluation questions are addressed utilizing comparative, descriptive and normative analysis. Since it is not an impact evaluation, however, none of its findings and conclusions may be utilized to attribute outcomes or impact specifically to OFDA’s DRR activities in Nepal. Comparative data analysis was utilized to assess statistically the extent to which the written test scores of DRR trainees tended to be retained over time, and whether there was any observable pattern of skills loss depending on the time that had elapsed since the training was received. Descriptive and normative analysis, based on structured interviews of training participants and other key informants, were utilized to address questions relating to such things as what key stakeholders believed to be the barriers to effective implementation of the skills the trainees had acquired. The analysis was supplemented by extensive review of documents pertinent to the evaluation questions. After ascertaining that in most cases, OFDA’s implementing partners had administered pre- and post-tests of training participants that might substitute as baseline information on skills levels, the evaluation team requested and received information on the numbers trained, as well as the average scores of the training participants, by subject matter category and year, within each program. The evaluation team also requested the names and contact points of the trainees, in order to track down as many as possible and re-test them on the same skills tests they had completed before and immediately after their training.10 The results of the comparison of pre- and post- tests with re-tests were then analyzed utilizing two-tailed t-tests of mean scores, at the 95% confidence level, to ascertain participants’ levels of skills retention. 11 Re-test procedures, structured interview guides and focus group discussion guides were developed during the inception phase, tested and clarified during the first mission with a select number of key stakeholders and training participants from both the PEER and BCIPN training programs and then translated into Nepali, with the collaboration of senior Nepali members of the evaluation team. The senior Nepali team then trained enumerators -- who were fluent in local languages other than Nepali, depending on need in the sites visited -- on how to utilize the data collection tools and accompanied them in field visits to carry out data collection oversight on a frequent basis. The evaluation team also conducted structured interviews and focus group discussions with the training participants who were re-tested in order to address several of the evaluation questions that went beyond skills retention, such as the degree to which the skills they learned helped training participants during the Gorkha earthquake of 2015 and the months of aftershocks that followed. Key informant interviews (KIIs) were also conducted, of key stakeholders (training instructors, central and municipal government officials and other donor representatives), to address specific evaluation questions. In all, for the PEER program, re-tests and structured interviews were conducted with 43 MFR, 96 CSSR and 44 HOPE trainees; and structured interviews were completed with 41 CADRE training participants. Four focus group discussions (FGDs) and 29 key informant interviews (KIIs) were also conducted with PEER training participants and key stakeholders. For the BCIPN program, re-tests and structured interviews were carried out with 244 Masons and 88 Engineers trained by OFDA’s implementing partner; and structured interviews were completed with 86 Municipal Officials and Social Mobilizers trained by OFDA’s implementing partner. Twenty FGDs and 35 10 The Inception Report established sampling targets for each training category according to the requirements of the Statement of Work (Annex A), and these targets were met. 11 The statistical tests examined whether it could be stated that, with a high degree of confidence, the trainees’ mean scores remained above what they were before they received the training, or in other words, whether the skills they learned tended to be retained. NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 13 KIIs were also completed with BCIPN training participants and key stakeholders. In addition, reviews were completed by evaluation team structural engineers of 300 building permits filed in 10 municipalities, for adherence to the NBC, followed by site visits to 20% of the buildings for which permits had been reviewed, and a rapid assessment walkabout in each of the municipalities. The evaluation team also was able to disaggregate the data on training participants who were re-tested and interviewed by sex and social grouping (approximated by the family name of the participant). The evaluation team’s findings and conclusions for the PEER and BCIPN programs are presented by evaluation question and sub-question in Section 4, below; Section 5 presents evaluation team’s recommendations. 3.2 DATA LIMITATIONS A number of data limitations pertained to this evaluation, including: Lack of baseline data. The evaluation team were informed initially that no baseline data had been collected for either program. This was ameliorated successfully, to the extent possible, by utilizing pre- and post-test scores of training participants as baseline data and re-testing to determine their retention of the skills they learned in the training. Unfortunately, while pre- and post-tests were available for MFR, CSSR and HOPE participants in the PEER program, they were apparently not performed for CADRE trainees. And, while pre- and post-tests were also available for Engineers and Masons who were trained under BCIPN, they were not conducted for Municipal Officials and Social Mobilizers who participated in orientation sessions. Lack of performance data. At the initiation of the evaluation, the evaluation team requested implementing partner M&E Plans, Performance Indicator Tracking Tables, baselines and targets, and full reporting of the performance data reported on all indicators by the implementing partners. While a good deal of information was supplied, the evaluation team did not receive reporting on output, outcome or impact-level indicators that could be utilized to assess program performance. Information was provided on numbers trained, and as noted above, based on this information the evaluation team requested, and received, the names and contact points of the trainees under both programs. This information was then utilized to re-test the trainees to assess the outcomes of the two programs. The data collection process was supplemented by in-depth structured interviews of those trainees who were re-tested, and by FGDs and KIIs of key stakeholders. Not possible to observe practical skills. While the technique of re-testing OFDA training participants for whom pre- and post-tests had been performed proved viable for the purpose of ascertaining their retention of theoretical skills, it was not possible to organize sessions within the Nepalese agencies whose staff had been recipients of PEER capacity building to observe their skills in practice. The reasons given were first, that practical training sessions could not be organized within the time frame of the evaluation, and second, that due to the hierarchical nature of decision making, there was little ability to act on such a request, particularly in the Army, Armed Police and Nepal Police. This was ameliorated to the extent possible by asking participants to self-rank their practical skills retention in response to structured interview questions. Potential ‘spillover’ effects. With no baseline study available the evaluation team initially thought that to address questions on retention of skills, in either program, it would be necessary to utilize a quasi￾experimental design according to which the skills of OFDA-trained (treatment) and non-OFDA-trained (comparison) professionals in each category could be compared. In practice, this procedure would have been less than ideal as a means to determine skills retention, in part because many of the non-OFDA-trained professionals received similar types of training from other sources – e.g., masons and engineers who were trained under programs funded by the UN Development Program (UNDP), Japan International Cooperation Agency, (JICA) and Swiss Intercooperation (HELVETAS), among others. The discovery of pre- and post-tests on most of the participants in training funded by OFDA meant that the quasi￾experimental method could be avoided in assessing the retention of skills by trainees. NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 14 Unavailability of data on trainees in courses ‘two levels removed from OFDA’s IP.’ All of the MFR, CSSR and HOPE training carried out under PEER 3 and 4 for which the evaluation team were provided data conformed to the definition of training one level removed from OFDA’s IP (that is, by instructors trained by OFDA and embedded within Nepalese counterpart institutions). MFR, CSSR and HOPE training two levels removed from OFDA’s IP may have been carried out within the counterpart institutions by their own instructors or hired by them; but no records on these trainees were made available to the evaluation team. For this reason, it proved impossible to recover either quantitative or qualitative data that could be utilized to address the question in respect to retention of skills by MFR, CSSR or HOPE trainees ‘two levels removed from OFDA’s IP.’ Training carried out by CADRE under PEER 3 was directly provided by NRCS, OFDA’s IP, but the program has not yet commenced in PEER 4; as far as is known, there are no data available on NRCS trainees in courses ‘two levels removed from OFDA’s IP’ that could be utilized to address the relevant evaluation question. Gender and social inclusion (GESI). The evaluation team found it difficult to assess the performance of the two programs in terms of compliance with GESI objectives based on the performance data reported by the implementing partners, and accordingly it was necessary to rely much more heavily on collection of qualitative information from Key Informant Interviews (KIIs) and Focus Group Discussions (FDGs) than might otherwise have been the case. The evaluation team did collect information on the sex and social grouping of the training participants re-tested and interviewed in the collection of data for the evaluation (see Section 4.3, below), and these are thought to be broadly representative of the participants in the PEER and BCIPN programs. NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 15 4. FINDINGS & CONCLUSIONS 4.1. FINDINGS & CONCLUSIONS FOR PEER EVALUATION QUESTIONS A: To what extent has the training provided to project beneficiaries resulted in improved skills for responding to disasters in their specific respective roles? Overall Findings. The evaluation team’s direct re-tests of hundreds of MFR, CSSR and HOPE professionals in Nepal found that the skills they acquired from OFDA-funded training to help them respond to disasters tended to be retained. For the MFR and CSSR trainees, the retention of skills acquired was found to be significant whether they were trained directly by expatriate OFDA instructors, or by OFDA￾trained Nepali instructors. There was no evidence to support the hypothesis that there was a pattern of progressive skills decline over time. In structured interviews, 88% of CSSR, 94% of HOPE and 93% of CADRE training participants stated that they actively participated in responding to the Gorkha earthquake, and they said that the training increased their preparedness to handle the challenges of that catastrophe.12 While earthquakes are rare events, accidents, fires and other more frequent disasters were also reported to be attended regularly, particularly by the CSSR and HOPE training participants, and this contributed to reinforcing their skills. MFR, CSSR and CADRE trainees also reported were better able to recall the practical skills parts of the OFDA training curricula that they regularly practiced, while the theoretical parts and the skills they identified as less useful such as “child birth” (MFR) or “dead body management” (CADRE) were less likely to be recalled. The evaluation team also found that, during a period of nearly two decades of PEER training activities in Nepal, integration of the training curricula and materials into host-country training programs has occurred, and so for example, OFDA’s key partner organizations now have their own DRR training capacities. Nonetheless, during the same period, although regional standards are maintained, adaptation of the PEER course content to local context and institutional capacities has been relatively limited.13 Management commitment to DRR training within the key emergency response organizations was identified by participants as a constraint to sustainable capacity building, again reducing the potential impact of the PEER program. Training participants and key informants also reported that emergency response organizations and health facilities at lower administrative levels, which are often the closest to disaster sites, cannot sufficiently benefit from DRR training without a sustainable institutional budget for training. And lack of sufficient funding by the GON of search and rescue equipment and medical materials purchase and maintenance remains a key constraint. Overall Conclusions. The emergency response professionals trained in MFR, CSSR and HOPE courses sponsored by OFDA have retained the skills they learned in those courses over time, and this is the case whether they were trained by expatriate OFDA instructors, or by Nepali instructors embedded within the respective response institutions. Both of these findings constitute evidence of impact sustainability. On the whole, training beneficiaries said that they highly valued and put to good use the skills they acquired from PEER training for a wide range of emergencies. Since emergency response professionals and volunteers attend to a wide range of emergencies, including road accidents and fires, it is reasonable to conclude that where those skills acquired through PEER training are exercised, they tend to be better 12 Note that all CSSR trainees must complete the MFR course (although not vice versa), and so MFR trainees were also quite involved in the Gorkha earthquake response. 13 See the detailed findings in the section below addressing sub-question A6, on pp. 37-39 for detailed explanation of the concept of adaptation utilized by the evaluation team. NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 16 retained. This conclusion tended to be confirmed in KIIs and FGDs, in which respondents said that topics such as attending “snake bite” and “high altitude sickness” were less used in their daily work, and hence less retained. It was also the case that the PEER training curricula and materials appear to have influenced the training programs of relevant governmental institutions (e.g., Nepal Police, Armed Police, Army, public hospitals) and NGOs (e.g., Nepal Red Cross Society). Nonetheless, without sustainable funding for the GON to provide DRR training to lower administrative levels, and some minimal sustainable budget for the purchase and maintenance of search and rescue equipment, and medical materials, it will be difficult to put the PEER skills training to good use in future emergencies, reducing the program’s potential impact. The findings and conclusions of the evaluation team with respect to each of the sub-questions of the Statement of Work relating to the PEER program are reviewed in the following sections. A1. How many skills did project beneficiaries retain from the trainings they received at key intervals of time after the training (6 months, 1 year, 2 years, 3 years) Findings The first part of the question relates to skills retention by type of skill. This was addressed by disaggregating the PEER 3 and 4 training participants by the types of training they received and asking the question whether the skills they acquired were retained. The second part of the question deals with retention of skills at key intervals, and to address this, the evaluation team looked at skills retention of the PEER 3 and 4 trainees over time. Performance data were made available to the evaluation team by the implementing partner on all beneficiaries of PEER 3 and 4 training programs who were systematically tested, before and after the training, on the knowledge gained in their respective courses (see Table 4).14 From comparison of the pre￾and post-test results for each group for which such baseline data could be retrieved (MFR, CSSR, HOPE), it is evident that there was a marked increase in their average scores immediately after the trainings (post-test) as compared with immediately before the training (pre-test). Table 4 Comparative Scores of PEER Training Participants MFR CSSR HOPE N Mean N Mean N Mean Re-test 43 57.9% 96 57.4% 44 65.8% Post-test 167 73.9% 166 76.5% 28 71.9% Pre-test 167 37.7% 166 51.7% 55 56.4% Table 4 shows that there was a marked increase in average scores immediately after training, for all three training programs, and that a significant proportion of the skills were retained in the re-tests conducted by the evaluation team. When 43 MFR, 96 CSSR and 44 HOPE training participants were re-tested, using exactly the same tests used by the PEER program immediately before and after they received the training, their average re-test scores had declined compared with average scores registered immediately after the training (post-test). However, despite the significant time that had elapsed, their re-test results remained above the average scores attained prior to commencing the training (pre-test). Moreover, when statistical tests were performed, 14 No pre- and post-test scores were available for CADRE trainees. NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 17 the average scores of training participants in each category were significantly higher in the re-tests as compared with average scores achieved in the pre-tests.15 These statistical results were supplemented by qualitative data gathered through structured interviews, KIIs and FGDs, which were also administered to CADRE trainees. Relatively high proportions of participants in the structured interviews (88% of HOPE, 77% of CSSR and 61% of MFR trainees, respectively) reported that they participated in skills refreshers and drills. Despite these positive structured interview results, respondents in KIIs and FGDs did report lack of regular refreshers and drills and stressed their importance for the retention of skills. In regard to the question of skills retention over time, the data on PEER trainees do not support the hypothesis that there was any significant trend of skills loss in the years following the training they received (see Table 5).16 As noted in the section above on data limitations, while the multiple choice pre, post and re-test results of PEER trainees measured retention of skills acquired in theory, it was not possible to observe the practical performance of skills acquired through PEER training through observation. Table 5 Re-tests of PEER Participants by Year of Training 2014 2015 2016 N Mean N Mean N Mean 46 60.1% 35 52.4% 24 56.8% Table 5 shows that there was no significant trend of skills loss among PEER trainees over time. The evaluation team ameliorated this data limitation by asking structured interview respondents to self-rank on a scale from 1-4 (low to high) the skills learned in the PEER training programs they participated in that were most retained in their work (see Tables 6-8)17 For example, most MFR respondents marked “basic life Table 6 Rank Ordering of Skills Retention by MFR Training Participants Training topics Mean Basic life support and cardio pulmonary resuscitation 3.56 Patient assessment and check vital sign 3.51 Lifting and moving patients 3.44 Fracture and fail test 3.42 Basic life and foreign body airway obstruction 3.28 Bleeding control and shocking chest wounds 3.23 Triage 3.14 Oxygen therapy 3.14 Child birth 2.70 15 Two-tailed t-tests of differences in mean scores of training participants in each group were utilized at the 95% significance level. 16 Two-tailed t-tests of differences in mean scores of training participants were utilized to test the null hypothesis that there were no significant differences in the mean score in any year as compared with the mean score (of 56.8%) for all years, at the 95% significance level. 17 Tables 6-8 are derived from structured interviews of 43 MFR, 96 CSSR, 44 HOPE and 41 CADRE training participants. NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 18 Table 6 is a self-ranking of which skills were retained best, according to the MFR trainees who were sampled. support and CPR” as the most retained skill, while “child birth” was lowest. CADRE trainees marked the two core areas, “basic life support and first aid” as the most retained skill, while “body management” was ranked the lowest. All these skills directly relate to the PEER training curricula. Self-ranking of skills retention was not carried out for HOPE participants as the course curricula were more focused on aspects of hospital preparedness for emergencies and less on the individual skills. Aspects of the curricula that were put into practice by the HOPE participants included “organizing medical response to disasters” (83.7%); “advising medical personnel on what to do in emergencies” (87.8%); and “activating Hospital Incident Command System (HICS)” (69.4%). Table 7 Rank Ordering of Skills Retention by CSSR Training Participants Trainings topics Mean Lifting and stabilizing loads 3.51 Operational safety triage 3.44 Shoring/Support methods 3.44 Rescue strategies and techniques 3.43 Construction materials, structures and damage types 3.42 Search and location techniques 3.42 Structural triage and the INSARAG marking system 3.42 Pre-Hospital Treatment 3.07 Table 7 is a self-ranking of which skills were retained best, according to the CSSR trainees who were sampled. Table 8 Rank Ordering of Skills Retention by CADRE Training Participants Training topics Mean Common hazards, securing family and preparedness 4.00 Basic life support and first aid 3.61 Role as a community first responder 3.54 Fire safety 3.44 Basic research and rescue techniques (load lifting and search methods) 3.27 Incident command system and triage 3.15 Water emergencies, floatation responder 2.95 Dead body management 2.88 Table 8 is a self-ranking of which skills were retained best, according to the CADRE trainees who were sampled. Conclusions The conclusion that the evaluation team draws from these findings is that MFR, CSSR and HOPE trainees sampled by the evaluation team retained a significant proportion of the skills that they had acquired during the PEER training. And there seemed to be no significant pattern of decline of skills over time. The decline in scores from post-test results after the trainings to re-test results during the evaluation was most likely due the fact that in the immediate aftermath of training, retention levels are at their highest point. Beyond this, it appeared that MFR, CSSR and CADRE trainees tended to recall the practical skills parts of NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 19 the OFDA training curricula that they regularly practiced, while the theoretical parts and the skills they identified as less useful such as “child birth” (MFR) or “dead body management” (CADRE) were less likely to be recalled. Since emergency response professionals and volunteers attend to a wide range of emergencies, including road accidents and fires, it is reasonable to conclude that where those skills acquired through PEER training are exercised, they tend to be better retained.18 This conclusion tended to be confirmed in KIIs and FGDs, in which respondents said that topics such as attending “snake bite” and “high altitude sickness” were less used in their daily work, and hence less retained. The self-ranking results from the structured interviews, confirmed by KII and FGD responses, tend to support the conclusion that the direct relevance of the skills they acquired to their jobs and their regular performance in all likelihood increased their skills retention levels. In other words, retention levels are higher for those practical skills directly relevant to their jobs and regularly performed as part of their work. A2. To what extent were project beneficiaries able to use the skills passed on in the training during the 2015 Gorkha Earthquake and aftershocks? Findings While a few of the training sessions were conducted after the Gorkha earthquake, the majority of those who responded in the structured interviews were already trained prior to the earthquake. In structured interviews, 88% of CSSR, 94% of HOPE and 93% of CADRE training participants stated that they actively participated in responding to the Gorkha earthquake. Armed Police Force respondents reported that a total of 45 emergency response teams were dispatched. A total of 35 PEER graduates from the Armed Police Force were deployed and mobilized for coordination and response within and outside Kathmandu Valley.19 According to an FGD statement from the Nepal Army “CSSR-trained Nepal Army personnel rescued 1,300 lives from collapsed structures in earthquake affected districts, which is around 43% of the total people under the collapsed structures.” A USAID/Nepal Disaster Risk Reduction, Reconstruction, and Resilience (DR4) Office Fact Sheet (June 2016) puts the total figure saved by PEER-trained personnel serving in the Nepal Army, the Nepal Police, and the Armed Police Force search- and-rescue teams at more than 1,675 lives. Table 9 Rank Ordering of PEER Skills Usefulness during and after Gorkha Earthquake PEER Course Mean Ranking by Surveyed Course Participants* (1=lowest, 4 = highest) CSSR 3.27 HOPE 3.26 CADRE 3.21 MFR 3.07 * Survey of 96 CSSR, 44 HOPE, 41 CADRE and 43 MFR course participants. Table 9 shows a self-ranking by surveyed course participants so the overall usefulness of the skills they acquired in PEER courses when confronting the disaster of the Gorkha earthquake and its aftermath. 18 In the structured interviews, 71.4 % of HOPE and 69.8 % of CSSR respondents reported that they participated in emergency responses to road accidents; and 85.4 % of CSSR and 70.7 % of CADRE respondents reported that they participated in emergency responses to fires. 19 Cited in Program Performance Report Program for Enhancement of Emergency Response (PEER) Stage 4 (2014-2019), Quarterly Report # 3 for the period April 01, 2015 – June 30, 2015, p. 11 NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 20 In structured interviews, PEER trainees said that the skills they acquired from training were either useful (3) or very useful (4) during and after the Gorkha earthquake. As presented in Table 9, above, the mean scores ranked between useful (3) and very useful (4) for all four training programs on this measure. The skills ranked highest in structured interviews by the PEER trainees in terms of their contribution to life saving actions they performed after the Gorkha earthquake in their respective roles are presented in Table 10, below. Table 10 Types of PEER Skills Ranked by PEER Trainees as Highest in Saving Lives after the Gorkha Earthquake PEER Course Skills Ranked as of Highest Value in Responding to Gorkha Earthquake (percentages of surveyed course respondents ranking the skill as highly useful)* CADRE First Aid (60%) MFR Cardio Pulmonary Resuscitation (42%) CSSR Tools, Equipment and Accessories Handling (42%) Search and Rescue Techniques (30%) * Survey of 41 CADRE, 43 MFR and 96 CSSR course participants. Table 10 shows the skills that the highest proportions of participants in PEER courses found to be most useful when they were responding to the challenges of the Gorkha earthquake and its aftermath. There is also considerable qualitative evidence to confirm these responses, in particular from KIIs and FGDs with beneficiaries in the Nepal Police and the Army, corroborating that the skills they acquired during PEER trainings were used in responding to the Gorkha earthquake and its aftermath. Tables 11-14, below, present self-rankings on a scale from 1-4 (low to high) by the participants in each PEER training course of the overall usefulness of what they had learned to the emergency response during and after Gorkha earthquake. Improved communication and coordination, within their organizations and with other emergency responders, as well as increased awareness within organizations of the key response actions and skills, were ranked high by MFR and HOPE. Openness of communities to their actions and suggestions was ranked relatively low, except for the CADRE participants who were the NRCS volunteers from within the communities. Table 11 Rank Ordering by MFR training participants of course usefulness during and after Gorkha Earthquake Useful themes of training during and after Gorkha Earthquake Mean Good understanding of what to do in such emergencies 3.28 Improved communications and coordination with other emergency responders 3.21 Your organization/institution was more aware of MFR 3.07 You made use of the skills acquired through MFR training 3.07 Your colleagues and the communities were open to your action and suggestions 3.02 Table 11 shows a ranking by trainees of those aspects of the MFR course that were most useful during and after the Gorkha earthquake. NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 21 Table 12 Rank Ordering by CSSR training participants of course usefulness during and after Gorkha Earthquake Useful themes of training during and after Gorkha Earthquake Mean Able to use equipment effectively 3.24 Communities were open to your actions and instructions 3.11 Table 12 shows a ranking by trainees of those aspects of the CSSR course that were most useful during and after the Gorkha earthquake. Table 13 Rank Ordering by HOPE training participants of course usefulness during and after Gorkha Earthquake Useful themes of training during Gorkha Earthquake Mean Improvement in communications and coordination within hospital in responding to emergencies through training 3.20 Staff was open to your actions and suggestions 3.13 Management was more aware of emergency preparedness and mass casualty response 3.11 Medical personnel have a greater understanding of what to do in such emergencies 3.09 Table 13 shows a ranking by trainees of those aspects of the HOPE course that were most useful during and after the Gorkha earthquake. Table 14 Rank Ordering by CADRE training participants of course usefulness during and after Gorkha Earthquake Useful themes of training during and after Gorkha Earthquake Mean Communities were open to assistance. 3.08 Community members have a greater understanding of what to do in emergencies. 2.71 Communities were more aware of what to do to reduce earthquake risks. 2.66 Conclusions From a range of structured interviews, confirmed by FGDs and KIIs, the evaluation team draws the conclusion that, on the whole, training beneficiaries highly valued and put to good use the skills they acquired from PEER training when responding to the Gorkha earthquake. It is very likely that PEER￾trained staff from the key emergency response organizations were instrumental in saving a significant number of lives and providing essential life support in the affected areas. A3. What barriers, if any, did they experience that prevented them from using the skills passed on in the training? Findings Approximately half of the respondents to structured interviews who were trained in MFR and HOPE skills said they did not encounter any barriers to putting the skills they acquired in the training to good use during the earthquake; although only 11 of 96 respondents (11.5%) from among CSSR trainees said the same. NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 22 Nonetheless, a high proportion of the CSSR participants (49%) and significant numbers of CADRE (19%) and HOPE (19%) respondents reported lack of proper equipment and maintenance of essential tools and equipment as major constraints to using the skills they acquired from PEER training during the earthquake, particularly outside Kathmandu valley. Respondents in KIIs and FGDs cited several barriers to effective use of the skills they had acquired in PEER capacity building activities. MFR and HOPE trainees reported limited availability of medical equipment as well as search and rescue equipment, which frequently had to be improvised, as a barrier to putting their skills to use in an emergency. They also cited lack of lack of budget for training and refreshers. Doctors and nurses in Kathmandu and Lalitpur cited inadequate tools and equipment needed for emergency response as a major barrier. CSSR trainees from the Nepal Police and Army cited inadequate communications and emergency response equipment as a critical barrier; poor procurement policy and lack of planning were cited as causes of this deficiency. CSSR and CADRE respondents to the structured interviews marked poor communication and coordination among emergency response stakeholders as among the key obstacles to effective earthquake response. Insufficient human resources, poor road conditions and lack of transport hindered adequate assistance to affected areas. Moreover, mobilization of the existing emergency response teams was thought to be poorly handled. Management strata, particularly in the security agencies, would have benefited from more focused orientation to PEER training outcomes to enable better management during the Gorkha earthquake and its aftermath. In KIIs, Nepal Police and Armed Police representatives stated that MFR and CSSR training helped them in responding to the Gorkha earthquake, although it could not prepare them for the complexities of the situation, especially dealing with pressures from the public on site. CSSR and CADRE respondents added crowd management and poor support from the communities to the complexities of field management. In the structured interviews, the lack of drills and refreshers by their respective organizations were reported in the KIIs and FGDs by CADRE and MFR respondents as a barrier to putting the training and skills to good use. Nepal Police and Nepal Army representatives responsible for CSSR and MFR training suggested pre￾positioning the most appropriate equipment and tools in sufficient numbers and their periodic maintenance so that they are available at the time of an emergency. They also acknowledged, however, that neither is achievable without adequate budget allocations, a point also raised by HOPE respondents in FGDs and KIIs. A 'Lessons Learnt Workshop on National Search and Rescue (SAR) Response during the 2015 Nepal Earthquake', organized by MoHA in 2016, cites barriers to effective SAR that are very similar to the findings from this evaluation.20 They include coordination, communication, training and equipment as among the core areas to improve for effective SAR. The same Workshop Report also identified availability of trained personnel as a challenge, an obstacle also cited in some KIIs due to redeployment of staff. Tables 15 – 18 provide a sampling of direct responses from CSSR, CADRE, HOPE and MFR participants to open-ended questions posed during FGDs and KIIs relating to key barriers and obstacles to using the skills they were trained in during PEER courses in practice. The findings in this section are based on qualitative data gathered in KIIs, FGDs and structured interviews of PEER trainees. 20 Lessons Learnt Workshop, National Search and Rescue (SAR) Response during the 2015 Nepal Earthquake, 10 March 2016. NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 23 Table 15 CSSR trainee responses to open-ended questions in FGD and KIIs Key barriers/obstacles to using thes skills Inadequate and poor quality equipment and tools Absence of latest technology during emergency response as used during the CSSR training Lack of public's faith in emergency response knowledge of security personnel Reasons for those barriers/obstacles Numbers of trained CSSR people are more than the number of tools/equipment set available for each of the security agencies Lack of clear policy provisions pre-positioning of tools/equipment General public are often unsupportive during disaster response Difficulties to control crowd How were those barriers prevented/can be prevented Pre-positioning the most appropriate equipment and tools in sufficient numbers Periodic checks and maintenance of equipment and tools Table 15 lists some of the more prevalent responses by the trainees themselves to open-ended questions on key barriers and obstacles to utilizing the skills they learned in the CSSR course Table 16 CADRE trainee responses to open-ended questions in FGD and KIIs Key barriers/obstacles to using the skills No provision of refresher trainings to CADRE trainees No frequent use of CADRE skills so the skills may be depleted Tools and equipment are inadequate Some CADRE trainees are ignorant of duties/responsibilities Dropout of many trainees, working in middle east countries Difficult to locate and search victims Difficult to reach affected areas due to geographical terrain Reasons for those barriers/obstacle Resource constraint for the management of tools/equipment CADRE trainees are scattered hence it is difficult to mobilize Poor coordination among the local authorities No realization of importance of CADRE training among the local government officials hence reluctant to allocate resources Inadequate personal protective equipment How were those barriers prevented/can be prevented? Developed a database (detail contact list) of CADRE trainees Managed tools and equipment through provision of fiscal budget at local government (to some extent) Run periodic review/reflection among the stakeholders Drills and simulation to sharpen the skills and knowledge Organized talk program on CADRE training and role of youth Table 16 lists some of the more prevalent responses by the trainees themselves to open-ended questions on key barriers and obstacles to utilizing the skills they learned in the CADRE course NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 24 Table 17 HOPE trainee responses to open-ended questions in FGD and KIIs Key barriers/obstacles to using the skills Absence of proper response capability and tools/equipment Recognition of medical responders by government and public in the field Poor mobilization of emergency response team: ambulance, fire brigade, Hospital administration not very supportive for the utilization of skills in the absence of contingency resources for effective response Limited availability of medical tools, equipment for medical response Lack of sufficient and safe space to treat disaster affected people Few steps missed like crowd control in the absence of refresher training How those barriers were prevented /can be prevented? Use of available tools and equipment Allocation of roles and responsibilities Table 17 lists some of the more prevalent responses by the trainees themselves to open-ended questions on key barriers and obstacles to utilizing the skills they learned in the HOPE course Table 18 MFR trainee responses to open-ended questions in FGD and KIIs Key barriers/obstacles to using the skills Identification of MFR; by public who were trained MFR or not. Lack of tools and equipment as the demand was high. Haphazard distribution of work and pressure from affected people. No regular refresher, difficult to remember the step-wise process. Poor coordination among organizations hence ad-hoc MFR response Insufficient trained MFR and demand was high Reasons for those barriers/obstacles In some cases, community do not trust MFR because of no identity Unavailability of books in Nepali affected learning Lack of instructor and participant workbook for refresher training In the absence of fiscal budget, refresher training was not organized Poor coordination among organizations due to ‘visibility’ competition How those barriers were prevented/can be prevented? Informed public by megaphone MFR are specially trained in life saving Mobilized the social leaders, political cadres to raise public awareness Allocated MFR to areas of large devastation and high mortality Table 18 lists some of the more prevalent responses by the trainees themselves to open-ended questions on key barriers and obstacles to utilizing the skills they learned in the MFR course Conclusions The principal conclusion the evaluation team derived from the structured interviews, KIIs and FGDs with PEER training participant and stakeholders is that, without some minimal sustainable budget for training, refreshers, equipment and materials purchase and maintenance, it will be difficult to put several parts of the PEER skills training to good use in future emergencies. Better coordination among the emergency agencies and authorities, as well as within the organizations, would facilitate a more effective use of skills acquired in training. Management strata, particularly in the security agencies, would have benefited from more focused orientation to PEER training outcomes to enable better management during the Gorkha earthquake and its aftermath. NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 25 A4. Was there any difference in the level of retention of beneficiaries from different iterations of the trainings - original training provided by OFDA’s IP / trainings one level removed from OFDA’s IP / trainings two levels removed from OFDA’s IP? Findings To address this question, the evaluation team requested that the implementing partner sort the trainees who were re-tested by the evaluation team into two groups: • Level 0, corresponding to trainees originally trained by OFDA who had gone on to become instructors; and • Level 1, corresponding to trainees who were trained by instructors trained by OFDA. The manner in which this was done was to request that the trainees who were re-tested be sorted into the two groups. Unfortunately, the information needed to sort the HOPE trainees into the two groups was not supplied by the implementing partner, so the analysis is limited to MFR and CSSR trainees. 21 Table 19, below, compares the mean scores of the Level 0 and Level 1 beneficiaries of the MFR and CSSR training programs on the knowledge gained in their respective courses. From a comparison of the pre- and post-test results it is evident that the scores of both Level 0 and level 1 trainees in MFR and CSSR improved significantly from pre-test to post-test. 22 On the other hand, there appeared to be little difference between the mean scores of Level 0 and Level 1 trainees, in either the MFR or the CSSR course. In addition, while Level 1 trainees followed the pattern observed for all MFR and CSSR trainees of reduced mean re-test scores as compared with the post-test scores, their mean re-test scores remained significantly above the mean pre-test scores, providing direct evidence of DRR skills being retained.23 The main difference with regard to Level 0 trainees – who went on to become OFDA instructors – is that their skills retention appears to have been higher (although the small numbers of Level 0 trainees who were re-tested precluded calculation of statistical tests). Table 19 Comparative Scores of Level 0 and Level 1 PEER Training Participants MFR CSSR N Mean N Mean Level 0 trainees – trained directly by OFDA Re-test 6 75.0% 4 61.5% Post-test 19 77.7% 19 73.3% Pre-test 19 36.2% 19 45.0% Level 1 trainees – trained by instructors who were trained by OFDA Re-test 28 53.8% 43 56.6% Post-test 148 73.4% 147 76.9% Pre-test 148 37.9% 147 49.3% Table 19 shows that there was a marked increase in average scores immediately after MFR and CSSR training, and that a significant proportion of the skills were retained in the re-tests conducted by the evaluation team for Level 1 trainees (i.e., those trained by Nepali instructors). 21 Not all MFR and CSSR trainees who were re-tested could be sorted into either Level 0 or Level 1, and those who could not be sorted were left out of these calculations, so that the numbers re-tested differ from those found in Table 4. 22 Two-tailed t-tests of the difference in mean scores were significant at the 95% confidence level. 23 Again, two-tailed t-tests of the difference in mean scores were significant at the 95% confidence level. NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 26 Training two levels removed from OFDA’s IP may have been carried out within agencies by the agencies’ own instructors or hired by them. Based on the quarterly reports to OFDA on the PEER training events, some of these trainers were also trained by PEER as instructors, master instructors and course coordinators. But no records on these trainees were made available to the evaluation team, so that it was not known how to contact them or whether they underwent pre- and post-tests at the time of their training that could be utilized as baseline data to assess their skills retention as a distinctive group. For this reason, it did not prove possible to recover either quantitative or qualitative data that could be utilized to address this question in respect to MFR, CSSR or HOPE trainees ‘two levels removed from OFDA’s IP.’ Training carried out by CADRE under PEER 3 was directly provided by OFDA’s IP, but the program was tabled in PEER 4 and then continued directly by ARC through NRCS, so there were no second or third levels of training provided by CADRE within the PEER program that the evaluation team could measure. It should also be noted that no pre- and post- test results for CADRE trainees were made available to the evaluation team that could be utilized statistically to measure skills retention for any levels. Conclusions Based on the evaluation team’s re-tests of trainees, the DRR skills acquired by the MFR and CSSR trainees were significantly improved, for both Level 0 and Level 1 trainees. And a significant proportion of these skills appear to have been retained over time, regardless of whether the training was performed directly by expatriate OFDA instructors, or by OFDA-trained Nepali instructors. While a greater proportion of the DRR skills of persons trained directly by OFDA appears to have been retained, this was most likely to have been because those trainees went on to become instructors in the same material, rather than because there was any intrinsic difference in the levels of training. A5. To what extent have aspects of the curriculum of PEER trainings been integrated into the work of Nepali disaster response institutions, both governmental and non-governmental? Findings Search and rescue, and medical response, are core areas of training for the emergency response organizations as part of their own military or police training. In KIIs and FGDs with representatives and beneficiaries of PEER training in the Nepal Police, Armed Police and Nepal Army, it was confirmed that these institutions have benefitted from the international standards embedded in the MFR and CSSR course curricula. Their trainers have been trained as instructors, master trainers and training coordinators by the PEER program, and training materials provided by OFDA have largely been integrated into their internal disaster management training. Nevertheless, those sections of the curricula that required sophisticated equipment or complex collapsed building simulations were difficult to practice due to limited resources. In the KIIs it was also stated that these institutions found it difficult to run full-length residential courses, similar to PEER training events, due to their high cost. KIIs and FGDs also confirmed that HOPE training curricula have influenced the disaster risk reduction training of WHO and health-related organizations in Nepal. KII and FGD respondents from the health sector in Kathmandu and Lalitpur said that parts of the HOPE training materials were used by different Government of Nepal hospitals (Bir Hospital, teaching hospitals, Army, Police and Armed Police Hospitals) and non-government hospitals (ALKA Hospital, Om Hospital, B&B Hospital, Norvic Hospital). A representative of the Ministry of Health said, however, that there was a need to institutionalize HOPE training into the National Health Training Center for outreach to clinics and primary health posts. The FGDs and KIIs supported the finding that emergency response organizations and health facilities at lower administrative levels, which are often closest to disaster sites, still do not benefit from these trainings. An Institute of Medicine (IOM) PEER coordinator informed the evaluation team that 14 hospitals (Zonal and Government Hospitals) are trained and funded by the Government of Nepal to allow the IOM of the Tribhuvan University to conduct new HOPE trainings. KIIs with PEER trainers confirmed that there is a NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 27 need for institutionalization of the HOPE and MFR training programs. CADRE has successfully integrated PEER training materials into its standard training due in part to the fact from the start it has been part of NRCS training with strong financial and technical support from Red Cross societies such as the American Red Cross (now with direct OFDA funding) and others. First aid at the community level being a traditional NRCS training, there was a lot of mutual integration when CADRE was under PEER 3. For example, the NRCS staff were trainers for CADRE as well as the NRCS volunteers being the recipients of it. Parts of CADRE material found their way into other NRCS trainings such as Community Based Disaster Risk Management and Community Based First Aid. According to one NRCS representative, OFDA curricula represent about 75% of the curricula utilized in their Community Based Disaster Risk Management training. Canadian Red Cross and British Red Cross also integrated parts of CADRE into their community based disaster risk management programs. The Ministry of Home Affairs accepts CADRE as the key community level training. From the structured interviews, it was reported that 44% of MFR, 44% of CSSR, 22% of HOPE and 58% of CADRE respondents stated that they received similar training from sources other than PEER, an indication that there is training complementary to PEER from which PEER trainees have been benefiting. The Nepal Disaster Risk Reduction Consortium (NRRC) described under “Flagship 2 ad Flagship 4” training similar to PEER undertaken or financed by various other organizations and donors.24 Conclusions From the available qualitative evidence compiled, the evaluation team concludes that aspects of PEER training curricula have contributed both to new knowledge and skills in emergency response organizations in Nepal, as well as to systematizing what was already part of their own training. International standards have been integrated into the first responder training programs of the security entities (Nepal Police, Armed Police Force and Army), and a trained cadre of instructors and master instructors in PEER curricula has been formed within those entities. Similarly, PEER training curricula on hospital preparedness have been integrated into the training programs of the major government and non￾governmental hospitals. In both cases, however, the PEER curricula have not been fully integrated into the Nepali institutions, essentially for budgetary reasons. The security entities cannot afford to run full-length residential courses, and to maintain equipment and supplies suitable for practical training exercises. The health sector institutions cannot afford to institutionalize hospital preparedness to lower administrative levels, although they may be the levels closest to potential disaster sites. In respect to training of community volunteers, while one correspondent estimated that 75% of the curricula offered by PEER overlaps with that provided by the NRCS, this is likely to be because, with support from the ARC and other Red Cross Societies, much of the PEER training curricula, mainly in First Aid, was already being offered in some form by the NRCS. Finally, large proportions of MFR, CSSR, HOPE and CADRE respondents stated that they had received similar training from sources other than PEER, an indication that training complementary to that provided by PEER has been carried out by NGOs and UN institutions. A6. To what extent have OFDA’s IPs and other Nepal-based institutions changed the original content of the trainings supported by the PEER project? Findings The origins of the MFR and CSSR training materials used in PEER 3 and 4 derive from similar courses developed for Latin America as early as in 1992, and introduced to the Asia region under PEER 1 and 2 in 24 A UN initiative that provides a platform for the coordination of DRR activities in Nepal; see http://www.un.org.np/coordinationmechanism/nrrc NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 28 1998, when Nepal was one of the pioneering countries within Asia. Originally MFR and CSSR training materials were developed by the Miami-Dade Fire Rescue Department and implemented under PEER 1 in partnership with ADPC and subsequently adapted in 2004 to the Asian context. ADPC initiated the design and conducted the pilot test of the HOPE course in 2002 under PEER 1. NSET continued to develop the HOPE course under PEER 2 in collaboration with Johns Hopkins University’s Center for International Emergency, and two other U.S. partners, International Resources Group (IRG) and Safety Solutions, Inc. (SSI). IRG was more focused on the overall program management, while SSI focused on MFR and CSSR. With the aim of institutionalizing HOPE in Asia, NSET developed a training course adapting the existing instructor training methods to the training of HOPE instructors. Due to the regional nature of the PEER program, the original training curricula are strictly followed to ensure quality and standardized practice. 'Copyright Notice' in the introductory pages of the Participant Workbooks, grants authorization to copy and distribute documents or portions of it with proper acknowledgement to USAID/OFDA “...provided that no modifications are made to the content…”25 Adaptations of Nepali versions of Workbooks appear to have been minimal, in part due to this restriction. MFR and CSSR Participant Workbooks still keep the logo of the Miami-Dade Fire Rescue Department on the cover as the original authors in the versions adapted for use in Nepal. Many respondents said that PEER training materials are still in English, as they were originally, which is problematic for purposes of enhancing understanding and dissemination to lower administrative levels. Use of sophisticated equipment and materials in the CSSR curricula that are either unavailable or hard to maintain in Nepal was also raised by respondents as less appropriate for the context in that country. An FGD of Nepal Army participants and KIIs of Nepal Police participants indicated that parts of CSSR training need specialized expertise to be understood, as they are too technical. The need to adapt training curricula and materials to local context has been raised by the participants during most training sessions and regional stock taking meetings, reported in a number of implementing partner Quarterly Reports. The need for adaptation was also highlighted in the end-of-PEER 3 report and an internal Program Evaluation. A number of course revision meetings were organized by ADPC and NSET at the regional and national levels to address this request. As a result, the HOPE User Workbook was revised twice in 2014 and 2016. Both the CSSR and the MFR Workbooks were also revised in 2016. However, the adaptations have reportedly been limited. Nepal’s emergency response organizations have adapted to varying degrees different parts of PEER training materials into their institutional training. In a KII, a Nepal Army representative said the training content has been changed and adapted for their own institutional training. Parts of CSSR that could not be simulated, such as vertical SAR or required sophisticated equipment, were not practiced. HOPE curricula in Nepal have eliminated “volcanic eruptions,” which are irrelevant to the country and, following the Gorkha Earthquake, added crowd management and communication topics. 26 Originally HOPE curricula only covered earthquakes; it was modified and adapted in PEER 3 to include multi-hazard training.27 CADRE has been adapted to various degrees for different NRCS training materials. Only two CADRE end user trainings were implemented as part of PEER 3. Others have been conducted by NRCS with American Red Cross, British Red Cross and Canadian Red Cross support. According to FGDs and KIIs with NRCS headquarters, as well branch office staff, 75% of the content remains the same from the original CADRE curricula funded by OFDA under PEER 3. Some new sections have been added and tailored to the Nepal and NRCS context. Parts of CADRE curricula also found their way also into other NRCS training materials. For example, the long existing NRCS training on Community-Based First Aid was reportedly modified using 25From HOPE Participant Workbook introduction; the restriction is repeated in other Participant Workbooks. 26 KII with head of IOM. 27 Ibid. NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 29 some content from CADRE. The Red Cross organizations have been supporting the NRCS financially and technically in this process of adaptation. IOM also acts as the institute to support adaptation and standards for HOPE. For CSSR and MFR there are multiple organizations. The Lessons Learnt Workshop on National Search and Rescue (SAR) Response during the 2015 Nepal Earthquake (cited above) stated that “there were also challenges that arose from different training standards and methodologies” and strongly argued in favor of adopting nationally standardized curricula. Conclusions The evaluation team has concluded that, during a period of nearly two decades of implementation, significant adoption of PEER training materials by Nepali institutions has been achieved. Practically all partner organizations now have their own training capacities. Nonetheless, due to the regional nature of the PEER program, the original training curricula are still strictly followed to ensure quality and standardized practice. In part for this reason, adaptations of Nepal versions of Workbooks appear to have been minimal; for example, the training materials are still in English. The need to adapt training curricula and materials to local context has been raised by the participants during most training sessions and regional stock-taking meetings There is a good understanding of what is useful, and what needs modifying. As emergency response organizations develop their own training capacities, and resources are accessed to take training to sub￾national levels, PEER curricula and materials are likely to be further adapted to the context in Nepal. 4.2. FINDINGS & CONCLUSIONS FOR BCIPN EVALUATION QUESTIONS B: To what extent has the project strengthened the capacity of local governments to effectively administer building code regulations? Overall Findings. The evaluation team’s direct re-tests of hundreds of masons and engineers tended to confirm that the skills they acquired from OFDA-funded training have increased their knowledge of earthquake resistant building techniques adhering to the National Building Code. And, while no baseline information on the knowledge of municipal government officials and social mobilizers prior to BCIPN training could be recovered from the implementing partner, in KIIs and FGDs they confirmed that BCIPN orientation sessions had enhanced their knowledge of disaster preparedness, the cause of earthquakes and challenges in implementing the NBC. The evaluation was not able to confirm, however, that those municipalities targeted by the BCIPN program performed better on adherence to the NBC than those that did not. This may have been the case, in part, because many other donor activities are providing capacity building similar to that funded by OFDA, in non-targeted municipalities, especially following the Gorkha earthquake. Key stakeholders cited a variety of factors that contributed to their ability to adhere to the NBC, including higher public demand for earthquake-resistant buildings following the earthquake and the orientation programs provided by donors. Factors inhibiting adherence to the NBC included the higher cost of construction, continued lack of house owner awareness, and lack of municipal enforcement. Finally, in regard to benefits to their livelihood, 69% of masons interviewed said that their wages had increased following the BCIPN training and 90% said that BCIPN training helped them to gain employment. Overall Conclusions. The principal conclusion the evaluation team derived from re-tests of masons and engineers was that OFDA-funded BCIPN training had increased their knowledge of earthquake-resistant building skills, as embodied in the NBC. Moreover, this knowledge had been retained over time, a key element of impact sustainability. The structured interviews, KIIs and FGDs confirmed that on the whole, training participants, including municipal officials, highly valued and put to good use the skills they acquired from BCIPN concerning adherence to the NBC. NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 30 While the economic constraints of house owners and geographic conditions contribute to non-adherence to the NBC, based on the evidence gathered from KIIs and FGDs, much depends on the extent to which masons – who are closest to the house owners -- are themselves informed and proactive. Nonetheless, while NBC adherence appears to have greatly improved following the Gorkha earthquake, municipal enforcement remains a major constraint. Unfortunately, the capacity of municipalities to enforce adherence to the NBC is as much an issue of resources and political will as it is of technical knowledge. The findings and conclusions of the evaluation team with respect to each of the sub-questions of the Statement of Work relating to the BCIPN program are reviewed in the following sections. B1. Did the project improve knowledge of the building code among the project’s key stakeholders, including engineers, housing construction contractors, masons and municipal government officials/social mobilizers? Findings The evaluation team addressed this question by disaggregating the BCIPN training participants by profession and types of training received, asking the question whether the training had significantly improved their knowledge of the National Building Code (NBC), and whether that knowledge was retained. Masons and engineers who were trained by the implementing partner were systematically tested before (pre￾tests) and after (post-tests) receiving BCIPN training through a multiple-choice test in order to measure whether the training had improved their knowledge on critical aspects of the NBC and earthquake resistant building techniques. These pre- and post- tests results were used as baseline data for the evaluation. The evaluation team used the same tests to measure the extent to which knowledge was retained (re-tests). Table 20 presents the scores of pre-, post- and re-tests of masons and engineers who received the BCIPN training. Table 20 Pre-, Post- and Re-tests Scores of Masons and Engineers benefiting from BCIPN Masons Engineers N Mean N Mean Re-test 245 74.2% 89 63.0% Post-test 985 74.0% 263 77.4% Pre-test 985 47.9% 263 56.9% Table 20 shows that there was a marked increase in average scores immediately after training, for both masons and engineers, and that a significant proportion of the skills were retained in the re-tests conducted by the evaluation team. When 245 masons and 89 engineers who had been training participants were re-tested using exactly the same tests used by the BCIPN program after a significant period of time had elapsed, the following findings were obtained. While the average scores of engineers had declined compared with average scores registered immediately after the training was conducted (post-test), they remained above the average scores attained prior to commencing the training (pre-test). In the case of masons, their re-test scores were essentially equivalent to their post-test scores. Moreover, when statistical tests were performed, the average scores of training participants in both categories were significantly higher in the re-tests as compared with average scores achieved in the pre-tests.28 No pre- and post-tests were conducted at the time municipal officials and social mobilizers were trained, and so it was not possible to perform tests of knowledge for this category. Instead, the evaluation team decided to rely on information gained from structured interviews of trainees in this category to address the evaluation question. 28 Two-tailed t-tests of differences in mean scores of training participants in each group were utilized at the 95% significance level. NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 31 In the focus group discussions and key informant interviews conducted in BCIPN-targeted municipalities, 29 municipal officials and social mobilizers said that their awareness of the importance of the NBC had been improved through the BCIPN orientation programs that were offered. Two focus group discussions30 revealed that the level of NBC knowledge among the municipal officials of Gorkha was poor when BCIPN trainings and orientation programs had not yet been delivered. It was also the case, however, that key stakeholders reported they had received similar training from multiple sources. The evaluation team carried out structured interviews of 68 municipal staff and social mobilizers who had not been trained by the BCIPN program, and of these, seventeen said they had received similar kinds of training from other donors or entities. The evaluation team also carried out structured interviews of 86 municipal staff and social mobilizers who had, in fact, been trained by the BCIPN program, and of these, fourteen said they had also received similar training from other donors or entities. As part of this process, the 86 municipal officials and social organizers who had received BCIPN training were asked whether the training had helped to improve their knowledge of earthquake disaster risk reduction. The results, presented in Table 21, tend to confirm the finding that BCIPN training helped them to better understand earthquake risk management, the usefulness of NBC, the usefulness of the building permit system and the role of municipal authorities in disaster preparedness. Table 21 Knowledge Gained from BCIPN Trainings (municipal officials and social mobilizers) Number of Answers Ratio (%) Training helped to better understand earthquake risk management No 3 3.5% Yes 83 96.5% Total 86 100.0% Training helped to better understand the usefulness of NBC No 7 8.1% Yes 79 91.9% Total 86 100.0% Training helped to better understand the role of municipal authorities in disaster preparedness No 9 10.5% Yes 77 89.5% Total 86 100.0% Training helped to better understand the usefulness of the building permit system No 10 11.6% Yes 76 88.4% Total 86 100.0% Table 21 shows the percentages of municipal officials and social organizers surveyed who stated that the various types of knowledge gained helped them to adhere to the NBC, with those types that were ranked highest presented first. Finally, based on structured interviews of key stakeholders, training similar to that provided by the OFDA￾funded BCIPN program was received by 12% of re-tested engineers, 30% of re-tested masons, 25% of municipal officials and social mobilizers interviewed who had not participated in OFDA-funded training, and 14% of municipal officials and social mobilizers who had. Conclusions Based on re-tests of masons and engineers trained by the BCIPN program, and t-tests of the average 29 This finding was drawn from focus group discussions and key informant interviews in Dharan, Dolakha, Karyabinayak and Putalibazar. 30 This finding was drawn from one focus group discussion with 14 masons and second one with 12 municipal staff. NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 32 improvements achieved by training participants, the evaluation team concludes that knowledge of critical aspects of the NBC and earthquake resistant building techniques was significantly increased; and those skills appear to have been largely retained over time. In addition, based on responses of municipal officials and social organizers to a structured questionnaire, it also appears that their knowledge of the building code was increased by the training they received from the BCIPN program. As with PEER training participants, the evaluation team believes that one very strong reason for the retention of skills acquired by the project’s key stakeholders, particularly masons and engineers, is the fact that they utilize the skills acquired on a routine basis in their line of work. A second reason is the presence of multiple other training programs on similar material, supported by other donors (e.g. UNDP, JICA, HELVETAS, etc.), especially following the Gorkha earthquake, which has reinforced the training supported by OFDA. B2. Did the project improve the quality of adherence to the building code in the project’s targeted municipalities? Findings To address this question, the evaluation team compared aspects of adherence to the NBC of seven BCIPN￾targeted and three non-BCIPN-targeted municipalities. 31 But this comparison did not end up being a feasible evaluation methodology, probably because of the spillover effects of other, similar training programs in non￾targeted municipalities (see Section 3.2, Data Limitations, above). In addition, it was reported in KIIs and FGDs, that the quality of adherence to the NBC improved not only as a result of BCIPN training programs, but also in reaction to the Gorkha earthquake. It was not possible to show statistically that adherence to the NBC improved in BCIPN-targeted municipalities had improved as compared with non-BCIPN-targeted municipalities. As such, the findings in this section only explore the quality of adherence to building codes in BCIPN-targeted municipalities and do not compare them with adherence in other municipalities. One measure of NBC adherence utilized by the evaluation team was to see whether the municipalities visited had any Building Permit Section, any Roster of Masons and any Public Display of Sample Drawings. Table 22 summarizes these critical aspects of municipal system preparedness in BCIPN-targeted municipalities. Table 22 Municipal Systems Facilitating Compliance with NBC in BCIPN-targeted Municipalities Number of BCIPN-targeted Municipalities Total Number of Municipalities visited 7 Building Permit Section Observed 7 Roster of Masons Present 4 Public Display of Sample Drawings 7 Mean Score 85.7% Table 22 shows the number of municipalities visited in which requisite aspects of municipal systems needed to implement the NBC effectively were in place, among the seven BIPCN-targeted municipalities visited by the evaluation team. 31 According to the implementing partner, municipalities targeted by BCIPN included Bhimeshwor, Biratnagar, Dharan, Gorkha, Karyabinayak, Putalibazar and Vyas. Municipalities not targeted by BCIPN included Pokhara, Lalitpur and Chautara. NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 33 The evaluation team also conducted extensive reviews of 30 building drawings in each of the municipalities visited. More precisely, the evaluation team carried out a simple count of the items in each building drawing that were filled in correctly, as opposed to incorrectly (or left unfilled) with regard to NBC provisions. 32 In each municipality, the evaluation team then visited six buildings, selected randomly from among the 30 for which building plans had been reviewed, and compared observable structural characteristics with the drawings accompanying the building permit that had been filed with the municipality. The degree to which they were applied as drafted was noted in each case. The results of this exercise for the BCIPN-targeted municipalities is shown in Table 23, below. Table 23 Adherence of Drawings to NBC and Buildings to Drawings in BCIPN-targeted Municipalities Municipality % Adherence of drawings to the NBC (A) % Adherence of buildings to drawings (B) Average of A and B Mean score Bhimeshwor 71% 96% 84% 80% Biratnagar 64% 94% 79% Dharan 92% 89% 91% Gorkha 76% 76% 76% Karyabinayak 79% 86% 83% Putalibazar 75% 76% 76% Vyas 76% 76% 76% Table 23 shows the percentage of building permit drawings reviewed that adhered to the NBC, and the percentage of buildings visited that adhered to the drawings, by municipality, in the seven BCIPN-targeted municipalities visited by the evaluation team’s structural engineers. The evaluation team’s structural engineers also conducted direct observation of new construction, and reconstruction, in rapid assessment walkabouts in each of the 10 municipalities. These observations, while qualitative in character, found that new or reconstructed buildings were tending to follow the NBC. Findings of the rapid assessment also included the following: • Use of “strong column, weak beam” was generally prevalent • Use of horizontal and vertical bands varied widely among municipalities • Low quality of concrete, especially excess water, was observed in most municipalities. Finally, in the KIIs and FGDs conducted in BCIPN-targeted municipalities, respondents reported improvement in adherence to the NBC improved following BCIPN interventions.33 Among others, the following results of BCIPN trainings were reported: • Change of column size34 (from 9”x9” or 9”x12” to 12”x12”) 32 The NBC provisions differ depending on the type of buildings and relate to general, structural, occupancy load, seismic design, concrete type or load combination standards 33 This finding was drawn from numerous focus group discussions and key informant interviews in Biratnagar, Dharan, Dolakha and Karyabinayak. 34 This finding was drawn from different focus group discussions and key informant interviews in Biratnagar, Dolakha and Karyabinayak. NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 34 • Change of horizontal bands35 in newly constructed buildings Conclusions Comparison of adherence with the NBC between BCIPN-targeted and non-BCIPN-targeted municipalities did not support the hypothesis that there were significant differences in terms of municipal systems. This was seconded by the examination of adherence of drawings to the NBC and buildings to drawings, which also tended to demonstrate that there was no significant difference between the BCIPN-targeted and non￾BCIPN-targeted municipalities. The evaluation team concluded that one reason for the lack of a marked difference being observed between BCIPN-targeted and non-BCIPN-targeted municipalities may be that numerous other donors and entities (e.g. UNDP, JICA, HELVETAS, etc.) have been supporting similar capacity-building initiatives in Nepal. Another may be that there was a general push toward improved municipal adherence to the NBC following the Gorkha earthquake. B3. What do key stakeholders in the construction process think is the most important reason they did or did not adhere to the building code? Findings In the structured interviews, KII and FGD respondents reported that numerous reasons contributed to their adherence to the NBC. A selection of the most prevalent reasons is presented below: • Hundreds of thousands of people were made homeless by the April 2015 Gorkha earthquake with entire villages flattened, across many districts of Nepal. As a result, people's risk perceptions changed. The earthquake made people aware about the weaknesses of non- engineered building construction and thus the importance of following the requirements of the NBC. • It was reported that awareness campaigns by social mobilizers and by other government programs on television or by distributing leaflets about earthquake resistant buildings helped to bridge the knowledge gap between house-owners and masons or engineers. These house owners’ orientation programs were instrumental to ensure NBC adherence as unaware house owners do not necessarily encourage masons to apply earthquake resistant construction techniques. • BCIPN training demonstrated that following the NBC in construction makes buildings earthquake resistant and thus helped to convince training recipients to ensure adherence to the NBC. • Municipalities are responsible to ensure the NBC is implemented. Therefore, not only does the leadership of the municipalities play a big part in enforcement of the NBC, but also the capacity of municipalities to enforce the NBC is also of paramount importance. • Different mechanisms to ensure adherence to the NBC were reported. For instance, some municipalities in Nepal have a penalty system for the buildings that do not adhere to the NBC. Others offer financial incentives (reconstruction grants). • A key informant interview with an executive officer of Pokhara Sub-Municipality revealed that the municipality put in place an electronic building permit system and the software rejects applications of new houses that do not comply with NBC. • A focus group discussion with 13 engineers in Lalitpur also reported that an electronic Building Permit System significantly reduced manual mistakes when verifying adherence to the NBC. • The Ministry of Federal Affairs and Local Development recently put in place a new system to evaluate 35 This finding was drawn from different focus group discussions and key informant interviews in Biratnagar and Dolakha. NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 35 the performance of executive officers, within which some criteria relate to the NBC, Disaster Management and masons/engineer trainings. It was reported that this system of performance evaluation has contributed to better enforcement of the NBC. • The main point of contact for the house owners during building construction are the masons. House owners do not necessarily meet any structural engineers during the process. Thus, the capabilities of masons to convince house owners about the benefits of constructing a house adhering to the NBC are fundamental. BCIPN training courses for masons improve these capabilities since there is a component focusing on the role of masons in convincing about the benefits of constructing earthquake resistant buildings.36 Respondents in structured interviews, KIIs and FGDs cited a number of reasons as contributing to their non-adherence to the NBC. A selection of the most prevalent barriers to adherence to the NBC is presented below: • It was reported in numerous instances that economic constraints of house owners and the higher cost of constructing NBC-compliant buildings (i.e. higher cost of materials, fee of engineers for supervising building construction) were the main obstacles to adhere to the NBC. • The building permit process is not consistent and the modalities differ from one municipality to another. Although the Local Self-governance Act provides a framework for the building permit process, it does not provide all the details for its implementation. Because of this lack of standardization of building permit processes, the extent to which buildings adhere to the NBC differs from one municipality to another. • A few informants revealed that the unavailability of certain materials in some parts of Nepal can contribute to the non-adherence to the NBC. For instance, it was reported that some materials required by the NBC for D-class buildings were not available in Chautara. • Due to the fact that numerous houses in some municipalities were not affected by the earthquake, some house owners did not see the need to comply with earthquake resistant and NBC standards. Thus, the lack of awareness of some house owners continues to contribute to their non-adherence to the NBC. • The municipality not being able to conduct construction supervision in a timely fashion due to inadequate number of technical staff was reported as a barrier to NBC adherence. • It was reported that brain drain of trained engineers in some municipalities like Vyas was an element affecting NBC adherence. • Geographic conditions (especially slope) and limited buildable land area make it difficult in some municipalities of Nepal to construct according to earthquake resistant and NBC standards. • It was reported in a few instances that, due to illiteracy, house owners had difficulties understanding orientation training content and the benefits of applying earthquake resistant construction techniques. • Municipalities’ processes to ensure NBC compliant buildings (e.g. a building permit system) can be lengthy and may be an obstacle to NBC adherence when house owners need to construct or reconstruct their houses quickly after a disaster. • The supervision of building work by qualified and experienced engineers is important to ensure NBC adherence. It was reported that only a small proportion of houses in the Kathmandu valley were constructed under the supervision of an engineer. A few reasons are listed below: o Although house owners have to hire an engineer for the design and drawings, it is not mandatory to hire engineers for the supervision of construction work. o People encounter difficulties finding available and competent engineers. o Sometimes, engineers do not necessarily know how to conduct supervision of construction work properly because earthquake resistant design is not a mandatory course in their 36 NSET’s Baseline Study Report on Risk Perception Survey in the Municipalities. Building Code Implementation Program in Municipalities of Nepal (BCIPN), 2015 contained a Knowledge, Attitudes and Practices Survey that tended to confirm that respondents acknowledge the responsibility of masons to help make the community safe from earthquake risk. NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 36 curriculum. Conclusions The conclusions the evaluation team drew from this evidence were that while there was strong increase of NBC adherence after the Gorkha earthquake, much depends on municipal enforcement, as well as the extent to which masons – who are closest to the house owners -- are themselves informed and proactive. While the economic constraints of house owners and geographic conditions contribute to non-adherence to the NBC, awareness campaigns do make a difference, as does the capacity of municipalities to enforce adherence to the NBC. B4. What was the effect of participation in the BCIPN program on the livelihoods of skilled construction labor in the project’s targeted municipalities? Findings In structured interviews of 244 trained masons, 69% said that their wages increased following the BCIPN training and 90% said that BCIPN training helped them to gain employment. In Karyabinayak municipality, for example, it was reported that, in urban areas, trained masons were in high demand and that there was a wage increase of 20%. This finding was reinforced by KIIs and FGDs in which respondents reported that masons trained in NBC￾compliant construction were more successful in gaining employment, and were able to charge higher wages; while masons untrained in NBC-compliant construction were not as successful in gaining employment, and were receiving lower wages. Although KIIs and FGDs in BCIPN municipalities tended to validate these findings, respondents in Vyas and Karyabinayak municipalities reported that masons’ wages did not increase in rural areas because house owners tended to have lower economic status in those places. In Gorkha municipality, numerous respondents reported that the masons were involved in more re-construction works following the earthquake and that BCIPN training did not particularly help them to get more work. 37 Conclusions The conclusion the evaluation team draws from these findings is that, while the Gorkha earthquake increased demand for masons’ services, training in NBC-compliant construction has been functioning as a prerequisite to their receiving higher incomes as a result of that demand. It is also to be noted, however, that masons reported that the NBC-compliant training was being provided by a variety of different donor entities in addition to BCIPN. 4.3 GESI DISTRIBUTIONS AMONG TRAINING PARTICIPANTS As presented in this section, GESI proportions sampled by the evaluation team are thought to be broadly representative of the PEER and BCIPN training participants in OFDA-funded programs. 4.3.1 GESI proportions sampled for PEER training participants The heavy proportion of males sampled by the evaluation team in several PEER training sub-activities (see Table 24, below) is attributable to institutions and professions dominated by males – including in the Nepal Police, Armed Police and Army, in the case of MFR and CSSR training; and to a lesser extent, for CADRE 37 This assertion is supported by one focus group discussion with masons and another one with municipal staffs as well as four key informant interviews, respectively with an engineer, a chief of DUDBC, a president of mason’s association and an administrative officer of the municipality. NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 37 community volunteers. The tendency is reversed for HOPE, trainees in which were dominated by female nurses. Table 24 Sex of PEER training participants re-tested and surveyed by the evaluation team Sex MFR CSSR HOPE CADRE Male 95.3% 99.0% 34.7% 70.7% Female 4.7% 1.0% 65.3% 29.3% In addition, while not included as a specific question in the structured interviews, the evaluation team was able to make an approximation of the social grouping of each training participant re-tested and surveyed, based on their surname; these results are presented in Table 25. As presented, the social groupings follow categorizations developed by the Government of Nepal. Table 25 Social groupings of PEER training participants re-tested and surveyed by the evaluation team Social groupings MFR CSSR HOPE CADRE Dalit 9.3% 10.4% 2.1% 0% Religious Minorities 0% 13.5% 0% 41.5% Adibasi/Janajati 30.2% 15.6% 50% 0% Disadvantaged Terai Caste Group (Pichadda Barga) 0% 0.0% 0% 9.8% Others (Bahun, Chhetri (both Terai and Hill), Thakuri, Sanyasi, Rana, Shaha, Rajput) 60.5% 60.4% 47.9% 48.7% 4.3.2 GESI proportions sampled for BCIPN training participants In respect to the heavy proportion of males sampled by the evaluation team in BCIPN training sub-activities (see Table 26, below), the heavy domination of males in the masons profession tends to be confirmed by the implementing partner’s reporting that only 1% of masons trained were female (see Section 1.2, above). Male domination of the engineering profession is also reflected in the numbers surveyed by the evaluation team, but while the implementing partner reports that the percentage of female engineers trained by BCIPN was 12%, the evaluation team was unable to verify that percentage based on the trained engineers it could locate and survey. On the other hand, the implementing partner reports that roughly 46% of municipal officials and social mobilizers trained by BCIPN were female, while females represented 71% of the numbers surveyed by the evaluation team. Table 26 Sex of BCIPN training participants re-tested and surveyed by the evaluation team Sex Masons Engineers Municipality officials/Social mobilizers Male 100% 100% 29.1% Female 0% 0% 70.9% In addition, as with PEER trainees, while not included as a specific question in the structured interview, the evaluation team was able to make an approximation of the social grouping of each BCIPN training participant re-tested and surveyed, based on their surname, and these results are presented in Table 27, below. As presented, the social groupings follow categorizations developed by the Government of Nepal. NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 38 Table 27 Social groupings of BCIPN training participants re-tested and surveyed by the evaluation team Social groupings Masons Engineers Municipal officials/ Social mobilizers Dalit 24.6% 6.7% 15.1% Religious Minorities 2.6% 3.4% 0% Adibasi/Janajati 52.9% 40.4% 10.5% Disadvantaged Terai Caste Group (Pichadda Barga) 4.7% 5.6% 2.3% Others (Bahun, Chhetri (both Terai and Hill), Thakuri, Sanyasi, Rana, Shaha, Rajput) 15.2% 43.8% 72.1% NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 39 5. RECOMMENDATIONS Section C.4.4 of the Evaluation Statement of Work states that the “evaluation will be designed to ensure that the findings … can directly inform future OFDA-funded DRR programming. The evaluation team will need to tailor the evaluation approach to the specific context of Nepal, but be able to produce a set of country￾specific recommendations as well as a set of general recommendations for OFDA DRR Programming.” 5.1 OFDA-WIDE RECOMMENDATIONS The immediate challenge presented to the evaluation team in respect to meeting these requirements in the Statement of Work is that the programs that are evaluated are restricted only to Nepal, and so recommendations for OFDA’s DRR activities in other countries must remain quite general in nature. Accordingly, the evaluation team have developed recommendations designed to help OFDA improve its DRR programming process, to enable improvements in future decision-making by OFDA about its DRR programs. In particular, while OFDA does require performance reporting within the context of its DRR awards, it is not mandated to conform to USAID’s ADS 201. Nonetheless, if the purpose is to improve OFDA’s programming decisions for future DRR programs, the Office might consider requiring delivery of Implementation Plans that articulate clear theories of change, performance indicators and targets for achievement of program objectives, and exit strategies for eventual program closure with sustainable impact.38 Second, as a disaster response institution, OFDA’s own theory of change must implicitly be to coordinate with development institutions (including USAID Missions), and host-country authorities, in the countries in which it sponsors DRR activities, to foster work on the longer-term development tasks required to achieve impact sustainability and scalability, and so it is recommended that this aspect be made explicit in its DRR theory of change and that of its implementing partners. . 5.2 NEPAL-SPECIFIC RECOMMENDATIONS The second challenge faced by the evaluation team, as noted in Section 2.3, is that the evaluation questions and sub-questions are focused very closely on observable outcomes related to OFDA’s interventions in disaster risk reduction in Nepal, and not on other aspects typical of performance evaluations, such as whether targets were met, any challenges requiring changes in strategic approach, or evidence of the sustainability and scalability of the results achieved. In other words, while evidence gathered to address the evaluation questions leads naturally to conclusions about the outcomes that have been achieved, they do not lend themselves easily to assessment of the implications of those conclusions for future OFDA programming. To address this challenge, the implications deriving from the principal conclusions of the evaluation are presented in Table 28, below, followed by recommendations for Nepal-specific programming stemming from each implication. 38 This would imply that OFDA’s awards would be in the form of cooperative agreements, and not grants. NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 40 Table 28 Nepal-Specific Conclusions, Implications and Recommendations Principal Conclusions Implications Recommendations PEER Program MFR, CSSR and HOPE skills imparted through PEER training have been retained over time. This was true for MFR and CSSR trainees whether the training was performed directly by OFDA, or by OFDA-trained instructors. Both of these program outcomes represent key elements of impact sustainability. During a period of nearly two decades of DRR capacity building in Nepal, the task of sustainably building the skills of emergency response professionals appears largely to have been achieved, and those skills are being put to good use in critical disaster situations. OFDA should continue to progressively hand off MFR, CSSR and HOPE capacity￾building activities in disaster risk reduction to GON entities in accordance with an established exit strategy. Skills retention levels appear to be higher for those practical skills directly relevant to trainees’ jobs and regularly performed as part of their work. Training beneficiaries highly valued and put to good use the skills they acquired from PEER for a wide range of emergencies, including the Gorkha earthquake in 2015. PEER training curricula and materials appear to have influenced the training programs of relevant governmental institutions. Aspects of PEER training curricula have contributed both to new knowledge and skills in emergency response organizations in Nepal, as well as to systematizing what was already part of their own training. A critical step in the process of sustainably achieving disaster risk reduction in Nepal will be to get Nepalese authorities to take ownership of PEER training activities going forward, adapting course content to local context, language and institutional capacities. This effort should include integration of the DRR training programs and materials into GON institutions and support for their dissemination to decentralized levels, as well as the establishment of mechanisms for periodic updates of the training materials and for ensuring that DRR skills are refreshed. The effort should also address involvement of senior levels of relevant Nepalese Ministries in DRR activities to help facilitate better management and coordination, as well Significant adoption of PEER training materials by Nepali institutions has been achieved; practically all partner organizations now have their own training capacities. As emergency response organizations develop their own training capacities and the NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 41 training is disseminated to sub-national levels, PEER curricula and materials will need to be further adapted to the context in Nepal. as ownership. Adaptation of course content to local context and institutional capacities has been relatively limited. Better coordination among the emergency agencies and authorities, as well as within the organizations, would facilitate a more effective use of skills acquired in training. Without the ability to project DRR training to lower administrative levels, and provide some minimal sustainable budget for the purchase and maintenance of search and rescue equipment, and medical materials, it will be difficult to put the PEER skills training to good use in future emergencies Another critical step in the process of sustainably achieving disaster risk reduction in Nepal will be to work with Nepalese authorities to ensure that they include sufficient funds in their budget for the purchase and maintenance of search and rescue equipment, and medical materials. Finally, it should address how to inventory and ensure adequate distribution to critical locations, as well as maintenance, of medical and search & rescue equipment and materials. BCIPN Program Earthquake-resistant building skills imparted through BCIPN training appear to have been largely retained over time, a key element of impact sustainability. Training in earthquake-resistant building skills has been successful among the engineers and masons trained by the BCIPN program. That task is being scaled up by many development institutions working in Nepal, especially following the Gorkha earthquake of 2015. OFDA should adhere to a role of piloting, standardizing and then progressively devolving training programs for masons, engineers and municipal staff in building code compliance to Nepalese agencies and development institutions. One very strong reason for the retention of skills acquired by the project’s key stakeholders, particularly masons and engineers, is the fact that they utilize the skills acquired on a routine basis in their line of work. A second reason is the presence of multiple other training programs on similar material, supported by other donors (e.g. UNDP, JICA, HELVETAS, etc.), especially following the Gorkha earthquake, which has reinforced the BCIPN training supported by OFDA. NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 42 Economic constraints of house owners, as well as geographic conditions, contribute to non-adherence to the NBC. Economic incentives function both as a barrier to improved adherence to the NBC – because of the increased cost to house owners – as well as a motivation – because those house owners want to hire only masons who are trained in earthquake-resistant building techniques. To follow up on the Knowledge, Attitudes and Practices (KAP) survey, a strategy should be developed on how best to encourage bottom up demand by house owners for more earthquake resistant buildings. Much depends the extent to which masons – who are closest to the house owners - - are themselves informed and proactive. Training in NBC-compliant construction has been functioning as a prerequisite for masons to get contracts with house owners and receive higher incomes for their work. BCIPN training participants, including municipal officials, highly valued and put to good use the skills they acquired from BCIPN concerning adherence to the NBC. OFDA will need to coordinate with development institutions and Nepalese agencies to foster work on the longer-term development activities required sustainably to achieve the inter-agency team’s DRR objectives. OFDA should consider enabling the conduct of a study or studies designed to frame the way forward in terms of the policy and institutional reforms needed to encourage building code compliance in Nepal. NEPAL DRR SUMMATIVE PERFORMANCE EVALUATION 43 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523 Tel: (202) 712-0000 Fax: (202) 216-3524 www.usaid.gov SUMMATIVE PERFORMANCE EVALUATION OF USAID/OFDA’S DISASTER RISK REDUCTION (DRR) PROGRAMMING IN NEPAL Contract No. AID-OAA-I-15-00028; Task Order: AID-OAA-TO-16-00039 Annexes – Evaluation Report July 27, 2017 This publication was produced at the request of the United States Agency for International Development. It was prepared by The Mitchell Group, Inc. (TMG) with the assistance of IMC Worldwide and the Center of Resilience Development (CoRD) in Nepal. NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 1 SUMMATIVE PERFORMANCE EVALUATION OF USAID/OFDA’S DISASTER RISK REDUCTION (DRR) PROGRAMMING IN NEPAL Annexes - Evaluation Report Contract No. AID-OAA-I-15-00028; Task Order: AID-OAA-TO-16-00039 July 27, 2017 DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for In￾ternational Development or the United States Government. NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) II TABLE OF CONTENTS LIST OF TABLES .............................................................................................................................III ANNEX A: STATEMENT OF WORK............................................................................................... 5 ANNEX B: EVALUATION TEAM - EXPERTISE, ROLES AND RESPONSIBILITIES............31 ANNEX C: UPDATED EVALUATION AND IMPLEMENTATION SCHEDULES..................37 ANNEX D: SOURCES OF INFORMATION ..................................................................................40 BIBLIOGRAPHY OF DOCUMENTS REVIEWED ...................................................................................... 40 LIST OF PERSONS/GROUPS INTERVIEWED........................................................................................... 50 STRUCTURED QUESTIONAIRE RESULTS – TABULAR FORMAT..................................................... 54 PEER – QUANTITATIVE DATA (RESULTS OF STRUCTURED INTERVIEWS)............................. 54 BCIPN – QUANTITATIVE DATA (RESULTS OF STRUCTURED INTERVIEWS)........................... 76 DATABASE OF PARTICIPANT RE-TESTS, POST-TESTS AND PRE-TESTS .................................... 92 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) III LIST OF TABLES Table 1 Identification and Background (CADRE Trainees) ...................................................................................... 55 Table 2 Training Session (CADRE Trainees)............................................................................................................ 57 Table 3 Post-training (CADRE Trainees).................................................................................................................. 59 Table 4 General Appreciation of Training (CADRE Trainees).................................................................................. 60 Table 5 Trainees’ Own Appreciation of Skills Retention (CADRE Trainees) ............................................................ 60 Table 6 Identification and Background (CSSR Trainees)........................................................................................... 61 Table 7 Training Session (CSSR Trainees)................................................................................................................. 62 Table 8 Post-training (CSSR Trainees) ...................................................................................................................... 64 Table 9 General Appreciation of Training (CSSR Trainees) ...................................................................................... 65 Table 10 Trainees’ Own Appreciation of Skills Retention (CSSR Trainees)............................................................... 65 Table 11 Identification and Background (HOPE Trainees)....................................................................................... 66 Table 12 Training Session (HOPE Trainees)............................................................................................................. 67 Table 13 Post-training (HOPE Trainees) .................................................................................................................. 70 Table 14 General Appreciation of Training (HOPE Trainees) .................................................................................. 70 Table 15 Identification and Background (HOPE Trainees)....................................................................................... 71 Table 16 Training Session (MFR Trainees)................................................................................................................ 72 Table 17 Post-training (MFR Trainees) ..................................................................................................................... 73 Table 18 General Appreciation of Training (MFR Trainees) ..................................................................................... 73 Table 19 Repetition/Refreshment of Practical Skills Training (MFR Trainees).......................................................... 74 Table 20 Trainees’ Own Appreciation of Skills Retention (MFR Trainees)................................................................ 75 Table 21 Identification and Background (Engineers & Sub-engineers)...................................................................... 77 Table 22 Knowledge Gained from BCIPN Trainings (Engineers & Sub-engineers) .................................................. 79 Table 3 Skills Improvement from BCIPN Trainings (Engineers & Sub-engineers).................................................... 79 Table 24 Use of Skills in Earthquake Resistant Building Techniques after BCIPN Trainings (Engineers & Sub￾engineers) ................................................................................................................................................................. 80 Table 25 Perception of Population on Building Earthquake Resistant Structures (Engineers & Sub-engineers)......... 80 Table 26 Major Barriers Ranking for NBC Enforcement (Engineers & Sub-engineers)............................................. 81 Table 27 Professional Benefits resulting from BCIPN training (Engineers & Sub-engineers) .................................... 81 Table 28 Professional Development following BCIPN training (Engineers & Sub-engineers)................................... 82 Table 29 Appreciation of BCIPN Training (Engineers & Sub-engineers).................................................................. 82 Table 30 Identification and Background (Masons) .................................................................................................... 83 Table 31 Knowledge Gained from BCIPN Trainings (Masons) ................................................................................ 84 Table 32 Post-training (Masons) ............................................................................................................................... 85 Table 33 Professional Benefits (Masons)................................................................................................................... 86 Table 34 Appreciation of BCIPN Training (Masons) ................................................................................................ 87 Table 35 Identification and Background (Municipality staff) ..................................................................................... 88 Table 36 Knowledge Gained through BCIPN (Municipality staff) ............................................................................ 89 Table 37 Post training (Municipality staff)................................................................................................................. 90 Table 38 Professional Benefits (Municipality staff).................................................................................................... 91 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) IV Table 39 Comparative Scores of PEER Training Participants.......................................Error! Bookmark not defined. Table 40 Re-tests of PEER Participants by Year of Training.........................................Error! Bookmark not defined. Table 41 Comparative Scores of BCIPN Training Participants .....................................Error! Bookmark not defined. NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 5 ANNEX A: STATEMENT OF WORK NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 6 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 7 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 8 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 9 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 10 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 11 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 12 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 13 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 14 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 15 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 16 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 17 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 18 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 19 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 20 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 21 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 22 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 23 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 24 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 25 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 26 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 27 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 28 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 29 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 30 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 31 ANNEX B: EXPERTISE, ROLES AND RESPONSIBILITIES OF THE EVALUATION TEAM The Mitchell Group, Inc. (TMG) provides OFDA with the highest quality professional and organizational experience. Complemented by IMC’s technical leadership in DRR and CoRD’s strength in local resilience, the evaluation team is well-equipped to offer comprehensive and rigorous evaluation services. In sum￾mary, the evaluation team is well-prepared in all areas for this scope of services including: • DRR analysis/evaluation • DRR in Nepal and the Region • Building Code implementation in Nepal • Operational aspects of Nepal • Complex USAID Performance Evaluation • Learning/Capacity Building Expertise • Solid Team Management The TMG Team has over thirty years of monitoring and evaluation experience to offer USAID/Nepal with rigorous technical strategies and evidence-based recommendations for effective disaster response. In total, TMG has completed evaluations in over 70 countries across a variety of technical fields performing performance, mid-term, final, and impact evaluations. TMG completed complex performance evaluations in the highly volatile regions of Afghanistan, Chad, Burkina Faso, Iraq, Niger, Nigeria and the eastern Democratic Republic of the Congo. IMC Worldwide has been practicing M&E across all of the technical areas in which it operates for the last decade: from DRR, social development, climate change and environment, innovation and livelihoods to transport infrastructure engineering and construction and management. This M&E includes numerous theory based evaluations particularly, at country and global levels and across a wide range of thematic ar￾eas. IMC deliver have delivered performance and operational evaluations, such as the performance review of DFID Nepal’s Program to Increase Resilience to Natural Disasters in Nepal in 2015. IMC have a dedi￾cated in-house team experienced in M&E, impact evaluation and learning who also provide technically backstopping, and this is complemented by a wide network of specialists with niche expertise. The variety of complex, multi-regional and multi-million dollar evaluations range from desk reviews and performance evaluations, to rigorous quasi-experimental and experimental mixed-methods impact evaluations. CoRD was established within Nepal, with the goal of building capacity and resilience across Nepal by means of research, training and design. Included in these endeavors are projects such as Support to Im￾plement National Building Code of Nepal in Three municipalities and six urbanizing VDC of Kathmandu Valley, funded by UNDP/DFID. From this CoRD are well acquainted with the programs, regions and stakeholders in which the BCIPN is focusing on. CoRD fielded Data Collection Team members (an all Nepali team), with language abilities allowing them to operate effectively in multiple regions across Nepal. Thanks to their work on other BCIPN and DRR and data collection projects, they have an extensive knowledge of the stakeholders, goals and objectives and challenges of this scope of services. NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 32 Roles and Responsibilities of Evaluation Team Members The figure below outlines the evaluation team’s reporting structure. The following outlines the roles and responsibilities of each evaluation team member. Core Team Under the strong leadership of the Evaluation Team Leader, is the Core Team which boasts both power￾ful technical and sector expertise and contains National staff to solidify the Team’s understanding of the environment, but also the nuances of the effectiveness of DRR programming and the stakeholders that are most affected. Evan Scott Thomas (Evaluation Team Leader) - Overall responsibility for the delivery of summative evaluation - Reporting back to the Technical Director, Jenkins Cooper. - Serving as TMG's point of contact with USAID/OFDA's Contracting Officer and Contracting Of￾ficer Task Order Representative. - Overseeing the general direction and implementation of the program, checking for any impending challenges and risks to the program. - Maintaining relations and communications with partners and stakeholders. - Leading and managing all discrete aspects of the summative evaluation (Including: the Inception re￾port, the Evaluation plan, the Data Collection Tools, the Monthly Progress Reports, and the Evalua￾tion report) - Key role in dissemination of research results, working with the Communication Specialist, and help￾ing to organize activities such as workshops as determined during the summative evaluation. - Providing technical guidance on discussions with key stakeholders, including the implementing part￾ners, key government officials and other experts. - Technical lead to refine the evaluation design, work plan, team composition and personnel inputs if required. - Technical lead on developing data collection instruments, in collaboration with CoRD data collection team. - Providing technical guidance to Data Collection Manager and Data Analysis during analysis of field based data collection and quality assurance approach. - Providing technical guidance in conducting surveys, focus group discussions and key informant inter￾views. - Leading on preparing all reports. - Leading on all regular reporting and management of resourcing of technical team. NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 33 - Leading on preparation, delivery and dissemination of all training and workshops. Hari Darshan Shrestha (2 roles) Deputy Team Leader - Deputizing for the Team Leader when needs arise. - Leading the Preparation of data collection phase (logistics, contact with external stakeholders/in￾formants etc.) - Assisting the Team Leader in establishing and maintaining a strong relationship with key stakeholders in Nepal and liaison with IMC and TMG when necessary, especially with visitors to Nepal. - Overall responsible for the Nepali team in country. Structural Engineering/ Building Codes Specialist - Overseeing the evaluation on the BCIPN component. - Providing technical guidance to Structural Engineers for the collection and analysis of field data. - Bringing international best practices in his technical areas, and in-depth knowledge of the local con￾text. The Performance Evaluation Specialist (Raymond Gervais) - Providing technical backstopping and provide technical inputs for all discrete aspects of the summa￾tive evaluation when instructed by the Team Leader. - Deputizing for the Deputy/Team Leader when needs arise. - Providing main support to TL for Evaluation Plan preparation - Designing the data collection tools and templates (Qualitative & Quantitative) - Supporting the design of pilots and ensure pilot testing. - Providing training of enumerators and supervisors (as per instructions from the Team Leader). - Conducting quality checks of quantitative and qualitative collection of field collected data (quality as￾surance of data collection and entry). - Contributing to Desk review and Inception report. - Analyzing field collected data. - Contributing to Final Evaluation Report - Bringing USAID M&E experience. Catherine Lowery (Training, Learning, Capacity Building Specialist) - Bringing extensive expertise in MEL and capacity building. - Supporting the Team Leader so that the evaluation uses an utilization focused approach (i.e. iterative learning process intended to formulate and disseminate lessons learned; accept feedback and refine methods of analysis and recommendations; and broaden the evaluation’s impact by exploring how lessons and recommendations can be shared). - Carrying out document reviews and provide summaries to the core team. - Supporting the Team in the development of the inception report. - Technical support to refine the evaluation framework, including evaluations questions and methodol￾ogies. - Providing technical support to the Team in the preparation of data collection instruments, in collabo￾ration with the Team Leader. - Providing support during the analysis of the data collected in the field. - Supporting the Team in the preparation of the final report and in summarizing the key findings. NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 34 Yasemin Aysan (DRR Specialist) - Bringing extensive expertise in in national and organizational capacity development in disaster risk management, vulnerability and capacity assessment, post-disaster recovery and reconstruction plan￾ning, disasters and development, urban disasters and shelter. - Carrying out document reviews and provide summaries to the core team. - Supporting the Team in the development of the inception report. - Providing technical support to refine the evaluation framework, including evaluations questions and methodologies. - Provide technical assistance on developing data collection instruments, in collaboration with the Team Leader. - Providing support during the analysis of the data collected in the field. - Supporting the Team in the preparation of the final report and in summarizing the key findings. Rumana Kabir (Community Based DRR, Gender and Social Inclusion Specialist) - Bringing extensive expertise in community based DRR, gender and social inclusion. - Carrying out document reviews and providing summaries to the core team. - Supporting the Team in the development of the inception report. - Providing technical support to the Team in the preparation of data collection instruments. - Providing support during the analysis of the data collected in the field. - Supporting the Team in preparation of the final report and in summarizing the key findings regarding DRR, gender and social inclusion. Management Support Team The Management Support Team brings experience in managing complex evaluations and projects (includ￾ing USAID assignments), combining experience in mobilizing and managing geographically disperse and moving teams of experts to ensure the evaluation is delivered on time and managing project risks to en￾sure value for money, and the safety of all the team. Jenkins Cooper (Technical Director) - Ensuring the highest standards of technical quality and timeliness in respect to all of the evaluation team's products, and to the performance of each of the evaluation team's members. Damien Faget (Evaluation Manager) - Reporting back to the Team Leader and the Technical Director. - Determining objectives, with the Team Leader and other team members, upon which completion of the evaluation can be measured. - Developing and update Project Operation Manual to facilitate the effective management and admin￾istration of this summative evaluation in accordance with the IMC Quality Management Systems. - Coordinating and managing the deployment of teams, including pre-deployment safety and security measures in collaboration with the IMC administration team; - Providing critical research support when needs arise. - Monitoring and manage quality assurance tasks, ensuring each evaluation output follows quality assur￾ance process. - Setting up and maintaining a consistent project filling system - Issuing all documentation required to team members - Tracking actual project fees, costs and expenses against the budget and report discrepancies. NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 35 Sajid Chowdhury/Ben Walker (Communications Specialist) - Leading the development of the branding and marking implementation plan. - Supporting the Team in the preparation of the reports and presentations in terms of formatting and graphic designs. Technical Support Pool Technical Support Staff provide technical know-how to the core team on an ad hoc basis and quality as￾surance in terms of M&E, social inclusion, gender, structural Engineering, disaster resilient infra￾structure and DRR. Catherine Renshaw (Technical M&E QA Specialist) - Providing technical backstopping for the team leader around key evaluation issues and challenges. - Reporting back to the Team Leader and Evaluation Manager with her findings after performing qual￾ity assurance of inception and final evaluation reports. - Reviewing key deliverables ensuring robustness of findings and highlighting any data limitations. - Applying international evaluation standards to the review of the evaluation report. Daniela Villar (Structural Engineer) - Bringing expertise in structural engineering and disaster resilient infrastructure. - Providing technical backstopping for the team in terms of structural engineering and disaster resilient infrastructure. - Reporting back to the Team Leader and Evaluation Manager with her findings after performing qual￾ity assurance of inception and final evaluation reports. - Applying international standards to the review of the inception and final evaluation reports. Data Collection Team The Data Collection Team is an all Nepali team, with language abilities which allow them to operate ef￾fectively in multiple regions across Nepal. Thanks to their work on other BCIPN and DRR and data col￾lection projects, they have an extensive knowledge of the stakeholders, goals and objectives and chal￾lenges of this scope of services. Ajay Chandra Lal (Data Collection Manager) - Primarily responsible for the overall management of the fieldwork and data collection (Qualitative & Quantitative). - Supporting the Team with regard to the design of data collection tools (Qualitative & Quantitative). - Supporting the design of pilots and pilot testing. - Providing training of enumerators and supervisors for the data collection. - Supporting the logistics associated with data collection - Setting up the co-ordination & oversight of data cleaning. - Supporting in appropriate communications and dissemination of information. - Reporting to the Deputy Team Leader. Negendra Amatya (Data Analyst) - Being involved in data analysis (Qualitative & Quantitative) and providing data analysis reports. - Supporting the Team with regard to the design of data collection tools (Qualitative & Quantitative). NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 36 - Being involved in quality/survey checks. - Leading data cleaning and collation. - Supporting in appropriate communications and dissemination of information. - Reporting to the Performance Evaluation Specialist and Deputy Team Leader. In addition to these senior members of the Data Collection Team, CoRD fielded an M&E/learning Spe￾cialist and Enumerator teams composed of, supervisors, enumerators (including structural engineers), and translators/interpreters who were involved in pilot testing, conducted data collection through direct ob￾servation, key informant interviews, focus group discussions and structured interviews. NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 37 ANNEX C: UPDATED EVALUATION AND IMPLEMENTATION SCHEDULES Evaluation Schedule An evaluation schedule is presented in the table below. Task Order contract deliverables and due dates for their completion are highlighted in boldface. Major tasks of the evaluation team and the timeline for their implementa￾tion are also shown. Evaluation Milestones and Deliverables* Schedule / Due Dates Evaluation Milestones and Deliverables* Approved Schedule / Due Dates Revised Schedule / Due Dates Post-Award Conference Call 18 October 2017 18 October 2017 Workplan 25 October 2016 25 October 2016 Kick off meeting 28 Oct 2016 28 Oct 2016 Branding and marking implementation plan 10 Nov 2016 10 Nov 2016 Inception report Draft 5 Dec 2016, Final 31 Dec 2016 Draft 5 Dec 2016, Final 31 Dec 2016 Evaluation plan Draft 12 Dec 2016, Final 7 Jan 2017 Draft 12 Dec 2016, Final 7 Jan 2017 Data collection tools Draft 12 Dec 2016, Final 7 Jan 2017 Draft 12 Dec 2016, Final 7 Jan 2017 Monthly progress reports/calls Last week of each month Last week of each month In-briefing in Kathmandu 13 Jan 2017 13 Jan 2017 Team planning/training 2-19 Jan 2017 2-19 Jan 2017 Instrument pre-testing 9-13 Jan 2017 9-13 Jan 2017 Revising / finalizing tools 16-20 Jan 2017 16-20 Jan 2017 Training of enumerators 23-27 Jan 2017 23-27 Jan 2017 Field work in selected sites 30 Jan - 24 Mar 2017 30 Jan - 24 Mar 2017 Data analysis 15 Mar - 14 Apr 2017 15 Mar - 14 Apr 2017 Exit briefing - presentation 17 Apr 2017 17 Apr 2017 Evaluation Report Draft 8 May 2017, Final 29 May 2017 Draft 23 May 2017, Final 23 June 2017 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 38 Electronic copies of raw data 29 May 2017 23 June 2017 Presentations in Washington and Kathmandu 5-9 June 2017 (dates TBD) 28 June 2017 5 July 2017 * Task Order deliverables are highlighted in boldface ** Revised deliverables dates approved by TOCOR NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 39 Evaluation Schedule The table below presents an updated implementation schedule detailing tasks and subtasks to be performed. 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 10-Oct 17-Oct 24-Oct 31-Oct 07-Nov 14-Nov 21-Nov 28-Nov 05-Dec 12-Dec 19-Dec 26-Dec 02-Jan 09-Jan 16-Jan 23-Jan 30-Jan 06-Feb 13-Feb 20-Feb 27-Feb 06-Mar 13-Mar 20-Mar 27-Mar 03-Apr 10-Apr 17-Apr 24-Apr 01-May 08-May 15-May 22-May 29-May 05-Jun 12-Jun 19-Jun 26-Jun 03-Jul 10-Jul 17-Jul 24-Jul 0. Start of contract (mobilization) 1.1. Post-award conference call 1.2. Inception preparation 1.3. Workplan 1.4. Kick off meeting with client 1.5. Preliminary meetings with government stakeholders, implementing agencies 1.6. Working with core team and briefing 1.7. Document review and assessment of data 1.8. Branding and Marking Implementation Plan (1st draft) 1.10. Refine scope of evalaution questions and methodology 1.11. Production of draft inception plan 1.12. Production of draft evaluation report 1.13. Production of draft data collection tools, database design (plus translation into Nepali) 1.14. QA, finalisation and submission of 1st draft inception report 1.15. QA, finalisation and submission of 1st draft evaluation plan & data collection tools 1.15. Preparation of data collection phase (logistcics, contact with external stakeholders/informants etc. ) 1.17. Response to clients' comments and submission of final evaluation plan, inception report, data collection tools and branding & marking implementation plan 2.1. Progress Call with OFDA 2.2. In-briefing in Kathmandu/Team planning 2.3. Training of all Supervisors and emurerators for Lalitpur 2.4. Piloting and Pretesting of data collection instruments 2.5. Revision and Finalization of data collection tools 2.6. Translation of revisions into Nepali 2.7. Training of all other enumeraters at local level 2.8. Data Collection/Field Work (KIIs, FGDs, Structured interviews, re￾tests, checks of adherance to building codes) 2.9. Surveys backchecked, quality checks 2.10. Translation of data into English 2.11. Data collation, cleaning 3.1. Data analysis 3.2. Data analysis workshop (internal) 3.3. Exit meeting with OFDA in Nepal (verification of findings) 3.4. Preparation of evaluation report (1st draft) 3.5. QA, finalisation and submission of evaluation report (1st draft) 3.6. Response to clients' comments and submission of final evaluation report 3.7. Electronic copies of raw data 3.8. Presentation of evaluation report to wider audience (Kathmandu) 3.9. Presentation of evaluation report to wider audience (Washington) June 17. July 17. Phase 3 - Data analysis & production of evaluation report period Mar-17 Apr-17 May-17 Phase 2 - Implementation period Phase 1 - Inception period Oct-16 Nov-16 Dec-16 Jan-17 Feb-17 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 40 ANNEX D: SOURCES OF INFORMATION BIBLIOGRAPHY OF DOCUMENTS REVIEWED PEER DOCUMENTS 1. Asian Disaster Preparedness Center (ADPC). 2010. Programme for Enhancement of Emer￾gency Response. Hospital Preparedness for Emergencies (HOPE). Program Review Meeting. Meeting Report. Bangkok: ADPD 2. Asian Disaster Preparedness Center (ADPC). 2010. PEER 3 Program. Quarterly Program Progress Report. October-December 2010. Bangkok: ADPC 3. Asian Disaster Preparedness Center (ADPC). 2010. PEER 3 Program. Quarterly Program Progress Report. April-June 2010. Bangkok: ADPC 4. Asian Disaster Preparedness Center (ADPC). 2010. PEER 3 Program. Quarterly Program Progress Report. July-Sept 2010. Bangkok: ADPC 5. Asian Disaster Preparedness Center (ADPC). 2012. Program for Enhancement of Emergency Response (PEER). Mid-Term Review (MTR). Bangkok: ADPC 6. Asian Disaster Preparedness Center (ADPC). 2012. PEER 3 Program. Quarterly Program Progress Report. October-December 2011. Bangkok: ADPC 7. Asian Disaster Preparedness Center (ADPC). 2012. PEER 3 Program. Quarterly Program Progress Report. January-March 2012. Bangkok: ADPC 8. Asian Disaster Preparedness Center (ADPC). 2013. PEER 3 Program. Quarterly Program Progress Report. October-December 2012. Bangkok: ADPC 9. Asian Disaster Preparedness Center (ADPC). 2014. PEER Program for Enhancement of Emer￾gency Response. Final Program Report. Bangkok: ADPC 10. Asian Disaster Preparedness Center (ADPC) and National Society for Earthquake Technol￾ogy-Nepal (NSET). 2005. Earthquake Resistant Construction of Buildings. Curriculum for Ma￾son Training. Guidelines for Training Instructors. Bangkok: ADPC, NSET, USAID 11. Asian Disaster Preparedness Center (ADPC). 2015. Program for Enhancement of Emergency Response (PEER) Stage 4 (2014-2019). Program Performance Report. Quarterly Report # 1 for the period October 01, 2014 - December 31, 2014. Bangkok: ADPC 12. Asian Disaster Preparedness Center (ADPC). 2015. Program for Enhancement of Emergency Response (PEER) Stage 4 (2014-2019). Program Performance Report. Quarterly Report # 2 for the period January 01, 2015 – March 31, 2015. Bangkok: ADPC 13. Asian Disaster Preparedness Center (ADPC). 2015. Program for Enhancement of Emergency Response (PEER) Stage 4 (2014-2019). Program Performance Report. Quarterly Report # 3 for the period April 01, 2015 – June 30, 2015. Bangkok: ADPC 14. Asian Disaster Preparedness Center (ADPC). 2015. Program for Enhancement of Emergency Response (PEER) Stage 4 (2014-2019). Program Performance Report. Quarterly Report # 4 for the period July 01, 2015 – September 30, 2015. Bangkok: ADPC 15. Asian Disaster Preparedness Center (ADPC). 2016. Program for Enhancement of Emergency Response (PEER) Stage 4 (2014-2019). Program Performance Report. Quarterly Report # 5 for the period October 01, 2015 – December 31, 2015. Bangkok: ADPC NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 41 16. Asian Disaster Preparedness Center (ADPC). 2016. Program for Enhancement of Emergency Response (PEER) Stage 4 (2014-2019). Program Performance Report. Quarterly Report # 6 for the period January 01, 2016 – March 31, 2016. Bangkok: ADPC 17. Asian Disaster Preparedness Center (ADPC). 2016. Program for Enhancement of Emergency Response (PEER) Stage 4 (2014-2019). Program Performance Report. Quarterly Report # 7 for the period April 01, 2016 – June 30, 2016. Bangkok: ADPC 18. Asian Disaster Preparedness Center (ADPC). 2016. PEER Stage 4 (2014-2019) Beneficiary￾Countries, Partner Organizations and Focal Points (as of 18 November 2016). Katmandu: NSET 19. Asian Disaster Preparedness Center (ADPC). 2010. Programme for Enhancement of Emer￾gency Response. Training Curriculum Framework. Community Action for Disaster Response (CADRE). Emergency Response Capacity Building for Non-Professional First Responders in the Community. Bangkok: ADPD 20. Kharel, S. C. 2016. PEER Course Coordinator's Report-CSSR. Kathmandu: NSET 21. Miami-Dade Fire Rescue Department. 2006. Collapsed Structure Search and Rescue Course. Instructor Workshop. Lesson Plan. Washington: USAID/OFDA 22. Miami-Dade Fire Rescue Department. 2011. Medical First Responder Course. Lesson Plan Complete. Washington: USAID/OFDA 23. Miami-Dade Fire Rescue Department. 2011. Medical First Responder Instructor Workshop. Pre-Work. Washington: USAID/OFDA 24. Miami-Dade Fire Rescue Department. 2013. Medical First Responder Course. Instructor's Guide Complete. Nepal Adapted Version. Washington: USAID/OFDA 25. Miami-Dade Fire Rescue Department. 2013. Medical First Responder Course. Participant Workbook. Nepal Adapted Version. Washington: USAID/OFDA 26. Miami-Dade Fire Rescue Department. 2013. Medical First Responder Course. Coordinator's Guide. Nepal Adapted Version. Washington: USAID/OFDA 27. Miami-Dade Fire Rescue Department. 2013. Collapsed Structure Search and Rescue Course. Exercise Guide. Nepal Adapted Version. Washington: USAID/OFDA 28. Miami-Dade Fire Rescue Department. 2013. Collapsed Structure Search and Rescue Course. Instructor's Guide. Nepal Adapted Version. Washington: USAID/OFDA 29. Miami-Dade Fire Rescue Department. 2013. Collapsed Structure Search and Rescue Course. Participant Workbook. Nepal Adapted Version. Washington: USAID/OFDA 30. Miami-Dade Fire Rescue Department. 2013. Collapsed Structure Search and Rescue Course. Coordinator's Guide. Nepal Adapted Version. Washington: USAID/OFDA 31. Miami-Dade Fire Rescue Department. 2016. Collapsed Structure Search and Rescue Course. Instructor Workshop. Facilitator's Guide. Washington: USAID/OFDA 32. National Society for Earthquake Technology-Nepal (NSET). 2010. MFR Course Grade Re￾port. National Police Academy. June 22 - July 04, 2010. Kathmandu: NSET 33. National Society for Earthquake Technology-Nepal (NSET). 2010. Collapsed Structure Search and Rescue (CSSR) Course. Grade Sheet. National Police Academy. Oct. 26 - Nov. 03, 2010. Kathmandu: NSET 34. National Society for Earthquake Technology-Nepal (NSET). 2010. MFR Course Grade Re￾port. National Police Academy. March 2-14, 2010. Kathmandu: NSET 35. National Society for Earthquake Technology-Nepal (NSET). 2010. Training for Instructors (TFI) Course. Grade Sheet. 20-24 June 2011. Kathmandu: NSET NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 42 36. National Society for Earthquake Technology-Nepal (NSET). 2011. MFR Instructors Work￾shop. Grade Sheet. June 26-30, 2011. Kathmandu: NSET 37. National Society for Earthquake Technology-Nepal (NSET). 2012. CSSR Instructor Work￾shop Grade Sheet. National Police Academy, Kathmandu. 09-15, Jan 2012. Kathmandu: NSET 38. National Society for Earthquake Technology-Nepal (NSET). 2013. Medical First Responder Course Grade Sheet. February 11-23, 2013. Kathmandu: NSET 39. National Society for Earthquake Technology-Nepal (NSET). 2013. Collapsed Structure Search & Rescue (CSSR) Course. APF, DMTC, Kurintar, Chitawon. March 21-28, 2013. Kathmandu: NSET 40. National Society for Earthquake Technology-Nepal (NSET). 2013. Training for Instructors (TFI) Course. Post-Course Evaluation. Kathmandu: NSET 41. National Society for Earthquake Technology-Nepal (NSET). 2013. Training for Instructors (TFI) Course. Pre-Course Evaluation. Kathmandu: NSET 42. National Society for Earthquake Technology-Nepal (NSET). 2014. MFR Instructors Work￾shop. Grade Sheet. APF, DMTC, Kurintar, Nepal. March 20-24, 2014. Kathmandu: NSET 43. National Society for Earthquake Technology-Nepal (NSET). 2014. CSSR Instructor Work￾shop Grade Sheet. APF, DMTC, Kurintar, Nepal. March 28-April 3, 2014. Kathmandu: NSET 44. National Society for Earthquake Technology-Nepal (NSET). 2015. Medical First Responder (MFR) Course Post-Test. Kathmandu: NSET 45. National Society for Earthquake Technology-Nepal (NSET). 2015. Medical First Responder (MFR) Course Pre-Test. Kathmandu: NSET 46. National Society for Earthquake Technology-Nepal (NSET). 2015. Hospital Preparedness for Emergencies (HOPE) Course. Pre/Post Test Results. 2-5 October 2015, Park Village. Kath￾mandu: NSET 47. National Society for Earthquake Technology-Nepal (NSET). 2015. MFR Course Grade Re￾port. Nepal Police Disaster Management Division. 1-13 December, 2015. Kathmandu: NSET 48. National Society for Earthquake Technology-Nepal (NSET). 2016. Briefing to USAID/OFDA M&E Team. Overview of NSET-PEER Activities. November 18, 2016. Kathmandu: NSET 49. National Society for Earthquake Technology-Nepal (NSET). 2016. Collapsed Structure Search & Rescue (CSSR) Course. Nepal Police Disaster Management Division, Samakhushi, Kath￾mandu, Nepal. January 7-14, 2016. Kathmandu: NSET 50. National Society for Earthquake Technology-Nepal (NSET). 2016. MFR Instructors Work￾shop Grade Sheet. Nepal Police, DMD, Kathmandu, Nepal. February 14-18, 2016. Kathmandu: NSET 51. National Society for Earthquake Technology-Nepal (NSET). 2016. CSSR Instructor Work￾shop. Grade Sheet. Nepal Police, DMD, Samakhushi, Kathmandu, Nepal. February 22-28, 2016. Kathmandu: NSET 52. National Society for Earthquake Technology-Nepal (NSET). 2016. Training for Instructors (TFI) Course. Nepal Police Disaster Management Division, Samakhushi, Kathmandu. February 8-12, 2016. Kathmandu: NSET 53. National Society for Earthquake Technology-Nepal (NSET). 2016. Hospital Preparedness for Emergencies (HOPE) Course. 26-29 October 2016, Park Village. Post- Test Result. Kath￾mandu: NSET 54. National Society for Earthquake Technology-Nepal (NSET). 2016. Briefing to USAID/OFDA M&E Team. Overview of NSET-PEER Activities. December 23, 2016. Kathmandu: NSET NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 43 55. National Society for Earthquake Technology-Nepal (NSET). 2016. PEER Courses and Meet￾ings conducted in Nepal in PEER Stage 3 (April 2009 to December 2013). Kathmandu: NSET 56. National Society for Earthquake Technology-Nepal (NSET). 2016. Collapsed Structure Search and Rescue Course. Post Test. Kathmandu: NSET 57. National Society for Earthquake Technology-Nepal (NSET). 2016. Collapsed Structure Search and Rescue Course. Pre-Test. Kathmandu: NSET 58. Program For Enhancement of Emergency Response (PEER). 2006. Hospital Preparedness for Emergencies (HOPE) Course. Training for Instructors. Instructor Workbook. Washington: USAID/OFDA, NSET 59. Program For Enhancement of Emergency Response (PEER). 2006. Hospital Preparedness for Emergencies (HOPE) Course. Training for Instructors. Lesson Plan. Washington: USAID/OFDA, NSET 60. Program For Enhancement of Emergency Response (PEER). 2009. Program for Enhancement of Emergency Response (PEER) Stage 3. Monitoring and Evaluation Plan, 2009-2010. Bangkok: Asian Disaster Preparedness Center 61. Program For Enhancement of Emergency Response (PEER). 2014. Master Instructors' Work￾shop. Facilitator's Guide. Washington: USAID/OFDA, ADPC, NSET, ARC 62. Program For Enhancement of Emergency Response (PEER). 2015. Medical First Responder Course. Kathmandu: NSET 63. Program For Enhancement of Emergency Response (PEER). 2015. Hospital Preparedness for Emergencies (HOPE) Course. October 2- 5, 2015. Kathmandu: NSET 64. Program For Enhancement of Emergency Response (PEER). 2016. CSSR End Users Course. March 31-April 04, 2016 (in Nepali). Kathmandu: PEER 65. Program For Enhancement of Emergency Response (PEER). 2016. MFR End Users Course. March 22-28, 2016 (in Nepali). Kathmandu: NSET 66. Program For Enhancement of Emergency Response (PEER). 2016. CSSR Instructors' Work￾shop (CSSRIW). Kathmandu: PEER 67. Program For Enhancement of Emergency Response (PEER). 2016. Training for Instructors (TFI) Course. Kathmandu: NSET 68. Program For Enhancement of Emergency Response (PEER). 2016. Hospital Preparedness for Emergencies (HOPE) Course. April 26-29, 2016. Kathmandu: NSET 69. Program For Enhancement of Emergency Response (PEER). 2016. CSSR End Users Course. April 17-21, 2016 (in Nepali). Kathmandu: PEER 70. Program For Enhancement of Emergency Response (PEER). 2016. MFR End Users Course. April 08-14, 2016 (in Nepali). Kathmandu: NSET 71. Program For Enhancement of Emergency Response (PEER). 2016. Community Action for Disaster Response. Instructor's Guide. Washington: USAID/OFDA, ADPC, ARC, NSET 72. Program For Enhancement of Emergency Response (PEER). 2016. Community Action for Disaster Response. Participant's Workbook. Washington: USAID/OFDA, ADPC, ARC, NSET 73. Program For Enhancement of Emergency Response (PEER). 2016. Emergency Response Ca￾pacity Building for Non-Professional First Responders in the Community. Training Curriculum Framework. Washington: USAID/OFDA, ADPC, ARC 74. Program For Enhancement of Emergency Response (PEER). 2016. Community Action for Disaster Response. Coordinator's Guide. Washington: USAID/OFDA, ADPC, ARC, NSET 75. Program For Enhancement of Emergency Response (PEER). 2016. Hospital Preparedness for Emergencies (HOPE) Course. Coordinator's Guide. Washington: USAID/OFDA, ADPC, NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 44 NSET 76. Program For Enhancement of Emergency Response (PEER). 2016. Hospital Preparedness for Emergencies (HOPE) Course. Participants Workbook. Washington: USAID/OFDA, ADPC, NSET 77. Program For Enhancement of Emergency Response (PEER). 2016. Hospital Preparedness for Emergencies (HOPE) Course. Post Test Exam. Washington: USAID/OFDA, NSET 78. Program For Enhancement of Emergency Response (PEER). 2016. Hospital Preparedness for Emergencies (HOPE) Course. Pre-Test Exam. Washington: USAID/OFDA, NSET 79. Program For Enhancement of Emergency Response (PEER). 2016. Hospital Preparedness for Emergencies (HOPE) Course. Pre-Test, Post Test Exam Key. Washington: USAID/OFDA, NSET 80. USAID. 2009. Award Letter: Program for Enhancement of Emergency Response (PEER) - Stage 3. Washington: USAID 81. USAID. 2010. Modification of Assistance Award 01: Program for Enhancement of Emergency Response (PEER) - Stage 3. Washington: USAID 82. USAID. 2010. Modification of Assistance Award 02: Program for Enhancement of Emergency Response (PEER) - Stage 3. Washington: USAID 83. USAID. 2010. Modification of Assistance Award 03: Program for Enhancement of Emergency Response (PEER) - Stage 3. Washington: USAID 84. USAID. 2012. Modification of Assistance Award 05: Program for Enhancement of Emergency Response (PEER) - Stage 3. Washington: USAID 85. USAID. 2012. Modification of Assistance Award 04: Program for Enhancement of Emergency Response (PEER) - Stage 3. Washington: USAID 86. USAID. 2013. Modification of Assistance Award 06: Program for Enhancement of Emergency Response (PEER) - Stage 3. Washington: USAID 87. USAID. 2014. Award Letter: Program for Enhancement of Emergency Response (PEER) - Stage 4. Washington: USAID 88. USAID. 2015. Modification of Assistance Award 01: Program for Enhancement of Emergency Response (PEER) - Stage 4. Washington: USAID 89. USAID. Disaster Risk Reduction, Reconstruction and Resilience DR4 Office. 2016. Fact Sheet. Program for Enhancement of Emergency Response (PEER) Stage IV. Washington: USAID 90. USAID. Disaster Risk Reduction Office. 2014. Fact Sheet: Program for the Enhancement of Emergency Response (PEER) Phase III. Washington: USAID BCIPN DOCUMENTS 1. Building Code Implementation Program. 2016. Activity Report of Bharatpur Sub-Metropolitan City Under Building Code Implementation Program in Municipalities of Nepal (BCIPN). Kath￾mandu: SNET 2. Building Code Implementation Program in Municipalities of Nepal (BCIPN). 2414. Notes of Teleconference among NSET, DRRO/USAID NEPAL, and USAID/OFDA Regional Office, Bangkok. Date: May 6, 2014. Bangkok: USAID 3. National Society for Earthquake Technology-Nepal (NSET). 2012. Program Performance Re￾port. Building Code Implementation Program in Municipalities of Nepal (BCIPN). Quarterly Report # 1: October 01 to December 31, 2012. Katmandu: NSET NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 45 4. National Society for Earthquake Technology-Nepal (NSET). 2013. Program Performance Re￾port. Building Code Implementation Program in Municipalities of Nepal (BCIPN). Quarterly Report # 2: January 01 to March 31, 2013. Katmandu: NSET 5. National Society for Earthquake Technology-Nepal (NSET). 2013. Program Performance Re￾port. Building Code Implementation Program in Municipalities of Nepal (BCIPN). Quarterly Report #3: April 01 to June 30, 2013. Katmandu: NSET 6. National Society for Earthquake Technology-Nepal (NSET). 2013. Program Performance Re￾port. Building Code Implementation Program in Municipalities of Nepal (BCIPN). Quarterly Report #4: July 1 to September 30, 2013. Katmandu: NSET 7. National Society for Earthquake Technology-Nepal (NSET). 2013. Activity Report: Program Launching Workshop on Building Code Implementation Program in Municipalities of Nepal (BCIPN). Kathmandu: NSET 8. National Society for Earthquake Technology-Nepal (NSET). 2014. Program Performance Re￾port. Building Code Implementation Program in Municipalities of Nepal (BCIPN). Quarterly Report #5: October 01 to December 31, 2013. Katmandu: NSET 9. National Society for Earthquake Technology-Nepal (NSET). 2014. Program Performance Re￾port. Building Code Implementation Program in Municipalities of Nepal (BCIPN). Quarterly Report #6: Jan 01 to Mar 31, 2014. Katmandu: NSET 10. National Society for Earthquake Technology-Nepal (NSET). 2014. Program Performance Re￾port. Building Code Implementation Program in Municipalities of Nepal (BCIPN). Quarterly Report #7: Apr. 01 to Jun. 30, 2014. Katmandu: NSET 11. National Society for Earthquake Technology-Nepal (NSET). 2014. Program Performance Re￾port. Building Code Implementation Program in Municipalities of Nepal (BCIPN). Quarterly Report #8: Jul. 01 to Sep. 30, 2014. Katmandu: NSET 12. National Society for Earthquake Technology-Nepal (NSET). 2014. Program Performance Re￾port. Building Code Implementation Program in Municipalities of Nepal (BCIPN). A Report on Planning Workshop- Biratnagar. Katmandu: NSET 13. National Society for Earthquake Technology-Nepal (NSET). 2014. Program Performance Re￾port. Building Code Implementation Program in Municipalities of Nepal (BCIPN). A Report on Planning Workshop-Nepalgunj. Katmandu: NSET 14. National Society for Earthquake Technology-Nepal (NSET). 2014. Program Performance Re￾port. Building Code Implementation Program in Municipalities of Nepal (BCIPN). A Report on Planning Workshop-Sauraha, Chitwan. Katmandu: NSET 15. National Society for Earthquake Technology-Nepal (NSET). 2014. PCIPN: Target vs. Accom￾plishments up to Mar. 2014. Katmandu: NSET 16. National Society for Earthquake Technology-Nepal (NSET). 2015. Program Performance Re￾port. Building Code Implementation Program in Municipalities of Nepal (BCIPN). Quarterly Report # 9: Oct. 01 to Dec. 31, 2014. Katmandu: NSET 17. National Society for Earthquake Technology-Nepal (NSET). 2015. Program Performance Re￾port. Building Code Implementation Program in Municipalities of Nepal (BCIPN). Quarterly Report #10: January – March 2015. Katmandu: NSET 18. National Society for Earthquake Technology-Nepal (NSET). 2015. Program Performance Re￾port. Building Code Implementation Program in Municipalities of Nepal (BCIPN). Quarterly Report # 11: April to June 2015. Katmandu: NSET 19. National Society for Earthquake Technology-Nepal (NSET). 2015. Program Performance Re￾port. Building Code Implementation Program in Municipalities of Nepal (BCIPN). Quarterly NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 46 Report #12: July to Sept. 2015. Katmandu: NSET 20. National Society for Earthquake Technology-Nepal (NSET). 2015. Baseline Study Report on Risk Perception Survey in the Municipalities. Building Code Implementation Program in Mu￾nicipalities of Nepal (BCIPN). Kathmandu: NSET 21. National Society for Earthquake Technology-Nepal (NSET). 2016. Program Performance Re￾port. Building Code Implementation Program in Municipalities of Nepal (BCIPN). Quarterly Report #13: Oct. to Dec., 2015. Katmandu: NSET 22. National Society for Earthquake Technology-Nepal (NSET). 2016. Program Performance Re￾port. Building Code Implementation Program in Municipalities of Nepal (BCIPN). Quarterly Report #14: Jan. to March, 2016. Katmandu: NSET 23. National Society for Earthquake Technology-Nepal (NSET). 2016. Program Performance Re￾port. Building Code Implementation Program in Municipalities of Nepal (BCIPN). Quarterly Report #15: April to June, 2016. Katmandu: NSET 24. National Society for Earthquake Technology-Nepal (NSET). 2016. Building Code Implemen￾tation Program in Municipalities of Nepal (BCIPN). 18 Nov 2016. Kathmandu: NSET 25. National Society for Earthquake Technology-Nepal (NSET). 2016. Monitoring & Evaluation Plan for Building Code Implementation Program in Municipalities of Nepal (BCIPN). 18 No￾vember 2016. Katmandu: NSET 26. National Society for Earthquake Technology-Nepal (NSET). 2016. Work Schedule for BCIPN M&E. Kathmandu: NSET 27. National Society for Earthquake Technology-Nepal (NSET). 2016. Building Code Implemen￾tation Program in Municipalities of Nepal (BCIPN). A program to support earthquake safer building construction in Nepal. 23 December 2016. Kathmandu: NSET 28. National Society for Earthquake Technology-Nepal (NSET). 2016. Monitoring & Evaluation Plan for Building Code Implementation Program in Municipalities of Nepal(BCIPN). 23 De￾cember 2016. Kathmandu: NSET 29. National Society for Earthquake Technology-Nepal (NSET). 2016. Training Course for Engi￾neers and Sub Engineers on Earthquake Resistant Design and Construction of Buildings. Pre￾Course Evaluation. Kathmandu: NSET 30. National Society for Earthquake Technology-Nepal (NSET). 2016. Training Course for Engi￾neers and Sub Engineers on Earthquake Resistant Design and Construction of Buildings. Post￾Course Evaluation. Kathmandu: NSET 31. USAID. 2012. Award Letter: Building Code Implementation Program in Municipalities of Ne￾pal (BCIPN). Washington: USAID 32. USAID. 2013. Modification of Assistance Award 01: Building Code Implementation Program in Municipalities of Nepal (BCIPN). Washington: USAID 33. USAID. 2014. Modification of Assistance Award 02: Building Code Implementation Program in Municipalities of Nepal (BCIPN). Washington: USAID 34. USAID. 2016. Modification of Assistance Award 03: Building Code Implementation Program in Municipalities of Nepal (BCIPN). Washington: USAID 35. USAID. Disaster Risk Reduction, Reconstruction and Resilience DR4 Office. 2016. Fact Sheet. Building Code Implementation Program in Municipalities of Nepal (BCIPN). Washington: USAID NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 47 WIDER LITERATURE 1. ACT Alliance Nepal Forum. 2016. ACT Response to the Gorkha Earthquake: Lessons Learnt. Kathmandu: ACT Alliance Nepal Forum 2. Bakrania, S. 2015. Urbanisation and urban growth in Nepal. Birmingham: Governance, Social Development, Humanitarian, Conflict (GSDRC) 3. Buescher, G., Shrestha, D. P., and Dhruba Gautam, D. 2013. Mid-Term Evaluation of the Comprehensive Disaster Risk Management Programme (CDRMP). Partnership between UNDP Nepal CO and the Government of Nepal. Katmandu: UNDP 4. Build Change. 2014. Seismic Retrofit of Housing in Post-Disaster Situations – Basic Engi￾neering Principles For Development Professionals: A Primer. Washington: USAID 5. Center of Resilient Development and Multi Disciplinary Consultant. 2015. Comprehensive Disaster Risk Management Programme. Project Completion Report. Kathmandu: UNDP 6. Center of Resilient Development and Multi Disciplinary Consultant. 2015. Report on Train￾ings to Engineers and Junior Engineers on NBC Compliant Building Design and Construc￾tion. Kathmandu: UNDP 7. Center of Resilient Development and Multi Disciplinary Consultant. 2015. Report on Build￾ing Permit System and Dedicated Implementation Support. Kathmandu: UNDP 8. CORD and MRB & Associates. 2013. Seismic Retrofitting Guidelines of Buildings in Nepal. Training Curriculum (Construction). Part II Presentation Material & Exercise for Trainee. Katmandu: UNDP & Government of Nepal 9. Giri, N. 2016. Implementation of Nepal National Building Code Through Automated Build￾ing Permit System. 10. Government of Nepal. 2016. Disaster Risk Reduction in Nepal: Achievements, Challenges and Ways Forward. Katmandu 11. Government of Nepal. 2016. Nepal: Striving for a Disaster Resilient Nation. Kathmandu: Government of Nepal 12. Government of Nepal. Ministry of Urban Development. Department of Urban Develop￾ment and Building Construction. 2015. National Plan of Action for Safer Building Construc￾tion (NaPA)-2072. Katmandu: DUDBC 13. Housing Recovery and Reconstruction. 2016. Nepal - Coverage of Mason Training on Earth￾quake Resistant Construction - DUDBC Curriculum (as of 17 Feb 2016). Katmandu: HRRP - Nepal 14. Murtaza, N., Arnborg, A., and Gautam, D. 2016. Final Evaluation Report. Nepal Earthquake Response NPL 151. Katmandu: Act Alliance 15. Nepal Administrative Staff College. 2014. Curriculum of DRM/CRM Training. Jawalakhel: Nepal Administrative Staff College 16. Nepal Risk Reduction Consortium. 2013. Flagship Programmes. Katmandu: NRRC 17. Nepal Risk Reduction Consortium. 2014. Safer Building Construction. Mapping Exercise. Analysis Report. Katmandu: NRRC 18. Nepal Risk Reduction Consortium. 2014. Documentation of Government Initiatives on Safer Building Construction. Katmandu: NRRC 19. Nepal Risk Reduction Consortium. 2016. DRR Mainstreaming Mapping Report. Katmandu: NRRC 20. OXFAM. 2014. GIS Based Mapping of Emergency Stockpiles in Kathmandu Valley. Kat￾mandu: OXFAM NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 48 21. Patton, M. Q. 2013. Utilization-Focused Evaluation (U-FE) Checklist. M. Q. Patton 22. Ramanagar, C. 2016. Implementation of 20 Years. SDMP (2015 - 2035) for KV. 23. Russel, B. 2016. Private Sector Engagement in Community Based Disaster Risk Management – Nepal. 24. SAARC Disaster Management Centre. 2011. Rapid Structural and Non-Structural Assess￾ment of School and Hospital Buildings in SAARC Countries. New Dehli: SAARC Disaster Management Centre 25. Taylor, G., Vatsa, K., Gurung, M., and Couture, E. 2013. Review of the Nepal Risk Reduc￾tion Consortium (NRRC). Final Report. Katmandu: UNDP & World Bank 26. The Government of Nepal. Ministry of Physical Planning and Works and Earthquake Risk Reduction and Recovery Preparedness Programme for Nepal. 2009. Recommendation for Update of Nepal National Building Code. Final Report. Katmandu: Government of Nepal & UNDP 27. The Interagency Common Feedback Project. 2016. Inter Agency Common Feedback Re￾port. January 2016 Feedback Report, Nepal. Katmandu: CFP 28. The Lutheran World Federation Nepal. 2016. From Relief to Resilience. LWF Nepal's Re￾sponse to the Gorkha Earthquake. Kathmandu: The Lutheran World Federation (LWF) Ne￾pal 29. United Nations Centre for Regional Development. 2010. Handbook. Building Code Imple￾mentation. Learning from Experience of Lalitpur Sub-Metropolitan City, Nepal. Kobe, Japan: UNCRD 30. USAID. 2010. Guide to Gender Integration and Analysis. Additional Help for ADS Chapters 201 and 203. Washington: USAID 31. USAID. 2012. ADS Chapter 203. Assessing and Learning. Washington: USAID 32. USAID. 2012. Gender Equality and Female Empowerment Policy. USAID Policy. Washing￾ton: USAID 33. USAID. 2012. Preparing Evaluation Reports. Washington: USAID 34. USAID. 2013. Checklist for Engendering Evaluations. Washington: USAID 35. USAID. 2013. Summary Checklist for Assessing USAID Evaluation Reports. Washington: USAID 36. USAID. 2013. ADS Chapter 205. Integrating Gender Equality and Female Empowerment in USAID's Program Cycle. Washington: USAID 37. USAID. 2013. Conducting Mixed-Method Evaluations. Washington: USAID 38. USAID. 2013. Gender Integration in Performance Plans and Reports. Washington: USAID 39. USAID. 2013. Focus Group Interviews. Washington: USAID 40. USAID. 2013. Gender Integration in MRRs and OPs. Washington: USAID 41. USAID. 2016. Gender Integration in Democracy, Human Rights, and Governance (DRG). Programming Toolkit. Washington: USAID 42. USAID. 2016. South Asia – Disaster Risk Reduction. Washington: USAID 43. USAID. 2016. USAID/OFDA Disaster Risk Reduction (DRR) Programs in South Asia. Wash￾ington: USAID 44. USAID Nepal. 2007. Gender and Inclusion Assessment USAID/Nepal. Washington: USAID 45. USAID Nepal Disaster Risk Reduction Office. 2013. U. S. Disaster Risk Reduction Strategic Framework - Nepal. Katrmandou: USAID-DRR Office 46. USAID/OFDA. 2013. Training for Instructors. Instructor Workbook. Washington: USAID/OFDA NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 49 47. USAID/OFDA. 2013. Training for Instructors. Lesson Plan. Washington: USAID/OFDA 48. USAID/OFDA. 2016. Disaster Risk Reduction in Nepal: Evaluation Statement of Work. Washington: USAID 49. World Bank and DFID. 2006. Unequal Citizens. Gender, Caste and Ethnic Exclusion in Ne￾pal. Summary. Kathmandu: World Bank and DFID NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 50 LIST OF PERSONS/GROUPS INTERVIEWED1 N Interviews or focus group discussions Affiliation, location Role Date 1 Stakeholder interviews Ministry of Defense (MoD), Kathmandu Under Secretary 16th January 2017 2 Ministry of Home Affairs (MoHA), Kathmandu Joint Secretary 16th January 2017 3 Ministry of Health, Kathmandu Division Chief, Curative Services 26th January 2017 4 Ministry of Urban Development, Kathmandu Secretary 16th January 2017 5 Ministry of Federal Affairs and Local Development (MoFALD), Kathmandu Under General 20th January 2017 6 Department of Urban Development and Building Construction (DUDBC), Kathmandu Director General 20th January 2017 7 Nepal Police, Kathmandu Police Inspector 22nd January 2017 8 Nepal Armed Police, Kathmandu DSP 20th January 2017 9 Karyabinayak Municipality Executive Officer, Engineers, Sub-Engineers, Masons 24th January 2017 10 Executive Officer 24th January 2017 11 Chautara Municipality Executive Officer & Municipal Engineer 2nd February 2017 12 Institute of Medicine (IOM), Kathmandu HOPE Coordinator (Doctor) January 2017 13 Nepal Red Cross Society (NRCS), Kathmandu District CADRE Instructor 23rd January 2017 14 Programme Coordinator January 2017 15 CARE, Kathmandu Consortium Program Manager, Disaster Risk Reduction and Resilience 25th January 2017 16 OXFAM, Kathmandu Sustainable Development Director, DRR & Climate Change Adaptation Program Manager 25th January 2017 17 British Red Cross, Tahachal Country Officer / CADRE Instructor 23rd January 2017 18 NSET, Kathmandu GESI and Environment Specialist 20th January 2017 19 Executive Director, Executive Member, Deputy Chief of Party, Project Manager 3rd February 2017 20 United Nations Development Program (UNDP), Kathmandu UNDP representative for Flagship 5 & Assistant Country Di￾rector 26th January 2017 21 World Health Organization, Kathmandu National Professional Officer 26th January 2017 22 Japan International Cooperation Agency, Kath￾mandu Representative & Associate Program Manager 26th January 2017 1 This list does not include the structured interviews’ informants (Recipients of BCIPN and PEER trainings). NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 51 N Interviews or focus group discussions Affiliation, location Role Date 23 Asian Development Bank, Kathmandu Administrative Assistant, Sr. Project Officer, Project Manage￾ment Specialist 1st February 2017 24 Focus Group Discus￾sions Biratnagar Sub Metropolitan City, Biratnagar 10 Masons 17th February 2017 25 Biratnagar Sub Metropolitan City, Biratnagar 6 Social Health Workers/Volunteers 17th February 2017 26 Dharan Sub Metropolitan City, Dharan 7 Engineers 18th February 2017 27 Bhimeshwer Municipality, Dolkha 11 Social Mobilizers 9th February 2017 28 Bhimeshwer Municipality, Dolkha 9 Masons 9th February 2017 29 Karyabinayak Municipality 9 Masons 2nd February 2017 30 Putalibazaar Municipality 10 Social Mobilizers 24th February 2017 31 Putalibazaar Municipality 10 Masons 24th February 2017 32 Vyas Municipality, Vyas 7 Municipal staff 14th March 2017 33 Vyas Municipality, Vyas 13 Masons 14th March 2017 34 DLPIU, MoUD, Chautara 6DLPIU Engineers 25th March 2017 35 NRA Information Center Hall, Chautara 6 Social Mobilizers, Ward Secretary 25th March 2017 36 Pipal Danda, Chautara 9 Masons 25th March 2017 37 Construction site, Gorkha 14 Masons 28th February 2017 38 Gorkha Municipality 12 Municipal Staffs 28th February 2017 39 Lalitpur Metropolitan City, Lalitpur 11 Masons 8th March 2017 40 Lalitpur Metropolitan City, Lalitpur 13 Engineers 8th March 2017 41 Lalitpur Metropolitan City, Lalitpur 9 Municipal\Non-technical Staffs 8th March 2017 42 Mason’s Association Building, Pokhara 15 Untrained Masons 19th March 2017 43 NRCS, HQ, Kalimati 8 CADRE Trainees 12th February 2017 44 Nepal Army, Sundarijaal 6 CSSR Trainees 22nd February 2014 45 Nepal Army Hospital, Chhauni, Kathmandu 10 HOPE Trainees 26th February 2017 46 Nepal Police, Disaster Management Section, Sa￾makhosi, Kathmandu 11 MFR Trainees 14th February 2017 47 Key informant inter￾views Department of Urban Development and Building Construction (DUDBC), Division Office, Bi￾ratnagar Senior Divisional Engineer / Engineer 15th February 2017 48 Local Building Construction Entrepreneurs (Ma￾son) Association, Biratnagar Treasurer and Office Manager 15th February 2017 49 Local Building Construction Entrepreneurs (Ma￾son) Association, Itahari President 16th February 2017 50 Local Building Construction Entrepreneurs (Ma￾son) Association, Ineruwa General Secretary 16th February 2017 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 52 N Interviews or focus group discussions Affiliation, location Role Date 51 Dharan Sub Metropolitan City Sr. Engineer, Division Chief, Urban Development Division 17th February 2017 52 Municipality, Charikot, Dolkha Senior Engineer 8th February 2017 53 Karyavinayak Municipality, Lalitpur Chief Executive Officer 22nd January 2017 54 Karyavinayak Municipality, Lalitpur Senior Engineer 22nd January 2017 55 Putalibazar municipality Executive Officer 22nd February 2017 56 Mason’s Association, Putalibazar Secretary 22nd February 2017 57 Nepali Congress (NC), Putalibazar Representative 25th February 2017 58 Ward, Putalibazar Secretary 26 February 2017 59 Vyas Municipality Senior Civil Engineer 13th March 2017 60 Nirman Byabasahi Sangh, Vyas President 15th March 2017 61 Division Office, DLPIU, Chautara Division Chief 26th March 2017 62 Nawa Buddha Nirman Sewa and Supplier, Chau￾tara Contractor 26th March 2017 63 Chautara Shangachowkgadi Municipality Chief Executive 27th March 2017 64 Chautara Shangachowkgadi Municipality Municipal Engineer 27th March 2017 65 Gorkha Municipality Engineer 27th February 2017 66 DUDBC, Gorkha Chief 1st March 2017 67 Mason’s Association, Gorkha President 1st March 2017 68 Gorkha Municipality Administrative Officer 2nd March 2017 69 Department of Urban Development, Lalitpur Met￾ropolitan City Senior Engineer., Division Chief 9th March 2017 70 Pokhara Sub-Municipality Executive Officer 17th March 2017 71 Pokhara Sub-Municipality Senior Engineer 19th March 2017 72 Mason’s Association Building, Pokhara Secretary 20th March 2017 73 Nepal Red Cross Society, Kathmandu Senior Program Officer 27th February 2017 74 NRCS, HQ, Kalimati CADRE instructor 9th February 2017 75 NRCS, HQ, Kalimati CADRE instructor 10th February 2017 76 NRCS, HQ, Kalimati CADRE instructor 10th February 2017 77 Nepal Army, Bhadrakali Captain 28th February 2017 78 Nepal Police, Headquarter, Kathmandu ASI 8th March 2017 79 Nepal Staff College, Nepal Police, Kathmandu DSP 28th February 2017 80 Nepal Police, Singh Durbar DSP 12th March 2017 81 Nepal Army Hospital, Chhauni Doctor 28th February 2017 82 Chhetrapati Hospital/Alka Hospital Neuro Physiotherapist 9th March 2017 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 53 N Interviews or focus group discussions Affiliation, location Role Date 83 Nepal Police Hospital, Kathmandu Senior Nurse 9th March 2017 84 Patan Hospital, Lalitpur Doctor 10th March 2017 85 Armed Police Force, Kathmandu DSP 23rd February 2017 86 Nepal Police Headquarter, Kathmandu Police Inspector 23rd February 2017 87 Nepal Police Headquarter, Kathmandu Police Inspector 26th February 2017 88 Armed Police Force, Kakani, Kathmandu DSP 26th February 2017 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 54 STRUCTURED QUESTIONAIRE RESULTS – TABULAR FORMAT PEER – Quantitative Data (Results of Structured Interviews) Table 1 Identification and Background (CADRE Trainees)........................................................................................................................................... 55 Table 2 Training Session (CADRE Trainees)................................................................................................................................................................. 57 Table 3 Post-training (CADRE Trainees) ...................................................................................................................................................................... 59 Table 4 General Appreciation of Training (CADRE Trainees) ...................................................................................................................................... 60 Table 5 Trainees’ Own Appreciation of Skills Retention (CADRE Trainees)................................................................................................................. 60 Table 6 Identification and Background (CSSR Trainees) ............................................................................................................................................... 61 Table 7 Training Session (CSSR Trainees)..................................................................................................................................................................... 62 Table 8 Post-training (CSSR Trainees)........................................................................................................................................................................... 64 Table 9 General Appreciation of Training (CSSR Trainees)........................................................................................................................................... 65 Table 10 Trainees’ Own Appreciation of Skills Retention (CSSR Trainees) ................................................................................................................... 65 Table 11 Identification and Background (HOPE Trainees) ........................................................................................................................................... 66 Table 12 Training Session (HOPE Trainees)................................................................................................................................................................. 67 Table 13 Post-training (HOPE Trainees)....................................................................................................................................................................... 70 Table 14 General Appreciation of Training (HOPE Trainees)....................................................................................................................................... 70 Table 15 Identification and Background (HOPE Trainees) ........................................................................................................................................... 71 Table 16 Training Session (MFR Trainees).................................................................................................................................................................... 72 Table 17 Post-training (MFR Trainees).......................................................................................................................................................................... 73 Table 18 General Appreciation of Training (MFR Trainees).......................................................................................................................................... 73 Table 19 Repetition/Refreshment of Practical Skills Training (MFR Trainees) .............................................................................................................. 74 Table 20 Trainees’ Own Appreciation of Skills Retention (MFR Trainees) .................................................................................................................... 75 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 55 Results of Structured Interviews – CADRE Trainees Num￾ber Ratio % Gender Male 29 70.7% Female 12 29.3% Total 41 100.0% Social groups Dalit 0 0.0% Adibasi/Janajati 17 41.5% Religious Minorities 0 0.0% Disadvantaged Terai Caste Group (Pichadda Barga) 4 9.8% Others (e.g. Bahun, Chhetri (both Terai and Hill), Thakuri, Sanyasi, Rana, Shaha, Rajput) 20 48.7% Total 41 100.0% Other trainings received No 17 41.5% Yes 24 58.5% Total 41 100.0% Part of task force or CDMC (Community Disaster Management Committee) No 28 68.3% Yes 13 31.7% Total 41 100.0% Table 1 Identification and Background (CADRE Trainees) NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 56 Num￾ber Ratio % Mean Me￾dian Mode Standard De￾viation Ways of finding out about training Organized by employer 23 56.1% Colleagues 2 4.9% Others 16 39.0% Total 41 100.0% Training helped to better understand emergency response No 0 0.0% Yes 41 100.0% Total 41 100.0% Training helped to better understand specifically earthquake response man￾agement No 1 2.4% Yes 40 97.6% Total 41 100.0% Training helped to better understand roles and responsibilities in in emer￾gencies No 0 0.0% Yes 41 100.0% Total 41 100.0% Training helped to better understand basic elements of first aid No 0 0.0% Yes 41 100.0% Total 41 100.0% Training helped to better understand how to conduct a triage No 2 4.9% Yes 39 95.1% Total 41 100.0% Training helped to better understand community light search and rescue No 2 4.9% NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 57 Num￾ber Ratio % Mean Me￾dian Mode Standard De￾viation Yes 39 95.1% Total 41 100.0% Extent to which understanding of the way to respond (actions to take) during earthquakes and emer￾gencies improved - Scale from 1 (very little) to 4 (very much) 41 3.12 3.00 3.00 .71 Extent to which understanding of community actions during disasters improved - Scale from 1 (very little) to 4 (very much) 41 3.27 3.00 4.00 .78 Extent to which understanding of the importance of fire and water safety improved - Scale from 1 (very little) to 4 (very much) 41 3.44 4.00 4.00 .63 Extent to which understanding of the role to play improved - Scale from 1 (very little) to 4 (very much) 41 3.41 3.00 3.00 .59 Extent to which understanding of the relationships with professional responders improved - Scale from 1 (very little) to 4 (very much) 41 3.22 3.00 3.00 .69 Table 2 Training Session (CADRE Trainees) Num￾ber Ratio % Mean Me￾dian Mode Standard De￾viation What was learnt in the training in terms of supporting communities and families was put into practice No 1 2.4% Yes 40 97.6% Total 41 100.0% What was learnt in the training in terms of advising community members on what to do was put into practice No 2 4.9% Yes 39 95.1% Total 41 100.0% What was learnt in the training in terms of improving community awareness was put into practice No 1 2.4% Yes 40 97.6% NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 58 Num￾ber Ratio % Mean Me￾dian Mode Standard De￾viation Total 41 100.0% Training enhanced confidence to talk to community members and decision makers No 0 0.0% Yes 41 100.0% Total 41 100.0% Involvement in Gorkha earthquake emergency response No 3 7.3% Yes 38 92.7% Total 41 100.0% Extent to which skills acquired during the training were used in Gorkha earthquake emergency re￾sponse - Scale from 1 (very little) to 4 (very much) 41 3.21 3.00 4.00 .81 Extent to which community members had a greater understanding of what to do during Gorkha earth￾quake emergency response - Scale from 1 (very little) to 4 (very much) 41 3.05 3.00 3.00 .77 Extent to which communities were open to assistance during Gorkha earthquake emergency response - Scale from 1 (very little) to 4 (very much) 41 3.08 3.00 3.00 .71 Extent to which communities were more aware of what to do to reduce Gorkha earthquake risks - Scale from 1 (very little) to 4 (very much) 41 2.71 3.00 2.00 .93 Extent to which skills acquired during the training were used during and after other emergencies - Scale from 1 (very little) to 4 (very much) 41 3.29 3.00 4.00 .80 Extent to which community members had a greater understanding of what to do during and after other emergencies - Scale from 1 (very little) to 4 (very much) 41 2.66 3.00 3.00 .71 Extent to which communities were open to assistance during and after other emergencies - Scale from 1 (very little) to 4 (very much) 41 2.97 3.00 3.00 .79 Extent to which communities were more aware of what to do to reduce disaster risks for other emer￾gencies - Scale from 1 (very little) to 4 (very much) 41 2.58 3.00 3.00 .79 Involvement in fire emergency response No 12 29.3% NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 59 Num￾ber Ratio % Mean Me￾dian Mode Standard De￾viation Yes 29 70.7% Total 41 100.0% Involvement in flood emergency response No 23 56.1% Yes 18 43.9% Total 41 100.0% Involvement in avalanche emergency response No 32 86.5% Yes 5 13.5% Total 37 100.0% Involvement in landslide emergency response No 28 71.8% Yes 11 28.2% Total 39 100.0% Involvement in other type of emergency responses No 22 62.9% Yes 13 37.1% Total 35 100.0% Table 3 Post-training (CADRE Trainees) Num￾ber Mean Me￾dian Mode Standard Devi￾ation Extent to which trainees think the training were useful for their work - Scale from 1 (very little) to 4 (very much) 41 3.54 4.00 4.00 .71 Extent to which trainees think topics covered in the training are relevant for their work or actions in the com￾munity - Scale from 1 (very little) to 4 (very much) 41 3.24 3.00 3.00 .80 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 60 Extent to which trainees think trainers were knowledgeable about field conditions - Scale from 1 (very little) to 4 (very much) 41 3.59 4.00 4.00 .55 Extent to which the balance between theory and practice met trainees’ expectations - Scale from 1 (very little) to 4 (very much) 41 3.51 4.00 4.00 .71 Table 4 General Appreciation of Training (CADRE Trainees) Num￾ber Mean Me￾dian Mode Standard De￾viation Trainees’ own appreciation of skills retention in terms of “Common hazards, securing family and preparedness” - Scale from 1 (very little) to 4 (very much) 41 4 4 4 1 Basic Life support (CAB) and first aid. Trainees’ own appreciation of skills retention in terms of “Basic Life support (CAB) and first aid” - Scale from 1 (very little) to 4 (very much) 41 3.61 4.00 4.00 .59 Trainees’ own appreciation of skills retention in terms of “Incident command system and triage (CQRST)” - Scale from 1 (very little) to 4 (very much) 41 3.15 3.00 4.00 .79 Trainees’ own appreciation of skills retention in terms of “Dead Body Management” - Scale from 1 (very little) to 4 (very much) 41 2.88 3.00 3.00 1.00 Trainees’ own appreciation of skills retention in terms of “Fire Safety” - Scale from 1 (very little) to 4 (very much) 41 3.44 4.00 4.00 .63 Trainees’ own appreciation of skills retention in terms of “Basic search and rescue techniques (Load lifting and search Methods)” - Scale from 1 (very little) to 4 (very much) 41 3.27 3.00 3.00 .71 Trainees’ own appreciation of skills retention in terms of “Water emergencies, floatation responder” - Scale from 1 (very little) to 4 (very much) 41 2.95 3.00 3.00 .95 Trainees’ own appreciation of skills retention in terms of “Role as a community first responder” - Scale from 1 (very little) to 4 (very much) 41 3.54 4.00 4.00 .60 Table 5 Trainees’ Own Appreciation of Skills Retention (CADRE Trainees) NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 61 Results of Structured Interviews – CSSR Trainees Number Ratio % Gender Male 95 99.0% Female 1 1.0% Total 96 100.0% Social groups Dalit 10 10.4% Adibasi/Janajati 13 13.5% Religious Minorities 15 15.6% Disadvantaged Terai Caste Group (Pichadda Barga) 0 0.0% Others (e.g. Bahun, Chhetri (both Terai and Hill), Thakuri, Sanyasi, Rana, Shaha, Raj￾put) 58 60.4% Total 96 100% Other trainings received No 54 56.3% Yes 42 43.8% Total 96 100.0% Table 6 Identification and Background (CSSR Trainees) Num￾ber Ratio % Mean Me￾dian Mode Standard De￾viation CSSR training helped to better understand the role you would play in case of emergencies No 1 1.0% Yes 95 99.0% Total 96 100.0% CSSR training helped to better understand the chain of command in an emergency No 4 4.2% Yes 92 95.8% NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 62 Num￾ber Ratio % Mean Me￾dian Mode Standard De￾viation Total 96 100.0% CSSR training helped to better understand search and rescue technique No 0 0.0% Yes 96 100.0% Total 96 100.0% CSSR training helped to better understand the way to manage a rescue operation No 2 2.1% Yes 94 97.9% Total 96 100.0% Extent to which understanding of the way to respond (actions to take) improved - Scale from 1 (very little) to 4 (very much) 96 3.39 3.50 4.00 .70 Extent to which understanding of how to use equipment improved - Scale from 1 (very little) to 4 (very much) 96 3.53 4.00 4.00 .68 Extent to which understanding of the role to play improved - Scale from 1 (very little) to 4 (very much) 96 3.40 4.00 4.00 .75 Extent to which understanding of the relationship with community and other emergency responders improved - Scale from 1 (very little) to 4 (very much) 96 3.39 4.00 4.00 .77 Table 7 Training Session (CSSR Trainees) Num￾ber Ratio % Mean Me￾dian Mode Standard De￾viation What was learnt in the training in terms of organizing disaster response was put into practice No 16 16.7% Yes 80 83.3% Total 96 100.0% No 9 9.4% NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 63 Num￾ber Ratio % Mean Me￾dian Mode Standard De￾viation What was learnt in the training in terms of advising community members what to do was put into practice Yes 87 90.6% Total 96 100.0% What was learnt in the training in terms of speaking with communities and authorities was put into practice No 2 2.1% Yes 93 97.9% Total 95 100.0% Involvement in Gorkha earthquake emergency response No 11 11.5% Yes 85 88.5% Total 96 100.0% Involvement in emergency response for road accidents No 29 30.2% Yes 67 69.8% Total 96 100.0% Involvement in fire emergency response No 14 14.6% Yes 82 85.4% Total 96 100.0% Involvement in flood emergency response No 32 33.3% Yes 64 66.7% Total 96 100.0% Involvement in avalanches emergency response No 71 75.5% Yes 23 24.5% Total 94 100.0% NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 64 Num￾ber Ratio % Mean Me￾dian Mode Standard De￾viation Involvement in landslide emergency response No 33 34.4% 1 Yes 63 65.6% Total 96 100.0% Involvement in other type of emergency responses No 80 83.3% Yes 16 16.7% Total 96 100.0% Extent to which skills acquired during the training were used during/after Gorkha earthquake emergency or other emergencies - Scale from 1 (very little) to 4 (very much) 96 3.27 3.00 4.00 .81 Extent to which equipment could be used effectively during/after the Gorkha earthquake or other emer￾gencies - Scale from 1 (very little) to 4 (very much) 96 3.24 3.50 4.00 .93 Extent to which communities were open to CSSR trainees’ actions and instructions during/after the Gorkha earthquake or other emergencies - Scale from 1 (very little) to 4 (very much) 96 3.11 3.00 4.00 .92 Table 8 Post-training (CSSR Trainees) Count Mean Median Mode Standard Devia￾tion Column N % Extent to which training was useful for search and rescue works - Scale from 1 (very little) to 4 (very much) 96 3.60 4.00 4.00 .62 Extent to which training was relevant for actions in emergencies - Scale from 1 (very little) to 4 (very much) 96 3.49 4.00 4.00 .68 Extent to which trainers were knowledgeable on emergency response practices - Scale from 1 (very little) to 4 (very much) 96 3.49 4.00 4.00 .70 Extent to which the balance between theory and practice met the trainees’ expectations - Scale from 1 (very little) to 4 (very much) 96 3.23 3.00 3.00 .76 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 65 Repetition/refreshment of practical skill training in search and rescue by trainees’ institution No 22 22.9% Yes 74 77.1% Total 96 100.0% Extent to which trainees think refreshment trainings improved their skills - Scale from 1 (very little) to 4 (very much) 96 3.41 3.50 4.00 .68 Table 9 General Appreciation of Training (CSSR Trainees) Count Mean Median Mode Standard Deviation Trainees’ own appreciation of skills retention in terms of “Construction materials, struc￾tures and damage types” - Scale from 1 (very little) to 4 (very much) 96 3.42 3.00 4.00 .63 Trainees’ own appreciation of skills retention in terms of “Structural triage and the INSARAG marking system” - Scale from 1 (very little) to 4 (very much) 96 3.42 4.00 4.00 .75 Trainees’ own appreciation of skills retention in terms of “Operational safety triage” - Scale from 1 (very little) to 4 (very much) 96 3.44 4.00 4.00 .65 Trainees’ own appreciation of skills retention in terms of “Search and location tech￾niques” - Scale from 1 (very little) to 4 (very much) 96 3.42 3.00 4.00 .63 Trainees’ own appreciation of skills retention in terms of “Tools, equipment and accesso￾ries” - Scale from 1 (very little) to 4 (very much) 96 3.38 3.00 4.00 .68 Trainees’ own appreciation of skills retention in terms of “Rescue strategies and tech￾niques” - Scale from 1 (very little) to 4 (very much) 96 3.43 4.00 4.00 .64 Trainees’ own appreciation of skills retention in terms of “Shoring/Support methods” - Scale from 1 (very little) to 4 (very much) 96 3.44 4.00 4.00 .66 Trainees’ own appreciation of skills retention in terms of “Lifting and stabilizing loads” - Scale from 1 (very little) to 4 (very much) 96 3.51 4.00 4.00 .62 Trainees’ own appreciation of skills retention in terms of “Pre-hospital treatment” - Scale from 1 (very little) to 4 (very much) 96 3.07 3.00 4.00 .90 Table 10 Trainees’ Own Appreciation of Skills Retention (CSSR Trainees) NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 66 Results of Structured Interviews – HOPE Trainees Number Ratio % Gender Male 17 34.7% Female 32 65.3% Social groups Dalit 1 2.1% Adibasi/Janajati 24 50.0% Religious Minorities 0 0.0% Disadvantaged Terai Caste Group (Pichadda Barga) 0 0.0% Others (Bahun, Chhetri (both Terai and Hill), Thakuri, Sanyasi, Rana, Shaha, Rajput) 23 47.9% Other trainings received No 38 77.6% Yes 11 22.4% Total 49 100.0% Table 11 Identification and Background (HOPE Trainees) Number Ratio % Ways of finding out about training Organized by employer 43 87.8% Colleagues 1 2.0% Others 5 10.2% Total 49 100.0% Training helped to better understand Hospital risk management No 1 2.0% Yes 48 98.0% NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 67 HOPE training helped to better understand the role to play in case of emergencies No 1 2.0% Yes 48 98.0% HOPE training helped to better understand the chain of command in mass casualty management team No 1 2.0% Yes 48 98.0% HOPE training helped to better understand the basic elements of mass casualty management No 1 2.0% Yes 48 98.0% HOPE training helped to better understand hospital preparedness plan for emergency management No 1 2.0% Yes 48 98.0% Total 49 100.0% Table 12 Training Session (HOPE Trainees) Num￾ber Ratio % Mean Me￾dian Mode Standard Deviation What was learnt in the training in terms of organizing medical response to disaster was put into practice No 8 16.3% Yes 41 83.7% Total 49 100.0% What was learnt in the training in terms of advising medical personnel on what to do was put into practice No 6 12.2% Yes 43 87.8% Total 49 100.0% What was learnt in the training in terms of activating HICS (Hospital Incident Command Sys￾tem) was put into practice 0 No 15 30.6% Yes 34 69.4% Total 49 100.0% Training was helpful when speaking with patients and hospital management 0 No 4 8.2% NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 68 Num￾ber Ratio % Mean Me￾dian Mode Standard Deviation Yes 45 91.8% Total 49 100.0% Involvement in emergency response for Gorkha Earthquake No 3 6.1% Yes 46 93.9% Total 49 100.0% Extent to which skills acquired in training were used in emergency response for Gorkha Earthquake - Scale from 1 (very little) to 4 (very much) 49 3.26 3.00 3.00 .65 Extent to which training improved communications and coordination within hospital in responding to Gorkha Earthquake emergency - Scale from 1 (very little) to 4 (very much) 49 3.20 3.00 3.00 .72 Extent to which training improved medical personnel’s understanding of what to do in such emergencies - Scale from 1 (very little) to 4 (very much) 49 3.09 3.00 3.00 .72 Extent to which staff was open to your actions and suggestions in Gorka earthquake emergency re￾sponse - Scale from 1 (very little) to 4 (very much) 49 3.13 3.00 3.00 .83 Extent to which management was more aware of emergency preparedness and mass casualty response in Gorka earthquake emergency response - Scale from 1 (very little) to 4 (very much) 49 3.11 3.00 3.00 .71 Involvement in emergency response for road accidents No 14 28.6% Yes 35 71.4% Total 49 100.0% Involvement in emergency response for fire 0 No 39 79.6% 1 Yes 10 20.4% Total 49 100.0% Involvement in emergency response for floods 0 No 36 73.5% NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 69 Num￾ber Ratio % Mean Me￾dian Mode Standard Deviation 1 Yes 13 26.5% Total 49 100.0% Involvement in emergency response for avalanches 0 No 39 79.6% 1 Yes 10 20.4% Total 49 100.0% Involvement in emergency response for landslides 0 No 34 69.4% 1 Yes 15 30.6% Total 49 100.0% Involvement in emergency response for epidemics 0 No 36 73.5% 1 Yes 13 26.5% Total 49 100.0% Involvement in other type of emergency responses 0 No 40 81.6% 1 Yes 9 18.4% Total 49 100.0% Extent to which skills acquired in training were used in other emergencies 49 3.20 3.00 3.00 .68 Extent to which training improved communications and coordination within hospital in responding to other emergencies - Scale from 1 (very little) to 4 (very much) 49 3.18 3.00 3.00 .63 Extent to which medical personnel have a greater understanding of what to in other emergency re￾sponses - Scale from 1 (very little) to 4 (very much) 49 3.06 3.00 3.00 .75 Extent to which staff was open to actions and suggestions in other emergency responses - Scale from 1 (very little) to 4 (very much) 49 3.22 3.00 3.00 .71 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 70 Num￾ber Ratio % Mean Me￾dian Mode Standard Deviation Extent to which management was more aware of emergency preparedness and mass casualty response in other emergency responses - Scale from 1 (very little) to 4 (very much) 49 3.10 3.00 3.00 .65 Table 13 Post-training (HOPE Trainees) Num￾ber Mean Me￾dian Mode Standard Devi￾ation Ratio % Extent to which training was useful in hospital emergency response work - Scale from 1 (very little) to 4 (very much) 49 3.43 3.00 3.00 .58 Extent to which training topics were relevant for work and actions in hospitals - Scale from 1 (very little) to 4 (very much) 49 3.18 3.00 3.00 .60 Extent to which instructors were knowledgeable about mass casualty response practices - Scale from 1 (very little) to 4 (very much) 49 3.31 3.00 3.00 .65 Extent to which the balance between theory and practice met trainees’ expectations - Scale from 1 (very little) to 4 (very much) 49 2.98 3.00 3.00 .63 Institution repeated drill /reviewed the hospital preparedness plan No 5 10.2% Yes 43 87.8% Don’t know 1 2.0% Table 14 General Appreciation of Training (HOPE Trainees) NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 71 Results of Structured Interviews – MFR Trainees Count Column N % Gender Male 41 95.3% Female 2 4.7% Total 43 100.0% Social groups Dalit 4 9.3% Adibasi/Janajati 13 30.2% Religious Minorities 0 0.0% Disadvantaged Terai Caste Group (Pichadda Barga) 0 0.0% Others (Bahun, Chhetri (both Terai and Hill), Thakuri, Sanyasi, Rana, Shaha, Rajput) 26 60.5% Total 43 100.0% Similar training(s) taken in the past No 24 55.8% Yes 19 44.2% Total 43 100.0% Table 15 Identification and Background (HOPE Trainees) Count Mean Me￾dian Mode Standard Devia￾tion Extent to which training helped to better understand the role to play in case of emergencies - Scale from 1 (very little) to 4 (very much) 43 3.37 3.00 4.00 .72 Extent to which training helped to better understand the chain of command in emergency situations - Scale from 1 (very little) to 4 (very much) 43 3.28 3.00 3.00 .73 Extent to which training helped to better understand the basic elements of first aid treatment /medical re￾sponse in emergencies - Scale from 1 (very little) to 4 (very much) 43 3.30 3.00 3.00 .67 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 72 Extent to which training helped to better understand specific medical interventions - Scale from 1 (very little) to 4 (very much) 43 2.77 3.00 3.00 .84 Extent to which training helped to better understand the principles of triage - Scale from 1 (very little) to 4 (very much) 43 3.40 4.00 4.00 .79 Extent to which understanding improved on action to be taken in emergencies - Scale from 1 (very little) to 4 (very much) 43 3.33 3.00 3.00 .68 Extent to which understanding improved on principles of stabilizing the condition of injured - Scale from 1 (very little) to 4 (very much) 43 3.42 3.00 4.00 .66 Extent to which understanding improved on the importance use of equipment - Scale from 1 (very little) to 4 (very much) 43 3.33 3.00 4.00 .78 Extent to which understanding improved on the role to play - Scale from 1 (very little) to 4 (very much) 43 3.51 4.00 4.00 .67 Extent to which understanding improved on the relationship with medical first responders in your organiza￾tion - Scale from 1 (very little) to 4 (very much) 43 3.21 3.00 3.00 .67 Extent to which understanding improved on the relationships with other/external emergency responders - Scale from 1 (very little) to 4 (very much) 43 3.12 3.00 3.00 .59 Table 16 Training Session (MFR Trainees) Count Mean Me￾dian Mode Standard De￾viation Extent to which what was learnt in the training in terms of organizing medical first response to disasters was put into practice - Scale from 1 (very little) to 4 (very much) 43 3.12 3.00 3.00 .73 Extent to which what was learnt in the training was shared in your organization was put into practice - Scale from 1 (very little) to 4 (very much) 43 3.65 4.00 4.00 .57 Extent to which training was helpful to speaking to affected people and leaders in the community - Scale from 1 (very little) to 4 (very much) 43 3.33 3.00 3.00 .68 Extent to which training was helpful to increase confidence in implementing MFR - Scale from 1 (very little) to 4 (very much) 43 3.63 4.00 4.00 .66 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 73 Count Mean Me￾dian Mode Standard De￾viation Extent to which skills acquired in training were used during and after Gorkha earthquake - Scale from 1 (very little) to 4 (very much) 43 3.07 3.00 4.00 1.06 Extent to which training improved communications and coordination with other emergency responders, during and after Gorkha earthquake - Scale from 1 (very little) to 4 (very much) 43 3.21 3.00 3.00 .86 Extent to which MFR personnel had a good understanding of what to do in such emergencies during and after Gorkha earthquake - Scale from 1 (very little) to 4 (very much) 43 3.28 3.00 4.00 .85 Extent to which colleagues and communities were open to action and suggestion during and after Gorkha earth￾quake - Scale from 1 (very little) to 4 (very much) 43 3.02 3.00 3.00 .91 Extent to which organization/institution was more aware of MFR during and after Gorkha earthquake - Scale from 1 (very little) to 4 (very much) 43 3.07 3.00 3.00 .83 Table 17 Post-training (MFR Trainees) Num￾ber Mean Median Mode Standard Deviation Extent to which training was useful in trainees’ work - Scale from 1 (very little) to 4 (very much) 43 3.37 3.00 4.00 .72 Extent to which sessions covered by training were relevant for trainees’ work and actions during emer￾gencies - Scale from 1 (very little) to 4 (very much) 43 3.42 4.00 4.00 .66 Extent to which instructors were knowledgeable about MFR practices - Scale from 1 (very little) to 4 (very much) 43 3.49 4.00 4.00 .59 Extent to which the balance between theory and practice met trainees’ expectations - Scale from 1 (very little) to 4 (very much) 43 3.35 3.00 3.00 .61 General appreciation of training 43 3.41 3.50 4.00 .52 Table 18 General Appreciation of Training (MFR Trainees) NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 74 Count Ratio % Mean Median Mode Standard Deviation Repetition/refreshment of practi￾cal skills training in MFR by train￾ees’ institutions No 17 39.5% Yes 26 60.5% Trainees’ own appreciation on how well they did at the repetition/refreshment of practical skills training 3.62 4.00 4.00 .70 Table 19 Repetition/Refreshment of Practical Skills Training (MFR Trainees) Count Mean Median Mode Standard Deviation Trainees’ own appreciation of skills retention in terms of “Patient assessment and check vital sign” - Scale from 1 (very little) to 4 (very much) 43 3.51 4.00 4.00 .51 Trainees’ own appreciation of skills retention in terms of “BLS and CPR” - Scale from 1 (very little) to 4 (very much) 43 3.56 4.00 4.00 .67 Trainees’ own appreciation of skills retention in terms of “BLS and FBAO” - Scale from 1 (very little) to 4 (very much) 43 3.28 3.00 3.00 .73 Trainees’ own appreciation of skills retention in terms of “Oxygen Therapy” - Scale from 1 (very little) to 4 (very much) 43 3.14 3.00 3.00 .77 Trainees’ own appreciation of skills retention in terms of “Fracture and fail test” - Scale from 1 (very little) to 4 (very much) 43 3.42 3.00 4.00 .63 Trainees’ own appreciation of skills retention in terms of “Bleeding control and shocking chest wounds” - Scale from 1 (very little) to 4 (very much) 43 3.23 3.00 3.00 .78 Trainees’ own appreciation of skills retention in terms of “Child Birth” - Scale from 1 (very little) to 4 (very much) 43 2.70 3.00 2.00 .83 Trainees’ own appreciation of skills retention in terms of “Lifting and Moving pa￾tient” - Scale from 1 (very little) to 4 (very much) 43 3.44 3.00 4.00 .59 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 75 Trainees’ own appreciation of skills retention in terms of “Triage” - Scale from 1 (very little) to 4 (very much) 43 3.14 3.00 3.00 .64 Overall trainees’ own appreciation of skills retention - Scale from 1 (very little) to 4 (very much) 43 3.27 3.22 3.22 .46 Table 20 Trainees’ Own Appreciation of Skills Retention (MFR Trainees) NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 76 BCIPN – Quantitative Data (Results of Structured Interviews) Table 21 Identification and Background (Engineers & Sub-engineers)........................................................................................................................... 77 Table 22 Knowledge Gained from BCIPN Trainings (Engineers & Sub-engineers)....................................................................................................... 79 Table 3 Skills Improvement from BCIPN Trainings (Engineers & Sub-engineers) ........................................................................................................ 79 Table 24 Use of Skills in Earthquake Resistant Building Techniques after BCIPN Trainings (Engineers & Sub-engineers)............................................ 80 Table 25 Perception of Population on Building Earthquake Resistant Structures (Engineers & Sub-engineers) ............................................................. 80 Table 26 Major Barriers Ranking for NBC Enforcement (Engineers & Sub-engineers) ................................................................................................. 81 Table 27 Professional Benefits resulting from BCIPN training (Engineers & Sub-engineers)......................................................................................... 81 Table 28 Professional Development following BCIPN training (Engineers & Sub-engineers) ....................................................................................... 82 Table 29 Appreciation of BCIPN Training (Engineers & Sub-engineers) ...................................................................................................................... 82 Table 30 Identification and Background (Masons)......................................................................................................................................................... 83 Table 31 Knowledge Gained from BCIPN Trainings (Masons)..................................................................................................................................... 84 Table 32 Post-training (Masons).................................................................................................................................................................................... 85 Table 33 Professional Benefits (Masons) ....................................................................................................................................................................... 86 Table 34 Appreciation of BCIPN Training (Masons)..................................................................................................................................................... 87 Table 35 Identification and Background (Municipality staff).......................................................................................................................................... 88 Table 36 Knowledge Gained through BCIPN (Municipality staff)................................................................................................................................. 89 Table 37 Post training (Municipality staff) ..................................................................................................................................................................... 90 Table 38 Professional Benefits (Municipality staff) ........................................................................................................................................................ 91 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 77 Results of Structured Interviews – Engineers & Sub-engineers Number Ratio % Gender Male 89 100.0% Female 0 0.0% Total 89 100.0% Social groups Dalit 6 6.7% Adibasi/Janajati 36 40.4% Religious Minorities 3 3.4% Disadvantaged Terai Caste Group (Pichadda Barga) 5 5.6% Others (Bahun, Chhetri (both Terai and Hill), Thakuri, Sanyasi, Rana, Shaha, Rajput) 39 43.8% Total 89 100.0% Any similar training received. No 62 69.7% Yes 27 30.3% Employment affiliation Self Employed 26 29.2% Salaried in small firm 3 3.4% Salaried in large firm 3 3.4% Others 57 64.0% Table 21 Identification and Background (Engineers & Sub-engineers) NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 78 Num￾ber Ratio % Ways of finding out about training NSET 7 7.9% Municipality 75 84.3% From organization/company 3 3.4% Colleagues mentioned 0 0.0% Social media 0 0.0% Others 4 4.5% Total 89 100.0% BCIPN training helped to better understand the engineering parameters of risk reduction No 2 2.2% Yes 87 97.8% Total 89 100.0% BCIPN training helped to better understand the engineering parameters of risk reduction the usefulness of the building code No 1 1.1% Yes 88 98.9% Total 89 100.0% BCIPN training helped to better understand technical knowhow on provisions enlisted in different sections of building code No 2 2.2% Yes 87 97.8% Total 89 100.0% BCIPN training helped to better understand the use of building code based checklist No 5 5.6% Yes 84 94.4% Total 89 100.0% No 6 6.7% NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 79 Num￾ber Ratio % BCIPN training helped to better understand the usefulness of building code checklists for the building code imple￾mentation Yes 83 93.3% Total 89 100.0% BCIPN training helped to better understand how to use commercial structural analysis software No 45 50.6% Yes 44 49.4% Total 89 100.0% Table 22 Knowledge Gained from BCIPN Trainings (Engineers & Sub-engineers) Count Mean Median Standard Deviation Extent to which trainees think BCIPN training improved skills in earthquake resistant building techniques with regard to designing parameters – Scale from 1 (very little) to 4 (very much) 89 2.93 3.00 .74 Extent to which trainees think BCIPN training improved skills in earthquake resistant building techniques with regard to selecting of construction materials – Scale from 1 (very little) to 4 (very much) 89 3.21 3.00 .82 Extent to which trainees think BCIPN training improved skills in earthquake resistant building techniques with regard to quality control – Scale from 1 (very little) to 4 (very much) 89 3.28 3.00 .71 Extent to which trainees think BCIPN training improved skills in earthquake resistant building techniques with regard to construction detailing – Scale from 1 (very little) to 4 (very much) 89 3.31 3.00 .78 Extent to which trainees think BCIPN training improved overall skills in earthquake resistant building techniques – Scale from 1 (very little) to 4 (very much) 89 3.19 3.00 .60 Table 23 Skills Improvement from BCIPN Trainings (Engineers & Sub-engineers) NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 80 Count Mean Me￾dian Mode Standard De￾viation Extent to which trainees think that skills acquired during the training in terms of design parameter were used – Scale from 1 (very little) to 4 (very much) 89 3.06 3.00 3.00 .87 Extent to which trainees think that skills acquired during the training in terms of selection of construction material were used – Scale from 1 (very little) to 4 (very much) 89 3.29 3.00 4.00 .77 Extent to which trainees think that skills acquired during the training in terms of quality control were used – Scale from 1 (very little) to 4 (very much) 89 3.43 4.00 4.00 .67 Extent to which trainees think that skills acquired during the training in terms of construction detailing were used – Scale from 1 (very little) to 4 (very much) 89 3.35 3.00 4.00 .72 Extent to which trainees think that skills acquired during the training in different earthquake resistant building tech￾niques were used – Scale from 1 (very little) to 4 (very much) 89 3.28 3.25 4.00 .64 Table 24 Use of Skills in Earthquake Resistant Building Techniques after BCIPN Trainings (Engineers & Sub-engineers) Count Mean Me￾dian Standard Devi￾ation Extent to which clients were open to adhering to the building code after the Gorkha earthquake – Scale from 1 (very lit￾tle) to 4 (very much) 89 3.01 3.00 .76 Extent to which earthquake resistant elements in new and old building were demanded after the Gorkha earthquake – Scale from 1 (very little) to 4 (very much) 89 2.75 3.00 .92 Extent to which general population was more aware of earthquake risk reduction practices after the Gorkha earthquake – Scale from 1 (very little) to 4 (very much) 89 2.91 3.00 .79 Overall anticipated population perception score on building earthquake resistant structures – Scale from 1 (very little) to 4 (very much) 89 2.89 3.00 .67 Table 25 Perception of Population on Building Earthquake Resistant Structures (Engineers & Sub-engineers) NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 81 Count Mean Me￾dian Standard Deviation Extent to which lack of knowledge of NBC in Building Permit System was a major barrier for NBC enforcement – Scale from 1 (very little) to 4 (very much) 89 3.07 3.00 .91 Extent to which capacity issues of municipalities were major barriers for NBC enforcement – Scale from 1 (very little) to 4 (very much) 89 2.84 3.00 .90 Extent to which unavailability of engineers was a major barrier for NBC enforcement – Scale from 1 (very little) to 4 (very much) 89 2.79 3.00 .91 Extent to which capacity issues of local engineers were major barriers for NBC enforcement – Scale from 1 (very little) to 4 (very much) 89 2.82 3.00 .85 Extent to which low incentives to adopt NBC (e.g. not required by insurance, for loan, etc.) were major barriers for NBC en￾forcement – Scale from 1 (very little) to 4 (very much) 89 2.81 3.00 1.00 Extent to which higher cost of construction was a major barrier for NBC enforcement – Scale from 1 (very little) to 4 (very much) 89 2.90 3.00 .88 Extent to which low demand by house owner was a major barrier for NBC enforcement – Scale from 1 (very little) to 4 (very much) 89 2.91 3.00 .91 Table 26 Major Barriers Ranking for NBC Enforcement (Engineers & Sub-engineers) Count Mean Median Standard Deviation Extent to which training helped to get additional financial benefits – Scale from 1 (very little) to 4 (very much) 89 2.33 2.00 .89 Extent to which training helped to get job promotion – Scale from 1 (very little) to 4 (very much) 89 2.37 2.00 .95 Table 27 Professional Benefits resulting from BCIPN training (Engineers & Sub-engineers) Count Mean Me￾dian Standard Devia￾tion Extent to which BCIPN training increased demand of trainees as designers – Scale from 1 (very little) to 4 (very much) 89 2.30 2.00 .96 Extent to which BCIPN training increased demand of trainees as trainers – Scale from 1 (very little) to 4 (very much) 89 2.21 2.00 1.06 NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 82 Extent to which BCIPN training increased demand of trainees as supervisors – Scale from 1 (very little) to 4 (very much) 89 2.83 3.00 .87 Extent to which BCIPN training increased efficiency in carrying out job – Scale from 1 (very little) to 4 (very much) 89 3.35 3.00 .68 Table 28 Professional Development following BCIPN training (Engineers & Sub-engineers) Count Mean Me￾dian Standard Devia￾tion Extent to which training was useful for engineers’ sub-engineers’ job – Scale from 1 (very little) to 4 (very much) 89 3.60 4.00 .62 Extent to which topics covered in training were relevant for work – Scale from 1 (very little) to 4 (very much) 89 3.64 4.00 .57 Extent to which trainers were knowledgeable about field conditions – Scale from 1 (very little) to 4 (very much) 89 3.54 4.00 .69 Extent to which the balance between theory and practice met trainees’ expectations – Scale from 1 (very little) to 4 (very much) 89 3.38 4.00 .79 Overall score of training appreciation – Scale from 1 (very little) to 4 (very much) 89 3.54 3.75 .54 Table 29 Appreciation of BCIPN Training (Engineers & Sub-engineers) Results of Structured Interviews – Masons Number Ratio % Gender Male 247 100.0% Female 0 0.0% Total 247 100.0% Social groups Dalit 47 24.6% Adibasi/Janajati 101 52.9% Religious Minorities 5 2.6% NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 83 Disadvantaged Terai Caste Group (Pichadda Barga) 9 4.7% Others (Bahun, Chhetri (both Terai and Hill), Thakuri, Sanyasi, Rana, Shaha, Rajput) 29 15.2% Total 191 100.0% Place of training Municipality 244 98.8% Village 3 1.2% Total 247 100.0% Similar training taken? No 217 87.9% Yes 30 12.1% Total 247 100.0% Employment affiliation Self-employed mason 31 12.6% Local contractor 128 52.0% Worker for contractor 81 32.9% Other 6 2.4% Total 246 100.0% Table 30 Identification and Background (Masons) Num￾ber Ratio % Mean Mode Me￾dian Standard De￾viation Ways of finding out about training NSET 9 3.6% Municipality 138 55.9% Trainee’ s organization/company 54 21.9% Colleagues 13 5.3% Social media 5 2.0% NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 84 Num￾ber Ratio % Mean Mode Me￾dian Standard De￾viation Others 27 10.9% Not mentioned 1 .4% Total 247 100.0% Training helped to better understand earthquake resistant building form/shape Yes 246 99.6% No 1 .4% Total 247 100.0% Training helped to better understand earthquake resistant con￾struction techniques No 2 .8% Yes 245 99.2% Total 247 100.0% Extent to which training helped to understand selection of construction materials – Scale from 1 (very little) to 4 (very much) 3.66 4.00 4.00 .56 Extent to which training helped to understand form/shape of building – Scale from 1 (very little) to 4 (very much) 3.70 4.00 4.00 .58 Extent to which training helped to understand construction detailing – Scale from 1 (very little) to 4 (very much) 3.73 4.00 4.00 .55 Extent to which training helped to understand repair and maintenance of existing building – Scale from 1 (very little) to 4 (very much) 3.02 4.00 3.00 1.13 Extent to which training helped to understand to make earthquake resistant building – Scale from 1 (very little) to 4 (very much) 3.53 4.00 3.75 .53 Table 31 Knowledge Gained from BCIPN Trainings (Masons) NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 85 Num￾ber Ratio % Mean Mode Me￾dian Standard De￾viation Skills acquired during the training in terms of selection of construction materials were used No 5 2.0% Yes 242 98.0% Total 247 100.0% Skills acquired during the training in terms of construction of new building were used No 4 1.6% Yes 243 98.4% Total 247 100.0% Skills acquired during the training in terms of repair and maintenance of existing building were used No 93 37.7% Yes 154 62.3% Total 247 100.0% Usefulness of new skills and knowledge acquired during training when discussing a contract No 34 13.8% Yes 213 86.2% Total 247 100.0% Extent to which house owners were open to apply the building code after Gorkha earthquake – Scale from 1 (very little) to 4 (very much) 3.23 4.00 3.00 .84 Extent to which earthquake resistant techniques for buildings were demanded after Gorkha earth￾quake – Scale from 1 (very little) to 4 (very much) 3.27 4.00 3.00 .83 Extent to which house owners were aware of earthquake resistant building after Gorkha earthquake – Scale from 1 (very little) to 4 (very much) 3.17 4.00 3.00 .95 Table 32 Post-training (Masons) NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 86 Num￾ber Ratio % Mean Mode Me￾dian Standard Devia￾tion Training helped to get higher wages No 77 31.2% Yes 170 68.8% Total 247 100.0% Training helped to get more masonry works No 26 10.5% Yes 221 89.5% Total 247 100.0% Training helped to get work satisfaction No 9 3.6% Yes 238 96.4% Total 247 100.0% Extent to which training increased demand of trained masons – Scale from 1 (very little) to 4 (very much) 3.50 4.00 4.00 .75 Extent to which training increased efficiency in carrying out masonry works – Scale from 1 (very little) to 4 (very much) 3.62 4.00 4.00 .61 Extent to which training increased number of contracts for masons – Scale from 1 (very little) to 4 (very much) 3.31 4.00 4.00 .86 Extent to which training helped to get higher wages – Scale from 1 (very little) to 4 (very much) 2.95 4.00 3.00 1.06 Table 33 Professional Benefits (Masons) NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 87 Mean Mode Me￾dian Standard Devia￾tion Extent to which training was useful for masonry works – Scale from 1 (very little) to 4 (very much) 3.74 4.00 4.00 .52 Extent to which technologies covered in training were useful for masonry works – Scale from 1 (very little) to 4 (very much) 3.68 4.00 4.00 .57 Extent to which trainers were knowledgeable about field conditions – Scale from 1 (very little) to 4 (very much) 3.80 4.00 4.00 .49 Extent to which the balance between theory and practice met trainees’ expectations – Scale from 1 (very little) to 4 (very much) 3.88 4.00 4.00 .42 General Appreciation of training – Scale from 1 (very little) to 4 (very much) 3.78 4.00 4.00 .39 Table 34 Appreciation of BCIPN Training (Masons) Results of Structured Interviews – Municipality staff Number Ratio % Gender Male 25 29.1% Female 61 70.9% Total 86 100.0% Social groups Dalit 13 15.1% Adibasi/Janajati 9 10.5% Religious Minorities 0 0.0% Disadvantaged Terai Caste Group (Pichadda Barga) 2 2.3% Others (Bahun, Chhetri (both Terai and Hill), Thakuri, Sanyasi, Rana, Shaha, Rajput) 62 72.1% Total 86 100.0% Similar training received? No 72 83.7% Yes 14 16.3% NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 88 Total 86 100.0% Roles in municipal administration Senior administrative staff 4 4.7% Junior administrative staff 6 7.0% Technician 1 1.2% Clerical support 2 2.3% Other 73 84.9% Total 86 100.0% Table 35 Identification and Background (Municipality staff) Number Ratio % Mean Median Mode Standard De￾viation Training received through Municipal office 72 83.7% Government 0 0.0% Others 14 16.3% Total 86 100.0% Training helped to better understand earthquake risk management No 3 3.5% Yes 83 96.5% Total 86 100.0% Training helped to better understand the usefulness of building code No 7 8.1% Yes 79 91.9% Total 86 100.0% No 10 11.6% Yes 76 88.4% NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 89 Training helped to better understand the usefulness of the building permit system Total 86 100.0% Training helped to better understand the role of municipal authorities in dis￾aster preparedness No 9 10.5% Yes 77 89.5% Total 86 100.0% Extent to which understanding of earthquake caused improved – Scale from 1 (very little) to 4 (very much) 86 2.83 3.00 3.00 .96 Extent to which understanding of disaster preparedness improved – Scale from 1 (very little) to 4 (very much) 86 2.95 3.00 3.00 .92 Extent to which understanding with regard to implementation of build￾ing code improved – Scale from 1 (very little) to 4 (very much) 86 2.55 3.00 3.00 1.01 Extent to which understanding with regard to the challenges of building code implementation improved – Scale from 1 (very little) to 4 (very much) 86 2.57 3.00 3.00 1.04 Table 36 Knowledge Gained through BCIPN (Municipality staff) Num￾ber Ratio % Mean Me￾dian Mode Standard De￾viation Use of knowledge acquired through BCIPN to support other municipal staff in building code im￾plementation No 3 3.5% Yes 83 96.5% Use of knowledge acquired through BCIPN to advise public on adopting the NBC No 2 2.3% Yes 84 97.7% Use of knowledge acquired through BCIPN to raise awareness and explain the NBC to con￾struction sector stakeholders No 27 31.4% Yes 59 68.6% NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 90 Num￾ber Ratio % Mean Me￾dian Mode Standard De￾viation Use of knowledge acquired through BCIPN when speaking to house owners and construction stakeholders No 9 10.5% Yes 77 89.5% Extent to which house owners were open to adhering to the building code after Gorkha earthquake – Scale from 1 (very little) to 4 (very much) 86 2.50 2.00 2.00 .93 Extent to which municipal authorities have a greater understanding of their role in enforcing NBC after Gorkha earthquake – Scale from 1 (very little) to 4 (very much) 86 2.66 3.00 3.00 .97 Extent to which BCIPN contributed to improve adherence to building permit system – Scale from 1 (very little) to 4 (very much) 86 2.57 3.00 3.00 .96 Extent to which BCIPN contributed to increase number of NBC complaint building permits issued – Scale from 1 (very little) to 4 (very much) 86 2.47 3.00 3.00 1.09 Extent to which BCIPN contributed to increase the number of NBC complaint buildings constructed – Scale from 1 (very little) to 4 (very much) 86 2.41 2.00 3.00 1.04 Extent to which lack of knowledge of NBC in Building permit system was a major barrier for NBC en￾forcement – Scale from 1 (very little) to 4 (very much) 86 2.81 3.00 4.00 1.10 Extent to which capacity issues of municipalities were major barriers for NBC enforcement – Scale from 1 (very little) to 4 (very much) 86 2.64 3.00 3.00 .89 Extent to which capacity issues of local professionals were major barriers for NBC enforcement – Scale from 1 (very little) to 4 (very much) 86 2.62 3.00 3.00 .94 Extent to which low incentives to adopt NBC (e.g. insurance, loan, etc.) were major barriers for NBC enforcement – Scale from 1 (very little) to 4 (very much) 86 2.74 3.00 2.00 1.04 Table 37 Post training (Municipality staff) NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 91 Count Mean Me￾dian Mode Standard Devia￾tion Extent to which training helped to get additional financial benefits – Scale from 1 (very little) to 4 (very much) 86 1.21 1.00 1.00 .58 Extent to which training helped to get a job promotion – Scale from 1 (very little) to 4 (very much) 86 1.28 1.00 1.00 .75 Extent to which training increased demands of trainees as trainers – Scale from 1 (very little) to 4 (very much) 86 1.30 1.00 1.00 .70 Extent to which training increased demand of trainees as supervisors – Scale from 1 (very little) to 4 (very much) 86 1.20 1.00 1.00 .57 Extent to which training increased efficiency in carrying out work – Scale from 1 (very little) to 4 (very much) 86 1.72 1.00 1.00 1.02 Table 38 Professional Benefits (Municipality staff) NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 92 DATABASE OF PARTICIPANT RE-TESTS, POST-TESTS AND PRE-TESTS – TABULAR FORMAT WITH T-TEST SCORES Table 39 Comparative Scores of PEER Training Participants MFR CSSR HOPE N Mean t-Stat* N Mean t-Stat* N Mean t-Stat* Re-test 43 57.9% 8.664 96 57.4% 3.060 44 65.8% 2.998 Post-test 167 73.9% 28.131 166 76.5% 18.329 28 71.9% 6.000 Pre-test 167 37.7% 166 51.7% 55 56.4% * t-Stats of post-tests against pre-tests and re-tests against pre-tests. Table 40 Re-tests of PEER Participants by Year of Training 2014 2015 2016 N Mean t-Stat* N Mean t-Stat* N Mean t-Stat* 46 60.1% 1.67 35 52.4% -2.198 24 56.8% 0.018 * t-Stats of annual means against mean score of 56.78% for all years. Table 19 Comparative Scores of Level 0 and Level 1 PEER Training Participants MFR CSSR N Mean t-Stat* N Mean t-Stat* Level 0 trainees – trained directly by OFDA Re-test 6 75.0% --- 4 61.5% --- Post-test 19 77.7% 9.404 19 73.3% 5.201 Pre-test 19 36.2% 19 45.0% Level 1 trainees – trained by instructors who were trained by OFDA Re-test 28 53.8% 11.153 43 56.6% 7.046 Post-test 148 73.4% 25.934 147 76.9% 17.068 Pre-test 148 37.9% 147 49.3% * t-Stats of post-tests against pre-tests and re-tests against pre-tests. Note: not all MFR and CSSR trainees who were re-tested could be sorted into either Level 0 or Level 1, and those who could not be sorted were left out of these calculations, so that the numbers re-tested differ from those found in Table 4. NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 93 Table 20 Pre-, Post- and Re-tests Scores of Masons and Engineers benefiting from BCIPN Masons Engineers N Mean t-Stat* N Mean t-Stat* Re-test 245 74.2% 26.630 89 63.0% 3.707 Post-test 985 74.0% 39.393 263 77.4% 14.107 Pre-test 985 47.9% 263 56.9% * t-Stats of post-tests against pre-tests and re-tests against pre-tests. NEPAL DRR EVALUATION - EVALUATION REPORT (ANNEXES) 94 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523 Tel: (202) 712-0000 Fax: (202) 216-3524 www.usaid.gov