This publication was produced at the request of the United States Agency for International Development. It was prepared independently by an independent evaluation consultant, Spencer Salime, hired by Miyamoto International, Inc. FINAL EVALUATION EBRP III MARCH 2023 FINAL EVALUATION OF EBRP III | i ABSTRACT The Ecuador Building Resilience Program (EBRP III) sought to reinforce the knowledge of people in the most vulnerable communities on topics related to safe construction methodologies and risk prevention; strengthen the technical capacity of professionals in the field of construction; contribute to the creation and implementation of public policies, ordinances and regulations that help promote safe construction; and promote, through awareness-raising campaigns, the importance of building technically and securely. The EBRP III program coverage area primarily corresponded to the rural areas of the canton of Portoviejo in the Manabí province. The program’s goal was to strengthen disaster resilience among vulnerable populations through targeted investments in capacity building and technical assistance for improved risk reduction and management. The program had a budget of $1,000,000.00 (one million dollars). It was launched on June 25, 2021, with an expected implementation period of 24 months, ending the program on June 23, 2023. The purpose of this evaluation was to measure and document the program’s outcomes and the sustainability of its impact. It includes lessons learned and strengths and challenges to inform and contribute to the improvement of planning processes for community resilience projects in Ecuador. To this end, the team carried out a participatory evaluation, which included the review of key program documents and semi-structured interviews with management personnel, local authorities, teachers and university students, and people from the communities that participated in the program’s training sessions, to gather their perceptions of the program, its relevance, effectiveness, efficiency, impact, and sustainability. The EBRP III program’s collaboration and partnerships with provincial and parochial GADs, the provincial government of Manabí, the National Risk and Emergency Management Service (SNGRE for its initials in Spanish), three of the most important local universities (ILEAM, PUCE, and USGP), the European Investment Bank (EBI), and World Vision, among others, allowed for the successful implementation of trainings, sensitization activities, and public awareness events, which have all empowered communities to manage various risks. The program managed to deliver trainings on safe construction methodologies and risk management to communities and local universities, and relevant technical courses such as “Seismic Reinforcement of Structures” and “Rapid Assessments of the Structural Vulnerability of Critical Buildings” to students and teachers from engineering and architecture schools at important local and national universities (UTM, PUCE-Manabí, PUCE￾Pichincha, UNESUM, and ULEAM); the technical teams of the cantonal GADs of Portoviejo, El Carmen, Flavio Alfaro, Manta, Pedernales, Sucre and Pichincha; the provincial governments of Esmeraldas, Manabí and Orellana; the technical teams of the cantonal GADs of Portoviejo, El Carmen, Flavio Alfaro, Manta, Pedernales, Sucre, and Pichincha; the Professional Association of Civil Engineers and the Professional Association of Architects of Manabí; as well as construction professionals in general. The program sensitized and created awareness in universities and schools in the province of Manabí on risk management and supported the implementation of local and national drills in coordination with the Risk Management National Service. The program worked in eleven municipalities of Manabí to replicate the EBRP methodology for assessing physical, social, and economic vulnerability. This helped promote DRM based on municipal evidence and the exchange of experience by leaders. As part of the multi-criteria risk analysis, the program produced flood susceptibility maps for the urban area of Portoviejo and the seven rural districts of the canton of Portoviejo and risk maps of threats such as landslides or floods, for 87 communities and seven rural districts of the canton of Portoviejo. Additionally, the program created educational material and methodologies for different training topics and technical material, such as manuals, guides, and books, to help spread and disseminate the knowledge imparted during the EBRP III program. ii | FINAL EVALUATION OF EBRP III It is worth mentioning that the program also supported bamboo artisans and builders associations created with the support of Miyamoto International during the program’s previous phase (EBRP II) As of March 2023, the EBRP III program reports the completion of 93% of its indicators and 91% of programmed activities. This report presents the details of each indicator and activity carried out. FINAL EVALUATION OF EBRP III | iii FINAL EVALUATION OF EBRP III March 2023 Award number: 720BHA21GR00139 DISCLAIMER: The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development of the United States Government. iv | FINAL EVALUATION OF EBRP III TABLE OF CONTENTS ABSTRACT.............................................................................................................................. i TABLE OF IMAGES.................................................................................................................vi TABLE OF CHARTS ...............................................................................................................vi LIST OF TABLES ....................................................................................................................vii ACRONYMS.........................................................................................................................viii EXECUTIVE SUMMARY .........................................................................................................ix EVALUATION PURPOSE...................................................................................................ix EVALUATION QUESTIONS..............................................................................................ix BACKGROUND.................................................................................................................ix EVALUATION QUESTIONS, DESIGN, METHODS, AND LIMITATIONS ........................ix FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS.............................................x EVALUATION PURPOSE AND EVALUATION QUESTIONS............................................... 1 EVALUATION PURPOSE ................................................................................................... 1 EVALUATION QUESTIONS .............................................................................................. 1 BACKGROUND..................................................................................................................... 3 EVALUATION METHODS AND LIMITATIONS ................................................................... 6 FINDINGS ............................................................................................................................ 10 OVERALL ASSESSMENT OF THE INFORMATION............................................................. 27 BEST PRACTICES ............................................................................................................. 27 LESSONS LEARNED......................................................................................................... 27 MAIN ACHIEVEMENTS ................................................................................................... 28 OPPORTUNITIES FOR IMPROVEMENT......................................................................... 29 UNEXPECTED RESULTS ................................................................................................. 30 CONCLUSIONS................................................................................................................... 32 RECOMMENDATIONS........................................................................................................ 33 ANNEXES............................................................................................................................. 34 ANNEX I: EVALUATION CALENDAR............................................................................ 34 ANNEX II: EVALUATION WORK STATEMENT............................................................. 35 ANNEX III: EVALUATION METHODS AND LIMITATIONS .......................................... 36 ANNEX IV: DATA COLLECTION TOOLS AND INTERVIEWS OF OFFICIALS............. 39 ANNEX V: SOURCES OF INFORMATION ..................................................................... 41 ANNEX VI: DECLARATION OF CONFLICTS OF INTEREST......................................... 42 ANNEX VII: MEMBERS OF THE EVALUATION TEAM.................................................... 43 ANNEX VIII: STATEMENT(S) OF DIFFERENCE .............................................................. 44 FINAL EVALUATION OF EBRP III | v ANNEX IX: PHOTOGRAPHS OF THE EVALUATION ................................................... 45 vi | FINAL EVALUATION OF EBRP III TABLE OF IMAGES Image 1 Geographic coverage map of the program...........................................................................................5 Image 2 Interview with Sandino León, Portoviejo GAD technical expert..................................................45 Image 3 Interview with Héctor Cedeño, architect, dean of the Faculty of Engineering and Architecture, ULEAM ............................................................................................................................45 Image 4 Interview with Fabricio Ponce Vélez, economist, President of Alhajuela GAD ........................46 Image 5 Interview with Wilmer Guillén, La Mocorita community leader ..................................................46 Image 6 Interview with Dolores Saltos, Alhajuela district spokesperson ...................................................46 Image 7 Quebrada de Guillén focus group.........................................................................................................47 Image 8 Community footprint in the Quebrada de Guillén community .....................................................47 Image 9 Entrance to land where guadua cane is grown. El Hormiguero community ..............................47 Image 10 Guadua cane tourist destination, Mocorita Community...............................................................48 Image 11 El Achiote community focus group ....................................................................................................48 Image 12 Focus group participants from La Mocorita Community; location: San Roque - La Esperanza community center................................................................................................................................48 Image 13 Visit to the Sembrando Esperanza Association – Pedernales......................................................49 Image 14 Treated guadua cane in the process of drying in the Production Center of the associations of artisans and bamboo builders of the Pedernales canton. ASOPROASE and ACERO VEGETAL ..............................................................................................................................................49 Image 15 Plants from the nursery of the Sembrando Esperanza Association - Pedernales....................49 TABLE OF CHARTS Chart 1 Context of the program according to the participant population...............................................10 Chart 2 Relevance of the program to population needs...............................................................................11 Chart 3 Why do you consider the program relevant?...................................................................................12 Chart 4 Did the Project contribute to a different use of resources?.........................................................16 Chart 5 Why did it contribute? ...........................................................................................................................16 Chart 6 What impacts were achieved with the project?...............................................................................17 Chart 7 Risk perception: People surveyed........................................................................................................17 Chart 8 People who perceive a likelihood of being affected by an earthquake .......................................18 Chart 9 What was the project’s contribution in generating regulations, processes, and policies?.....18 Chart 10 Safe construction training attendees (day 1)...................................................................................20 Chart 11 Safe construction training attendees (day 2)...................................................................................20 Chart 12 Evaluations after two months of training.........................................................................................21 Chart 13 People who passed the evaluations with more than 70% -Retention of knowledge-...........21 FINAL EVALUATION OF EBRP III | vii Chart 14 Percentage of people who passed the participatory assessment on hazards, vulnerability, and response capacity with more than 7 points..............................................................................22 Chart 15 Percentage of people who passed the participatory assessment on hazards, vulnerability, and response capacity two months later. .........................................................................................22 Chart 16 Structure reinforcement training.......................................................................................................23 Chart 17 Theory training on rapid assessments of structural vulnerability in buildings. .......................23 Chart 18 Practical training on rapid assessments of structural vulnerability in buildings......................24 Chart 19 Ongoing actions according to leaders..............................................................................................25 Chart 20 Actions to be implemented to replicate the Project, according to the communities ..........25 LIST OF TABLES Table 1 Number of evaluation participants by gender...................................................................................1 Table 2 General program information...............................................................................................................4 Table 3 Complementary contributions to the program................................................................................4 Table 4 Complementary contributions from the local counterpart ..........................................................4 Table 5 Report planning ........................................................................................................................................5 Table 6 Number of participants per sector......................................................................................................6 Table 7 Number of samples per district ...........................................................................................................6 Table 8 Number of samples per community....................................................................................................7 Table 9 Number of women trained on earthquake-resistant construction methodologies.................7 Table 10 Number of women who participated in the program evaluation process.................................7 Table 11 Research techniques and methods used in the evaluation.............................................................8 Table 12 Research criteria used in the evaluation.............................................................................................8 Table 13 Guidelines, 2021-2025 Opportunity Creation Plan.......................................................................10 Table 14 Purpose 1 indicators of the EBRP III program and their degree of achievement...................12 Table 15 Purpose I activities of the EBRP III program and their degree of achievement ......................13 Table 16 Purpose 2 activities of the EBRP III program and their degree of achievement.....................13 Table 17 Purpose 3 indicators of the EBRP III program and their degree of achievement...................14 Table 18 Purpose 3 activities of the EBRP III program and their degree of achievement.....................15 Table 19 Description of program indicators ....................................................................................................36 Table 20 Evaluation methods and limitations...................................................................................................36 viii | FINAL EVALUATION OF EBRP III ACRONYMS APGRE Spanish acronym for Association of Risk Management Professionals of Ecuador BHA Bureau of Humanitarian Assistance DRM Disaster Risk Management DRR Disaster Risk Reduction DVI Detailed Vulnerability Inspections EIB European Investment Bank EP-ULEAM Spanish acronym for Public company for the Production and Strategic Development of the Eloy Alfaro Secular University of Manabí GAD Spanish acronym for Decentralized Autonomous Governments GPM Spanish acronym for Manabí Provincial government MIDUVI Spanish acronym for Ministry of Urban Development and Housing MTOP Spanish acronym for Ministry of Transport and Public Works NGO Non-Governmental Organization PUCE Spanish acronym for Pontificial Catholic University of Ecuador RVI Rapid Vulnerability Inspections SGR Spanish acronym for Risk Management Secretariat SNGRE Spanish acronym for National Risk and Emergency Management Service ULEAM Spanish acronym for Eloy Alfaro Secular University of Manabí UNESUM Spanish acronym for Southern Manabí State University USAID U.S. Agency for International Development USGP Spanish acronym for San Gregorio University of Portoviejo FINAL EVALUATION OF EBRP III | ix EXECUTIVE SUMMARY EVALUATION PURPOSE Measure and document the program’s outcomes and impact, including lessons learned, strengths, and challenges, to inform future programs aimed at strengthening community resilience in Ecuador and other locations. EVALUATION QUESTIONS • To what extent have the activity’s interventions followed the planned implementation plan —timelines, participant selection, composition/amounts of transfer of resources, inputs and service delivery, and outputs — and completed the expected objectives, purposes, and outcomes? /Criteria: RELEVANCE, EFFECTIVENESS. • Have the interventions reached the target groups within the target areas (homeowners, construction workers, community councils, engineers, and government officials)? /Criteria: EFFICIENCY. • What (expected and unexpected, positive and negative) changes did the selected beneficiaries, community members, and other stakeholders associate with the activity’s interventions? /Criteria: IMPACT/SUSTAINABILITY The key sources/audience for this evaluation were technical experts (engineers and architects), public officials, students from universities and schools, community leaders, and the benefited men and women who trained in rural communities and urban areas of the province of Manabí. Their representativeness, participation, and geographic location were paramount to defining them as key informants for the program. BACKGROUND The USAID/BHA Ecuador Building Resilience Program (Award 720BHA21GR00139) was launched on June 25, 2021, with a 24-month planned implementation period (21 months implemented), ending on March 2023 and a total budget of $1,000,000.00. The goal of EBRP III was to strengthen disaster resilience among vulnerable populations through targeted investments in capacity building and technical assistance for improved risk reduction and management. The program was designed as a continuation of phases one and two (EBRP I and EBRP II), which were implemented following the April 16, 2016 earthquake. These programs aided the affected communities by helping them develop construction skills for seismic-resistant houses and bamboo construction and offering training on risk management. EBRP III provides evidence of the following purpose statements: 1. Strengthen the essential knowledge and skills for seismic-resistant construction and techniques for resilient households through training sessions and technical assistance in vulnerable communities. 2. Assess the structural vulnerability of critical infrastructure from Portoviejo, guided by the 2020-21 multi-criteria risk analysis findings. 3. Expand the multi-criteria risk analysis and create a communication conduit of risk awareness and coordinated risk management actions to peri-urban and rural communities of the Municipality of Portoviejo. EVALUATION QUESTIONS, DESIGN, METHODS, AND LIMITATIONS The evaluation consisted of gathering inputs and verifying the implementation of the objectives and impacts planned for the program. It was a participatory evaluation that combined qualitative and quantitative approaches; in other words, it reviewed documental information and gathered primary x | FINAL EVALUATION OF EBRP III information from communities, technical teams, managers, and partners through interviews and focus groups. This evaluation included: • Semi-structured interviews with key informants, managers, technical experts, and leaders. • Focus groups with rural community organizations’ representatives and beneficiaries. • Interviews with government authorities, local GAD authorities, and district councils • Interviews with directors and professors of the universities that participated in the program • Interviews with professionals, such as engineers and architects, involved in planning and implementation. While selecting the sample to evaluate, the team sought out representatives from different levels, integrating gender, generational, and geographic location approaches. FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS The evaluating team verified program progress and outcomes through a desk review, an analysis process, and focus groups with key informants. The achievement of the planned indicators, outcomes and progress, and the fulfillment of expected activities was verified by the achieved indicators, thus, obtaining the quantitative progress that is complemented by interviews with key informants (qualitative approaches). Based on these criteria, the main findings of the evaluation are: Relevance The EBRP III program has been relevant to state planning and development policies regarding the right to housing and safe conditions and the reality of the target population/geographic area being a vulnerable and high-risk area regarding earthquakes and natural disasters. The targeted area has great environmental potential because it is the province with the highest production of guadua cane in the country. People can harvest and promote these crops to build safe housing for the population. Effectiveness The EBRP III program has reached a high level of accomplishment of objectives and expected outcomes. Compliance with indicators and outcomes within the stipulated timeframe and using estimated resources in both technical and financial areas serve as evidence. It is worth mentioning that not all indicators were accomplished due to the early closing of the program (three months ahead of schedule), because of external factors, such as the COVID outbreak, which defined a change in strategy and is explained below. Efficiency Throughout implementation, the program promoted technical improvement of construction processes, optimization of materials and design, construction, and monitoring processes; Thus, promoting construction standards within the GAD ordinances that allow for better quality control and relevance of buildings. At the same time, the program offered training for construction professionals and engineering and architecture students. Impact The main impacts of the program were: Training the rural population of Portoviejo Canton on earthquake-resistant safe construction management and risk management; and knowledge on the application and treatment of the guadua cane. Moreover, universities have made new requests for training courses on this topic and simultaneously disseminated information on construction regulations among authorities. The value given to the guadua cane and environmental care are other impacts achieved. FINAL EVALUATION OF EBRP III | xi Sustainability The program defined sustainability strategies, signed agreements with universities (so they can continue replicating the training using the Miyamoto guidelines), and promoted the use of materials and tools. On the other hand, there is the impact of the GADs regarding control mechanisms for construction and the establishment of ordinances to this effect. It is worth mentioning that with the shift in local authorities (mayors), the continuity of these training processes and the establishment of technical parameters must be emphasized to ensure quality in housing construction and safety for its inhabitants. Conclusions • The EBRP III program is the third, progressive, and final phase after two previous programs aimed at promoting resilience in the communities affected by the 2016 earthquake that struck Manabí and Esmeraldas. • The program’s main objective was to generate resilience in rural communities of the Portoviejo canton, promoting skills among individuals so that they know and acknowledge traditional construction processes under technical principles and regulations and teaching individuals about risk management. • The program promoted construction with environment-friendly materials, such as guadua cane, that favor tradition, and care for the environment, the preservation of native species, and the beautification of the surroundings. • The implementation and perfection of technical skills by construction professionals, such as architects, engineers, and technicians, has had a positive impact by promoting the adoption of construction regulations and standards. • The program forged strategic partnerships between Academia, partner agencies, and the GADs to develop better construction processes that are efficient and safe, as well as to apply and define regulatory standards in the region. • The participation of students in community trainings has created the possibility for them to apply what has been learned and, at the same time, has strengthened the connection and experience of values such as reciprocity, respect, and care for the environment. • The participation of women in technical training sessions represents great progress in the development of a process to improve knowledge and reduce gaps in access. It has also promoted the participation of youth and people with disabilities. Recommendations • Better define sustainability actions to maintain the learning by participants at the community and technical levels and consolidate and expand knowledge to new areas. • Disseminate the principal methodologies, materials, and knowledge to other cantons in the province of Manabí and the country, promoting the exchange of knowledge. • Develop, together with Academia, support for the organization, production, and commercialization processes of the participating associations linked to bamboo, creating possibilities for the generation of bamboo products for the market. • Resume dialogue with the new local authorities, emphasizing the importance of continuity in the definition and control over the construction processes, whether with traditional materials or bamboo. • Promote and disseminate the experiences of the production practices of the associations in the locality and with students, generating work plans and joint practices to strengthen the production outcomes and the commitment of young people to the community. xii | FINAL EVALUATION OF EBRP III • Expand the dissemination and implementation of technical trainings to other municipalities and cities in the country, focusing on having safe cities. FINAL EVALUATION OF EBRP III | 1 EVALUATION PURPOSE AND EVALUATION QUESTIONS This evaluation was carried out for each of the program’s components: Policy creation and planning, training development, and awareness-raising/community mobilization. The program’s timeframe includes implementation and monitoring activities from June 25, 2021, to March 31, 2023. EVALUATION PURPOSE Measure and document the program’s outcomes and impact, including lessons learned, strengths, and challenges, to inform future programs aimed at strengthening community resilience in Ecuador and other locations. Evaluation participants Technical experts, staff from different universities, GADs participating in the program, the National Risk and Emergency Management Service, the provincial government, external partners (EIB and World Vision), local community leaders, and beneficiaries trained in the communities and districts participated in this investigation. Regarding the community evaluation, the evaluation team selected a sample of people with the greatest relevance to and participation in the program, those with the ability to make decisions and act during the different activities implemented in the communities. The community focus groups were selected based on their participation, interest, and centrality. A detailed table of evaluation participants can be found below. Table 1 Number of evaluation participants by gender. PARTICIPANTS MEN WOMEN METHOD Key informants Directives 6 2 Interviews Technical experts 7 2 Interviews Community leaders 1 2 Interviews Target population Participants (communities) 12 13 Focus group TOTAL 26 19 45 Author: Buen Vivir Consultancy, March 2023 The list of participants is shown in Annex V. Sources of Information (item 5.1) EVALUATION QUESTIONS Based on the program's components, the program identified two types of audiences for the research. A detailed description can be found in Annex IV. Collection tools and data analysis. Questions asked to experts and officials General information: Names, age, sex, position, functions or relation with the project, institution — community, name of the institution— community, work time, and relationship to project. 1. What was the context in which the program was implemented? 2. Do you consider the program relevant to the needs of the population? Why? 3. What were the main lessons you learned from the program? 4. Did the project contribute to a different use of resources? (Human, technical, or material) How? 2 | FINAL EVALUATION OF EBRP III 5. What was the project’s contribution to generating standards, processes, and policies? 6. What continuity actions will be carried out by your institution? 7. What impact was attained by the project? 8. What would you improve in new interventions? 9. Comments and suggestions. Questions asked to community participants General information: Names, age, position in the community, district, community, time participated in the project (month/year). 1. How were the participants selected for the training? (Criteria) 2. What changes were accomplished at the individual level? (Describe) 3. How did these training sessions contribute to community development? 4. What messages do you remember from the campaigns? 5. What lessons learned have you been able to replicate in your community? 6. What were the challenges faced in the project? 7. What are the next steps for replicating the project? 8. Observations – Comments FINAL EVALUATION OF EBRP III | 3 BACKGROUND This external final evaluation was carried out for the Ecuador Building Resilience Program (EBRP III), financed by USAID/BHA and implemented by Miyamoto International. The evaluation covers all three components: policy creation and planning, capacity building and training, and raising awareness/community mobilization. The program’s time frame includes implementation and monitoring activities from June 25, 2021, to March 31, 2023. Additionally, the objective aligns with the vision of the Bureau of Humanitarian Assistance (BHA) to “Provide the base for a transforming change and self-sufficiency” and its mission “Reduce physical, social, and economic impact from disasters by supporting the population at risk to develop stable knowledge in coordination with the affected population, host governments, local and international NGOs and the private sector.” Furthermore, by applying an integrated risk management strategy focused on strengthening community resilience to natural disasters and developing more efficient public policies and plans, the activity addressed all four priority actions according to the Sendai Framework for Disaster Risk Reduction, (i) Understanding disaster risk; (ii) strengthening disaster risk governance to manage disaster risks; (iii) investing in disaster risk reduction for resilience, and (iv) enhancing disaster preparedness for effective response and “Building Back Better" in recovery, rehabilitation, and reconstruction. This evaluation assesses the EBRP III program and its three components described below: 1. Strengthen the essential knowledge and skills for seismic-resistant construction and techniques for resilient households through training sessions and technical assistance in vulnerable communities. 2. Assess the structural vulnerability of critical infrastructure from Portoviejo, guided by the 2020-21 multi-criteria risk analysis findings. 3. Expand the multi-criteria risk analysis and create a communication conduit of risk awareness and coordinated risk management actions to peri-urban and rural communities of the Municipality of Portoviejo. The program was planned to last 24 months, but instead it ran from June 2021 until March 2023, a total of 21 months. The early closure was due to the termination of resources allocated to the program, which, according to program management, resulted from an upsurge in the pandemic in June and July 2022. The Ecuadorian government issued sanitary and regulatory measures regarding mass events, which made it necessary to postpone some activities and redesign program strategies, e.g., activities for raising risk management awareness. Apart from planning delays, these changes produced unexpected additional costs that affected the budget allocated to the program, and the completion of indicator D02 (community action plans) and activity 2.3 of purpose 2 (technical training on DVI (Detailed Vulnerability Inspections) methodology to senior engineers from ULEAM and the municipalities and the supervised deployment to assess three high priority buildings. Another factor that affected program implementation was the election period (January and February 2023), where new provincial, cantonal, and district authorities were elected. This delayed the review, approval, and implementation processes for the proposals presented by the EBRP III, which relied on the political will of the new officials. Timelines in Annex I. Calendar include details of the execution. As for the financial aspect, a summary is shown below: 4 | FINAL EVALUATION OF EBRP III Table 2 General program information. AWARD NUMBER AWARD DATE FUNDING LEVELS IMPLEMENTING PARTNERS 720BHA21GR00139 June 25, 2021 $1.000.000 Miyamoto International Author: Buen Vivir Consultancy, March 2023 Source: Program Management, March 2023 Table 3 Complementary contributions to the program NAME OF PARTNERS WHO CONTRIBUTED TO THE PROGRAM * DATE AMOUNT CONTRIBUTED (Dollars) EBI (Supported activities from the three purposes of the program) from October 2021 to May 2022 $ 95,000.00 California Polytechnic State University (Supported the construction of the classroom for training sessions) August 2022 – January 2023 $ 20,651.14 TOTAL $ 115,651.14 *Resources were managed directly by contributors (partners). Author: Buen Vivir Consultancy, March 2023 Source: Program Management, March 2023 Table 4 Complementary contributions from local counterparts PROGRAM’S LOCAL COUNTERPART DATE APPROVED AMOUNT Eloy Alfaro Secular University of Manabí (ULEAM) (Venues for awareness-raising events) September – October, 2022 $ 3,600.00 “Sembrando Esperanza” and “Acero Vegetal” Associations (Workforce) November – December, 2022 $ 1,440.00 TOTAL $ 5,040.00 *Resources valued according to market price Author: Buen Vivir Consultancy, March 2023 Source: program management, March 2023 Note: Contributors themselves directly managed all complementary contributions granted to the EBRP III program. The EBRP III program was only the implementer of the activities carried out with such contributions. The total sum of the complementary contributions from program partners and collaborators reached $120,691.14 (One hundred twenty thousand, six hundred ninety-one dollars and fourteen cents). Annex V: SOURCES OF INFORMATION (Item 5.2) includes a breakdown of these complementary contributions. FINAL EVALUATION OF EBRP III | 5 The program was generally implemented according to plan (permanent monitoring and follow-up), and completed the scheduled periodic report submissions described below: Table 5 Report planning TYPE OF REPORT REPORTING PERIOD First Annual Report From June 25, 2021, to September 30, 2021. First Semi-Annual Report From October 1, 2021, to March 31, 2022. Second Annual Report From October 1, 2021, to September 30, 2022. Author: Buen Vivir Consultancy, March 2023 Source: Program Management, March 2023 On December 13, 2022, the EBRP III program received a visit from the donor (BHA). The hypothesis (Theory of Change) for this program was: “IF the population living in high-risk disaster areas have a broad awareness of risk, access to essential risk management resources, better practices, and are guided by public policies informed by current risk data and expert risk analyses, THEN resilience to disasters will improve, and disaster vulnerability will decrease.” Note: This information comes from the EBRP III program’s technical proposal. The geographic coverage of the EBRP III program included the canton of Portoviejo from the province of Manabí in Ecuador. The intervention in the cantons of Sucre, Jama, and Pedernales in the province of Manabí focused on supporting bamboo artisans and builders and on risk management awareness-raising events in the canton of Pedernales. The program selected the canton of Portoviejo as the coverage area to give continuity, complement the work completed in Phase II, and address communities and rural districts that had not been attended to in the area. The following are the maps of the program’s coverage area: EBRP III coverage: The map on the left shows the cantons of the Manabí province where the program intervened. The map on the right shows rural districts of the Portoviejo canton that comprise the EBRP III program's main coverage. Image 1 Geographic coverage map of the program 6 | FINAL EVALUATION OF EBRP III EVALUATION METHODS AND LIMITATIONS The evaluation consisted of gathering inputs to verify the execution of objectives, achievements, and impacts planned in the program. It was a participatory evaluation that combined the participation of beneficiaries through interviews with technical teams, strategic partners, and authorities. To carry it out, the consultant team first designed a work plan which Miyamoto reviewed and approved. See Annex II. Evaluation Work Statement. Desk Review The consultant team reviewed: • The technical proposal of the program • Interagency agreements • The program Monitoring and Evaluation Plan • Program annual and semi-annual reports • Program monitoring and follow-up database • Program archives • Teaching and technical materials for trainings • Specialized publications produced by the program • Ordinances under review Sample selection and representativeness Regarding key actors and partners of the program, the consultant team selected and interviewed a sample from 100% of the areas and levels involved, i.e., participating technicians, strategic partners, and authorities. Table 6 Number of participants per area/level PARTICIPANTS NUMBER METHOD Key informants Managers 8 Interviews Technical experts 9 Interviews Community leaders 3 Interviews Target population Participants from communities 25 Focus group TOTAL 45 Author: Buen Vivir Consultancy, March 2023 This external evaluation decided to take a representative sample of districts; of the seven districts, two were chosen (a sample representing 29% of the total). Table 7 Number of samples per district Number of participating districts Sample number Percentage 7 2 29% Author: Buen Vivir Consultancy, March 2023 FINAL EVALUATION OF EBRP III | 7 Regarding the 53 communities that participated in program trainings on seismic-resistant construction and risk management, the consultant team took a sample of 6 communities, which corresponds to 11% of the total. Table 8 Number of samples per community Number of participating communities Sample number Percentage 53 6 11% Author: Buen Vivir Consultancy, March 2023 The team, which included a consultant, an information gathering expert, and a systematization expert, carried out field visits in the territory. The participants selected for the evaluation process were chosen based on their participation, leadership, and involvement in the program. The evaluation team gave special attention and compliance to gender equity during program implementation and its evaluation. Thus, the team integrated the participation of women in both focus groups and programmed interviews, resulting in the participation of 19 women (42% of all participants). Table 9 Number of women trained on earthquake-resistant construction methodologies Total number of participants Number of women Percentage 1,434 701 49% Author: Buen Vivir Consultancy, March 2023 Table 10 Number of women who participated in the program evaluation process Total number of participants Number of women Percentage 45 19 42% Author: Buen Vivir Consultancy, March 2023 The team can affirm that the percentage of women interviewed for this evaluation (42%) was very similar to the number of women who participated in the program (49%); therefore, gender representativeness in this evaluation was met. Application of Techniques and Data Collection The evaluation team applied quantitative and qualitative data gathering methods. The data collected from interviews with staff members and focus groups, highlighted the beneficiaries’ perceptions regarding: • Participation in training activities • Tools, channels of communication, and key messages in outreach materials • Selection of municipal buildings for evaluation • Data collection sources for risk analyses; and • Integration of risk data into relevant policies/plans. Additionally, during the program implementation, periodic evaluations of the participants of the training events were carried out. These evaluations produced a broad, detailed, and consistent database used to analyze the following variables: geographic scope, age, marital status, level of education, trade/occupation, disability, and learning level. The information has been integrated into some of the analyses in this report. Moreover, the collected, sorted, and tabulated information can show how to implement new and optimized procedures for future programs. 8 | FINAL EVALUATION OF EBRP III Coordination Miyamoto International, through its EBRP III program team, offered the evaluating team all the necessary assistance and information to develop the evaluation. In this regard, see the following: - Program files, documentation, and database. - Contacts of the main program actors. - Community calls for the organization of on-site evaluations. - Transportation of the evaluation team within the program’s territory. The team analyzed the main trends to determine relevant findings with the data collected during the evaluations. The findings of this analysis are listed in Section 5 of this report (Findings, Conclusions, and Recommendations). Limitations of the Evaluation - Not all focus groups in the evaluation process met the expected attendance levels because people in the communities prioritize their daily tasks that provide them with the means and resources for subsistence. - The evaluation of the program was implemented in a short period, facing external factors, such as increasing public insecurity and complex climate conditions. However, the objective was accomplished. A summary of the techniques and methods of the research and its criteria are presented in tables 11 and 12: Table 11 Research techniques and methods used in the evaluation Research techniques Target group Responsibility Desk review of the program’s description, policies, guidelines, and plans. Review of secondary sources relevant to the topic. - National authorities - Local authorities - Project experts Evaluator and program manager Direct qualitative techniques Focus groups Semi-structured interviews Indirect –Observation - Communities - Technical experts - Local GAD Authorities Evaluator and field expert Quantitative techniques/Numeric analysis and trends -Communities -Trained participants Evaluator and data entry expert Source: EBRP III program Technical Evaluation Proposal, CIBV, February 2023 Table 12 Research criteria used in the evaluation Criteria Relevant information Techniques used RELEVANCE: Analysis of relationship between project objectives vs. demand needs. Was the project design relevant to the problems you/they are facing? Were the project objectives and design relevant given the context? Document review, interviews, focus groups. FINAL EVALUATION OF EBRP III | 9 EFFECTIVENESS: The extent to which the project has achieved the objectives and outcomes and benefited the target population. Has the program reached the expected number of beneficiaries? Are the beneficiaries satisfied with the quality and service of the project? Specifically, in what aspects were the beneficiaries dissatisfied? Document review, technical reports, interviews, and focus groups. EFFICIENCY: Relationship between the quality and quantity of programming outcomes and the resources and means used to achieve them. What has been the performance in terms of projected outcome indicators? Did the program achieve the number of projected training workshops? Provide information about the problems encountered and the measures taken to remedy them. Have there been any unforeseen effects? Document review, technical reports, interviews, and focus groups. IMPACT: Measurement of behavioral and structural changes as an effect of the intervention of a program on a given population at the individual, family, local, regional, and national levels. How did the program contribute to the achievement of the goal? To what extent could it be said that the institutions' capacity has been increased? How would the impact of the project have been increased? Document review, technical reports, interviews, and focus groups. SUSTAINABILITY: Estimates whether the outcomes of the project have had or are likely to have lasting effects after the project has been completed. Will the project's effects remain over time? Will the project continue with the activities after project resources are spent? Document review, technical reports, interviews, and focus groups. Source: EBRP III program Technical Evaluation Proposal, CIBV, February 2023 Furthermore, the monitoring and follow-up plan proposed by the project was taken into account, and it is included in Annex III. 10 | FINAL EVALUATION OF EBRP III FINDINGS The review and verification of compliance with program planning was carried out for objectives, outcomes, activities, and indicators, with the following findings described below. Concerning the program evaluation criteria, the following findings are available: Relevance of the Program based on Context, Public Policies, and Target Population It indicates whether the program was adapted to the environment and the economic, political, and social conditions that may have had an impact on the tended geographic sectors. RELEVANCE OF THE PROGRAM BASED ON CONTEXT To demonstrate the adequacy of the program based on the context of this intervention, key informants were interviewed: managers, technicians, and community leaders, as follows: Source: Interviews with key informants, February 2023 Author: Buen Vivir Consultancy, March 2023 Of all the key informants, 28% indicated that there was an adequate context to train communities in safe construction (recently, the Manabí earthquake occurred in 2016).Then, 24% stated that an adequate context is necessary to promote rural development in the province, and 21% consider it important to train the population in risk management. To a lesser extent, 10% believe that GADs require technical assistance, 10% say that training in bamboo cane is in line, and 7% say that support is needed after the 2016 earthquake. RELEVANCE/COMPLIANCE WITH REGULATIONS, AND PUBLIC OFFICES It is consistent with Article 340 of the Ecuadorian Constitution, which recognizes that people have the right to a safe and healthy habitat and adequate and decent housing, regardless of their social and economic situation. The Opportunity Creation Plan (2021-2025) developed by the National Planning Secretariat, the governing body for planning in the Ecuadorian State, is related to Objective 5 of the social axis (to protect families, guarantee their rights and services, eradicate poverty, and promote social inclusion). Goal 5.4.1. Reduce the housing deficit from 58% to 48.44% Table 13 Guidelines, 2021-2025 Opportunity Creation Plan OBJECTIVE GOAL INDICATOR Chart 1 Context of the program according to the participant population FINAL EVALUATION OF EBRP III | 11 Objective 5. Protect families, guarantee their rights and services, eradicate poverty, and promote social inclusion. 5.1.1. To reduce extreme poverty by income rate from 15.44% to 10.76%. Occurrence of extreme poverty by income. 5.1.2. To reduce the child labor rate (5 to 14 years old) from 6.10% to 4.42%. Child labor from 5 to 14 years of age. 5.2.1. To reduce the femicide rate per 100,000 women from 0.87 to 0.80. Femicide rate per 100,000 women. 5.2.2. To reduce the adequate employment gap between men and women, from 33.50% to 28.45%. Adequate employment gap between men and women (15 years of age and older). 5.2.3. To reduce the wage gap between men and women from 15.34% to 11.27%. The labor income gap between men and women. 5.3.1. To increase the percentage of people covered by one of the social security schemes. Percentage of individuals covered by any of the contributive public social security schemes. 5.4.1. To reduce the housing deficit from 58.00% to 48.44% Housing deficit. Source: Opportunity Creation Plan, National Planning Secretariat, 2021-2025 In other words, it is relevant since it is framed within the social axis of benefiting families and their access to rights and services. It is relevant because housing is a key factor in a person’s development and contributes to the quality of life of families, their security, and welfare. RELEVANCE: OPINIONS OF COMMUNITY LEADERS AND TECHNICAL STAFF ON THE PROGRAM Chart 2 Relevance of the program to population needs Source: Interviews with key informants, February 2023 Author: Buen Vivir Consultancy, March 2023 12 | FINAL EVALUATION OF EBRP III According to the opinions of stakeholders, 100% of them agree with the relevance of the program since it is focused on the needs of the population. Chart 3 Why do you consider the program relevant? Source: Interviews with key informants, February 2023 Author: Buen Vivir Consultancy, March 2023 Breaking down the question of relevance, 33% of respondents agreed that community needs were met, 18% stated that the relevance of the program was adequate since the province/region is prone to natural disasters, and 18% also stated that the use of guadua cane was important since it fosters respect towards nature. Program Effectiveness: Achievement of Objectives, Indicators, and Results Extent to which the project has achieved the objectives and outcomes and benefited the target population. Purpose 1: Strengthen essential knowledge and skills in seismic-resistant construction and resilient housing techniques through training and technical assistance in vulnerable communities. Purpose 1 includes 2 indicators (D06 and D07) and 8 activities. Compliance with Purpose 1 indicators Table 14 Purpose 1 indicators and their degree of achievement PROGRAM INDICATORS AND THEIR ESTIMATED DEGREE OF ACHIEVEMENT SUBSECTOR: CAPACITY BUILDING AND TRAINING PROGRAM GOAL GOAL REACHED PERCENTAGE OF ACHIEVEMENT 1 D06a: Number of people trained in disaster preparedness, DRR and/or DRM: TECHNICAL TRAINING ON CONSTRUCTION METHODOLOGIES 1,000 1,434 143.40% 2 D06b: Number of people trained in disaster preparedness, DRR and/or DRM: TECHNICAL TRAINING ON STRUCTURAL REINFORCEMENTS 200 200 100% 3 D07: Percent of individuals who retain disaster preparedness, DRR and/or DRM skills and knowledge two months after training 800 820 102.50% FINAL EVALUATION OF EBRP III | 13 Source: EBRP III General Report: Miyamoto, as of March 2023. All Purpose 1 indicators were fully met, significantly exceeding program goals. The EIB contributed to achieving these indicators. Verification source: Data obtained comes from participant records, attendance lists, workshop reports, and participant evaluations. A summary of the data is provided in Annex 5. Sources of Information. Achievement of Purpose 1 Activities Table 15 Purpose I activities and their degree of achievement PROGRAM ACTIVITIES AND THEIR ESTIMATED DEGREE OF ACHIEVEMENT No. Description of activities PROGRAM GOAL GOAL REACHED 1 Activity 1.1. Formation of Steering Committee. 100% 100% 2 Activity 1.2. Community Mobilization and Beneficiary Selection. 100% 100% 3 Activity 1.3. Technical Training Development and Delivery. 100% 100% 4 Activity 1.4. Communications and Public Awareness. 100% 100% 5 Activity 1.5. Links to Bamboo Builders Associations and Other Industry Groups. 100% 100% 6 Activity 1.6. Community Accountability Events. 100% 100% 7 Activity 1.7. Systematization of Program Products and Outcomes. 100% 100% 8 Activity 1.8. Monitoring and Evaluation. 100% 100% Source: EBRP III General Report: Miyamoto, as of March 2023. All Purpose 1 activities were fully accomplished. Verification sources: meeting records, list of participants, event reports, certificates of participation, advertising spots, and photographic records. The data is included in Annex V. Sources of information. Purpose 2: Assess the structural vulnerability of critical infrastructure in Portoviejo guided by findings from the 2020-21 seismic risk analysis. Achievement of Purpose 2 indicators: Not applicable. Purpose 2 does not include any indicators but it does consider 3 activities. Achievement of Purpose 2 Activities Table 16 Purpose 2 activities and their degree of achievement PROGRAM ACTIVITIES AND THEIR ESTIMATED DEGREE OF ACHIEVEMENT No. Description of activities PROGRAM GOAL GOAL REACHED 1 Activity 2.1. Prioritization and selection of critical municipal buildings for structural vulnerability assessments. 100% 100% 2 Activity 2.2. Technical training of ULEAM and municipal engineers in the RVI methodology and supervised deployment to assess 25 essential buildings. 100% 100% 14 | FINAL EVALUATION OF EBRP III 3 Activity 2.3. Technical training of ULEAM and municipal senior engineers in the DVI methodology and supervised deployment to assess three high priority essential buildings 100% 0% Source: EBRP III General Report: Miyamoto, as of March 2023. All Purpose 2 activities were fully accomplished, except Activity 2.3, which could not be achieved due to the limitations and regulations imposed by the COVID-19 pandemic, forcing a change of strategy in the implementation plan of the program. Verification sources: building selection reports, videoconferences, attendance records, photographic records, evaluations, and technical material. Data is included in Annex V. Sources of Information. Purpose 3: Expand the seismic risk analysis and create a conduit for communication of risk knowledge and coordinated DRM actions in peri-urban and rural communities of Portoviejo municipality. Purpose 3 considers: indicators D01, D02, D03, D04, D05, D15, D16, C01 and C02, and 4 activities. Achievement of Purpose 3 Indicators Table 17 Purpose 3 indicators and their degree of achievement PROGRAM INDICATORS AND THEIR ESTIMATED DEGREE OF ACHIEVEMENT SUBSECTOR: POLICY AND PLANNING PROGRAM GOAL GOAL REACHED PERCENTAGE OF ACHIEVEMENT 1 D15: Number of DRR strategies, policies, disaster preparedness and contingency response plans drafted or revised to reflect improved information and procedures. 3 4 133.33% 2 D16: Number of strategies, policies, disaster preparedness, and contingency response plans that communities and/or governments are adopting and/or using. 3 2 66.67% 3 C01: Number of people participating in discussions regarding national risk reduction strategies as a result of the program. 50 66 132% 4 C02: Existing risk data is incorporated into the municipal DRR and response policy and plans. YES YES 100% SUBSECTOR: CAPACITY BUILDING AND TRAINING PROGRAM GOAL GOAL REACHED PERCENTAGE OF ACHIEVEMENT 5 D05: Percentage of people who perceive/recognize a high probability of being severely affected by a specific hazard. 25% 54.93% 219.72% SUBSECTOR: AWARENESS RAISING / COMMUNITY MOBILIZATION PROGRAM GOAL GOAL REACHED PERCENTAGE OF ACHIEVEMENT 6 D01: Number of community members that mobilized and completed a participatory hazard, vulnerability, and capacity assessment. 500 1,141 228.20% 7 D02: Number of community action plans developed based on participatory hazard, vulnerability, and capacity assessments. 7 3.15 45% 8 D03: Number of public awareness campaigns and/or drills completed 1 30 3,000% FINAL EVALUATION OF EBRP III | 15 9 D04: Number of individuals reached through public awareness campaigns and/or participating in drills 4,750 5,264 110.82% Source: EBRP III General Report: Miyamoto, as of March 2023. All Purpose 3 indicators were fully met, except for D02, which was not met due to delays generated by the limitations and regulations imposed by the COVID-19 pandemic, forcing the reprogramming of activities related to this indicator as well as a change of strategy in the implementation of the program. On the other hand, indicator D16 has been partially achieved since Portoviejo’s GAD adopted two of the four proposals presented by the program; only one additional proposal has not been adopted to comply with the indicator in its entirety. The delay with indicator D16 was because the Municipal GADs entered a period of elections to appoint new authorities and representatives, generating delays in the procedures and processes related to adopting the proposals presented by the EBRP III program. It is estimated that the pending processes related to this indicator will be reactivated once the new authorities of the Municipal GADs take office in May 2023 and that there is the necessary political will to do so. Verification sources: Regulations for implementing the operation of the Portoviejo canton’s risk reduction committee, training guide on disaster risks and the main effects of climate change, a guide for systematizing the training experience on climate change in communities, and a draft of the municipal ordinance that enables and creates incentives for the construction, conservation, and sustainable use of guadua cane and other bamboos in the Manta canton. The data is included in Annex V. Sources of Information. Table 18 Purpose 3 activities and their degree of achievement PROGRAM ACTIVITIES AND THEIR ESTIMATED DEGREE OF ACHIEVEMENT No. Description of activities PROGRAM GOAL GOAL REACHED 1 Activity 3.1. Review and Analysis of Existing Risk Data from the Expanded Target Area. 100% 100% 2 Activity 3.2. Integrate Results of Data Analysis into DRM Policy and Planning Processes. 100% 60% 3 Activity 3.3. Develop and Enhance Tools for Data Management and Municipal Planning 100% 100% 4 Activity 3.4. Promote Replication of Evidence-Based DRM Policy and Planning at the Municipal Level through Leadership Exchanges 100% 100% Source: EBRP III General Report: Miyamoto, as of March 2023. All Purpose 3 activities were fully achieved except for Activity 3.2, which was partially achieved since the entire digital repository of the data analyzed by the program was delivered to the Municipal GAD of Portoviejo. The implementation of these was missing in the planning and DRR policies. Program Efficiency: Achievement of Activities, Outcomes, and Resources The achievement of the planned objectives and activities vs. the activities carried out in Component 1 is reported as follows: PROGRAM EFFICIENCY AS PERCEIVED BY LEADERS 16 | FINAL EVALUATION OF EBRP III Chart 4 Did the Project contribute to a different use of resources? Source: Interviews with key informants, February 2023 Author: Buen Vivir Consultancy, March 2023 95% of people interviewed responded that the program helped the community to use resources differently; 5% (corresponding to 1 person) did not answer the question. Chart 5 Why did it contribute? Source: Interviews with key informants, February 2023 Author: Buen Vivir Consultancy, March 2023 Regarding the different use of resources, 33% believe that the program allowed them to improve the use of tools and materials in the use of guadua cane, which is a material whose value increased in the communities. 28% think the project promoted collectivity, leading to human resource changes. 25% consider that the project helped with technical skills, as there was an impact on university regulations and training. Finally, 14% confirmed that the project allowed them to optimize their work time by having more knowledge on the subject. Program Impacts IMPACT: AS PRECEIVED BY LEADERS FINAL EVALUATION OF EBRP III | 17 Chart 6 What impacts were achieved with the project? Source: Interviews with key informants, February 2023 Author: Buen Vivir Consultancy, March 2023 The main impact described by 28% of those interviewed was the generation of safe and resilient communities, which allowed them to be prepared for any hazard. 17% asserted that the educational opportunities improved since universities have become interested in these issues and generated related career paths. This goes along with the improvement in the qualification of professionals, where 17% of them explain that they are better prepared with technical knowledge. 14% agree that families currently have a culture of risk management and safe construction. Chart 7 Risk perception: People surveyed Source: EBRP III source database, February 2023 Author: Buen Vivir Consultancy, March 2023 Out of a total of 1,247 people surveyed about their perception of risk, 619 people (49.64%) were female, and 628 (50.36%) were male. 18 | FINAL EVALUATION OF EBRP III Chart 8 People who perceive a likelihood of being affected by an earthquake Source: EBRP III source database, February 2023 Author: Buen Vivir Consultancy, March 2023 Out of a total of 685 respondents who perceive a probability of being affected by an earthquake, 50.66% (347 people) were male and 49.34% (338 people) were female. With both indicators, we can infer that the population has been made aware of the risks that exist in the province, and the risks of an earthquake occurring again, making them more alert and resilient to the events that may occur. IMPACT: CONTRIBUTIONS TO REGULATIONS, PROCESSES, AND POLICIES Chart 9 What was the project’s contribution in generating regulations, processes, and policies? Fuente: Interviews with key informants, February 2023 Source: Buen Vivir Consultancy, March 2023 According to the generation of regulations, processes, and policies, 29% of the people surveyed stated that there was an improvement in construction regulations, 26% stated that project control systems were improved and 17% said that there were improvements in technical studies thanks to the participation of universities. IMPACT ON COMMUNITIES THROUGH ASSOCIATIVE PRACTICES FINAL EVALUATION OF EBRP III | 19 The EBRP III program supported the four associations of bamboo artisans and builders that were formed with the support of the previous phase of the program (EBRP II) in the cantons of Sucre, Jama, and Pedernales. Support provided by the EBRP III program to the organizations of the Pedernales canton (ASOPROASE (Sembrando Esperanza [In English, Sewing Hope]) and ASOSERCONACEVE (Acero Vegetal [in English, Plamt Steel]) consisted of the following: - Construction of a social housing prototype with concrete and bamboo, which serves as housing for the guard of the associations’ production center. - Construction of a 10-meter-long pool for the bamboo preservation process. - Construction of machines to process bamboo into commercial-sized slabs. - Construction of a training classroom (54m2). - Construction of the power line and installation of electricity connection for the associations’ production center. - Support for the construction of the associations’ plant nursery. - Support applying the ballast layer for the production center’s access road. - Training in crop management. The EBRP III program managed and implemented the training with the support of the Manabí Provincial Government’s technical staff. - Development of the associations’ web pages to promote their products and services. The impact of the support to these associations has been very representative. The associations are generating sources of employment for the families of their members as well as for the families of the surrounding area. The purchase, processing, and final marketing of guadua cane, either as a material, a processed product, or as part of the construction, contribute to the economic reactivation of the sector. On the other hand, the associations' bylaws establish their commitment to training community members in bamboo construction and hand craft making. For this purpose, they already have a training classroom which was built with the support of the program. The support provided by the EBRP III program to the Sucre canton association, ASOSERCONSANI [In English: San Isidro Construction Services Association], consisted in managing, by way of ULEAM’s legal department, the legalization of the association’s land so they can create/establish their production center. The activities carried out by the associations have a positive impact on the communities. It was not possible to support ASOSERCONJA (Asociacion de Servicios de Construccion Jama), the association from the canton of Jama because for work reasons, most of its members and its board were dispersed in Manabí and other provinces. Program Sustainability SUSTAINABILITY: LESSONS LEARNED FROM THE TARGET POPULATION The first component of sustainability includes the participation and development of technical skills through training and the permanence of this learning over time: 20 | FINAL EVALUATION OF EBRP III Chart 10 Safe construction training attendees (day 1) Source: EBRP III source database, February 2023 Author: Buen Vivir Consultancy, March 2023 Regarding training on earthquake-resistant construction methods and risk management, a total of 1,506 people participated in the first day of training, of which 772 were men (51.26%), and 734 were women (48.74%). Chart 11 Safe construction training attendees (day 2) Source: EBRP III source database, February 2023 Author: Buen Vivir Consultancy, March 2023 Regarding training on earthquake-resistant construction methods and risk management, a total of 1,434 people participated on the second day of training, of which 733 (51.12%) were men and 701 (48.88%) were women. FINAL EVALUATION OF EBRP III | 21 Chart 12 Evaluations after two months of training -Retention of knowledge￾Fuente: EBRP III source database, February 2023 Author: Buen Vivir Consultancy, March 2023 As for the participants who took the evaluations to measure knowledge retention two months after the training on earthquake-resistant construction and risk management, a total of 877 people were evaluated, of which 422 (48.12%) were men, and 455 (51.88%) were women. Chart 13 People who passed the evaluations with more than 70% -Retention of knowledge￾Source: EBRP III source database, February 2023. Author: Buen Vivir Consultancy, March 2023 Of the 877 people evaluated (two months after the training), a total of 820 people (394 men (48.05%) and 426 women (51.95%)) passed the evaluations in construction methods and risk management with a score of more than 70%. 22 | FINAL EVALUATION OF EBRP III Chart 14 Percentage of people who passed the participatory assessment on hazards, vulnerability, and response capacity with more than 7 points. Source: EBRP III source database, February 2023 Author: Buen Vivir Consultancy, March 2023 Of the 1,141 people who completed the participatory assessments on hazards, vulnerability, and response capacity, 87.4% (998 people) passed with a score equal to or greater than 70%; of these, 49.70% were men, and 50.30% were women. Chart 15 Percentage of people who passed the participatory assessment on hazards, vulnerability, and response capacity two months later. Source: EBRP III source database, February 2023. Author: Buen Vivir Consultancy, March 2023 Of the 662 people who took the participatory assessments on hazards, vulnerability, and response capacity two months after completing the training, 95.77% (633 people) passed with a score equal to or higher than 70%, of which 46.68% were men and 53.32% were women. SUSTAINABILITY: LEARNING FROM UNIVERSITY TECHNICAL EXPERTS (TEACHERS AND STUDENTS) AND TECHNICAL TEAMS FROM OTHER INSTITUTIONS. FINAL EVALUATION OF EBRP III | 23 Chart 16 Structure reinforcement training Source: EBRP III source database, February 2023 Author: Buen Vivir Consultancy, March 2023 As part of the program’s Purpose I, training sessions on “Seismic Strengthening of Structures” were held for university professors and students, technical teams from the Municipal GAD of Portoviejo, various public and professional institutions and professional freelancers (engineers and architects) freely practicing their occupation. A total of 200 people participated in the training, of which 148 people took the evaluation, and 82 people (71 men and 11 women) passed with a score of more than 70%. As can be seen in the chart, 63% of the trained individuals who passed the training courses on structural reinforcement in buildings and houses are freelance professionals (engineers and architects) linked to the practice of design, construction, and building assessments. Therefore, it is considered that in the short term, they will replicate what they have learned and provide sustainability to the acquired knowledge within their projects. 16% of participants were university teachers and students, and the remaining 21% were professionals from the technical teams of different public institutions, mainly from Municipal GADs. All participants are potential replicators of what they learned and will contribute to the sustainability. Chart 17 Theory training on rapid assessments of structural vulnerability in buildings. Source: EBRP III source database, February 2023 Author: Buen Vivir Consultancy, March 2023 24 | FINAL EVALUATION OF EBRP III As part of the Purpose 2 of the program, training sessions on “Rapid Structural Vulnerability Assessments of Buildings” were held for university teachers and students, technical teams from the Portoviejo Municipal Government, various public institutions, and professional freelancers (engineers and architects). A total of 207 people (152 men and 55 women) participated in the training sessions and met participation requirements. Of those trained, 38.16% were freelance independent professionals, 43.96% were engineering and architecture students from different universities in the country, 11.11% were engineering and architecture professors, 1.93% were post￾graduate professionals, and 4.83% were professionals from the technical teams of municipal GADs and the Ministry of Public Works. Attendance to these training indicates the capacity transferred in construction professionals and institutions to carry out structural vulnerability assessments in buildings and houses. The considerable number of university professors in the training sessions leads us to believe that knowledge will be sustained by systematically replicating what was learned. A similar effect could be expected from the participation of university students and construction professionals, especially those on technical teams from public agencies, where new policies and procedures can be implemented concerning what has been learned. Chart 18 Practical training on rapid assessments of structural vulnerability in buildings Source: EBRP III source database, February 2023. Author: Buen Vivir Consultancy, March 2023 The chart shows the number of participants that attended the on-site rapid building assessments carried out as part of the practical training exercise. In total, 136 people (97 men and 39 women) participated. The highest attendance percentage corresponds to ULEAM students and freelance professionals (engineers and architects), with a 37.50% attendance from each group. The total attendance by university students from different universities was 42.65%, the attendance of university professors reached 12.50%, the attendance of post-graduate students and professionals reached 1.47%, and the attendance from technical expert teams from public institutions reached 4.41%. All participants have the necessary profile to sustain and replicate what they learned. SUSTAINABILITY: FUTURE ACTIONS FINAL EVALUATION OF EBRP III | 25 Chart 19 Ongoing actions according to leaders Source: Key informant interviews, February 2023. Author: Buen Vivir Consultancy, March 2023 The respondents were asked about what actions they will take to give continuity to the program and 25% commented that they would promote and disseminate the information, 20% detailed that it is important to improve the training processes because these are topics that have currently gained relevance, 18% commented that academic learning would be improved, taking this information into account for university curricula, and 15% seek to replicate the project in other cantons or communities to reach more people. Chart 20 Actions to be implemented to replicate the Project, according to the communities Source: Key informant interviews, February 2023. Author: Buen Vivir Consultancy, March 2023 Participants were asked about the next steps to be taken to replicate the program and give continuity to the project. Of those asked, 31% of those interviewed said that once they were trained, they would pass on the knowledge to their community, as well as to their family and friends, 31% specified that they planned to do more projects with guadua cane (bamboo), such as buildings and furniture with what they had learned, and 23% said that, with the knowledge they had acquired, they would be able to start a productive venture. SUSTAINABILITY THROUGH EDUCATIONAL CENTERS 26 | FINAL EVALUATION OF EBRP III The EBRP III program has signed agreements with four major local universities: ULEAM, PUCE, USGP, and UTM. As of the date of this evaluation, three of these universities (ULEAM, PUCE, and USGP) are committed to continuing the training processes developed by the EBRP III program, specifically in teaching earthquake-resistant construction methods with conventional materials and bamboo, as well as risk management. Through their engineering and architecture schools, the universities will be able to replicate the knowledge acquired from the EBRP III program and will give sustainability to its purposes. On the other hand, the “Ángela Rosa Cevallos” Artisan Training Center of the Portoviejo Municipal GAD has formally expressed its interest in continuing the implementation of trainings done by the program. Note: the above-mentioned educational centers have requested the donation of the teaching materials and tools used by the EBRP III program to develop the training on earthquake-resistant construction methods and risk management and to be able to implement the same methodology applied by Miyamoto International during the implementation of the program. This could be implemented through community involvement, with which all universities are obliged to comply as part of their responsibilities with the program and with the community. FINAL EVALUATION OF EBRP III | 27 OVERALL ASSESSMENT OF THE INFORMATION BEST PRACTICES - The program’s socialization and promotion processes were key to achieving positive outcomes, as they generated interest and involvement from the beginning. The methodology used, the involvement of the community, the participation of the authorities and institutions, the materials used, the promotion and implementation of the processes, and the technical and management level of the implementing team members made it possible to develop all the activities related to the program. - The improvement of the methodologies used for the training processes of the EBRP program, which have been validated and adjusted in the previous phases during the last five years, has generated proven teaching-learning procedures with high acceptance from community members, technical experts, and authorities. - The growing dissemination of knowledge and the high demand from the communities and their spokespeople to replicate the EBRP program and its training is reflected in the number of people trained, which exceeds the figures planned in the program’s indicators. - Well-developed associativity is one of the best resources for strengthening individual, group, and community capacities. It is a highly effective tool for generating development opportunities with broad coverage for the common good and boosting and improving families' and their communities' resilience and response capacity. - The link between academia, the cooperation, and state agencies (SNGRE, Municipal and parochial GADs, and the Provincial Government) generated commitment and active participation, this being the key to managing the sustainability of the program’s objectives over time since the capacity for replication and dissemination of what was learned is very high. - The progressive collection of data on performance, participation, and related activities of the participants allowed for the organization of the results and the filing of the information in general. The necessary supporting documents, duly ordered and processed, are available to underpin the actions and achievements of the program. - Maintaining active and propositional communication between the organization executing the program and its main strategic partners helps develop the intervention and carry out coordinated work aimed at common objectives in a better way. It also generates credibility and a more significant and positive impact on the areas of intervention or coverage of the program. LESSONS LEARNED - Self-building in rural and peri-urban areas is one of the factors that most contributes to the vulnerability of families to hazards such as earthquakes, floods, and landslides. Although no one can eradicate this reality, it can be improved and regulated. - The training developed with the program on earthquake-resistant construction methods generates and reinforces skills and trades, such as master builders, construction assistants and artisans, and producers and marketers of bamboo products. This creates income opportunities and motivates further training. - Training courses in rural areas should include at least one module that advises participants on production processes using local resources. The module should preferably be related to the central theme of the training, which motivates participation and local development. - The contents, teaching materials, methodologies, language, safety and security protocols, and everything that is implemented for trainings, evaluations, surveys, and other activities carried out in the communities must be designed to be highly inclusive and in accordance with the local 28 | FINAL EVALUATION OF EBRP III reality, culture, and customs. In addition, it must enhance the abilities and skills of the people as well as their needs. - All the program’s actions must be oriented at guaranteeing its sustainability; that is, to ensure the continuity of its purposes over time once the external assistance has ended. In the case of training and outreach, this can be achieved through universities, as they are the institutions with the capacity to replicate and certify learning, and through governmental institutions linked to the program with cooperation and participation agreements. - Production and commercial support and advice should be maintained for new organizations until their members acquire the knowledge, experience, and capacity for large-scale production and commercialization to strengthen outcomes and expand their impacts. - The program team found that developing a strategic plan to promote the program’s activities contributes to greater participation in the training. The promotion of the program and the invitation to participate in organized events is best done in this context through face-to-face assemblies (socializations), strategically placed posters in the communities, and fliers distributed in assemblies or public spaces. The team found that care should be taken not to leave the promotion of the program and the calls for participation in the hands of local community leaders, as they do not always have the support of all the community members they represent. - All the agreements and appointments/sessions made with the communities and their representatives must be strictly complied with. - The best motivation for the members of an organization is the opportunities offered to work, improve their income, train, and progress. - Awareness-raising campaigns on risk issues should preferably be conducted in person, as this promotes the interest of participants and allows for greater interaction, thus providing a valuable space for discussion. - It is essential to territorialize the risk, that is, to work –to the extent possible– with the communities in the area, accompanying them in each of the processes involved in the program. This will ensure a more significant impact on the participants, a greater acceptance of the program, and greater collaboration and participation of its beneficiaries. MAIN ACHIEVEMENTS The program achieved coverage of 8,105 beneficiaries, a number higher than the 6,500 beneficiaries initially foreseen in the program, that is, a 25% increase in the number of direct beneficiaries. FINAL EVALUATION OF EBRP III | 29 The program worked closely with four universities. These universities were: Eloy Alfaro Secular University of Manabí (ULEAM for its initials in Spanish); Pontifical Catholic University of Ecuador – Manabí campus (PUCE-M for its initials in Spanish); Manabí Technical University (UTM); and San Gregorio University of Portoviejo (USGP for its initials in Spanish). The high level of participation and commitment of public institutions, such as the Portoviejo Municipal GAD, the National Risk and Emergency Management Service (SNGRE for its initials in Spanish), and the Manabí Provincial Government (GPM for its initials in Spanish) was also important. The program achieved representative gender equity through the participation of the direct beneficiaries of the program (52% men and 48% women). Participation was characterized by diverse inclusion, representing an interesting gathering process. The following people actively participated in the training: 54 illiterate people, one blind person, 19 people with disabilities, and 60 senior citizens (>65 years old). The European Investment Bank, The European Union, and the Latin America Investment Facility (LAIF) also participated in and made a representative contribution to the program. OPPORTUNITIES FOR IMPROVEMENT • It is essential to implement mechanisms to standardize and digitalize field data collection and tabulation; this facilitates processes, saves time and resources, and is more environmentally friendly by minimizing paper use. • Regarding community events: take advantage of open applications and training events to create spaces where beneficiaries receive guidance on specific production processes that could be useful to them; for example, teaching them how to use the available resources in the area to make profitable enterprises. This promotes community benefit and creates additional interest from participants. • It is important to have a place in the country for meetings with donor representatives (USAID/BHA) to inform them about the work being done through the program and to be able to have timely feedback. 30 | FINAL EVALUATION OF EBRP III • Create medium-range or technical-scope programs that allow for a progressive professionalization of the local labor force, the generation of new skills, and future job opportunities both at the community level and in the urban sector. • It is essential to provide organizational and commercial support to the production processes of bamboo associations that have been trained. The resources they obtain through the program provide a basis, but help is needed for the development of strategic and business plans that will allow them to strengthen their processes and provide for a better future for the member families. • Another significant opportunity for improvement is the possibility of creating alliances with local authorities or the business sector (tourism-related service providers, etc.), to find opportunities for joint work and identify the demand for products or services related to bamboo construction or the creation of handcrafted furniture, with a greater local identity, the area’s architectural beauty, and, at the same time generating income for the participating families. UNEXPECTED RESULTS 1. An agreement reached between the EBRP III program and the EIB, The European Union, and LAIF. This agreement was characterized by significant support from the EIB to the EBRP III program in the form of inputs, materials, equipment, tools, transportation, technical personnel, and support for the program. The following was achieved through this agreement: - Trained more people on safe construction methods. - Raised awareness and sensitized a greater number of people on risk management. - Supported associations of bamboo artisans and builders. - Edited and printed teaching material for the program: training manuals on safe construction, training guides on climate change, multi-hazard maps for the program’s coverage area, and two technical books on Seismic Strengthening of Structures and Simplified Methods for the Analysis and Rapid Evaluation of Structures. - Created signs for the evacuation routes in the Crucita district of the canton of Portoviejo. 2. Held the national and international event titled the “Technical Exchange on Guadua Bamboo”, which included the participation of national and international presenters from the following countries: Colombia, the United States, Peru, Mexico, and Ecuador. Eighty-two people attended the event, primarily technical experts, and professionals from countries such as Venezuela, Mexico, Peru, Spain, Colombia, and Ecuador. 3. Support for the Associations: Representative support was provided to three of the associations created with the support of the EBRP program. No. DESIGNATION NAME LOCATION (CANTON) SUPPORT PROVIDED TO THE ASSOCIATIONS BY THE EBRP III PROGRAM 1 ASOPROASE ASOCIACIÓN DE PRODUCCIÓN ARTESANAL PEDERNALES - Construction of plant nursery for reproduction and commercialization. It promotes the reproduction of endangered trees. FINAL EVALUATION OF EBRP III | 31 SEMBRANDO ESPERANZA [Spanish for Planting Hope Handicraft Production Association] - Construction of a 60m2 social housing prototype with concrete and bamboo. It also serves as a house for the associations’ production center guard. - Construction of a pool for the bamboo preservation process. - Construction of a classroom for training and other events related to the activities of the associations. - Repair of access roads to the associations’ production center. - Set up of the associations' production center's power lines and electrical service connections. - Training on forestry issues to members of the associations. This was carried out with the support of forestry technical experts from the Manabí Provincial Government - Advice and support in the presentation of projects to other non-governmental institutions. 2 ASOSERCONACEVE ASOCIACION DE SERVICIOS DE CONSTRUCCION ACERO VEGETAL [Spanish for Plant Steel Construction Services Association] PEDERNALES 4 ASOSERCONSANI ASOCIACION DE SERVICIOS de CONSTRUCCION SAN ISIDRO [Spanish for San Isidro Construction Services Association] SUCRE - Support in the legalization process of the land acquired by members of the association to build the production center. - Ongoing consultancy with experts on bamboo production, processing, and marketing. 4. Technical Exchange between the EBRP III program, the California Polytechnic State University, and the associations of artisans and bamboo builders of Pedernales. The EBRP III program trained university students in bamboo construction methodologies. Then, with the support of the associations' members and the students' direct participation, the training classroom was built using bamboo as the primary construction material. This classroom provided the associations with an appropriate space for future trainings. 32 | FINAL EVALUATION OF EBRP III CONCLUSIONS • The EBRP III program is the third progressive and conclusive phase of two previous programs to promote resilience in the communities affected by the 2016 earthquake in Manabí and Esmeraldas. This final phase perfected the training processes and estimated impacts. • The main objective of the EBRP III program has been to generate resilience in rural communities of the Portoviejo canton, boosting their skills to know and recognize traditional construction under principles and technical regulations and learning about risk management. • The program promoted construction with environmentally friendly materials such as guadua cane, favoring the cultural tradition and the environmental care and preservation of native species of the area and the beautification of the environment, promoting the cultivation and use of bamboo and the capacity to commercialize it. • The development and improvement of technical skills by construction professionals, such as architects, engineers, and technical experts, has positively impacted the adoption of building standards and regulations. • Strategic alliances have been developed between academia, partner agencies, and GADs to develop better, more efficient, and safe construction processes that will undoubtedly benefit the newly designed homes and their inhabitants. • The participation of students in community internships has made it possible for them to apply their knowledge and put it into practice while at the same time reinforcing the ties with communities and experience of values in society, such as reciprocity, respect, and environmental care. • The participation of women in technical trainings has been a significant step forward in developing a process to improve knowledge and reduce gaps in access to resources, promoting the feedback of young people and people with disabilities. • There is a noticeable interest on the part of local universities to continue the training implemented by the EBRP III program and to include these new processes and knowledge in their research and curricula. FINAL EVALUATION OF EBRP III | 33 RECOMMENDATIONS • Define continuity processes so that the knowledge gained by participants is maintained at the technical expert and community levels to consolidate and expand this knowledge to new areas. • Disseminate the main methodologies, materials, and training to other cantons in the province of Manabí and the country, generating an exchange of information and a wealth of knowledge. • The methods used, especially in community training on earthquake-resistant construction and risk management methodologies, should be widely patented and documented to promote their application in the country and, if possible, at the regional level. • To develop, together with academia, support for the organizational, production, and commercial processes of participating associations linked to bamboo, creating possibilities for the generation of resources and marketing products. • Emphasize the importance of continuity in the definition and control of building processes, whether traditional or bamboo, by resuming dialogue and promotion with new local authorities. • Promote training in an intentional, active, and participatory manner, especially for women and young people, defining processes to strengthen local and associative leadership because there is an initial process that requires support and reinforcement to ensure more significant impacts on income and higher productivity and performance in the processes of supplying guadua cane for the building sector. • Promote and spread the experiences of associative production in the area as well as in the student environment, generating work plans and joint practices to strengthen the production outcomes and the commitment of young people to the community. • Expand the dissemination and development of technical experiences to other municipalities and cities in the country to have safe cities. • Create the necessary conditions and agreements so that local universities can give continuity to the training and methodologies implemented by the EBRP III program. 34 | FINAL EVALUATION OF EBRP III ANNEXES ANNEX I: EVALUATION CALENDAR Timeline with data collection, baseline, and subsequent data collection dates. ACTIVITY DESCRIPTION DATE 1. Pre-baseline data collection July and August 2021 2. Presentation of baseline September 2021 3. Subsequent data collection during the implementation of the program From October 2021 to December 2022 4. Subsequent data collection during the evaluation of the program From: January 2023 to March 2023 Please find the timeline of both the implementation and execution of the program at the following link: http://bit.ly/3loghm1 FINAL EVALUATION OF EBRP III | 35 ANNEX II: EVALUATION WORK STATEMENT Please find the program evaluation statement or work plan at the following link: https://bit.ly/404K8i6 36 | FINAL EVALUATION OF EBRP III ANNEX III: EVALUATION METHODS AND LIMITATIONS The consulting team used the methodology proposed in the program’s Monitoring and Evaluation plan. Table 19 Description of program indicators EBRP III PROGRAM INDICATORS SUBSECTOR: POLICY AND PLANNING D15: Number of DRR strategies, policies disaster preparedness, and contingency response plans written or revised to reflect improved information and procedures D16: Number of DRR strategies, policies, disaster preparedness, and contingency response plans that are being adopted or utilized by communities and/or governments C01: Number of people participating in discussions regarding national risk reduction strategies as a result of the program C02: Existing risk data is incorporated into municipal DRR and response policy and plans SUBSECTOR: CAPACITY BUILDING AND TRAINING D05: Percent of individuals perceiving/ recognizing a high likelihood of being severely affected by specific hazard D06: Number of individuals trained in disaster preparedness, DRR and/or DRM: TECHNICAL TRAINING ON CONSTRUCTION METHODS AND TECHNICAL TRAINING ON STRUCTURE REINFORCEMENT D07: Percent of individuals who retain disaster preparedness, DRR and/or DRM skills and knowledge two months after training SUBSECTOR: AWARENESS RAISING/COMMUNITY MOBILIZATION D01: Number of individuals in communities mobilized and completed a participatory hazard, vulnerability, and capacity assessment D02: Number of community action plans developed participatory hazard, vulnerability, and capacity assessment D03: Number of public awareness campaigns and/or drills completed D04: Number of individuals reached through public awareness campaigns and/or participating in drills Table 20 Evaluation methods and limitations Instruments /Method Information to be collected Checklist/follow-up form Implemented by: National Program Manager Review the standard checklist to report on relevant indicators (D01, D02, D03, D04, D15, D16) for semi-annual and final reports. This checklist will document progress toward indicator targets. Verifiable items: Meeting notes, photographs, Project reports, direct observation, and conversations/interviews with multiple stakeholders. See description of indicators in Table No.19 FINAL EVALUATION OF EBRP III | 37 Attendance records and registration of participants Implemented by: National Program Manager Verification of log sheets monitored and observed by the National Program Manager or training facilitators. Track all unique individuals, including gender, the communities/agencies, and the total number of trainings/meetings. Verifiable items: Attendance records to evaluate indicators D06 and C01. Desk review Implemented by: Consultant agency The document review will be used during the baseline and end-line to assess the context related to indicators D15 and D16, which includes ensuring an understanding of existing response plans and policies, capacity gaps, and the key institutions to engage. Verifiable items: Plans, policies, and reports to understand previous or existing communication campaigns on risk related to D03 and D04. Surveys based on the beneficiaries Implemented by: National Program Manager, support staff, and consultant agency Surveys implemented by technical experts from Miyamoto and ULEAM 1) A beneficiary-based survey related to D5 was carried out at the beginning and end. The baseline used a simple random sampling in all communities, with a representative sample of 100 respondents, collecting data by target area and gender. The end-line survey used probability sampling to reach direct beneficiaries. Data were broken down by gender. Surveys were conducted in person, as it was appropriate for the audience. (2) Post-training questionnaires will be implemented for training program participants two months after completion to measure knowledge retention related to D07. Verifiable items: Baseline / end-line surveys. Training reports. Semi-structured key interviews Implemented by: National Program Manager Evaluating consultant agency Semi-structured interviews were conducted at the baseline and involved representatives of all partners and stakeholder organizations, including ULEAM, SGR, MIDUVI, and community councils. These interviews were conducted in person or via Zoom, depending on the interviewee’s preference Verifiable items: Discussion guides aimed at understanding the contextual information related to indicators D02, D03, D15, D16, and C02. The people interviewed will be selected by a non-probabilistic purpose sampling for each organization and gender, age, and geographic area See description of indicators in Table No.19 Focus Group Baseline and end-line focus groups can be carried out using structured discussion guides related to the implementation of training, preferred campaign channels/messages (D01, D02, D03, D04, D05), and the status of DRR strategies. 38 | FINAL EVALUATION OF EBRP III Implemented by: National Program Manager Evaluating consultant agency Verifiable items: The discussion guides were adapted according to the participants, and all discussion guides will be sent with the initial and final report, with personal information removed from it. These focus groups were conducted in person, at least for the baseline. Direct observation Implemented by technical experts from Miyamoto and ULEAM Consultant agency Direct observation was used to corroborate information from the monitoring checklist, specifically for Indicators D01, D02, D03, and D04, to observe public awareness campaign activities, and during participation in events, meetings, technical follow-up visits, and support. Validate attendance records for indicators related to the implementation of training and discussions, including D06 and C02. Verifiable items: Semi-annual reports, which may include photos, videos, press clippings, and social media links or analysis. See description of indicators in Table No.19 Source: Monitoring and Evaluation Plan for the EBRP III program FINAL EVALUATION OF EBRP III | 39 ANNEX IV: DATA COLLECTION TOOLS AND INTERVIEWS OF OFFICIALS INTERVIEW GUIDE FOR TECHNICAL EXPERTS AND OFFICIALS PROGRAM EVALUATION ECUADOR BUILDING RESILIENCE PROGRAM (EBRP III) GENERAL INFORMATION - Name - Age - Gender - Position, role or relationship with the project - Institution of community: GAD •Universities •Cooperation •Community - Name of institution or community - Work time/relationship with the project QUESTIONS 1. What was the context in which the program was developed? 2. Do you consider the program relevant to the needs of the population? Why? 3. What were the main lessons learned from the program? 4. Did the project contribute to a different use of resources (human, technical, or material)? 5. Why? 6. What was the project’s contribution to generating standards, processes, and policies? 7. What continuity actions will be carried out by your institution? 8. What impacts were attained with the project? 9. What would you improve in new interventions? 10. Comments and suggestions INTERVIEW GUIDE FOR FOCUS GROUPS – COMMUNITIES PROGRAM EVALUATION ECUADOR BUILDING RESILIENCE PROGRAM (EBRP III) GENERAL INFORMATION List of participants - Name - Age - Gender - Positioning the community 40 | FINAL EVALUATION OF EBRP III - Districts •Alhajuela •Crucita •Abdón Calderón •Chirijos •Pedernales - Community - Time of participation in the Project (month/year) QUESTIONS 1. How were the participants selected for training? (Criteria) 2. What changes were achieved at the individual level? (Describe) 3. How did these trainings contribute to community development? 4. What messages do you remember from the campaigns carried out? 5. What lessons have you been able to replicate in your community? 6. What difficulties did you encounter in the project? 7. What are the future steps to replicate the project? 8. Remarks – comments FINAL EVALUATION OF EBRP III | 41 ANNEX V: SOURCES OF INFORMATION No. DESCRIPTION LINK 5.1 List of participants: people who were interviewed during the program evaluation. https://bit.ly/3Lk7mg8 5.2 Breakdown of contributions made to the program by partner institutions and organizations. https://bit.ly/3TnJ8ni 5.3 Verifiable program items: it contains the database and all the supporting information generated during the implementation of the three purposes of the EBRP III program. EBRP III Deliverables 42 | FINAL EVALUATION OF EBRP III ANNEX VI: DECLARATION OF CONFLICTS OF INTEREST The signed conflict of interest declarations of evaluation team members are available at the following link: https://bit.ly/3LwiJBT FINAL EVALUATION OF EBRP III | 43 ANNEX VII: MEMBERS OF THE EVALUATION TEAM Consultant: Sayla Yánez Alvarado, Sociologist, Central University of Ecuador. Master’s Degree in Latin American Cultural Studies, Simón Bolivar Andean University. Technology in Personal Development –Online Modality– CRE-SER Institute of Personal Development. Social researcher. Diploma in Gender and Poverty, ILO - Italy, online modality. Specialization seminars in social management, project design, gender, culture, and social organizations. Consultant. Twenty years of experience in international cooperation and designing, implementing, monitoring, and evaluating programs and projects. Teacher. Coordinator and facilitator of the School of Solidarity-based Social Economy. Consultancy and work with indigenous populations, Afro-Ecuadorians, and peri-urban sectors. Speaker at international events on cooperation, development, and social organization. Facilitator of strategic planning, continuous improvement, and total quality processes. Advisor to women’s organizations and in public policy design. Facilitator in leadership, human rights and development, organization, and education in values and principles. Trainer in schools for parents and couples, and workshops for young people in the church office for social care. Management of methodologies and techniques of popular education and communication. Published didactic guides for training organizations. Assistant: Sofía Daniela Saltos Yánez, organizational psychologist, Pontifical Catholic University of Ecuador. Experience in human talent selection and training. Training courses on database management and report writing. Researcher in social and community welfare studies. Collection of essential information and preparation of management and administrative reports. Assistant: María Dayana Álava Peñafiel, Bachelor’s Degree in Library and Information Sciences, Technical University of Manabí. Experience in conducting surveys. Knowledge of computer sciences and telecommunications. Diploma in Educational Inclusion. Collection of baseline information and interviews. 44 | FINAL EVALUATION OF EBRP III ANNEX VIII: STATEMENT(S) OF DIFFERENCE Not applicable. FINAL EVALUATION OF EBRP III | 45 ANNEX IX: PHOTOGRAPHS OF THE EVALUATION Other information is available at the discretion of the evaluation teams and USAID and, in some cases, of implementing partners. Image 2 Interview with Sandino León, Portoviejo GAD technical expert Image 3 Interview with Héctor Cedeño, architect, dean of the Faculty of Engineering and Architecture, ULEAM 46 | FINAL EVALUATION OF EBRP III Image 4 Interview with Fabricio Ponce Vélez, economist, President of Alhajuela GAD Image 5 Interview with Wilmer Guillén, La Mocorita community leader Image 6 Interview with Dolores Saltos, Alhajuela district spokesperson FINAL EVALUATION OF EBRP III | 47 Image 7 Quebrada de Guillén focus group Image 8 Community footprint in the Quebrada de Guillén community Image 9 Entrance to land where guadua cane is grown. El Hormiguero community 48 | FINAL EVALUATION OF EBRP III Image 10 Guadua cane tourist destination, Mocorita Community Image 11 El Achiote community focus group Image 12 Focus group participants from La Mocorita Community; location: San Roque - La Esperanza community center FINAL EVALUATION OF EBRP III | 49 Image 13 Visit to the Sembrando Esperanza Association – Pedernales Image 14 Treated guadua cane in the process of drying in the Production Center of the associations of artisans and bamboo builders of the Pedernales canton. ASOPROASE and ACERO VEGETAL Image 15 Plants from the nursery of the Sembrando Esperanza Association - Pedernales I Note: The photographic record of the EBRP III program with its three purposes can be seen in the following link: http://bit.ly/3ZP77y3 50 | FINAL EVALUATION OF EBRP III [Provide contact information on the back cover. This can be USAID headquarter information, or specific to the USAID mission or operating unit that commissioned the report.] U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523