This publication was produced at the request of the United States Agency for International Development. It was prepared independently by an independent evaluation consultant, Mildred Guerrero Echegaray, hired by Miyamoto International, Inc. FINAL EVALUATION PREPARE MEXICO JUNE 2023 FINAL EVALUATION OF PREPARE MEXICO June 2023 Award Number: 720FDA20GR00086 DISCLAIMER: The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development of the United States Government. TABLE OF CONTENTS ACRONYMS.............................................................................................................................2 EXECUTIVE SUMMARY.........................................................................................................1 EVALUATION PURPOSE ......................................................................................................4 BACKGROUND.......................................................................................................................5 EVALUATION METHODS AND LIMITATIONS................................................................7 LIMITATIONS .............................................................................................................................................................8 FINDINGS, CONCLUSIONS AND RECOMMENDATIONS .............................................9 FINDINGS....................................................................................................................................................................9 RELEVANCE AND ADAPTABILITY............................................................................................................9 EFICCIENCY ....................................................................................................................................................12 PARTICIPATION ............................................................................................................................................15 EFFECTIVENESS ..............................................................................................................................................17 SUSTAINABILITY............................................................................................................................................33 IMPACT .............................................................................................................................................................35 LESSONS LEARNED ......................................................................................................................................37 RECOMMENDATIONS..........................................................................................................................................39 ANNEXES...............................................................................................................................41 ANNEX I. ...................................................................................................................................................................41 TIMELINE OF THE EVALUATION ............................................................................................................41 ANNEX II. ..................................................................................................................................................................42 EVALUATION STATEMENT OF WORK ................................................................................................42 ANNEX III. .................................................................................................................................................................54 TOOLS FOR COLLECTING INFORMATION FROM PRIMARY SOURCES .................................54 ANNEX IV..................................................................................................................................................................60 SOURCES OF INFORMATION..................................................................................................................60 ANNEX V...................................................................................................................................................................61 USAID DISCLOSURE OF REAL OR POTENTIAL CONFLICT OF INTEREST FOR EXTERNAL EVALUATION TEAM MEMBERS ................................................................................................................61 ANNEX VI. ...............................................................................................................................................................63 CV OF CONSULTANT RESPONSIBLE FOR THE EVALUATION ...................................................63 2 | EVALUATION OF THE PREPARE MEXICO ACRONYMS ARISE Private Sector Alliance for Disaster Resilient Societies BHA Bureau of Humanitarian Assistance CDMX Mexico City (Spanish acronym) CENAPRED National Center for Disaster Prevention (Spanish acronym) CETI Industrial Technical Education Center (Spanish acronym) CICEJ State College of Civil Engineers of Jalisco (Spanish acronym) CMIC Mexican Chamber of the Construction Industry (Spanish acronym) COMICIJ Metropolitan School of Civil Engineers of Jalisco (Spanish acronym) CONOCER National Commission for the Certification and Standardization of Labor Competencies DAMAHR College of Engineers and Architects of the State of Oaxaca (Spanish acronym) DAC Development Assistance Committee DDA Detailed Damage Assessment DFRPS Final Disposal of Post-Earthquake Waste (Spanish acronym) DIF Integral Family Development System (Spanish acronym) DRM Disaster Risk Management DRR Disaster Risk Reduction ISC Construction Safety Institute (Spanish acronym) IIEG Institute of Statistical and Geographic Information (Spanish acronym) IMEPLAN Institute for Planning and Development Management (Spanish acronym) ITESM Monterrey Institute of Technology (Spanish acronym) MI Miyamoto International OECD Organization for Economic Co-operation and Development PDA Preliminary Damage Assessment PREPARE Preparing to Lessen the Social and Economic Impact of Earthquakes in Mexico RDA Rapid Damage Assessment SEMADET Secretariat of Environment and Territorial Development (Spanish acronym) SGIRPC Secretariat of Integral Risk Management and Civil Protection (Spanish acronym) UNAM The National Autonomous University of Mexico (Spanish acronym) USAID U.S. Agency for International Development FINAL EVALUATION OF PREPARE MEXICO | 1 EXECUTIVE SUMMARY The PREPARE MEXICO program, Preparing to Lessen the Social and Economic Impact of Earthquakes in Mexico, follows several years of Miyamoto International's collaboration with various government and private sector partners in Mexico City and Zapopan to strengthen the capacity of the disaster risk response system. The program started September 30th, 2020, with an initial duration of 24 months that was extended to 33 months, and with the overall objective to: Strengthen and institutionalize the risk management and response capacity of public disaster authorities and their private sector partners, in earthquake-prone and highly vulnerable urban environments. The program was structured around three specific objectives. The first two, designed to deepen the impact and give sustainability to the results achieved in previous years, were: Facilitate the incorporation of risk management and response best practices, strategies and tools into public policy and planning processes; and Strengthen public-private collaboration in the application of risk management and response tools. The actions planned within these two objectives were highly relevant, as they were designed with the participation of local partners with whom the program had worked with in previous phases, and therefore responded to their mandates, needs and priorities. The third specific objective of the program, which sought to increase knowledge of basic seismic-resistant construction techniques among the vulnerable population of high-risk urban settlements, was incorporated as a pilot project, in recognition of the fact that it is the lower-income urban population who build their homes directly without the intervention of professionals or in compliance with safety norms or standards, given the state's inability to control these constructions. To ensure the relevance of this component, it was implemented in partnership with the local government agencies of Zapopan that work directly with the population of vulnerable settlements, among which they are recognized and accepted, and adapted the strategies and contents of the workshop on basic seismic-resistant construction techniques to the physical characteristics of the specific context and to the construction terminology used locally. The COVID-19 pandemic forced the program to make adaptations in the modalities used in all the products and processes that were part of the program, with very good results in the case of the community workshops, which had to reduce their capacity to 30 people, thus achieving a strong exchange among the participants, which produced a better understanding of the contents taught. However, the pandemic also caused significant delays in the start of the program since several of its local institutional partners were engaged in dealing with this health emergency. This delay, plus the slowness of the work processes with government officials, forced the program to extend for 10 additional months, ending in June of 2023. The program was very effective in taking advantage of the experience and internal knowledge of its local partners, both in the processes led directly by Miyamoto and in those entrusted to specialized external consultants. Miyamoto took advantage of the knowledge and experience through the establishment of technical spaces for joint, inter-institutional and interdisciplinary work and reflection, in which all public and private organizations and institutions linked to the development of products were given a decisive space for participation, both because of their legal mandate and their contextual knowledge. However, the available financial resources could have been used more efficiently in the third program component, where the coverage of the population participating in the community workshops on seismic-resistant construction could have been considerably expanded. In all the program's activities, there was broad participation of local partners and beneficiaries, and the incorporation of the most vulnerable population groups was ensured, such as women and single mothers. For this purpose, mechanisms were established that allowed their almost parity and equal participation in the earthquake-resistant construction workshops. The program's technical implementation team was able to exercise close, empathetic, and proactive leadership to obtain relevant results adapted to the realities of the context, with a high level of ownership by the final beneficiaries. However, it was not possible to generate, among the various participants, networks that 2 | FINAL EVALUATION OF PREPARE MEXICO would allow for the follow-up and socialization of the program's achievements and products, to support their sustainability. The program was effective in achieving its three specific objectives, which were fully met. Under Objective 1, the program worked with the mayors offices of Xochimilco and Benito Juárez in Mexico City, and the municipality of Zapopan in Guadalajara, in the collegiate construction and validation of the protocols and procedures needed to effectively implement post-earthquake damage assessment in buildings, in the development of a mobile application to record and manage the information collected in the assessments, and in the formulation of a methodology for locating sites for the disposal of large volumes of solid waste debris caused by a large earthquake, as quantified in the seismic vulnerability assessment of the city of Zapopan prepared in the previous phase of the program. Under Objective 2, the program worked with private professional associations in CDMX and Guadalajara, to strengthen their capacities in construction damage assessment, training approximately 1,500 professionals and instructing trainers within these associations, leaving them with the capacity to continue training their associates. With the participation of several universities and technological institutes, a teaching program was also developed on the “Introduction to Post-seismic Damage Assessment in Construction in the Risk Management environment,” which has already been introduced in 11 academic institutions. Under Objective 3, 176 inhabitants of vulnerable communities in Zapopan were trained in basic seismic-resistant construction techniques through practical workshops held in the communities themselves, where all participants were able to directly experience the notions learned and received printed manuals with all the information shared. All the products developed within the framework of the program were formally handed over to the agencies in charge of their implementation, which took full ownership of them. Although, there are still major obstacles that limit their contribution to the achievement of the general objective, regarding the institutionalization of the procedures and tools created. Political tensions, both internal and international, have had an impact on weakening collaboration with the local government agencies in Mexico City that participated in the program and are responsible for providing continuity to its results, which puts the institutionalization of these products in these agencies on hold. On the other hand, differences in government structures and power struggles among the different municipalities that make up the Zapopan metropolitan area make it difficult to adopt unified tools and procedures among all of them, which seriously compromises the disaster response capacity of the city. The lack of administrative continuity in governmental entities, and of a long-term political will and vision, makes it difficult to take advantage of the methodology developed for the location of debris disposal sites, which requires authorities to make decisions and invest resources in the expropriation of the necessary land, to manage the large volumes of debris that a big earthquake would produce in Zapopan, to reduce the social and economic impact on the city. In the program’s second component, although the technical capacity of the private sector to support the government in post-disaster damage assessments was significantly strengthened, the necessary mechanisms to link this installed capacity with the decision-making bodies was not established, to be able to make effective use of it in the event of an emergency. In the universities, the adoption of the courses developed was made difficult by the complexity and slowness of the bureaucratic processes necessary to make changes to the official contents of careers linked to construction. Despite the great interest shown by the beneficiary organizations in giving the program results continuity, and the emphasis placed on generating technical and operational capabilities to replicate the knowledge acquired and to use and adapt the procedural and technological tools developed, the sustainability of the results is limited due to the aforementioned obstacles. Therefore, the most sustainable results were the ones achieved with more autonomous and flexible organizations, such as professional associations, which have greater agility to incorporate changes and make decisions, and with the population of vulnerable communities, who have total autonomy in their decision making with respect to the construction of their homes, as they are free from the control mechanisms of the state. FINAL EVALUATION OF PREPARE MEXICO | 3 Therefore, the result obtained in the third component of the program had the greatest potential for impact, to the extent that its coverage can be expanded, since a very high percentage of the population builds their homes on a day-to-day basis without the participation of professionals or adherence to technical standards. While the impact of the other components depends on institutional variables that are not assured and may take many years to occur, the training of the population in safe construction techniques can be put into practice from the very moment the individuals participate in the workshops, as shown by the findings of this study. To the extent that this workshop could be generalized to reach large contingents of the population of these sectors, its impact on the increase in construction standards, and therefore in the resistance capacity of buildings to earthquakes and other hazards, would be very high and very significant in social and economic terms. The following are the main recommendations to deepen the impact and sustainability of the results: For the authorities: • Create a permanent training program for communities in basic seismic-resistant construction techniques, relying on agencies that work directly with the communities and on universities to teach the course, and link it to assistance programs for the delivery of construction materials, to increase its impact. • Establish communication and information management mechanisms between universities, professional associations, and government agencies to strengthen their joint responsibility, harmonize training with the tools used by the institutions and regulatory changes, and have an updated record of the human resources trained to intervene in an emergency. • Implement and further develop the RDA mobile application, incorporating functions that allow for the management and coordination of humanitarian organizations involved in emergency response. For Miyamoto: • Estimate the monetary value of the impact of each of the proposed interventions actions to convince the government agencies called upon to participate in them, since it is a very useful piece of information both for the justification and for the appropriation by their beneficiaries. • Deepen participatory diagnostics in the intervention design stage, to improve the Theory of Change by taking into account uncontrollable risks and threats, and to generate a system of objectives, results and indicators that is better adapted to the reality of the context. 4 | FINAL EVALUATION OF PREPARE MEXICO EVALUATION PURPOSE The main purpose of this final external evaluation was to assess the progress achieved in the implementation of the PREPARE Mexico program (Preparing to Lessen the Social and Economic Impact of Earthquakes in Mexico), based on the expected results and objectives. The evaluation covers the 33 months of the program, between September 2020 and June 2023, and takes into account all its objectives, results and indicators, whether planned or not. In addition, it assesses the incorporation of the strategic principles underpinning the intervention, such as: broad stakeholder participation to favor inter-institutional collaboration and the construction of public-private networks, promoting the balanced participation of women, the elderly and other marginalized groups; the use of local and international experience for the exchange and transfer of technical knowledge, leveraging lessons learned and Miyamoto International's experiences in response to more than 100 earthquakes in the last 30 years; and integrated assessment, planning and technical assistance to optimize the impact and costs of the intervention through information exchange and mutually reinforcing coordinated actions. The evaluation also identifies impacts, lessons learned, and proposes strategic and operational recommendations that will contribute to the sustainability of the results achieved. The specific objectives of the evaluation were as follows: 1. Evaluate the effectiveness and impact of program activities according to the established objectives and indicators. 2. Identify and document the lessons learned from the various activities and processes undertaken, as well as the impact of the program and its unplanned results, both positive and opportunities for improvement, and to identify the program's sustainability measures. FINAL EVALUATION OF PREPARE MEXICO | 5 BACKGROUND Mexico ranks eighth in the world in terms of its exposure to natural and human-induced hazards, including seismic hazards, according to the World Bank. 1 In recent years, significant efforts have been made in the country to educate the population about disaster risk and to promote the political, legal and institutional reforms needed to strengthen risk management and emergency response capacity, such as the harmonization of building codes with international standards, the development of detailed risk atlases for most urban areas, the development of earthquake response plans, the hiring of risk management professionals in municipal and national civil protection offices, and the establishment of an extensive network of collaboration with technicians and experts from universities, professional associations, the business community, and civil society. However, this emergency response system still has profound weaknesses, as was demonstrated during the 2017 earthquake in Mexico City. There, the experience lived in the assessment of the damage suffered by buildings as a result of the earthquake, for which a significant number of professionals and students of engineering and architecture careers were mobilized, evidenced a series of problems that prevented the proper functioning of these assessments and emphasized the need for coordination and training prior to disasters, in terms of methodologies, protocols and tools to be used, between civil society professionals and responsible government agencies. The main problems identified in these damage assessment processes were 2: • Inadequate coordination between the different emergency response actors • Lack of a clear methodology for collecting damage assessment information • Lack of adequate training of individuals conducting the assessments, which led to contradictory and unreliable results and recommendations • Lack of standardization of the information collected • Duplication of activities and information • Lack of a centralized and reliable information system for decision making by the authorities • Slow and inefficient delivery of assistance and aid to affected persons On the other hand, in the municipality of Zapopan, which is part of the Guadalajara Metropolitan Area, the second largest in the country, there have been no earthquakes of considerable intensity for more than 100 years. Nevertheless, it also presents a high exposure to seismic risk, which was measured in the risk assessment study of the built environment conducted by Miyamoto in the previous phase of the PREPARE program. This study, which used probabilistic analysis methodologies, estimated that about 60% of the buildings in the municipality could be damaged by a major earthquake, generating an estimated 2.5 million cubic meters of construction debris whose final disposal would require significant foresight and management measures by the competent authorities. The high exposure to seismic risk in Mexico is closely linked to the processes of disorderly and accelerated urban expansion and to the low technical standards of self-construction and self-produced housing practices carried out by large segments of the lower-income population. Lacking access to other urban housing options, this population occupies land on the outskirts of large cities, often unsuitable for construction, building settlements that are highly vulnerable to natural hazards, without the authorities having the capacity to supervise these constructions and ensure compliance with urban planning regulations and safe construction standards. Since 2017, Miyamoto International has been working with local government and private sector partners to strengthen disaster risk response system capacities through two strategies: In Mexico City the focus has been on strengthening government and private sector capacity to conduct post￾earthquake structural damage assessments, supporting SGIRPC and ISC in the development and incorporation of RDA and DDA tools and methodologies into disaster response policies, and training 1 Miyamoto International: USAID/OFDA May 2020, Technical Proposal. Preparing to Lessen the Social and Economic Impact of Earthquakes in Mexico. 2 USAID/BHA Prepare Mexico: Baseline Report. 6 | FINAL EVALUATION OF PREPARE MEXICO public and private sector professionals in the application of these tools and methodologies. In Zapopan, the program has assisted the relevant agencies in the technical training of personnel from the institutions of the response and risk management system, the incorporation of the most relevant findings of the seismic risk assessment conducted in 2019 within the framework of plans and policies, the development and validation of the RDA tool, and the study of the legal and regulatory municipal framework for the development of a plan for the disposal of debris produced by natural disasters. The current phase of the Prepare program started in September 2020, with an initial duration of two years that was extended to 33 months, and with the overall objective to: Strengthen and institutionalize the risk management and response capacity of public disaster authorities and their private sector partners in earthquake-prone and highly vulnerable urban environments. The direct beneficiaries of the program were: 540 public and private actors in the risk management and emergency response sector, through technical discussions, planning and training activities to operationalize risk management strategies and tools, and 150 heads of households and semi-skilled masons from vulnerable communities, trained in seismic-resistant construction techniques in Zapopan. Indirect beneficiaries were estimated to be around 3,727,500 people living in high-risk areas of Mexico City and Zapopan. To contribute to the achievement of its general objective, the program was structured around three specific objectives, framed within the Risk Management Policies and Practices Subsector, which were: • SO1: Facilitate the incorporation of good practices, strategies and tools for risk response and management in public planning processes. • SO2: Strengthen public-private collaboration in the application of risk management and response tools. • SO3: Increase knowledge of basic seismic-resistant construction techniques among the vulnerable population of high-risk urban settlements. The main activities developed within the framework of these objectives were: 1. Development of a planning framework to operationalize the use of RDA and DDA technical tools and resources, developing protocols and procedures for their implementation at the municipal level, and implementing earthquake response simulations to validate them. 2. Development of a digital platform for the collection, organization, and analysis of rapid and detailed damage assessment data. 3. Support the training of a greater number of engineers and architects in the application of RDA and DDA methodologies and tools. 4. Strengthening the operational capacity of debris management in Zapopan, through the development of a site selection methodology for the disposal of solid waste generated by earthquakes. 5. Development of a Structural Earthquake Damage Assessment curriculum to be included in architecture and engineering programs at local and national universities. 6. Implementation of training workshops on proper construction techniques for heads of households and masons from low-income communities in Zapopan, as a pilot activity on a limited scale, which will be evaluated for full transfer to municipal authorities for continuation. The program implementation model was based on the following strategic principles: Broad stakeholder participation, with the objective of strengthening inter-institutional collaboration and public-private networks, through joint work in the design, planning and implementation of activities aligned with their institutional roles and mandates; Leveraging in-house expertise to share and transfer technical knowledge and training, using lessons learned by Miyamoto International experts in their intervention in over 100 earthquakes over the past thirty years, and the organization's management structure in overseeing and supporting the program; and Integrated assessment, planning and technical assistance, which improve understanding of seismic risk to optimize impacts and resource utilization through coordinated actions that share information and reinforce each other. FINAL EVALUATION OF PREPARE MEXICO | 7 EVALUATION METHODS AND LIMITATIONS The theoretical and methodological framework of the evaluation was guided by the criteria-based evaluation model 3 , complemented by two evaluative approaches: 1) systemic evaluation, which describes how the design and implementation of any intervention or system translates into results through a chain of causal relationships, and 2) democratic evaluation 4, which is a participatory process centered on the organizations and which is highly attuned to the cross-cutting approaches of the intervention. Based on this triple approach and on secondary documentation, we were able to define the logical skeleton of the project, the assumptions that comprised it, the factors that influenced it and its sequential cause-effect logic. The evaluation questions were the backbone of the final evaluation because they delimit the dimensions to be investigated and assessed. These were taken from the information provided in the terms of reference, and were complemented with others, arising from the document review and the needs and expectations of the program's technical team, gathered through consultation. These were used to prepare the evaluation matrix (see Annex 2), which relates each question with the sources of information to answer them, and with the methods for collecting and analyzing the information. The phase of collecting and analyzing information from primary and secondary sources was carried out between May and June 2023. For the collection of information from primary sources, a qualitative sample of 25 people representative of the different stakeholders was chosen, which does not seek statistical representativeness but rather the potential of the process of theoretical understanding of the evaluated intervention. To achieve this, the depth of the analysis is a key element, which rules out covering the number of key informants that would give statistical representativeness to the evaluation, as it would be unfeasible to implement that with the available resources. In the qualitative methodology used, the sample was sufficient to respond to the objective of the study, as their testimonies served as an element of contrast of the theory underpinning the program and the data collected in its monitoring system with the empirical reality, but not so large as to prevent an in-depth analysis. A total of 22 of the 25 interviews requested were conducted (see Annex 1), with people from the implementing technical team, and representatives of public and private organizations that are members of the municipal DRM system, academic institutions, and vulnerable communities in Zapopan participating in the program. The information obtained from the analysis of secondary sources was triangulated with the verbalizations issued by the key informants, who had a committed participation, providing high quality information. The merging of both (interviews with facts), led to the identification of argumentative ruptures in the interviews given in terms of "should be", especially present in the evaluative processes. The analytical process was carried out by contrasting the descriptive narratives of the informants with secondary information capable of ordering milestones, stages, and actors, and on the other hand, by superimposing this descriptive narrative with intersubjective verbalizations, coming from the (explicit or underlying) assessments of the interviewees. In addition, it was possible to obtain quantitative data from the project's secondary sources (monitoring and evaluation), to which a descriptive or content analysis of the numerical data was carried out to determine the final state of achievement of the logical framework indicators, their coverage and scope. This analysis made it possible to establish causal hypotheses of an introductory nature, whose value and interpretative sense acquired the dimension of an evaluative finding in its triangulation with qualitative methods. 3 The evaluation criteria of the Development Assistance Committee (DAC) of the Organisation for Economic Co-operation and Development (OECD) were agreed in 1991. This document contains the five evaluation criteria that have been used over the last 25 years: relevance, effectiveness, efficiency, impact and sustainability. These criteria have served to homogenize evaluation practice, contributing to learning and improvement in international cooperation for sustainable development. 4 Evaluative model created by Barry Mac Donald (1967) or "democratic approach", by democratizing the use of knowledge, it encourages informed participation and accountability of the different audiences. 8 | FINAL EVALUATION OF PREPARE MEXICO LIMITATIONS The political situation of the current electoral campaign in Mexico has produced administrative changes and in the public agenda of local governments in Mexico City and at the federal level, making it difficult to access key informants such as the municipal government institutions that participated in the program, especially those of the Benito Juarez and Xochimilco municipalities, due to the change of objectives and direction of these governmental entities. On the other hand, Mexico's foreign policy, which has been adopting increasingly rigid positions regarding the international context, has had a decisive impact on the direction of USAID-funded programs and projects. Tensions between the Mexican and U.S. governments have been escalating in recent months, reaching their peak with the letter that Mexican President Andres Manuel Lopez Obrador addressed to U.S. President Joe Biden, accusing USAID of financing organizations opposed to the Mexican government. This marked a turning point from which the collaboration that had been established with the municipalities of Xochimilco and Benito Juarez, as well as with other institutions, was interrupted, affecting their participation in the consultation spaces necessary for this evaluation. This impossibility to consult Mexico City's governmental actors generated an information gap that could not be filled from the other available sources of information, affecting the understanding of the processes implemented and the results achieved, especially in relation to component 1, whose main actors were these institutions. Another limitation was the time factor, since in the course of the intervention, several situations delayed some of the planned processes, so that at the time of the evaluation (one month before the end of its implementation period), there were still very important products that had not been completed. Therefore, they were not able to be evaluated by their recipients, such as some of the post-earthquake damage assessment courses, which had been implemented very recently, and the mobile application for the management of post-earthquake damage assessment data in structures, whose development was delayed for 9 months because CENAPRED did not authorize its implementation at the federal government level. However, the evaluation counted on the receptiveness and collaboration of an important group of representatives of the organizations and institutions that participated in the PREPARE Mexico program, especially those located in Zapopan, where the political and electoral situation has not had such an impact on the program's course. Despite the limitations, it was possible to gather a wide variety of informative inputs, with which it was possible to build a solid base for the confirmation of the identified findings and the elaboration of a well-founded evaluation report that, in line with the democratic evaluative approach, made ample use of the consulted actors' own words to illustrate the value judgments issued. FINAL EVALUATION OF PREPARE MEXICO | 9 FINDINGS, CONCLUSIONS AND RECOMMENDATIONS FINDINGS RELEVANCE AND ADAPTABILITY The evaluation of relevance consists of assessing the appropriateness of the objectives and interventions of the activity to the context in which it was carried out, and its adaptability to unforeseen situations emerging from that context. Its analysis studies the quality of the diagnosis that supports it, judging its correspondence with the needs observed in the beneficiary population.5 In assessing the relevance, the evaluation also takes into account the analysis of the issues to which the program responds to, which was included in the technical proposal submitted in 2020 to USAID/BHA6, and will answer the following evaluation questions: 1. To what extent does the program respond to the priorities and mandates of the institutions and organizations that are part of the DRR system in Mexico? 2. To what extent has the Covid-19 crisis and other situations of the context had an impact on the results achieved, and what adaptation strategies were adopted? 3. How were the contents of the workshops adapted to the characteristics of the construction processes in the informal neighborhoods of Zapopan? 4. To what extent does the program respond to the priorities of the informal neighborhood communities of Zapopan? Is there a perception of seismic risk in these communities? The PREPARE Mexico program emerged as a continuation of the previous phases of PREPARE implemented in the country (and regionally), to operationalize and give sustainability to the results achieved, and was designed with the participation of agencies that are part of the disaster response system, such as the SGIRPC, authorities of the municipalities of Xochimilco and Benito Juarez in CDMX, and civil protection and fire department authorities in Zapopan. These agencies, which are immersed in the process of establishing planning frameworks for risk management and disaster response, participated in the identification and formulation of the program's objectives, actions and strategies. They pointed out the need to develop protocols and procedures for the use of tools developed in the previous phases, such as the rapid damage assessment forms, and systems for recording and managing the information gathered, to avoid contradictions and ensure that it can be used in a timely and reliable manner for decision making by authorities. "What we were able to visualize in the participation we have had in this type of events is that there are situations where the same agency sends to evaluate buildings more than once, and each time by different people, and not even the data from one and others match..." To this end, these agencies submitted in writing their needs and priorities, in addition to their commitment to use their human resources in the program, where they also identified as a priority the need to significantly increase the capacity to deploy damage assessors to meet the expected demands after a major earthquake. This can only be achieved by strengthening the collaboration of government agencies with private organizations such as universities and professional associations, which are the ones that have the necessary experts to perform these assessments. In the case of Zapopan, the findings of the seismic risk assessment conducted in the previous phase, which quantified a debris volume of approximately 2.5 million cubic meters, evidenced the need to develop an effective debris management plan that includes institutional roles, mandates, and resources, that is beyond the current technical and budgetary capacities of the municipality. This led to the inclusion of the methodology for selecting debris disposal sites as one of the main products of the program. 5 OECD DAC Evaluation Management Manual. 6 Miyamoto International: USAID/OFDA May 2020, Technical Proposal. Preparing to Lessen the Social and Economic Impact of Earthquakes in Mexico. UNDERSOFT COMPANY INFORMANT 10 | FINAL EVALUATION OF PREPARE MEXICO "In the scenario used for the seismic risk assessment, approximately 2.5 million cubic meters of debris were generated. To implement the logistics to handle that volume, we needed to develop a protocol and procedures to select the disposal areas that met the requirements to avoid massive contamination from the disposal of this debris. That's how we saw the importance of continuing in this second stage." Recognizing, on the other hand, the incapacity of the authorities and the insufficiency of public resources to control the self-construction of housing in irregular settlements, which are the most vulnerable and fastest growing sectors in the cities, led the municipal authorities of Zapopan to consider the need to train the population that builds their homes or directly manages their construction, in adequate and seismic-resistant construction techniques, to increase the quality and safety of informal construction. As a result, it was decided to include this component in the program, taking advantage of the lessons learned from a similar initiative previously carried out by Miyamoto in Ecuador and the knowledge acquired by the organization in its work with construction of irregular settlements in countries such as Haiti. The materials used and techniques taught were adapted to the context of the Guadalajara region, the characteristics and main flaws of self-built constructions, and the terms and language used in the communities, with the participation of local experts from the Jalisco College of Civil Engineers. "People start building with wood and flimsy material and then replace it, but without laying a proper foundation. In the work done with MI, although not a large number of people were trained, there was a great deal of interest, especially from housewives. Although they are not necessarily the ones who are building their houses, gluing bricks, they are the ones who sensitize their husbands and family members to use more appropriate techniques and seek the support and advice of specialists in the field." The various unforeseen situations of the context that arose during the implementation of the program influenced its results and conditioned the execution of the planned activities, requiring some adaptations to be made. The COVID-19 pandemic caused delays in the start of the program, due to the high level of dedication that local actors such as the Civil Protection and Fire Departments, directly involved in public health and safety issues, were providing at that time, which reduced their availability to participate in the programmed activities. The health protection measures prevented the holding of face-to-face meetings and courses and the uncertainty about the duration of these measures, forced the team to make adjustments, such as the design of the academic curriculum of rapid damage assessment with two modalities, one face-to-face and the other virtual, which resulted in much more work than what was originally planned for the development of this product. In the community workshops on seismic-resistant construction techniques, with a high practical content which could only be implemented in person, the restrictions due to the pandemic made it difficult to respond to the calls made, due to people's fear of leaving their homes to attend meetings, requiring home visits to explain the importance of the course, and the provision of all the protective measures demanded by the health authorities. These measures also forced the reduction of the capacity of the groups to a maximum of 30 people, when originally, they were to be implemented with groups of 100 people, as had been done in other countries, which had a very positive influence. This allowed for greater interaction between students and instructors, resulting in a better adaptation of the contents and methodologies to the conditions and needs of the attendees, and a better understanding and fixation of the topics covered. The convening process was also difficult due to the lack of interest of the communities in the seismic issue, since in Zapopan this type of event has not occurred for many years, so there is no perception of this risk among the communities. The topics addressed in the workshops, however, are highly relevant within the processes of social production of housing carried out by the lower income sectors of the population of the city of Zapopan, since they were not only focused on the measures of ZAPOPAN CIVIL PROTECTION COORDINATION INFORMANT ZAPOPAN CIVIL PROTECTION COORDINATION INFORMANT FINAL EVALUATION OF PREPARE MEXICO | 11 resistance of buildings to high intensity earthquakes, but also addressed the risks that most frequently affect people living in these irregular settlements, such as floods or landslides, and the measures that the inhabitants can implement to avoid them, from the very moment they choose the site where to build their homes. Although the processes of social production of housing in irregular settlements have in common the non-compliance with safe construction standards due to the lack of control and the absence of professional intervention, they are very heterogeneous in their execution modalities, ranging from pure self-construction, carried out entirely by the members of the families, to the contracting of local masons managed by the contracting families, which is called self-production. In this range of options, which can be interspersed in the successive stages of growth of a home depending on the family's investment capacity at each of these moments, the family has a preponderant role in decision making, from the choice of the land and the location of the home to the purchase and storage of materials, the definition of the spaces, the hiring and management of the labor force, etc. Therefore, in order to advance towards the set objective of raising quality standards in the social production of housing, it was very pertinent that the workshops on earthquake-resistant construction techniques be aimed not only at masons or people working in the field of informal construction, but also at all members of the community, since any of them at some point in their lives, as heads of households, will exercise management roles in the construction of their homes. "All the people who attended the courses have a need to expand, improve or build their homes, to a greater or lesser extent. So, it is obvious that sooner or later they will use what they have learned. What we are looking for is to create awareness of what they must observe. For example, that they observe that the space for the foundation is free of garbage, that they do not build in ravines or on hillsides, that they do the heel test to see that the soil is not filled, that they have the correct reinforcement of the columns, the correct angle in case of a seismic event so the column does not open. Dr. Kit Miyamoto has it very well studied and says: just putting a couple of rods there, with concrete, makes all the difference in a seismic event." The call made in the communities to attend these workshops placed emphasis on attracting female participation in these courses. Although construction has traditionally been a male occupation, the high incidence of single-parent households headed by women in these areas makes this emphasis on female participation very relevant. Since single mothers are the least likely to be able to improve or expand their homes, both because they are unfamiliar with the construction trade and because they lack the resources to hire labor to do so, the basic construction skills taught in these courses are very important for them. Even in cases where they do hire labor or where their husbands or other family members participate in the construction of their homes, women are the ones who are most concerned about ensuring a safe shelter for their families, and they are often the ones who manage and direct the expansion and improvement of their homes. The call, which was led by the Colmenas and the DIF, proved to be well adapted to the specific cultural and social context of the intervened sectors, since it achieved an almost equal participation in terms of gender, as was its objective. A contextual situation not foreseen in the design of the program, which had an important impact on the planned activities, was the drift of the country's foreign policy which, as already mentioned, evolved towards increasingly rigid positions towards the United States and specifically towards its international cooperation agency, USAID, the financial partner of this program. This generated delays in the approval of certain processes by the competent authorities, as happened with the mobile application for the collection and management of data in the post-earthquake damage assessments, which was originally going to be developed jointly with the federal government. Given its refusal, after nine months of waiting, it was developed at the last minute with the participation of the municipal government of Zapopan, reducing its potential impact and decreasing the time available for its implementation and evaluation within the time frame of the program. The influence of the country's internal politics also proved to be more important than was considered in the contextual risk analysis of the intervention. In this design, which counted on the active and MIYAMOTO'S TECHNICAL TEAM INFORMANT 12 | FINAL EVALUATION OF PREPARE MEXICO committed participation of the institutional actors from the previous phases of the program, the threat that the ups and downs of partisan politics, such as the beginning of the electoral pre-campaign for the 2024 elections which could produce significant changes, even within the same governmental period, was not sufficiently measured. These changes have taken place in recent months, with high-level officials of the Xochimilco and Benito Juarez mayors' offices in CDMX leaving their positions to dedicate themselves to the electoral campaign, producing an interruption in their relations with the program and a change of direction in their operational priorities. This called into question the appropriation and continuity of the products and processes developed with their support. Based on this trust in the institutional actors of the moment, no mechanisms were foreseen to counteract this vulnerability, in a country context characterized by political caudillismo, with its consequent weakening of democratic institutionality. Despite these shortcomings, the implementation of the program's actions was characterized by a flexibility to adapt to the emerging needs of its local partners, as manifested with the incorporation of a new component that was introduced at the request of the Coordination of Civil Protection and Risk Reduction of CDMX, which was the equipping of a classroom truck with earthquake simulation equipment, generators, screens and computers for interactive activities, in order to disseminate knowledge about earthquake risk prevention among children and elderly people in communities far from the urban center. EFICCIENCY The analysis of the efficiency of development cooperation projects and actions refers to the study and assessment of the results achieved in comparison with the resources used, and of compliance with the schedule and planning of activities, and answers the following questions: 1. To what extent has the program taken advantage of the internal experience and knowledge led by the country's organizations and institutions? 2. Was the program's distribution of human, material, and temporal resources sufficient and adequate to achieve its objectives? The evidence analyzed allows us to conclude that the PREPARE Mexico program was very efficient and took advantage of the experience and internal knowledge of its local partners, establishing technical spaces for joint, inter-institutional and interdisciplinary work and reflection, with the participation of each of the organizations and institutions linked to the processes and products developed, both for their legal mandate and technical knowledge. "We had many meetings from the beginning, it was very complicated to link so many institutions and organizations involved in the issue, and I think that the efforts were able to take advantage as much as possible of so many different visions from the expertise of each of the parties involved (...) Miyamoto's team did its liaison work so that each institution, agency, organization or academy could join the work and know where the discussion was at and what the objective was." This work methodology, already tested in previous Miyamoto experiences, was used in the development of protocols and procedures for post-earthquake damage assessment, which were developed specifically with each of the participating local governments of Xochimilco, Benito Juarez and Zapopan, taking into account their differences in terms of government structures, actors involved and their legal powers, and resources available to each entity. The program's technical team gathered the initial information and presented the working group with a basic pre-designed protocol structure, with the stages and steps to be fulfilled. Based on this structure, detailed discussions were held on each stage, building a matrix of responsibilities that established the mode and sequence of action of each of the actors involved in the various emergency scenarios envisaged. "What we did was to give them an initial structure of how to organize the work, but they filled in absolutely everything, expressing the way in which they attended the emergency, so that the protocol could be viable, and we were not the ones defining things for them." INFORMANT OF THE INSTITUTE OF STATISTICAL AND GEOGRAPHIC INFORMATION FINAL EVALUATION OF PREPARE MEXICO | 13 Similarly, in the processes led by external specialized organizations, due to their technical specificity, the consultation with the interested parties was maintained at all times, so that the result would not only respond to the needs and requirements of the institutions involved, facilitating their appropriation of these products, but also so that their usefulness would be extended to other potential users, foreseeing in their development adaptation mechanisms for other institutional contexts within the country. The efficiency of the work in these spaces was, however, weakened in some cases, by the lack of a detailed mapping of the actors that had to be present, or because the actors convened were not informed of the specific objectives sought, which led to unnecessary delays: "I feel that time was wasted in the first meetings. It was not clear which actors had to be incorporated, so the work done was lost when they were gradually incorporated (...) The lack of socialization about the program also played a role, we did not find out what it was about until we were there. So there was a lack of socialization, surely some entities or agencies did not attend because they did not know what it was about, nor what they were going there for". In terms of human resources, the program had a robust and interdisciplinary local implementation team consisting of four professionals with backgrounds in civil engineering, public administration, social work, and economics, in addition to the regional management team consisting of Miyamoto's Director and Coordinator for Latin America and the Caribbean, the technical support team of engineers from the California-based team, and the organization's international program, finance, procurement and communications specialists. 7 "The work teams must be robust. In Mexico we were fortunate to have four members. It's good that you can rely on the back office in the California and Washington offices, but it's never the same as having people who support you locally, and people who are experts. I recognize that each and every one of them in their area of expertise is very good. Having an economist like Diego in this group gives us a broader, much more transcendent perspective on things, which helps us see impacts that we would not have seen if he had not been with us." Regarding the use of available resources in the third component of the program, in which workshops on seismic-resistant construction techniques were implemented in vulnerable communities, these could have been used to impact a greater number of people in the communities, despite the fact that the established indicator of 150 people was more than met by reaching 176 people. The most costly and difficult part of this part of the intervention was the elaboration of the scale models of various construction and structural components, and of the printed materials that were left as guides for the participants. These instruments could have been more profitable by increasing the number of people served through the workshops, whose implementation costs were comparatively much lower, thanks to the support of the Colmenas and the DIF, which provided their facilities for the workshops. The efficiency of the calls for participants was not high either, since in most of the nine workshops held the maximum quota of 30 people was not reached, leaving an average of 19.55 participants, as shown in the following table. This fact corroborates what was said by the attendees interviewed for this evaluation, that in their environment there are many people who would have been interested in enrolling, but did not find out about the existence of these courses: 7 Miyamoto International: USAID/OFDA May 2020, Technical Proposal. Preparing to Lessen the Social and Economic Impact of Earthquakes in Mexico. MIYAMOTO'S TECHNICAL TEAM INFORMANT INFORMANT OF THE INSTITUTE OF STATISTICAL AND GEOGRAPHIC INFORMATION MIYAMOTO'S TECHNICAL TEAM INFORMANT 14 | FINAL EVALUATION OF PREPARE MEXICO Figure 1. Number of Participants in Workshops on Seismic-Resistant Construction Techniques8 Workshop Date nº of participants 10/2/2021 20 10/16/2021 10 10/23/2021 31 11/6/2021 13 11/7/2021 8 11/20/2021 14 11/28/2022 13 2/20/2022 39 3/20/2022 30 AVERAGE 19.77 participants 66% of the established quota per workshop Another rather costly activity of the program, in terms of economic resources invested, was the development of the methodology for selecting sites for post-earthquake solid waste disposal, which has not been profitable within the framework of this intervention, but has a very large potential impact, and a possibility of being used by many other territories and governments inside and outside the country. The impact of its use depends, however, on political, economic, and institutional factors that are beyond the scope of this program and are not assured in the immediate future, which may call into question the efficiency of the investment made in it. "The applicability of the methodology for identifying sites for waste disposal has a major challenge, which is that the sites identified as suitable or adequate are privately owned and acquiring them involves legal processes of expropriation or purchase that are complicated.” Regarding compliance with the schedule, the program execution period was marked by exceptional situations, such as the COVID-19 emergency, which affected the work mechanics. But, apart from these, the planning did not adequately calibrate the time needed to carry out work processes with local institutional partners, whose multiple activities and commitments made them difficult to address, and who in many cases had to start by raising awareness of the issues to be addressed, which required more time than expected. The complexity and lack of correspondence with the local reality of the administrative procedures required by USAID for purchases, expenditures and contracting was also a factor in the delay of planned actions: "For example, for purchases over 500 dollars, you need to go through a whole administrative process, look for three suppliers, etc. They suggest that the payment be made directly from the U.S. headquarters to Mexico. They suggest that the payment be made directly from the central office in the USA to Mexico, and you know what an international transfer is, which has a cost, and not everyone has a bank account (...) The per diem in Guadalajara is very cheap and those amounts are not enough to comply with the security issues required by the donor. They are excluding conditions, impossible to meet at the same time". 8 Own elaboration based on: USAID/BHA PREPARE México: Community training, Seismic resistant construction tables. MIYAMOTO'S TECHNICAL TEAM INFORMANT INFORMANT OF THE INSTITUTE OF STATISTICAL AND GEOGRAPHIC INFORMATION FINAL EVALUATION OF PREPARE MEXICO | 15 PARTICIPATION The participation analysis focuses on determining which actors were involved in the different stages of program planning and implementation, assessing their impact on decision making, and answering the following questions: 1. To what extent has the involvement of national stakeholders determined and helped develop the program's outputs and outcomes? 2. To what extent were stakeholders' priorities and needs incorporated in the design of program actions? 3. Did the methodology and strategies implemented allow for the inclusion of the most vulnerable people in the communities? 4. To what extent have the outputs and outcomes included accountability to the affected population? The broad participation of stakeholders to strengthen inter-institutional collaboration and the establishment of public-private networks for the improvement of risk management and disaster response capacity is the first of the strategic principles that cut across all the actions implemented in the program. In its fulfillment, all the processes set in motion and the products developed to achieve the objectives were based on the decisive intervention of a broad group of local public and private organizations involved in the issues addressed, either because of their regulatory role, their knowledge and experience or the importance of their resources to contribute to achieving them. To achieve this active participation, technical working groups were established for the joint design, discussion and validation of the different products developed within the framework of the program, with the participation of a wide variety of entities: bodies from different levels of government, public and private educational institutions, professional organizations, private initiative, all of which have a role to play within the national civil protection system, based on the capabilities of each one of them. "All the products have the virtue that they can be linked and adapted to the needs of the majority of actors who participated in the program, both from the public and private sectors, because they were built with contributions from them, in a collegial manner, and that therefore there is not a dispersed vision but a vision with a direction about what is pursued with those objectives." This same participatory methodology was followed both in the processes led directly by Miyamoto's technical team, and in those that for reasons of technical specificity were conducted by specialized external consultants. The first group was used to discuss the post-earthquake damage assessment protocols, which were developed for the mayor's offices of Xochimilco and Benito Juarez in CDMX and the municipality of Zapopan in Guadalajara, where in each of the three cases the agencies involved in their emergency response procedures, participated in each of the three cases. In Zapopan, the authority for this matter is held by a single entity, the coordination of Civil Protection and Firefighters, which has its own rescue groups and establishes a direct relationship with universities and professional associations for the evaluation of damages. However, since they are part of a metropolitan coordination of the city of Guadalajara, representatives of the neighboring municipalities of Tlaquepaque and Tlajomulco de Zúñiga were also called to participate in this process. In the case of CDMX, work was carried out with different entities within each of the municipalities, which together complement the various roles in emergency response. In the second group, a large group of professional and academic institutions participated in the discussions around the development of the academic curriculum on post-earthquake damage MIYAMOTO'S TECHNICAL TEAM INFORMANT 16 | FINAL EVALUATION OF PREPARE MEXICO assessment9, and the preparation of the Methodology for the Selection of Sites for the Final Disposal of Waste, where the institutions that are part of the disaster risk reduction system and other public and private organizations that handle important information for the management of debris and the preparation of this particular product, were invited to participate. “The external consultancy was well combined with the participation and appropriation of the product by the local stakeholders, (...) they did their best to incorporate everyone's suggestions and comments, they sat down with us in a private meeting and we verified with them each one of the points we made.” The broad participative approach used achieved a result that is difficult when products are developed through external consultancies, which was a high level of ownership of the results by the stakeholders. However, this approach was weakened by the lack of follow-up and socialization of the progress achieved, not only among all the stakeholders that at some point collaborated in its development, but also towards the outside. This follow-up and socialization would be necessary for the establishment and maintenance of public-private networks that collaborate in the sustainability and impact of the results obtained, which is one of the objectives of the participatory approach, and which according to the people consulted was not achieved. The testimonies obtained, both from officials of local government agencies, and from members of professional associations, university students and inhabitants of vulnerable communities, repeatedly expressed the lack of information after their participation, on the progress and products of the various activities of the program, as a shortcoming that weakened their chances of continuing to support these initiatives, which they all considered very important. One of the objectives of the implementation of the earthquake-resistant construction workshops in irregular settlements was the accessibility of these courses to the most vulnerable people in these communities, and the equal participation of men and women, given the importance of women as heads of household and mothers in the management of the construction, improvement, and expansion of their homes. "(...) We tried to do it in different locations so that people would not have to travel long distances. We wanted the community not to have to go to the courses, but to bring the courses closer to the community. They were implemented on weekend days so that people did not have to miss working hours. Another issue we also foresaw was food. We provided them with a strong meal and healthy snacks for both the attendees and their children." To achieve this, the program relied on the Directorate of Municipal Delegations and Agencies, the Colmenas, and the DIF, as agencies of the local government of Zapopan that work directly with the population of these sectors and whose headquarters are located within them, to facilitate the attendance of interested people. In addition, during the workshops (which were held on weekends to accommodate the participants' free time), food was offered to the attendees and their children, and childcare was provided so that mothers who had no one to leave their children with, could attend. There were no requirements or conditions to participate, except that the attendees should be over 15 years of age for safety reasons in the handling of heavy construction elements during the practical exercises. This way, people of any age and sex could attend, whether they were familiar with construction activities. It was striking to see the large participation of very young individuals (teenagers), when one could think that it would not be a subject of interest to them, although it will affect them directly when, in the near future, they form their families and want to build their homes. 9 Alianza del Sector Privado para Sociedades Resilientes ante Desastres; Colegio de Arquitectos de la Ciudad de México; Instituto Tecnológico y de Estudios de Occidente; Instituto Tecnológico y de Estudios Superiores de Monterrey; Tecnológico Nacional de México, Campus Colima; Instituto Tecnológico de Tláhuac; Universidad de Guadalajara; Universidad Iberoamericana, Campus Ciudad de México; Universidad Nacional Autónoma de México; Universidad Popular Autónoma del Estado de Puebla; Universidad Tecnológica de Guadalajara. INFORMANT OF THE INSTITUTE OF STATISTICAL AND GEOGRAPHIC INFORMATION MIYAMOTO'S TECHNICAL TEAM INFORMANT FINAL EVALUATION OF PREPARE MEXICO | 17 The following table shows the age and sex of the participants in the workshops, demonstrating how well received they were, both among women, who participated almost equally with men, and among young people, who accounted for 18.75% of the total number of participants. Figure 2. Participants In Earthquake-Resistant Construction Workshops According To Age And Sex 10 SEX AGE TOTAL 15 - 20 years 21 – 59 years ≥ 60 years Absolute Relative Women 12 65 4 81 46,02% Men 21 69 5 95 53,97% TOTAL Absolute 33 134 9 176 100 % Relative 18.75% 76.13% 5.11% EFFECTIVENESS The evaluation of effectiveness measured and assessed the degree of achievement of the initially planned objectives, that is, it sought to judge the intervention in terms of its orientation towards results, also taking into account the results and effects not foreseen but emerging from the processes put in place. The analysis of this criterion also assessed the actions and strategies of the program that contributed more effectively to the achievement of the planned results and objectives, and those that worked less well, or even hindered progress, and that constituted areas of improvement for future interventions. For the analysis of this criterion, the evaluation did not limit itself to verifying compliance with the indicators established in the program design, but considered the entire results chain, since it was observed that some of the indicators were not really useful for evaluating the effectiveness of the program, either because they were not designed specifically for it and instead were generic for the entire program at the regional level, or because they focused only on verifying the quantitative compliance of the activities carried out without measuring how these contributed to the results obtained, or because, as in the case of indicator 1. 3, they referred to results and products already achieved in previous phases of the program. This section will answer the following evaluation questions: 1. To what extent have the goal and specific objectives of the program been achieved? 2. Has coordination and communication between organizations, institutions and agencies and public-private collaboration increased as a result of participation in the project? 3. Has DRA deployment capacity been scaled up as a result of this collaboration? 4. Has the private sector been involved in contingency plans and municipal emergency response protocols? 5. In Zapopan, was progress made in incorporating the findings of the city's vulnerability assessment conducted in 2019 into risk management plans? 6. In CDMX, was progress made in the development of the necessary operational framework to operationalize the deployment of the RDA and DDA technical capacity strengthened in the previous phase, with 249 specialists trained? The key informant consultations and document review conducted in this evaluation, led to the conclusion that the program fulfilled to a high degree its planned specific objectives, and contributed significantly to the achievement of the overall objective of strengthening and institutionalizing the risk management and disaster response capacity of authorities and their private partners in earthquake-prone and 10 Own elaboration based on: USAID/BHA PREPARE Mexico: Community training, Seismic resistant construction tables. 18 | FINAL EVALUATION OF PREPARE MEXICO highly vulnerable urban areas in Mexico. These achievements, however, cannot be attributed entirely to this phase of intervention, but are the result of the evolution and consolidation of the progress achieved in the implementation in Mexico of the previous stages of the PREPARE program. For each of the three program components, the progress made towards the objectives formulated in its planning, the changes perceived and the most effective interventions, and the obstacles encountered in the achievement of those objectives are presented below. Specific Objective 1 Progress Towards Formulated Objective Component 1 of the program was guided by objective I of the program which was to facilitate the incorporation of best practices, strategies and tools for risk management and disaster response into public policy and planning processes. To achieve this, activities were planned for the elaboration of three products: 1. Protocols and procedures for damage assessment in emergencies for the municipalities of Xochimilco and Benito Juarez in Mexico City, and Zapopan in Guadalajara. 2. Mobile application for information management in structural damage assessments. 3. Site Selection Methodology for Earthquake Debris Disposal in Zapopan. Protocols and procedures for damage assessment in emergencies In the development of these products, the program worked with the mayors’ offices of Xochimilco and Benito Juarez in Mexico City, and with the municipality of Zapopan in Guadalajara, which showed interest in participating in this work due to the emergency response planning processes in which they were already involved. These offices were willing to sign memorandums of understanding with Miyamoto International to commit their human resources to a joint development process. The process began with a workshop on operational protocols held by MI in December 2021, which was attended by representatives from the three levels of government (state, federal and municipal), academia and the private sector. Inter-institutional and multidisciplinary technical working groups were formed, which were similar in all three cases, but the actors participating in each varied according to structure of each local government for emergency response. In Zapopan there is a single authority for risk management, which brings together Civil Protection and Firefighters, and they have their own rescue groups, which allows them to make alliances directly with universities and professional associations to carry out damage assessment work. However, in the CDMX municipalities the Civil Protection departments only work in prevention, and depend, for emergency response, on other existing entities within the municipalities themselves. "The work methodology was similar. First, they were given an approach on how to set up the protocol, and then an initial procedural proposal to have a baseline on which to work. Based on that baseline, we held a workshop in which we obtained a clear definition of who was responsible, who the support was, what type of support each institution was going to provide, in order to translate it into communication matrices and then into action procedures." The protocols developed with the stakeholders of each of the entities involved were validated through earthquake drills, where all the authorities of each of the municipalities met on a scheduled date to form the emergency committee, applied the processes and formats designed to deal with the emergency, and made adjustments based on the observations of the specialists who participated. In the case of Zapopan, this drill coincided with a real seismic event, which tested, under the pressure and nerves of the moment, the validity of the designed procedures. These protocols were formally delivered to the respective authorities in each case, who, thanks to their participation in their design, are fully equipped with these tools, which are adapted to their resources, mandates and work procedures to make them more effective and efficient. MIYAMOTO'S TECHNICAL TEAM INFORMANT FINAL EVALUATION OF PREPARE MEXICO | 19 Mobile application for information management in structural damage assessment As a complement to the development of these protocols, and at the request of the SGIRPC and the ISC in CDMX, a specialized firm, Undersoft, was hired to design a mobile application for the collection of information in the evaluation of post-earthquake damage to structures and for the unified management of the information collected, in order to make these processes more efficient, eliminate duplications, repetitions and delays, and have reliable and timely data for decision making by the emergency response authorities. The process of developing this tool was greatly affected by the political context that arose during the program's implementation period. Originally, the application was to be developed with the participation of key federal government institutions, in order to harmonize it with the procedures and tools they use and to extend its application to the entire country. The National Center for Disaster Prevention (CENAPRED), which is the governing body of the risk management system at the federal level, was expected to support the project, but due to the growing tensions between the Mexican government and the U.S. government, the Ministry of Foreign Affairs required the implementation of additional prior procedures to be considered by other counterparts, and ended up denying the authorization of the application to carry it out as planned. It was then decided to carry it out with the participation of the municipal governments and mayors' offices of Zapopan, Xochimilco and Benito Juarez, who were participating in the other components of the program, who had the autonomy to decide their participation and who expressed their interest in doing so, since they are the first responders in their territories in the event of catastrophic incidents. The development of the tool involved the staff of the emergency response agencies of these three local government agencies, who first established the requirements to be met by the application, and then participated in the feedback of its different aspects of design, such as the graphic environment, the experience of users as evaluators and as emergency managers, etc. Due to the delay in the start of this work, caused by the aforementioned political restrictions, at the time of the consultation stage of this evaluation, the tool had not yet been delivered. However, within the scope of this component, a training on the use of the tool is planned (both for the administrator users and for those who have the role of evaluators), user manuals will be delivered, and technical support will be provided for several months after its delivery, to solve any unforeseen issues that may arise in its use. The testimony of the consulting firm in charge of the application’s design shows the great progress in capacity building that the use of this instrument by emergency response institutions will mean: "The tool can be operated from a cell phone with or without internet, in situations where communication services may stop working. It is not only a mobile application, but a system that allows us to visualize the information, since it has dashboards, with maps that enable the identification of areas that are safe, those that have a medium risk and those that are totally unsafe, to avoid risk situations, not only for the population, but also for the personnel working in the field. The system allows to modify the status of each building directly, to avoid generating duplicate records and procedures, allows to generate reports, count the affected population and the type of affectation, export the information to visualize it in other tools, so that decision makers can see in real time what is happening in the territory, to make timely and appropriate decisions to save lives." The mobile application can be used with iOS and Android operating systems, and has the characteristic that, although it was designed specifically for the three participating municipalities, it was built in such a way that other municipalities or regions can be incorporated at any time. Methodology for the selection of debris disposal sites The third product under this component, which is the methodology for selecting sites for debris disposal, was developed specifically with the local government of Zapopan, since it was based on UNDERSOFT COMPANY INFORMANT 20 | FINAL EVALUATION OF PREPARE MEXICO quantitative data from the probabilistic assessment of seismic risk in the built environment of this city, and was developed as a complement to the Post-Earthquake Waste Management Plan, developed in the previous program’s phase. The objective of this methodology is to mitigate the impact of waste disposal caused by earthquakes. If these impacts are reduced, the population will be able to return more quickly to the levels of well￾being in which they were before the event. Likewise, by reducing the negative impacts caused by poor waste management, the deterioration of environmental quality in the exposed communities is avoided, both in the vicinity of the site of the event and around the waste collection and final destination sites. For its preparation, the company Enlace Ambiental was contracted, and different agencies of the local government of Zapopan and the state of Jalisco participated, such as the Secretary of the Environment, the Institute of Statistical and Geographic Information and many other organizations and experts who contributed their knowledge and experience. This wide participation was necessary due to the scope of the implications of waste disposal and the restrictions that must be taken into account for the identification of suitable sites. The methodology considers the temporal dimension of emergency response by first identifying land located within the city that has no built-up use, such as sports fields, parking lots or vacant lots, which can be used as temporary deposits when debris needs to be removed quickly to clear roads and rescue victims, and, then, identifying sites that can be used for the final disposal of debris, which must comply with a series of regulatory requisites of the different levels of government, legal, infrastructure, physical, environmental, social and safety requirements. Several probable sites that meet the established requirements are selected, and then their conditions are verified and evaluated in the field, in order to establish a hierarchical order among them. This is an indispensable step, since the information on the basis of which the existing alternatives are screened is often incomplete, or not updated and does not correspond to the reality of the site. "We visited all four sites and found that none of them were optimal, because of other contextual conditions. One was located next to a gas plant, which is a hazardous facility, and had a funnel-like relief, which made it vulnerable to flooding. Then we went to another place which was excellent, spacious and flat, but adjacent to areas where many people are involved in criminal activities. So there you consider other types of variables that maybe the technical analysis does not give you...” In the development of this tool, the lessons learned from its previous application in other countries were used, and its design contemplates the ease of updating the base information for site identification, such as changes in legislation and regulations that affect it, and the possibility of adapting it to other contexts. Its elaboration was accompanied by the implementation of a tutorial workshop for emergency response system officials, where the methodology was explained step by step, in a theoretical and practical way, and which was delivered together with the general report, so that in the future the officials who are going to use it can continue to be trained. As part of this activity, and to facilitate the sustainability of its results, the PREPARE Mexico program will also deliver to the corresponding administration, the hardware equipment necessary to run this methodology, since conventional equipment does not have the capacity to implement it, due to the large amount of information that supports it. Unplanned products Within objective 1, an unplanned action was included, requested by the coordination of Civil Protection and risk prevention of Mexico City, which was the provision of equipment for a mobile classroom (in a bus) that was created with the intention of disseminating awareness and knowledge about risk prevention and mitigation in communities far from the urban center, especially focused on the youth population of rural settlements. In agreement with the donor, the program donated in 2021 the earthquake simulation equipment, generators, computers and screens for the operation of the MIYAMOTO'S TECHNICAL TEAM INFORMANT FINAL EVALUATION OF PREPARE MEXICO | 21 mobile classroom, which was put into operation a few months ago, with much delay, because of the Mexican government's restrictions on donations, due to the COVID-19 health emergency. Figure 3. Compliance with the Indicators of the Specific Objective 111 Indicator Goal Degree of Compliance 1.1 Number of hazard risk reduction plans, policies, strategies, disaster preparedness and contingency plans developed and in place 6 100% achieved 1.2 Number of people participating in discussions about national risk reduction strategies as a result of the program 40 Target exceeded 1.3 National and local risk assessment, hazards data, and vulnerability information is available within target areas Y Achieved 1.4 Percentage of disaster risk management and response actors citing improvements in municipal-level preparedness at program end 90 100% achieved Regarding the degree of compliance with the indicators planned to measure component/objective 1, the table above shows that the first of them was 100% achieved, since the three protocols and procedures for post-earthquake damage assessment in buildings, and the three guides for the use of the mobile application for information management in RDA, were developed for the municipalities of Xochimilco and Benito Juarez in CDMX, and for the municipality of Zapopan in Guadalajara. However, the indicator states that these developed plans are in operation, a very ambitious condition that has not been fully achieved, since, although the current administrations of the institutions of the risk management system participated in the development of these products, their definitive institutionalization depends on other factors external to the program: "We know that protocols have to be developed, they have to be validated, they have to be authorized and then socialized. We have not yet completed all those steps. If this month we are concluding this stage of the Prepare project, we hope to have the product ready to be able to, through a protocol act, invite the authorities to make the deliverables, and that they will be our starting point to initiate these processes of socialization of the protocol...” Indicator 1.2, on the other hand, set a very low target, because in the development of the different products planned within component 1, a large number of stakeholders from different areas were consulted, given the variety of these products and the breadth of their implications. Indicator 1.3 refers more to a seismic risk and vulnerability assessment that was carried out during the previous phase of the program, so it is our opinion that it should not be counted as an achievement of this stage of implementation. Regarding indicator 1.4, which refers to the percentage of risk management and disaster response stakeholders who cite improvements in preparedness at the municipal level due to the program, all informants that participated in this evaluation who belong to those entities cited improvements and advances in the prevention and response capacity of the system, so we consider that this indicator has been reached at 100%. This evaluation has, however, the already mentioned weakness that it was not possible to consult actors from the Xochimilco and Benito Juárez municipalities, due to the internal political situation that in recent months cut off the collaboration that these institutional actors maintained throughout the program. It is noteworthy that none of the indicators of Component I mentioned the development and implementation of management tools such as the methodology for site selection for the disposal of earthquake debris, one of the most significant products for progress towards the specific objective of 11 Own elaboration based on: USAID/BHA PREPARE México: Performance Report March 2023 ZAPOPAN CIVIL PROTECTION COORDINATION INFORMANT 22 | FINAL EVALUATION OF PREPARE MEXICO this component, which illustrates a certain inadequacy of the indicators established for measuring this progress. Beyond these advances in the fulfillment of the indicator goals, all the evidence analyzed in this study affirms that the PREPARE Mexico program advanced decidedly in the achievement of its specific Objective I of facilitating the incorporation of best practices, strategies and tools for risk management and disaster response in public policy and planning processes. Perceived changes and more effective interventions The most important changes and potentialities cited by the people interviewed, as a result of the interventions implemented in this component, have to do with the consolidation of a paradigm shift within the country's risk management and emergency response system. A change that goes from a basically reactive and erratic intervention by the authorities in response to disasters affecting the territory and the population, to a planned and strategic response, capable of using public and private resources in a systematic and rational manner. The damage assessment protocols and procedures developed, establish a set of actions and actors that allow timely decisions based on reliable information, to address emergencies in an efficient and effective way to minimize damage and accelerate the recovery of the affected territories. Within this change of vision, the mobile application is a decisive operational leap, which constitutes one of the first real efforts being made in the country to incorporate technology in emergency response, and whose use will overcome the multiple deficiencies that have been detected in the assessment of damage and distribution of aid in recent catastrophic events recorded in the country, such as the 2017 earthquake in Mexico City. But the most novel aspect is the management of debris that may be caused by an earthquake or other catastrophic event, an issue that had not been given the importance it deserves, which was evidenced in the probabilistic assessment of the seismic vulnerability of the city of Zapopan prepared in the previous phase of the program: "(...) the issue of debris disposal also, because of its novelty, because we always focus on the treatment of the injured, the rescue of people, the emergency per se, but we do not think that if I have a good debris management it will facilitate the rescue and care of the victims, I will not generate a negative environmental or ecological impact, nor how much it will cost us to move those two million cubic meters of debris and where I have to send those who have chemical products, biological remains, etc. I think this opens the door for the institutions to think about these issues.” The most effective strategy used in the actions carried out under this specific objective was that of technical work in interinstitutional and interdisciplinary teams, where the representatives of the organizations called upon to use each of the products developed were given a leading role, so that the products would respond as closely as possible to their needs, priorities, and procedures for action. This strategy was used in the elaboration of post-earthquake damage assessment protocols and procedures, directly coordinated by the Miyamoto technical team, but also in the elaboration of the methodology for the selection of sites for debris disposal, which due to its technical specificity was contracted to a specialized consulting team that, despite being external to the program, was able to maintain the approach of broad consultation with stakeholders that is part of its strategic principles, in order to maintain the relevance and usefulness of the products elaborated. Main challenges to achieving the program's objectives The main challenge to achieving Objective 1 was the institutionalization of the products elaborated, a task that has different degrees of difficulty for each of the products and the different local administrations involved. This institutionalization is essential to overcoming the political vicissitudes that deeply affect the continuity of public administrations in Mexico, which are exacerbated by the ZAPOPAN CIVIL PROTECTION COORDINATION INFORMANT FINAL EVALUATION OF PREPARE MEXICO | 23 short terms of local governments, but which, as we have seen above, can occur even within government terms, even without a change of political party or authorities. "It usually happens that with changes in government, things are either postponed or directly cancelled, because they are technical processes, but also legal. It is very recurrent that many policies are frustrated by these changes. Even a technical body such as ours is affected by political changes. There should be a lot of political will to be able to give continuity to something that is identified as necessary, as a priority…” The post-earthquake damage assessment protocols and procedures, which is probably the product of this component that will be most easily used and applied by the local authorities that participated in its development (since it is adapted to their existing resources and forms of working), must, however, pass a series of bureaucratic requirements before their full application. The unification of these procedures in all the local administrations of the country, or at least in all those that make up a metropolitan area such as Guadalajara, which is a necessary condition for their efficient functioning, is also hindered by the differences in the operational structures of these administrations, often made up of different types of instances and actors, and by the existing power relations between them. In the case of the mobile application, its implementation will be easier in the municipality of Zapopan, since it responds to a section of its development plan that specifically establishes the use of this type of tool, but more complex for the municipalities of Xochimilco and Benito Juarez, where there is no such mention and will depend on the vision of institutional development that the successive administrations of these municipalities have. However, it is the methodology for locating debris disposal sites that faces the greatest obstacles to its institutionalization and application by local governments, since it has to do with the legal status of ownership of the identified land, which will need long and costly expropriation processes that require political and administrative will and continuity. A drawback that always arises when working with governmental entities, and which was among the causes that motivated the need for an extension of the program period, is the slowness and difficulty of the work of co-creation of products with different governmental entities, whose diverse and tight political and institutional agendas (within which this program was just one more number), condition the agreement of common spaces and the continuity of their participation, and therefore hinder the progress of the processes that depend on it. Some of the informants also report certain planning failures in some of the activities implemented, which hindered the start of their processes, causing unnecessary delays. For example, in the beginning of the development of the methodology for locating sites for waste disposal, the necessary participating entities had not yet been identified and instead were gradually incorporated into the process, causing setbacks and repetition of the work stages already advanced. Specific Objective 2 Progress Towards Formulated Objective Component II’s specific objective was to strengthen public-private collaboration in the application of disaster response tools, and to achieve this it focused on scaling up the capacity for post-earthquake damage assessment in the private professional and academic sector, with the development and implementation of the following products: • Academic Curriculum for the university course “Introduction to Post-seismic Evaluation of Constructions in the Integrated Risk Management Environment” • Teaching Manual for the course “Introduction to the Post-seismic Evaluation of Constructions in the Integrated Risk Management Environment” • Implementation of a Training of Trainers in the RDA course. INFORMANT OF THE INSTITUTE OF STATISTICAL AND GEOGRAPHIC INFORMATION 24 | FINAL EVALUATION OF PREPARE MEXICO The program hired a multidisciplinary group of professional experts in education and pedagogy, but also in engineering and architecture, to develop the university curriculum for the course “Introduction to Post-seismic Evaluation of Constructions in the Integrated Risk Management Environment”, based on the material provided by Miyamoto, to make it operational and digestible for higher level university students of architecture, civil engineering and construction. The course design was made with the participation of specialists from a large group of universities. The objective of the course is not to train building damage assessors, since that responsibility can only be assumed by legally accredited professionals, but to ensure that university students can effectively support damage assessment brigades led by professional experts, which are formed to respond to emergencies that affect the built infrastructure of the city, since it has been proven in previous events that the participation of students without prior preparation in damage assessment of buildings, produces very bad results: "If there is no knowledge, this collaboration turns into chaos, instead of giving confidence and security to the citizens, you instill more fear. This basic knowledge that we had in this course gives you the ability to distinguish if it is a superficial crack and does not threaten the structure or if it is structural and if it does threaten it, and that also depends on the type of structure of each building, it is very complex.” But another objective of this course, not less important, is to make students aware of the risks inherent in the urban built environment and introduce them to the knowledge of its management system, and the roles that they can play as future professionals within this system and in strengthening their communities to deal with these risks, depending on the professional practice option in which they decide to develop their careers: "The course as such does not oblige those who study it to become damage assessors in the future, but it does leave them with the seed that whoever wants to can do it. Everything that has to do with integral risk management was taken into account, which is a cycle, so that they can identify the elements of risk management in their daily lives and can exercise self-care, or even so that when they become engineers or architects and it is their turn to design, they know that these risks must be taken into account and that there are measures and decisions that influence this risk.” In the participatory process of developing these products, presentations were made of the approaches, discussions and voting that took place to decide on the final options. The meetings were recorded and the minutes were sent to all participants together with the document constructed, so that the experts could make their contributions, adding photographs, bibliographies, etc. The course, of a theoretical-practical nature, is based on the formation of competencies, and is structured in two units. The first focuses on the factors involved in risk due to many types of natural and human-induced hazards and disturbing phenomena and on the integrated management of these risks, including their regulatory, conceptual and operational aspects. It is illustrated with examples of the main disasters that have occurred in the world and in Mexico in the last 50 years, as well as the governmental actions that have been developed to counteract them.12 The second unit, which focuses on rapid damage assessment in buildings, reviews the basic concepts of structural engineering and the response of structures to seismic events, and presents the RDA process and tools, including the safety aspects of the brigade members, and their role within the Seismic Emergency Plan and the Incident Command System. The course, which was developed in two versions, a face-to-face version (which was the originally planned modality), and a virtual version that had to be incorporated due to the restrictions imposed 12 USAID/BHA PREPARE México, november 2021: Introducción a la evaluación post-sísmica de construcciones en el entorno de la Gestión Integral de Riesgos. Manual del docente. INFORMANT OF THE ACADEMIC PROGRAM DESIGN CONSULTANT TEAM STUDENTS AT THE NATIONAL UNIVERSITY OF KNOWLEDGE AND RESEARCH FINAL EVALUATION OF PREPARE MEXICO | 25 by the COVID-19 pandemic, was structured in a very segmented manner, so that its implementation would be adaptable to the different types of academic plans of the various educational institutions. "One of the most positive aspects of the program has been that the course material came already developed and resolved, and what we had to do was choose how we were going to dose it in order to deliver it directly to the different profiles of our students.” In addition to the academic curriculum, the participating universities were provided with a Teaching Manual and a Sylabus, which detail the teaching methodology step by step and include the exercises, evaluations, etc. They were also given interactive presentations, and a pilot group of teachers were given a training course for trainers. The course is already being implemented in several universities and technological institutes in Mexico City, Guadalajara and other regions of the country, such as UNAM, the National Technological Institute and the Technological Institute of Monterrey. In Zapopan, the main academic partner of the program was the Centro de Enseñanza Tecnológica Industrial (CETI), a public educational center that offers technical and higher education programs related to construction to students from a lower/middle socioeconomic level. The third activity under Component II was the Training of Trainers courses, which were implemented in various professional associations and universities, to generate in these institutions the capacity to continue teaching damage assessment courses autonomously, with the objective of increasing the number of trained professionals needed by the cities of Zapopan and CDMX to be able to respond to a contingency such as a high magnitude earthquake. At the same time, the post-earthquake damage assessment courses continued to be given in different professional associations, reaching a total of approximately 1,500 professionals in architecture, civil engineering, and related professions, trained to act in cases of emergency: "We already have people trained within our ranks who are going to give the training, because we are going to replicate it in Chapala, which is also a seismic zone, and in Guzmán, which also has an active fault. We did not manage to do it in these municipalities within the program, but we already have people who took the course and we hope that doing it with them will have a good result, and that they will perform well.” Regarding the measurement of the degree of compliance with the indicators under Component/Objective II, the following table shows that all of them were fully achieved. Five of the indicators were surpassed; 2.1 because 20 private sector businesses directly engaged in DRM activities, 2.3 because there were three protocols and procedures for post-earthquake damage assessment developed that incorporate the private sector, 3.2 because a total of 735 students were educated in DRR (691 in RDA/DDA), and 3. 3 because by the end of the PREPARE Mexico program, eight educational institutions had incorporated the course on “Introduction to Post-Seismic Evaluation of Constructions in the Integrated Risk Management Environment” into their academic curriculum, either as an independent subject or, what has been more frequent, as part of one or some other subjects that are already part of the academic programs of these institutions. Some of these academic institutions are the School of Architecture of the UNAM, the Instituto Tecnológico de Estudios Superiores de Monterrey, the Universidad Iberoamericana, and the Centro de Enseñanza Técnico Industrial in Zapopan. For indicator 3.4, which measured the retention of knowledge acquired in the RDA courses, two months after their implementation, the results were 83% in Zapopan and 84% in Mexico City, exceeding the established goal of 70%. TEACHER AT THE INDUSTRIAL TECHNICAL EDUCATION CENTER CETI INFORMANT OF THE COLLEGE OF ARCHITECTS AND URBAN PLANNERS OF JALISCO 26 | FINAL EVALUATION OF PREPARE MEXICO Figure 4. Degree of Compliance with the Indicators of the Specific Objective 2 13 Indicator Goal Degree of Compliance 2.1 Number of private sector business directly engaged in response or DRR-related activities as a result of the program 6 Exceeded at 333% 2.2 Targeted total number of individuals indirectly benefiting from DRR-related program activities 3,727,500 Achieved 2.3 Number of government disaster contingency plans that incorporate private-sector aspects as a result of the program 2 Exceeded at 150% 2.4 Public-private partnerships included in city or municipal emergency response protocols yes Achieved 3.1 Number of DRR curricula developed, by education level 2 100% Achieved 3.2 Number of students educated on DRR 500 Exceeded at 138% 3.3 Number of DRR-related programs established within educational institutions 2 Exceeded at 400% 3.4 Percentage of RDA course participants retaining knowledge 2 months later 70% Exceeded at 120% It can be concluded that the actions implemented within Component II, as well as the collegiate elaboration of procedures and methodologies that were part of Component 1, produced significant progress towards the objective of strengthening public-private collaboration in the application of disaster response tools. It is noteworthy, however, that the program's planning matrix did not include an indicator to measure one of the most effective actions for the achievement of the general objective of this component, which was the training in damage assessment and the Training of Trainers in professional associations. Both of these directly impact the possibility of fruitful collaboration between the private professional sector and the public administration, and grant autonomy to its members to continue strengthening this collaboration, once the PREPARE Mexico program is concluded. Perceived changes and more effective interventions Before the PREPARE Mexico program, in dealing with emergencies due to earthquakes, landslides, floods, which affect buildings and urban infrastructure, collaboration was established between government disaster response agencies and professional associations that have the technical capacity to make evaluations and reports. In recent cases, university students from construction-related careers were involved as collaborators in the damage assessment brigades. However, the great qualitative changes introduced by the PREPARE program in this field were the raising of awareness among authorities and private professional circles on the forcefulness and magnitude that such collaborations has in order to be able to respond on the necessary scale in the event of a catastrophe of major proportions, and the development and socialization of an operational and technical planning framework within which such collaboration can take place in a fruitful and effective manner. "I believe that the objective of strengthening public-private collaboration in emergency response has been achieved, because both the institutions are aware that the governments need help, and already know the techniques to deal with disasters, and the government already knows that they alone will not be able to deal with a major emergency and must turn to professionals. In some cases this was already working, and what we did was to give it more formality, so that it is not only when the emergency arrives that a cry for help is sent out, and the entity called to help does not even know how to attend to that emergency...". 13 Own elaboration based on: USAID/BHA PREPARE México: Performance Report March 2023 MIYAMOTO'S TECHNICAL TEAM INFORMANT FINAL EVALUATION OF PREPARE MEXICO | 27 All the professionals from different associations and academic institutions consulted highly valued the technical skills acquired, not only in terms of the mandate of collaboration with public entities that many of them have, but also of their professional strengthening, since the preparation of technical opinions on buildings is a service that is also in great demand by many companies and individuals in the private sector. "We almost always support the government when there is a contingency, but here with the course they gave us an order, because it is not the same to support without knowledge of the cause than to support with a with a sense of direction. We always go, we make judgments, but not with the methodology, the precision, the speed that this course gives us, it is the first time we have had an approach like this. For us this experience is molten gold, because among the functions of the school is to support the government and the citizens, and this course gave us the certainty that we already have the tools to do so in the face of a disaster of this nature.” The joint participation of government professionals and technicians in the courses implemented, generated specific collaboration agreements, in some cases like the one described in the following testimony: "I even invited people from the state university who have campuses all over the state, and because of the diversity of the regions they have many problems in their buildings. This strengthened our relationship with the public sector, because the public servants who were sitting there asked us to schedule a second course almost exclusively for them because they thought it was very good. They also asked us for a quote for the appraisal of all the educational buildings they have; they gave us a list of 25 buildings.” Among the most effective elements and strategies for achieving the objectives sought, the informants mentioned the following: the fact that the damage assessment courses were given in virtual mode, in addition to the face-to-face option, which extended the course’s coverage to professionals from other territories, the teaching methodology used, with real examples and exercises in the field, and the evaluations carried out before and after the course, which made it possible to highlight the participants' deficiencies. In terms of the academic curriculum developed, it was very appropriate that, as a result of exhaustive research, numerous examples of national and international disasters and the strategies and actors deployed to deal with them were incorporated into the first module of the course. This first module focused on integrated risk management contributed greatly to generating among the students a perception of the multiple risks in the territories they inhabit, which is an indispensable condition for them to be able to become involved later in the work of prevention and management. In the development of this curriculum by the contracted consulting team, the participation of local experts was decisive, as they broadened the vision of the specialists to achieve a more complete and adequate product for their purposes. Main challenges to achieving the program's objective A first obstacle that arose in the implementation of this component was the difficulty that exists within the academic institutions to modify curricula to introduce the course “Introduction to Post-seismic Evaluation of Constructions in the context of Integrated Risk Management” as an independent subject, given the slowness of the bureaucratic procedures that govern their structures. This forced most of the universities to incorporate the contents of this course within other existing subjects, often in a fragmentary manner, which was possible because it was designed with this possibility in mind. However, this might have also discouraged some academic institutions with which approaches had been made that did not yield the expected results. But the greatest challenge was the institutionalization of public-private collaboration, which sought to be strengthened with the actions implemented under this component. While collaboration is more INFORMANT OF THE COLLEGE OF CIVIL ENGINEERS OF THE STATE OF MEXICO INFORMANT OF THE COLLEGE OF ARCHITECTS AND URBAN PLANNERS OF JALISCO 28 | FINAL EVALUATION OF PREPARE MEXICO robust between governments and professional associations, it is still very weak with the academic sector, where there is a lack of links and agreements to strengthen it. For example, building a roster of students who have taken the course, and where some of those who participated in the courses are missing a follow-up after they have finished, which would allow them to build and maintain these links. "As for the objective of strengthening the response capacity through the public-private partnership, it would have to be investigated, because at the end of the course, it simply remained there. There was no active participation that we would have had after the course. There has not been any information after the end of the course, so we do not know if there is really a follow-up to this course. Independently of that, it depends on the educational policies of each institution, to be able to change or link it, because sometimes they remain hermetic about their ways of working, but this change does not happen overnight". Specific Objective 3 Progress Towards Formulated Objective Component III of the program aimed to increase knowledge of basic seismic-resistant construction techniques among the vulnerable population of high-risk urban settlements, through a workshop on basic seismic-resistant construction practices in which 176 people from irregular settlements in Zapopan participated. As mentioned above, the indicators established in the program's logical framework for this specific objective are not very useful to fully measure the progress made towards the stated objective. The following table shows that the first two indicators were exceeded, while the last one was not met. With these numbers, it could be concluded that the objective sought was achieved by approximately 75%, however, their analysis shows that these indicators did not respond, in their formulation, to an exhaustive diagnosis of the context in which this component was implemented. Figure 5. Degree of Compliance with the Indicators of the Specific Objective 3 14 Indicator Goal Degree of Compliance Number of people participating in training 150 Exceeded to 118.66% Percentage of people who retain skills and knowledge after two months 70% Exceeded to 108% Percentage of attendees at joint planning meetings who are from the local community 90% Achieved Percentage of training participants applying knowledge of seismic resistant construction techniques at program end 70% Achieved The testimonies collected in this evaluation show that the established objective was fully achieved, since the people who attended the 9 workshops not only obtained a consistent and well-founded knowledge on basic earthquake-resistant construction techniques, but also on the different types of risks that threaten the safety of their homes in these settlements, such as floods and landslides, in addition to seismic events. This learning is corroborated by the results of the evaluations carried out subsequently, as expressed by the value reached by indicator number 4.2. The workshops were very effective in awakening this awareness in the participants and in explaining the most basic techniques to avoid these risks. This was achieved by using methodological strategy that favored practical exercises with real scale models of the construction elements, through which all attendees could directly experience these techniques, and to a very successful management that generated among the participants an environment of trust and open communication, conducive to mutual learning. 14 Own elaboration based on: USAID/BHA PREPARE México: Performance Report March 2023 STUDENTS AT THE NATIONAL UNIVERSITY OF KNOWLEDGE AND RESEARCH FINAL EVALUATION OF PREPARE MEXICO | 29 "The methodology was that we presented the slides of the manual, we commented on them, and then we did several dynamics. One where they assembled wooden models that represented the foundation and the columns, and then they also did activities such as cutting steel and building armatures to scale, and they also started to build sections of walls with blocks.” The fact that the workshops were held in places located within the communities themselves helped to keep the topics discussed well rooted in their context by using examples taken directly from their immediate environment, awakening the interest of the participants, and stimulating the exchange of experiences among them: "The content of the training was very good, because they talked to us about the hills where people build their houses, and they told us about the impact that an earthquake can have in those places. They were talking to us about something that is our daily home. It was not like they brought a program and implemented it where it was not required. No, they implemented it in the right place, because we were located there between hills.” The front end work of adapting the contents of the workshops to the specific local context, in terms of the most commonly used construction methods, the different types of terrain and the construction terminology known in the sectors, was key to ensuring the understanding and applicability of all the notions covered, as well as providing the participants with manuals that explicitly and graphically reflect all the concepts and notions taught, as a necessary support when implementing them in practice. However, despite the fact that the first indicator, which proposed the participation of 150 people in these workshops, was exceeded, reaching 176 people, it can be seen that, as explained in the efficiency section, with the work and investment made in adapting the contents, building the scale models and preparing the manuals, the coverage of the workshops could have been much more extensive, since most of them did not reach the maximum quota of 30 participants. This means that if the 9 workshops held had covered the established quota, 270 people would have been trained, and the indicator would have been exceeded by 180%, at no additional cost. This translates into a low effectiveness in the convocation carried out, which was affected by the fear generated among the population by the COVID-19 pandemic, to leave their homes to participate in face-to-face training, as these were. Home visits were carried out to explain the importance of the subject to a population that does not have a formed perception of the seismic risk present in its territory, since other risks are much more visible and recurrent. The testimonies collected from participants in the courses showed this low effectiveness and scope of the message, when they repeatedly commented that among their relatives there were many people who would have been interested in attending these workshops, if they had known about their existence and importance. Regarding indicators 4.3 and 4.4, their formulation expresses a lack of knowledge of the conditions of the socioeconomic context where the action is implemented, which makes the goals set for them unrealistic. Indicator 4.3 refers to the participation of people from the local communities in the joint course planning meetings, without taking into account that many of the inhabitants of these sectors work in the Guadalajara metropolitan area and have to make long daily commutes from their homes to their jobs, which means that they have very little time for the rest of their family obligations, and therefore the courses had to be held on weekends in order to facilitate their attendance. In this scenario, it was absurd to expect that 90 percent of the participants in the course planning meetings would be inhabitants of the communities, when these were technical meetings to discuss the contents and pedagogical and operational strategies to be used. The reality was that these meetings were not attended by inhabitants of the communities, but by the project's technical staff, local construction experts, the professionals in charge of giving the workshops, and the representatives of the Colmenas, the DIF and the Directorate of Municipal Delegations and Agencies, which are the government entities that work directly with the population and therefore know the conditions needed to assure their WORKSHOP INSTRUCTOR INFORMANT OF THE DIRECTORATE OF DELEGATIONS AND MUNICIPAL AGENCIES 30 | FINAL EVALUATION OF PREPARE MEXICO participation, and who were assumed to belong to the communities, although they are not always, in order to be able to comply with the indicator. "Before carrying out the courses, we held worktables, where we organized the workshops with the representatives of the program and the agencies involved. There we discussed the pros and cons of what we were going to do with the workshops so that they would be the most profitable and beneficial. And during the field work we also continued with the meetings to evaluate what we had already executed and what had not turned out so well, in order to adjust and redirect it.” Indicator 4.4 reflects a lack of knowledge of the construction processes in irregular settlements because, although those attending the workshops acquired the knowledge to apply appropriate techniques in the construction of their homes, and efforts were made to locate people who were involved in construction processes at the time of their realization, the temporality of these processes depends on many factors in addition to these. In the first place, the availability of economic resources of the inhabitants to buy the materials and pay for the labor, but also family priorities, where there are times when it is more important to prioritize health, or the education and wellbeing of the children than the improvement or expansion of their house. These are long construction processes, which sometimes last a lifetime15, and which advance slowly adapted to the family needs and possibilities of each moment. "The most positive thing about the course was the willingness of all the people who participated because, even though they knew they were not going to use it immediately, they have been replicating it as much as they could. Perhaps what caused them a bit of frustration is that at home they did not have the material to be able to do what they did here with the materials brought for the course. The frustration of knowing that now they knew how to do it, but they didn't have the material to do it." In addition, the program did not establish a monitoring mechanism for the possible application of construction techniques by workshop participants, which would have required the program to keep contact with each of them to quantify this indicator. However, the interviews conducted show that people are applying the knowledge acquired, in the small construction projects in their homes, or in ways so subtle and at the same time so important, such as the decision to wait before continuing to build until they can buy the materials to make foundations suitable for the construction they want to do, or the application of the criteria learned in the choice of the land to buy to build their house in the future. "One person nearby here told me that she wants to build more, but she was waiting before starting construction to have everything she needs to have a good foundation, and to do it the way the teachers taught them. They did raise awareness with the course. Before they didn't consider it because they didn't know about it, but now they are being more cautious when it comes to building." Perceived changes and more effective interventions The changes perceived as a result of the implementation of the third component of the program have been numerous and very positive, according to the informants consulted. The workshops did not only serve to create and strengthen capacities on safe construction techniques, as was their specific objective, but also to sensitize and create awareness among the population on the vulnerability to risks that exist in the territories they occupy, and on how the decisions they make at the moment of acquiring land to build their homes, or deciding on the location of their homes on that land, influence this vulnerability: 15 Some researchers on the construction processes in informal neighborhoods have qualified them as a permanent work, for example: Bolívar, Teolinda (1987): “La production du cadre bati dans les barrios a Caracas: Un Chantier Permanent.” INFORMANT FROM THE SAN JUAN DE OCOTÁN COLMENA INFORMANT FROM THE SAN JUAN DE OCOTÁN COLMENA INFORMANT OF THE DIRECTORATE OF DELEGATIONS AND MUNICIPAL AGENCIES FINAL EVALUATION OF PREPARE MEXICO | 31 "This does generate awareness, and in particular it generated it for me, because we were trying to buy another piece of land that they were selling to us very cheaply, but which is behind a stream, and this course also helped us to think that in the long run this land could cause us a problem due to some kind of landslide, and the reinforcement we would have to do to the construction to reinforce it against this possible risk would be more expensive. There we were able to apply the knowledge we acquired to say, this works for us and this does not work for us.” Participants report that the knowledge acquired will help them save money in the construction of their homes, as they will be able to calculate exactly the amount of materials they will need for each phase of construction, minimizing waste, and will also be able to demand an adequate execution that ensures the durability of the works carried out, avoiding unnecessary repairs or reconstruction. Another important change perceived in the testimonies was related to the knowledge of the masons. The population often hires masons, especially for the execution of components of structures that they are not capable of carrying out themselves, trusting that they have, due to their experience, the knowledge to execute them in an adequate and safe manner. However, in the workshops it became evident that the masons do not use techniques adequate for the seismic and other risks present in these territories. "The people who are dedicated to construction sometimes do not have technical knowledge. Here almost everyone says that you have to excavate to one meter, one meter 20 and that's it, but sometimes the land is no good, they have mud, they have mud and they are not good materials for foundations, but they believe, because of tradition or because they have done it other times, that this is the right thing to do and it is not so." The workshops reinforced the importance of strengthening the masons’ technical knowledge in these areas, but also of the inhabitants themselves to have the necessary knowledge so that they can supervise the work done by the masons in their homes, and demand that they comply with the proper techniques learned. The workshops were also proof of the importance of inviting to the courses both people with experience in construction and people without any knowledge, so that these exchanges and learning could be generated: "And one thinks that hiring the masons is enough, but sometimes not even the masons have the necessary degree, but they have learned by what they have seen, but not what is technically the best. And what we learned teaches us that we have to do the best we can, because it is our investment and our patrimony." The important and active participation of women in these workshops produced significant changes that have a great potential for social impact in these sectors. On the one hand, it showed that, contrary to general belief, women do work directly in the construction of their homes and urban settlements, as several testimonies transmit, and also in the construction trade, sometimes helping their husbands who carry out this work. This finding helps to dismantle the prejudice that construction is not a women's trade. "There were many single mothers, because they were previously given grants, and with those grants they build and when they don't have the money to pay for labor they do it directly, so the impact among them was great. They said that the course was very good for them because they already know how to do it correctly.” But more important was the empowerment that this knowledge produced in the women participants, allowing them to better manage the work they contract out to third parties in their homes, with less risk of being cheated or robbed. This gives them greater autonomy as well from their husbands, and facilitates the possibility of them improving their homes by themselves, for those whose economic PARTICIPANT IN THE SEISMIC-RESISTANT CONSTRUCTUION WORKSHOP INSTRUCTOR OF SEISMIC-RESISTANT CONSTRUCTION WORKSHOPS PARTICIPANT IN THE WORKSHOPS ON SEISMIC-RESISTANT CONSTRUCTION INFORMANT FROM THE SAN JUAN DE OCOTÁN COLMENA 32 | FINAL EVALUATION OF PREPARE MEXICO condition does not allow them to hire labor for construction. The following testimonies give an account of these changes brought about by their participation in the workshops: "The women who participated said that now they would no longer need to ask their husbands for support when they wanted to build a fence, but above all they mentioned that, if one day a bricklayer came to work, they would know the amount of material they had to use so that the material would not be stolen or wasted, and they could calculate how much they needed to buy the materials." There was also a significant participation of young people of both sexes who, although did not reach a very high proportion among the attendees (18.75% of participants between 15 and 20 years of age), did show a great interest in the topics discussed, which went beyond that of adults, whose main concern was the welfare and safety of their families. These adolescents expressed their interest in learning more about construction in general, thus refuting the generalized conception that young people are manifestly disinterested: "We have the conception that construction as a profession is poorly valued, but I perceived that the course awakened the young people's interest to learn, to know more and to put it into practice, especially by being able to compare what I learn empirically with what I can obtain from a more scientific basis. The women saw it as the possibility of improving their homes and the security of their families, but the young people saw it from a broader perspective, as the possibility of developing themselves and perhaps seeing it as a source of income.” The practical methodology of the workshops, based on exercises carried out by all the participants with construction materials and scale models of the structural components, was the strategy most highly valued by all those interviewed, together with the successful conduct of the instructors, who always stimulated participation and the exchange of experiences, valuing the heterogeneous knowledge that each of the participants had. The location of the workshops in the communities themselves, not only facilitated the access of the inhabitants to them, but also allowed the use of examples of the immediate environment that were familiar to them in the explanations, which were very useful for the understanding of the concepts addressed. A positive consequence of the unforeseen events of the context, such as the impossibility of holding meetings with many people due to the health restrictions imposed by the COVID-19 pandemic, proved to be of great help in achieving the proposed objectives, since it allowed for close interaction between instructors and participants: "(...) I really liked the fact that we were small groups, because that way we could pay more attention without being distracted. When there are large groups, one person gets asked a question and another may be thinking about something else, but when there are small groups, we are all paying attention because you may get the next question. We were a group of 9 or 10 people, and they made a kind of horseshoe shape and the engineer stood in front of us and asked us each one's opinion and experience". To enable people from the most vulnerable groups in the communities to attend, the appropriate conditions were created, such as the proximity of the location of the workshops, as mentioned above, but also the adaptation to the schedules in which people could attend, an arduous process due to the different activities and commitments of the participants. In addition, due to the length of the workshops, which lasted up to 8 hours at a time, childcare and meals were provided for the participants and their children, which made it possible for single mothers to attend, who otherwise would not have been able to participate: "The support for women was very important, because there are many of us who cannot take courses because we have children and no one to take care of them. The fact that they offered a type of daycare while moms and dads were taking the course is something very good, which for me was a very important INFORMANT FROM DIF INFORMANT FROM THE VILLA DE GUADALUPE COLMENA INFORMANT FROM THE VILLA DE GUADALUPE COLMENA PARTICIPANT IN THE WORKSHOPS ON SEISMIC-RESISTANT CONSTRUCTION FINAL EVALUATION OF PREPARE MEXICO | 33 support, and also the fact that they provided food so they could be here during the several hours that the course sessions lasted.” Main challenges to achieve the program's objectives The COVID-19 pandemic was a challenge when calling individuals to participate in the workshops, which required house-to-house visits to explain the importance of the proposed course. It also required all health protection measures to be taken, including hiring people to take the temperature of the attendees, etc. But this call was also difficult, and did not have the desired impact, because the subject of the course was focused on seismic risk, when the population of the city of Zapopan is not aware of this risk, which has not materialized there for many years, as it has in other areas of the country. This required raising awareness about seismic risk to attract people to the workshops, which was not foreseen in the program planning. Another major difficulty, which required great flexibility and effort to overcome, was the limited time availability of the individuals to whom the call was addressed, as expressed in this testimony: "One obstacle was the work schedules, because we as a municipality and the personnel that came from the program were dedicated almost 24 hours a day, but the people in the communities did not, they had a job, a family and other situations, and it was very difficult to agree on that. The solution was to form different groups in order to somehow shape the schedules in which we could all attend. We had to give up our family schedules, our rest schedules." SUSTAINABILITY The sustainability assessment focuses on the evaluation of the continuity over time of the positive effects generated by the intervention once aid is withdrawn. In the field of cooperation, this concept is linked to the enhancement of key development factors and ownership of the process by aid recipients. The sustainability analysis answers the following questions: 1. To what extent have program activities contributed to sustainable changes or results, and what factors have contributed to achieving sustainability? 2. Have mechanisms been established to maintain the public-private partnership beyond the life of the program? 3. How was the exit strategy and sustainability plan established? The sustainability of most of the results obtained in the PREPARE Mexico program is high, since both the planning and the implementation of the actions involved local stakeholders, who took ownership of the products and responsibility for their implementation. The development of all the products incorporated the criterion that, although they were designed for use by the specific government institutions that participated in their co-design, they have built-in mechanisms so that they can be updated following regulatory or other changes and adapted to the use of other administrations and territories. As for the damage assessment protocols and procedures, for example, those were adapted to the resources, mandates and ways of working of the risk management and emergency response agencies of the participating local governments and are therefore perfectly operational. Despite this, the greatest difficulty arises in the unification of these protocols and tools developed in the successive phases of the PREPARE program, among the different local governments in which metropolitan areas as vast as Mexico City and Guadalajara are divided. This unity, which is fundamental for achieving the efficiency and effectiveness in disaster response that these actions seek, is made difficult by the differences between the governance structures of these municipalities and mayors' offices, in addition to factors such as partisan political interests and power struggles. INFORMANT OF THE DIRECTORATE OF DELEGATIONS AND MUNICIPAL AGENCIES INFORMANT FROM THE VILLA DE GUADALUPE COLMENA 34 | FINAL EVALUATION OF PREPARE MEXICO In the case of the methodology for locating solid waste disposal sites, which also meets the conditions of flexibility in adapting to different contexts and ease of updating, there are important factors that threaten its sustainability. In the municipality of Zapopan, for which this product was developed, the main obstacle is the legal status of the ownership of the land identified as suitable, which would require long and costly expropriation processes, which escape the short-term objectives of local government periods and defy the lack of continuity that characterizes these administrations. As this is a new issue, which is not generally considered in the country's risk management plans and policies, there is no awareness of its transcendence, neither in the political class nor in public opinion, to help prioritize the necessary steps and investments to solve the problems that prevent the implementation of the results of the application of this methodology. And these same reasons make it difficult for this methodology to be adopted and replicated by other local administrations or to be scaled up to the state and federal levels of government. In the program's exit strategy and sustainability plan, it is proposed that the individuals trained in the workshops under component 3 would serve as multipliers of the knowledge acquired among their acquaintances, neighbors and relatives. This strategy, however, has its drawbacks, because it can be effective among participants working in construction-related occupations, who can directly disseminate the criteria and techniques learned in their daily practice among their peers. But among participants who have nothing to do with construction, and who may takes years before they can apply what they have learned in their own homes, it is feasible to say that they can share general safety criteria such as the location of buildings, etc., but it is more difficult for them to transmit accurately technical details that condition the behavior of buildings in the event of earthquakes. “To trasmit it to other people, I feel that it is not the same as going to the course, because in the course we lived it because we did it. What I can tell someone else, is like saying it to the air. Yes, I can transmit it, but there is nothing like the dynamics we had there, from how to band the rods, the way to arrange the partitions, all of that, to do it with a physical example is better.” However, the testimonies collected convey a great interest in giving continuity to these workshops, not only on the part of the Colmenas, the DIF and the Directorate of Municipal Delegations and Agencies, which were the local authorities involved in their implementation and to which the PREPARE program provided all the material and methodological inputs necessary to replicate them, but also in other organizations outside this component, which nevertheless value the potential impact that this can have, and are considering implementing similar activities themselves: "Regarding training in construction techniques for the communities, here we have a space where we had planned to give it, and we already had people interested, especially women, who are the most empowered in matters of their homes, but the program ended and we did not have enough time. We are going to see if with what they explained to us we can design it ourselves to be able to give it. In the communities here in Guadalajara there is interest in this topic. Here not even 10% is done with architects, most of it is self-built housing. We contacted many housewives through our social work office and among them there was a lot of interest in how to put up a wall, divide a room, put a little roof, put a light switch or a light bulb in their house.... that is, people who have their own money but do not have the means to hire an architect..” One factor that would give greater sustainability to these workshops would be to try to link them to other assistance programs that donate construction materials in vulnerable communities, so that participating families can have the opportunity to directly implement in practice the lessons learned in improving the safety of their buildings. The program's exit strategy, which seeks to avoid the dependence of its local partners on the donor, contemplates the formal delivery to these partners of all the products developed jointly with them, the having training of trainers so that they can replicate the knowledge and continue strengthening the response and risk management capacity of the system's actors, and the institutionalization of the achievements produced. But it can be seen that this strategy, which has been fully implemented in all INFORMANT OF THE COLLEGE OF ARCHITECTS AND URBAN PLANNERS OF JALISCO PARTICIPANT IN THE WORKSHOPS ON SEISMIC-RESISTANT CONSTRUCTION FINAL EVALUATION OF PREPARE MEXICO | 35 the actions executed, can obtain better results depending on the degree of autonomy of each of the beneficiary partners. Thus, for example, rapid damage assessment courses have greater sustainability among private professional organizations, which have appropriated them and are already replicating to take advantage of them, than among universities, which face the rigidity of their governance structures, which hinder the possibility of including these courses in their programs. "(...) The school itself can deploy a strategy on its own to deal with an emergency situation. The authorities should do it, but sometimes the authorities are slower, and we, who already have 200 people here trained in this area, if there is any type of disaster we can organize ourselves. We are about to send a list of the people who have been trained by municipalities, so that if they are needed, they will be aware that these trained technical resources are available. All of our professionals who have taken these courses, without exception, are directors responsible for building and urbanization projects and works, and we are empowered by law to say, if there is a risk on a property, to evacuate it and call Civil Protection to cordon it off". But in local government structures, we have already seen that there are other types of external factors that permanently threaten the sustainability of the results obtained, in spite of all the care taken to achieve their full appropriation. Foreign and domestic political situations that affect the continuity of the results, such as the electoral campaign in Mexico City, compromise the use of the products and processes designed with the participation of the mayor's offices of Xochimilco and Benito Juárez. IMPACT In this section the evaluation assesses the effects and impacts that were generated by the program, whether expected or not, direct and indirect, collateral or induced. In theory, impact analysis focuses on determining the net effects attributable to the action, but this requires another type of intervention design that isolates the influence of external factors, to determine whether the changes generated are direct results of the project's actions. However, under a contribution to the results approach, which recognizes that changes are generated because of many factors, this report identifies the following impacts as possible contributions to change from program actions in combination with other contextual factors beyond its control: "The most impactful part could be the community courses, because I believe that in this city only about 10% of the constructions have an engineering project and construction supervision to ensure that they meet the construction standards that would make them safe in the event of a major earthquake.” The very great impact, not only social but also economic, that the large-scale replication of a component included in this program as a pilot project, such as the community workshops on basic seismic-resistant construction techniques, can have, was correctly calibrated by the person who made the previous judgment, within the effects that can be produced by the very diverse actions that were implemented within the framework of this intervention. Indeed, in countries where there is no capacity to control the quality of construction, neither in governmental structures nor in specialized professional circles, the greatest effect that can be achieved in reducing the social and economic impact of earthquakes is in the direct training of the millions of people who build their own homes day after day. In Mexico, as in all Latin American countries, the lower-income population has been forced to build their own housing, given the inability of governments to offer them affordable living options in cities. Researchers on the subject estimate that, in a city like Caracas, the inhabitants of irregular sectors have built more than twice the amount of housing that has been built jointly by governments and the private sector in the last decades, generating, through individual action to solve their urgent needs, urban fabrics with very particular characteristics 16. 16 Josefina Baldó Ayala, 2023: “A los constructores de la ciudad” INFORMANT OF THE COLLEGE OF ARCHITECTS AND URBAN PLANNERS OF JALISCO INFORMANT OF THE COLLEGE OF ARCHITECTS AND URBAN PLANNERS OF JALISCO 36 | FINAL EVALUATION OF PREPARE MEXICO In this titanic task they have had no advice from technicians and professionals involved in construction and urban planning, nor from governments, which have criminalized and persecuted them, so that their constructions, in which they have invested a lifetime of work, often present serious vulnerabilities that threaten the safety of their families. In many cases, they are also burdened with the social stigma that directly associates the inhabitants of these territories with vagrancy, crime, and other vices. Under these circumstances, a training program for this vast population, based on the lessons learned from the pilot experience implemented in this activity, would have an immense social impact, both in the recognition of their rights as legitimate inhabitants of the cities, in the awareness of the population about the existing risks and the need to invest in their prevention, and in the reduction of vulnerability to earthquakes and other natural disasters and therefore the possible human and economic impact of these on society as a whole. Economic Impacts: • For the inhabitants of disadvantaged communities, it has a high economic value to acquire knowledge about the most suitable land to build their houses, about the dosage and calculation of the materials to be used and about the construction techniques to make their houses safe from various types of natural disasters, They will be able to make much better use of the economic resources they invest in the construction of their homes by knowing how to choose the safest locations in the event of hazards, not having to hire professional labor for all the construction processes and components, not wasting construction materials, and not having to repeat or repair parts already built in the event of collapse or deterioration caused by deficient construction techniques. • The young participants trained in the construction techniques workshops can use that knowledge for the masonry trade in their communities and beyond. • For municipal governments, having validated action protocols and an agile computer tool such as the mobile application for the evaluation of damage to buildings after earthquakes will mean great savings by avoiding the waste of available resources in the duplication and repetition of processes, poor distribution of aid, etc. Another product of the program, such as the methodology for locating sites for debris disposal, has a high potential economic impact, by collaborating in a faster recovery of the territories affected by an earthquake and avoiding the excess cost and waste of resources involved in the management of debris in emergency situations, without having taken the necessary precautions to do so. • For families affected by disasters of various causes, accessing official aid in a timely manner will enable them to recover their livelihoods more quickly and reduce their dependence on state aid. • For the professionals of the associations, taking the courses on rapid and detailed assessment of damage to buildings is an asset within their professional practice, since this is a highly demanded and well-paid service, not only by local governments in case of emergencies, but also by individuals and private companies, including transnational ones. Social Impacts: • To reduce human suffering and unnecessary loss of life due to natural disasters whose impacts can be lessened. • The incorporation of programs on risk management and damage assessment in university courses related to construction can have a strong and lasting impact on reducing the vulnerability of cities, by integrating anti-seismic design principles into the variables that are normally considered as determining factors for the work of architects and engineers as designers of buildings and service infrastructures. It can also have a strong impact on young people, by making them aware of the risks in their environment, which for many go unnoticed, and of the responsibilities that they are called upon to assume, as future professionals, in the management of these risks. • Having a well-articulated damage assessment system will help to provide a more respectful and humane treatment to individuals and families affected by disasters, eliminating the re- FINAL EVALUATION OF PREPARE MEXICO | 37 victimization caused by the repeated assessment of damages and losses by different institutions that act in an uncoordinated manner. • The centralization of damage assessment information and transparency in its collection and management will generate greater public confidence in the role of the government's emergency response system institutions. Political Impacts: • The efficiency of the systems, protocols and methodologies created within the framework of the program will enable public institutions to fulfill the social purposes and responsibilities for which they were created. • Establish a political culture of planning as opposed to reacting to events, and a systemic and collaborative management among the different state agencies. • The availability of reliable information for decision-making will make it possible to channel public policies in a way that is more in line with the country's reality. LESSONS LEARNED 1. When the strategic working principles guiding the intervention, such as broad stakeholder participation and the use of local knowledge and experience, are adequately integrated into the elaboration of products through specialized external consultancies, the results obtained are highly technical and at the same time completely rooted in the real conditions of the environment, and their full appropriation by the beneficiaries is enhanced. To achieve this, the profile of the contracted specialists is very important. In addition to technical solvency, they must have knowledge of public policies and pedagogical and participatory methodologies, so that the products are adapted to the resources and degree of preparation of the officials of the institutions called upon to apply them. 2. Having a robust and multidisciplinary local technical team for program implementation is important to be able to understand and respond to the different dimensions and the great complexity of the issues and situations which are addresses in these types of interventions. 3. Mechanisms are needed to ensure continuity in the implementation of a program during the transition periods between its different phases and technical management teams. These mechanisms should be operational throughout the life of the program, so that the information and instruments generated can be used by the new team members and their partners to provide adequate continuity to their actions. About Component I 4. Working in interinstitutional and multidisciplinary technical groups with broad stakeholder participation and a close, empathetic, and proactive leadership is vital for taking advantage of local experience and knowledge in the construction of products that are useful for the purposes and beneficiaries for which they were created. 5. Before starting a project, even before making the decision to finance it, it is important to have the approval of the government entities that will be partners in its implementation and will be responsible for giving use and continuity to the products. 6. Despite the relationships cultivated during years of work with local authorities, the priorities of these governmental entities are greatly influenced by situations of the national and international political context that are totally beyond the control of the program and that may neutralize its effects and impacts. For this reason, it is essential to implement, in parallel, awareness-raising actions to build public awareness of the importance of these changes, and to work on the construction of networks between public entities and private sector stakeholders, so that they can question and put pressure on the institutions to give continuity to the efforts and achievements obtained. About Component II 7. It is much more difficult to introduce change in highly structured and bureaucratic institutions 38 | FINAL EVALUATION OF PREPARE MEXICO such as universities than in more flexible and autonomous organizations such as professional associations. It is necessary to gauge the impact that can be achieved with each of these types of organizations when it comes to investing efforts in working together with them. 8. In the academic curriculum for the course “Introduction to Damage Assessment in Construction”, the first part, on the different types of risks and the functioning of the systems for their integral management, is the one that has the greatest impact on university students, because it makes them aware of a reality that surrounds them but of which they were not aware, and opens up perspectives on the responsibilities that they can exercise as future professionals within the existing structures for the prevention of and response to these risks. "The importance for architects to know how the design of buildings, the shapes, the enclosures, have a great influence on their seismic resistance capacity, that no one teaches us that in architecture school, and then one makes design decisions based on other considerations, without taking into account the risk that can put the users of these buildings we design. To realize how there are also construction details that we do not take into account and that nobody supervises, and that these issues should be given much more importance in universities and be mandatory in the curricula of all careers that have to do with the design and execution of buildings.” About Component III 9. Involving the local government entities of Zapopan that directly work with the population of vulnerable sectors, have a wide recognition and social acceptance among its population and are located within them, was a decisive element to achieve their participation in the earthquake-resistant construction workshops, in their adequacy and effectiveness, and in the possibility of appropriation by these entities to give them continuity. 10. The population of irregular and vulnerable communities, as the direct builders of a high proportion of the city's buildings, can be the best guarantors of higher construction standards and the safety of buildings in the face of disastrous events, if they are given specialized training and technical tools adapted to their priorities and their environment. 11. The knowledge and awareness of the population about the risks to which their buildings are subjected is essential for them to be able to assess the need to incorporate specific technical elements in the construction of their homes, which may initially make it more expensive, but in the long run will represent savings in terms of repairs, reconstruction and loss of material goods and the lives of their families. 12. The interaction, in the workshops on seismic-resistant construction techniques, between academic experts and local construction workers and inhabitants who were unfamiliar with the techniques, but knowledgeable about their conditions and needs, was essential for the emergence of knowledge that was solidly rooted in reality and, therefore, useful and usable for all participants. 13. Introducing the younger generations to the knowledge of construction techniques appropriate to their environment can have a very high social impact, since they are the ones who in the immediate future will manage the construction of their own homes, and who may be interested in further training for construction in the low-income sectors of the cities a trade with a better technical basis that will raise the safety standards of the entire built environment. 14. Involving women in earthquake-resistant construction workshops has a high potential for empowerment and change in gender relations, both within their families and in their communities, by giving them greater autonomy over the improvement of their homes, and giving them ownership of knowledge that is very valuable, but to which women have not had access, traditionally, due to the gender stereotypes that surround the construction trade. 15. The replicability of the knowledge on seismic-resistant construction techniques cannot be entrusted to the people who participated in the workshops, for many reasons: Many of them will probably not be involved in construction again until their economic situation allows them to purchase the necessary inputs (land, materials, etc.), which can mean several years of INFORMANT OF THE COLLEGE OF ARCHITECTS AND URBAN PLANNERS OF JALISCO FINAL EVALUATION OF PREPARE MEXICO | 39 savings; The specificity of the technical details that make the difference in the resistance of a construction to an earthquake can be easily lost as it is spread by word of mouth; The importance of direct practice in workshops with real materials and scale models for the full understanding of the concepts explained, among people with a very heterogeneous knowledge base and with a terminology on materials and construction elements that varies significantly from one place to another. RECOMMENDATIONS For the authorities of the risk management and disaster response system: 1. Generate updated and detailed knowledge on the conditions of the urban context that influence the possibilities of risk management and adequate response to emergency situations, such as everything related to road infrastructure, geography, geology and hydrography of the terrain at the appropriate scales, etc., so that methodologies such as the location of waste disposal sites can yield more accurate and have useful results. 2. Create a committee with the participation of various agencies and actors that participated in this program, to follow up on its results and keep all interested parties, both governmental and from the private and academic sectors, informed of the progress of its products, so that they can become involved in its sustainability. 3. Establish communication and information management mechanisms between universities, professional associations and risk management and emergency response institutions in order to strengthen their ties of co-responsibility, harmonize the training of professionals and students with the tools used by government agencies, update the information in professional courses and university curricula with new data and official regulations, and know what technical resources trained in damage assessment in buildings are available at any given time to respond to an emergency that may arise. 4. Incorporate the use of the mobile application for rapid damage assessment in disaster response protocols, and continue developing it to take advantage of its full potential, incorporating, for example, functions that make it possible to manage and coordinate the work of the different humanitarian aid organizations involved in assisting the affected population, without generating duplication, deficiencies or mismanagement in the delivery of aid, coordination and administration of emergency groups, etc. 5. Work on the unification of emergency response tools and procedures among the different administrations that make up each metropolitan area of the city, which is absolutely necessary for efficient and effective risk management and emergency response. 6. Create a permanent training program for communities on basic seismic-resistant construction techniques, relying on the agencies that work directly with the communities, such as the Colmenas, the DIF and the Directorate of Delegations and Municipal Services, and taking advantage of the tools and learning generated in the pilot workshops implemented in this program. Expand the coverage of this program, establishing agreements with universities to support the teaching component: "If we do a focused job with the universities, with the engineering schools, I think we could strengthen this component of community training in self-construction techniques. That is one of the strategies that we as a municipality could adopt, to sign an agreement with the universities, so that through products or methodologies such as those that Miyamoto can bring us, we can give continuity to this program.” For the instances of social attention to vulnerable communities in Zapopan: 7. Continue implementing seismic-resistant construction workshops in communities, emphasizing the participation of women and young people, using the media and social networks to strengthen the dissemination of the calls and expand the coverage of participants. Associate these workshops with existing social assistance programs that provide construction materials, in order to enhance their direct impact. ZAPOPAN CIVIL PROTECTION COORDINATION INFORMANT 40 | FINAL EVALUATION OF PREPARE MEXICO For Miyamoto: 8. Generate tools, such as a manual of instructions, protocols and procedures, on how to manage USAID and other major donor projects, to facilitate and shorten the learning curves of the technical implementation teams at the beginning and in the transition between the different phases of the interventions. 9. Establish a system to regularly record and share experiences and key information on the processes carried out in the interventions, so that the accumulated knowledge is not lost when changes are made in the technical personnel assigned to the projects and programs, which should be gradual to ensure an adequate transition. 10. Estimating the monetary value of the impact of each intervention of the activity proposed by the organization, can be a very convincing argument for the government agencies that are called upon to participate in them, and a very useful piece of information both for their justification and for their appropriation by the beneficiaries. 11. Work with official course certification bodies, so that the training provided by Miyamoto in professional environments can certify the skills acquired. This would add much value to these trainings, even in the private sector of national and transnational industry, which usually hires this type of structural damage assessment services. 12. Deepen participatory diagnostics in the problem identification and intervention design stages, to improve the Theory of Change by taking into account uncontrollable risks and threats, and generate a system of objectives, results and indicators that are better adapted to the reality of the context, and that allows the proper measuring of the results and progress towards the objectives, rather than remaining at the level of compliance with activities. The technicians who manage the program and who have first-hand knowledge of the problems encountered in their implementation in the field should always participate in the planning phases of the interventions. "The objectives must be challenging and achievable. I would set different objectives and measure them in a more transcendent way. We have two extremes, very diffuse objectives that you can easily achieve, and very specific ones, that count the participants when we should value the quality, or the knowledge acquired...” For BHA: 13. Start any activity by financing an identification and formulation phase in which all stakeholders participate, to incorporate their conditions, needs and priorities in the design of the interventions to be implemented. 14. Continue to proactively cultivate relationships with authorities and decision-makers, in order to have their commitment to participate and institutionalize the results of the interventions. 15. Given the political instability and lack of continuity in government institutions, push programs to generate social awareness and networks among civil society organizations, the private sector and communities, so that they can serve as elements of pressure on governments, and can support each other autonomously, without the intermediation of the latter, in initiatives of community benefit. 16. Make the implementation of activities coincide with the administrative periods of the government entities participating in them, so that changes in government do not negatively affect the implementation processes put in place. MIYAMOTO'S TECHNICAL TEAM INFORMANT FINAL EVALUATION OF PREPARE MEXICO | 41 ANNEXES ANNEX I. TIMELINE OF THE EVALUATION Spaces for consultation with key informants DATE INTERVIEWED PERSON INSTITUTION / ORGANIZATION 17/05/2023 Javier Macías PREPARE program technical team 17/05/2023 Verónica Hernández Community Center La Colmena de San Juan de Ocotán 17/05/2023 Enmanuel López Cortés Centro de Enseñanza Técnica Industrial CETI 18/05/2023 Rosa Guevara Directorate of Municipal Delegations and Agencies 18/05/2023 Miguel Juárez y Claudio Canales Undersoft, developer of the mobile application 19/05/2023 Alhy Núñez Zapopan Civil Protection Coordination 19/05/2023 Miriam Luna RDA university curriculum development team 19/05/2023 Rigoberto Román López y Eduardo Parra Ramos Enlace Ambiental, developer of the methodology for the identification of debris disposal sites 31/05/2023 Érika Estrada Community Center La Colmena de Villa de Guadalupe 01/06/2023 Yirah Barraza y Miriam Ascencio Institute of Statistical and Geographic Information of Jalisco IIEG 02/06/2023 Silvia Álvarez y Susana Moreno DIF Zapopan 05/06/2023 José López, Javier Macías, Diego Barrón y Tadeo Cepeda PREPARE program technical team 09/06/2023 María Elena González Ruiz College of Architects and Urban Planners of the State of Jalisco 12/06/2023 Jesús Valdez Miyamoto International 13/06/2023 Sandra Miranda College of Civil Engineers of the State of Mexico 20/06/2023 Javier Macías y Tadeo Cepeda PREPARE program technical team 24/06/2023 Irwin Mazarego y Gabriel Guzmán International University of Knowledge and Research UNICI 27/06/2023 Joel Alejandro Mendoza Mesa de Oriente Community 27/06/2023 Edgar Aparicio Villas de Guadalupe Community 27/06/2023 Darío Rea La Colmena Community 29/06/2023 Nicolás Juárez Instructor of basic seismic-resistant construction practices workshop 15/08/2023 Peter Schecter USAID 42 | FINAL EVALUATION OF PREPARE MEXICO ANNEX II. EVALUATION STATEMENT OF WORK PLAN DE EVALUACIÓN PROGRAMA PREPARE MÉXICO Preparación para reducir el impacto social y económico de los terremotos MAYO 2023 FINAL EVALUATION OF PREPARE MEXICO | 43 1. INTRODUCCIÓN 1.1 Finalidad y objetivos de la evaluación El propósito fundamental de esta evaluación final externa es valorar los avances alcanzados en el proceso de implementación del programa PREPARE MÉXICO, en función de los resultados esperados y los objetivos previstos en su planificación. Se espera, también, que en el proceso de evaluación se identifiquen impactos, lecciones aprendidas, y se propongan recomendaciones de carácter estratégico y operativo, que colaboren en la sostenibilidad de los resultados alcanzados. Objetivos específicos • Evaluar la efectividad e impacto de las actividades del programa de acuerdo a los objetivos e indicadores establecidos. • Identificar y documentar las lecciones aprendidas de las diversas actividades y procesos emprendidos, así como el impacto del programa y sus resultados no planificados, tanto positivos como oportunidades de mejora, así como identificar las medidas de sostenibilidad del mismo. 1.2 Alcance y cobertura de la evaluación La evaluación final abarcará los 32 meses de implementación del Programa PREPARE MÉXICO Preparación para Reducir el Impacto Social y Económico de los Terremotos, entre septiembre de 2020 y junio de 2023, y tendrá en cuenta todos sus objetivos, efectos, resultados e indicadores, planificados o no, además de los principios estratégicos con los que trabaja de forma transversal, como son : una amplia participación de las partes interesadas para favorecer la colaboración interinstitucional y la construcción de redes público-privadas, promoviendo la participación balanceada de mujeres, personas mayores y otros grupos marginalizados; el aprovechamiento de la experiencia local e internacional para el intercambio y transferencia de conocimientos técnicos, potenciando las lecciones aprendidas en las experiencias de Miyamoto International en respuesta a más de 100 terremotos en los últimos 30 años; y la evaluación, planificación y asistencia técnica integradas, para optimizar el impacto y los costos de la intervención, a través de intercambio de información y de acciones coordinadas que se refuerzan mutuamente.17 El análisis de los resultados del programa se basará en los criterios de pertinencia, eficacia, eficiencia, participación, sostenibilidad e impacto de la Organización para la Cooperación y el Desarrollo Económico/Comité de Ayuda al Desarrollo (OCDE/CAD). Así, la evaluación valorará la ejecución final del programa, identificando los puntos fuertes y las limitaciones, a través del análisis de los criterios predefinidos. Evaluará principalmente el grado de consecución de los objetivos específicos y sus resultados subyacentes desde un punto de vista cuantitativo y cualitativo, para el beneficiario en general, y de forma diferenciada para mujeres y hombres. Por último, basándose en las pruebas recogidas, la evaluación formulará recomendaciones programáticas y orientaciones estratégicas para futuras intervenciones. 2. RESUMEN DEL PROGRAMA 2.1 Justificación El programa PREPARE MÉXICO, implementado por Miyamoto Internacional, surge en 2017, a raíz de los terremotos que ocurrieron ese año que demostraron que, a pesar de los importantes esfuerzos realizados en el país para fortalecer la capacidad de manejo de riesgos y respuesta a desastres de las autoridades, todavía persisten profundas debilidades que limitan la eficacia del sistema, y que evidencian la necesidad de la aprobación y entrenamiento, previos a las emergencias, de metodologías, protocolos 17 USAID/OFDA PREPARE Mexico: Technical Proposal. Preparing to Lessen the Social and Economic Impact of Earthquakes in Mexico. 44 | FINAL EVALUATION OF PREPARE MEXICO y herramientas de respuesta, y de la coordinación entre la sociedad civil profesionalizada y las agencias gubernamentales responsables. 18 Una de las más importantes debilidades identificadas es la escasa capacidad de evaluación de daños post sismo en edificaciones, en la ciudad de México invirtió 18 meses después de los eventos sísmicos, dejando en un limbo a muchos edificios que no habían recibido el dictamen legal que estableciera su demolición o su reparación.19 Entre las causas de este mal funcionamiento se señalaron la falta de coordinación entre los diferentes actores de la respuesta a emergencias, la ausencia de una metodología y una herramienta de evaluación adecuadas y unificadas, la falta de información oficial disponible, confiable y estandarizada que permitiera a los organismos competentes tomar decisiones acertadas, y el déficit de especialistas capacitados para dirigir estas evaluaciones y emitir dictámenes técnicos vinculantes. Por otra parte, la exposición al riesgo sísmico en las zonas urbanas está estrechamente vinculada a los bajos estándares de calidad en las prácticas constructivas, especialmente en las zonas de asentamientos informales, en las que habita un alto porcentaje de la población de las ciudades y que constituyen los sectores que más aceleradamente crecen dentro de ellas. En estos sectores, tanto las edificaciones como su entorno se construyen de manera autoproducida por sus habitantes, sin ninguna supervisión de las autoridades y con poca implicación de profesionales de la construcción, por lo que los esfuerzos realizados en el fortalecimiento de las normativas urbanísticas y estructurales no producen ningún efecto, lo que exacerba su vulnerabilidad ante todo tipo de amenazas. La fase actual del programa PREPARE, implementada entre septiembre de 2020 y junio de 2023 en los municipios de Zapopan, Guadalajara y Xochimilco y Benito Juárez en Ciudad de México, se diseñó para abordar estas vulnerabilidades y debilidades del sistema de gestión de riesgos y respuesta a desastres, en continuación de la primera fase que se implementó entre los años 2018 y 2020, en la cual se elaboraron evaluaciones, herramientas y guías, que en esta fase debían evolucionar para ser utilizadas en casos de emergencia por los organismos responsables, adaptándolas a las normativas locales, desarrollando planes de acción detallados, clarificando las responsabilidades y alcance de la participación de cada una de las instituciones y socios del sector privado involucrados, y fortaleciendo la generación de recursos humanos capacitados para ponerlas en práctica. 2.2 Diseño Técnico del Programa OBJETIVO GENERAL Fortalecer e institucionalizar la capacidad de respuesta y gestión de riesgos de las autoridades públicas de desastres y sus socios del sector privado, en asentamientos urbanos altamente vulnerables. OBJETIVO ESPECÍFICO INDICADORES 1. Facilitar la incorporación de buenas prácticas, estrategias y herramientas de manejo de riesgos y respuesta a desastres en los procesos de políticas públicas y planeamiento. 1.1 Número de planes, políticas y estrategias de reducción de riesgos, preparación y contingencia, desarrollados y en funcionamiento. 1.2 Número de personas participando en discusiones sobre estrategias de reducción de riesgos. 1.3 Evaluaciones de riesgos, datos e información sobre amenazas y vulnerabilidad, disponibles. 1.4 Porcentaje de actores de manejo de riesgos y respuesta, que citan mejoras en la preparación a nivel municipal. 2. 2.1 Número de empresas del sector privado directamente involucradas en actividades de respuesta o reducción de riesgos. 18 USAID/OFDA PREPARE Mexico: Baseline Report 19 USAID/OFDA PREPARE Mexico: Technical Proposal. Preparing to Lessen the Social and Economic Impact of Earthquakes in Mexico. FINAL EVALUATION OF PREPARE MEXICO | 45 Fortalecer la colaboración público￾privada en la aplicación de herramientas de gestión y respuesta. 2.2 Número de personas que se benefician indirectamente de las actividades de reducción de riesgos por desastres. 2.3 Número de planes gubernamentales de contingencia que involucran al sector privado. 2.4 Asociaciones público privadas incluidas en protocolos de respuesta municipal a emergencias. 3.1 Número de currículos de RRD desarrollados, por cada nivel educativo. 3.2 Número de estudiantes educados en RRD. 3.3 Número de programas relacionados con RRD, establecidos dentro de las instituciones educativas. 3.4 Porcentaje de participantes en los cursos de ERD que retienen los conocimientos adquiridos después de dos meses. 3. Incrementar el conocimiento de técnicas básicas de construcción sismo-resistente, entre la población vulnerable de los barrios de alto riesgo. 4.1 Número y género de personas participando en entrenamientos. 4.2 Porcentaje de los participantes que retienen las habilidades y conocimientos adquiridos después de dos meses. 4.3 Porcentaje de participantes en las reuniones de planificación conjunta, que pertenecen a las comunidades locales. 4.4 Porcentaje de los participantes entrenados que aplican los conocimientos y técnicas de construcción sismo-resistente al finalizar el programa. BENEFICIARIOS INDIRECTOS BENEFICIARIOS DIRECTOS 600.000 personas en alto riesgo sísmico en los municipios de Zapopan en Jalisco, y Xochimilco y Benito Juárez en Ciudad de México 50 representantes y autoridades nacionales y municipales de Reducción de Riesgo ante Desastres y del sector privado. 500 profesionales y técnicos entrenados en evaluación de daños de edificaciones afectadas por terremotos, y 150 personas de comunidades vulnerables entrenadas en técnicas de construcción sismo-resistente. 2.3 Partes interesadas Siguiendo el enfoque estratégico del programa, se ha involucrado ampliamente a las partes interesadas de los distintos niveles de gobierno y del sector privado en la planificación e implementación de las acciones, así como en el mapeo de recursos, capacidades y oportunidades de colaboración, y en la definición de roles y compromisos institucionales. Algunos de los actores que han tenido una participación más sostenida son los siguientes: CIUDAD GOBIERNO NACIONAL Y ESTADAL GOBIERNO LOCAL SOCIEDAD CIVIL/ SECTOR PRIVADO Zapopan Instituto de Información Estadística y Geográfica IIEG SEMADET Jalisco IMEPLAN Unidad de protección civil y bomberos de Jalisco PC y B Tlaquepaque Coordinación municipal de protección civil y bomberos Servicios Municipales Comisaría de Zapopan Obras Públicas DOPI SEMADET Zapopan UAG, CETI, UTEG, UNE, Instituto Tecnológico de Colima, Instituto del Conocimiento e Investigación CICEJ, COMICIJ, ARISE Undersoft, Enlace Ambiental Ingenieros 46 | FINAL EVALUATION OF PREPARE MEXICO DIF Zapopan Colmenas Habitantes de comunidades informales de la ciudad Ciudad de México SGIRPC CNPC CONAIC ITC Municipalidad de Xochimilco Municipalidad de Benito Juárez Protección civil Xochimilco Protección civil Benito Juárez UNAM, UPAEP, UI, ITESM, UNIRED, UDG, National technological of Mexico Colima y Tláhuac, ITESO, UTEC CMIC, CMOC, ARISE, ADOC, ERUM. DAMAHR 3. METODOLOGÍA DE EVALUACIÓN 3.1 Matriz de Evaluación A partir del análisis detallado de la documentación clave del programa, se ha elaborado la siguiente matriz de evaluación como marco metodológico del trabajo evaluativo, en la cual se formulan preguntas de evaluación que buscan abarcar todas las dimensiones y criterios de análisis del objeto de estudio, incluyendo los enfoques estratégicos transversales y la afectación por la crisis del Covid-19. Esta matriz relaciona cada una de las preguntas con los informantes clave y fuentes de información disponibles para contestarlas, y con las herramientas de investigación que se aplicarán con cada una de estas fuentes de datos. CRITERIOS Y PREGUNTAS DE EVALUACIÓN HERRAMIENTAS DE INVESTIGACIÓN FUENTES DE INFORMACIÓN PRIMARIAS FUENTES DE INFORMACIÓN SECUNDARIAS PERTINENCIA Y ADAPTABILIDAD ¿En qué medida el programa responde a las prioridades y mandatos de las instituciones y organizaciones que forman parte del SRRD de México? Análisis documental Entrevistas Semiestructura￾das Grupos de discusión Funcionarios de instituciones y organizaciones participantes Técnicos del programa Habitantes de comunidades Línea de base Propuesta técnica Documentación de contexto Informes de seguimiento ¿En qué medida se adaptaron las actividades del programa a los hallazgos de la línea de base? ¿En qué medida el programa responde a las prioridades de las comunidades de barrios informales de Zapopan? ¿Existe una percepción del riesgo sísmico en estas comunidades? ¿En qué medida la crisis del Covid-19 y otros cambios del contexto han tenido un impacto en los resultados alcanzados, y qué estrategias de adaptación se adoptaron? ¿Cómo se adaptaron los contenidos de los talleres a las características de los procesos constructivos en los barrios informales de Zapopan? ¿Quién participó en esta adaptación? EFICIENCIA FINAL EVALUATION OF PREPARE MEXICO | 47 ¿Fue suficiente y adecuada la distribución de recursos humanos y materiales del programa para la obtención de sus objetivos? Análisis documental Entrevistas semiestructura￾das Partes interesadas participantes en las actividades Técnicos del programa Propuesta técnica Informes de seguimiento Informes de productos ¿En qué medida el programa ha aprovechado la experiencia y conocimiento interno liderado por las organizaciones e instituciones del país? ¿En qué medida se ha aprovechado el conocimiento y experiencia producido en otros países de la región? PARTICIPACIÓN ¿En qué medida la participación de las partes interesadas locales ha determinado y ayudado a desarrollar los productos y resultados del programa? Análisis documental Entrevistas semiestructura￾das Grupos de discusión Partes interesadas participantes en las actividades Técnicos del programa Habitantes de las comunidades Informes de seguimiento Documentos del programa Tablas de participación de actores ¿En qué medida los productos y resultados han incluido la participación y la rendición de cuentas a la población afectada, incluidas las consideraciones de género? ¿En qué medida se incorporaron las prioridades y necesidades de las partes interesadas en el diseño de las acciones del programa? ¿La metodología y estrategias implementadas permitieron la inclusión de las personas más vulnerables de las comunidades? EFICACIA ¿En qué medida se ha logrado la meta y los objetivos generales del programa? Análisis documental Entrevistas semiestructura￾das Grupos de discusión Partes interesadas participantes en las actividades Técnicos del programa Funcionarios de las instituciones involucradas Habitantes de las comunidades Línea de base Propuesta técnica Informes de seguimiento Tablas de indicadores Tablas de participación de actores ¿Qué actividades del programa han sido más efectivas para alcanzar los objetivos establecidos? ¿Cuáles han sido los principales desafíos para lograr las metas y objetivos del programa? ¿Ha aumentado la coordinación y comunicación entre organismos, instituciones y agencias como resultado de la participación en el proyecto? En ZPP, ¿Se avanzó en la incorporación de los hallazgos de la evaluación de vulnerabilidad de la ciudad realizada en 2019 en los planes de gestión de riesgos? En CDMX, ¿se avanzó en el desarrollo del marco operativo necesario para operativizar el despliegue de la capacidad técnica de ERD y EDD fortalecida en la fase anterior, con 249 especialistas formados? 48 | FINAL EVALUATION OF PREPARE MEXICO ¿Las herramientas desarrolladas en el marco del programa han sido puestas en práctica por las partes interesadas? Informes de productos ¿Se han establecido los currículos de RRD dentro de las instituciones educativas? ¿Ha mejorado la colaboración y coordinación público-privada? ¿Se ha logrado escalar la capacidad de despliegue de ERD gracias a esa colaboración? ¿Los participantes en actividades de fortalecimiento y creación de capacidades han demostrado que han retenido el conocimiento adquirido después de terminadas las capacitaciones? ¿Se ha involucrado al sector privado en los planes de contingencia y los protocolos de respuesta municipal a emergencias? ¿Qué lecciones se pueden aprender de PREPARE MEXICO que puedan ser replicadas en otros casos del país y la región? SOSTENIBILIDAD ¿En qué medida las actividades del programa han contribuido a cambios o resultados sostenibles, y qué factores han contribuido a lograr la sostenibilidad? Análisis documental Entrevistas semiestructura￾das Grupos de discusión Técnicos del programa Partes interesadas participantes Habitantes de las comunidades Informes de seguimiento Informes de productos ¿Se han establecido mecanismos para mantener la colaboración público-privada más allá de la vida del programa? ¿En qué medida el programa ha contribuido a mejorar la autosuficiencia de las organizaciones e instituciones participantes? ¿Cómo se estableció la estrategia de salida y el plan de sostenibilidad? IMPACTO ¿En qué medida los cambios logrados han provocado o pueden provocar efectos sociales, económicos y/o políticos directos o indirectos en la zona de intervención? Entrevistas semiestructura￾das Partes interesadas Técnicos del programa Habitantes de comunidades Documentos del programa ¿Cómo pueden potenciarse estos efectos y qué factores del contexto pueden condicionar el logro de estos efectos e impactos? 3.2 Principios rectores de la Evaluación Los principios y enfoques que guiarán el proceso de evaluación son los siguientes: Participación de actores: Se facilitará la inclusión de representantes de los diversos grupos meta del programa, los equipos de las organizaciones socias en su implementación y otros socios estratégicos FINAL EVALUATION OF PREPARE MEXICO | 49 pertinentes, porque su participación legitima el ejercicio evaluativo y reflexivo e incrementa la apropiación de los resultados y recomendaciones de mejora. Enfoque Apreciativo: La evaluación se focalizará en lo que existe, en las aspiraciones de lo que se desea lograr y en lo que ha funcionado bien, identificando ejemplos de buenas prácticas en las que se puede basar una futura etapa de intervención. Los desafíos encontrados en el logro de los objetivos se valorarán como oportunidades de aprendizaje y mejora, para identificar lecciones aprendidas y áreas temáticas en las que se podrían realizar cambios para mejorar el desempeño y los resultados. Análisis consensuado: Durante las fases de la evaluación se tendrán espacios de diálogo consensuado con el equipo técnico del programa, para la toma de decisiones en relación con el proceso evaluativo y para ir validando los hallazgos y los resultados de la evaluación. Prioridades transversales: Se analizará el tratamiento que se le ha dado a la incorporación de los enfoques estratégicos transversales priorizados por el programa, como el de género, enfoque sistémico y el enfoque vecinal. Rigurosidad en la recolección y procesamiento de datos: La evaluación se llevará a cabo cumpliendo siguiendo un sistemático proceso de recopilación y procesamiento de la información cuantitativa y cualitativa. En este sentido, será fundamental el proceso de revisión previa de la documentación de que se dispone del proyecto y el contexto, y el consenso de los criterios de la evaluación y las herramientas de recolección de información en función de los datos que se requieren. La triangulación de la información se utilizará como un mecanismo para evitar posibles sesgos, y como garantía de alcanzar resultados fiables. Respeto y confidencialidad: En el proceso de recolección de información, se tendrá especial cuidado por el respeto alos derechos de todas las personas que participan como informantes, como son: el respeto a la dignidad humana, el respeto al consentimiento libre e informado, la privacidad y la confidencialidad, la inclusión, y el reconocimiento del potencial de daño y la maximización de los beneficios para todas las partes interesadas. 3.3 Enfoque metodológico Partiendo de estos principios básicos, se busca llevar a cabo un trabajo de calidad bajo un modelo de Evaluación Sistémica que aporte conclusiones sobre la operativa del programa y sus perspectivas de sostenibilidad. El modelo de Evaluación Sistémica permitirá evaluar las diferentes dimensiones implicadas en el programa, teniendo en cuenta que éste describe la forma en que el diseño, la estructura y los procesos, se traducen en resultados e impactos, por medio de las diferentes relaciones establecidas. 50 | FINAL EVALUATION OF PREPARE MEXICO Se plantea un análisis de contribución que determine los factores que han permitido u obstaculizado el progreso hacia los resultados esperados y que permita establecer con un elevado nivel de credibilidad las relaciones causa-efecto, valorando críticamente la teoría del cambio con una perspectiva sistémica. Así, se revisará o validará el marco teórico que establece relaciones entre Necesidades – Recursos – Actividades – Productos - Resultados (en el período establecido), y su sostenibilidad futura, a lo largo de las diferentes dimensiones del proyecto (diseño, estructura, procesos (gestión) y objetivos/resultados). Para ello la evaluación combinará métodos mixtos (cualitativos y cuantitativos) para investigar las relaciones causales entre los productos y los resultados, y aplicará un sistema de triangulación de la información a los datos recogidos de las diversas fuentes identificadas mediante herramientas que se elaborarán para cada una de ellas. 3.4 Herramientas de recolección de información La información de datos cuantitativos y cualitativos existentes en la documentación del programa será complementada con datos nuevos que serán recolectados de las fuentes primarias, mediante la aplicación de entrevistas semiestructuradas y grupos de discusión a una muestra cualitativa de informantes clave20, de manera que sirvan como elementos de contraste de la teoría que sustenta el programa con su realidad empírica. La muestra, de carácter cualitativo, no busca la representatividad estadística, sino la potencialidad del proceso de comprensión teórica de la intervención evaluada, para lo cual la profundidad del análisis es el elemento clave. La necesidad de este análisis detallado de las unidades seleccionadas en la muestra impide abarcar el número de informantes que daría representatividad estadística a la evaluación, ya que sería inviable su implementación. En las metodologías cualitativas como la que aquí se plantea, la muestra debe ser suficiente para responder al objeto de estudio, pero no tan grande que impida su análisis en profundidad. De esta manera, el diseño de la muestra a estudiar no es aleatorio sino intencional, ya que se busca abarcar los diferentes discursos y significados que los individuos involucrados asignan al objeto de estudio según las diferentes posiciones de cada uno de ellos ocupa en la estructura sociopolítica en la que este se inserta. Bajo estos criterios, se ha definido la realización de entrevistas semiestructuradas y grupos de discusión, los cuales se realizarán a distancia a través de plataformas digitales. La cantidad y tipo de espacios de consulta y los actores consultados podrán cambiar durante el transcurso de la evaluación, en función del criterio de saturación de la información obtenida, por la disponibilidad de las fuentes y del tiempo para el periodo de consulta, las posibilidades de agrupación de distintos actores en un solo espacio de consulta, o por el interés de incorporar informantes que no se han identificado en la revisión documental. La propuesta inicial, a depurar con la participación del equipo técnico del programa, es la siguiente: GRUPO DE ACTORES INFORMANTE CLAVE / INSTITUCIÓN REPRESENTADA HERRAMIENTAS A UTILIZAR Equipo técnico de implementación Javier Macías Entrevistas semiestructuradas José López Tadeo Cepeda Diego Barrón Undersoft Enlace ambiental Ingenieros 20 Luis Mena Martínez 2017: Muestra cualitativa, una propuesta integradora. En Investigación cualitativa en Ciencias Sociales, volumen 3. Departamento de sociología y comunicación, Universidad de Salamanca. FINAL EVALUATION OF PREPARE MEXICO | 51 Instituciones nacionales y estadales Coordinación nacional de protección civil CNPC Entrevistas semiestructuradas Instituto de Información Estadística y Geográfica IIEG Instituto de Planeación y Gestión del Desarrollo IMEPLAN Sistema de Gestión Integral de Riesgos y Protección Civil SPIRPC Instituto Técnico de la Construcción ITC Instituciones municipales en Zapopan Protección civil y bomberos de Zapopan Entrevistas semiestructuradas Municipalidad de Zapopan Comisaría de Zapopan Obras públicas DOPI SEMADET Zapopan DIF Zapopan Colmena comunidades del norte Colmena comunidades del sur Instituciones municipales en Cuidad de México Municipalidad Xochimilco Entrevistas semiestructuradas Protección civil Xochimilco Municipalidad Benito Juárez Protección civil Benito Juárez Organizaciones de la sociedad civil / sector privado en Zapopan Centro de Enseñanza Técnica Industrial CETI Entrevistas semiestructuradas Universidad de Guadalajara Colegio estatal de ingenieros civiles de Jalisco CICEJ o Colegio Metropolitano de Ingenieros Civiles de Jalisco COMICIJ Alianza del sector privado para sociedades resilientes ante desastres ARISE Organizaciones de la sociedad civil / sector privado en Ciudad de México Universidad Autónoma de México UNAM Entrevistas semiestructuradas Instituto Tecnológico de Monterrey ITESM Cámara mexicana de la Industria de la Construcción CMIC Colegio de Ingenieras y Arquitectas del Estado de Oaxaca DAMAHR Habitantes de comunidades del norte de Zapopan Albañiles y trabajadores/as de la construcción Grupos de discusión Mujeres y jefes/as de hogar 52 | FINAL EVALUATION OF PREPARE MEXICO Habitantes de comunidades del sur de Zapopan Albañiles y trabajadores/as de la construcción Grupos de discusión Mujeres y jefes/as de hogar Análisis documental: Se basa en un proceso, ya se ha iniciado, donde se revisa y analiza la documentación clave generada por el programa como la propuesta técnica, informe de línea de base, datos del sistema de monitoreo y evaluación, informes de avance presentados a USAID/BHA, informes de productos específicos, actas de reuniones con las instituciones y socios y documentación relevante del contexto, utilizando el método del análisis cualitativo del contenido. Se realiza, asimismo, la explotación o análisis secundario de los datos cuantitativos existentes en los informes de seguimiento del programa, tablas de valoración de los indicadores, listados de participación en las actividades y otros. A partir de estos análisis, se redactan los diferentes guiones de entrevista y formularios de encuesta, según cada grupo de partes interesadas y la información que se requiere para contestar las preguntas de evaluación y contrastar la información encontrada en las fuentes documentales. Las técnicas de recolección de información de fuentes primarias que se utilizarán, son las siguientes: Entrevistas semiestructuradas: Tienen como objetivo recabar información detallada y explicativa sobre la percepción y valoración de los actores vinculados al programa y los equipos técnicos responsables sobre los procesos puestos en marcha para la implementación de las acciones y sobre los resultados e impactos alcanzados. Grupos de discusión: Es una entrevista grupal alrededor de un tema o experiencia específica, en la cual se busca la interacción entre los participantes, como método para generar información, sentidos y significados de la experiencia discutida Reuniones de discusión y reflexión: A partir del informe preliminar, se realizará una reunión virtual con participación del equipo técnico, para la presentación y discusión de los resultados preliminares de evaluación, en la cual se estimulará la reflexión conjunta como medio de contraste y retroalimentación de estos resultados para determinar prioridades, formular recomendaciones útiles y realistas y realizar los ajustes necesarios para la elaboración de la versión final del informe. 3.5 Análisis y producción de resultados Este es el momento de descomponer los procesos, a partir de la información recogida, en los elementos y etapas que los constituyen, para comprender sus lógicas internas y las diferentes relaciones que se establecieron entre sus sujetos, identificar las contradicciones y conflictos que los marcaron, y las interrelaciones que se establecieron entre sus distintos aspectos. La propuesta de integración de distintos métodos de recolección de datos y la triangulación de distintas fuentes para responder a las interrogantes, permite superar las limitaciones de cada fuente de datos individual, mejorando la fiabilidad de las constataciones cuando la información de distintas fuentes de datos converge, y profundizando la comprensión del programa, sus efectos y su contexto. La interpretación de los datos se basará principalmente en la frecuencia y secuencia de sucesos (por ejemplo, en diferentes muestras, o en momentos diferentes) y en patrones de co-ocurrencia (con el fin de identificar conceptos fuertemente relacionados dentro del conjunto de documentos o registros). Los resultados se sintetizarán y clasificarán por cada una de las preguntas de evaluación, realizando triangulaciones entre las distintas respuestas a cada una para determinar patrones recurrentes y en qué medida estas respuestas convergen, corroborando los hallazgos y conclusiones, o se diversifican, en cuyo caso hay que profundizar el análisis para determinar las implicaciones y explicaciones de estas divergencias. FINAL EVALUATION OF PREPARE MEXICO | 53 Por último, la comparación de los resultados con la línea de base y los indicadores de resultados, permitirá evaluar el rendimiento de las intervención con respecto a lo planificado en su diseño. 4. ETAPAS DE EVALUACIÓN Y CRONOGRAMA El proceso de evaluación tendrá una duración de 8 semanas, una semana más del tiempo especificado en los TdR, tomando en cuenta la interrupción de actividades por el asueto navideño, y estará dividido en las siguientes etapas de trabajo: ETAPAS ACTIVIDADES/PRODUCTOS Etapa 1 PLANIFICACIÓN (2 semanas) Reuniones de trabajo con coordinadores y oficiales de programa, para acordar el proceso operativo de la evaluación, con énfasis en su enfoque formativo. Definición del Plan y Cronograma de trabajo, incluyendo metodología y calendario de reuniones con el equipo de coordinación del programa para el seguimiento del proceso. Definición de muestra de sujetos participantes y grupos meta, para la recolección de información de fuentes primarias. Lectura, revisión y análisis de documentación del proyecto y su contexto. Diseño y validación de instrumentos y guías para la recolección de información de fuentes primarias. Producto 1: PLAN DE EVALUACIÓN Etapa 2 RECOPILACIÓN DE INFORMACIÓN, ANÁLISIS E INTERPRETACIÓN (3 semanas) Organización de los aspectos logísticos y de coordinación para la realización de entrevistas y encuestas. Aplicación virtual de instrumentos de recolección de información, entrevistas semiestructuradas y encuestas Degrabación de entrevistas y vaciado, ordenamiento, sistematización y análisis de la información obtenida. Etapa 3 REDACCIÓN Y RETROALIMENTACIÓN DE INFORME PRELIMINAR (2 semanas) Elaboración del informe preliminar de evaluación. Presentación de resultados, hallazgos y aspectos sustantivos de la evaluación, y retroalimentación del informe preliminar con el equipo técnico del programa y organizaciones socias. Producto 2: INFORME PRELIMINAR DE EVALUACIÓN Producto 3: PRESENTACIÓN DE HALLAZGOS EN POWER POINT Etapa 4 REDACCIÓN Y PRESENTACIÓN DEL INFORME FINAL (2 semanas) Ajuste y presentación de informe final de evaluación con los cambios y ajustes sugeridos, Producto 4: INFORME FINAL DE EVALUACIÓN, en español e inglés, incluyendo documentos de soporte en anexos. Producto 5: RESUMEN EJECUTIVO DE LA EVALUACIÓN 54 | FINAL EVALUATION OF PREPARE MEXICO ANNEX III. TOOLS FOR COLLECTING INFORMATION FROM PRIMARY SOURCES A. INITIAL EXPLORATORY INTERVIEW WITH THE PROGRAM COORDINATOR PREPARE MEXICO 1. 1. Several program documents mention that the methodology of Technical Working Groups would be used to prepare the main products of the intervention, such as the protocols and procedures for the institutionalization of the ERD and EDD tools. But in the review of the participation tables, it is seen that the people who participated in discussions did so, in a vast majority, in a single session (84.16% in Zapopan where there are no people who have participated in more than two meetings and 76, 78% in MXC, where there are a few people who participated in 3, 4, 5 and 6 meetings), which is not consistent with work processes in technical groups. What, then, was then the methodology used, and who participated? 2. What was the selection criteria for the municipalities in which you worked? 3. What was the reason for requesting a no cost extension of the project? 4. Regarding component three, I have read that it was the replica of a program previously implemented in Ecuador, but to which adaptations were made to the specific context. What adjustments were made to it and who participated in that adaptation process? 5. Do you consider that this component was pertinent within the context of informal settlements in the city? Are they adapted to the forms and strategies followed by the construction processes in the neighborhoods? What percentage of the population of Zapopan lives in informal settlements? 6. 6. It is mentioned that the effectiveness of this component depends on the population being aware about the seismic risk, but the program does not include actions to sensitize public opinion about this threat. Does this awareness exist among the population? Is there any complementary program that takes care of this necessary awareness? 7. What was the criteria for choosing the participants in these earthquake-resistant construction techniques courses and what methodology was used in their implementation? 8. Are the lessons learned from these workshops being applied on the ground? 9. The Monitoring and Evaluation Plan mentions the evaluation processes of all capacity building workshops and courses, and also the implementation of one of these courses as a Pilot Plan, in January 2022. Were the lessons learned from that pilot implemented to improve actions? What changes were made? 10. Are there reports on the results of the simulations and earthquake drills carried out? Were changes to protocols and procedures made as a result of those actions? What methodology was used to collect these learnings? 11. How were the simulation and simulation exercises implemented, and who participated in them? 12. Were the computer systems for managing the information and data collected in the post￾earthquake damage assessments finally developed? In whose hands were they left? Were the officials of the responsible entities trained in its management? B. INTERVIEW WITH THE PROGRAM IMPLEMENTATION TECHNICAL TEAM 1. What were the tools and procedures developed in the previous phase of Prepare, and what are the advances with respect to them, produced in this phase of the program? 2. Were the baseline findings taken into account in the design and planning of the program's actions? For example, in the definition of the contents and methodologies of the RDA course curriculum. FINAL EVALUATION OF PREPARE MEXICO | 55 3. Given that the CDMX authorities already had free training courses in civil protection that included RDA, and that in PREPARE II specific training in RDA and DDA had already been given to technical personnel and trainers from 16 municipalities, what was the the goal of creating a new course? Were these previous courses (or their evaluations) used as inputs to create it? 4. To what extent were the conditions, priorities and needs of the interested parties incorporated into the design of the program's actions? 5. Did the universities participate in defining the contents and methodologies of the RDA and DRM course curriculum? 6. What work groups were created, which actors participated and how did they work in each city of the program? How was the knowledge generated from one to the other participants in the work tables transmitted, since different representatives of the institutions participated in each meeting? 7. To what extent did the participation of local stakeholders help to shape and develop the products and results of the program, and was the experience and internal knowledge of the organizations and institutions in the country used? 8. In CDMX, was there progress in the development of the necessary operational framework to operationalize the deployment of the technical capacity of RDA and DDA strengthened in the previous phase, with 249 specialists trained? 9. Has public-private collaboration and coordination improved? Has the RDA deployment capacity been scaled thanks to that collaboration? 10. In Zapopan, was there progress in incorporating the findings of the city's vulnerability assessment carried out in 2019 into risk management plans? 11. What was the methodology used in the earthquake simulation and drill carried out, and what were the results obtained from them? Were these findings shared with the interested parties and incorporated into the improvement of the tools and protocols? 12. Are the learning and knowledge acquired in the RDA courses retained after the courses are implemented? 13. What are the obstacles for RDA courses in universities to be accredited and allow their graduates to conduct RDA in emergencies? What is their role restricted to if that is not achieved? 14. Have RDA and DRM curricula been established within educational institutions? 15. To what extent does the program respond to the priorities of the communities of informal neighborhoods in Zapopan? Is there a perception of seismic risk in these communities? 16. How were the participants chosen? Did the methodology and strategies implemented allow the inclusion of the most vulnerable people in the communities? 17. How were the contents of the workshops adapted to the characteristics of the construction processes in the informal neighborhoods of Zapopan? Who participated in this adaptation? 18. To what extent has the Covid-19 crisis had an impact on the strategies and actions of the program and on the results achieved? What other changes in the context, such as political changes, have affected or may affect the program and its effects? 19. Was the distribution of human and material resources of the program sufficient and adequate to achieve its objectives? 20. What actions of the program have been the most effective in reaching the established objective? 21. What have been the main obstacles and challenges in achieving the goals and objectives of the 56 | FINAL EVALUATION OF PREPARE MEXICO program? 22. Has coordination and communication between agencies and institutions increased as a result of their participation in the program? 23. Have the tools developed in the framework of the project been put into practice by the stakeholders? Have they been or will they be used by the organization to which you belong? 24. Are the learnings obtained in the seismic resistant construction courses retained after the courses are finished? Are they being put into practice properly? 25. Are the inhabitants of the communities participating in the seismic resistant construction courses replicating the knowledge acquired to other people in their environment? Are they receiving any follow-up or advice to do it correctly? 26. What lessons can be learned from PREPARE MEXICO that can be replicated in other cases in the country and the region? 27. To what extent have program activities contributed to sustainable changes or results over time, and what factors have contributed to achieving that sustainability? 28. To what extent can the changes achieved cause direct or indirect social, economic or political effects in the area of intervention? 29. What recommendations could you identify for improving the program in a future expansion phase? C. INTERVIEW WITH DRM OFFICIALS FROM LOCAL GOVERNMENTS 1. To what extent were the conditions, priorities and needs of the interested parties incorporated into the design of the program's actions? 2. To what extent did the participation of local stakeholders help to shape and develop the products and results of the program, and was the experience and internal knowledge of the organizations and institutions in the country used? 3. In CDMX, was there progress in the development of the necessary operational framework to operationalize the deployment of the technical capacity of RDA and DDA strengthened in the previous phase, with 249 specialists trained? 4. Has public-private collaboration and coordination improved? Has the RDA deployment capacity been scaled thanks to that collaboration? 5. In Zapopan, was there progress in incorporating the findings of the city's vulnerability assessment carried out in 2019 into risk management plans? 6. What was the methodology used in the earthquake simulation and drill carried out, and what were the results obtained from them? Were these findings shared with the interested parties and incorporated into the improvement of the tools and protocols? 7. To what extent does the program respond to the priorities of the communities of informal neighborhoods in Zapopan? Is there a perception of seismic risk in these communities? 8. To what extent has the Covid-19 crisis had an impact on the strategies and actions of the program and on the results achieved? What other changes in the context, such as political changes, have affected or may affect the program and its effects? 9. Was the distribution of human and material resources of the program sufficient and adequate to achieve its objectives? 10. What actions of the program have been the most effective in reaching the established objective? FINAL EVALUATION OF PREPARE MEXICO | 57 11. What have been the main obstacles and challenges in achieving the goals and objectives of the program? 12. Has coordination and communication between agencies and institutions increased as a result of their participation in the program? 13. Have the tools developed in the framework of the project been put into practice by the stakeholders? Have they been or will they be used by the organization to which you belong? 14. What lessons can be learned from PREPARE MEXICO that can be replicated in other cases in the country and the region? 15. To what extent have program activities contributed to sustainable changes or results over time, and what factors have contributed to achieving that sustainability? 16. To what extent can the changes achieved cause direct or indirect social, economic or political effects in the area of intervention? 17. What recommendations could you identify for improving the program in a future expansion phase? D. INTERVIEW WITH EMPLOYEES OF COLMENAS, DIF AND DIRECTORATES OF MUNICIPAL AGENCIES 1. To what extent were the conditions, priorities and needs of the interested parties incorporated into the design of the program's actions? 2. To what extent did the participation of local stakeholders help to shape and develop the products and results of the program, and was the experience and internal knowledge of the organizations and institutions in the country used? 3. To what extent does the program respond to the priorities of the communities of informal neighborhoods in Zapopan? Is there a perception of seismic risk in these communities? 4. How were the participants chosen? Did the methodology and strategies implemented allow the inclusion of the most vulnerable people in the communities? 5. How were the contents of the workshops adapted to the characteristics of the construction processes in the informal neighborhoods of Zapopan? Who participated in this adaptation? 6. To what extent has the Covid-19 crisis had an impact on the strategies and actions of the program and on the results achieved? What other changes in the context, such as political changes, have affected or may affect the program and its effects? 7. Was the distribution of human and material resources of the program sufficient and adequate to achieve its objectives? 8. What actions of the program have been the most effective in reaching the established objective? 9. What have been the main obstacles and challenges in achieving the goals and objectives of the program? 10. Are the learnings obtained in the seismic resistant construction courses retained after the courses are finished? Are they being put into practice properly? 11. Are the inhabitants of the communities participating in the seismic resistant construction courses replicating the knowledge acquired to other people in their environment? Are they receiving any follow-up or advice to do it correctly? 12. What lessons can be learned from PREPARE MEXICO that can be replicated in other cases in the country and the region? 58 | FINAL EVALUATION OF PREPARE MEXICO 13. To what extent have program activities contributed to sustainable changes or results over time, and what factors have contributed to achieving that sustainability? 14. To what extent can the changes achieved cause direct or indirect social, economic or political effects in the area of intervention? 15. What recommendations could you identify for improving the program in a future expansion phase? E. INTERVIEW WITH INFORMANTS FROM ACADEMIC INSTITUTIONS 1. To what extent were the conditions, priorities and needs of the interested parties incorporated into the design of the program's actions? 2. Did the universities participate in defining the contents and methodologies of the RDA and DRM course curriculum? 3. What work groups were created, which actors participated and how did they work in each city of the program? How was the knowledge generated from one to the other participants in the work tables transmitted, since different representatives of the institutions participated in each meeting? 4. To what extent did the participation of local stakeholders help to shape and develop the products and results of the program, and was the experience and internal knowledge of the organizations and institutions in the country used? 5. Has public-private collaboration and coordination improved? Has the RDA deployment capacity been scaled thanks to that collaboration? 6. Are the learning and knowledge acquired in the RDA courses retained after the courses are implemented? 7. What are the obstacles for RDA courses in universities to be accredited and allow their graduates to conduct RDA in emergencies? What is their role restricted to if that is not achieved? 8. Have RDA and DRM curricula been established within educational institutions? 9. To what extent has the Covid-19 crisis had an impact on the strategies and actions of the program and on the results achieved? What other changes in the context, such as political changes, have affected or may affect the program and its effects? 10. Was the distribution of human and material resources of the program sufficient and adequate to achieve its objectives? 11. What actions of the program have been the most effective in reaching the established objective? 12. What have been the main obstacles and challenges in achieving the goals and objectives of the program? 13. Has coordination and communication between agencies and institutions increased as a result of their participation in the program? 14. Have the tools developed in the framework of the project been put into practice by the stakeholders? Have they been or will they be used by the organization to which you belong? 15. What lessons can be learned from PREPARE MEXICO that can be replicated in other cases in the country and the region? 16. To what extent have program activities contributed to sustainable changes or results over FINAL EVALUATION OF PREPARE MEXICO | 59 time, and what factors have contributed to achieving that sustainability? 17. To what extent can the changes achieved cause direct or indirect social, economic or political effects in the area of intervention? 18. What recommendations could you identify for improving the program in a future expansion phase? 60 | FINAL EVALUATION OF PREPARE MEXICO ANNEX IV. SOURCES OF INFORMATION • USAID/BHA PREPARE México, 2022. Protocolo de Evaluación de Daños para Emergencias • USAID/BHA PREPARE México, 2023. Protocolo ED Zapopan • USAID/BHA PREPARE México, marzo 2022. Protocolo ED Xochimilco • USAID/BHA PREPARE México, marzo 2022. Protocolo ED Benito Juárez • USAID/BHA PREPARE II, 2019. Estudio de Riesgo Sísmico Zapopan • USAID/BHA PREPARE II. Metodología de Evaluación Rápida de Daños en edificaciones afectadas por sismos • USAID/BHA PREPARE II. Plan de Gestión de Residuos Post Sismo • USAID/BHA PREPARE México. Metodología para la selección de sitios para la disposición final de Residuos en Zapopan • USAID/BHA REPARE Mexico. Semi-Annual Performance Report. April-September 2021 • USAID/BHA PREPARE Mexico. Semi-Annual Performance Report. October 2021-March 2022 • USAID/BHA PREPARE Mexico. Semi-Annual Performance Report. April-September 2022 • USAID/BHA PREPARE Mexico. Semi-Annual Performance Report. October 2022-March 2023 • USAID/BHA PREPARE Mexico. Participation Indicator Tracking Tool Zapopan 2023 • USAID/BHA PREPARE Mexico. Participation Indicator Tracking Tool Mexico City 2023 • USAID-BHA PREPARE Mexico. Baseline Report • MIYAMOTO INTERNATIONAL, 2019. Technical Proposal. Preparing to Lessen the Social and Economic Impact of Earthquakes in Mexico (PREPARE Mexico) • USAID/OFDA PREPARE Mexico. Monitoring and Evaluation (M&E) Plan • USAID/BHA PREPARE México. Introducción a la evaluación post-sísmica de construcciones en el contexto de la Gestión Integral de Riesgos. Manual del docente, noviembre 2021 • USAID/BHA PREPARE México. Introducción a la evaluación post-sísmica de construcciones en el contexto de la Gestión Integral de Riesgos. Manual del estudiante, noviembre 2021 • USAID/BHA PREPARE México. Manual de construcción sismo-resistente. Reducción de riesgo de desastres, abril 2021 • PREPARE Mexico Community Training Seismic-Resistant Construction. Nº 1-9 FINAL EVALUATION OF PREPARE MEXICO | 61 ANNEX V. USAID DISCLOSURE OF REAL OR POTENTIAL CONFLICT OF INTEREST FOR EXTERNAL EVALUATION TEAM MEMBERS Name Mildred Guerrero Echegaray Title Consultant in external final evaluation of PREPARE Mexico Program Organization Independent Consultant Evaluation Position X Team Leader Team member Evaluation Award Number (contract or other instrument) USAID Activity(s) Evaluated (Include activity name(s), implementer name(s) and award number(s), if applicable) USAID/BHA Preparing to Lessen the Social and Economic Impact of Earthquakes in Mexico (“PREPARE Mexico”) Implemented by Miyamoto International I have real or potential conflicts of interest to disclose. Yes X No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USAID operating unit managing the activity(s) being evaluated or the implementing organization(s) whose activity(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose activities are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the activity(s) being evaluated, including involvement in the activity design or previous iterations of the activity. CONTINUED If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 62 | FINAL EVALUATION OF PREPARE MEXICO 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose activity(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose activity(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular activities and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Date 22/07/2023 Signature FINAL EVALUATION OF PREPARE MEXICO | 63 ANNEX VI. CV OF CONSULTANT RESPONSIBLE FOR THE EVALUATION 1. PERFIL 1. PROFILE Architect-Urbanist with Master's degrees in Development Politics and Processes and in Local Development. With extensive experience in research applied to development, in academic research and teaching, and in writing and producing publications in the field of self-produced housing in Latin American informal urban settlements. In this field, I have received several awards for academic research, such as the Award for Theory, History and Criticism of Architecture at the XX Pan￾American Architecture Biennial of Quito, in 2016, and the 1st National Housing Award of the National Housing Council of Venezuela, in 1993. 2. EDUCATION • UNIVERSIDAD DE VALENCIA, Spain Master’s degree in Management and Promotion of Local Development. 2013-2014 • UNIVERSIDAD POLITÉCNICA DE VALENCIA, Spain Master’s degree in Development Politics and Processes. 2010-2013 • UNIVERSIDAD CENTRAL DE VENEZUELA (UCV), Caracas Architecture. 1979-1984 • American Language Academy, Washington DC, USA English as a Foreign Language. 1978-1979 3. WORK EXPERIENCE 3.1. Project Evaluation • Miyamoto International: Final evaluation of the program funded by USAID/BHA: “Preparing to lessen the social and economic impact of earthquakes in Mexico. PREPARE MEXICO”. May-July 2023. • Miyamoto International / Global Communities: Final evaluation of the program funded by USAID/BHA: “Guatemala City earthquake reduction program. PREPARE GUATEMALA”. December 2022 - March 2023. • Miyamoto International: Final evaluation of the program funded by USAID/BHA: “Strengthening the deployable capacity for damage assessment in Costa Rica. PREPARE COSTA RICA. December 2023 – March 2023. • GOAL: Systematization of the program funded by USAID: “Cash Response Preparedness Programming in Latin America: Honduras, Guatemala, El Salvador and República Dominicana”. April-August 2022. MILDRED GUERRERO ECHEGARAY Social Researcher, Specialist in Housing, Development and Project Evaluation 64 | FINAL EVALUATION OF PREPARE MEXICO • Heifer Internacional: Systematizacion of the program “PARADELANTE, Livestock business advice for the strengthening of enterprises and businesses”. Guatemala, June-August 2022. • Entraide & Fraternité: Final evaluation of the program “Making the Earth turn fair: Food Sovereignty for all”, chapters of Guatemala and Nicaragua. March-July 2022. • Latin American and Caribbean Network for Democracy - REDLAD: final evaluation of the second phase of the project funded by the Office of Western Hemispheric Affairs of the Department of State of the United States: “Participation of Civil Society in the Summit of the Americas PASCA”. November 2021 – June 2022. • SOLIDARIDAD INTERNACIONAL / Basque Agency for Development Cooperation: Systematization of the project "San Jerónimo Sasha Productive Development Axis, an option of the original peoples of the Caribbean Coast of Nicaragua" May-July 2021. • WE EFFECT: Mid-term evaluation of the project “Strengthening the organizational capacity, leadership and advocacy of the Coordinating Board of Honduran housing cooperatives" February￾May 2021. • Latin American and Caribbean Network for Democracy - REDLAD: Midterm and final evaluation of the first phase of the project funded by the Office of Western Hemispheric Affairs of the Department of State of the United States: “Participation of Civil Society in the Summit of the Americas PASCA”. June 2020 – July 2021. • MÉDICOS DEL MUNDO: Final evaluation of the project “Strengthened health care and protection of migrants and displaced persons forced by violence, reinforcing the civil society responses in Guatemala, El Salvador and Honduras”. October 2020 – February 2021. • JOVESOLIDES: final evaluation of the project: " NOTHING TRIVIAL. Elderly people of the Valencian Community committed to the SDGs through Gamification ". May-June 2019. • ACTION AGAINST HUNGER: Final evaluation of the project: "No Hunger Generation. An artistic and creative proposal for the first generation of Valencians capable of ending hunger. Phase I". July 2018. • ASOCIACIÓN MADRE CORAJE: Final evaluation of the project: “Improving our futuro II. Qipa Kausayninchikta Allinchaspa II Apurímac Perú". March 2017 • FUNDACIÓN DEL VALLE: Final evaluation of the project: "Kametsa Asaiki: The Good Living in the Peruvian Amazon. ", 2012. 3.2. Management of Humanitarian Aid projects • Fundación de Atención Integral Juvenil FUNDINIL: Management of the project funded by UNICEF: WASH in Communities, implemented in El Peñón de Tapipa, Venezuela, Miranda State. November 2022 – May 2023. 3.3. Base Lines • GLOBAL FUND FOR CHILDREN: Proyecto "Empowering teenage girls. Analysis and context mapping ". Guatemala, Honduras and Nicaragua. September-December 2018. • DIAKONÍA: Project "Improving access to justice for Women Survivors of Gender-Based Violence in Honduras, within the Eurojustice Program". Honduras, May-July 2017. • ALIANZA POR LA SOLIDARIDAD: Project "Reflecting on inequality, global justice and human rights in the Valencian university community within the framework of the Objectives of Sustainable Development". March 2017. 3.4. Academic Research FINAL EVALUATION OF PREPARE MEXICO | 65 • Latin American Council of Social Sciences CLACSO / UCV: “Houses of infinite deprivation: Germ of cities for all?” Caracas-Ecuador, June-December 2014. • Universidad Politécnica de Valencia. Master en Políticas y Procesos de Desarrollo: “Reviewing the contours of architecture. A look from the architectural practices and their link with the development processes”. Diciembre 2013. • SELAVIP / UCV / Fundación Polar: “Experimental office of attention to housing in the Barrio Julián Blanco”. Caracas 2003-2005. • Ministry of Urban Development MINDUR / UCV: “Basic technical and social criteria for the establishment of norms for the improvement, consolidation and urban planning of slums”. Caracas, 1994. • National Housing Council / UCV: “Densification and Housing in Caraqueños Slums”. Caracas, 1993. • Council for Scientific and Humanistic Development / UCV: “Densification in Caraqueños informal settlements”. Caracas, 1992. • Ministry of Urban Development MINDUR / UCV: “Study of costs and quality indicators in popular housing”. Caracas, 1988. • Council for Scientific and Humanistic Development / UCV: “The production of housing and urban conditioning in informal settlements of the Caracas Metropolitan Area”. Caracas 1986. 3.5. Teaching • Federation of Entities for Co-development and International Cooperation FEDACOP: Instructor in the workshop: “Identification, Formulation and Monitoring of development projects within the Logical Framework approach”. Valencia, Spain 2012. • Universidad Central de Venezuela, School of Architecture and Urbanism: Professor of the postgraduate course: “Urban popular habitat”. Caracas, March - June 2003. • Universidad Central de Venezuela, School of Architecture and Urbanism: Professor of the Urban Studies Sector subject: “Analysis of Informal Urban Settlements”. Caracas, September 1998 - June 2005. 3.6. Technical Assistance • General Directorate for Cooperation and Solidarity, Generalitat Valenciana: Technical assistance in the Department of Monitoring and Evaluation of International Cooperation Projects. Valencia, Spain, August - December 2011. • PNUD / Fundacomun: " Technical review and design proposals in the Resettlement Plan of the Urban Improvement Project in Barrios de Caracas CAMEBA ". Caracas, October - December 2004. 3.7. Compilation, writing, translation, graphic design and edition of publications • Consejo latinoamericano de Ciencias Sociales CLACSO/Universidad Central de Venezuela UCV: Compilation, writing and edition of the book: “Houses of infinite deprivation. Germ of cities for all?", Quito 2014. https://www.clacso.org.ar/librerialatinoamericana/libro_detalle.php?id_libro=905&pageNum_rs_libros =1&totalRows_rs_libros=886 • Community Development Resources Association, South Africa: Spanish translation and texts final edition of: “BAREFOOT GUIDE I to working with organisations and social change”, and 66 | FINAL EVALUATION OF PREPARE MEXICO “BAREFOOT GUIDE II Learning practices in organisations and social change”. 2012 – 2013. https://issuu.com/dougreeler/docs/bfg1_espanoles • Red CYTED XIV.b / Fundación para el Progreso de la Humanidad FPH / UCV: Graphic design and edition of the book “Neighborhoods in transformation”. Caracas, 2006. • Fundacomun / UCV: Graphic design and edition of the book: “Slums and land ownership. A discussion”. Caracas, 2003. • EPFL / FPH / UCV: Writing, graphic design and edition of the book: “New Opportunities: Planning and participating”. Caracas, 1996. • National Housing Council / Fundación Polar / UCV: Graphic design and edition of the book: “City Makers”. Caracas, 1995. • FPH / UCV: Wrinting, graphic design and edition of the biannual newsletter “Cities of the People. Latin America for the rehabilitation of slums”. Numbers 1 to 14. Caracas, 1995 – 2002. 3.8. Management of construction works and architectural projects. • With Visitación Inversiones SL, Valencia, Spain, 2018-2021: Preparation of projects and budgets, tender management, technical and financial management of construction and architectural supervision in refurbishment of a building of 14 tourist apartments in Valencia, with a total investment amount of more than 400,000 euros. • With MrMoc SRL, Chimoio, Mozambique, 2014-2017: Project development and technical and financial management of works of holiday complexes in Vilankulos, Inhambane Province, management of the participatory design process of a housing development for the relocation of displaced population in Zembe, Manica Province, and works management in the refurbishment of a permanent residence in Beira, Sofala Province. • With UMA, real estate projects and promotions, Benicarló, Spain, 2006-2008: Elaboration of projects and budgets, client relations and coordination of execution of changes and reforms in the construction of a development of 200 vacational homes in Benicarló, Castellón. • With Mutable, architectural projects and constructions, Caracas, Venezuela, 1992-2005: Development and coordination of projects and architectural supervision of construction in institutional and educational buildings and in different developments and renovations of single and multi-family housing. 4. AWARDS AND HONOURS • XX Pan American Architecture Biennial Quito 2016: Prize for Theory, History and Criticism of Architecture, to the book: "Houses of infinite deprivation. Germ of cities for all?" CLACSO / UCV 2014. • Ibero-American Science and Technology Program for Development of CYTED XIV.b Network: Internship scholarship at the Office of the Architect of the Community of Holguin, Cuba. January - February 1996. • National Housing Council: 1st National Housing Research Prize 1993, for the project: “Densification and housing in Caracas slums”. Caracas, UCV / CONAVI. • National Council for Scientific and Technological Research, 1st National Meeting Expo-Vivienda 86: 1st Prize for Novel Researchers in the field of Housing, to the presentation: “Accessibility, Improvement and Growth of Housing in the Informal Urban Settlements” Caracas, FAU / UCV 1986. 5. PUBLICATIONS FINAL EVALUATION OF PREPARE MEXICO | 67 • “La Extensión universitaria en Venezuela”, In: Investigació Acció Participativa en Universitats Politècniques: el cas d’Utòpika a la Universitat Politècnica de València. Editorial UPV, Valencia, 2012. • “La Casa de barrio caraqueña: momentos y problemas de una tipología arquitectónica que muta”. In: Primeras Jornadas de Investigación de la Escuela de Arquitectura Carlos Raúl Villanueva. Ediciones de la Facultad de Arquitectura y Urbanismo UCV. Caracas, 2003. https://arquitecturarazonada.wordpress.com/2013/10/16/la-casa-de-barrio-caraquena-momentos-y￾problemas-de-una-tipologia-arquitectonica-que-muta/ • “Los barrios venezolanos de la transitoriedad a la permanencia”. In: Voces Solidarias. Contribución a la participación para el diseño. Ediciones de la Facultad de Arquitectura y Urbanismo UCV. Caracas, 2001. https://arquitecturarazonada.wordpress.com/2013/08/18/de-la￾transitoriedad-a-la-permanencia/ • “Hombres e instituciones clave de proyectos urbanos”. In: Nuevas oportunidades, Participar y planificar. Escuela Politécnica Federal de Lausanne, Fundación para el progreso de la humanidad, UCV. Caracas, 1996. • “Densificación y vivienda en los barrios caraqueños. Contribución a la determinación de problemas y soluciones”. Ministerio del Desarrollo Urbano y Consejo Nacional de la vivienda, Caracas, 1994. • “¿Arquitectura con arquitectos? La experiencia de la Oficina del arquitecto de la comunidad de Holguín”. Boletín Ciudades de la Gente nª 2, FAU/UCV. Caracas, 1996. 6. PROFESSIONAL TRAINING • IDB OFICCE OF EVALUATION AND OVERSIGHT Responding to COVID-19. Lessons from previous responses to Public Health Crisis. Junio 2020. • REDINFOR PERÚ Systematization of Experiences of Social Programs and Projects. March - april 2018. • UNIVERSIDAD POLITÉCNICA DE VALENCIA Specialization course in elaboration of baselines. August - september 2012. • UNIVERSIDAD POLITÉCNICA DE VALENCIA – Red Utópika Participatory Action Research. June 2012. • UNIVERSIDAD POLITÉCNICA DE VALENCIA Cooperation and Sustainable Development in Architecture and Urbanism. Valencia 2009 7. LANGUAGES NATIVE SPANISH HIGH LEVEL PROFESSIONAL ENGLISH MEDIUM LEVEL PORTUGUESE U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523