1 [Month Year] This publication was produced at the request of the United States Agency for International Development. It was prepared independently by the Florida International University’s Extreme Events Institute. Volcano Disaster Assistance Program (VDAP) 2020 Mid-Term Evaluation Final Report May 2021 2 The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Acknowledgements This report would not have been possible without the cooperation of national volcanic observatories of Argentina, Chile, Colombia, Costa Rica, Ecuador, El Salvador, Guatemala, Indonesia, and Perú, as well as the contribution of the Volcano Disaster Assistance Program (VDAP) training´s beneficiaries from 16 countries in Africa, Asia, and Latin America, who collaborated with the online survey, and the inputs provided by the U.S. Mission Disaster Relief Officers located in several Latin American and Southeast Asia countries. The evaluation team is grateful to the VDAP/ United States Geological Survey (USGS) officials who shared their valuable knowledge and experience in this VDAP collaboration, they are the main actors of this important program. We are particularly grateful to David Ramsey, M&E counterpart at VDAP/USGS, and Gari Mayberry, USAID-USGS Natural Hazards and Disaster Risk Reduction Team Lead at USAID/ Bureau for Humanitarian Assistance (BHA), for their collaboration, commitment, and guidance throughout the evaluation. Evaluation Team Leader Juan Pablo Sarmiento VDAP/USGS David Ramsey USAID/BHA and USGS/VDAP Gari Mayberry USAID/M&E Anna Jacobson FIU Evaluation Team Members Extreme Events Institute: Meenakshi Chabba, Gabriela Hoberman Independent Consultants Alvaro Amigo and Carlos Cardona from SERNAGEOMIN, Chile Allison Huisa Patricia Bittner This evaluation was conducted by Florida International University’s Extreme Events Institute, under the “Volcano Disaster Assistance Program (VDAP) 2020 Mid-Term Evaluation,” cooperative agreement #720FDA20CA00023 with the United States Agency for International Development’s Bureau for Humanitarian Assistance (USAID/BHA). Suggested Citation: Sarmiento, J. P., Ramsey, D., Cardona, C., Amigo, A., Huisa, A., Chabba, M., Hoberman, G. (2021). Volcano Disaster Assistance Program (VDAP) 2020 Mid-Term Evaluation. United States Agency for International Development (USAID). 3 Contents Abstract ............................................................................................................................................. i Acronyms & Abbreviations................................................................................................................ ii Executive Summary........................................................................................................................... 1 Evaluation Purpose and Evaluation Questions .................................................................................. 6 Program Background......................................................................................................................... 9 Evaluation Methods and Limitations............................................................................................... 11 Findings .......................................................................................................................................... 16 Response to the Evaluation Questions....................................................................................................16 Findings by Central Topics.......................................................................................................................27 Challenges................................................................................................................................................47 Conclusions..................................................................................................................................... 49 Recommendations .......................................................................................................................... 50 Annexes - Content........................................................................................................................... 52 Annex I: Evaluation Statement of Work .......................................................................................... 53 Annex II: Evaluation Methods – Survey Results............................................................................... 66 Annex III: Data Collection Instruments............................................................................................ 70 Annex IV: Sources of Information.................................................................................................... 93 o Reported Use of VDAP Tools o List of Persons Interviews o VDAP Donated Instrumentation FY 2018-2020 o References Annex V: Disclosure of Conflicts of Interest ...................................................................................... 127 i Volcano Disaster Assistance Program (VDAP) 2020 Mid-Term Evaluation Abstract This report presents the 2020 mid-term evaluation of the performance and outcomes at the global level of the Volcano Disaster Assistance Program (VDAP), a joint initiative between the United States Agency for International Development (USAID) and the United States Geological Survey (USGS). The evaluation focuses on the program’s effectiveness, relevance, and sustainability. The 2020 mid-term evaluation was commissioned by and reports to the USAID´s Bureau for Humanitarian Assistance (BHA) and the USGS-VDAP and was conducted by Florida International University’s Extreme Events Institute (FIU-EEI). Beneficiary countries expressed satisfaction with the quality and level of technical support they receive from VDAP, which has allowed them to achieve tremendous technical competency through consulting, training of professionals, scientific understanding, and technical knowledge in volcano studies. The countries especially mention the strong ties forged by the deeply helpful, patient, close and considerate working partnerships that VDAP personnel have built with all partner agencies through the years. The current mid-term evaluation confirms the effectiveness and relevance of the VDAP program, which has demonstrated its adaptability to the distinct needs of beneficiary countries, using different state-of-the-art tools and the latest technological developments. Despite the instability of some of the national agencies involved in volcanic monitoring, the reduced human and budgetary resources, and the turnover of dedicated staff, this evaluation found sustainability of the VDAP-initiated processes in all 10 observatories of the nine countries evaluated. Some VDAP services can be improved, and new initiatives can be addressed. However, all of these will require a careful analysis of feasibility and appropriateness while maintaining the technical quality and leadership that has characterized the VDAP program since its inception. ii Acronyms & Abbreviations APS Annual Program Statement ASGMI Association of Iberoamerican Geological and Mining Surveys CalVO California Volcano Observatory CENAPRED National Center for Prevention of Disasters, Mexico CONRED National Coordinator for Disaster Reduction, Guatemala COSUDE The Swiss Agency for Development and Cooperation CSAV Center for the Study of Active Volcanoes, Hawaii CVGHM Centre for Volcanology and Geological Hazard Mitigation, Indonesia CVO Cascades Volcano Observatory dMODELS Software package for modeling crustal deformation near active faults and volcanic centers DOAS Differential optical absorption spectroscopy DRR Disaster Risk Reduction EOS Earth Observatory, Singapore FIU EEI Florida International University, Extreme Events Institute FLIR Forward-looking infrared technology FY Fiscal Year GPS Global Positioning System gTOOLS Software dedicated to the processing of relative-gravity data HVO Hawaiian Volcano Observatory IAA Interagency Agreement IAVCEI International Association of Volcanology and Chemistry of the Earth's Interior IASPEI International Association for Seismology and Physics of the Earth's Interior IGP-OVS Geophysical Institute of Peru, Southern Volcano Observatory IG-EPN Geophysical Institute, Escuela Politécnica Nacional, Ecuador INGEMMET Geological Mining and Metallurgical Institute, Peru INSIVUMEH National Institute of Seismology, Volcanology, Meteorology and Hydrology, Guatemala IRD Institute of Research for Development, France InSAR Interferometric synthetic aperture radar INVOLCAN Canarian Volcanology Institute, Spain IVANS International Volcano Activity Notification System JICA Japan International Cooperation Agency MARN Ministry of the Environment and Natural Resources, El Salvador NOVAC Network for Observation of Volcanic and Atmospheric Change NVEWS National Volcano Early Warning System NVMS National Volcanic Monitoring Services OVDAS Southern Andes Volcano Observatory, Chile OVI Observatorio Volcanológico del Ingemmet, Peru iii OVSICORI Observatorio Vulcanológico y Sismológico, Costa Rica PHIVOLCS Philippine Institute of Volcanology and Seismology PRA Post Response Assessment RAS Selected Robotics and Autonomous Systems RNVV National Volcanic Monitoring Network, Chile RSAM Real-time Seismic-Amplitude Measurement RVO Rabaul Volcanological Observatory, Papua New Guinea SGC Colombian Geological Survey SEGEMAR OAVV Geological Mining Service, Argentine Observatory for Volcanic Surveillance SERNAGEOMIN National Geology and Mining Service, Chile SINAGERD Disaster Risk Management Secretariat, Peru SRC Seismic Research Centre of the University of the West Indies SWARM Software for visualization and analysis of seismic activity TAGA Threat Assessment and Gap Analysis TET The Evaluation Team UNESCO United Nations Educational, Scientific and Cultural Organization UNGRD National Unit for Disaster Risk Management, Colombia USAID/BHA United States Agency for International Development, Bureau for Humanitarian Assistance USG United States Government USGS/VDAP United States Geological Survey, Volcano Disaster Assistance Program VALVE Software to visualize different types of volcano monitoring data in one place 1 Executive Summary The Executive Summary summarizes the key sections of the report: Evaluation Purpose and Evaluation Questions; Project Background; Evaluation Methods and Findings; and Conclusions. Evaluation Purpose and Evaluation Questions This report documents the 2020 mid-term performance evaluation, FY2018–FY2020 period, of the Volcano Disaster Assistance Program (VDAP), a joint initiative between the United States Agency for International Development and the United States Geological Survey (USAID - USGS). The evaluation focuses on the program’s effectiveness, relevance, and sustainability. The 2020 mid-term evaluation was commissioned by and reports to the United States Agency for International Development’s Bureau for Humanitarian Assistance (USAID/BHA) and the USGS/VDAP. It was carried out by Florida International University’s Extreme Events Institute (FIU-EEI). This evaluation prioritizes the 15 countries that have benefitted from the VDAP program between FY2018 and FY2020 and that inform VDAP’s decisions on strategies, activities, and future funding levels. Five main questions (with 24 sub-questions), prepared by VDAP-USGS, guide the assessment and reporting of this mid-term evaluation. The five evaluation questions are: 1. What is the effectiveness of tools provided by VDAP on improving volcano observatories’ ability to monitor their volcanoes and forecast eruptions? (tools include data, software, databases, and monitoring equipment). 2. How effective have VDAP’s trainings, workshops, and exchanges been in preparing counterparts at volcano observatories to monitor their volcanoes and forecast eruptions? 3. To what extent has stakeholder effectiveness improved due to VDAP programming? (stakeholders include volcano observatory staff, at-risk populations, affected governments, USGS, USAID/BHA, and the USG). 4. To what extent has stakeholder coordination and leveraging improved due to VDAP programming? 5. What are the recommendations for future VDAP programming? For the purpose of this evaluation, ‘effective’ is the degree to which something is successful in producing a desired result, and ‘effectiveness’ is the degree to which objectives are achieved and the extent to which targeted problems are solved, in other words ‘doing the right thing.’ Project Background The Volcano Disaster Assistance Program (VDAP) is a joint USGS/USAID-BHA program, designed to assist foreign governments to mitigate the effects of volcanic unrest and eruptions. The primary goal of the VDAP is to reduce loss of life and economic damage in countries that experience volcanic eruptions. Under the current USGS/USAID-BHA agreement, VDAP must comply with a mid-term program performance evaluation. The VDAP was established on August 6, 1986 in the wake of the devastating volcano-induced mudflow that destroyed the city of Armero, Colombia on November 13, 1985, killing more than 25,000 people. This tragic event, along with several others in the 1980s, demonstrated the vulnerability of the ever-increasing number of people in developing countries worldwide that are exposed to volcanic hazards. Since 1986, VDAP has worked in multiple countries and regions to (a) strengthen volcano monitoring capacities and (b) integrate mitigation and response components through technology transfer, training, and rapid response to volcano unrest and eruptions. The VDAP is headquartered at the USGS Cascades Volcano Observatory in Vancouver, Washington and has a team of approximately 20 geologists, geophysicists, and engineers. Total funding from 2 USG toward VDAP is $21.8 million from the beginning of FY 2018 through the end of FY 2020 – the time period this evaluation is examining. Evaluation Methods and Findings In order to address VDAP’s evaluation questions, research design began with an extensive literature review, followed by a mixed research method that included qualitative approaches, such as interviews, online short questionnaires, and document reviews, and quantitative approaches, such as surveys. This evaluation prioritized the 14 countries (15 agencies) that benefitted from the VDAP program between FY2018 and FY2020. From this list, the evaluation team (TET) selected those countries (nine) which, according to VDAP, received higher levels of technical cooperation from the Program. One of these nine countries—Peru—has two official organizations that work on volcano monitoring, under two different ministries. Thus, evaluation centered the study on 10 agencies from nine countries and conducted interviews, field visits and sent online questionnaires. However, the survey was conducted in 16 countries (including the original 14) to gather primary data (Table 1). FIU-EEI followed the ethical Institutional Review Board (IRB) regulations when conducting the different evaluation methods. Table 1. Evaluation Methods by Beneficiary Country Country Interviews Remote Field Visits US MDROs Questionnaire Survey Argentina X X X X Chile X X X X Colombia X X X X Costa Rica X X X X DRC* X Ecuador X X X X El Salvador X X X X Guatemala X X X X Indonesia X X X X Mexico X Nicaragua X Peru IGP OVS X X X X Peru INGEMMET X X X Philippines X PNG* X * DRC - Democratic Republic of the Congo; PNG - Papua New Guinea. Following an integrative process—triangulation—the FIU evaluation team analyzed the data from different perspectives, i.e., from the viewpoint of: the final recipients of the capacity-building initiatives; the national geological services; VDAP team members; and the U.S. Mission Disaster Relief Officers (MDROs) in selected countries. The main limitation TET found during the evaluation corresponded to the restrictions imposed by the COVID-19 pandemic; this limitation was mitigated by conducting online interviews and remote guided visits. The second limitation was due to the absence of a unique and updated database of the beneficiaries of training and capacity building program. This required a continuous effort to build databases of training attendees with co￾organizers in several countries. The third limitation detected was the hesitation of some national counterparts to share data on the availability of equipment and networks and the operational status of the equipment donated by VDAP, thus limiting the analysis to the instruments reported as delivered by VDAP. 3 The key findings reflect the essence of the countries’ responses to the five core evaluation questions, as determined through interviews and surveys. 1. What is the effectiveness of tools provided by VDAP on improving the ability of volcano observatories to monitor their volcanoes and forecast eruptions? The various tools provided by VDAP, such as data, software, databases, and monitoring equipment, have all demonstrated their vital importance to the volcano observatory teams. No one modality has been identified as more effective than another. Every observatory’s infrastructure is different and may require customized solutions to problems; the availability of technical expertise varies by partner organization (e.g., volcano observatories, volcano monitoring systems, national geological services). Two particular VDAP tools, 1) threat assessment and gap analysis (TAGA) and 2) the event tree training, have been widely used to guide observatory operations and investigations. TAGA has been instrumental for ranking and prioritizing volcanic hazards, along with volcanic assessments in eight out of the ten observatories. With VDAP support, Guatemala, Nicaragua, El Salvador, and Peru conducted a first TAGA assessment to rank and prioritize volcanic threat and the two Peruvian observatories have continued their volcanic risk assessment. Guatemala also updated its ranking with advice from VDAP and SERNAGEOMIN. Nicaragua and Indonesia have not updated their volcanic threat assessment studies. Some countries do not use TAGA, but rather utilize other approaches to prioritize their work, as in the case of Colombia and Ecuador. All ten observatories that were evaluated regard training on event trees, which build scenarios to assess volcanic risk, as important. Although some countries have not been able to apply the method, they are hoping to do so as soon as resources become available. Satellite sketches are considered a vital resource for improving volcanic monitoring and have served as supporting documents to ensure proper evacuation. Most countries cannot easily access satellite images and the illustrations provided by VDAP have been valuable, providing a better understanding of what is occurring, especially for volcanoes in remote locations. Even if countries could access imaging data, there is neither the software nor the capacity to process it, or both. Countries suggest that VDAP might create a knowledge hub and a potential new delivery system for RS data of these multi-country image databanks for channelized, convenient, and direct access to all participating countries. Initial software training has provided observatories a foundational platform that has then been improved or adapted to each country’s needs. Some countries (Costa Rica and Colombia) have been able to develop their own software for monitoring events, ranging from lahar to deformation to gas emissions, which they occasionally share with other countries. VDAP’s donations of equipment to countries in recent years respond to the need to strengthen monitoring instrument networks during eruptive cycles. The donated equipment was essential for decision making during volcanic crises: Colombia acknowledges the essential support received during the Nevado del Huila eruption (2007–2010); similarly, Chile, for the Chaiten volcano (2008-2009); Ecuador, for the Cotopaxi eruptions (2015) and Sangay (for an ongoing eruptive cycle); El Salvador, for the San Miguel volcano (2015); the response to unrest at Agung, Indonesia (2018); Guatemala, for the eruption of the Fuego volcano (major eruption in 2018 and specific reactivations in subsequent years); the Morne Plat Pays volcanic unrest in Dominica (2019); and the same year in Papua New Guinea for the eruption of the Ulawun volcano. Other instances include the rapid-onset eruption at Taal in The Philippines (2020) and more recently, with escalated volcanic activity at La Soufrière volcano in Saint Vincent and the Grenadines in 2021. 2. How effective have VDAP’s trainings, workshops, and exchanges been in preparing counterparts at volcano observatories to monitor their volcanoes and forecast eruptions? Most beneficiaries of VDAP’s capacity-building activities indicated their preference for attending face-to-face workshops and on-the-job training. When asked if these programs are up-to-date, and if they should be maintained by VDAP, most survey respondents agreed that VDAP tools are valid, relevant, and updated. The heads of national observatories identify the international training program at the Center for the Study of Active 4 Volcanoes (Hawaii), known as the CSAV course, as one of the most useful, particularly in terms of comprehensively addressing volcanic studies and building relationships with other people around the world. Binational exchanges have also helped to strengthen literacy about volcanic risk; promote understanding of volcanic disasters and effective mitigation options; and provide scientists, emergency managers, and first responders with the opportunity to learn from similar events and improve disaster preparedness in communities located near volcanoes. 3. To what extent has stakeholder effectiveness improved due to VDAP programming? VDAP's work has boosted the capabilities of national volcanological observatories to develop a critical mass of professionals and monitoring systems that varies from country to country, according to internal policies and budget allocations. Colombia currently has a system of consolidated volcanological observatories, which has dealt with volcanic crises of great magnitude, such as those that occurred at the volcanoes Galeras and Nevado del Huila. During the Nevado del Huila eruptions in 2007 and 2008, VDAP’s assistance with equipment, training, and the interaction with VDAP experts helped effectively forecast three eruptive pulses, which, in turn, safeguarded more than 100,000 lives in the volcano’s area of influence. Another notable case is Chile, where two strong eruptions of the Llaima and Chaitén volcanoes in 2008 led to the creation of the National Volcanic Surveillance Network Project (RNVV), on the recommendation of VDAP professionals. Since the consolidation of the RNVV Project in Chile, more than 15 volcanic crises have been managed, thanks to effective forecasts, thus preventing the loss of human life. One of these 15 crises, the case of the explosive eruption of Calbuco volcano in April 2015, illustrates how, despite a short alert period (three hours), it was possible to evacuate and save the lives of 7,000 people from the effects of pyroclastic flows and lahars that destroyed homes and infrastructure in their wake. Similarly, in Ecuador, the effective management and forecast of the Guagua Pinchincha, Tungurahua, Cotopaxi, Sangay, Reventador, and other eruptive processes has been possible due to the creation of the volcano observatory, with VDAP’s assistance. Peru, where VDAP's support has been fundamental for consolidating observatories, has increased the quality of the country’s technical processes, services, and operational capabilities, assuring proper response during recent volcanic crises and preventing loss of human life. In Indonesia, VDAP’s assistance with satellite sketches, which provided supporting documentation, was the most useful resource. VDAP has a long trajectory of involvement in international volcano monitoring and volcanic crises management. Its direct experience in managing volcanic crises can be transferred to the USGS domestic program. The possibility of professional exchanges with other countries, leading to gaining a broader context and perspective on how other countries address volcano hazards, would benefit both organizations, the domestic USGS program, and international partners. During crisis situations, two circumstances can create an opportunity for scientific diplomacy: first, the existence of trust in the suitability of those who can assist and advise on specific issues, and second, the availability of communication channels for direct exchange. Over the years, VDAP has used a distinct way of reaching out to counterparts in different countries, under the principle of technical-scientific collaboration, aimed at strengthening countries’ response to crises and building their capacities for understanding and monitoring volcanic threats. 4. To what extent has stakeholder coordination and leveraging improved due to VDAP? Undoubtedly, VDAP’s material assistance, complemented by the advocacy and endorsement its representatives provide, has opened and taken advantage of windows of opportunity within the countries, resulting in the appropriation of budgets for the acquisition of equipment and investment in infrastructure. Equipment donated by VDAP in Indonesia was used to leverage an expansion plan for monitoring equipment, resulting in an important equipment purchase in 2018. A similar experience was mentioned by the volcanology area coordinator in Guatemala. VDAP’s assistance has also helped agencies to achieve international presence and recognition. In the field of volcanology, excellent relationships exist between cooperating agencies, donors, and 5 beneficiaries. Despite an absence of formal agreements between cooperating agencies and donors in certain circumstances, and the fact that often each works in a specialized niche, TET observed no competition but rather complementarity between them. Most countries note that VDAP is in a privileged position worldwide to identify where the expertise of the different groups lies and establish communication channels within and among regions. Due to VDAP’s work in promoting cooperation among countries, it is in a unique position to map the availability of expertise, equipment, and infrastructure in the region and promote and support horizontal exchanges among countries. Informally but decidedly, VDAP has already been supporting horizontal cooperation, taking advantage of its emerging technical leadership in the region. 5. What are the recommendations for future VDAP programming? Information from satellite observations that VDAP has provided in times of volcanic crises, either through sketches, verbal communication, or via e-mail, is a powerful and highly appreciated short-term decision￾making tool for countries. Nevertheless, not having direct access to the images/data (due to distribution restrictions) prevents a deeper analysis, comparison between images, and use for hazard modeling by the countries themselves. VDAP’s response and communications, whether in-person, by email, or through teleconferencing, is considered par excellence. Countries who have a wealth of data at their disposal, waiting to be processed, and seek to increase collaboration and scale up joint efforts to develop scientific publications and papers. Participating countries also suggested: (1) the creation of a VDAP platform/website with a search engine/tool (chronologically or topically organized, with analyzed content to enrich user experience) through which users could have easier access to information; and (2) acquisition of hardware. Countries with advanced capabilities, such as Colombia, Costa Rica, and Chile, now aspire to graduate from having just software capabilities to acquiring independent hardware capabilities, thus allowing them to dissociate from their dependence on university laboratories, which have distinct priorities, such as publishing. With ample subject area experts and electronic engineers, these countries would like to become self-reliant in terms of assembling the instruments they need and build the networks they require for volcanic monitoring. Finally, other suggestions include a need to increase the availability of tools; strengthen training for satellite image processing; have access to the USGS VolcView; support for drone monitoring for volcanic surveillance, especially during eruptive crises events (and also for crater lake sampling); and training for preparing samples for electron microscopy). Conclusions All countries expressed satisfaction with the quality and level of technical support they receive from VDAP (whether through consulting, training of professionals, scientific understanding, and technical knowledge), which has allowed them to tremendously improve their technical competence. Along with the high-quality scientific and technical nature of the partnerships that VDAP has created, countries were deeply appreciative of the human component of VDAP’s collaboration. This report identifies three areas of challenges for VDAP regarding the current demand for services from beneficiary countries: (1) infrastructure; (2) personnel, administrative and financial support; and (3) international relations. If VDAP considers expanding the scope of its program by strengthening some services and initiating new ones, as identified and proposed by this evaluation, it will require a careful analysis of the level of effort involved in the process and the need for the expansion of the VDAP team. 6 Evaluation Purpose and Evaluation Questions Evaluation Purpose The overarching goal of this evaluation was to conduct an up-to-date programmatic mid-term performance evaluation, FY2018–FY2020 period, of the Volcano Disaster Assistance Program (VDAP) at the global level, focusing on the program’s effectiveness, relevance, and sustainability. VDAP is a joint initiative between the United States Agency for International Development (USAID) and the United States Geological Survey (USGS). The results of this evaluation by FIU-EEI will inform the USAID’s Bureau for Humanitarian Assistance (USAID/BHA) and the USGS/VDAP on VDAP’s performance in 14 selected beneficiary countries (hereafter, countries). While different types of evaluations have been carried out over the lifetime of the VDAP, the last country-level evaluation, the 2012 Evaluation of the USAID/OFDA-USGS Volcano Disaster Assistance Program in Indonesia, is now eight years old. A new and more programmatic mid-term evaluation at this time would benefit from the advances in technologies, better capacities in data processing, and knowledge of volcanic hazards, constituting a window of opportunity to revisit and review VDAP’s capacity development mechanisms. This evaluation prioritizes the countries benefitted by the VDAP program between FY2018 and FY2020 and focuses its efforts on assessing the effectiveness, relevance, and sustainability of VDAP to assist foreign governments to mitigate the effects of volcanic unrest and eruptions. Total funding from USG toward VDAP is $21.8 million from the beginning of FY 2018 through the end of FY 2020 – the time period this evaluation is examining. The current Interagency Agreement (IAA) for VDAP between USAID/BHA and USGS will end on September 30, 2022. A new 5-year IAA for VDAP is being prepared. The VDAP M&E Team will work with management from VDAP, USGS, and USAID to address recommendations made from this external evaluation and to make changes to VDAP programming as appropriate to be incorporated in annual work plans during the new IAA. For the purpose of this evaluation, ‘effective’ is the degree to which something is successful in producing a desired result and ‘effectiveness’ is the degree to which objectives are achieved and the extent to which targeted problems are solved, in other words ‘doing the right thing.’ This evaluation seeks to inform VDAP’s decisions on strategies, activities, and future funding levels. The main audiences to which this evaluation report is directed are: VDAP, USAID/BHA, and USGS. For this evaluation, our team at FIU-EEI reached out to VDAP’s senior managers and the technical team. Within USAID/BHA, TET considered the offices of Technical and Program Quality, Monitoring and Evaluation, and the regional offices for Latin American and the Caribbean and Southeast Asia. In USGS, TET reached out to the senior administration. Finally, VDAP’s counterparts overseas and the national volcano monitoring systems’ managers and technical teams could benefit from information contained in this evaluation. 7 Evaluation Questions This assessment was designed around a series of questions proposed by VDAP-USGS. There are five major evaluation questions: 1. What is the effectiveness of tools provided by VDAP in terms of improving the volcano observatories’ ability to monitor their volcanoes and forecast eruptions (tools include data, software, databases, and monitoring equipment)? 2. How effective have VDAP’s trainings, workshops, and exchanges been in preparing counterparts at volcano observatories to monitor their volcanoes and forecast eruptions? 3. To what extent has stakeholder effectiveness improved due to VDAP programming (stakeholders include volcano observatory staff, at-risk populations, affected governments, USGS, USAID/BHA, and the USG)? 4. To what extent has stakeholder coordination and leveraging improved due to VDAP programming? 5. What are the recommendations for future VDAP programming? Each evaluation questions contains sub-questions as noted below (24 sub-questions in total). FIU: Volcano Disaster Assistance Program (VDAP) 2020 Mid-Term Evaluation Evaluation Questions 1. What is the effectiveness of tools provided by VDAP in terms of improving the volcano observatories’ ability to monitor their volcanoes and forecast eruptions (tools include data, software, databases, and monitoring equipment)? • Which VDAP tools are most useful to observatories? (Which VDAP tools are used? How are they used? Who uses them?) • How have threat assessment and gap analysis (TAGA) been used to guide observatory operations and investigations? • Which VDAP partners, who received event tree training from VDAP, are using event trees on their own? How has the event tree process been used to assess and communicate risk and aid forecasts? • In what ways do periodic satellite data observations and sketches assist in observatory decisions? What remote sensing data is most useful (e.g., paragraph form from Reston or jpgs)? • How does VDAP-provided software affect observatory operations? How has VDAP-provided software spurred homegrown software? How many partners are using software VDAP has provided for monitoring? Which software packages are they using? • How effectively has VDAP-donated equipment been used at volcanoes that have been active to provide data for decision making? 2. How effective have VDAP’s trainings, workshops, and exchanges been in preparing counterparts at volcano observatories to monitor their volcanoes and forecast eruptions? • Is the current format for conducting trainings, workshops, and exchanges effective (e.g., are regional workshops as effective as observatory-based training sessions, are general or specific trainings more effective, etc.). Are any changes required? • Which trainings, workshops, and exchanges are most useful to participants? How important a role does CSAV and other VDAP training play in the human resources development of volcano observatory staff around the world? 8 • How have countries participating in binational exchanges used the knowledge transferred during the exchanges to further disaster risk reduction (DRR) in their countries? • Did VDAP events ensure the involvement of women and assist men and women equally? 3. To what extent has stakeholder effectiveness improved due to VDAP programming (stakeholders include volcano observatory staff, at-risk populations, affected governments, USGS, USAID/BHA, and the USG)? • Has VDAP assistance led to a reduction in the loss of lives and property due to volcanic eruptions? • Has the ability of government authorities to forecast and respond to eruptions improved as a result of VDAP assistance? • What is the impact of VDAP on domestic USGS programs? • Has VDAP assistance promoted science diplomacy? 4. To what extent has stakeholder coordination and leveraging improved due to VDAP programming? • Has VDAP material assistance been leveraged by host countries to obtain more support from national or foreign donor sources? • Were VDAP activities well-coordinated with other donors to avoid duplication of effort? • How does VDAP facilitate collaboration/cooperation within/between countries? What binational or multi-agency ‘agreements’ have been fostered by VDAP? • How has partnering with VDAP fostered the development and operations of counterpart agencies over the last 2.5 years? 5. What are the recommendations for future VDAP programming? • What changes should be made regarding how VDAP makes forecasts and/or communicates information about them to improve effectiveness? • Does VDAP engage in outreach at appropriate levels? Is the type of outreach appropriate (social media, publications, USGS outreach products, website, factsheets)? • Is reporting timely (e.g., IVANS, and daily updates)? • Should VDAP develop new tools for partners? Are there tools that partners want that VDAP doesn’t provide? • Should VDAP make any changes to the way the program is monitored or evaluated? • Are there any recommendations, concerns, or feedback counterparts would like to provide? 9 Program Background VDAP is a joint USGS/USAID-BHA program designed to assist foreign governments to mitigate the effects of volcanic unrest and eruptions. The primary goal of the VDAP is to reduce loss of life and economic damage in countries that experience volcanic eruptions. VDAP has had 5-year planning and budgeting cycles. The current cycle began in October 2017 and ends in September 2022. Under the current USGS/USAID-BHA agreement, VDAP must comply with a mid-term program performance evaluation. VDAP was established in August 6, 1986, as a joint initiative between the United States Agency for International Development (USAID) and the United States Geological Survey (USGS), after the devastating volcano-induced mudflow that destroyed the city of Armero, Colombia on November 13, 1985, killing more than 25,000 people. This tragic event along with a few others in the 1980s, demonstrated the vulnerabilities of the increasing number of people exposed to volcanic hazards in developing countries worldwide. Since 1986, VDAP has worked in multiple countries and regions to (a) strengthen volcano monitoring capacities and (b) integrate mitigation and response components through technology transfer, training, and rapid response to volcano unrest and eruptions. The VDAP program is headquartered at the USGS Cascades Volcano Observatory in Vancouver, Washington, and has a team of approximately 20 geologists, geophysicists, and engineers. The VDAP program contributes decisively to the fulfillment of the priority actions proposed by the U.N. Sendai Framework for Disaster Risk Reduction (SFDRR) 2015–2030: (1) Understanding disaster risk; (2) Strengthening disaster risk governance to manage disaster risk; (3) Investing in disaster risk reduction for resilience; and (4) Enhancing disaster preparedness for effective response. The VDAP program is mainly associated with SFDRR priority actions 1 and 4. Logic Model for the Program Evaluation This VDAP performance evaluation is built on the evidence-based research strategy (EBRS). The EBRS uses prior research in a methodical and transparent way to answer specific questions in a valid and efficient manner. A key aspect of EBRS is that it is based on practice, and actively involves the participation of implementing actors in defining the objectives and research questions. In this study, the evaluation questions have been prepared by the VDAP team and are followed by an in-depth review, upon discussions with USAID/BHA and USGS/VDAP counterparts, who will be part of the evaluation. This process ensures that the results of the study respond to defined needs and expectations of beneficiaries and partner organizations, in other words, from and for practice. The design of this study will advance the measurement of the different processes involved in VDAP’s capacity development strategy, and thus contribute to a better understanding of the program’s performance and outcomes to inform future decisions. Figure 1 describes the Logic Model for the Program Evaluation, designed for the VDAP mid￾term performance evaluation. 10 Figure 1. Logic Model Development Program Planning 11 Evaluation Methods and Limitations To address VDAP’s evaluation questions, the research design began with an extensive literature review, followed by a mixed research method that included qualitative approaches, such as interviews, online short questionnaires, and document reviews and quantitative approaches, such as surveys conducted in selected countries to gather primary data (Figure 2). Figure 2. Selection of Evaluation Methods This evaluation prioritizes the 14 countries that have benefitted from the VDAP program between FY2018 and FY2020. From this list, the evaluation team (TET) selected those which, according to VDAP, received higher levels of technical cooperation from VDAP, resulting in 10 countries. During the early stages of the evaluation there was a change in the countries initially selected in the evaluation proposal as indicated in Table 1. Mexico, which was originally included, had not requested any significant technical support in the last five years. Consequently, VDAP requested that two Peruvian institutions be included instead: the Geophysical Institute of Peru, Southern Volcano Observatory (IGP-OVS) and the Geological Mining and Metallurgical Institute (INGEMMET), which provides in-country volcanic monitoring services. Both Peruvian institutions have received important assistance from VDAP in past years and show different institutional features, strengths, and needs. In this way, the evaluation centered the study in 10 agencies from nine countries and conducted interviews, field visits and sent online questionnaires. The survey, however, was conducted in 16 countries (which include the original 14) to gather primary data. Table 1 summarizes the evaluation methods used by country. 12 Table 1. Evaluation Methods by Country Country Interviews Remote Field visits US MDROs Questionnaire Survey Argentina X X X X Chile X X X X Colombia X X X X Costa Rica X X X X DRC X Ecuador X X X X El Salvador X X X X Guatemala X X X X Indonesia X X X X Mexico X Nicaragua X Peru IGP OVS X X X X Peru INGEMMET X X X Philippines X PNG X DRC-Democratic Republic of the Congo. PNG – Papua New Guinea. FIU-EEI followed the Institutional Review Board (IRB) regulations for an approved research protocol. Using an integrative process—triangulation—the FIU-EEI evaluation team analyzed the data from different theoretical positions and perspectives, i.e., of the final recipients of the capacity building initiatives, national geological services, VDAP team members, and that of the U.S. Mission Disaster Relief Officers (MDROs) in selected countries. Survey The survey provided the means to conduct long-term follow-up with participants who graduated and benefitted from the training provided by VDAP￾USGS. The training refers to events (courses, workshops, seminars, on-the-job training) organized and financed exclusively by VDAP as well as other events jointly organized with other agencies and partners, in which VDAP provided a certain level of technical and/or financial sponsorship. The survey had three main objectives: • Monitor the retention of knowledge among participants of the VDAP training and capacity building program. • Measure attitudes and practices associated with VDAP’s learning process. • Follow-up of volcano monitoring, forecast of eruptions, and emergency management practices. 13 VDAP provided the FIU-EEI evaluation team with lists of attendees at different training events, going back to 2013, along with information about the individuals and their associated agencies. However, since this survey would be conducted online, TET suggested to USAID and VDAP that the entire trained population from 2013, and not just the population benefited in FY2018-FY2020, should be included. This would allow for a broader consultation, transcending the nine selected countries. TET then contacted the concerned agencies to obtain updated and current information on training attendees (names and email addresses). From a total of 209 trainee records, TET was able to identify 176 individuals (several people had attended more than one training session); of these, seven addresses were no longer current. The final survey population was made up of 169 individuals from 16 countries. Of these, 107 corresponded to individuals trained in the period FY2018-FY2020. As an additional output, TET generated a consolidated database of VDAP trained individuals. The survey questionnaire (33 questions) was designed using the USAID-KAP Survey Model and focused on knowledge, attitudes, and practice/implementation. It was constructed using pre-defined questions, allowing us to gather quantitative and qualitative information from interviewees. The research was conducted in Spanish for Latin American countries and in English for Asian and African countries. FIU-EEI sent the invitation to participate in the survey by email to the beneficiaries of VDAP training, who had 45 days to respond using the Qualtrics online platform survey. A total of 92 valid surveys from 16 countries were received (Table 2), 62% men and 38% female, with a 54.4% response rate. The response rate was adequate considering that the average response rate obtained in other continuing education and professional development programs is 40% (Lambert, 2014). Table 2. Response to the survey by country. Country # Responses Percentage 1 Argentina 6 6.5% 2 Cameroon 1 1.1% 3 Chile 8 8.7% 4 Colombia 6 6.5% 5 Democratic Rep. of Congo 1 1.1% 6 Costa Rica 4 4.3% 7 Ecuador 8 8.7% 8 El Salvador 7 7.6% 9 Guatemala 6 6.5% 10 Indonesia 9 9.8% 11 Mexico 5 5.4% 12 Nicaragua 5 5.4% 13 Papua New Guinea 1 1.1% 14 Peru 17 18.5% 15 Philippines 7 7.6% 16 Singapore 1 1.1% Total 92 100.0% The data obtained was processed in SPSS Version 19 (SPSS Inc., Chicago, IL, USA) for statistical analysis for frequencies and descriptive analysis. An independent t-test was employed to assess the differences in knowledge, attitude and practices by age, gender, and qualification among study subjects. The confidence interval was fixed at 95% for the present study. Information about the survey outputs is presented in the following section on Findings, specifically in the subsections Main Topics and Response to the Evaluation Questions. Information on the demographics of the surveyed population can be found in Annex II, page 67. 14 Interviews Interviews with 10 coordinators from national volcano monitoring systems were conducted in the nine countries selected: Argentina, Chile, Colombia, Costa Rica, Ecuador, El Salvador, Guatemala, Indonesia, and Peru. The semi-structured interviews lasted between 30–45-minutes and were designed jointly with VDAP to help understand if VDAP assistance has contributed to improving volcano monitoring, forecasting eruptions, and strengthening emergency management capabilities. The interviews were conducted online in Spanish for Latin American countries and in English for Indonesia, using the Zoom platform. All interviews were recorded (with previous authorization by interviewees). Fourteen hours of interviews are available. Transcripts of the interviews were prepared using a predesigned format, which facilitated the subsequent analysis. The semi-structured interview layout used can be found in Annex III, page 83. Questionnaires to VDAP Team Members A short questionnaire was sent to VDAP team members who provide technical assistance to foreign countries in order to include their perspective in this evaluation. Of fifteen questionnaires sent, nine were completed and returned, containing information on topics such as: major challenges to providing long-term technical assistance; the role of U.S. Missions abroad; VDAP’s impact on domestic programs; and comments and recommendations to improve VDAP’s cooperation. The short VDAP team member questionnaire can be found in Annex III, page 82. Questionnaires to U.S. MDROs A short questionnaire was sent to the U.S. Mission Disaster Relief Officers (MDROs) to understand their role within the broad scope of cooperation that VDAP provides to its national counterparts. The MDROs were located in the nine selected countries: Argentina, Chile, Colombia, Costa Rica, Ecuador, El Salvador, Guatemala, Indonesia, and Peru. Three completed questionnaires were received, containing information on central topics such as the role of U.S. Missions abroad; existing formal agreements between the U.S. Government (USG) and host countries; involvement of the central government, national emergency office, and/or ministry of foreign affairs in VDAP’s cooperation; and comments and recommendations to improve VDAP’s cooperation. The short MDROs questionnaire can be found in Annex III, page 81. Remote Field Visits As the design of the VDAP Mid-term Evaluation was beginning in February 2020, the realization that the COVID-19 pandemic would severely restrict our movements within and among countries came to the forefront. It also made it impossible to carry out field visits to observe the monitoring rooms and the operation of networks and equipment. To a greater or lesser extent, this scenario coincided with the reality on the ground in the nine selected countries through December 2020. For this reason, once the surveys and interviews were completed, the directors of the national volcanic monitoring systems were asked to carry out a virtual visit to the physical facilities where these monitoring systems operate. Of the 10 requests sent, TET received five affirmative responses with additional information on organization, facilities, equipment and networks (Chile, Colombia, Costa Rica, El Salvador [provided a 15 short video], and Peru-INGEMMET). Two national agencies in Argentina and Peru (IGP-OVS) were unable to participate in virtual visits, as their facilities were under construction. Three countries indicated restricted access to their observatories due to COVID-19 or did not respond at all to the request for virtual visits (Ecuador, Guatemala, Indonesia). Four hours of video of the (remote) guided field visits are available. Transcripts of the field visits were prepared using a predesigned format, which facilitated the subsequent analysis. The form layout used to capture information on organization, facilities, equipment, and networks can be found in Annex III, page 85. Limitations The main limitation experienced in this 2020 VDAP mid-term performance evaluation centers on mobility restrictions due to the COVID-19 pandemic, which impacted all the countries participating in this evaluation and were in place during the 10 months in which the evaluation was conducted. The activity most impacted by the pandemic were the remote visits. However, TET must acknowledge that due to the pandemic, there was also greater willingness to accept online surveys, participate in remote interviews, and respond to questionnaires that national counterparts sent by mail. The second limitation experienced was the absence of a unique and updated database of the beneficiaries of VDAP training and capacity building programs. Therefore, it was necessary to build the VDAP trainee database with information from dispersed databases, activity reports, and through consultation with instructors and organizers. On some occasions, the information was non-existent or simply out-of-date. The third limitation detected was the hesitancy of some national counterparts to share data on equipment, availability of networks, and the operational status of the equipment donated by VDAP. Only five of the nine selected countries provided this information. Countries gave the following reasons for the hesitancy to provide information: no records, no detailed information, the fact that sharing information on equipment information might discourage VDAP from assigning new equipment to them, the out-of￾service status of VDAP donated equipment, and a failure to install donated equipment. Finally, in Indonesia, strict government guidelines regulate/control the interaction that non￾national agencies or organizations can have with technical and scientific counterparts. The MDRO in Indonesia is aware of these circumstances, and in his response to this evaluation, he requests VDAP’s collaboration in complying with these requirements in the future, which include registering the assistance, sharing information on the financial investment, and details on all activities, including collaboration on research (additional details in item 76, page 45). 16 Findings The following section is divided in two subsections: 1) Response to the Evaluation Questions, and 2) Findings by Central Topics. Response to the Evaluation Questions The responses to the evaluation questions have been prepared using a mixed research method, including qualitative approaches (interviews, online short questionnaires, and document reviews), and quantitative approaches (surveys conducted in selected countries). The responses are constructed using an integrative process, combining different perspectives and coincident responses when applicable, indicating the method/source while respecting anonymity. Unlike the executive summary, where the answers are focused on the five core questions of the evaluation, the answers in this section focus on the sub-questions to reflect greater specificity. To delve into the issues addressed in the responses, a number is included in square brackets that corresponds to a link to the corresponding paragraphs of the later section on central issues, where complementary information is found. 1. Effectiveness of tools provided by VDAP on improving volcano observatories’ ability to monitor their volcanoes and forecast eruptions • Which VDAP tools are most useful to observatories? (Which VDAP tools are used? How are they used? Who uses them?) The various tools provided by VDAP, such as data, software, databases, and monitoring equipment, have all demonstrated their vital importance to the observatory teams. No one modality has been identified as more effective than another. As a VDAP team member indicated, there are two general rules: (1) every observatory’s infrastructure is different and may require customized solutions to problems; and (2) the available technical expertise varies by the partner organization. Some have more capable staff than others. This is why in some observatories, certain VDAP tools are used more than in others. However, based on the answers provided by survey respondents and interviewees, TET did not find a VDAP tool included in the current portfolio that is not being used. [41-48] P.93 • How have threat assessment and gap analysis (TAGA) been used to guide observatory operations and investigations? All observatories included in this evaluation agree that the hazard assessment training provided by VDAP is a fundamental steppingstone for ranking and prioritization of volcanic hazards, along with assessments. With VDAP support, Guatemala, Nicaragua, El Salvador, and Peru, conducted their first-ever assessments to establish a ranking and prioritization of the volcanic threat. The two Peruvian observatories have continued their risk assessments in southern Peru using the Ewert et al. (2005) methodology, with support from VDAP. Guatemala updated the study based on Ewert (2007) with French and UK institutions, acknowledging the advice of VDAP and SERNAGEOMIN. Nicaragua, and Indonesia have not updated the studies. Indonesia acknowledges that there have been cases of volcanoes that previously had not been classified as high risk at their time of their eruption. [25] 17 Some countries in this study do not use TAGA, but rather other approaches to prioritize their work, as in the case in Colombia and Ecuador. As stated by Chile, “threat assessments and gap analysis are exercises that should be performed periodically” to revaluate any recently published studies, demographic changes, changes in infrastructure, and technical advancements, such as sensors and telecommunications. [25] • Which VDAP partners, who received event tree training from VDAP, are using event trees on their own? How has the event tree process been used to assess and communicate risk and aid forecasts? All 10 observatories contacted as part of the evaluation highly appreciate this method and consider it essential as a method to reduce volcanic risk, by building possible scenarios, to be important. Event trees require that specialties come together and work collaboratively to provide more realistic outcomes and consequently support gap analysis. [27, 28] The countries that have not been able to apply the event tree method are hoping to do so as soon as resources become available [29, 30] There have been on-site event tree trainings conducted during volcanic crises in Guatemala, 2018 (Fuego volcano); Nicaragua 2016 (Momotombo volcano); Peru 2016 (Ubinas volcano); Ecuador 2015 (Cotopaxi volcano); and Indonesia 2014 (Sinabung volcano). Two virtual event tree trainings were also conducted (with Argentina/Chile in 2018, and Chile in 2020) [Annex IV page 95]. • In what ways do periodic satellite data observations and sketches assist in observatory decisions? What remote sensing data is most useful (e.g. paragraph form from Reston or jpgs)? Satellite sketches are considered a vital resource that improve volcanic monitoring and have served as supporting documents to ensure proper evacuation. The majority of the countries do not have easy access to satellite images and the illustrations provided by VDAP has been a valuable input, providing a better understanding of what is occurring, especially for volcanoes in remote locations [36-40]. Specific needs include quantification of thermal anomalies, amount of gas in the atmosphere, detection of ash in the atmosphere and on the ground, measuring the height of an ash and/or gas plume, topographic changes (and determination of lake-level changes), interferometry, and meteorological data (wind, cloud cover, rainfall for DOAS data processing). Even if countries could access data on images, there is neither the software nor the capacity to process it, or both. Countries suggest that VDAP might create a knowledge hub and a potential database of these multi-country image databanks for channelized, convenient and direct access to all participating countries. The software (which could be commercial) and the capacity to process could be centralized to create robust image databases with organized metadata. A hassle-free, seamless access system with clarity in organization and reach could be provided to all countries. Other international networks of experts, such as ASGMI and IAVCEI, could be brought in for increased information-sharing. With sound procedures, policies and coordinated management, this information and knowledge hub could benefit everyone. [36] 18 • How does VDAP-provided software affect observatory operations? How has VDAP provided software spurred homegrown software? How many partners are using software VDAP has provided for monitoring and which software packages? It is important to note that most of the observatories are using 'Earthworm' as their main data acquisition software and they use 'Swarm' as software for the immediate display of seismic data. Both are provided by VDAP and are constantly being updated and improved with new versions. One of the greatest contributions to the evaluation of the impact of lahars is the LAHARZ software, which is based on the statistical relationships of the volume of lahars vs. flooded area. VDAP specializes in training on the use of this model through workshops and communication with professionals from the observatories, since the software must be installed on local servers. The LAHARZ software has been used to evaluate the impact zone in cases of pyroclastic flows, based on the concept of the "energy cone" or H/L cone. The ASH3D model software has recently been used to monitor and forecast volcanic plumes and surface ash accumulation during explosive eruptions, allowing real-time execution of simulations where the user establishes input parameters. The software is available on an online platform to which countries have been given access. Initial software training has given observatories a foundational platform. From this point, they have been able to improve or adapt it to each country’s needs. Software for events ranging from lahars to deformation to monitoring of gas emissions is either currently being used or countries have been able to develop their own software and occasionally share it with other countries. Software packages such as NOVAC, dMODELS, InSAR and GPS are being used by several countries in this evaluation. [41-44] • How effectively has VDAP-donated equipment been used at volcanoes that have been active to provide data for decision making? In reviewing the list of equipment donated by VDAP in recent years (Annex IV, page 125), it is clear that the donation process has been influenced by the need to strengthen instrument networks during eruptive cycles. Evidence that VDAP-donated equipment proved essential for decision-making is exemplified by cases mentioned in interviews and others extracted from the VDAP response reports: Ecuador, during the Cotopaxi eruptions (2015) and Sangay (for an ongoing eruptive cycle); El Salvador, for the San Miguel volcano (2015); the response to unrest at Agung, Indonesia (2018); Guatemala, for the eruption of the Fuego volcano (major eruption in 2018 and specific reactivations in subsequent years); the Morne Plat Pays volcanic unrest in Dominica (2019); and the same year in Papua New Guinea for the eruption of the Ulawun volcano. [46-50, 57] [Annex IV pages 115-121]. Chile recognizes the importance of VDAP in strengthening their monitoring and support capacity in volcanic crises such as those that occurred during the Chaiten volcano crisis (2009). Colombia acknowledges the essential support received during the Nevado del Huila eruption (2007-2010), where the assistance provided by VDAP (an AFM system and some seismic stations) was essential during the Huila volcanic unrest, allowing an effective management of the crisis. Finally, two more cases show how VDAP-donated equipment proved essential for decision-making: Taal in the Philippines (2020), and more recently in 2021 with escalated volcanic activity at La Soufriere Volcano in Saint Vincent and the Grenadines. [Annex IV page 119] 19 2. Effectiveness of VDAP’s trainings, workshops, and exchanges in preparing counterparts at volcano observatories to monitor their volcanoes and forecast eruptions. • Is the current format for conducting trainings, workshops, and exchanges effective (e.g. Are regional workshops as effective as observatory-based training sessions, are general or specific trainings more effective, etc.)? Should there be any changes? Survey respondents and interviewees emphasize how these learning opportunities comprehensively cover volcanic studies, including monitoring, modelling, and survey of information. The survey inquired about the level of effectiveness of the different VDAP capacity building activities. The majority of the survey respondents show a clear preference for attending face-to-face workshops and on-the-job trainings. Seventy-three trainees (79% of total respondents) selected these two options as the first order of effectiveness or the most effective VDAP capacity-building activities. Binational exchanges were ranked second in terms of preference, or as the second order of effectiveness by 26 trainees (28% of respondents); on-the-job training dominates the third ranking with 29 trainees (32% of respondents); and online trainings ranks fourth in effectiveness by 56 people (61% of the respondents). When asked if these programs are up-to-date and if they should be maintained by VDAP, a majority of survey respondents agreed that VDAP tools are valid, relevant, and updated. A Peruvian representative expressed that “the training provided by VDAP to personnel working at the Southern Volcanological Observatory on volcanic monitoring, volcanic seismology, volcanic deformation, gases, hazard mapping, etc. is very important. These trainings should be more permanent and there should be some quotas for IGP specialists.” Transcending the capacity building purpose of these activities, survey respondents and interviewees agree on the importance of networking. “It is the opportunity for colleagues from different countries of the world to interact and share experiences during the course…It’s networking.” (Ovsicori, Head of the National Volcanic Network, Costa Rica). Another dimension in which trainings prove useful is to fill in gaps in countries with few experts in volcanology with advanced degrees (master’s or doctoral degrees). In one recorded interview, the suggestion was made to identify opportunities for more embedded, real￾world training. The traditional format of short-term (3-4 days) workshops or online training has certain limitations in terms of knowledge retention and skills development. Training and skills, when not put to use, wither away. Countries (e.g., Indonesia) suggest that embedded training improves the length of retention of knowledge and skills and would ensure they are kept up to date and/or upgraded. With regard to more formal staff training, several countries acknowledge that formal graduate academic programs (master’s degree and PhD) at the national level on volcanic studies are not offered and there are few opportunities for postgraduate studies abroad. The countries suggest VDAP directly offer scholarships or advocate with other agencies (e.g., USAID, Fulbright Program, State Department) to support higher education opportunities for young professionals. • How important a role does CSAV play in the human resource development of volcano observatory staff around the world? The heads of national observatories identify the international training program of the Center for the 20 Study of Active Volcanoes (Hawaii), known as the CSAV course, as one of the most useful, particularly in terms of comprehensively addressing volcanic studies and in building relationships with other people around the world. More than half (51%) of the survey respondents indicated they have completed the CSAV course (47 participants). It is interesting to see that participation has trended upward; in the late 1990s there were three participants per year – by 2019 there were 10 participants in that year who benefited from the course. [31-34] The VDAP survey included a set of questions based on the KAP method to assess knowledge, attitudes, and practice with regard to volcano monitoring, forecasting volcano crises, and emergency management. When analyzing the result of the KAP index (a composite index that includes three subindices: knowledge, attitudes, and practice) it was found that the group that had completed the CSAV course had statistically significant higher scores as compared to the group of survey respondents who had not taken the CSAV course: the Knowledge subindex (M = 0.537 vs. M = 0.468) and Practice subindex (M = 0.545 vs. M = 0.506). See information on the KAP index in Annex II page 69. This result is consistent with the positive subjective appreciation of the CSAV by those who completed the training and the heads of the national volcanic monitoring systems interviewed. [35] • How have countries participating in binational exchanges used the knowledge transferred during the exchanges to further disaster risk reduction (DRR) in their countries? The participation in binational exchanges has strengthened relationships between countries and between local and national authorities within a country. Communities are becoming more interested in learning about ‘transitory meeting points’ in case of a volcanic eruption (as in Chile) and learning from each other’s experiences with volcanic crises. Binational exchanges have also helped to strengthen literacy about volcanic risk; promote understanding of volcanic disasters and effective mitigation options; help scientists, emergency managers, and first responders by learning from events to improve disaster preparedness in communities located near volcanoes; provide a window for revision of school curricula to address volcanic risk; and even, as in the case of Colombia, to promote ‘geo-tourism’ (in collaboration with UNESCO) to strengthen communities for sustainable development. One important challenge that has been identified by some countries is that while binational exchanges are key in advancing knowledge transfer and sharing, the cycle of political turnover makes it more difficult to achieve long-term decision making. For example, in Chile, the participants who represented political-administrative institutions in the 2015 binational exchange have already been replaced in their positions. However, representatives from technical entities still hold their positions (Civil Protection and Volcanological Observatory). [55, 56] • Did VDAP events ensure the involvement of women and assist men and women equally? Women form a growing presence in the field of volcanology in VDAP-supported countries and their representation coincides with a general universal trend. Based on the information provided by the 10 national volcanic monitoring systems, approximately 35% of the workforce are women, of whom 10% are in positions of power. Similar results were obtained in the survey conducted in December 2020 targeting VDAP training beneficiaries: 62% were men and 38% women. 21 These results are consistent with the results obtained using different methods regarding participation by gender, with a male/female ratio of 3:1. However, in terms of women in positions of power, the ratio tends to be lower [14-17]. 3. Stakeholder effectiveness improved due to VDAP programming. • Has VDAP assistance led to a reduction of lives and property lost from volcanic eruptions? VDAP’s work has boosted the capabilities of national volcanological observatories in beneficiary countries by developing a critical mass of professionals and monitoring systems that varies from country to country, according to internal policies and budget allocation. Back in 1985, a catastrophic eruption of the Nevado del Ruiz volcano in Colombia claimed the lives of about 25,000 people. This catastrophic experience marked the beginning of the growth of volcanology in Latin America. Colombia followed the recommendation of international scientists to create the first Latin American Volcanological Observatory in the city of Manizales, where USGS scientists trained local professionals in monitoring and instrumentation techniques, many of whom have become global references in topics related to volcanic surveillance. Colombia currently has a system of consolidated volcanological observatories, which have dealt with volcanic crises of great magnitude, such as those that occurred in the Galeras and Nevado del Huila volcanoes. In the latter case, during its eruptions in 2007 and 2008, the assistance provided by VDAP—equipment, training, and the interaction with VDAP experts—helped to effectively forecast three eruptive pulses, thus safeguarding the lives of more than 100,000 people in its area of influence. Another notable case is Chile, where two strong eruptions of the Llaima and Chaiten volcanoes in 2008 led to the creation of the National Volcanic Surveillance Network Project (RNVV), on the recommendation of VDAP professionals. The objective was to strengthen the Southern Andes Volcanological Observatory and monitor 45 active volcanoes across the country. Since its creation, the RNVV has successfully managed more than 15 volcanic crises, without loss of human life. Most of the Chilean observatory professionals who have participated in managing these crises have, at some point, interacted with VDAP professionals, through courses or during the technical assistance provided at the time of the crises. VDAP’s support has been fundamental in the consolidation of observatories in Ecuador and Peru, increasing their quality and operability and assuring proper response during recent volcanic crises, which has prevented loss of human life. Ecuador effectively managed and forecasted eruptions of the Guagua Pinchincha, Tungurahua, Cotopaxi, Sangay, and Reventador eruptive processes, among others. Similarly in Peru, the two observatories (OVI and OVIGP) have jointly responded effectively to the current eruption of the Sabancaya volcano. Mount Agung in Indonesia offers a recent example of lives that were safeguarded in 2017 and 2018. According to the VDAP Indonesia Fact Sheet: "During increasing volcanic unrest in September and October 2017, more than 130,000 people were evacuated from their homes near Mount Agung. CVGHM and VDAP staff collaborated to install and repair monitoring equipment, improve computer systems, model potential eruption scenarios, analyze seismic and ground deformation data, and develop new strategies for sampling gases in volcanic clouds. After a month of intense earthquake activity, the volcano temporarily quieted in mid-October, but then finally erupted in late November 2017 and into the following year. The eruption was relatively small and no one was injured, but the volcanic episode 22 at Mount Agung taught valuable lessons to CVGHM and their VDAP collaborators, and further strengthened a notable decades-old partnership. " The TET found that there are no indicators geared to measuring reduction of lives and property lost within the 17 indicators selected by VDAP to report to USAID. The need for this indicator has recently been acknowledged at international level and the United Nations Office for Disaster Risk Reduction began a discussion in 2019 with member nations to include a new disaster risk indicator on avoiding lost lives and property as a way to measure the effectiveness of risk reduction interventions; discussions about its implementation continue. The event tree scenarios generated with VDAP software constitute an important advance to guide the development of more complex mathematical probabilistic risk models. These exercises would help to contrast forecasts with observations, allowing evaluators to respond to the query on avoiding the loss of lives and property with higher levels of evidence. • Has the ability of the government authorities to forecast and respond to eruptions improved as a result of VDAP assistance? VDAP’s assistance contributes to the partner’s credibility and trust by training personnel through workshops [31-34]. Evidence of VDAP’s assistance in forecasting and response is exemplified by the experience of Colombia, which after the creation of its three volcanological observatories, has effectively managed and predicted challenging volcanic eruptions, such as those that occurred in the Volcanoes Galeras, Nevado del Huila, and more recently, the reactivation of Nevado del Ruiz. In Chile, since the consolidation of the RNVV Project, more than 15 volcanic crises have been managed with effective forecasts and no loss of human lives in challenging situations such as the eruptions of the volcanoes Villarrica (2015); Cordón Caulle (2011 eruption VEI 3-4); Calbuco (2015 eruption VEI 3-4), among others. Likewise in Ecuador, since the creation of its observatory, the country has been able to effectively manage and forecast the eruptions of Guagua Pinchincha, Tungurahua, Cotopaxi, Sangay, Reventador, among other eruptive processes. Observatories have been able to provide data to government authorities, for example, by working with “volcanic risk management advising regional and local authorities on the preparation of contingency plans for volcanic activity” (Peru’s OVS) and by “participating in discussion tables and providing evidence and recommendations for changes to volcanic alerts” (Peru’s INGEMMET). More specifically, in Indonesia, VDAP’s assistance with satellite sketches was the most useful resource and supporting document. The satellite sketches played a vital role in response to the 2010 Merapi eruption by convincing their governor of the urgency of the evacuation and the extent of evacuation parameters needed at the time [36, 37]. In general, the equipment support provided by VDAP has been crucial; but just as important, or more important, has been the development of the professionals at the volcanological observatories, many of whom have participated in VDAP training courses, interacting with VDAP professionals, and sharing their experiences in professional meetings promoted by VDAP (e.g., LAVAS Group and GEOVOL). All these VDAP trainings have greatly contributed to improving the technical capabilities of observatory professionals to effectively forecast volcanic eruptions. [77-81] • What is the impact of VDAP on domestic USGS programs? Due to its long international trajectory in volcano monitoring and volcanic crises management, VDAP has direct experience in managing volcano crises that can be transferred to the USGS domestic 23 program (e.g., 2018 Kilauea eruption). The possibility of professional exchanges with other countries would help gain broader context and a wider perspective for the domestic USGS program on how these countries address volcano hazards. Exposure of USGS scientists in the domestic program to different volcanic/geologic areas, particularly from observatories where eruptions are rare (e.g., CVO, CalVO), would contribute to the development of scientists in the USGS and benefit international partners as well. [72-74] • Has VDAP assistance promoted science diplomacy? Over the years, VDAP has consolidated a distinct way of reaching out to counterparts in different countries under the principle of technical-scientific collaboration, aimed at strengthening countries’ response to crises and building capacities for understanding and monitoring volcanic threats. The bridges that VDAP has created between nations allow them to face a common problem, in this case addressing volcanic threats, which require building constructive international partnerships. During crisis situations, two circumstances can create an opportunity for scientific diplomacy: first, the existence of trust in the suitability of those who can assist and advise on specific issues, and second, the availability of communication channels for direct exchange. An iconic case of science diplomacy is related to the 2008 Chaitén volcano crisis in Chile, where the VDAP team, together with the U.S. ambassador, met President Michelle Bachelet. VDAP’s endorsement was key to the establishment of the new National Volcanic Watch Network program. More recently, in cases such as the assistance provided to Argentina and Indonesia, it has become clear that this cooperation is not limited to the technical-scientific field, but transcends it, enhancing interactions between scholars and officials engaged in diplomacy within the nations involved. In Argentina, the U.S. embassy in Buenos Aires supported the Chile-Argentina workshop in 2019, endorsing its organization and convening neighboring countries to join forces and plan together. In the case of Indonesia, what began as a relationship of trust on scientific and technical assistance is being transformed and formalized, as the government of Indonesia is now working on a MOU that involves the U.S. Embassy, the Ministry of Foreign Affairs, and the Ministry of Energy and Mineral Resources, which oversees Indonesia’s Center for Volcanology and Geological Hazard Mitigation (CVGHM). According to the MDRO, VDAP should be prepared to register the assistance, share information on the financial investment, and details on all activities, including collaboration on research. [75-76] 4. Stakeholder coordination and leveraging improved due to VDAP programming • Has VDAP material assistance been leveraged by host countries to obtain more support from national or foreign donor sources? Undoubtedly, VDAP’s material assistance, complemented by the advocacy and endorsement provided by VDAP representatives, has served to open and take advantage of windows of opportunity within the countries, resulting in the appropriation of budgets for the acquisition of equipment and investment in infrastructure. The case of Indonesia is worth mentioning, where equipment donated by VDAP was used to leverage an expansion plan on monitoring equipment, resulting in an important equipment purchase made in 2018. A similar experience was mentioned by the volcanology area coordinator in Guatemala. 24 Even those countries that have reached an acceptable level of self-sufficiency in the purchase of equipment and networks (Colombia, Costa Rica, Peru), indicate that they do not have the capacity to immediately access and use these resources in times of emergency, a situation where VDAP has been able to do so, with an agility that no other international donor or partner has been able to match [48, 50, 51, 52]. Noteworthy is the case of Argentina, where VDAP’s technical advice, and not its material assistance, has served to evaluate existing equipment and propose, integrate, and expand networks financed with national resources (central and provincial government) [71, 76]. VDAP’s assistance has also helped national agencies achieve international presence and recognition. • Were VDAP activities well-coordinated with other donors to avoid duplication of effort? In the field of volcanology, excellent relationships exist between cooperating agencies, donors, and beneficiaries. However, based on survey results and interviews, the environment for international cooperation on volcanic hazards is limited, in contrast to important needs (e.g., equipment, training, research). The technical agencies and donors that are active in this field tend to remain in specific cooperation niches, including short courses, donation of equipment and networks, guided visits, and in some cases, sponsorship for higher education offerings (MSc and PhD) and research. Seminars and congresses are the spaces where different co-operators frequently converge [59-67], thus showing that there is complementarity rather than competition among them. • How does VDAP facilitate collaboration/cooperation within/between countries? What binational or multi-agency "agreements" have been fostered by VDAP? Most countries note that VDAP is in a privileged position worldwide to identify where the expertise of different groups lies as well as to establish communication channels throughout the regions. Considering that VDAP has been promoting cooperation among countries, it is in a position to map the expertise, equipment, and infrastructure available in the region and to promote and support horizontal exchanges among Latin American countries. Informally but decidedly, VDAP has already been supporting horizontal cooperation, taking advantage of the technical leadership that has emerged in the region. This is how Chile, Colombia, and Costa Rica have shared software, training, and provided technical assistance in their areas of expertise [7, 51, 68, 69, 70, 71]. The 2020 agreement between Argentina, Chile, and the U.S. on volcano studies is a clear example of science diplomacy, developed in a period in which relations between Argentina and the U.S. were not at their best. The sponsorship of VDAP was decisive in achieving this agreement and the U.S. embassy endorsed the meetings that ended in a formal agreement between the countries involved [71, 76]. 5. Recommendations for future VDAP programming • What changes should be made for how VDAP does forecasts or communicates them to improve effectiveness? A common point raised during interviews was that people significantly appreciate the information from satellite observations that VDAP provided in times of volcanic crises, either through sketches, verbal communications, or via e-mail. This information is used mainly for short-term decision-making. Nevertheless, not having direct access to the images/data prevents a deeper analysis, comparison 25 between images, and their use for hazard modeling by the countries themselves [36-40]. • Is outreach about VDAP done at appropriate levels? Is the type of outreach appropriate (social media, publications, USGS outreach products, website, factsheets)? Countries have access, on a regular basis, to VDAP’s outreach tools, such as the websites, brochures, posters, publications, and social media. In addition, daily VDAP email communications keep countries aware of and updated on news—and knowledge—on developments in the field of volcanology. All of this valuable information is highly appreciated. VDAP’s response and communication, whether in-person, email or through teleconferencing, is considered par excellence. One interviewee suggested that VDAP experts create a blog to directly connect with country professionals. In the words of an El Salvadorian official, “The creation of blogs can be a very interesting undertaking for communication among different members.” Some countries expressed special interest in increasing and scaling up joint efforts on scientific publications with VDAP. With a wealth of data at their disposal waiting to be processed, some countries (for example, Peru) look to increase collaboration with VDAP to jointly publish scientific papers, thus strengthening the scientific knowledge base and technical know-how of the countries’ professionals, giving them an opportunity to use the wealth of data at their disposal. As a Chilean official remarked, “What is important to our day-to-day work are the communications and information updates that VDAP provides by email, for example, the description of a volcanic crater that is developing, is emitting lava, there are explosions…This information has been of tremendous help in our daily lives, more so than simply a description of a scientific article ... For us, the impact has more to do with access, through VDAP, to resources that we do not have.” • Should VDAP develop new tools for partners? Are there tools that partners want that VDAP doesn’t provide? Participating countries provided a host of suggestions on tools that VDAP can improve upon and provide. Prominent among these are remote sensors and new computer software for volcanic monitoring, possibly with constant or systematic methods of data collection. Countries also look for automated systems for processing data on volcanic monitoring, seismicity, GPS, and gasses quickly and effectively. There were two specific suggestions: o VDAP platform/website: The vast information accessible on VDAP’s platform could be made more user-friendly by introducing a search engine/tool to enable easier access to information, organized chronologically or topically, and add analyzed content to enrich the user experience. o Acquire hardware capability: Countries with advanced capabilities, such as Colombia, Costa Rica, and Chile now aspire to graduate from having just software capabilities to acquiring independent hardware capabilities. They would like to distance themselves from their dependence on nearby university laboratories, which have different priorities, such as publishing. With ample subject area experts and electronic engineers, these countries would like to become self-reliant in terms of assembling the instruments they need and build the networks they require for volcanic monitoring. Finally, other suggestions include a need to increase the availability of tools; strengthen training for satellite image processing; have access to the USGS VolcView; support for drone monitoring for 26 volcanic surveillance, especially during eruptive crises events (and also for crater lake sampling); and training for preparing samples for electron microscopy. • Should VDAP make any changes to the way the program is monitored or evaluated? For the 2020 VDAP Mid-Term Evaluation, the FIU-EEI evaluation team analyzed VDAP M&E system. The VDAP M&E Team (Gari Mayberry and Dave Ramsey) worked with all of VDAP in FY 2015 and FY 2016 to develop a monitoring and evaluation plan that is now fully implemented and has been adjusted for the current IAA that runs until the conclusion of FY 2022. Details about 17 performance indicators related to overall VDAP objectives as defined in the program logical framework are compiled by all VDAP staff members on a quarterly basis using the web-based interface developed by VDAP computer scientist Dianna Norgaard in FY 2018. The interface has been refined over FY 2019 and FY 2020 and some additional data output capabilities have been added to assist with quarterly and annual report compilation. The interface populates a VDAP M&E database that can easily be accessed to fulfill queries such as where VDAP responses have occurred or where monitoring instrumentation has been donated. In addition to the detailed program performance on indicators, the report includes information on the scientific production of the VDAP team where professional and scientific excellence is exercised and reaffirmed. To complement the M&E system arranged to report to USAID and the USGS, VDAP has worked on different initiatives aimed at evaluating the effectiveness of processes and technical/scientific advances. These initiatives include multiple joint investigations, scientific publications, and equally important, regular and systematic contact with national counterparts. This informal communication has made it possible to gather a significant number of anecdotes that account for performance during volcanic crises, progress in programs and projects, institutional achievements, personal successes, as well as difficulties and challenges. This 2020 VDAP evaluation is a joint collaboration between VDAP, USAID, FIU-EEI, and a panel of international experts in the field, focusing on VDAP’s program effectiveness. The mid-term evaluation’s main difference in comparison to VDAP’s periodic reporting lies in the extensive consultation that has been made to reach out to different stakeholders such as the volcano observatory staff in beneficiary countries, VDAP/USGS, USAID/BHA, and USG (U.S. Embassy and USAID) located in the U.S. and 16 other countries in three regions, Africa, Latin America, and Southeast Asia. For the purpose of a mid-term evaluation, TET consider this approach appropriately fulfills the exercise’s purpose. TET proposes strengthening the existing M&E system by: (1) Designing a registration and tracking system for the beneficiaries of the different training and capacity building activities. The database of beneficiaries of the VDAP training compiled during this evaluation would be a starting point; (2) Designing a registration and tracking system for the VDAP donated equipment and instruments; and (3) Designing new reporting indicators on life and property safeguarded/loss of life and property avoided. Future mid-term evaluations could incorporate an economic component to broaden the scope, including efficiency in the program assessment. This economic component may include cost analysis per services and processes (e.g., training, technical assistance, labs); training on cost-benefit analysis, early warning systems cost-benefit analysis; and economic impact assessment incorporating different VDAP components. VDAP would also benefit from a fully external evaluation, involving other stakeholders in the evaluating process such as USGS, donor agencies, and international experts. 27 • Is there any feedback counterparts would like to provide? In general, all countries expressed appreciation of VDAP’s crucial role in building strong and capable volcano monitoring organizations in each of the participating countries in the VDAP network. Countries acknowledged VDAP’s vast and unparalleled contribution in technical training and knowledge sharing, capacity building, reliable and prompt support during crises, technical support, and in building an international support network. The VDAP courses, visits, and exchanges in the areas of software, satellite data processing, consulting, and HR training have increased significantly the technical competencies in all the countries. Continued support is essential for their ongoing efforts to face potential volcanic crises. Along with the high-quality scientific and technical nature of the partnerships that VDAP has created, countries were deeply appreciative of the human component of VDAPs’ collaboration. Countries commend the strong ties and the deeply helpful, patient, close, and considerate working partnership that VDAP personnel have built with all partner agencies through the years. The collaboration is doubly meaningful due to VDAP’s respectful collaboration and treatment of participating countries as equals. Partner countries feel that these soft skills of genuine collaboration make for easy and fluid access to information and knowledge. Unlike other international actors, USGS/VDAP is not academically extractive in their practice, instead guiding country professionals to be independent and lead their own research and findings. Countries suggest VDAP convene a network of volcano experts from VDAP participating countries and other international entities who can “e-gather” in response to volcano activity/unrest/events in any part of the world. Countries in distress can seek direct advice and trouble-shooting help at their time of need. At the same time, experts will gain experience in problem solving. In all, this multi-lateral collaboration could be a win-win situation during and beyond emergency crises. As an official stated, “Communication channels should be opened up. VDAP has global coverage and the convening capacity to do so. They have a global perspective on the volcanic problem, not like us, who are trying to do a good job, but here locally. VDAP has this global perspective, while one sticks to a particular volcano’s behavior; and can facilitate communication with colleagues in another country who have already lived a similar experience.” Findings by Central Topics After careful analysis of the data and information gathered through the different methods, the FIU-EEI evaluation team identified 15 central themes: National Volcanic Monitoring Services (NVMS) Equipment and IT Hardware NVMS: Gender Participation Binational Experiences Issuance of Alerts Response TAGA and Event Tree Training Non-U.S. International Cooperation International Training Program - Center for the Study of Active Volcanoes (CSAV), Hawaii Impact of VDAP On Domestic USGS Programs Remote Sensing U.S. Embassy and USAID Involvement Software Associations and Networks 28 National Volcanic Monitoring Services (NVMS) 1. Based on the interviews with volcanic monitoring services, TET identified different types of organizations within the state, and a variety of geographic locations. Position of volcanic monitoring services within the State structure 2. The 10 selected observatories have different organizational structures within the state; the three predominant types of NVMS are: independent or autonomous; academic; and integrated with other services. 3. In the first type of situation, entities such as institutes or centers can be considered autonomous when they have a certain level of autonomy in budget or decision making. This is the case in Chile and Indonesia, where a geological monitoring system is located in a ministry, with a certain degree of autonomy. Colombia offers another example, where monitoring services are located within the national geological service, with its own budget and office space. Finally, IGP in Peru has a clear budget allocation and independent physical space assigned to the agency. 4. Secondly, TET identified a series of independent institutes or research centers that operate within an academic setting, such as a university. This is the case with the observatory OVSICORI in Costa Rica, which is attached to the Universidad de Costa Rica; in Ecuador, the institute also operates in an academic setting at the Escuela Politécnica Nacional. 5. Thirdly, El Salvador and Guatemala are examples of countries where these services are ‘integrated with other services.’ In both countries, the NVMS is part of a territorial institute where volcanic monitoring is one among other hazards being monitored. 6. One country does not fit into any of these three categories, as it is still evolving. Argentina’s model was created from an initiative at the central level, but complying with provincial authority. It aims to develop a certain level of autonomy. As of now, its status and path to autonomy are still in transition. There is growing collaboration with local academic institutions (geophysics, geodesy) and international institutions (National Geographic Institute, Spain) to build capacity and train human resources. The head of the National Volcanic Network says, “SEGEMAR is the governing body at national level, with the responsibility of monitoring volcanoes.” 7. When analyzing the different ways in which NVMS are organized, it is important to keep in mind the number of people assigned to these agencies and how this influences the monitoring structure in terms of self-sufficiency. From our analysis, four countries (Costa Rica, Chile, Colombia, and Ecuador) show a potential for becoming self-reliant in the generation of software and the capacity to assemble and produce equipment and monitoring systems. These countries have already provided support to other laboratories and observatories in the LAC region. Geographic location 8. There are two main geographic location models for NVMS in Latin America. The first is based on a centralized model, often located in the country’s capital. The second acknowledges the location closer to the hazards under study, in the local context/region in which the mostly occur. 29 9. The centralized model includes countries that have chosen to structure their volcanic monitoring systems within academic settings in the countries’ capital, such as in the case of Ecuador, where the Instituto Geofísico is located in the Escuela Politécnica Nacional in the country’s capital, Quito. In this same type of centralized model, Costa Rica’s Observatorio Vulcanologico y Sismologico, is based in the Universidad Nacional (OVSICORI-UNA) in San Jose. 10. The case of Argentina is still developing as the Observatorio Argentino de Vigilancia Volcanica (OAVV), does not yet have a facility specifically designated for this activity. As mentioned before, the OAVV is currently part of the Servicio Geologico Minero Argentino (SEGEMAR), with headquarters in Buenos Aires, but plans to build a new facility in the city of Bariloche. 11. Among the countries with a decentralized organization of volcanic monitoring systems are Chile, Peru, and Colombia. In the case of Chile, the Servicio Nacional de Geología y Minería – RNVV￾OVDAS is located in Temuco, a city in central Chile, although the central administration of the RNVV as well as the volcanic hazard assessment group is based in the capital city, Santiago. For Peru, both institutes considered in this evaluation, the Instituto Geofísico del Peru, Observatorio Vulcanologico del Sur (OVS), and the Observatorio Vulcanologico del INGEMMET are located in Arequipa, a region in southern Peru that is framed by Peru´s three most active volcanoes. The two observatories have a common mission, but they belong to different ministries; the interrelationship, scope, and cooperation between both agencies is an important issue that deserves future discussion. In Colombia, the Servicio Geologico Colombiano has its headquarters in Bogota, but there are three volcano observatories located strategically in the cities of Manizales, Pasto, and Popayan. 12. In Indonesia, the VDAP beneficiary country selected in the Southeast Asian region, there is a geographical decentralization. Two large NVMS divisions (East and West) and 74 observatories are distributed throughout the extensive territory and monitor the 76 active volcanoes (of the total of 147 in the country). 13. In summary, about half of the countries under study have centralized operations in the capital cities, such as Argentina, Costa Rica, Ecuador, El Salvador, and Guatemala. Other countries have a decentralized structure of daily monitoring operations, as in Chile, Colombia, Indonesia, and Peru. National Volcanic Monitoring Services - Gender Participation 14. Women form a growing presence in the volcanology field in VDAP-supported countries, generally following a universal trend. Based on the information provided by the 10 national volcanic monitoring systems, approximately 35% of the workforce are women, of whom 10% are in positions of power. A position of power is the power a person holds in the organizational hierarchy of a particular office or rank in a formal organizational system (e.g., team leader, area coordinator, chief, director). Men Women Total Women in Power Total 307 168 475 17 Percentage 65% 35% 100% 3.6% of total 10.1% of total women 30 15. With regard to participation by gender, an analysis of the surveys that targeted the beneficiaries of the VDAP training program revealed that of the 92 people surveyed in 16 countries, 57 were men (62%) and 35 women (38%), thus showing a small increase in the participation of women, compared with the results mentioned above. 16. Regarding the interviews conducted in the 10 observatories, three were with women (30%) and seven were with men (70%). Regarding participation by gender within the VDAP team that answered the questionnaire, three were women (33.3%) and six were men (66.6%). 17. TET can conclude a consistency in the results obtained for participation by gender in the different survey methods used, with a male/female ratio of 3: 1. Regarding positions of power, the ratio tends to be lower for women. Issuance of Alerts 18. All observatories included in the study are directly involved in issuing alerts about the volcanic threat. The technical alert level of each monitored volcano is based on an evaluation of the activity observed through instrument networks and the probability of impact of the expected or ongoing eruptions. In all cases, there is a close relationship with national emergency organizations on volcanic risk assessment and associated decision-making. In some countries, technical alerts are binding for the civil protection system, but the vast majority of alerts are provided by a technical advisory body on which the civil protection systems base their decisions. Some peculiarities are recorded below. 19. The Volcanic Observatory in Chile has been working with VDAP to analyze local alert experiences using quantitative measures. It is acknowledged that currently, alerts are still qualitative in many dimensions and that is why the technique proposed by VDAP called ‘event trees’ is key, as it allows risk to be quantified in probabilistic terms. In Chile, the prioritization is a result of geology. First, volcanoes with activity in the last 10,000 years are identified. The second step is to assess the level of activity and exposed infrastructure. The process is based on the EWS system but adapted to local conditions. In 2007, Chile constructed a ranking of volcanoes and updates the database every two years for the 100 volcanoes considered active in the country. Volcanic monitoring is prioritized within the seismology discipline. The National Civil Protection System defines three commands for the EWS: technical, coordinating, and authority. The technical command issues the alert (Observatory), the coordinating command advises the authorities (Civil Protection), and authorities (Mayor) make the decisions and allocate resources. 20. The National Volcanic Network of Colombia’s Geological Service does not use the term alert to designate a warning; it refers to the level of volcanic activity. Technical information on different levels of volcanic activity will impact the alerts, and the responsibility then falls to the authorities, mayors, and governors. Two communication protocols indirectly influence decision-making processes into change alert levels. 21. The observatory in El Salvador is responsible for monitoring volcanoes, which can transition from a pre-warning stage to a warning stage. In the warning stage, a special report is issued, and all information is transferred to the civil protection agency, which takes charge of issuing future warnings. The last stage is the emergency stage; the observatory has its own protocols. 22. In the case of Argentina, the monitoring of volcanoes follows the guidelines of the National Volcano Early Warning System (NVEWS), developed by the USGS. Regarding the monitoring of volcanoes, 31 the focus is on seismic monitoring along with deformation methodologies, geochemistry of gases and fluids, and monitoring cameras. The Argentine Volcanic Surveillance Observatory (SEGEMAR) is in charge of issuing alerts, as indicated by the head of National Volcanic Network: “OAVV of SEGEMAR is responsible for the official issuance of volcanic technical alerts on volcanoes in the Argentine Republic or volcanoes in neighboring countries that may affect the Argentine territory.” 23. The Observatorio Vulcanologico del Sur (OVS) in Peru, categorizes its active volcanoes based on the Ewert et al. (2005) NVEWS methodology. Monitoring networks were set up at 12 volcanoes in southern Peru. These volcanoes are monitored with seismic equipment, both real-time geodesic and visual. Based on this monitoring, vulcanological bulletins and alerts on ash dispersion and the descent of lahars are prepared. The active volcanoes have the most seismic stations, deformation equipment and video cameras. “OVS makes recommendations to government authorities to change or maintain volcanic alert levels (volcanic traffic light), according to the analysis of geophysical parameters. Likewise, official alerts are issued on the occurrence of eruptions, ash dispersion and lahars descent.” (PERU IGP-OVS). Peru’s Observatorio Vulcanologico INGEMMET bases its alert system on the VDAP (NVEWS), with an assessment and monitoring of volcanological hazards. This forms the basis of the national early warning system for volcanoes. 24. To provide an extra-regional perspective, alerts are issued in Indonesia at the technical level by the volcanologists of the CVGHM, which is placed at the highest level in the country. “It [alerts]is only issued by the volcanologists at our Center, which is already in the highest level in Indonesia. It means—the presidential level is below that.” (CVGHM, Indonesia). Threat Assessment and Gap Analysis and Training on Event Trees This section refers to two VDAP tools that are widely appreciated by institutions and individuals who have benefited from using them: TAGA and event trees. Threat Assessment and Gap Analysis (TAGA) 25. All countries included in this evaluation agree that the hazard assessment training provided by VDAP is a fundamental steppingstone for ranking and prioritization of volcanic hazards, along with assessments. As stated by Argentina, implementing "the methodology of the National Volcanic Early Warning System… has been the backbone in planning for the Volcanic Hazard Assessment Program.” TAGA is a tool that can and should be used to minimize or prevent volcanic crises by comparing the current instruments network with the ideal instruments network for each volcano, based on ranking. Chile believes being able to create a gap analysis “makes it possible to establish whether the country and/or institution have the adequate resources to maintain adequate surveillance of the volcanic risk variable.” In Peru, both observatories, IGP-OVS and INGEMMET, worked with VDAP on volcanic ranking. OVS has since continued risk assessment in southern Peru, using the Ewert et al. (2005) methodology, with VDAP support. INGEMMET works on classifying volcanoes (through assistance from VDAP on improving their monitoring system and ranking network), based on parameters such as level of danger, volcanic activity, and surrounding population, as it is "important to improve their knowledge." 26. However, some countries in this study take a different approach using other ways to rank volcanoes. As Chile stated, "Although at the beginning we very much replicated the [VDAP] methodology, the progress that has been made in recent years has caused us to ask ourselves internally if these 32 parameters and their outcomes represent the national reality." Many apply TAGA variations that best reflect their specific surroundings, along with their individualized and present volcanic characteristics, such as population concentrations and data from current activity. For example, Colombia does not have a formal document on volcanic hazard prioritization. Their monitoring is based on "degree of explosiveness, vulnerability, exposure of communities" and uses a ranking based on their terms. They work to strengthen and add monitoring networks for volcanoes that show more instability and then work toward getting the less active volcanoes monitored. El Salvador initially received VDAP technical support and is now working on a study regarding which volcanoes can be included in the current field of 20 monogenetic volcanoes and which are not being investigated enough to be accurately classified. Ecuador does not use TAGA as a tool but rather as an approach for monitoring and allocating personnel, based on their local knowledge of volcanoes. Lastly, Indonesia has not updated its volcanic hazard assessment since the 1980’s when they worked with VDAP to classify their volcanoes by hazard type, frequency of occurrence, societal risk, and explosivity. Indonesia acknowledges that there have been volcanoes which, at the time of eruption, had not been previously classified as high risk. As stated by Chile, “threat assessment and gap analysis are exercises that should be performed periodically” to reevaluate any recent and applicable published studies, demographic shifts, additional or changes in infrastructure, and technical advancements, such as sensors and telecommunications. Event Tree Method 27. Using ‘event trees,’ a method for volcanic hazard prevention by determining the probability of volcanic eruption, has been implemented in every country participating in this evaluation. As stated by Chile, “Event trees will play a fundamental role because they allow quantifying in probabilistic term.” Factors that contribute to the variations in the way in which the event tree method is applied from country to country include: improving the method with additional knowledge and “implementation according to [their] own needs” (Colombia); shifting the focus of the method, based on their knowledge of their volcanoes (Ecuador); and insufficient personnel to apply the method (Guatemala). These countries are still developing, or are hoping to develop when possible, hazard prevention methods based on the initial event tree training provided by VDAP. 28. Applying the event tree method has contributed to valuable and opportune outcomes within the observatories and with the community. As stated by Indonesia, “The good thing about event trees is that you have to work together. You have to accept inputs from different specialties and it's a team effort.” Ecuador views event trees as a useful source of information for national authorities, providing research and reports and helping define possible scenarios. Peru’s IGP-OVS has been able to use the information produced by event trees to create volcanology bulletins that target national, regional, and local authorities. Chile sees event trees as a way to guide the prioritization of volcanic monitoring and help with issuing alerts by shifting the view from a “scenario evaluation” to a “quantitative evaluation.” 29. Argentina and Chile have used the event tree method to create a hazard map of the Laguna del Maule Volcanic Complex. This contributed to a binational (Argentina-Chile) map of volcanic hazards to better understand the volcanic complex and make a long-term evaluation of possible scenarios. Costa Rica used the method several years ago during the Turrialba crises and the Tenorio, Rincon de la Vieja, Arenal, and Miravalles volcanoes. Costa Rica has worked with the University of Costa Rica and the National Emergency Commission who are now using event trees on their own. Ecuador 33 began using event trees during the 2015 Cotopaxi eruption and continues to use the method independently. Indonesia started using event trees after their training in 2014. Peru’s observatories have both used event trees, specifically, IGP-OVS with the Ubinas volcano and IGP-OVS and INGEMMET with the Sabancaya volcano. 30. Countries have faced certain difficulties that have hindered or complicated the use of event trees. For example, Guatemala was able to use the method twice before personnel cuts resulted in a loss of staff that were the driving force for preparing event trees. This turnover caused overall responsibilities to fall on the remaining staff and the method had to be put on hold. As Guatemala stated, creating event trees “requires people that have a fairly consolidated knowledge of the volcanic activity, both historical and current,” which currently staff does not have. The training in El Salvador that began in January/February 2020 was cut short due to the pandemic. They have used what they were able to learn, began making event trees for the San Miguel volcano, and want to begin working on the San Salvador volcano. El Salvador has not had a chance to apply the method during an active eruption but has been using them as a preventative measure. Chile has had virtual meetings with VDAP on the event tree method. Colombia uses event trees as monitoring exercises. Overall, event trees are found to be very useful. Countries that are not able to apply the method, hope to do so very soon because it helps with gap analysis. Countries that are currently using the method believe that it is an approach that continues to evolve and improve, and can be used when an anomaly is present and is the “easiest method compared to the deterministic approach,” as stated by Indonesia—a country that hopes to receive further training and greater insights from different countries and activities. International Training Program - Center for the Study of Active Volcanoes (CSAV) 31. Respondents to the survey and interviewees agree on the importance of the International Training Program (CSAV course), organized by The Center for the Study of Active Volcanoes (CSAV) in Hawaii since 1990. In recent years, it has included a stay in Washington State, near the Cascades Volcano Observatory (CVO) and Mt. St. Helens. As stated by Argentina, “CSAV is the main training course for all volcanological observatories, as it provides participants with a complete overview of all methodologies and techniques used in volcanological observatories worldwide.” All countries interviewed praised the CSAV as an extremely important, pertinent, unique, and incomparable experience. Countries such as Argentina and Colombia mentioned that they experience difficulties in recruiting specialized human resources and that they do not offer advanced education and specializations in volcanology; the CSAV course, which provides a broader overview of technical aspects of volcanology, is a way to fill that knowledge gap. Ecuador values the course because it is the longest and most comprehensive training available for volcanology professionals to improve knowledge of complex volcanoes. The CSAV course allows for the discovery of new techniques that can be applied in their home countries. Guatemala mentions how attending CSAV has sparked ideas, once thought not to be feasible, to become possible by applying techniques learned in the course. Peru (IGP-OVS) highlighted the important value of CSAV training on the topic of risk communication with the press and public. 32. The CSAV course provides an updated overview of what is occurring in the field of volcanic monitoring and presents the opportunity to learn about techniques, models, and ideas that “take longer to arrive in the country,” as stated by Chile. Participants also value learning about the crisis management work of the Hawaiian Volcano Observatory (HVO) and what can be applied in their national observatories to improve techniques. The newly acquired information is then transferred 34 back to the countries through valuable presentations, workshops, and hands-on equipment demonstrations, thus contributing to improved volcanic hazard forecasting and development of new methodologies. Ecuador has specifically implemented the technique, learned at the CSAV course, of holding weekly meetings, where every specialty is present and shares new data on two or three volcanoes, adjusting priorities accordingly. Rather than meeting solely in the moment of crises, this approach allows everyone in their observatory to be aware of the current situation. 33. Another valuable outcome of the course is how it allows participants to become more well-rounded scientists with an awareness of the benefits of an interdisciplinary approach to volcanic monitoring. A correspondent from Peru’s INGEMMET states, “[CSAV] has allowed me to open up and understand globally what volcanology is all about. Before participating in this course, for example, I was only focused on geology and volcanoes… Now I am interested in other topics that before perhaps did not seem relevant to me, such as the gas chemistry and seismology.” Networking is a valuable outcome of this course because information about how different approaches work in the United States and other countries can be shared among the participants, “which helps improve the interrelation between the volcanological observatories of different countries,” as stated by Argentina. Countries such as El Salvador and Costa Rica, mention that a variety of specialists have attended the CSAV training, “not only volcanologists but also the technicians, and colleagues from electronics and others have taken this course.” The participation of professionals from different specialties depends on the individual observatory’s total number of employees and how they distribute participation in the course. 34. Participation in the course varies from country to country. For example, Argentina’s observatory is up and coming, and consequently, only two people have attended the CSAV course, whereas in Indonesia, about 20 people have attended. From the countries included in the evaluation, the people who participated in the course are rotated, allowing different employees to attend each year. According to the survey conducted by FIU in 2020, 47 people (51% surveyed) who had attended CSAV since 1999 responded to the survey. 35. The VDAP survey included a set of questions based on the KAP method to assess knowledge, attitudes, and practice/implementation among VDAP training beneficiaries. The first component, knowledge, constitutes the relationship between volcano monitoring, forecasting eruptions, and emergency management. The second component, attitude, follows the knowledge component, and is fundamental to understanding how these measures have a value attached that could translate or apply to these topics, while indicating there is a change in behavior and or attitudes. The third component that is assumed in this framework is practice, where a close assessment of outcomes takes place (e.g., at the local level, it would involve the assessment of outcomes that follow successful evacuations alerts or local interventions determined by a land use management plan or community action). The responses use a Likert scale, which is integrated into a KAP composite index made up of three sub￾indices: knowledge, attitude, and practice. When analyzing the result of the KAP index in the 92 surveys received, it was found that the group that completed the CSAV course had significantly higher scores compared to the group of survey respondents who had not taken the CSAV course (the knowledge subindex [M = 0.537 vs. M = 0.468] and practice subindex [M = 0.545 vs. M = 0.506]). This result is consistent with the positive subjective appreciation of the CSAV by those who completed the course and the heads of the national volcanic monitoring systems interviewed. This KAP method study will be submitted to a peer review for later publication. 35 Remote Sensing 36. All observatories included in the evaluation consider remote sensing technology and satellite sketches, provided by VDAP, as an essential component of volcanic monitoring and crisis prevention. This very detailed information on crater conditions (e.g., lava extension, dome growth) allows monitoring of a volcanic process. This is very valuable for observatories that do not usually have the resources to perform flyovers and periodic field surveys, and allows validation of local observations. As Ecuador states regarding satellite sketches, “they are quite useful because it allows us to confirm things that we have seen or realize things that are happening that we don't have access to. It is an important input for our work.” Argentina mentioned that because the country has a strong national space agency that provides satellite images to SEGEMAR, they do not regularly request sketches from VDAP. Nonetheless, Argentina has received remotely sensed data from VDAP for the major eruptions of the Calbuco and Cordón Caulle volcanoes, which complemented the information they had at that time. 37. Aside from Argentina, the remaining countries do not have adequate remote sensing and/or do depend on VDAP for remote sensing equipment and satellite sketches that have proved to be vital for evacuation decisions and monitoring of volcanoes in remote areas. In Indonesia, the sketches delivered for the Merapi eruption in 2010, were the most significant contribution to decisions regarding an important evacuation. Without the sketches, the pyroclastic flows would have been underestimated and would have caused a tragedy; instead, CVGHM used the sketches as a supporting document to prove the severity of the situation to the governor. 38. In countries such as Chile, Colombia, Ecuador, El Salvador, and Peru’s IGP-OVS and INGEMMET, whenever an anomaly or volcanic unrest presents itself, the observatories communicate with VDAP to access remote sensing data or sketches, which provide qualitative and quantitative information that is not readily available. Colombia began receiving sketches in 2007-2008 showing the activity of Huila Volcano; it began receiving sketches for Nevado de Ruiz when the dome formed in 2015 (it is complex and difficult to observe). They also believe it is important to have remote sensors and data at their observatory. Ecuador receives images every 1-2 weeks and has remote sensors in place because they do not have equipment that can perform overflights or camera equipment to help gauge the level of current danger. Remote sensors, therefore, provide effective and rapid answers. Peru’s INGEMMET also receives periodic sketches and additional information when domes form. 39. Indonesia mentions an important 2010 contribution, when VDAP helped develop the first network on North Sulawesi, where monitoring conditions are difficult. CGVGHM was having difficulties transferring data from different islands to the observatories on North Sulawesi and with help and some equipment from VDAP, were able to transfer information by satellite and internet. 40. Several countries still need further assistance, are aware of gaps in their training that need to be strengthened, and/or see where there is room for improvement in remote sensing. For example, Guatemala believes that remote sensing and analysis of satellite images would be an important step to understanding the volcanic threat. However, the country lacks continuity of work because the Guatemalan observatory “does not have the capacity… department, or even a person who is in constant communication and working on the issue [remote sensing monitoring].” Costa Rica would like to have a “concrete” workshop that focuses on remote sensing because it is difficult to find available data and finding data with the necessary attributes consumes a large amount of time. They 36 would appreciate a course on “how to collect and process the most useful information available, such as thermal anomaly quantification, atmospheric gas quantification, ash detection in atmosphere/on the ground, topographic changes, interferometry, and meteorological data.” Most countries have a wealth of data on images, but either have no software or capacity to process it, or both. Colombia believes that having access to a robust database with organized metadata, software (which could be commercial), and centralized processing capacity would be beneficial to everyone. However, currently, this capacity is non-existent, and countries are not aware of image banks that are available. A hassle-free, seamless access system, clearly organized, could be provided to all countries. Other international networks of experts, such as ASGMI and IAVCEI, could be brought in to increase information sharing. With sound procedures, policies, and coordinated management, this could become an information and knowledge hub that benefits everyone. Peru’s IGP-OVS requires technological assistance to automate systems that processes volcanic monitoring data (seismic, GPS, remote sensing, and gas data) quickly and effectively. Finally, Argentina is interested in VolcView which would facilitate monitoring of volcanoes in remote and high places. Software 41. Much of the software and training on the software that is used extensively in the observatories was initially provided by VDAP. Most observatories are using 'Earthworm' as their main data acquisition software, and they use 'Swarm' as the software for immediate display of seismic data (both provided by VDAP). The software is constantly being updated and improved and new versions have improved the rapid analysis and visualization of seismic signals, which is extremely important for continuous monitoring of volcanoes. VDAP software training has allowed some countries to modify the software as needed. Argentina, Ecuador, and Peru’s INGEMMET use the software provided by VDAP, which allows processing and uses databases of computer systems to improve the initial product. Guatemala uses software initially produced by VDAP and has found it easy to modify to the situation in their country. In the past 1.5 years, the INVISUMEH observatory has been able to write computer programs that helps analyze data from different stations. Nonetheless, Guatemala states that they “could not say that INVISUMEH is already creating its own software. In fact, almost 80 percent of the software that INVISUMEH uses to monitor volcanoes come from VDAP.” 42. Certain countries have created software and shared it with other countries. For example, Costa Rica developed deformation monitoring software and has shared it with Colombia and Peru. 43. Indonesia mentions that they use Earthworm and receive software assistance from France. They have developed a software called MAGMA Indonesia and “have used some data from Swarm and Valve to provide live seismogram information for the public;” it is becoming easier as data becomes digitized. Additional information collected from the VDAP survey states that major advancements have been made possible through VDAP support. The following summary of products that have resulted through these developments or from current projects is neither country-specific nor comprehensive, but includes: automated geodetic monitoring processes; NOVAC system and multiGAS stations implemented; python-based real-time analytic tools; automatic unloading and processing systems of the country’s GNSS stations with GAMIT/ GLOBK; dMODELS for inclinometers; software for InSAR, tilt, and GPS data processing for quantifying volcanic deformation. 44. Argentina and Chile have mentioned certain areas they would like to improve. Argentina states that they want to see “new computer software applied to monitoring that can be implemented in different volcanological 37 observatories” and are interested in deformation monitoring. Argentina also would like to have access to VolcView, which has been used to analyze Alaskan volcanoes that they consider similar to the large and isolated Argentinian volcanoes. Chile would like to take steps to develop software and hardware with the electrical engineering group at the National Geology and Mining Service. Equipment and IT hardware 45. This section has been prepared based on two different data sources: (1) the list of VDAP-donated equipment (2018-2019), and (2) the remote visits conducted at the volcanological observatories in Chile, Colombia, Peru, Costa Rica, and El Salvador. VDAP-donated equipment 46. Based on the list provided by VDAP of equipment donated between 2018 and 2020 (Annex IV, page 125), TET divided the equipment into five categories, according to the volcanic surveillance methods and/or specific area to be strengthened (Figure 3a). One can observe that the category receiving the most donations is volcanic seismology (22 instruments), followed by geochemistry (20 instruments). Figures 3a and 3b. General distribution of VDAP donated equipment 2018-2020. 47. In volcanic surveillance, volcanic seismology and acoustic stations are considered as one of the most effective tools to carry out permanent monitoring, due to their state of development, and considering that they have been widely used worldwide for the observation of precursor signals of eruptive activity. For this reason, all efforts aimed at strengthening seismic and acoustic networks in active volcanoes will result in effective volcanic crisis management. 48. During the last decade, the study of the geochemistry of magmatic fluids has experienced a significant advance in terms of the development of new tools to better observe and monitor magmatic fluids and gaseous emissions toward the surface (e.g., DOAS, multiGAS, IR cameras, among others). The donation of this type of instrument by VDAP has allowed: (1) a better understanding of volcanic processes and the behavior of fluids emitted toward the surface; (2) determination of baselines prior to volcanic crises and their evolution during eruptive processes, (3) establishment of activity thresholds, and (4) more informed decisions to achieve an effective (a) (b) 38 technical response during volcanic crises, with multiparametric data analysis that allows the building of conceptual models of monitored volcanic systems. 49. Regarding the equipment that contributes to the monitoring and study of surface processes and emitted products, the donated equipment included rain gauges (probability of generation of secondary lahars), FLIR cameras (taking the temperature of emitted products), and dredge (for taking rock samples). There is an important effort to support the strengthening of data telemetry through the donation of digitizers and accessories for digitizers, lightning protection equipment, as well as the modernization of analogue instrumentation. In a smaller number, but not less important, are instruments related to geodesy, specifically from two GNSS stations provided for the Rabaul volcano (Papua New Guinea). 50. Four countries lead the list of donations by country (Figure 3b): Ecuador (12 instruments), Papua New Guinea (10 instruments), Guatemala (nine instruments) and El Salvador (six instruments). These four countries account for about 71% of all donations since 2018. This suggests that the donation process has been influenced by the need to strengthen instrument networks during recent eruptive cycles in the aforementioned countries, as in Ecuador, with the Cotopaxi eruptions (2015 ) and Sangay (with an ongoing eruptive cycle); Papua New Guinea, with the eruption of the Ulawun volcano (2019); Guatemala, with the eruption of the Fuego volcano (2018 major eruption and specific reactivations in subsequent years); and El Salvador, with the San Miguel volcano (2015). Remote visits to volcanological observatories in Latin America 51. The main objective of the virtual visits to the volcano observatories of Chile, Colombia, Peru, Costa Rica and El Salvador was to observe the organization, facilities, equipment, and networks available in VDAP beneficiary countries. Special emphasis was placed on the operational status of the equipment donated by VDAP and how the equipment has impacted the routine tasks of the observatories. During the visits, two main aspects were identified: (1) there are countries that, due to their level of development in terms of volcanic monitoring and their current budgetary capacity, have received no equipment (e.g., Argentina), or a low number of instruments from VDAP in recent years in response to specific needs (e.g., Chile and Colombia), and (2) countries that have received equipment on a regular basis from VDAP. In the first group, VDAP’s support focused on strengthening human capital and the exchange of experiences. It is important to note that even with this level of self￾sufficiency, Chile and Colombia could require equipment assistance during volcanic crises as administrative or regulatory constraints may prevent them from purchasing new equipment in a short period of time. Both countries recognize the importance of VDAP in strengthening their monitoring and support capacity in volcanic crises such as those that occurred in Chile with the Chaiten volcano (2008-2009) and the Nevado del Huila eruption in Colombia (2007-2010). In the latter case, an AFM system (surveillance of lahars, see Figure 4, Surveillance of emitted products) and some seismic stations were essential during the Huila volcanic unrest, allowing for effective management of the crisis. 39 Figure 4. Equipment reported by the volcano observatories during the virtual visits. Donations made between 2007 to 2020. 52. Costa Rica has received instruments from VDAP to respond to the Turrialba volcano (2016) and Rincón de la Vieja (2019) crises. This support is considered extremely important. It should be noted that Costa Rica, in turn, has supported the assembly of equipment and provided technical support to nearby Central American countries. 53. VDAP’s support in terms of equipment numbers, methodologies, and infrastructure has been significant for Peru and El Salvador (Figure 4). In the case of Peru, the two volcanological observatories, INGEMMET and the IGP-OVS, have received comprehensive support in the form of instruments that include sensors, assistance to strengthen data telemetry, and hardware and electronic components. These volcanological observatories are in the process of consolidating their surveillance networks and their monitoring centers. The donation of two data servers for the OVI (IT equipment in Figure 4) has considerably increased data storage capacity and allowed the use of specialized software for the processing and modelling of data from multi-parameter surveillance networks. In the case of El Salvador, the donation of instruments has been diverse, highlighting the importance of equipment for analysis of emitted products (binocular magnifying glass and polarized light microscope), data servers, and seismological and geochemical equipment. An important finding indicates that the donated seismic sensors have not yet been installed due to a lack of capacity to install them. VDAP could assist them in the future. 54. Costa Rica made an interesting suggestion: organize a regional ‘collective’ of equipment carrying ‘regional property tags,’ i.e., equipment owned by the region and not any one country that can be dispatched and shared in times of need. The ‘collective’ would include instruments such as seismological stations, multigas stations, and GPS stations and could cross borders without customs problem. This would facilitate cooperation among countries in the region. Binational experiences 55. Between FY 2018 and FY 2020, VDAP organized four binational exchanges between the U.S. and Latin American countries. For those attending these bilateral exchanges, the events constituted a unique opportunity to strengthen relationships between individuals, agencies, and countries. A binational 40 exchange constitutes a unique opportunity for experiential learning. It is built on the basis of enabling a space for interaction among communities from different nations that share common volcano hazards and looking for a learning and motivational opportunity. A 2020 article about this binational experience was published in the Journal of Applied Volcanology (Driedger et al., 2020), where the authors highlight one of the main features of these encounters: “Travel to a disaster area that included conversations with original first responders and survivors could give officials valuable knowledge and the credibility back home as second-hand witnesses to eruption disaster and recovery.” 56. The following information about the Binational Exchanges, organized between 2017 and 2020, was extracted from a report prepared by the VDAP team, which is included in Annex IV, page 105. • U.S./Colombia Binational Exchange – (18–25 November 2017). A binational encounter in risk management and education was held in Popayán, hosted by the Colombian Geological Service (SGC). Popayán is home to one of SGC’s volcano observatories and is near Nevado del Huila, Puracé, and Sotará volcanoes. Objectives of this exchange were to bring attention to existing volcano education efforts, identify common elements of success, and select a path forward toward creation of a national volcano education curriculum for Colombia. • Colombia and Ecuador/U.S. Binational Exchange (14-19 October 2018). Geoscientists, emergency managers, and first responders from Colombia and Ecuador attended a binational exchange in Washington State. Participants learned about the geology, eruption history, and potential hazards of Mount Baker and Glacier Peak in northern Washington. Participants attended an emergency preparedness exercise in Whatcom County featuring an eruption scenario for Mount Baker, the northernmost volcano in the U.S. Cascades which poses potential hazards to nearby Canada. • U.S./Ecuador Binational Exchange, Ecuador (2–8 June 2018)– Emergency managers representing the U.S. Forest Service, the State of Washington, and four counties in Washington State that lie partially within hazard zones around the Cascades volcanoes traveled to Ecuador from for a binational exchange focused on preparedness and response to volcanic eruption crises. They were joined by scientists from the USGS Cascades Volcano Observatory and VDAP. The U.S. delegation met with counterparts from the Instituto Geofisico, Escuela Politecnica Nacional, SNGRE (Ecuador’s National Risk and Emergency Management Service), and Ecu911 (Ecuador’s Integrated 911 Security Service) to learn about lahar detection and notification systems and other volcanic risk mitigation measures at Cotopaxi and Tungurahua volcanoes, which have much in common with Cascades volcanoes. Two members of OFDA’s LAC staff also attended the exchange. • U.S./Colombia Binational Exchange, Colombia (24-28 February 2020). In commemoration of the 35-year-anniversary of the eruption of Nevado del Ruiz volcano on November 13, 1985, the Servicio Geológico Colombiano (SGC) and its partners sponsored the Fifth National Biennial of Children and Young People Living in Volcanic Risk zones (commonly referred to as ‘The Bienal’). Organizers assembled Colombian students and teachers for four days of intensive focus on the scientific and cultural aspects of volcanoes adjacent to these at-risk communities. Three U.S. officials from the small town of Darrington, WA participated, including the town’s mayor, school district superintendent, and coordinator of youth programs. Their attendance was aspirational, because while Darrington has not taken steps to reduce risk from for the nearby 41 Glacier Peak volcano, they intend to do. The U.S. travelers had responded to the 2014 Oso landslide (Washington State, 43 fatalities), which had many of the same characteristics as the 1985 Nevado del Ruiz lahar tragedy. Response 57. Between FY 2018 and FY 2020, VDAP responded to several volcanic crises, on-site and remotely. The following information was extracted from a report prepared by the VDAP team, which is included in Annex IV 115. • Agung, Indonesia (three on-site response teams, September 2017). The first team arriving in Agung assisted with seismic interpretation, assessing needs for monitoring instruments, and laying the groundwork and planning for two response teams that followed. The second VDAP response team arrived in Bali in early October. They assisted CVGHM with pyroclastic density current modeling; seismic data display and interpretation; GPS data retrieval and interpretation; researching the eruptive history of Agung; developing a conceptual eruptive model; satellite data interpretation; and upgrading and repairing the network of monitoring instrumentation and observation stations around Agung. The third VDAP response team arrived in Bali at the end of October as the second team was departing. They continued to assist with seismic interpretation; improving the monitoring network and observation stations; sampling cold springs around the volcano for chemical analyses that could help determine the presence of magmatic gases; and preparing and using the Aeroterrascan drone for gas monitoring flights. The team departed Bali on November 20. Finally, VDAP met at the Cascades Volcano Observatory on December 14 and conducted a Post-Response Assessment (PRA) for the response to Agung. • Fuego, Guatemala (On-site response, June 2018). VDAP responded onsite from June 13-30. Many other VDAP professionals helped remotely. VDAP assisted counterparts at the National Institute of Seismology, Volcanology, Meteorology and Hydrology (INSIVUMEH) to assess this large eruption and analyze the probabilities of another similar event occurring. VDAP helped counterparts recognize areas that could be at risk from future lahars or pyroclastic flows, using remote sensing and hazards mapping. VDAP provided remote sensing updates for INSIVUMEH and prepared situation reports for OFDA (now BHA) during the crisis. • Morne Plat Pays, Dominica (remote response, 2019) involved the shipment of seismic and telemetry gear to Dominica to monitor volcanic unrest. Counterparts at the Seismic Research Centre (SRC) of the University of the West Indies installed the equipment. • Ulawun, Papua New Guinea (on-site response, 2019). A capacity building trip ended up being an on-site response when Ulawun erupted explosively on June 26, its first eruption in nearly 20 years. VDAP assisted RVO to prepare helicopter gas and photogrammetry surveys of the volcano, installing a NOVAC gas monitoring station at the Ulamona mission, and training RVO staff on how to analyze and interpret data streaming back to the observatory from the new gas instrumentation at Ulamona. • Taal in the Philippines (remote response, January 2020). VDAP assisted counterparts at PHIVOLCS to develop a forecast, using the probabilistic event tree method for activity at Taal following the initial explosive eruption. VDAP provided considerable remote sensing support 42 and geodetic and InSAR modeling of ground deformation. VDPA temporarily loaned a FLIR camera to measure the temperature of cracks forming around Taal. • Sangay in Ecuador. (remote responsese, June 2020.) Similar to what occurred in PNG in 2019, as VDAP was hosting a virtual seminar with counterparts at IG-EPN in Ecuador in early June, 2020, there was explosive activity and ash emissions at the Sangay volcano. The focus shifted to a discussion of recent events at Sangay and how to improve monitoring. VDAP assisted counterparts at IG-EPN to develop a strategy for placement of additional instrumentation to monitor the eruption and provided remote sensing support through access to available imagery and image analysis tools. 58. These responses provide ample evidence of the importance of the previous relationships that VDAP has built with the different countries, its unique capacity to provide technical assistance during a volcanic crisis, and its ability to mobilize and install equipment on a short notice. These responses undoubtedly offered an important and irreplaceable opportunity for capacity building at the national level, a fact that has been widely ratified in the surveys and the interviews carried out. Although surveyees and interviewees have indicated a preference for in-person activities, the pandemic has reinforced the benefits of remote assistance, allowing the continuity of many of the services that VDAP provides. Non-U.S. International Cooperation 59. This evaluation contains references to several international cooperation activities with the countries. However, the type of assistance varies significantly from one cooperating agency to another and from one recipient country to another. International cooperation has been provided from countries such as France, Germany, Italy, Mexico, Singapore, Spain, Switzerland, and Japan. In addition, the Central and South American countries that are part of the evaluation have, in one way or another, interacted with each other and provided assistance. 60. The countries that have benefited from the France's Institut de Recherche pour le Développement (IRD) cooperation in the long term are Ecuador, Indonesia, and Peru’s IGP-OVS and INGEMMET. IRD, as stated by Ecuador “has provided some instruments in a much more limited way, but above all, it has allowed us to train the staff” and makes a point in saying that “The success or the achievements that the institute [IG-EPN] has reached are due to its trained personnel; eight out of 10 Ph.D.- level employees at the institute were trained in France, where scholarships covered the training of master's and doctoral level candidates. According to Peru’s IGP-OVS, which works directly with Clermont-Ferrand and the Clermont-Auvergne (French research centers associated with the IRD), “normally these scholarships are fully paid or almost at 80 percent, so that has also made it easier for us go to France.” 61. Every country, with the exception of Ecuador, mentions the assistance and support of the Japan International Cooperation Agency (JICA). Colombia, Costa Rica, El Salvador, Guatemala, Indonesia, Peru’s IGP-OVS and INGEMMET have all received assistance from JICA in the form of monitoring, donations, tools, and training, which have contributed to volcanic monitoring and have strengthened volcanological hazard studies. Guatemala has worked with JICA to produce volcanic risk maps for the Pacaya volcano, the Fuego volcano, and the Santa Maria volcano. Argentina states that they have attended short courses with JICA, “however there was no support for the development of volcanic monitoring capabilities…but there has been a long history of funding for other specific projects.” 43 Indonesia states that JICA “gives very strong support with their scholarship... the JICA Monbukagakusho scholarship [for graduate studies.]” Colombia also benefited from new ideas from JICA and is now working with UNESCO Global Geoparks on the Nevado de Ruiz project, which “will strengthen communities for sustainable development.” 62. Costa Rica and Chile face certain challenges with international cooperation because they have been ranked a middle-income country and as such, technical assistance and international aid is limited. Germany has helped Costa Rica with some tectonic analysis. 63. The Swiss Agency for Development and Cooperation (known in Spanish as COSUDE), has asked the OVSICORI to become involved with a Central American early warning system project because they are the only country with the institutional capacity in the region. The project is now managed by Costa Rica’s National University. 64. Switzerland has helped Guatemala by improving the monitoring and alert system after the eruption of the Fuego volcano and wants to establish “a monitoring system that involves civil protection, communities, and also INSIVUMEH.” 65. Peru’s IGP-OVS has worked directly with the University of Geneva on specific topics of research and with the Canarian Volcanology Institute (INVOLCAN), located in Spain, developing specific projects. 66. Argentina, El Salvador, and Peru’s IGP-OVS have worked on short-term projects with Italy’s research groups. 67. Indonesia has worked with Singapore’s Earth Observatory Singapore (EOS) and strengthened the monitoring system by using selected robotics and autonomous systems (RAS). 68. In the Central and South American countries, joint agreements, tools, and training have been shared. For example, El Salvador, Guatemala, and Nicaragua work together by sharing seismic networks, exchanging equipment, and providing measurements, due to the active volcanoes in the region. El Salvador has a good relationship with the University of Costa Rica and both have created workshops together. Finally, Guatemala collaborated with Chile to gain further insight into volcanic monitoring; even though the formal collaboration has ended, the contacts and relationships formed continue. Guatemala has also been working with technicians in Costa Rica and has worked on issues with Mexico’s National Center for the Prevention of Disasters (CENAPRED), but no long-term outcome has resulted. 69. Colombia and Ecuador work together on projects and joint efforts because they share a high seismic threat zone in the north of Ecuador and south of Colombia, where there are active volcanos at the border between the two countries. As Ecuador stated, “we have collaboration agreements, we exchange instruments, we receive data from the Colombian network for Cerro Negro and part of the south coast [and vice versa].” 70. Peru’s IGP-OVS and Chile have certain inter-institutional agreements and conduct joint research work, such as on the Casiri volcano. IGP-OVS believes that their participation is very important because they have communicators, sociologists, and psychologists that improve their survey and work with the Catholic University of the North in Chile. 44 71. VDAP has been instrumental in fostering cooperation between Argentina and Chile. Argentina was previously governed by Chile’s volcanic technical alerts, which presented complexities in Argentina. Now agreements have been signed by the highest authorities in both countries and they will work together on long-term on alert levels. Argentina is working on installing the equipment and the first seismic, geodetic and geochemical data exchange tests in neighboring Andean volcanoes will begin soon. Furthermore, in terms of hazard assessment, Chile has published the first ‘tri-national’ hazard map (Argentina-Chile-U.S.) for the Laguna del Maule volcanic complex, where VDAP support was essential for the development of the project. Impact of VDAP on domestic USGS programs 72. The following section was prepared based on inputs from the VDAP team and reviewing institutional reports. Due to its long international trajectory in volcano monitoring and volcanic crisis management, VDAP has: • Direct experience in managing volcano crises that can be transferred to the USGS domestic program. • A long track record of professional exchanges with other countries that would help the USGS domestic program gain a broader context and perspective on how other countries address volcano hazards. This would benefit both organizations, the U.S. domestic program and international partners. • Exposure to different volcanic/geologic areas, particularly those where eruptions are rare (e.g., CVO, CalVO), would contribute to the development of USGS scientists. 73. In practical terms, there are specific references to the impact of VDAP on domestic USGS programs: • The domestic program relies on the VDAP cache for potential domestic eruption responses, i.e., “the domestic program only has a very limited cache by itself.” • Because VDAP´s work focuses on reducing volcanic risk in developing countries, VDAP has gained a vast experience (a high number and all type of active volcanos). The USGS domestic program is excellent but its expertise is limited to the existing volcanos in the U.S. territory. However, a VDAP expert said: “… our staff can easily plug holes in domestic eruption responses, and we can play leadership roles in some other projects. For example, VDAP staff have familiarity with remote sensing, use of UAS, forecasting with event trees, field methods for installing equipment, and databases…We have led workshops for staff of the domestic program on several of these topics.” • Technical advances sometimes come into the USGS from VDAP. (e.g., “… the 1996-now Aleutian Expansion (of seismic monitoring on the Aleutian Islands) used instrumentation practices standardized by VDAP for rugged areas with infrequent access. Arguably those practices have resulted in much higher reliability of the Aleutian monitoring networks than would have been the case otherwise.”) • Domestic researchers benefit from exposure to VDAP information on eruptions occurring around the world. • The VDAP International Charter allows for support to domestic crises (e.g., 2018 Kilauea eruption). • Some VDAP capabilities can be used as models to develop similar activities at the domestic level (e.g., the NVEWS 24/7 watch office could use VDAP remote sensing capabilities as a reference). 45 • Many of the tools VDAP provides to partners come from domestic USGS/VSC developments. Software commonly used in VSC for monitoring volcanoes, such as Swarm, are maintained by VDAP’s software developer. Swarm benefits from enhancements and bug fixes made by VDAP. Although, NOVAC software is not yet used as widely in VSC, it will be installed in Alaska and will benefit from changes made by VDAP. Updates made to NOVAC software were used during the 2018 Kilauea eruption. • Earthworm, Winston, and Valve documentation, compiled by VDAP, can also be utilized by the USGS domestic program to train new staff. 74. Finally, the impact of VDAP on the professional development level of those who work in the program deserves to be mentioned. In this regard, a VDAP team member indicated: “Personally, I view my time with VDAP as a post-grad education that has an equivalent or greater impact on my understanding of volcanic systems as compared to my formal schooling.” U.S. Embassy and USAID Involvement 75. The U.S. Mission abroad has played a critical role in the implementation of the VDAP/USGS program in the different regions. The following section was prepared based on inputs from the VDAP team and the MDROs, gathered though the questionnaire sent to them, as well as from the interviews with the national volcano monitoring systems. USAID missions and U.S. Embassies: • are primary beneficiaries of VDAP expertise, receiving briefings and useful information regarding existing volcano hazards in the country. • can expedite the passage of personnel and equipment through immigration and customs (Costa Rica, Indonesia, and Philippines). • deliver timely assistance with shipping and customs of volcano monitoring equipment used in capacity building initiatives (e.g., Costa Rica - NOVAC workshop 2015). • provide communications and other technical devices to VDAP staff during capacity building and eruption response operations (Papua New Guinea–U.S. Embassy provided satellite phones and GPS trackers to VDAP staff. Ulawun, 2019). • assist VDAP in organizing workshops and seminars (for example, facilitating and providing logistics and administrative support to the LAVAS meeting in Pasto, Colombia in 2019, at which an Embassy Science Fellow facilitated the presence of a VDAP staffer who then helped organize the workshop). 76. Last but not the least, the USAID missions and U.S. Embassies exercise scientific diplomacy together with VDAP, convening neighboring countries overseas to join forces and plan together to achieve efficient volcanic hazard monitoring and develop a response capacity to volcanic crises. (The U.S. embassy in Buenos Aires supported Chile-Argentina workshop in 2019.) In crisis situations there are two circumstances that create an opportunity for scientific diplomacy: first, the existence of trust in the suitability of those who can assist and advise on such specific issues, and second, the availability of communication channels for direct exchange, without restrictions. The situation posed by the MDRO in Indonesia could become an exception, as the Government of Indonesia is now working on a new MOU that involves U.S. Embassy, the Ministry of Foreign Affairs, and the Ministry of Energy and Mineral Resources, which oversees Indonesia's Center for Volcanology and Geological Hazard Mitigation (CVGHM). According to the MDRO, VDAP should be prepared to register the assistance, share information on the financial investment, and details on all activities, including collaboration on research. 46 Associations and Networks 77. Respondents to the survey and interviewees recognized the importance of networks and associations in advancing knowledge, exchanging experiences, and networking in volcanic studies. Likewise, they recognize the support that VDAP has provided to these initiatives: (1) co-organizing events or parallel sessions; (2) participating by conducting lectures, courses, workshops, and practices on equipment installation and operation, data collection and analysis, among others; (3) co-financing events; (4) extending invitations and financial support to meetings´ attendees worldwide. VDAP makes significant contributions to support the participation of professionals from the observatories in the conferences, workshops, and courses they organize. Some of the existing associations and networks are included below. 78. International Association of Volcanology and Chemistry (IAVCEI): This is the international volcanology association par excellence, which together with other organizations, such as the seismology association (IASPEI), is grouped under the International Union of Geodesy and Geophysics (IUGG). In addition, within IAVCEI there are commissions that carry out their own activities. Of particular interest to the observatories is the WOVO (World Organization of Volcano Observatories). Although it has been somewhat inactive, an effort is underway from the Singapore EOS to create a platform called WOVOdat, where the observatories can share experiences and volcanic monitoring data. It should be noted that VDAP and USGS members are part of various commissions, and specific activities are sometimes derived from these associations. 79. Asociación Latinoamericana de Volcanología (ALVO), [Latin American Volcanology Association]: This is a ‘local’ version of the IAVCEI, but does not compete with it. The ALVO was created in 2010 to foster collaboration among Latin American countries and has been gaining strength ever since. On its board of directors are representatives from different regions and also a student representative. ALVO has played an important role in establishing initiatives among academic and professional observatory teams in the region. In particular, two meetings of Latin American observatories were organized (both in Arequipa, Peru) in 2015 and 2018, financed by IAVCEI and VDAP. Additionally, internships have been carried through exchanges of professionals and students in the region. Last year the first ALVO congress was organized in Antofagasta, Chile. Although carried out virtually, resources from USAID￾BHA facilitated the participation of young students and researchers. Although VDAP was not directly involved in the organization, they did participate in the congress through exhibits and demonstrations. 80. Asociación Latinoamericana de Sismólogos en Volcanes (LAVAS) [Latin American Association of Volcano Seismologists]: Unlike IAVCEI and IASPEI, this association is oriented to the Latin American community, with strong support and funding from VDAP since its inception. Workshops and meetings are frequently held, and have been a significant source of shared experiences and progress in the region, in addition to the transfer of analysis tools. 81. Asociación Latinoamericana de Geodesia Volcánica (GEOVOL) [Latin American Association of Volcanic Geodesy]: Focuses on surface deformation analysis associated with volcanic processes. It also has strong VDAP support. As in the case of LAVAS, there are VDAP specialists behind these groups. 82. The Iberoamerican Association of Geological and Mining Surveys (ASGMI): Aims to strengthen associated geology and mining services in Latin America and the Caribbean. It promotes the generation and dissemination of geoscientific knowledge, and contributes to international cooperation, scientific and technical issues such as the prevention and mitigation of disasters due to natural phenomena or the 47 effects of climate change. There have been talks of establishing teams that could provide assistance to countries during volcanic crises, if required/requested. 83. In recent years, VDAP has organized and financed various scientific thematic workshops in the region (e.g., on numerical modelling techniques, gas geochemistry, deformation, among others). However, the first meeting on operational (not scientific) aspects, the Latin American Region Volcano Observatory Information Technology (IT) Summit, held in Guatemala in April 2019, deserves attention. In addition, to bringing together professionals from the region who work in IT aspects for the first-time, the summit discussed challenges that volcanological observatories face in terms of data storage, availability, and processing. This will allow VDAP to better focus its efforts in the operational aspects of observatories in the coming years. 84. The CSAV international course, organized by the University of Hawaii, has a wide scope and relevance in training professionals from different parts of the world on hazard monitoring and evaluation techniques. Scholarships from VDAP have made it possible for numerous professionals in the region to participate in the latest CSAV courses. VDAP professionals are part of the academic staff of the course and in recent years, the program includes a visit to the State of Washington (U.S.), near the Cascades Volcano Observatory (CVO) and Mt. St. Helens. [31-35] 85. Finally, VDAP, in conjunction with INGV in Italy, has conducted four VOBP (Volcano Observatory Best Practices) workshops. The last one, in Mexico in 2019, dealt with the observatories' operations during volcanic crises. These meetings are closed and professionals from different countries of the world are invited. VDAP not only supports accommodation and transport for selected participants from the Americas, but actively participates in the organization and preparation of publications that summarize the discussions held. Challenges 86. Understanding counterparts and the environment in which a technical cooperation program takes place is essential to achieving effectiveness and sustainability. Through discussions with the VDAP team, FIU-EEI was able to identify characteristics and particular situations that generate challenges for the program. These challenges are grouped into three categories: (1) infrastructure; (2) personnel, administrative and financial; and (3) international relations. 87. Infrastructure • Every observatory’s infrastructure is different and may require customized solutions to problems. • In many countries, deficiencies in basic infrastructure (internet and email access and security constraints) place limitations on the use, sharing, and storage of large volumes of data. • There is a lack of specialized equipment. “Poor access to the equipment that allows proper fulfillment of the training. For example, petrologic monitoring may require access to a scanning electron microscopy, or a high-functioning microscope, but access to such equipment may be rare or intermittent.” 88. Personnel, Administrative and Financial • The technical expertise that is available varies by partner organization. The staff in some organizations have greater capacity than in others. • The comments of VDAP experts included concern about staff turn-over: “[qualified personnel] switch to a different part of the organization,” or “they [qualified personnel] leave the organization to 48 seek a better-paying job,” and “In some countries, staff with good skills…often move on quickly to higher paying jobs in the private sector.” • Changes in policies and political appointees in the institutions that oversee the local observatories result in changes to the points of contact and program priorities, disrupting efficient communication, long-term stability, and progress in VDAP partner’s programs. • Counterparts have limited technical expertise in fields such as computer hardware and software, networking, field engineering, and project organization. A VDAP expert indicated: “VDAP has a software developer, but does not have other staff with extensive experience in systems management or networking to assist with non-software issues or in making recommendations regarding an observatory’s IT infrastructure.” • Many partners depend on VDAP to supply instrumentation and parts, but struggle to purchase spare parts and maintain instrumentation. A VDAP expert indicated: “Instruments for our discipline [in situ geochemical monitoring] are self-made, and while most counterparts possess the technical ability to use such instruments and to interpret data, obtaining spare parts and consumables can be very difficult, impossible, or prohibitively expensive (e.g. a bottle of calibration gas that costs $300 in the U.S. can cost 5-6 times that overseas, if it’s available at all)…Some partners are also reticent to ask for “too much” and instruments sometimes go unrepaired and unused as a result.” 89. International Relations • Political relationship between the U.S. and the country (e.g., Nicaragua). 90. Other challenges to providing long-term technical assistance are associated with VDAP’s internal circumstances. VDAP, USGS, and USAID may wish to consider these. These challenges are grouped into two categories: staff limitations and equipment, parts and software. 91. Staff limitations • VDAP geochemists are overwhelmed with requests for assistance. “VDAP currently only has two half-time geochemists on staff. We often can’t keep up with requests for assistance and are often stuck providing short-term fixes to systemic issues that would require longer time commitments to address. The geochemistry staff provides significant technical support, in engineering, IT, data analysis and interpretation.” • Some VDAP experts dedicate only half of their time to the program, limiting their ability to interact with partners or take on meaningful new areas of collaborations (e.g., assisting partners with publishing their work). 92. Equipment, parts, and software • Limited availability of instrumentation: VDAP has not been able to provide all instrumentation requested due to limited supply of custom parts and availability of staff for assembly. A VDAP expert indicated: “Instruments for measuring volcanic gases cannot be purchased commercially but are instead assembled from commercial and custom components by VDAP staff.” • Volcano geochemistry analysis software has several constraints. The software is not keeping up with state-of-the-art research in the field (the software is developed by researchers in their spare time). A VDAP expert indicated: “…it suffers from bugs, is often written in proprietary programming languages (e.g. MATLAB), and is not always intuitive for non-experts.” 93. The VDAP team has also worked to overcome the challenges identified. Following are some of the suggestions. 94. - Increasing professional capacity 49 • Hire more VDAP geochemists and computer scientists. • Hire a permanent system administrator. “This would allow our computer scientist to focus on software development while the system administrator can focus on other IT issues like hardware procurement, hardware donations, software install and support, managing VDAP servers, consulting partners on IT infrastructure issues, etc.” • To deal with the limited availability of instrumentation, VDAP would need to hire more technical staff, and attempt to contract with external companies for sourcing components and assembling instruments. One alternative is to take advantage of the existing capabilities in national volcano monitoring services to assemble instruments (e.g., Chile, Colombia and Costa Rica) [39, 52]. • To face software constraints, hire more VDAP computer scientists to assist VDAP research staff with scientific computing needs. Pursue contracts with external programmers when possible. 95. - Equipment, parts, and software • VDAP could provide some specialized equipment or could try and foster collaboration with other organizations who can help our partners gain access to the needed equipment. 96. - Infrastructure • Encourage large institutions to host data on servers and to build/maintain good web tools that allow remote users to analyze data and manipulate it on screen without the responsibility of hosting/storing large volumes of data. 97. If, as identified in this evaluation, it is proposed to expand the program´s scope by strengthening some services and expanding to new ones, it would be necessary to make a careful analysis of the levels of effort involved, as well as the need to incorporate new disciplines into the team. Conclusions All countries expressed satisfaction with the quality and level of technical support they receive from VDAP, whether through consulting, training of professionals, and scientific and/or technical knowledge in volcanic studies. This has allowed them to reach very high levels of technical competence. Along with the high-quality scientific and technical nature of the partnerships that VDAP has created, countries were deeply appreciative of the human component of VDAP’s collaboration. Countries commend the strong ties forged between the deeply helpful, patient, close, and considerate VDAP personnel and all partner agencies through the years. This is particularly meaningful because of VDAP’s respectful collaboration and treatment of participating countries as equals. Partner countries feel that these soft skills of genuine collaboration make for easy and fluid access to information and knowledge. Unlike other international actors, USGS/VDAP are not academically extractive in their practice, rather they guide country professionals to be independent and lead their own research and findings. The current evaluation confirms the effectiveness and relevance of the VDAP program, which has demonstrated its ability to adapt to the different needs of the countries, using different state-of-the￾art tools, and taking advantage of new developments in technology. Despite instability in some national agencies involved in volcanic monitoring, the reduced human and budgetary resources they are assigned, 50 and the turnover of in staffing in the 10 observatories in the nine countries, TET witnessed sustainability in the processes initiated by VDAP. From the professional development point of view, the effectiveness and relevance of the VDAP program was widely confirmed by the 92 technicians from the 16 countries surveyed. There are some VDAP services that can be improved and new initiatives that can be addressed (see details in the recommendation section), all of which require a careful analysis for feasibility and appropriateness so as to maintain the leadership and quality that has characterized the VDAP program since its inception. Recommendations Specific recommendations were included toward the end of the ‘response to the evaluation questions’ on pages 24-26. In this section, TET will highlight some of the study´s recommendations organized under three categories: VDAP´s M&E; VDAP's strategies and services, and VDAP's organization and team composition. Monitoring & Evaluation The recommendations to strengthen the existing VDAP M&E system include the design and implementation of: • A registration and tracking system for the beneficiaries of the different training and capacity building activities. • A registration and tracking system for VDAP equipment and instruments donated. • New reporting indicators on life and property safeguarded/loss of life and property avoided. Strategies and Services Regarding VDAP’s strategies and services, recommendations fall into four categories: (1) multi￾lateral collaboration; (2) learning opportunities; (3) exchanges; and (4) images databank and knowledge hub. Multi-lateral collaboration: Several countries suggest that VDAP, thanks to its global experience and convening capacity, can lead more open and robust multilateral and international collaborations, partnerships, and a multi-channeled network of ideas, information, technical knowledge, and consulting. Fostering an “e-gather” in response to volcano activity/unrest/events in any part of the world could trigger direct advice and trouble-shooting support at their time of need, and a learning environment for the volcanic scientific community (page 27). Learning Opportunities: Countries had useful and innovative suggestions for trainings and workshops to address knowledge and skills gaps. Suggestions included: embedded, real-world training, and the support in Higher Education. [19] Countries also suggested certain subject areas that could be strengthened or addressed via VDAP training, such as event trees; remote sensing; drone monitoring for volcanic surveillance, especially during eruptive crises events. [25, 33] 51 Exchanges: Countries had interesting and valuable suggestions for the nature and types of potential exchange with VDAP, extending onsite VDAP visits; increasing binational exchanges with other discipline networks; convening international groups during volcanic unrest events; exploring the possibility of a regional ‘collective’ of equipment, carrying ‘regional property tags,’ that can be dispatched to the countries in times of crisis. [55, 56] Databank of Images and Knowledge Hub: Countries suggest that VDAP should create a knowledge hub and a potential database of these multi-country image databanks for channelized, convenient, and direct access by all participating countries. [40] With sound procedures, policies, and coordinated management this be information and knowledge hub could benefit everyone. It is worth mentioning that certain countries, which are still in initial stages of developing their volcanological strengths, seek to expand and strengthen their capabilities and build a self-sufficient volcano agency, an ‘observatory.’ They express a need for support to build knowledge centers, acquire the ability to make high-profile threat and risk assessments, have a strong monitoring network, remote sensing units with ability to analyze the satellite images, and a strong forecast and communications unit. As of now, most of these elements are either missing or at the very initial stages of development in countries such as Guatemala, El Salvador, and Nicaragua. Organization and Team Composition Based on feedback provided by the VDAP team, there are limitations on VDAP’S ability to respond to the current demand for services, particularly regarding staff and equipment, parts and software. [91, 92]. If, as identified in this evaluation, it is proposed to expand the scope of VDAP by strengthening some services and/or developing new ones [94-97], it would be necessary to make a careful analysis of the levels of effort this would entail and enlarge the VDAP team accordingly, in both numbers and areas of expertise. 52 Annexes Contents Annex 1. Statement of Work .................................................................................................................. 53 1. Description of Program/Project/Activity to be Evaluated ...........................................................53 2. Background ..................................................................................................................................53 3. Purpose ........................................................................................................................................53 4. Questions.....................................................................................................................................54 5. Methods.......................................................................................................................................54 6. Deliverables and Timeline............................................................................................................55 7. Team Composition .......................................................................................................................55 8. Scheduling and Logistics...............................................................................................................57 Annex 1a. Evaluation Questions and Sub-Questions...........................................................................58 Annex 1b. Evaluation Questions - Implications for Program Evaluation..............................................60 Annex 1c. Evaluation Mapping.............................................................................................................64 Annex II: Evaluation Methods – Survey Results ....................................................................................... 66 Results by Country ................................................................................................................................66 Age Group .............................................................................................................................................66 By Sex ....................................................................................................................................................67 Undergraduate Studies.........................................................................................................................67 Graduate Degree...................................................................................................................................67 The composite KAP Index......................................................................................................................69 Annex III: Data Collection Instruments.................................................................................................... 70 VDAP 2020 Evaluation – Qualtrics Survey Questionnaire.....................................................................70 Questionnaire for MDROs.....................................................................................................................81 Questionnaire for VDAP Representatives.............................................................................................82 Protocols for Interviews with Heads of Volcano Observatories ...........................................................83 VDAP Remote Field Visit .......................................................................................................................85 Annex IV. Sources of Information ........................................................................................................... 93 Reported Use of VDAP Tools..........................................................................................................................93 Training Portfolio Training Portfolio -- CSAV......................................................................................................................93 Training Portfolio – Probabilistic Event Trees.......................................................................................95 VDAP Activities Related to NOVAC........................................................................................................98 Threat Assessment and Gap Analysis (TAGA) Training .......................................................................101 Seismic Analysis and Interpretation Trainings....................................................................................103 VDAP Binational Exchanges FY 2018 – FY 2020............................................................................................105 Software VDAP-Provided Software and Observatory Decisions –dMODEL .......................................................110 VDAP-Provided Software and Observatory Decisions – NOVAC Software .........................................112 VDAP-Provided Software and Observatory Decisions – Swarm..........................................................113 VDAP-Provided Software and Observatory Decisions – Valve............................................................114 VDAP Responses FY 2018 – FY 2020.............................................................................................................115 List of Persons Interviewed ..........................................................................................................................122 National Agencies Leading Volcano Monitoring Activities..................................................................123 VDAP Donated Instrumentation FY 2018‐2020 ............................................................................................125 References....................................................................................................................................................126 Annex V: Disclosure of Conflicts of Interest........................................................................................... 127 53 Annex 1. Statement of Work U.S. Agency for International Development, Bureau for Humanitarian Assistance (BHA) Florida International University - Extreme Events Institute Volcano Disaster Assistance Program (VDAP) 2020 Mid-Term Evaluation Statement of Work 1. Description of Program/Project/Activity to be Evaluated FIU’s current proposal responds to the need to conduct a mid-term program performance evaluation of The United States Geological Survey’s Volcano Disaster Assistance Program (VDAP). The goal is to improve USGS/USAID-BHA’s understanding of the performance and outcomes of the VDAP program at the global level, prioritizing the countries benefitted between FY2018 and FY2020, and focusing on the program’s effectiveness, relevance, and sustainability. 2. Background The Volcano Disaster Assistance Program (VDAP) is a joint USGS/USAID-BHA program designed to assist foreign governments to mitigate the effects of volcanic unrest and eruptions. The primary goal of the VDAP is to reduce loss of life and economic damage in countries that experience volcanic eruptions. Under the current USGS/USAID-BHA agreement, VDAP needs to comply with a mid-term program performance evaluation. The VDAP was established in 1986 after the devastating volcano-induced mudflow that destroyed the city of Armero, Colombia on November 13, 1985, killing more than 25,000 people. This tragic event along with a few others in the 1980s, demonstrated the vulnerabilities of the increasing populations exposed to volcanic hazards in developing countries worldwide. Since 1986, VDAP has worked in multiple countries and regions to (a) strengthen volcano monitoring capacities and (b) integrate mitigation and response components through technology transfer, training, and rapid response to volcano unrest and eruptions. The VDAP program is headquartered at the USGS Cascades Volcano Observatory in Vancouver, Washington, and has a team of approximately 20 geologists, geophysicists, and engineers. The VDAP program contributes decisively to the fulfillment of the priority actions proposed by the Sendai Framework for Disaster Risk Reduction (SFDRR) 2015–2030: (1) Understanding disaster risk; (2) Strengthening disaster risk governance to manage disaster risk; (3) Investing in disaster risk reduction for resilience; and (4) Enhancing disaster preparedness for effective response. The VDAP program is mainly associated with SFDRR priority actions 1 and 4. 3. Purpose FIU uses the Theory of Change to define the overarching evaluation goal: An up-to-date programmatic mid-term evaluation to provide USGS/BHA with better information about VDAP’s performance and outcomes at the global level, focusing on the program’s effectiveness, relevance, and sustainability (See Figure 1). This evaluation will prioritize the countries benefitted by VDAP program between FY2018 and FY2020, and will inform VDAP’s decisions on strategies, activities, and future funding levels. As a contribution to this overarching goal, FIU’s Disaster Risk Reduction Laboratory will conduct a mid-term program performance evaluation of the VDAP. 54 4. Questions VDAP-USGS prepared five evaluation questions, with 24 sub-questions (See Annex 1 Page 7). The five evaluation questions are: 1. What is the effectiveness of tools* provided by VDAP on improving volcano observatories’ ability to monitor their volcanoes and forecast eruptions? (*tools = data, software, databases, and monitoring equipment) 2. How effective have VDAP’s trainings, workshops, and exchanges been in preparing counterparts at volcano observatories to monitor their volcanoes and forecast eruptions? 3. To what extent has stakeholder* effectiveness improved due to VDAP programming? (*stakeholders= volcano observatory staff, at-risk populations, affected governments, USGS, USAID/BHA, and the USG) 4. To what extent has stakeholder* coordination and leveraging improved due to VDAP programming? 5. What are the recommendations for future VDAP programming? For the purpose of this evaluation, ‘effective’ is the degree to which something is successful in producing a desired result, and ‘effectiveness’ is the degree to which objectives are achieved and the extent to which targeted problems are solved, in other words ‘doing the right thing.’ 5. Methods To address VDAP’s evaluation questions, the research design would begin with an extensive literature review, followed by a mixed research method, including qualitative approaches such as interviews, case studies, and document reviews, and quantitative approaches like surveys conducted in the selected countries to gather primary data (more details in Annex 2 Page 8), following an Institutional Review Board (IRB)-approved research protocol. Finally, using an integrative process—triangulation—the evaluation would analyze the data from different perspectives and theoretical positions (i.e., final recipients of the capacity building initiatives, national geological services, VDAP members, and other stakeholders). The preparation of the protocols for interviews and field visits, as well as the survey questionnaire, include a review by VDAP-USGS and USAID / BHA. Pretesting will be carried out to assure validity of the questionnaire, estimate the time required, and avoid problems for interviewers or respondents. Semi-structured interviews will be conducted to cover the 5 evaluation questions. Having open￾ended questions will allow for a discussion with the interviewee rather than a straightforward question and answer format. It is likely that the interviewee will cover some of the secondary questions in their answers, but if this does not happen the interviewer should include them. The interview’s output will be processed using the technique called ‘thematic analysis’ (Braun & Clarke, 2006). This technique consists of six phases: 1) familiarization with the data and identification of emerging topics; 2) coding the data; 3) searching for themes, identifying common patterns and categories among the selected codes; 4) reviewing themes; 5) defining and naming themes; 6) writing up the code book. The excerpts in the code book will be analyzed and summarized in Labels or Key points. The learning component of the VDAP's capacity development strategy will use the Knowledge, Attitudes and Practices (KAP) method in the survey design. A KAP survey allows the researcher to go beyond just knowledge retention to characterize the population using VDAP training, collecting information on what is known, believed and done in relation to the different topics addressed. The survey will be online, using the Qualtrics platform. The invitation to participate in the survey will be made via email, using the VDAP’s training and exchange participant lists from the last two years (exact date to be established with VDAP-USGS and USAID / BHA). Prior to sending the emails inviting survey responses, FIU will seek affiliate institutions’ endorsements in addition to the endorsement by VDAP and USAID / BHA, to ensure the greatest possible participation. There will be a section in the survey about crisis response that will be carried 55 out in all 14 countries because all of them have experienced confirmed eruptions in the last 25 months (See Evaluation Mapping Annex 3 Page 14). Additionally, FIU will apply the competence-based approach to analyze the different volcano observatory positions, including job requirements, job descriptions, and performance evaluation. The outputs of this approach will be compared with existing entry level profiles, on-the-job training provided (by VDAP and other sources) in order to determine the appropriateness of trainings and tools provided by VDAP. Most likely, the results will vary from one country to another, given the different modalities of higher education and the differences in work experience in the region. The competence-based approach will be complemented with an analysis of the existing volcano observatory equipment, its preventive and corrective maintenance, as well as its replacement /renewal plans. Knowing the current status of the observatories’ capabilities including personnel, standard procedures, facilities, equipment, technologies, and administration, will inform future assistance decisions. Pilot Country FIU will pilot the evaluation design and tools in a LAC country. Based on the existing capabilities, FIU suggests one of the following two countries: Colombia, or Costa Rica. Both countries have active volcanic observatories, institutional support, and unrestricted access. The intention is to test the different evaluation techniques and tools. Likewise, we want to compare results between in-person interviews and visits to observatories, with remote interaction using audiovisual media, thinking of an alternative for countries where there may be restricted access (e.g., COVID19) or other security limitations. 6. Deliverables and Timeline Activities & Deliverables Aug￾2020 Sep￾2020 Oct￾2020 Nov￾2020 Dec￾2020 Jan￾2021 Feb￾2021 Mar￾2021 Apr￾2021 May￾2021 1. Evaluation design 2. IRB approval. (Compliance with quality, ethics, and data protection) 3. In-briefing – Kick-off meeting Oct 21st 4. Literature review 5. Field work, Interviews, Survey 6. Mid-term briefing Dec 20th 7. Out-briefing Mar 27th 8. Draft Evaluation Report Apr 27th 9. Presentation of Evaluation Report May 20th 10. Submission of Final ER and Electronic Copies of Raw Data May 31st 7. Team Composition Roles and Responsibilities The evaluation will be led and implemented by FIU with participation of members of VDAP-USGS and USAID / BHA. The following table summarizes the roles and responsibilities of the institutions involved in the different stages of the evaluation. FIU VDAP-USGS USAID/BHA 1. Evaluation design Prepare design & tools Provide feedback Provide feedback 2. IRB approval. Prepare protocols and request approval/exemption. Obtain IRB credentials Obtain IRB credentials if the representative leads surveys, processes or analyses data Obtain IRB credentials if the representative leads surveys, processes or analyses data 56 FIU VDAP-USGS USAID/BHA 3. In-briefing – Kick-off meeting Active participation Active participation Active participation 4. Literature review Conduct literature review N/A N/A 5. Field work, interviews, survey Conduct field work Participation encouraged Participation encouraged 6. Mid-term briefing Prepare and present Active participation Active participation 7. Out-briefing Prepare and present Active participation Active participation 8. Draft Evaluation Report Prepare and submit Provide feedback Provide feedback 9. Presentation of Evaluation Report Prepare and present Provide feedback Provide feedback 10. Submission of Final ER and electronic copies of raw data Prepare and submit N/A N/A FIU Team The evaluation team will consist of FIU team members, internationally recognized experts from the Americas region, and representatives from USAID/BHA-M&E and USAID/BHA Preparation, Strategic Planning, and Mitigation (PSPM). A specialist for the EAP region will be hired as a consultant and integrated into the team to conduct field work and interviews in that region. This specialist will be selected jointly by USGS￾VDAP, USAID/BHA, and FIU at a later date. FIU has included a consultant position in the study’s budget. The Bios of the FIU team and international experts are as follows: Juan Pablo Sarmiento, M.D., M.P.H. Dr. Juan Pablo Sarmiento is a Research Professor and Associate Director for Research at FIU’s Extreme Events Institute. He is also the Director of the Disaster Risk and Resilience in the Americas program, funded by USAID/BHA. He is a Medical Doctor and Surgeon (U. Rosario, Colombia) with a M.Sc. in Public Health, Specialty in Health Promotion and Social Development (U. Bordeaux, France & UPN, Spain), and has an M.A. in Project Management (UCI, Costa Rica). He has a Specialization Degree in Medical Education (U. Sabana, Colombia). Dr. Sarmiento has also post-graduate studies in Disaster Management (Oxford Brookes U., U.K), High Level Public Administration (ESAP-Colombia), and a residence in Nutrition (Tufts U., U.S.A.). He has more than three decades of professional experience in evidence-based research/performance evaluation, health, and risk and disaster management. He has worked as consultant for the Pan American Health Organization, and as Technical Manager of the regional risk management program, implemented by IRG in Latin American and the Caribbean for USAID/BHA-LAC. Dr. Sarmiento has also been involved in disaster risk advisory work in Asia (Nepal) and the South Pacific region. Recently, he was appointed as member of the World Meteorological Organization's Working Group on Societal and Economic Research Applications (WG SERA). Alvaro Amigo, Ph.D. Dr. Amigo is the Director of the Chilean Network for Volcanic Surveillance, Sernageomin. He is an elected member of the Chilean Geological Society Executive Committee. He holds a PhD degree in Earth Sciences (U. of Bristol, U.K.) funded by the Dorothy Hodgkin Postgraduate Award. He has also a degree as Geologist (U. of Chile, Chile) and a BSc. (U. of Chile, Chile). His research focuses on improving the understanding of recent volcanic activity along the Central, Southern and Austral Volcanic Zones of the Andes and the application of volcanology concerning hazards, risk and communication with stakeholders. Dr Amigo has provided technical support in volcano monitoring and hazards assessments to Guatemala (2018-present) and Nicaragua (2016-2017) under the Chile’s Ministry of Foreign Affairs. He was also the Co-coordinator of the 2015 “United States-Chile Binational Exchange for Volcanic Risk Reduction” supported by USAID-BHA. He was the PI of the Fondecyt research 11130671 “The influence of unloading of ice and glacial advance/retreats on explosive volcanism in Northern Patagonia” (2013 – 2017); and Associate Researcher in the Fondap Grant 15090013 “Andean Geothermal Center of Excellence” (2012–present), NSF-IES Grant 1411779 “Collaborative Research: Dynamics of caldera-scale rhyolitic magma systems” (2013–2017), and Cyted Grant 410RT0492 “Iberoamerican network for modeling and monitoring volcanic ash and aerosols, and their impact in the infrastructure and air quality” (2010– 2012). 57 Carlos Cardona, Ph.D. Dr. Cardona is the coordinator of the Volcano Seismology team at the Chilean Volcano Observatory, Sernageomin. He was the head of the Chilean Volcano Observatory, between 2015-2016, and the scientist coordinator of the Popayan Volcano Observatory of the Colombian Geological Service, between 2007-2010. He holds a PhD degree in Earth Sciences (U. of Concepción, Chile), he has also a degree as Geologist (U. of Caldas, Colombia), his research focuses in the seismic behavior of the rhyolitic volcanic system and geophysical conceptual model of the magma plumbing systems. Dr. Cardona has provided technical support in volcano monitoring and seismology observations to Peru (2017). He was involved in the NSF-IES Grant 1411779 “Collaborative Research: Dynamics of caldera-scale rhyolitic magma systems” (2013–2018). He has worked in volcano monitoring for 25 years, and he has participated into the professional teams that did the scientist management of scientific teams that handled more than 15 volcano crises in Colombia (including Nevado del Ruiz, Nevado del Huila and Galeras volcanoes) and Chile (including Cordón Caulle, Villarrica, Calbuco, Copahue and Nevados del Chillán volcanoes, and others minors eruptions) as senior volcano seismologist. Meenakshi Chabba, M.S. Meenakshi Chabba is Coordinator of Research Programs at the Extreme Events Institute in Florida International University (FIU). Meenakshi obtained her Master of Science in Environmental Studies (2012) from FIU with specialization in Environmental Economics, Environmental Policy, Resource Management, and Geographic Information Systems. She has a M.Sc. and B.Sc. in Zoology, and a Bachelor’s in Science Education from the U. Delhi, India. Meenakshi’s research interests include ecosystem services valuation, environment resource management, benefit cost analysis of disaster risk reduction, and sustainable development. Her master’s thesis focused on the quantification and valuation of carbon storage by the mangrove forests in Everglades National Park, Florida, and it was recently published in the Environment Science & Policy Journal. Her research has also been published in the International Journal of Mass Emergencies and Disasters, Ad￾Minister, and the International Journal of Ecological Economics and Statistics. As part of FIU’s DRR team, Meenakshi has designed and developed cost benefit analysis methodologies for academic and practical applications. She is part of the adjunct faculty for the Earth and Environment Department at FIU and her teaching interests include Ecology of South Florida, Global Environment and Society, and Introduction to Environment Sciences. 8. Scheduling and Logistics Due to the constantly changing crisis of COVID 19, the planned schedule will be revisited once information is received on the award’s approval by USAID. The Safety & Security Plan will guide the review of the situation and the appropriate measures to take. The aspects related to staffing, travel arrangements, and communications will be centered at FIU-EEI, under the responsibility of the project coordinator, Juan Pablo Sarmiento. These procedures will be governed by the agreement between USAID and FIU, and the regulations that rule FIU as a public university of the State of Florida. 58 Annex 1a. Evaluation Questions and Sub-Questions Evaluation Questions and Sub-Questions FIU: Volcano Disaster Assistance Program (VDAP) 2020 Mid-Term Evaluation Evaluation Questions 1. What is the effectiveness of tools provided by VDAP in terms of improving the volcano observatories’ ability to monitor their volcanoes and forecast eruptions (tools include data, software, databases, and monitoring equipment)? • Which VDAP tools are most useful to observatories? (Which VDAP tools are used? How are they used? Who uses them?) • How have threat assessment and gap analysis (TAGA) been used to guide observatory operations and investigations? • Which VDAP partners, who received event tree training from VDAP, are using event trees on their own? How has the event tree process been used to assess and communicate risk and aid forecasts? • In what ways do periodic satellite data observations and sketches assist in observatory decisions? What remote sensing data is most useful (e.g., paragraph form from Reston or jpgs)? • How does VDAP-provided software affect observatory operations? How has VDAP-provided software spurred homegrown software? How many partners are using software VDAP has provided for monitoring? Which software packages are they using? • How effectively has VDAP-donated equipment been used at volcanoes that have been active to provide data for decision making? 2. How effective have VDAP’s trainings, workshops, and exchanges been in preparing counterparts at volcano observatories to monitor their volcanoes and forecast eruptions? • Is the current format for conducting trainings, workshops, and exchanges effective (e.g., are regional workshops as effective as observatory-based training sessions, are general or specific trainings more effective, etc.). Are any changes required? • Which trainings, workshops, and exchanges are most useful to participants? How important a role does CSAV and other VDAP training play in the human resources development of volcano observatory staff around the world? • How have countries participating in binational exchanges used the knowledge transferred during the exchanges to further disaster risk reduction (DRR) in their countries? • Did VDAP events ensure the involvement of women and assist men and women equally? 3. To what extent has stakeholder effectiveness improved due to VDAP programming (stakeholders include volcano observatory staff, at-risk populations, affected governments, USGS, USAID/BHA, and the USG)? • Has VDAP assistance led to a reduction in the loss of lives and property due to volcanic eruptions? 59 • Has the ability of government authorities to forecast and respond to eruptions improved as a result of VDAP assistance? • What is the impact of VDAP on domestic USGS programs? • Has VDAP assistance promoted science diplomacy? 4. To what extent has stakeholder coordination and leveraging improved due to VDAP programming? • Has VDAP material assistance been leveraged by host countries to obtain more support from national or foreign donor sources? • Were VDAP activities well-coordinated with other donors to avoid duplication of effort? • How does VDAP facilitate collaboration/cooperation within/between countries? What binational or multi-agency ‘agreements’ have been fostered by VDAP? • How has partnering with VDAP fostered the development and operations of counterpart agencies over the last 2.5 years? 5. What are the recommendations for future VDAP programming? • What changes should be made regarding how VDAP makes forecasts and/or communicates information about them to improve effectiveness? • Does VDAP engage in outreach at appropriate levels? Is the type of outreach appropriate (social media, publications, USGS outreach products, website, factsheets)? • Is reporting timely (e.g., IVANS, and daily updates)? • Should VDAP develop new tools for partners? Are there tools that partners want that VDAP doesn’t provide? • Should VDAP make any changes to the way the program is monitored or evaluated? • Are there any recommendations, concerns, or feedback counterparts would like to provide? 60 Annex 1b. Evaluation Questions - Implications for Program Evaluation Question Stakeholder Involved - Location Method to gather data/information Evaluation Team Expertise needed Comments 1. What is the effectiveness of tools* provided by VDAP on improving volcano observatories’ ability to monitor their volcanoes and forecast eruptions? *tools= data, software, databases, and monitoring equipment NGS (National Geological Service) – Volcano monitoring team members Local Universities Other Central Level and volcano observatories Interviews covering the subsets mentioned in the comments’ column. Capacity building expert /volcanologist Familiarity with the topics mentioned in the comments’ column. Knowledge about: • Suite of VDAP tools available for observatories • Threat Assessment and Gap Analysis (TAGA) • Event tree process & risk communication • Subset satellite data, observations, and sketches for observatory decisions • VDAP-provided software and observatory operations • VDAP-donated equipment and its functionality for decision making • Which VDAP tools are most useful to observatories? (Which VDAP tools are used? How are they used? Who uses them?) NGS (National Geological Service) – Volcano monitoring team members Local Universities Other Central Level and volcano observatories Interviews – Subset VDAP tools Capacity building expert /volcanologist (familiarity with suit of VDAP tools) Knowledge about the suite of VDAP tools available for observatories • How have threat assessment and gap analysis (TAGAs) been used to guide observatory operations and investigations? Interviews – Subset TAGAs Capacity building expert/volcanologist (familiarity with TAGAs) Knowledge about threat assessment and gap analysis (TAGAs) • Which VDAP partners, who received event tree training from VDAP, are using event trees on their own? How has the event tree process been used to assess and communicate risk and aid forecasts? Interviews – Subset event tree process & risk communication Capacity building expert/volcanologist (familiarity with event tree process & risk communication) Knowledge about event tree process & risk communication • In what ways do periodic satellite data observations and sketches assist in observatory decisions? What remote sensing data is most useful (e.g., paragraph form from Reston or jpgs)? Interviews – Subset satellite data observations and sketches for observatory decisions Capacity building expert/volcanologist (familiarity Subset satellite data observations and sketches for observatory decisions) Knowledge about Subset satellite data observations and sketches for observatory decisions 61 • How does VDAP-provided software affect observatory operations? How has VDAP provided software spurred homegrown software? How many partners are using software that VDAP has provided for monitoring? -- Which software packages are these? • Interviews – Subset VDAP￾provided software and observatory operations Capacity building expert/volcanologist (familiarity VDAP-provided software and observatory operations) Knowledge about VDAP-provided software and observatory operations. • How effectively has VDAP-donated equipment been used at volcanoes that have been active to provide data for decision making? • Interviews – Subset VDAP￾donated equipment; its functionality for decision making Capacity building expert/volcanologist (familiarity VDAP-donated equipment; its functionality for decision making) Knowledge about VDAP-donated equipment and its functionality for decision making 2. How effective have VDAP’s trainings, workshops, and exchanges been in preparing counterparts at volcano observatories to monitor their volcanoes and forecast eruptions? NGS (National Geological Service) – Volcano monitoring team members Local Universities Other Central Level and volcano observatories • Interviews with decision makers & managers– Subset Training • Survey targeting different training beneficiaries. • Observation – Ongoing training • Statistical analysis of existing VDAP training / TraiNet data • Review of institutional reports Capacity building expert/volcanologist (familiarity with VDAP training, CSAV, exchanges, others). Their input is critical for evaluation tools preparation Knowledge about: • VDAP training modalities (regional workshops, observatory-based training) • VDAP capacity building activities (VDAP training, CSAV, exchanges, others) • Binational exchanges organized by VDAP Access to VDAP training statistics / TraiNet data • Is the current format for conducting trainings, workshops, and exchanges effective (e.g., Are regional workshops as effective as observatory-based training sessions, are general or specific trainings more effective, etc.)? Should there be any changes? NGS (National Geological Service) – Volcano monitoring team members Local Universities Other Central Level and volcano observatories • Interviews with decision makers & managers– Subset Training • Survey targeting different training beneficiaries. • Observation – Ongoing training • Review of institutional reports Capacity building expert Knowledge about VDAP training modalities (regional workshops, observatory-based training) • Which trainings, workshops, and exchanges are the most useful to participants? How important a role does CSAV and other VDAP training play in the human resource development of volcano observatory staff around the world? • Interviews with decision makers & managers – Subset Training • Survey targeting different training beneficiaries. • Review of institutional reports Capacity building expert/volcanologist (familiarity with VDAP training, CSAV, exchanges, others). Their input is critical for evaluation tools preparation Knowledge about VDAP capacity building activities (VDAP training, CSAV, exchanges, others) • How have countries participating in binational exchanges used the knowledge transferred during the exchanges to further Disaster Risk Reduction (DRR) in their countries? • Interviews with decision makers & managers – Subset Training • Survey targeting different training beneficiaries. • Review of institutional reports Capacity building expert/volcanologist (familiarity with binational exchanges organized by VDAP). Their input is critical for preparation of evaluation tools. Knowledge about binational exchanges organized by VDAP 62 • Did VDAP events ensure the involvement of women and assist men and women equally? • Statistical analysis of existing VDAP training / TraiNet data Statistics Access to VDAP training statistics / TraiNet data 3. To what extent has stakeholder* effectiveness improved due to VDAP programming? *stakeholders= volcano observatory staff, at-risk populations, affected governments, USGS, USAID/BHA, and the USG National Emergency Office NGS (National Geological Service) – Volcano monitoring team members Local Universities Other Central Level and volcano observatories VDAP team USGS Department Emergency Management (States) • Interviews • Review of institutional reports • Case study preparation (if cases are found) Program evaluator /volcanologist. Their input is critical for evaluation tools preparation • Agree with USAID on the time-period to be considered • Anticipate event list for the countries selected • Agree with USAID on the scope of this activity because of using USAID funds • Revise science diplomacy approaches • Has VDAP assistance lead to a reduction of lives and property lost from volcanic eruptions? • Interviews with decision makers – Subset EWS impact • Review of institutional reports Program evaluator /volcanologist. Their input is critical for evaluation tools preparation Agree with USAID on the time-period to be considered Anticipate event list for the countries selected • Has the ability of the government authorities to forecast and respond to eruptions improved as a result of VDAP assistance? National Emergency Office NGS (National Geological Service) – Volcano monitoring team members Local Universities Other Central Level and volcano observatories • Interviews with decision makers – Subset EWS impact • Review of institutional reports Program evaluator Agree with USAID on the time-period to be considered Anticipate event list for the countries selected What is the impact of VDAP on domestic USGS programs? VDAP team USGS State Department Managers • Interviews – Subset EWS domestic impact • Review of institutional reports Program evaluator Agree with USAID on the scope of this activity because of using USAID funds • Has VDAP assistance promoted science diplomacy? VDAP team National Emergency Office NGS (National Geological Service) • Interviews – Subset EWS impact Science diplomacy • Review of institutional reports Program evaluator Revise science diplomacy approaches 4. To what extent has stakeholder* coordination and leveraging improved due to VDAP programming? VDAP team National Emergency Office NGS (National Geological Service) Volcano monitoring team • Interviews • Review of institutional reports Program evaluator Agree with USAID on the time-period to be considered Identify with VDAP other possible donors to contact • Has VDAP material assistance been leveraged by host countries to obtain more support from National or foreign donor sources? VDAP team NGS (National Geological Service) • Interviews – Subset coordination and leveraging impact • Review of institutional reports Program evaluator Agree with USAID on the time-period to be considered Anticipate case list for the countries selected 63 • Were VDAP activities well￾coordinated with other donors to avoid duplication of effort? VDAP team NGS (National Geological Service) VDAP team NGS (National Geological Service) • Interviews – Subset coordination and leveraging impact • Review of institutional reports Program evaluator Identify with VDAP other possible donors to contact • How does VDAP facilitate collaboration/cooperation within/between countries? What binational or multi-agency "agreements" fostered by VDAP? • Interviews – Subset coordination/leveraging impact • Review of institutional reports • Analysis of institutional agreements Program evaluator Agree with USAID on the time-period to be considered Anticipate agreement list for the countries selected • How has partnering with VDAP fostered the development and operation of counterpart agencies over the last 2.5 years? VDAP team National Emergency Office NGS Volcano monitoring team • Interviews – Subset coordination and leveraging impact • Review of institutional reports Program evaluator 5. What are the recommendations for future VDAP programming? VDAP team National Emergency Office NGS (National Geological Service) Volcano monitoring team • Interviews • Review of institutional reports Volcanologist Program evaluator Evaluation Team Access to outreach material (social media, publications, USGS outreach products, website, factsheets) • What changes should be made for how VDAP does forecasts or communicates them to improve effectiveness? VDAP team National Emergency Office NGS Volcano monitoring team • Interviews – Subset recommendations - forecasts/communication • Review of institutional reports Program evaluator / volcanologist. Their input is critical for evaluation tools preparation and the result analysis • Is outreach about VDAP done at appropriate levels? Is the type of outreach appropriate (social media, publications, USGS outreach products, website, factsheets)? • Interviews – Subset recommendations - outreach • Review of institutional reports Program evaluator Access to outreach material (social media, publications, USGS outreach products, website, factsheets) • Is reporting timely (e.g., IVANS, and daily updates)? VDAP team NGS • Interviews – Subset recommendations - reporting Volcanologist The questionnaire must include data quality, timing, and the existence of protocols. • Should VDAP develop new tools for partners? Are there tools that partners want that VDAP doesn’t provide? VDAP team NGS Volcano monitoring team • Interviews – Subset recommendations – new tools • Institutional reports Volcanologist • Should VDAP make any changes to the way the program is monitored or evaluated? VDAP team • Interviews – Subset recommendations – M&E • Institutional reports Program evaluator • Are there any recommendations, concerns, or feedback counterparts would like to provide? VDAP team National Emergency Office NGS Volcano monitoring team • Interviews – Subset recommendations – other feedback • Institutional reports Evaluation Team 64 Annex 1c. Evaluation Mapping Based on the VDAP Work Plan FY 2020 and the VDAP Country Profiles, those countries that have benefited most from VDAP actions in the last two years were identified to form Group 1 countries. Subsequently, countries other than Group 1 that benefited from training workshops or exchange visits between countries were identified to form Group 2. Finally, those countries remaining in the list, not included in Groups 1 and 2, were included in Group 3. Group Countries Group 1 Indonesia, Guatemala, Peru, Colombia, Ecuador, Argentina, Chile, Costa Rica and Mexico Group 2 PNG, El Salvador, DRC, Philippines Group 3 Nicaragua The mapping evaluation table describes how field visits and interviews will focus on Group 1. The Capacity Development Impact Survey, and the Training & Exchange Statistical Analysis will target Groups 1 and 2. Institutional References Review, Capacity Development Needs Survey and the Crisis Response Survey will target Groups 1, 2, and 3. High Priority Countries Country Interviews Field visit (Observatories) Survey Capacity Develop. Impact Institutional References Review Statistical Analysis Training, exchange Survey Capacity Develop. Needs Survey Crisis Response* Indonesia X X X X X X X PNG X X X X X Guatemala X X X X X X X Nicaragua X X X Peru X X X X X X X Medium Priority Countries Colombia X X X X X X X Ecuador X X X X X X X El Salvador X X X X X Low Priority Countries Argentina X X X X X X Chile X X X X X X X Costa Rica X X X X X X X 65 Country Interviews Field visit (Observatories) Survey Capacity Develop. Impact Institutional References Review Statistical Analysis Training, exchange Survey Capacity Develop. Needs Survey Crisis Response* DRC X X X X X Philippines X X X X X Mexico X X X X X X PNG - Papua New Guinea; DRC-Democratic Republic of the Congo. *Smithsonian Global Volcanism Program (See next page) Under this strategy, FIU seeks to prioritize evaluation in those countries that have benefited most from VDAP actions in the last two years but extends the needs analysis and the recent history of crisis response to all the countries on the list of VDAP cooperation. We consider this approach to be consistent with a Midterm evaluation, extending the scope of some evaluation topics to other countries to inform USGS-VDAP's future decisions. 66 Annex II: Evaluation Methods – Survey Results Details on the demographics of the survey respondents are included in this annex, as well as additional information that was partly used in answering some of the evaluation questions of the present study. From a total of 209 trainee records, TET identified 176 individuals as several people have received more than one training. Of these 176, 7 addresses were no longer in use, which led to a final target population of 169 individuals who were invited to complete the online survey. As an additional output, a consolidated database of VDAP trained individuals was generated. The survey questionnaire with 33 questions was designed according to USAID-KAP Survey Model, focusing on knowledge, attitudes, and practice/implementation. A total of 92 valid surveys from 16 countries were received (Table 1), with a 54.4% response rate. This is a good response rate, considering that the average response rate obtained in other continuing education and professional development programs is 40%. The average NSSE institutional response rate for these six institutions was about 40% (ranging from 35% to 69%), which is consistent with most NSSE institutions (Lambert et al. 2014). Results by Country Country Targeted N Survey Respondents Response Rate % Argentina 8 6 75 Cameroon 1 1 100 Chile 15 8 53 Colombia 15 6 40 Democratic R. of Congo 3 1 33 Costa Rica 9 4 44 Ecuador 13 8 62 El Salvador 15 7 47 Guatemala 14 6 43 Indonesia 17 9 53 Mexico 6 5 83 Nicaragua 14 5 36 Papua New Guinea 3 1 33 Peru 28 17 61 Philippines 7 7 100 Singapore 1 1 100 Total 169 92 54.4 Age Group Age Group No. Survey Respondents % 20-35 42 46.2% 36-50 33 36.3% 51-65 16 17.6% 91 100% 67 By Sex Gender No. Survey Respondents % M 57 62.0% F 35 38.0% 92 100% Undergraduate Studies Undergraduate studies No. Survey Respondents Civil Engineer 7 Geologist 19 Geophysics 6 Geophysical Engineer 9 Chemistry 9 Electronics Engineer 6 Geological Engineer 11 Physics 6 Other (Geodesy Engineer, Environmental Science, IT, Industrial Engineer, Engineer Surveyor, Petroleum engineer, Surveyor, Biologist, technician) 19 Graduate Degree Graduate studies N Survey Respondents Master 35 PhD 9 Specialization 3 Other 6 Without 7 60 Have you participated in the International Training Course at the Center for the Study of Active Volcanoes (CSAV), Hawaii? Cohort No. Survey Respondents Cohort No. Survey Respondents 2019 10 2013 2 2018 9 2012 2 2017 2 2011 2 2016 6 2000-2009 4 2015 4 <1999 4 2014 2 TOTAL 47 68 Have you received training or technical assistance from VDAP (other than CSAV) in any of these areas? Type of training or technical assistance provided by VDAP N Survey Respondents a) Seismology 38 b) Acoustic sensors 7 c) Geochemistry 8 d) Deformation 4 e) Geology 31 f) Monitoring of surface activity and morphological changes 20 g) Strainmeter h) Magnetometers i) Gravimetry / Microgravimetry 3 j) Electrical potential k) Borehole instrumentation 2 l) Others 20 Have you received training or technical assistance from VDAP for the use of the following software? Software, training or technical assistance provided by VDAP N Survey Respondents dMODELS 67 Swarm 33 NOVAC Software 17 gTOOLS 5 Válve 5 In fulfilling your role with your country's volcanic monitoring team over the past year, indicate the percentage (%) of your time spent in the following areas Areas where survey respondents spent their time (%) N Survey Respondents 81-100% 61-80% 41-60% 21-40% 1-20% 0 Seismology 6 8 2 8 31 37 Geodesy 4 8 5 5 15 55 Physical Vulcanology 2 8 9 6 24 43 Geochemistry 3 6 4 7 16 56 Hydrology 0 1 0 1 9 81 Remote Sensing 2 2 5 7 28 48 Computer Systems Engineering 2 0 2 5 16 67 Electronics - Communications Engineering 2 3 2 3 4 78 Community Outreach 1 3 1 3 25 59 Risk Communication 1 3 1 3 38 46 Work with risk and emergency management organizations 1 3 1 0 35 52 Administration (Planning, Human Resources, Finance, Logistics, Reporting) 1 1 0 3 5 32 69 A significant number of respondents report a greater dedication of time to seismology, geodesy, physical vulcanology, and geochemistry. Area % of Dedication Hydrology 81 Electronics - Communications Engineering 78 Computer Systems Engineering 67 Community Outreach 59 Geochemistry 56 Geodesy 55 Work with risk and emergency management organizations 52 Remote Sensing 48 Risk Communication 46 Physical Vulcanology 43 Seismology 37 Administration (Planning, Human Resources, Finance, Logistics, Reporting) 32 A smaller number of people report activity in the areas of hydrology (19%), electronics- communications engineering (22%), computer systems engineering (33%), and community outreach (41%). Administration is the common activity reported by the largest number of people (68%). The composite KAP Index The KAP index is a composite index, which includes three subindices: knowledge, attitudes, and practice. The questions used to explore each subindex can be found in Annex III pages 74-79. The following formulas represent each subindex: K ((Q14 1+2+3+4+5+6.../12) + (Q15 1+2+3+4+5/5) + Q16+Q17+Q18)/15 (scale 0-1) A (Q19+Q20+Q21+Q22+Q23)/20 (scale 0-1) P ((Q24 1+2+3+4+5.../15) + (Q25+Q26+Q27+Q28)/16)/2 (scale 0-1) The following formula explain the composite KAP index: KAP Index (K+A+P)/3 (results presented in %) In the coming months, the TET will work on a manuscript on the use of the KAP method in this VDAP evaluation. 70 Annex III: Data Collection Instruments VDAP 2020 Evaluation – Qualtrics Survey Questionnaire Q1 Florida International University (FIU) is evaluating participants from the different trainings carried out by the United States Geological Survey’s Volcano Disaster Assistance Program (VDAP) to understand how the content was delivered, knowledge acquired, and results ultimately obtained. To this end, FIU, VDAP, & USAID designed a KAP survey of 33 questions /statements with a breakdown into three main dimensions: knowledge, attitude, and practice to address the links between volcano monitoring, forecast eruptions, and emergency management. ------------------------------------------------------------------------------------------------------------------------------ ConsentKey points you should know about this study: Proposal: The study seeks to understand how the VDAP assistance content was delivered, knowledge acquired, and results finally obtained. To this end, FIU, VDAP, & USAID designed a KAP survey of 33 questions/statements with a breakdown into three main dimensions: knowledge, attitude, and practice to address the links between volcano monitoring, forecast eruptions, and emergency management. Procedures: If you decide to participate, you will be asked to complete an online survey about your experience and your perception of contents delivered. Duration: This survey will take about 10 minutes to complete. Risks: No known risks are associated with this study. Benefits: No direct benefits expected. Alternatives: There are no known alternatives available to you other than not taking part in this study. Participation: Participating in this research project is voluntary. -------------------------------- Please answer the questionnaire as honestly and carefully as possible. Q2 Place of residence o Country ________________________________________________ o City ________________________________________________ Q3 Please confirm the age group you belong to: o 20-35 years o 36-50 years o 51-65 years o More than 65 years Q4 Sex o Female o Male 71 Q5 Field of study in Bachelor’s ________________________________________________________________ Q6 Field of study in postgraduate degree (Master’s and/or Doctorate) ________________________________________________________________ Q7 Please list short trainings, certificates, internships (please indicate type of training and year) ________________________________________________________________ Q8 Have you participated in the Annual Course at the Center for the Study of Active Volcanoes (CSAV), Hawaii o Yes o No Q8a Please indicate the year you completed the CSAV. ________________________________________________________________ Q9 Have you received training or technical assistance from VDAP (other than CSAV) in any of these areas? ▢ Seismology ▢ Acoustic sensors ▢ Geochemistry ▢ Deformation ▢ Geology ▢ Monitoring of surface activity and morphological changes ▢ Strainmeter ▢ Magnetometers ▢ Gravimetry / Microgravimetry ▢ Electrical potential ▢ Borehole instrumentation ▢ Others 72 Q9a If Others, indicate which one(s) ________________________________________________________________ Q10 Have you received training or technical assistance from VDAP for the use of the following software? ▢ dMODELS ▢ gTOOLS ▢ NOVAC Software ▢ Swarm ▢ Valve Q10a If any of the above answers is yes, do you use it nowadays? o Yes o No Q11 In fulfilling your role with your country's volcanic monitoring team over the past year, indicate the percentage (%) of your time spent in the following areas: Seismology : _______ Geodesy : _______ Physical Vulcanology : _______ Geochemistry : _______ Hydrology : _______ Remote Sensing : _______ Computer Science - Computer Systems Engineering : _______ Electronics - Communications Engineering : _______ Community Outreach : _______ Risk Communication : _______ Work with risk and emergency management organizations : _______ Administration (Planning, Human Reources, Finance, Logistics, Reporting) : _______ Total : ________ 73 Q12 Do you use satellite data observations and sketches provided by VDAP assist in observatory decisions? o Yes o No Q12a If Yes, what remote sensing data is most useful (e.g. paragraph form from Reston or jpgs)? ________________________________________________________________ Q13 Do you consider that VDAP has promoted the development of your own software (homegrown) in your country and institution? o Yes o No Q13a Yes, briefly describe the type of own software (homegrown) developed ________________________________________________________________ 74 Q14 Point out the level of knowledge you have in the following areas None Basic Intermediate Advanced Seismology o o o o Acoustic sensors o o o o Geochemistry o o o o Deformation o o o o Geology o o o o Monitoring of surface activity and morphological changes o o o o Strainmeter o o o o Magnetometers o o o o Gravimetry / Microgravimetry o o o o Electrical potential o o o o Borehole instrumentation o o o o Telemetry Data o o o o 75 Q15 Point out the level of knowledge and experience you have in handling the following software? a) None b) Basic c) Intermediate d) Advanced dMODELS o o o o gTOOLS o o o o NOVAC Software o o o o Swarm o o o o Valve o o o o Q16 You have the knowledge and experience to contribute decisively to a national-scale volcano threat assessment, including existing knowledge (eruptive histories, conceptual models, etc.) and monitoring capabilities. o None o Basic o Intermediate o Advanced Q17 You have the knowledge and experience to contribute decisively to establish a relative threat ranking of volcanoes essential to guide development of long-term monitoring networks and geologic, hazard, and risk mapping projects in advance of an unrest crisis or eruption. o None o Basic o Intermediate o Advanced Q18 You have the knowledge and experience to contribute decisively to establishing graphical representations of general, mutually exclusive, initial events (e.g. unrest, eruption, size of eruption) to increasingly specific, and non-mutually exclusive phenomena, as a single eruption can 76 produce multiple hazardous products (e.g. lahars and ash fall). Events include conditional probabilities. o None o Basic o Intermediate o Advanced Q19 Express your level of agreement with the following sentence: To build a common language and understanding and to establish trust and credibility, observatories engage with stakeholders (from civil authorities to the general public) at all phases of the emergency cycle; ensure constant flow of information; continuously evaluate and improve effectiveness of communication strategies. o Strongly disagree o Disagree o Agree o Strongly agree Q20 Express your level of agreement with the following sentence: Volcano observatories play an important role in disaster risk reduction. They generate hazard forecasts and scenarios that contribute to decisions regarding evacuations, air traffic, relocation and land use planning, among other topics. o Strongly disagree o Disagree o Agree o Strongly agree Q21 Express your level of agreement with the following sentence: Volcano observatories should communicate directly and regularly with mitigation authorities such as emergency management and civil defense agencies, police, military, aviation authorities, government authorities, at all appropriate levels (national to local). o Strongly disagree o Disagree o Agree o Strongly agree 77 Q22 Express your level of agreement with the following sentence: Volcano observatories should communicate directly with communities exposed to volcanic hazards. o Strongly disagree o Disagree o Agree o Strongly agree Q23 Express your level of agreement with the following sentence: Working on volcanic monitoring necessarily implies working as a team with professionals and technicians from different disciplines, where all contribute decisively to the achievement of the proposed objectives. o Strongly disagree o Disagree o Agree o Strongly agree Q24 How often have you used the following software in the last 2 years? Never Rarely Often Frequently dMODELS o o o o gTOOLS o o o o NOVAC Software o o o o Swarm o o o o Valve o o o o Q25 Express your level of agreement with the following sentence: The event tree documentation includes: 1) A summary of monitoring data; 2) Information from a database or with analog volcanoes; 3) A list of remaining questions; 4) An interpretation that 78 integrates all observations (a conceptual model); and 5) A description of how the group arrived at stated probabilities (semi-quantitative) o Strongly disagree o Disagree o Agree o Strongly agree Q26 Express your level of agreement with the following sentence: Event trees are valid for a stated time period. They are often constructed for specific periods of significance operation: (i.e. .1 month for local gov't and evacuations; 6 months for estimating observatory staffing needs; >1 year for relocation/land use planning) o Strongly disagree o Disagree o Agree o Strongly agree Q27 Express your level of agreement with the following sentence: When generating a volcano forecast over a specified time frame, we should consider: current status (based on monitoring data), 2) What to expect next (monitoring data + eruptive history + global data), 3) How big? (eruptive history + global data+ monitoring data), and 4) What kind of hazard? (eruptive history + global data) o Strongly disagree o Disagree o Agree o Strongly agree Q28 Express your level of agreement with the following sentence: When assigning uncertainties to a particular event or manifestation under very limited data, we 79 assign probabilities in qualitative/semi-quantitative fashion (i.e. "Roughly equal probability": 50% probability; "More likely than not": 70% probability; and "Almost certain": 90% probability) o Strongly disagree o Disagree o Agree o Strongly agree Q29 Have you been involved in a professional team handling a volcanic crisis? o Yes o No Q29a If so, please list the year and name of each of the volcanoes you have been involved with in the past 5 years. Year Volcano Name Type of Volcano Activity 1 2 3 4 5 6 7 8 9 10 Q30 Order the different forms of training provided by VDAP according to their effectiveness, so that the most effective are in the first places ______ Online Training ______ Face-to-face workshops ______ Binational exchanges ______ On-the-job training (e.g. equipment installation and maintenance, mentoring during volcano crises, etc.) Q31 Should there be any change in the way VDAP provides training and technical assistance? ________________________________________________________________ 80 Q32 Which of VDAP outreach means are you aware of? (social media, publications, USGS outreach products, website, factsheets), Comment on the appropriateness of the outreach type (s). ________________________________________________________________ Q33 Are there any recommendations, concerns, or feedback you would like to provide to VDAP? ________________________________________________________________ For follow-up purposes, would you share your email address with us? ________________________________________________________________ Thanks for participating in this survey. 81 Questionnaire for MDROs USAID Contract: 720FDA20CA00023 Volcano Disaster Assistance Program (VDAP) 2020 Evaluation Florida International University (FIU) is evaluating the impact of the United States Geological Survey’s Volcano Disaster Assistance Program (VDAP) in enhancing volcano monitoring, forecasting eruptions, and strengthening emergency management capabilities in different countries around the world. To this end, FIU, VDAP, & USAID have designed a short questionnaire geared toward MDROs of countries where VDAP has been actively involved in providing technical assistance to local institutions. Answering these questions would take a few minutes of your time and help us understand the level involvement of different stakeholders, MDRO´s perception of this collaboration, as well as comments and recommendations to improve this cooperation. Country: __________________________________ Local VDAP Counterparts: _______________________________________________________ Questions 1. Has the USAID mission or the US Embassy been involved in the cooperation program of the VDAP / USGS program in the country in the last 3-5 years? if yes, then could you please provide some details? (e.g., facilitating communications, meetings, visa procedures, customs) 2. Is there a formal / signed agreement between the USG and the host country for VDAP´s cooperation? 3. Is there involvement of the central government, national emergency office, and / or the ministry of foreign affairs in VDAP´s cooperation? 4. In general, how would you rate this cooperation? (e.g., regarding the relations between the two countries, capacity building, response to emergencies of volcanic origin) 5. Are there any recommendations, concerns, or feedback you would like to provide? 82 Questionnaire for VDAP Representatives USAID Contract: 720FDA20CA00023 Volcano Disaster Assistance Program (VDAP) 2020 Evaluation Florida International University (FIU) is evaluating the impact of the United States Geological Survey’s Volcano Disaster Assistance Program (VDAP) in enhancing volcano monitoring, forecasting eruptions, and strengthening emergency management capabilities in different countries around the world. To this end, FIU, VDAP, & USAID have designed a short questionnaire geared toward VDAP representatives who have been actively involved in providing technical assistance abroad to local institutions. Answering these questions would take a few minutes of your time and help us understand this collaboration, as well as comments and recommendations to improve this cooperation. Questions 1. What are the major challenges to provide long-term technical assistance to the different countries benefited by the VDAP program? Do you have any suggestions on how to address them? 2. Has the USAID mission or the US Embassy been involved in the cooperation program of the VDAP / USGS program in the country in the last 3-5 years? If yes, then could you please provide some details? (e.g., facilitating communications, meetings, visa procedures, customs) 3. Do you have any suggestions about new tools or areas of cooperation that VDAP could develop for partners? 4. What is the impact of VDAP on domestic USGS programs? 5. Are there any recommendations, concerns, or feedbacks you would like to provide? 83 Protocols for Interviews with Heads of Volcano Observatories Is in your organization a H.R. capacity development strategy? Are there core competences defined for the different Observatory positions? Are there performance evaluations defined for the different Observatory positions? 1. What is the effectiveness of tools* provided by VDAP on improving volcano observatories’ ability to monitor their volcanoes and forecast eruptions? 1.1. Which of the following VDAP tools are in use in this observatory? Data Software Databases Monitoring equipment 1.2. How have threat assessment and gap analysis (TAGAs) been used to guide observatory operations and investigations? (Chile, El Salvador, Guatemala, Perú, Colombia) 1.3. Who is using event trees on their own? (Indonesia, Perú, Ecuador, Nicaragua, Guatemala, Argentina, Chile) How has the event tree process been used to assess and communicate risk forecasts? Is the observatory actively involved in issuing volcano alerts? Specific experiences: 1.4 In what ways do periodic satellite data observations and sketches provided by VDAP assist in observatory decisions? What remote sensing data is most useful (e.g., paragraph form from Reston or jpgs)? 1.5 How effectively has VDAP-donated equipment provided data for decision-making at active volcano sites? Specific experiences: 2. How effective have VDAP’s trainings, workshops, and exchanges been in preparing counterparts at volcano observatories in monitoring their volcanoes and forecast eruptions? 2.1 Is the current format for conducting trainings, workshops, and exchanges effective (e.g., are regional workshops as effective as observatory-based training sessions, are general or specific trainings more effective, etc.)? Should there be any changes? 2.2 From the volcano observatory perspective: Which trainings, workshops, and exchanges are the most useful to participants? How important a role does CSAV and other VDAP trainings play in the human resource development of volcano observatory staff around the world? How is knowledge transferred after the activity is completed? 2.3 How have countries participating in binational exchanges used the knowledge transferred during the exchanges to further Disaster Risk Reduction (DRR) in their countries? 2.4 What is the current status of women’s representation in the Volcano Observatory staff? Number of Men: Number of Women: Women in leading positions: 3. To what extent has stakeholder effectiveness improved due to VDAP programming? 3.1 Has VDAP assistance led to a reduction of lives and property lost from volcanic eruptions? Specific experiences: 84 3.2 Has the ability of the government authorities to forecast and respond to eruptions improved as a result of VDAP assistance? Specific experiences: 3.3 Science Diplomacy ARG, ECU, NIC, INDONESIA Has the USAID mission or the US Embassy been involved in the cooperation program of the VDAP / USGS program in the country? Is there a formal/signed agreement for this cooperation provided by VDAP? Is there involvement of the central government and/or the ministry of foreign affairs? 4. To what extent has stakeholder* coordination and leveraging improved due to VDAP programming? 4.1 How has partnering with VDAP fostered the development and operation of counterpart agencies (emergency agencies, local authorities) over the last 2.5 years? 4.2 Has VDAP material assistance been leveraged by host countries to obtain more support from national or foreign donor sources? Specific experiences: 4.3 Have VDAP activities been well-coordinated with other donors to avoid duplication of effort? Specific experiences: 4.4 How does VDAP facilitate collaboration/cooperation within/between countries? What binational or multi￾agency "agreements" have been fostered by VDAP? Specific experiences: 5. What are the recommendations for future VDAP programming? 5.1 VDAP works closely with the volcano observatories in forecasting– in person during responses or via teleconference. Is there a way to improve this type of technical support? 5.3 Which of the VDAP outreach means are you aware of? (social media, publications, USGS outreach products, website, factsheets) Is the type of outreach interesting for your organization? / Comment on the appropriateness of the outreach type(s). 5.4 Should VDAP develop new tools for partners? (Are there tools that partners want that VDAP does not provide?) 5.5 Are there any recommendations, concerns, or feedback you would like to provide? 85 VDAP Remote Field Visit Country Place Institution Date Conducted by Local Counterpart Built Area Number of Employees Functional structure Availability of protocols EWS / Alert - When analyzing the volcanoes in your country for threat assessments, what kind of system is being used to prioritize monitoring such as seismic, deformation, hydrology, remote sensing, and gas? 86 - Does this system have a preliminary and/or clear breakdown of specific points that are being taken into consideration for categorizing the monitoring level? Indicate below the number of equipment the observatory has in each category, specifying whether this equipment is currently in use or not. Equipment/network/system/facility Number Is the equipment/network/system/facility currently in use Equipment donated by VDAP (Take into account condition according to the attached table, equipment condition) Comments 1 - Surveillance Methodologies a) Seismology: - Short uniaxial period seismometers (1 Hz) - Triaxial short period eismometers (1 Hz) - Medium Band Seismometers (30') - Broadband Seismometers(120's) - Accelerometers. b) Acoustic sensors: - Broadband infrasound sensors. 87 Equipment/network/system/facility Number Is the equipment/network/system/facility currently in use Equipment donated by VDAP (Take into account condition according to the attached table, equipment condition) Comments - Short-term infrasound sensors - Infrasound arrays. c) Geochemistry - DOAS stations. - FLYSPEC Stations - Multigas Equipment - FTIR - IR Cameras - UV Cameras - Radon Stations - CO2, H2S, CH4 diffusion chambers. - Thermocouples, pHmeter, conductivity meter. - Radiometers - On-site fluid sampling. - Satellite geochemical monitoring international agencies (IMO, TROPOMI) - Gas and fluid geochemistry laboratory. d) Deformation 88 Equipment/network/system/facility Number Is the equipment/network/system/facility currently in use Equipment donated by VDAP (Take into account condition according to the attached table, equipment condition) Comments - GNSS stations - Electronic tiltmeters - LIDAR terrestrial - AIR LIDAR - INSAR - Leveling Lines - EDM measurements - Photogrammetric surveys. e) Geology - Petrographic laboratory and sample analysis • Analysis under binocular magnifying glass • Analysis with electronic microscope • Screening Instruments • Rock cutting • SEM • Electronic microprobe 89 Equipment/network/system/facility Number Is the equipment/network/system/facility currently in use Equipment donated by VDAP (Take into account condition according to the attached table, equipment condition) Comments • Other: please specify - Ashmeters - Modeling software of issued products (Laharz, flow2d, ashfall, etc) - Volcanic hazard mapping - Vulnerability and risk assessment - Volcano Ranking f) Monitoring of surface activity and morphological changes - Photo cameras - Video cameras - Thermal cameras - Drones - Mudflow network - Rain gauges - Weather station - Regular overflights - Access to high resolution satellite images 90 Equipment/network/system/facility Number Is the equipment/network/system/facility currently in use Equipment donated by VDAP (Take into account condition according to the attached table, equipment condition) Comments f) Strainmeter g) Magnetometers h) Gravimetry i) Electrical potential j) Borehole 2 - Communications infrastructure. - Analogical telemetry - UHF Telemetry - Satellite Telemetry - Sending data via internet - Fiber optics - Energy systems for remote sites 3 - Systems infrastructure - Datacenter - Headquarters energy backup system - Software development group - Computer support group - Contingency site. - - Network administration group 91 Equipment/network/system/facility Number Is the equipment/network/system/facility currently in use Equipment donated by VDAP (Take into account condition according to the attached table, equipment condition) Comments - Database administration. 4 - Monitoring center infrastructure - Monitoring room - Meeting room - Conference room - Warehouse for storage of equipment and supplies - Equipment testing laboratory (sensors and communications) - Electronics laboratory - Portable crisis network - Exclusive web page of the volcanological center - Volcanological museum - Group of diffusion and social communication - Social network web manager Has the observatory received equipment donated from VDAP in the last five years, please describe it? You can use the comment box of the equipment described above to indicate in detail the conditions of this equipment based on the classification scheme indicated below: To consider when addressing equipment donated by VDAP 92 Inventory and current condition CONDITION CODE 1 E = Used-Reconditioned N = New U = Used-Usable Without Repairs R = Used-Repairs Required L = Lost, stolen, damaged CONDITION CODE 2 OP = Operational NO = Non-operational CONDITION CODE 3 1 = Excellent 2 = Good 3 = Fair 4 = Not Being Used https://controller.psu.edu/equipment-condition-codes Has the observatory you received technical support from VDAP on the installation, use, maintenance of instruments described above during the last five years, please describe? To consider when addressing software - Are they using software provided by VDAP? - If yes, how does VDAP-provided software affect observatory operations? - Are they using homegrown software? If yes, describe them? Has VDAP provided technical assistance? - Are they using other software packages commercial or provided by other partners/donors? Are there any recommendations, concerns, or feedback counterparts would like to provide to VDAP? 93 Annex IV. Sources of Information Reported Use of VDAP Tools Argentina Chile Colombia Costa Rica Ecuador El Salvador Guatemala Indonesia Peru INGEMMET Peru IGP Probabilistic Event Trees R R R R R R R R R R Threat Assessment and Gap Analysis R R N R N R R R R R dMODELS R R R R R R INA R R R NOVAC Software N R R R R R R R R R Swarm R R R R R R R R R R Valve N N N R INA INA INA R N N Satellite data observations & sketches (provided by VDAP) R R R R R R R R R R R - Reported in use; N - Reported not in use; INA - Information Not Available Training Portfolio -- CSAV VDAP Midterm Evaluation Questionnaire 1. Activity description (purpose, target audience, objectives, content, duration, pre-requisites), reference and training materials? The Center for the Study of Active Volcanoes (CSAV), on the Big Island of Hawaiʻi, operates out of the University of Hawaiʻi at Hilo. The Center is a training and outreach program founded by Robert W. Decker. CSAV's mission is to provide information on volcanic and natural hazards that occur in Hawaiʻi and worldwide. CSAV has been operating since 1989 and is a cooperative program of the University of Hawaiʻi at Hilo, the U.S. Geological Survey (USGS) Hawaiian Volcano Observatory (HVO), and the University of Hawaiʻi at Mānoa (UHM). The CSAV International Training Program is designed to assist developing nations in attaining self￾sufficiency in monitoring volcanoes. The field-based training emphasizes volcano monitoring methods—both data collection and interpretation— in use by the USGS. Participants are taught the use and maintenance of volcano monitoring instruments and techniques to assess volcanic hazards. Participants also learn the interrelationship of scientists, governing officials, and the news media during volcanic crises. There have been 264 scientists and technicians from 30 countries who have attended the CSAV International Training Program since 1990. The course is an introduction to a variety of volcano monitoring techniques rather than a detailed training with just one technique, hence seismologists who attend the CSA training learn about deformation, gas geochemistry, and physical volcanology as well as geophysics. The course is not geared toward academics, but rather, addresses working in a crisis response mode, focusing on forecasting and rapid response to save lives and property. 94 The six-week training course is divided into two sections. The Hawaiʻi Section is held at the University of Hawaii at Hilo and covers physical geology, webcams, gas geochemistry, rock identification, geologic mapping, lahars, photogrammetry, seismology, remote sensing, deformation, and dealing with the press. Field work sites are Kīlauea and Mauna Loa volcanoes. The Washington section is held in Vancouver, Washington and includes work at the Cascades Volcano Observatory (the home office of VDAP) as well as field work at Mount St. Helens. Topics covered are stratigraphic sections, mapping volcanic deposits, engineering and assembly of monitoring instrumentation, public outreach, probabilistic event trees, and the relationship of scientists to media and Civil Defense. VDAP staff instruct counterparts through both sections of the course. Attendees are early-career scientists and are nominated by their volcano observatories/institutions. VDAP funds the travel and attendance of foreign counterparts. Course size is generally limited to 12 students to ensure participants receive “hands-on” training throughout the instruction. 2. Assessments conducted Darcy Bevens (bevens@hawaii.edu) of CSAV at UH Hilo conducts participant assessments of different sections during the course. 95 Training Portfolio – Probabilistic Event Trees VDAP Midterm Evaluation Questionnaire 1. Activity description (purpose, target audience, objectives, content, duration, pre￾requisites), reference & training materials. One format for framing volcanic eruption forecasts is known as volcanic event trees. Probabilistic volcanic event trees are frequently used to facilitate discussion, reach consensus, evaluate uncertainty, and help scientists assess the likelihood of different hazardous phenomena before and during volcanic crises. Event trees are graphical representations of successive possible events or outcomes (represented by numbered nodes), with the branches leading from general, mutually exclusive, initial events (e.g., unrest, eruption, size of eruption) to increasingly specific, and non￾mutually exclusive phenomena, as a single eruption can produce multiple hazardous products (e.g., lahars and ash fall). Events are assigned conditional probabilities, the likelihoods of a single event given that the previous event has occurred. Figure 3 Data Stream Flows Because event likelihoods are based on subjective interpretation of multiple data streams (e.g., Figure 3), there can be significant uncertainty to likelihood determinations. Furthermore, the communication of event probabilities may differ depending upon the audience. In some cases, probabilities are used for internal discussion only (e.g., Wright et al. 2019), whereas in other cases probabilities are regularly released to the public (e.g., White Island eruption probabilities on Geonet webpage, 2019). In still other cases, quantitative probabilities are used to inform emergency management decisions (e.g., Newhall 1982). Furthermore, the nature of communication of probabilities is dependent upon audience and can include relative, absolute, and comparative discussion of likelihoods (e.g., Doyle et al. 2014). 96 Event tree training are usually conducted in person and require a range of different discipline specialties to participate in the training. In-person trainings range from ½ day to 2 ½ days in length. As an example of a training (over 2 days), see current materials (power points, training agenda, workbook materials) for the CSAV course in a folder linked at: https://drive.google.com/drive/folders/1Vo_pD8IcuNENHnn7QX81Puf85TKrOqJH?usp=sh aring In this folder you will find: • Power point presentations of: a. Mount St. Helens (MSH) eruptive history (Pallister and Wright presentation) b. How event trees work (Wright and Ogburn presentation) c. History of seismicity and unrest at MSH d. A post-exercise review of what actually happened at MSH in 2004-2008 • Figures at 4 time steps (part of the exercise workbook) a. Sept. 26 b. Sept. 30 c. Oct. 2 d. post-Oct. 2 • A blank event tree with a pre-formatted structure for MSH • Reference materials a. Event tree papers b. Seth's MSH seismicity paper c. Example waveforms of seismic types at MSH (including VT, hybrid, LF - perhaps we should add ice quakes? and rockfalls?) d. A description of alert levels (as of 2004 and as of now) e. MSH fact sheets f. VEI distribution of all MSH eruptions as recorded in GVP g. MSH hazard map (Pierson version, OF95-947) h. A map of Lahar simulation runs (with disclaimer: simulations run over x year topography and to be used for this exercise only) i. Ash3D runs (again, with disclaimer) j. A wind rose diagram 2. Does the staff of the observatories receive training in the use of this tool? Or is it the result of a self-study based on VDAP / USGS technical documents? All observatories receive training upon request (see above). All trainings have been in person, but two virtual event tree trainings have also been conducted (with Argentina in 2018 and Chile in 2020). 3. Has VDAP updated these technical materials? Yes. The training exercise is also tailored to meet the needs of the partner countries. 97 4. Is there a database of participants/attendees to this type of capacity building activity? Key points of contact for previous on-site event tree trainings (Full lists of participants can be provided). Chile 2020 remote consultation María Angélica Contreras Geologist from OVDAS – working in volcano monitoring Temuco maria.contreras@sernageomin.cl Argentina/Chile 2018 remote consultation Manuela Elissondo Geologist from SEGEMAR manuelaje@yahoo.com.ar Guatemala 2018 onsite training (Fuego volcano) Amilcar Palma Seismologist from INSIVUMEH, aeroca@insivumeh.gob.gt Nicaragua 2016 onsite training during crisis at Momotombo volcano Eveling Espinoza Geologist from INETER eveling.espinoza@gf.ineter.gob.ni Peru 2016 onsite training (Ubinas volcano) Marco Rivera Geologist from IGP mrivera@igp.gob.pe Ecuador 2015 onsite training during Cotopaxi crisis Ben Bernard Geologist from Instituto Geofisico bbernard@igepn.edu.ec Indonesia 2014 onsite training (Sinabung volcano) Supriyati Andreastuti Geologist and Socialization expert, CVGHM s7andreastuti@gmail.com All past CSAV students since 2013 have also taken this training. 98 VDAP Activities Related to NOVAC VDAP Midterm Evaluation Questionnaire What is NOVAC? The Network for Observation of Volcanic and Atmospheric Change (NOVAC) is a community of volcano observatories and research institutions that together develop and apply ultraviolet differential optical absorption spectroscopy (DOAS) instruments to measure volcanic gas emission rates. The collected data are used for assessment of volcanic activity, eruption forecasting, research on volcanic processes, and studying the atmospheric impact of volcanic degassing. Together, the institutions participating in NOVAC run the largest instrument network for monitoring gas emissions from volcanoes around the world. VDAP began supporting NOVAC efforts in 2014 in two different ways. For one, VDAP has organized two NOVAC workshops (Costa Rica 2015 and Peru 2018) and sponsored observatory staff from partner institutions to attend these. These workshops follow on from regular workshops that NOVAC was holding between 2005 and 2010 with funding from the European Union (which is no longer available) and allow observatory staff to present the results and interpretations of their gas measurements to the rest of the community for discussion. Secondly, VDAP supports partner institutions in maintaining and expanding their gas monitoring networks by donating DOAS instruments and in some cases providing on-site assistance/training in installing, configuring, and running the instrumentation. These two support modes will be addressed separately below. NOVAC workshops 1. Activity description (purpose, target audience, objectives, content, duration, pre￾requisites), reference & training materials? VDAP has organized two NOVAC workshops in the last 5 years. Both were organized in close collaboration with the host institutions (OVSICORI in Costa Rica and INGEMMET in Peru). The purpose of the workshops is to bring together observatory staff working with scanning DOAS instruments to monitor volcanic gas emissions such that they can share data and experience. Since most of the staff have been working with these instruments for a number of years, the target audience is generally adept at the basic concepts of the measurements. VDAP will often invite two persons from each partner institution to attend, with typically one scientist and one field engineer attending. Often, some additional training may be planned in association with the workshop. E.g., a 1-day introduction into DOAS was held for partners new to NOVAC prior to the 2015 workshop, and an advanced spectroscopy workshop was held with select, more advanced partners after the 2018 workshop. During the workshop, every institution is expected to give a 20-minute presentation on volcanic gas monitoring in their country. Then there is time for discussion. In addition to these country￾specific talks, members of the NOVAC community are invited to give additional presentations on overarching topics (e.g., new instrumentation ideas, new data analysis techniques, etc.). The workshops typically go for 5 days with a 1-day field trip to a volcano. During the field trip, they typically visit one or multiple monitoring stations to demonstrate their design (which differs slightly from country to country) and provide an opportunity for members that brought instrumentation to make some test measurements of their own. There are no formal pre-requisites, but most attendees have experience with the instrumentation and interpretation of geochemical volcano monitoring 99 parameters. During each workshop, all presentations were collected and provided to the attendees in digital form. At the Peru workshop, a post-workshop survey for VDAP M&E were also conducted. 2. Does the staff of the observatories receive training in the use of this tool? Or is it the result of a self-study based on VDAP / USGS technical documents? Most attendees are already familiar with the NOVAC instruments. Separate from the workshops, VDAP supports individual partners with additional basic training as needed (see section 2 of this document). Though VDAP plays a major role in organizing and sponsoring these workshops, the majority of the content comes from the attendees. 3. Has VDAP updated these technical materials? Yes, VDAP organizes the workshops and the dissemination of workshop content in digital form after each workshop. 4. Is there a database of participants/attendees to this type of capacity building activity? Summary reports of the last two workshops will be attached. These include participant lists on the last page. Additional information is available on the website www.novac-community.org. An EOS article regarding the 2015 NOVAC workshop is available here: https://eos.org/meeting￾reports/monitoring-gas-emissions-can-help-forecast-volcanic-eruptions NOVAC capacity building 1. Activity description (purpose, target audience, objectives, content, duration, pre￾requisites), reference & training materials? VDAP supports individual partner institutions in building, maintaining, and expanding their capacity for geochemical monitoring of volcanoes. If requested by a partner institution, this can include donating NOVAC scanning DOAS instrumentation, providing advice remotely for troubleshooting existing instruments or interpreting collected data, and in some cases can involve on-site training/assistance. On-site assistance is typically provided to partners who are relatively new to the NOVAC community, as they have the highest need for training. In the past 5 years, NOVAC-related on-site assistance has been given to CVGHM (Indonesia, 2 visits), INGEMMET (Peru, 2 visits), and RVO (Papua New Guinea). Oftentimes, an on-site visit will be preceded by a shipment of NOVAC instrumentation. This instrumentation will then be assembled and installed on-site with the partners. Trips usually last about 3 weeks, and sometimes not all the instrumentation will be installed by the end of the trip, but usually a plan will be made for the partners to finish the installations on their own. The partners are also trained in how to evaluate the data to obtain robust gas emission rates, and how these might be interpreted in combination with other monitoring parameters. During these on-site visits, VDAP staff typically work with one or multiple geochemists from the partner observatory, as well as one or multiple field engineers responsible for installing/maintaining monitoring equipment. The objectives of each visit include the installation of new or maintenance of existing NOVAC monitoring sites and the training of observatory staff in their use. After such an effort, the hope is that VDAP could send additional instrumentation in the future (e.g., for an 100 eruption response) without the need for an on-site presence (although this usually takes more than a single visit). VDAP will work with observatory staff at any level, so there are no formal prerequisites, but most observatories have staff with some level of experience in geochemical data collection, analysis and interpretation and field engineering. The NOVAC community has developed manuals that describe the functionality of the instrumentation and are being provided. 2. Does the staff of the observatories receive training in the use of this tool? Or is it the result of a self-study based on VDAP / USGS technical documents? Most NOVAC partners have experience with the NOVAC tools, but VDAP sometimes provides basic training to individual partners who are relatively new to NOVAC or provides specialty training to more advanced partners. 3. Has VDAP updated these technical materials? VDAP is not solely responsible for the technical manuals but does participate in keeping them updated. 4. Is there a database of participants/attendees to this type of capacity building activity? Below is the contact information of partners with whom VDAP has had recent NOVAC capacity￾building activities: CVGHM, Indonesia Syegi Kunrat Geochemist syegikunrat@gmail.com Ugan Saing Geochemist ugan_saing@vsi.esdm.go.id Sulus Setiono Field Engineer sulus@vsi.esdm.go.id INGEMMET, Peru Fredy Apaza Geochemist fapaza@ingemmet.gob.pe Pablo Masias Geochemist pmasias@ingemmet.gob.pe Rabaul Volcano Observatory, Papua New Guinea Kila Mulina Geochemist kila_mulina@mineral.gov.pg Ima Itikarai Chief of Staff ima_itikarai@mineral.gov.pg John Bosco Field Engineer john_bosco@mineral.gov.pg 101 Threat Assessment and Gap Analysis (TAGA) Training VDAP Midterm Evaluation Questionnaire 1. Activity description (brochure: purpose, target audience, objectives, content, duration, pre-requisites), reference & training materials Through application of a National Volcano Early Warning System (NVEWS) framework, developed by the USGS and VDAP, partner countries develop national-scale volcano threat assessments and monitoring strategies for their nation’s volcanoes. Threat assessment and gap analysis (TAGA) trainings/workshops typically last 1 week with the end result being a systematic volcano threat ranking highlighting priority volcanoes, projects, and investigations that can be used to justify and guide funding from domestic and foreign donor sources to achieve the greatest benefit in terms of risk reduction. During TAGA workshops, multi-disciplinary teams develop volcano monitoring strategies that typically include 4 levels of monitoring (lowest level 1 “minimal monitoring” to highest level 4 “well monitored in real-time”) that are assigned to each volcano based on its threat level (for example, the highest threat volcanoes require the highest level of monitoring). Current monitoring levels at volcanoes are determined and compared with the desired state to develop a gap analysis. The volcanoes with the biggest monitoring gaps between their current level and their target level are given priority for using domestic and foreign resources to improve monitoring networks and investigations. The gap analyses have evolved from being primarily about monitoring to include knowledge gaps as well (such as eruptive histories, conceptual models, etc.). During these workshops, preliminary target volcanoes are identified for future monitoring installation and upgrades as well as scientific investigations based on the completed volcano threat assessment and monitoring and knowledge gap analysis. TAGA workshops provide our partner agencies with a relative threat ranking of volcanoes that is used to guide development of long-term monitoring networks and geologic, hazard, and risk mapping projects in advance of an unrest crisis or eruption. 2. Does the staff of the observatories receive training in the use of this tool? Or is it the result of a self-study based on VDAP / USGS technical documents? Yes, the staff receive hands-on training and by the end of the week they have a draft TAGA. During the TAGA workshop VDAP experts work with the local observatory staff to conduct hands on training as the analysis is actually conducted. Foreign colleagues then draft the final report. TAGAs are meant to be dynamic and updated routinely, as volcanic activity occurs and when gaps in monitoring or knowledge are reduced. By providing the week-long training and working through all aspects of a TAGA, partner agencies are equipped to update their own TAGAs. 3. Is there a database of participants/attendees to this type of capacity building activity? Guatemala Points of contact: Amilcar Rocas aeroca@insivumeh.gob.gt Gustavo Chigna gustavo.chigna@gmail.com 102 Nicaragua Points of contact: Eveling Espinoza eveling.espinoza@gf.ineter.gob.ni William Martinez William.martinez@ineter.gob.ni El Salvador Points of contact: Manuel Diaz mdiaz@marn.gob.sv Eduardo Gutierrez egutierrez@marn.gob.sv Peru Points of contact: Orlando Macedo Was the Chief Author. Moved on from IGP to UNSAA Orlando Macedo omacedos@unsa.edu.pe Marco Rivera IGP mrivera@igp.gob.pe Rigoberto Aguilar raguilar@ingemmet.gob.pe 103 Seismic Analysis and Interpretation Trainings VDAP Midterm Evaluation Questionnaire 1. Activity description (purpose, target audience, objectives, content, duration, pre￾requisites), reference & training materials. The VDAP seismology group has conducted and sponsored a variety of trainings in seismic analysis and associated tools to partner volcano observatories in the past three years (in addition to teaching a week of CSAV training each summer which we will not describe here, assuming that topic has been covered elsewhere). Trainings in 2020 were limited to ongoing remote support of partner volcano observatories, due to COVID-19 restrictions here and abroad. VDAP trainings in 2018 and 2019 ranged from targeted trainings for individual volcano observatories (MARN, El Salvador; INSIVUMEH, Guatemala; INGEMMET and IGP, Peru), and larger trainings such as the LAVAS 5 meeting (details below). Generally, training is provided to the observatory staff responsible for volcano seismology interpretation, so ability level of the trainees varies dramatically. In addition, VDAP colleagues are sent to external training sessions that will improve their data processing skills (i.e., sending staff from Papua New Guinea to software training classes). Details on trainings are given below. In 2019, VDAP co-organized and co-hosted the premier international workshop for volcano seismologists in Latin America. The 10-day LAVAS 5 workshop, located in Pasto Colombia, included 71 participants from 12 countries and 16 VDAP partner organizations. This workshop brought together the leading volcano seismologists from the US and all Latin American countries. Each participant shared information on seismicity at the volcanoes in their country and their monitoring strategies. Because most of this information does not exist in the published literature, this exchange of information was a powerful mechanism for all attendees to gain new insights from the experience of others that may be helpful in monitoring their own volcanoes, through discussion and sharing techniques. VDAP reserved several days at the end of the workshop for hands on software demonstrations. Three software packages were shared by the USGS. Training materials included software packages and summaries of volcano seismic data from many countries. In 2018, VDAP taught a week-long course to 18 leading seismologists from three organizations in Peru (IGP, INGEMMET, and UNSA). The course introduced students to a variety of advanced seismic methods commonly used to analyze seismic activity at volcanoes, including earthquake location methods, seismic velocity analysis, seismic source analysis, and seismic cataloging and forecasting methods. VDAP assisted INGEMMET by providing software for analysis of infrasound data and troubleshooting some software problems for them. In 2018 and 2019, in response to the disastrous eruption of Fuego volcano in Guatemala, a visit to INSIVUMEH was done to provide software that would send alarms on dangerous lahars, software for infrasound data analysis and interpretation, and software for analyzing ‘earthquake families’. Trainings for these software packages and for seismic data interpretation techniques and strategies in general occurred over the course of multiple ~week-long visits. These software packages are now part of their standard observatory operating system. In 2019, VDAP taught a week-long training course to 13 staff of MARN, El Salvador, on advanced seismic analysis techniques and strategies. VDAP installed two USGS software packages on this trip, 104 one of which they are using operationally and for research on San Miguel volcano. Although follow up training in 2020 was delayed due to the global pandemic, continued support has been provided to them remotely in the use of these tools. 2. Does the staff of the observatories receive training in the use of this tool? Or is it the result of a self-study based on VDAP / USGS technical documents? Both methods are used to train partners, but the former method is used primarily. Typically, introductory training is done in person and continuously supported remotely. 3. Has VDAP updated these technical materials? VDAP updates training presentations for CSAV annually, and generally updates presentations for individual training sessions to tailor the information to the trainees’ needs. These presentations are shared with trainees along with technical manuals that exist. 105 VDAP Binational Exchanges FY 2018 – FY 2020 FY 2018 U.S./Colombia Binational Exchange – From November 18 through 25, two U.S. education/emergency management officials and USGS Cascades Volcano Observatory outreach coordinator Carolyn Driedger visited Colombia to learn about volcano hazards mitigation activities within schools and their broader communities, and to share information about similar efforts in the U.S. This Binational Encounter in Risk Management and Education was held in historic downtown Popayán and was hosted by the Colombian Geological Service (SGC). Popayán is home to one of SGC’s volcano observatories and is located near Nevado del Huila, Puracé, and Sotará volcanoes. Objectives of this exchange were to bring attention to existing volcano education efforts, identify common elements of success, and select a path forward toward creation of a national volcano education curriculum for Colombia. Marta Calvache of SGC leads a discussion at the U.S./Colombia binational exchange on a future Colombian national volcano curriculum that can help improve the safety of Colombian communities. Recurring themes during the exchange were the integration of hazards and safety education with environmental education, the teaching of respect for the volcano and its ecology, and teaching with consideration for the cosmology as interpreted by the local indigenous people. Another important recognition is that coordination of educational messaging by schools, families, and community authorities is essential for effective education and risk management. FY 2019 Colombia and Ecuador/U.S. Binational Exchange, Washington – From October 14 through October 19 in Washington State, 10 geoscientists, emergency managers, and first responders from Colombia and Ecuador attended a binational exchange in Washington State. Participants learned about the geology, eruption history, and potential hazards of Mount Baker and Glacier Peak in northern Washington. They attended an emergency preparedness exercise in 106 Whatcom County featuring an eruption scenario for Mount Baker, the northernmost volcano is the U.S. Cascades which poses potential hazards to nearby Canada. Participants in the Colombia and Ecuador/U.S. binational exchange visited the Unified Emergency Coordination Center in Whatcom County, Washington. Mount Baker was chosen for this exercise in this exchange because of the cross-border cooperation between the USA and Canada that will be necessary to respond to an eruption. Similar cross-border cooperation will also be needed should Chiles-Cerro Negro erupt along the border between Colombia and Ecuador. Participants also visited the new Mount Rainier lahar detection system, which is based in part on lahar detection systems developed by VDAP and counterparts from Colombia and Ecuador, and used at volcanoes in those countries. U.S./Ecuador Binational Exchange, Ecuador – Emergency managers representing the U.S. Forest Service, the State of Washington, and four counties in Washington State that lie partially within hazard zones from Cascades volcanoes traveled to Ecuador from June 2 through 8 for a binational exchange focused on preparation and response to volcanic eruption crises. They were joined by scientists from the USGS Cascades Volcano Observatory and VDAP. The U.S. delegation 107 met with counterparts from the Instituto Geofisico, Escuela Politecnica Nacional, SNGRE (Ecuador’s National Risk and Emergency Management Service), and Ecu911 (Ecuador’s Integrated 911 Security Service) to learn about lahar detection and notification systems and other volcanic risk mitigation measures at Cotopaxi and Tungurahua volcanoes, which have much in common with Cascades volcanoes. Two members of OFDA’s LAC staff also attended the exchange. Participants in the U.S./Ecuador binational exchange visited Ecu911 (Ecuador’s Integrated 911 Security Service) to learn about lahar detection and notification systems and other volcanic risk mitigation measures at Cotopaxi and Tungurahua volcanoes. Participants saw examples of infrastructure development, notification and warning systems, and community networks that have developed prior to or during recent volcanic eruptions at these volcanoes. They learned about the challenges of managing volcanic emergencies. Emergency managers observed a Cotopaxi eruption crisis simulation exercise, after which they provided feedback to Ecuadorian emergency managers and scientists with suggestions for improvement for future exercises. Participants interacted with emergency managers, politicians, and city officials at the national, regional, and local levels. U.S. participants reported that primary lessons learned from this exchange included: the importance of coordination between emergency managers in different jurisdictions and agencies (several discussions occurred between Washington State and county emergency managers during this trip to better coordinate their plans and procedures in the future), the role of uncertainty in scientific forecasts and the challenges of communicating that uncertainty to stakeholders, the importance of relationship building prior to crises, and the importance of community-based risk education, response planning, and recovery. The host countries benefited from this exchange, some activities that were organized for the meeting would not have been conceived otherwise. Ecuadorian counterparts reported that the Cotopaxi eruption exercise was scheduled specifically to coincide with this exchange. Community meetings in Latacunga and Banos were prompted by the exchange and offered opportunities to 108 recognize the important roles of vigias (citizen volcano monitors), community leaders, and scientists in successful management of crises, despite rocky beginnings. FY 2020 U.S./Colombia Binational Exchange, Colombia – Between February 24 and 28, the Servicio Geológico Colombiano (SGC) and its partners sponsored the Fifth National Biennial of Children and Young People Living in Volcanic Risk zones (commonly referred to as ‘The Bienal’). This event was the fifth in a series held over the past decade. Organizers assembled Colombian students and teachers for four days of intensive focus on the scientific and cultural aspects of volcanoes adjacent to these at-risk communities. This 2020 event was held in New Armero/Guayabal Colombia in commemoration of the 35 years anniversary of the eruption of Nevado del Ruiz volcano on November 13, 1985. Approximately 240 students and teachers from the departments of Nariño, Cauca, Quindío, Caldas and Tolima attended the event. Travelers from USA observed the event, interacted with students, teachers, and their counterpart officials, and compared information about lahar hazards and risk reduction measures in the USA and Colombia. For fellow travelers, USGS staff chose three officials from the small town of Darrington, WA, including the town’s mayor, school district superintendent, and coordinator of youth programs. Their attendance was aspirational, because while they have not taken steps for risk reduction from nearby Glacier Peak volcano, they intend to do so. The US travelers had responded to the 2014 Oso landslide tragedy (43 fatalities), which had many of the same terrible characteristics of the 1985 Nevado del Ruiz lahar tragedy. USGS staff distributed packets of educational materials to ~50 teachers, including classroom activities from the Mount Rainier Educator Guide (Spanish) that are sufficiently generic for use in Colombia, brochures about lahar safety and Mount St. Helens, post cards displaying Cascade volcanoes, a fact sheet about volcanic hazards (Spanish), and a colorful classroom poster about how science begins with curiosity (Spanish). Public Information Officers for SGC and the schools received a copy of selected portions of the USGS-CVO News Media Management Guide (Spanish). VDAP funds paid for translation of all items listed above into Spanish. 109 Mayor Dan Rankin talks with the Mayor of Armero-Guayabal, discussing the City of Darrington’s response to the Oso landslide in 2014 and learning about the 1985 lahar in Armero. In additional to Bienal activities, US travelers were invited to the national civil defense training center located near Mariquita, and to primary and secondary schools in New Armero/Guayabal. US travelers from Darrington and the USGS returned with a stronger bond, and a common understanding of lahar hazards. This interaction will enable them to work on risk reduction measures more effectively in coming years. 110 VDAP-Provided Software and Observatory Decisions – dMODEL VDAP Midterm Questionnaire 1. Type of software, description per type (What does the application do? Intended audience/User interface). dMODEL is a MATLAB-based software package, developed as a practical, working tool for geoscientists interested in applying mathematical modeling to investigate volcano and earthquake deformation. A general understanding of the concepts and calculus behind the models is recommended to enhance the application of the models. As a companion to the software package, the U.S. Geological Survey has published a paper with detailed introduction to the mathematics of each model (Battaglia et al., 2013). Most of the models are taken from other credited or standard sources, but VDAP has derived or re-formatted a number of equations. All formulae have been checked for typographical errors existing in the original references and confirmed against numerical models (COULOMB 3, earthquake.usgs.gov/research/modeling/coulomb and COMSOL Multiphysics, www.comsol.com). The manual and MATLAB scripts are open source and intended for teaching and research. If any of the scripts are used in research submitted for publication, the authors must cite the USGS publication and JVGR paper. Intended audience: Staff at volcano observatories and academic institutions. Battaglia, M., Cervelli, P. F., & Murray, J. R. (2013). dMODELS: A MATLAB software package for modeling crustal deformation near active faults and volcanic centers. Journal of Volcanology and Geothermal Research, 254, 1-4. Battaglia, M., Cervelli, P. F., & Murray, J. R. (2013). Modeling Crustal Deformation Near Active Faults and Volcanic Centers--a Catalog of Deformation Models (pp. 96-p). US Department of the Interior, US Geological Survey. 2. Is there on the job training for software users? Yes. People attending the annual, international volcanology school at CSAV (UH Hilo) are trained in the use of the software. Workshops on the use of the software are run every year at volcano observatories, conferences, and meetings. Mentoring of individuals or small groups is also done. This year, online training was offered because of COVID-19. 3. Software maintenance (including updates) Software is updated every year and distributed to people attending the summer international CSAV training at UH Hilo, staff of partner institutions (volcano observatories in LAC and Philippines) and people attending GEOVOL meetings. Software is open source and available for free to anybody who asks for a copy of the code. 4. Data management (associated with the software) dMODELS is released with GPS, tilt and InSAR datasets to help those testing the code. These datasets are public. 5. Hardware associated with software comments The MATLAB version of dMODELS runs under Windows 10, Linux or MacOS. The compiled, executable version runs only under Windows 10. 111 VDAP-Provided Software and Observatory Decisions – gTOOLS VDAP Midterm Evaluation Questionnaire 1. Type of software, description per type (What does the application do? Intended audience/User interface). gTOOLS is an open-source software dedicated to the processing of relative-gravity data. The software is available both in MATLAB and as a compiled executable to be run under the free MATLAB Runtime Compiler (MCR). gTOOLS is particularly suited for time-lapse (temporal) gravity monitoring, where high precision is required. The software binds together single-task processing modules within a very simple user interface, based on a single text file. The modular structure allows the user to easily update the software. The code allows the fast treatment of raw gravity data from a network composed by a base station and several monitoring sites. Data processing includes the removal of measurements noise, and correction for Earth tides, ocean loading and instrument drift. Instrumental drift, gravity differences between the base station and monitoring sites, and associated errors are obtained by least square analysis of the data set. When different occupations of the network are completed, the software computes a single solution for the whole survey. The program has been tested in the field in Hawaii, Mount St. Helens, and several other volcanoes monitored by the USGS. Intended audience: Staff at VDAP partner institutions, volcano observatories, and academic institutions. 2. Is there on the job training for software users? Mentoring of people interested in learning how to use the software is provided. The first major workshop about use of this software (Pasto, Colombia, September 2020) has been cancelled because of COVID-19. 3. Software maintenance (including updates) The latest release is from 2019. 4. Data management (associated with the software) gTOOLS is released with a gravity dataset from Mount St. Helens to help people testing the code. This dataset is public. 5. Hardware associated with software comments The MATLAB version of gTOOLS runs under Windows 10, Linux or MacOS. 112 VDAP-Provided Software and Observatory Decisions – NOVAC Software VDAP Midterm Evaluation Questionnaire 1. Type of software, description per type (What does the application do? Intended audience/User interface). Network for Observation of Volcanic and Atmospheric Change (NOVAC) is the largest instrument network for monitoring gas emissions from volcanoes around the world. The two primary software used by the community to acquire, evaluate, and display SO2 data from the instruments are NOVAC Program (stationary stations) and MobileDOAS (mobile stations). They are used to monitor approximately 42 volcanoes in Costa Rica, El Salvador, Guatemala, Mexico, Nicaragua, Chile, Colombia, Ecuador, Peru, DRC, Iceland, Italy, U.K., Indonesia, Papua New Guinea, and Philippines. In collaboration with partners from Chalmers University in Sweden, the VDAP Computer Scientist has been making several enhancements to the software since 2018. One critical update incorporated new driver package to interface the instrument allowed volcano observatories to use modern computing equipment (64-bit operating systems). Without this update the software risked being phased out. Data analysis is also now significantly more efficient. Other changes helped improve usability and stability of the software. The changes were presented at the 6th NOVAC Workshop held in April 2018. Immediately after the release, the new version of software was used to collect and analyze gas emission data during the Kilauea eruption. 2. Is there on the job training for software users? Training for the NOVAC Software is typically provided at the NOVAC Workshop. Software maintenance (including updates) VDAP, in collaboration with Chalmers University, makes a new release of NOVAC Software with new features and bug fixes about once a year. This is distributed through http://novac￾community.org/. 113 VDAP-Provided Software and Observatory Decisions – Swarm VDAP Midterm Evaluation Questionnaire 1. Type of software, description per type (What does the application do? Intended audience/User interface). Swarm is a visualization and analysis software for seismic activity. It is lightweight, cross-platform, and used by professional and amateur scientists, students, and hobbyists world-wide. It is used daily at all U.S. volcano observatories and used 24/7 in real-time mode for monitoring during major crisis, such as the 2018 Kilauea eruption. It is also used in a similar manner at volcano observatories in many other countries. VDAP seismologists use the software in volcanic crisis management to assist foreign governments. VDAP also teaches Swarm to about 15 volcano scientists from around the world at the Center for the Study of Active Volcanoes each year. Swarm was developed at AVO around 2004, but since 2017 the VDAP Computer Scientist, Diana Norgaard, has been the primary developer and maintainer. The new features added to Swarm include ability to calculate and plot event (e.g., earthquake) locations, tag and classify events in helicorder, import & export events in internationally standardized format, ability to import waveforms in format commonly used in Asia, etc. Numerous bug fixes and minor enhancements have also been made. User documentation has been updated to reflect all changes made since 2011 when it was last updated. The new features were primarily focused on benefiting our partner volcano observatories in developing countries that have few alternatives for seismic event identification and tracking outside of Swarm. Swarm is also bundled as part of Earthworm, an open-source seismic acquisition & processing software. Swarm website: https://volcanoes.usgs.gov/software/swarm/index.shtml 2. Is there on the job training for software users? Swarm is part of CSAV training provided by VDAP. The seismologists may also provide additional training (best to get details from them.) 3. Software maintenance (including updates) VDAP makes periodic releases of Swarm with new features and bug fixes each year. This is distributed through the USGS website. 114 VDAP-Provided Software and Observatory Decisions – Valve VDAP Midterm Evaluation Questionnaire 1. Type of software, description per type (What does the application do? Intended audience/User interface). Valve is a visualization and analysis software for volcano monitoring data. The primary data types supported are for deformation, seismic, and gas; but other time series data can be imported and visualized as well. The primary purpose of the software is to allow users to visualize different types of volcano monitoring data in one place. The system consists of a database and data importer backend and web user interface. The system was developed within the Volcano Science Center many years ago and is currently maintained by a developer at HVO. Although each USGS volcano observatory has an installation, distribution of the software is not done widely by VDAP due to the complexity of installing, configuring, and maintaining the software. 2. Is there on the job training for software users? VDAP Computer Scientist provided onsite installation and configuration assistance to CVGHM (Indonesia) and OVSICORI (Costa Rica) in 2018. On-site training to system administrator and demonstration of the software to users was completed during the installation visit. Technical and user documentation was also written by VDAP and distributed to these observatories. 3. Software maintenance (including updates) Maintenance is currently focused only on bug fixes and is performed by the developer at HVO. On rare occasions when an update is necessary, VDAP aids the partner country in its implementation. 4. Is there a database of participants/attendees to this type of capacity building activity? IT contacts for the software (may be able to provide additional user contacts). Ferry Rusmawan CVGHM ferryrusmaone@gmail.com Christian Garita Hidalgo OVSICORI christian.garita.hidalgo@una.cr 115 VDAP Responses FY 2018 – FY 2020 VDAP responded to 2 volcanic crises during FY 2018. Both were on-site responses. Agung, Indonesia- VDAP began a major on-site response to unrest at Agung, Indonesia on September 18 (during the fourth quarter of FY 2017) that extended to November 20 in the first quarter of FY 2018. Several VDAP team members were already in Indonesia for planning meetings and advanced volcano seismology training when the unrest began. They immediately transitioned and became the first VDAP response team for Agung, assisting with seismic interpretation, assessing monitoring instrumentation needs, and laying the groundwork and planning for two response teams that followed. The second VDAP response team arrived in Bali in early October. They assisted CVGHM with pyroclastic density current modeling, seismic data display and interpretation, GPS data retrieval and interpretation, researching the eruptive history of Agung, developing a conceptual eruptive model, satellite data interpretation, and upgrading and repairing the network of monitoring instrumentation and observation stations around Agung. The second team also researched the possibilities of conducting gas monitoring flights at Agung using instrumentation in helicopters and drones. VDAP response teams assisted counterparts at CVGHM with upgrading and repairing the network of monitoring instrumentation and observation stations around Agung. The third VDAP response team arrived in Bali at the end of October as the second team was departing. They continued to assist with seismic interpretation, improving the monitoring network 116 and observation stations, sampling cold springs around the volcano for chemical analyses that could help determine the presence of magmatic gases, and preparing and using the Aeroterrascan drone for gas monitoring flights. The team departed Bali on November 20. VDAP members met at the Cascades Volcano Observatory on December 14 and conducted a Post Response Assessment (PRA) for the response to Agung. The group discussed what went well during the response as well as suggestions for improvement. The results will be used to help VDAP improve its ability to respond to future volcanic crises. Fuego, Guatemala- On June 3, an explosive eruption at Fuego, Guatemala generated pyroclastic flows that descended multiple drainages on the slopes of the volcano and inundated populated areas. Over 100 people were killed as villages and agricultural fields were buried. It was the largest eruption of the Fuego since 1974. VDAP assistance was requested, and geophysicist Andy Lockhart responded onsite from June 13 through June 30. Many experts from VDAP helped remotely. VDAP assisted counterparts at National Institute of Seismology, Volcanology, Meteorology and Hydrology (INSIVUMEH) in assessing what had happened during this large eruption and what the probabilities of another similar large event occurring could be. VDAP helped counterparts prepare for large lahars that heavy rains began generating through mobilization of the new pyroclastic flow deposits and to recognize areas that could be at risk from future lahars or pyroclastic flows via remote sensing and hazards mapping. VDAP provided remote sensing updates for INSIVUMEH and prepared situation reports for OFDA during the crisis. Andy Lockhart worked on improving communications between government agencies in Guatemala for a better understanding of the risks posed by continuing hazards from Fuego and for an ability to issue warnings of volcanic activity to potentially affected populations. Andy met with counterparts at INSIVUMEH and emergency managers at the National Coordinator for Disaster Reduction (CONRED) to establish an observer system to keep watch on Fuego and to issue alerts when hazardous activity occurred. Andy also worked with INSIVUMEH to determine locations for monitoring instrumentation for improved eruption and lahar detection. Response activities continued through the fourth quarter of FY 2018. Andy Lockhart, Rowdy LaFevers, Jay Wellik, and John Lyons traveled to Guatemala from July 23 to August 5 to assist INSIVUMEH with the installation of new monitoring instrumentation and telemetry, and to train counterparts at INSIVUMEH on the installation, maintenance, and use of the new seismic and infrasound monitoring equipment. Andy continued to meet with managers of government agencies in Guatemala to improve communications between agencies. Andy also discussed the capabilities of the new seismic and infrasound monitoring equipment for detecting lahars and the ability of the cooperating agencies to issue early warnings of lahars to potentially affected populations. Jay trained counterparts at INSIVUMEH on the installation of RSAM alarm software and the interpretation of seismic data. 117 Temporary bridge constructed over Rio Achiguate drainage affected by pyroclastic flows and subsequent lahars from Fuego in Guatemala. Rowdy and John trained counterparts at INSIVUMEH in installation techniques, troubleshooting, and maintenance for new monitoring equipment and telemetry. They assisted counterparts with the installation of an infrasound array at an existing monitoring site. John provided training at INSIVUMEH on the installation of infrasound alarm software and the interpretation of infrasound data. Rowdy assisted with the logistics and installation of two new combined infrasound and seismic monitoring stations located along drainages that were affected by pyroclastic flows on June 3 and continue to be affected by subsequent lahars. Andy returned to Guatemala again from September 10 through the end of the fiscal year to assist counterparts at INSIVUMEH and emergency managers at CONRED with hazard assessment of Barranca Lajas on the east and Rio Cenizas on the south-southwest side of Fuego. Infrastructure and populated areas are at risk from continuing lahars in both drainages. Andy updated the U.S. Ambassador on the situation at Fuego and the institutional challenges and needs at INSIVUMEH. Rowdy returned during September 16 through 21 to assist counterparts at INSIVUMEH with replacing infrasound sensors damaged by heavy rains. He also helped them to harden monitoring installations against lightning. Jay returned during September 18 through 22 to teach a short course in volcano seismology for INSIVUMEH seismologists and volcanologists as well as a few students who may be potential contractors with INSIVUMEH. 118 Counterparts from INSIVUMEH check instrumentation at a newly installed seismic and infrasound monitoring station for Fuego in Guatemala. Remotely, VDAP assisted counterparts at INSIVUMEH in assessing the probabilities of another large eruptive event occurring at Fuego as well as helping to delineate areas at risk from future pyroclastic flows. VDAP also helped to determine areas at risk from future lahars and assisted with the presentation of this information to CONRED so that they can take actions to reduce risk. VDAP continued to provide remote sensing updates for INSIVUMEH and situation reports for OFDA (today USAID/BHA) through the end of FY 2018. VDAP was involved in one remote response and one on-site response during FY 2019. VDAP responded remotely to Morne Plat Pays in Dominica and responded on-site to Ulawun in Papua New Guinea during the third quarter of FY 2019. The remote response to Morne Plat Pays involved the shipment of seismic and telemetry gear to Dominica for installation by counterparts at the Seismic Research Centre (SRC) of the University of the West Indies to monitor volcanic unrest. The SRC had reported that seismic monitoring on the island had been mostly wiped out by Hurricane Maria in late 2017. The new equipment will be installed by SRC staff in consultation with VDAP. The data from these new installations will help counterparts at the SRC to more adequately monitor unrest and forecast potential activity at Morne Plat Pays. 119 The on-site response to Ulawun began as a capacity building trip. VDAP geochemists Christoph Kern and Cindy Werner traveled to Rabaul, Papua New Guinea on June 24, planning to assist counterparts at the Rabaul Volcano Observatory (RVO) in installing 1 to 3 NOVAC-style SO2 monitoring stations on Ulawun volcano, which lies on the border of West New Britain and East New Britain provinces. However, their mission quickly became diverted into an on-site response when Ulawun erupted explosively on June 26, its first eruption in nearly 20 years. The VDAP team spent the next several days at RVO assisting and advising counterparts in their monitoring of the ongoing events at Ulawun. After activity ebbed, Kern traveled to Port Moresby to brief U.S. Embassy and USAID staff on the eruption. The team then traveled to West New Britain where they assisted RVO staff in preparing helicopter gas and photogrammetry surveys of the volcano. They later set up a forward staging area in Bialla, 50 km southwest of Ulawun, and from there they supported RVO engineers in installing a NOVAC gas monitoring station at the Ulamona mission, approximately 11 km northwest of the volcano’s summit. The VDAP team concluded their response by traveling back to Rabual and training RVO staff on how to analyze and interpret data streaming back to the observatory from the new gas instrumentation at Ulamona. Kila Mulina (RVO) discusses strategies for helicopter gas observations with Cynthia Werner and Christoph Kern (VDAP) prior to a gas survey at Ulawun volcano on July 2, 2019. VDAP was involved in two remote responses to volcanic crises in FY 2020. During the second quarter of FY 2020, VDAP responded remotely to a rapid-onset eruption at Taal in the Philippines on January 12. VDAP assisted counterparts at PHIVOLCS with developing a forecast using the probabilistic event-tree method for activity at Taal following the initial explosive eruption. VDAP provided considerable remote sensing support through access to available imagery as well as interpretations derived from classified imagery. VDAP collaborated with counterparts at PHIVOLCS on geodetic and InSAR modeling of ground deformation. 120 Counterparts from PHIVOLCS used a VDAP-donated FLIR camera to measure the temperatures of ground cracks around Taal. Some of the ground cracks appeared on Taal Island while others developed in villages surrounding Taal Caldera, damaging roadways and buildings. Photo courtesy PHIVOLCS. A FLIR camera temporarily donated by VDAP was used by PHIVOLCS to measure the temperature of cracks forming around Taal due to ground deformation to determine if these cracks could become active fissure vents. VDAP also issued 10 IVANS (International Volcano Activity Notification System) situation reports to U.S. Government partners in response to activity at Taal through the month of January. During the third quarter of FY 2020, VDAP responded remotely to an eruption at Sangay in Ecuador. VDAP was already hosting virtual seminar talks with counterparts at IG-EPN in Ecuador when explosive activity and ash emissions at Sangay increased significantly in early June. The virtual meetings shifted focus to a discussion of recent events at Sangay and how monitoring of the volcano could be improved. Existing monitoring sites near the volcano were failing due to ash-covered solar panels preventing batteries from recharging adequately. VDAP assisted counterparts at IG-EPN with developing a strategy for placement of additional instrumentation to monitor the eruption. VDAP provided remote sensing support through access to available imagery and image analysis tools. VDAP also prepared to send a seismic station, two NOVAC DOAS scanners for volcanic gas detection, and corresponding telemetry gear as the third quarter drew to a close. 121 Sangay volcano in Ecuador on June 10, 2020. Photo by Jorge Anhalzer, courtesy Patty Mothes (IG￾EPN). 122 List of Persons Interviewed VDAP Midterm Evaluation National Volcanic Monitoring Systems − Argentina SegemAR OAVV: Sebastián García − Chile SERNAGEOMIN OVDAS: Alvaro Amigo, Carlos Cardona, Rodrigo Ordenes − Colombia SGC: Gloria Patricia Cortés, Cristian López − Costa Rica OVSICORI: Marino Protti − Ecuador EPN IG: Silvana Hidalgo − El Salvador MARN: Carlos Demetrio Escobar − Guatemala INSIVUMEH: Amilcar Roca − Indonesia CVGHM: Devy Kamil Syahbana − Peru IGP-OVS: Luisa Diomira Macedo, Marco Rivera, Jose Alberto del Carpio, Janneth Aiwa, and Ivonne Lazarte − Peru INGEMMET: Rigoberto Aguilar Contreras, Edu Taipe Maquerhua VDAP/USGS (Answered short questionnaires via email) − Battaglia, Maurizio − Griswold, Julia P − Kelly, Peter − Kern, Christoph − Lockhart, Andrew B − Lowenstern, Jacob B − Norgaard, Diana L − Pesicek, Jeremy D − Prejean, Stephanie G USAID – MDROs (Answered short questionnaires via email) − Callaghan, Tim (Costa Rica) − Seuc, Jason (Indonesia) − (Peru) 123 National Agencies Leading Volcano Monitoring Activities Group Countries Group 1 Indonesia, Guatemala, Peru, Colombia, Ecuador, Argentina, Chile, Costa Rica and Mexico Group 2 PNG, El Salvador, DRC, Philippines Group 3 Nicaragua Group 1 Organization Point of Contact Indonesia Centre for Volcanology and Geological Hazard Mitigation (CVGHM) Andiani Djarwoto andianidjarwoto@gmail.com Devy Kamil Syahbana devy.syahbana@gmail.com CVGHM Guatemala INSIVUMEH Amilcar Roca aeroca@insivumeh.gob.gt INSIVUMEH Peru Instituto Geológico Minero y Metalúrgico (INGEMMET) Observatorio Vulcanologico Rigoberto Aguilar Contreras pmasias@ingemmet.gob.pe Edu Taipe Maquerhua edtaipe@ingemmet.gob.pe INGEMMET Instituto Geofísico del Perú IGP Observatorio Vulcanológico del Sur (OVS) Luisa Diomira Macedo lmacedo@igp.gob.pe Hernando Tavera htavera@igp.gob.pe Marco Rivera mrivera@igp.gob.pe Jose Alberto del Carpio, Janneth Aiwa, Ivonne Lazarte IGP Colombia Servicio Geológico Colombiano Martha Lucía Calvache mcalvache@sgc.gov.co Directora de Geoamenazas del Servicio Geológico Colombiano Gloria Patricia Cortés gpcortes@sgc.gov.co Cristian López cmlopez@sgc.gov.co Diego Mauricio Gómez dgomez@sgc.gov.co Observatorio Vulcanológico de Manizales Ecuador Departamento de Geofísica – Instituto Geofísico Escuela Politécnica Nacional IGEPN Silvana Hidalgo shidalgo@igepn.edu.ec IGEPN Argentina Observatorio Argentino de Vigilancia Volcánica (OAVV) Servicio Geológico Minero Argentino (SEGEMAR) Sebastián García sebastian.garcia@segemar.gov.ar SEGEMAR Instituto CEDIAC Universidad de Cuyo, Mendoza Gregorio Boixart - Instituto CEDIAC gboixart@gmail.com Chile Servicio Nacional de Geología y Minería SERNAGEOMIN Alvaro Amigo alvaro.amigo@sernageomin.cl Jefe Red Nacional de Vigilancia Volcánica (RNVV) Carlos Cardona carlos.cardona@sernageomin.cl Rodrigo Ordenes rodrigo.ordenes@sernageomin.cl Jefe Observatorio Volcanológico de los Andes del Sur (OVDAS) Costa Rica Observatorio Vulcanológico y Sismológico de Costa Rica OVSICORI Marino Protti OVSICORI mf.angarita52@gmail.com Mexico Departamento de Vulcanología del Instituto de Geofísica Claudia Rivera – UNAM 124 Group 1 Organization Point of Contact Universidad Nacional Autonoma de Mexico (UNAM) claudia.rivera@ciencias.unam.mx Universidad de Colima Rogelio García - Universidad de Colima rgflores@ucol.mx Group 2 Organization Point of Contact PNG Rabaul Volcano Observatory Ima Itikarai ima_itikarai@mineral.gov.pg RVO El Salvador Dirección General del Servicio Nacional de Estudios Territoriales, Ministerio de Medio Ambiente y Recursos Naturales DGSNET-MARN Carlos Demetrio Escobar descobar@marn.gob.sv DGSNET-MARN University of El Salvador Rodolfo Olmos rolmos99@yahoo.com U. El Salvador DRC Goma Volcano Observatory Geochemistry and Environmental Department 142, Avenue du Rond-Point Quartier des Volcans, Goma-RD Congo Charles Balagizi, PhD balagizi.charles@gmail.com Goma Volcano Observatory Philippines Philippine Institute of Volcanology and Seismology (DOST￾PHIVOLCS) Geology and Geophysics Research and Development Division Bryan J. Marfito bryan.marfito@gmail.com Science Research Specialist I Earthquake Section Group 3 Organization Point of Contact Nicaragua Instituto Nicaragüense de Estudios Territoriales INETER Eveling Espinoza eveling.espinoza@gf.ineter.gob.ni – INETER 125 VDAP Donated Instrumentation FY 2018‐2020 Fiscal Year Volcano Item Date Quantity Donee Was equipment installed with VDAP assistance? Country FY 2018 Quarter 1 Agung UAS multigas Nov-17 1 CVGHM yes Indonesia FY 2018 Quarter 1 Agung Multigasstation Nov-17 1 CVGHM no (conditions at volcano too hazardousforinstallation) Indonesia FY 2018 Quarter 1 Agung Campaignmultigas Nov-17 1 CVGHM yes(portable equipment for use throughoutIndonesia) Indonesia FY 2018 Quarter 2 Fuego Broadband seismic station Mar-18 1 INSIVUMEH yes Guatemala FY 2018 Quarter 4 Rabaul seismic monitoring stations Jul‐18 2 RVO No Papua New Guinea FY 2018 Quarter 4 Rabaul deformationGPS Jul‐18 2 RVO No Papua New Guinea FY 2018 Quarter 4 Fuego seismic and infrasound Jul‐18 2 INSIVUMEH Yes Guatemala FY 2018 Quarter 4 San Miguel seismic monitoring station Jul‐18 1 MARN No El Salvador FY 2019 Quarter 1 Fuego Observatory opsroom equipment and telemetry Oct‐18 1 INSIVUMEH No Guatemala FY 2019 Quarter 1 Sabancaya DOAS Oct‐18 1 INGEMMET No Peru FY 2019 Quarter 2 Fuego Broadbandseismometers (along with 6 infrasound detectors) Feb‐19 2 INSIVUMEH No Guatemala FY 2019 Quarter 3 Morne Plat Pays seismic and telemetry gear Apr‐19 3 SGC No Dominica FY 2019 Quarter 3 Ulawun NOVAC scanning DOAS and telemetry Apr‐19 3 RVO Yes Papua New Guinea FY 2019 Quarter 3 San Salvador analog upgrade equipment for observatory Apr‐19 3 MARN Yes El Salvador FY 2019 Quarter 3 Cotopaxi 3 repaired seismometers and 4 rain gauges Apr‐19 1 IG‐EPN Yes Ecuador FY 2019 Quarter 3 Popocatepetl 4 seismometers and 3 rain gauges Jun‐19 4 CENAPRED No Mexico FY 2019 Quarter 3 CerroQuemado seismic station with rain gauge Jun‐19 1 INSIVUMEH Yes Guatemala FY 2019 Quarter 4 Rabaul seismic stations 01/07/2019 2 RVO No Papua New Guinea FY 2019 Quarter 4 Cotopaxi portable MultiGAS 01/07/2019 1 IG‐EPN No Ecuador FY 2019 Quarter 4 San Salvador portable MultiGAS 01/07/2019 1 MARN No El Salvador FY 2019 Quarter 4 Nyiragongo portable MultiGAS 01/07/2019 1 GVO No Democratic Republic of the Congo FY 2020 Quarter 1 Nevados de Chillan dredge 17/10/2019 1 SERNAGEOMIN No Chile FY 2020 Quarter 1 Fuego IT supplies and lightning protection 25/10/2019 1 INSIVUMEH No Guatemala FY 2020 Quarter 1 multiple MultiGAS and DOAS spare parts, plus water sampling equipment. 14/11/2019 1 MARN No El Salvador FY 2020 Quarter 2 Turrialba moxa computer for NOVAC scanning DOAS 31/03/2020 1 OVSICORI No Costa Rica FY 2020 Quarter 2 Taal FLIR camera 29/01/2020 1 PHIVOLCS No Philippines FY 2020 Quarter 3 Multiple in Ecuador Calibration Gases(116L): 10 ppm SO2, 3000 ppm CO2 (Qty 3), 10 ppm H2S (Qty 3), 4 SS DFR regulators 02/06/2020 6 IG‐EPN No Ecuador FY 2020 Quarter 4 Sangay Seismic station 23/07/2020 1 IG‐EPN No Ecuador FY 2020 Quarter 4 Sangay Scanning DOAS 23/07/2020 2 IG‐EPN No Ecuador FY 2020 Quarter 4 Sangay Telemetry repeater gear 23/07/2020 1 IG‐EPN No Ecuador FY 2020 Quarter 4 Fuego Digitizers and telemetry gear 21/07/2020 1 INSIVUMEH No Guatemala FY 2020 Quarter 4 Rabaul Accessoriesfor digitizers 18/08/2020 1 RVO No Papua New Guinea 126 References Banks, Norman G. 1987. «Volcano Early Warning and Disaster Assistance Program (VDAP)». First Annual Report. Vancouver, WA: U. S. Geological Survey. ———. 1989. «Volcano Early Warning and Disaster Assistance Program (VDAP)». Second Annual Report. Vancouver, WA: U. S. Geological Survey. Berman, Laine, Ann von Briesen Lewis, John Lockwood, Erlinda Panisales, y Joeni Hartanto. 2012. «Evaluation of The USAID/OFDA USGS Volcano Disaster Assistance Program in Indonesia». Evaluation. Washington D.C.: This publication was produced at the request of the United States Agency for International Development. It was prepared independently by International Business & Technical Consultants, Inc. (IBTCI). Ewert, John W., Marianne Guffanti, y Thomas L. Murray. 2005. «An Assessment of Volcanic Threat and Monitoring Capabilities in the United States: Framework for a National Volcano Early Warning System». Open-File Report. Open-File Report. Reston, VA: U.S. Geological Survey. Ewert, John W., Angela K. Diefenbach, y David W. Ramsey. 2018. «2018 Update to the U.S. Geological Survey National Volcanic Threat Assessment». Open-File Report 2018-5140. Open-File Report. Reston, VA: U.S. Geological Survey. Fitzpatrick, Jessica. 2016. «30 Years Saving Lives from Volcanoes». USGS.gov. News (blog). 2 de agosto de 2016. https://www.usgs.gov/news/30-years-saving-lives-volcanoes. Hammelton, Raynal, Michael James, Richard Greene, y Grant Heiken. 1993. «Evaluation of The Volcano Disaster Assistance Program in Indonesia Guatemala, Ecuador an the Philippines». Evaluation. Washington D.C.: United States Agency for International Development, Office of U.S. Foreign Disaster Assistance, Division of Prevention, Mitigation and Preparedness. Lambert, Amber D., and Angie L. Miller. 2014. "Lower Response Rates on Alumni Surveys Might Not Mean Lower Response Representativeness". Educational Research Quarterly 37 (3): 40-53 Miller, Dan, John W. Ewert, y Andrew B. Lockhart. 1993. «The USGS/USAID Volcano Disaster Assistance Program: The Next Five Years». Vancouver, WA: U.S. Geological Survey Cascades Volcano Observatory. Moran, S.C., Freymueller, J.T., LaHusen, R.G., McGee, K.A., Poland, M.P., Power, J.A., Schmidt, D.A., Schneider, D.J., Stephens, G., Werner, C.A., and White, R.A., 2008, Instrumentation recommendations for volcano monitoring at U.S. volcanoes under the National Volcano Early Warning System: U.S. Geological Survey Scientific Investigations Report 2008-5114, 47 p. Pallister, John S. 2015. «Volcano Disaster Assistance Program: Preventing volcanic crises from becoming disasters and advancing science diplomacy». En Global Volcanic Hazards and Risk, 379-85. Cambridge, U.K.: Cambridge University Press. Tabachnick BG, Fidell LS. Using Multivariate Statistics. 4th ed. Allyn & Bacon; 2001. doi:10.1037/022267 USAID/OFDA. 2018. Geological Hazards Sector Update. http://www.usaid.gov/what-we-do/working-crises￾and-conflict/responding-times-crisis. USGS. 1998. «Mobile Response Team Saves Lives in Volcano Crises». 064-97. USGS Fact Sheet. Vancouver, WA: U.S. Geological Survey. 127 Annex V: Disclosure of Conflicts of Interest Name Juan Pablo Sarmiento Title Associate Director for Research, Extreme Events Institute Organization Florida International University Evaluation Position? ☒ Team Leader Team member Evaluation Award Number (contract or other instrument) Cooperative agreement #720FDA20CA00023 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) Volcano Disaster Assistance Program (VDAP) I have real or potential conflicts of interest to disclose. Yes ☒ No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date April 27, 2021 128 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523