FINAL PROGRAM RESULTS REPORT: PROMOTING RISK AND EMERGENCY PREPAREDNESS (PREP) JUNE 2017 – NOVEMBER 2018 November 30, 2018 FINAL PROGRAM RESULTS REPORT: PROMOTING RISK AND EMERGENCY PREPAREDNESS (PREP) JUNE 1, 2017 – NOVEMBER 30, 2018 AGREEMENT NUMBER: AID-OFDA-G-17-00086 CHIEF OF PARTY: MARTIN ROMAN EVALUATOR: SERGIO URIOSTE, INDEPENDENT CONSULTANT PHOTOGRAPH CREDITS: ACDI/VOCA DISCLAIMER: The opinions expressed in this document are those of the author and do not necessarily represent the views of the United States Agency for International Development or those of ACDI/VOCA. EVALUATOR’S THANKS: The evaluator would like to recognize the efforts of PREP program staff, who provided support and time for the development of his activities. The evaluator also thanks the program beneficiaries, local leaders, and members of the Municipal Council for Risk Management and Reduction (MCRMR) of Caazapá and Nanawa, and the community risk management teams (CRMTs), for offering their time and knowledge to discuss evaluation topics. Promoting Risk and Emergency Preparedness (PREP) Final Program Results Report November 2018 i TABLE OF CONTENTS Executive Summary ..........................................................................................................................................1 Introduction .......................................................................................................................................................3 PROBLEM STATEMENT .......................................................................................................................3 PROGRAM BACKGROUND ..............................................................................................................3 PROGRAM RELEVANCE AND PERTINENCE ...........................................................................4 Qualitative Assessment of Activities............................................................................................................5 SECTOR: RISK MANAGEMENT POLICY AND PRACTICE ...............................................5 OUTCOME 1 – INFORMATION SYSTEMS IMPROVED ...................................................5 OUTCOME 2 – DISASTER RISK MANAGEMENT ORGANIZATIONS STRENGTHENED ...............................................................................................................................7 SECTOR: AGRICULTURE AND FOOD SECURITY ............................................................ 12 OUTCOME 3 – SUSTAINABLE FOOD PRODUCTION INCREASED..................... 12 Quantitative Analysis of Indicators............................................................................................................ 19 Indicator 1: Number of agroclimatic risk and socioeconomic vulnerability maps developed .................................................................................................................................................................. 19 Indicator 2: Number of people trained in disaster preparedness, mitigation, and management........................................................................................................................................... 20 Indicator 3: Number of people passing final exams or receiving certificates.......................... 20 Indicator 4: Percentage of people trained who retain skills and knowledge after two months.................................................................................................................................................... 20 Indicator 5: Number of people benefiting from seed systems/agricultural input activities.. 21 Indicator 6: Number of project households with increased access to targeted vegetables produced................................................................................................................................................. 21 Indicator 7: Number of floating gardens implemented in the target area as a result of program technical assistance.............................................................................................................. 21 Conclusions and Recommendatins............................................................................................................ 23 Annexes........................................................................................................................................................... 25 ANNEX A. EVALUATION METHODOLOGY................................................................................. 25 ANNEX B. QUESTIONNAIRE TO EXPERTS ................................................................................... 29 Promoting Risk and Emergency Preparedness (PREP) Final Program Results Report November 2018 ii ACRONYMS CRMT Community Risk Management Team DANA Damage Assessment and Need Analysis DBRMR Departmental Board for Risk Management and Reduction of Caazapá DEAG Agrarian Extension Directorate DMH National Hydrology and Meteorology Directorate DRR Disaster Risk Reduction EWS Early Warning System FAB First Aid Brigade FAO Food and Agriculture Organization (of the United Nations) FCA/UNA Faculty of Agronomy in the National University of Asunción FCS Food Consumption Score FFRB Forest Fire Response Brigade GOP Government of Paraguay HFIAS Household Food Insecurity and Access Scale HHS Household Hunger Scale IICA Inter-American Institute for Cooperation on Agriculture INFONA National Forestry Institute ISS Integrated Sensor Suite LOP Life of Project M&E Monitoring and Evaluation MCRMR Municipal Council for Risk Management and Reduction NPDRMR National Policy on Disaster Risk Management and Reduction OFDA Office of U.S. Foreign Disaster Assistance PREP Promoting Risk and Emergency Preparedness Program RM Risk Management SEN National Emergency Secretariat SENACSA National Animal Health and Quality Service SENAMHI National Meteorological Service of Peru SSAEC Secretariat of Social Action and Emergency of Caazapá ToR Terms of Reference MAG/RMU Risk Management Unit of the Ministry of Agriculture USAID United States Agency for International Development Promoting Risk and Emergency Preparedness (PREP) Final Program Results Report November 2018 1 EXECUTIVE SUMMARY The purpose of the final program results report is to assess the impact and measure the results and indicators of the Promoting Risk and Emergency Preparedness (PREP) program, implemented by ACDI/VOCA and funded by the United States Agency for International Development’s Office of Foreign Disaster Assistance (USAID/OFDA). The evaluator’s opinions expressed in this document are supported by documental and testimonial evidence provided by local authorities and beneficiary populations in the program’s intervention area, as well as ACDI/VOCA Paraguay staff. The evaluator conducted a document review to provide background and a logic framework for the evaluation, held consultative meetings with key stakeholders, and conducted focus group discussions and surveys with beneficiaries. The primary objective of the PREP program was to increase the ability of the Government of Paraguay (GOP) and vulnerable households in program intervention areas to prepare for, mitigate, and adapt to shocks and stresses caused by natural hazards using structural and nonstructural strategies that can be replicated in other communities. Based on quantitative and qualitative evidence, the evaluator concludes that PREP has met this objective. Specifically, the program successfully: ▪ Strengthened the GOP’s multilevel linkages and abilities for effective disaster risk reduction (DRR) preparedness and response at all levels. GOP institutions are now in better condition to manage risks and respond to disasters in the target regions. ▪ Strengthened vertical linkages among DRR institutions, in accordance with the National Policy on Disaster Risk Management and Reduction (NPDRMR). ▪ Developed tools, procedures, guidelines, protocols, and best practices that have been made available to the GOP and stakeholders and can be replicated in other regions in Paraguay. ▪ Improved knowledge and understanding of risk management options and strengthened monitoring capacities of risk management initiatives. ▪ Contributed to strengthening the ability of vulnerable households to prepare for, mitigate, and adapt to shocks and stresses caused by natural hazards through a people￾centered, multisector approach. In addition, PREP created and incorporated a culture of prevention and resilience to natural disaster events in communities and families. ▪ Contributed to increased sustainable food production for households vulnerable to cyclical climatic events in target intervention areas, decreasing the probability of an acute decline in food access or consumption due to weather disturbances through a prevention and mitigation strategy. PREP provided national government and local actors with tools and capacities to proactively monitor and manage risks, and it built capacity to strengthen national and local resilience to disaster risks. At the national level, PREP strengthened DRR governance systems by improving information systems (e.g. development of agroclimatic risk and socioeconomic vulnerability maps, national water balance model). At the local level, PREP strengthened institutional capacities through workshops, technical assistance, and in-kind grants for recently formed Municipal Councils for Risk Management and Reduction (MCRMRs) and Community Risk Management Teams (CRMTs), volunteers, and other stakeholders. The workshops covered relevant topics such as: MCRMR and CRMT creation and capacity building; DRR concepts and DRR tool application for mainstreaming in development programs, plans, and policies; early recovery; emergency response strategic plans; and contingency planning. PREP also implemented local capacity development to increase sustainable food production for households vulnerable to cyclical climatic events. Promoting Risk and Emergency Preparedness (PREP) Final Program Results Report November 2018 2 Key takeaways and lessons learned from program implementation include: • To avoid duplication of efforts, it is effective to coordinate stakeholders on an inter-institutional platform targeted at improving information systems and identifying synergies between sectors. • Positive program results can leverage funding from public and private donors for additional activities beyond the scope of the program, such as the World Banks’s complementary funding for the water balance model and risk map development. • Under an institutional framework like the CRMT, organized communities can and do play a key role in reducing the impact of, preparing for, and responding to disasters. • Providing DRR training to CRMTs created a new language to proactively manage situations before, during, and after a disaster and helped them understand DRR and the terminology used in relation to risk, vulnerability, and hazards. • The participation of CRMT representatives in MCRMRs provides an opportunity for vulnerable communities to represent their concerns at a municipal level and play a role in developing preventive approaches, instead of solely receiving top-down emergency response support. • Diversified and secure access to food can be achieved through horticulture plots at the household level. The year-round availability of vegetables, such as lettuce, tomatoes, cabbage, bell peppers, and beets, helped provide new sources of vitamins, minerals, and protein. The following recommendations aim to contribute to the sustainability of PREP activities: • The National Emergency Secretariat (SEN) and local governments should schedule new training processes and allocate resources to reinforce newly acquired knowledge, support the newly formed institutional structures, and create long-lasting changes in practices and attitudes within MCRMRs and CRMTs. • Municipal and departmental authorities must follow up by providing political support to MCRMRs and Departmental Board for Risk Management and Reduction (DBRMR) to maintain, increase, and expand the activities and goods generated by PREP. • Future DRR programming should build on this program’s successful development of structural and non-structural solutions to address DRR management. Beneficiaries from Caracarai proudly showing their vegetable garden in full production stage. Promoting Risk and Emergency Preparedness (PREP) Final Program Results Report November 2018 3 INTRODUCTION The objective of PREP’s final evaluation is to measure final quantitative and qualitative results and indicators and to provide conclusions and recommendations. To achieve the objective, this evaluation aims to respond the following questions: • Have PREP’s activities contributed to strengthening the ability of the GOP and vulnerable households to prepare, mitigate, and adapt to shocks and stresses caused by natural hazards using structural and nonstructural strategies? • Can PREP’s strategy be replicated in other communities? • Have the tools and capacities provided by PREP helped communities, local governments, and organizations to proactively monitor and manage risks? • Has PREP contributed to increasing sustainable food production for households vulnerable to cyclical climatic events? The evaluator conducted surveys, in-depth interviews, and focus group discussions with a total of 221 persons, including local government authorities, community leaders, and beneficiaries in the intervention areas to collect quantitative and qualitative data to answer the evaluation’s questions. For detailed information regarding the evaluation methodological design and application, refer to Annex A. PROBLEM STATEMENT Paraguay has experienced severe climate variances in recent years, testing the GOP’s ability to manage and mitigate disaster risks and vulnerable households’ ability to manage shocks and stresses caused by natural hazards. According to the World Bank report “Agricultural Sector Risk Analysis for Paraguay (2018)”, poverty in Paraguay is directly linked to climatic adverse events with catastrophic impacts for small producers, due to their low capacity to manage agricultural risk. Frosts, droughts, and heavy rains, among other events, affect smallholders’ subsistence due to their insignificant level of participation in the different agricultural value chains, and the fact that these adverse climatic events often force them to leave their crops and households to evacuate to safer areas. In conclusion, smallholder farmers often absorb the greatest weather-induced agriculture-sector losses, given their lack of adequate risk management mechanisms. The GOP established institutions, policies, and resources to manage the country’s vulnerability to natural disasters, such as the National Emergency Secretariat (SEN) and the National Policy on Disaster Risk Management and Reduction (NPDRMR) at the national level, and the Secretariat of Social Action and Emergencies of Caazapá (SSAEC) at the departmental level. However, these institutions do not have enough financial and human resources and tools to effectively implement this strategy at the departmental, municipal, and community levels. Paraguay’s departmental and municipal institutions often do not have clear DRR prevention protocols or action plans. The centralized, Asunción-based SEN does not have prepared local focal points to collaborate with most departmental and municipal institutions, and therefore cannot provide sufficient regional coverage to towns and rural communities. In addition, departmental and municipal governments and communities affected by emergencies often demonstrate a low level of preparation and adaptation capacity to cope with cyclical climatic emergencies and continue to suffer from the same vulnerabilities year after year. The agricultural sector is particularly susceptible to these climatic events. PROGRAM BACKGROUND In June 2017, ACDI/VOCA began implementation of the PREP program, which aimed to strengthen the ability of institutions, communities, and households to prepare, mitigate, and adapt to shocks and stresses caused by natural hazards through a combination of training, technical assistance, and critical in- Promoting Risk and Emergency Preparedness (PREP) Final Program Results Report November 2018 4 kind grants. It closely worked with the GOP (national and local) and other stakeholders to build skilled human resources, improve systems for information collection and analysis, and promote strong vertical linkages between communities and institutions at the municipal, departmental, and national levels. PREP coordinated with local stakeholders to ensure long-term sustainability in target geographic areas. The program designed activities to be replicated in other parts of the country in order to help communities and institutions across Paraguay achieve a greater level of disaster preparedness, risk mitigation, and adaptation to climate-induced shocks and stresses. The program focused its activities in the departments of Caazapá and Presidente Hayes in Paraguay. In the department of Caazapá, PREP worked in 10 target communities within the municipalities of Caazapá, Coronel Maciel, Fulgencio Yegros, and Tres de Mayo. In the department of Presidente Hayes, PREP targeted the municipality of Nanawa; as well as the neighboring agricultural community of Beterete Cué. The program focused its activities on two sectors to accomplish this objective: ➢ Sector 1: Risk Management Policy and Practice: Providing the national government and local actors (local government and communities) with tools and capacities to proactively monitor and manage risks. ➢ Sector 2: Agriculture and Food Security: Increasing sustainable food production for households vulnerable to cyclical extreme climatic events. PROGRAM RELEVANCE AND PERTINENCE ACDI/VOCA designed PREP to align with SEN’s 2015–2018 Strategic Plan and the NPDRMR guidelines, which both highlight the need to update national databases and produce valid risk scenario models for risk management resource allocation and decision-making. All Risk Management Policy and Practice sector activities respond to three main components of the NPDRMR: i) strengthening institutional capacities, ii) education, communication, and citizen participation, and iii) knowledge management and technology. PREP’s contribution to the NPDRMR included the design, development, production, and launch of technical and computer models to analyze risks in the agricultural sector for national and local decision-makers. In addition, PREP’s work in decentralizing DRR from the national through the departmental, municipal, and community levels aligns with NPDRMR guidelines. At the local level, PREP provided structural and nonstructural solutions to increase households’ sustainable food production, especially to prepare vulnerable families for cyclical climatic events. PREP provided tailor-made solutions and training based on the needs found during the baseline and considering the local environment, availability of materials, animals raised in the region, dietary habits, and households’ expectations. PREP provided technical assistance for food security activities for each beneficiary through personalized home visits conducted by locally hired technicians, using a peer-to-peer methodology and in local Guaraní language. Finally, PREP’s design and start-up activities are the result of a participatory process whereby ACDI/VOCA engaged with local stakeholders, community organizations, and municipal and departmental authorities to orient and inform activities. During program start-up, ACDI/VOCA coordinated the selection of the intervention areas, target population, and the relevance of proposed activities through field visits, studies, and data collection with potential beneficiaries and authorities. PREP presented and discussed the information with GOP stakeholders and received approval from USAID/OFDA. Promoting Risk and Emergency Preparedness (PREP) Final Program Results Report November 2018 5 QUALITATIVE ASSESSMENT OF ACTIVITIES SECTOR: RISK MANAGEMENT POLICY AND PRACTICE To increase institutional planning capacity and allow for the decentralization of DRR institutional response capacities from the national to the local level, PREP promoted initiatives; developed tools, procedures, manuals, and systems; and trained national and local personnel. The key takeaways under this sector are: • PREP reinforced the roles of the MAG and SEN, in line with their respective national policies and strategies, through the design, development, and launch of technical and computer models for national or local decision making for risk management and disaster risk reduction. • To avoid duplication of efforts, it is effective to coordinate stakeholders on an inter-institutional platform targeted at improving information systems and identifying synergies between sectors. • Involving GOP technicians starting at design and during the progressive stages of tool development contributed to the sustainability and empowerment of local institutions. • Positive program results can leverage funding from public and private donors for additional activities beyond the scope of the program, such as the World Banks’s complementary funding for the water balance model and risk map development. • Under an institutional framework like the CRMT, organized communities can and do play a key role in reducing the impact of, preparing for, and responding to disasters. • Providing DRR training to CRMTs created a new language to proactively manage situations before, during, and after a disaster and helped them understand DRR and the terminology used in relation to risk, vulnerability, and hazards. • The participation of CRMT representatives in MCRMRs provides an opportunity for vulnerable communities to represent their concerns at a municipal level and play a role in developing preventive approaches, instead of solely receiving top-down emergency response support. OUTCOME 1 – INFORMATION SYSTEMS IMPROVED Technological tools, such as water balance models and risk maps, increased the GOP’s capacities to produce, understand, and update risk scenarios before, during, and after climatic events. Through a partnership with the Inter-American Institute for Cooperation on Agriculture (IICA), PREP created national-level agricultural risk maps and municipal-level socioeconomic vulnerability maps in Caazapá. PREP trained 400 technicians—hailing from the public and private sectors in Asunción as well as the three most agriculturally productive departments of the country—to use the tools to integrate information regarding climatic events, increasing their ability to engage in data-driven decision-making and to develop appropriate DRR programs for vulnerable communities. Develop agroclimatic risk and socioeconomic vulnerability maps In order to develop these maps, PREP created a national water balance model, conducted a national characterization of family agriculture and livestock production, and gathered socioeconomic and food security data. Creation of a water balance model. PREP created the water balance tool with support from IICA and the World Bank, and the technical leadership of the National Hydrology and Meteorology Directorate Promoting Risk and Emergency Preparedness (PREP) Final Program Results Report November 2018 6 (DMH) and the Risk Management Unit of the Ministry of Agriculture (MAG/RMU). The technical partners helped PREP to determine the daily water requirement of eleven family farming, pasture-based, and technical crops, including corn, wheat, soybeans, cassava, sesame, beans, cotton, sweet potatoes, canola, peanuts, and sugar cane. This tool can analyze historical information about each crop, from sowing to harvest, to determine the water content in the soil for the selected crop at a specific time and place. This information is contrasted with the real need for water that the plant requires at each stage of its development, to determine if there is a water deficit or excess. To create this tool, PREP carried out an exhaustive process of research and data collection of three specific components: 1) the climatic component with hydrometeorological information of the DMH of the last 30 years; 2) soil component, including information on soil types at the country-level; and 3) a crop component (sowing dates, phenological phases, and depth of roots). The DMH and MAG/RMU secured the sustainability of this activity by involving their technicians in all processes from the activity’s outset, and by signing inter-institutional cooperation agreements with PREP and IICA, in which they committed to supply virtual space in their server for software installation and to assign programmers for system maintenance and use. As evidence of this model’s usefulness and replicability, the World Bank joined PREP’s initiative by investing approximately $100,000 to expand the coverage of the national water balance model, including the analysis of cash crops, amplifying the positive impact of this activity in the country. National characterization of family agriculture and livestock production. Required for the agricultural risk maps, PREP developed a characterization of family agriculture and livestock production at the national level. The characterization contains information about critical periods of cultivation for the main family agricultural crops and measures their vulnerability levels to adverse meteorological phenomena with special emphasis on drought, flooding, and frost. Similar information is available for livestock production and pasture management in smallholder agriculture, which is important to cope with their susceptibility to adverse climatic events. The national characterization of livestock production in smallholder agriculture and pasture management includes a national-level map of soil use capacity and land use planning to assess soil vulnerability. The water balance model tool is live on the web and available for public use on the website http://bhag.meteorologia.gov.py/. By performing a statistical analysis of this information, users can determine the level of climatic risks that could affect a specific crop in a given area. For example, if this tool shows that corn has a water availability below 50 percent in a specific region, it means that it is affected by a water deficit (stress by drought). When the water availability exceeds 100 percent, it shows that the soil does not have the capacity to continue storing water. These factors can affect crop yields, and the analysis of this information allows technicians and smallholder producers to take preventive measures such as determining the best time to sow, or to take corrective measures such as determining when to irrigate the crops. It also allows producers to monitor the climate-soil-plant system, the probability of extreme events, planning and zoning in productive regions, and the determination of climatic indexes associated with agricultural insurance. The collection, systematization, and processing of relevant existing data from national registers for the national characterization of family agriculture and livestock production is the result of a participatory, coordinated process of the risk management working group created on MAG/RMU’s initiative. This working group seeks to bring together local actors; GOP national institutions such as the DMH, SEN, the General Directorate of Statistics, Surveys, and Census; international organizations such as USAID, the World Bank, FAO, and IICA; and academia including the National University of Asunción and the Catholic University of Asunción. Promoting Risk and Emergency Preparedness (PREP) Final Program Results Report November 2018 7 Risk map development and training. The national-level agricultural risk maps show a geographical representation of the potential effects of a climatic event in the agriculture and cattle sectors. These risk maps are the result of combining the existing information mentioned in steps one and two and using algorithms to analyze the level of susceptibility of selected crops or animals to adverse weather events. To analyze the information available, PREP developed a multi-criteria analysis database including a national-level geographic information system (GIS) that integrates information for analysis and management of agroclimatic risks. It can measure the risks of drought, water deficits, water excess, frost, or thermal stress of the eleven crops, including at the level of family agriculture or cash crops, as well as for natural or implemented pasture, and of cattle, pigs, goats, sheep, and poultry. Through this tool, PREP generated 500 maps, including probability, risk, soil degradation, and auxiliary maps. Socioeconomic vulnerability map development and training. To align disaster risk management planning between the national and local levels, PREP developed smallholder farmer socioeconomic vulnerability and food security maps in all 11 municipalities of Caazapá. These maps were created by layering additional municipal data—such as socioeconomic metadata from the Agricultural Extension Directorate’s register of assisted family farming and from the social cabinet of the presidency’s integrated social information system—onto the databases of information consolidated for the aforementioned national-level risk maps. For the 11 municipal socioeconomic vulnerability maps created for Caazapá, PREP produced a manual of procedures for data quality control and database use/updating. In addition, PREP conducted various training sessions and workshops with GOP stakeholders to present and disseminate the maps and explain the procedures, techniques, and technology used in the maps. OUTCOME 2 – DISASTER RISK MANAGEMENT ORGANIZATIONS STRENGTHENED To improve disaster risk response and recovery at the local level, the GOP has prioritized the decentralization of DRR services from SEN’s national level to the local level by improving the communication channels between the local, departmental, and national levels. In addition, the NPDRMR PREP widely disseminated the national-level risk maps at the national and regional levels. At the local level, several cooperatives and production associations approached the MAG/RMU, requesting to sign cooperative agreements to advise them on the use of the maps. The Itaipu Binational Dam also requested the same. Private insurance companies requested access to the information to generate or improve their agricultural insurance products. At the international level, peer organizations recognized the MAG/RMU’s achievement in preparing the product, including the Ministries of Agriculture of Argentina, Uruguay, and Chile. The World Meteorological Organization (WMO) presented the tools as an achievement at its regional meeting in Argentina, and the National Meteorological Service of Peru requested support and cooperation to set up a similar tool in Peru. This demonstrates the usefulness of PREP’s tools, with the potential to produce a valid risk scenario model for decision-making that can be replicated throughout the region. Over 400 technicians from the private and public sectors were trained to develop and use agricultural risk maps. Promoting Risk and Emergency Preparedness (PREP) Final Program Results Report November 2018 8 prioritizes moving away from a reaction-oriented disaster response to a comprehensive, prevention￾oriented approach based on management and reduction of risks. Under this framework, PREP engaged in five concrete activities: 1) create and strengthen MCRMRs, 2) create First Aid Brigades (FABs), Damage Assessment and Need Analysis (DANA) teams, and Forest Fire Response Brigades (FFRBs), 3) create CRMTs, 4) Strengthen vertical linkages among DRR institutions, and 5) implement early warning systems (EWS). Ultimately, PREP assisted these local DRR platforms to develop the tools and capacities to proactively monitor and manage risks in their respective regions. As a result, PREP strengthened Paraguay’s national disaster risk management institutional capacity. Create and Strengthen MCRMRs According to the guidelines of the risk management and reduction tool kit developed by SEN, and ACDI/VOCA’s previous DRR experience, evidence shows that MCRMRs that effectively involve local authorities, the private sector, and community leaders have a better chance of identifying risks that affect their region and proposing preventive measures and potential solutions. Under this framework, PREP formed and strengthened four MCRMRs in the target municipalities of Caazapá and strengthened the recently established MCRMR in Nanawa. Through a participatory process, the five MCRMRs selected a mix of local actors to join their technical and operational committees, including municipal authorities, representatives from the private sector, police and health personnel, representatives of the Agrarian Extension Directorate (DEAG), firefighters, teachers, university students, and community members. Through signed resolutions, municipal and departmental authorities formally acknowledged each MCRMR as a municipal entity. In addition, PREP worked with each MCRMR to draft a budget proposal for their municipal board to allocate financial resources to the newly formed councils. At the time of this report, three of the five MCRMRs had obtained approval of their municipal budget resolutions. All the MCRMRs have adopted procedures, manuals, statutes, and public resolutions following best practices. In Nanawa, PREP promoted operational agreements between the municipalities’ three main DRR actors—the mayor, the MCRMR, and the municipal board—to ensure sustainable cooperation. To support MCRMR administrative capacities, PREP distributed important in-kind grants (e.g. IT equipment) to the five MCRMRs, while the municipal governments assigned office spaces to coordinate operations and hold meetings. To strengthen disaster response capacity, PREP provided emergency response inputs (e.g. corrugated metal rooftops, mattresses, blankets, wood beams), while the municipal governments provided warehouse spaces to safely store these items. PREP also provided uniforms and banners to each MCRMR to strengthen its institutional framework, and to promote their visibility as focal points in risk management in their districts. Five MCRMRs were created and strengthened in four municipalities of Caazapa and in Nanawa Promoting Risk and Emergency Preparedness (PREP) Final Program Results Report November 2018 9 Create FABs, DANA Teams, and FFRBs Once MCRMRs were established, PREP proceeded to build an operative framework by forming specialized brigades within each MCRMR that ensure a coordinated institutional response during emergencies. PREP formed one FAB and one DANA team within each MCRMR in the four target municipalities in Caazapá and in the municipality of Nanawa. PREP additionally formed one FFRB within each of the four Caazapá MCRMRs, as the department is particularly affected by forest fires. PREP ensured that the newly formed brigades have the necessary emergency response skills by implementing USAID/OFDA's RDAP courses conducted by certified instructors from the National Training Network in Emergencies and Disasters (RED), in coordination with SEN’s training unit. These trainings—including the basic life support course (BLSC), the DANA course, and the Basic Forest Firefighter Course (BFFC)—prepared participants to handle emergency situations that occur in their communities and to disseminate their knowledge to their community members and neighbors. The courses had significant participation by women. For instance, women composed 59 percent of the participants in the BLSC and 32 percent of the DANA courses. After the training courses, PREP developed manuals and protocols for the specialized brigades and provided in-kind equipment to facilitate their fieldwork, including first aid equipment for FABs, tactical backpacks for the DANA teams, and firefighting equipment for the Caazapá FFRBs. Create CRMTs PREP formed 10 CRMTs in target communities in Caazapá.1 CRMTs are a proven mechanism to decentralize disaster risk planning at the community level, as they provide a link between local families and the MCRMRs. More importantly, CRMTs are the primary source of real-time information on local conditions during emergencies. PREP understood the importance of local knowledge as one of the main sources of information, and developed their strategy, including CRMT formation, with local actors. In these groups, community representatives create the official criteria for how their community will prevent and respond to emergencies and disasters. A coordinator leads each team, consisting of eight to 10 local volunteers chosen by each community and composed of several sub-units that coordinate activities to ensure food safety and the adequate conduction of 1 In Nanawa, PREP worked at the municipal level. Nanawa’s small geographic size and population eliminates the need to create CRMTs within the PREP-supported MCRMR. Therefore, PREP concentrated its efforts on creating CRMTs within the MCRMRs in Caazapá. On October 23, 2018, the level of the Paraguay River rose about 2.3 meters above normal levels, forcing 25 families from Nanawa to evacuate their homes. The local DANA team deployed to the field to collect primary information about affected families, which they provided to Nanawa’s MCRMR and the SEN. With this information, SEN and Nanawa’s MCRMR helped set up an emergency camp to relocate the affected families, and distributed corrugated metal rooftops, mattresses, blankets, and wood beams to them. This was the first time a local institution in Nanawa provided direct assistance to residents recurrently affected by floods. CRMTs received first aid training and received grants of equipment to locally apply their new skills. Promoting Risk and Emergency Preparedness (PREP) Final Program Results Report November 2018 10 prevention and response activities (divided into health, damage assessment and needs analysis, evacuation and shelters, and rescue). PREP strengthened the local capacity of CRMTs, enabling them to effectively prepare community risk maps, draft family and community emergency plans, and evaluate scenarios to respond to emergencies. PREP trained CRMT members, emphasizing the importance of planning and prevention as key mechanisms to address DRR and providing them with tools, procedures, and knowledge to prevent or respond to disasters. Within this framework, PREP found the creation and strengthening of CRMTs to be an excellent mechanism to promote a DRR culture at the community and household level. Strengthen Vertical Linkages among Disaster Risk Reduction Institutions PREP strengthened institutional linkages between the SSAEC, the MCRMRs, and the CRMTs in the targeted intervention areas of Caazapá by promoting regular quarterly meetings where the CRMTs, MCRMRs, and SSAEC could meet, exchange lessons learned, and plan joint actions for the future. At the departmental level, PREP promoted the creation of the DBRMR in Caazapá, formally recognized by a departmental resolution signed by all its members. Presided over by the departmental governor, the DBRMR consists of various secretariats of the Caazapá government, the departmental-level National Police representative, and the Firefighters Volunteer Corp, among others. This board is essential to increase DRR operative capacity at the departmental level and to integrate into the CRMT-MCRMR-DBRMR-SEN coordination hierarchy. With PREP’s support, The DBRMR defined their DRR roles and responsibilities at the community, municipal, departmental, and national levels, increasing effective communication strategies across these levels. To increase national-municipal coordination, PREP supported the DMH by granting two automatic weather stations installed in the municipalities of Tres de Mayo and Fulgencio Yegros. The information from these weather stations allows the DMH and MAG/RMU, to generate previously nonexistent hydrometeorological data for this region of the country, monitor relevant local meteorological conditions, and use the resulting data for adequate risk management. Both stations are integrated with the DMH’s national network of automatic meteorological stations. Likewise, both municipalities provided the space to assemble the automatic meteorological stations, while the DMH has committed to cover operation and maintenance costs. Inauguration event of a weather station granted by PREP. Institutional Coordination Framework Promoting Risk and Emergency Preparedness (PREP) Final Program Results Report November 2018 11 Implement EWSs PREP developed EWSs, which consist of a combination of training, protocols, and provision of essential equipment. PREP promoted an EWS linkage to ensure that the SEN, DBRMR, MCRMR, and CRMTs coordinate. PREP gathered contact information for radio stations from target areas and shared it with the head of SEN’s Early Warning unit, who then started providing weather forecasts and alerts to the local stations through e-mail or WhatsApp for dissemination. In addition, coordinators of the DBRMR, MCRMRs and CRMTs can access the SEN’s general emergency alerts via email and WhatsApp. The coordinators can distribute this information to their representatives or community leaders, who then cascade the information to households in each community. Each of the 10 target communities of Caazapá created a phone tree system so that upon receipt of an alert, each person can notify another family about the risk and disseminate information as quickly as possible. The EWS model includes community feedback on the accuracy of the alerts, that is then transmitted back to SEN to improve their reports. In addition, PREP granted radio communication equipment to assist CRMTs in local communities to stay in stable contact with their MCRMRs, DBRMR, and first responders, even when cellular service is down. As part of the EWS, PREP developed 15 community risk maps for target locations. The maps include information about specific risks that affect each community and resources located within the mapped areas. Based on this exercise, PREP provided 150 reflective signposts made with durable materials and placed them in sites identified through the community risk maps to identify risk areas, evacuation routes, and safe areas in each community. Additionally, PREP developed one map of each of the five target municipalities, and a map of the department of Caazapá. All maps were printed and handed out to the CRMTs, MCRMRs, and the DBRMR. These tools serve to identify emergency protocols at the family and community levels, as well as for first responders to determine an action plan when responding to emergencies in specific communities. In October 2018, heavy rains and strong winds damaged a considerable number of homes in PREP’s intervention area in the department of Caazapá. Program beneficiaries had the opportunity to experience how the PREP-supported EWS works. SEN’s early warning office generated an emergency alert that was disseminated to the Caazapá MCRMRs, who then in turn distributed it their CRMTs who then disseminated the alert to their community members and neighbors. After the extreme climatic event, trained DANA volunteers performed damage assessment and needs analyses of the affected households and delivered reports to their respective MCRMR. The MCRMRs quickly responded in coordination with Caazapá’s DBRMR representative, assisting the affected families by providing metal roofing granted by PREP to repair damaged homes. This experience demonstrated the theory put successfully into practice. PREP granted VHF radios to beneficiaries to facilitate contact between victims and first responders during emergencies. Promoting Risk and Emergency Preparedness (PREP) Final Program Results Report November 2018 12 SECTOR: AGRICULTURE AND FOOD SECURITY PREP’s successful results within the agricultural and food security sector relied on an easily adoptable approach, introducing low-level technologies and techniques for sustainable food production. Small producers quickly adopted the proposed technologies, incentivized by the availability of local material to construct the structures, as well as their efficiency, simplicity, low cost, and low environmental risk. The key takeaways and lessons learned under this sector were: • Hiring local technicians who live in the implementation zone and are fluent in the local Guaraní language is an extremely valuable contribution to ensure a positive connection to and rapport with the beneficiary families. • Through home garden production at the household level, PREP diversified the food available to beneficiary families, improving overall household food security. The year-round availability of lettuce, tomatoes, cabbage, bell peppers, and beets provide new sources of vitamins and minerals in beneficiaries’ diets. • PREP replicated the San Pedro floating garden pilot in Nanawa based on their previous experience. Even though both areas are similarly affected by floods, the Nanawa gardens were not as successful. A barrier analysis demonstrated that factors such as competing priorities with commercial livelihoods and river conditions contributed to the outcomes. For future programs, we recommend that an in-depth analysis is conducted prior to replicating the floating gardens to analyse the local context, determine applicability, and design appropriate behaviour change interventions to ensure sustainability. OUTCOME 3 – SUSTAINABLE FOOD PRODUCTION INCREASED To implement PREP’s strategy to improve agricultural production and food security, PREP conducted the following activities: a) implemented demonstration horticulture plots and small animal husbandry plots, b) replicated the plots in homes, c) trained households in food preservation and planning, and d) replicated the floating gardens pilot. Interviews with beneficiaries demonstrated that ACDI/VOCA’s technicians conducted extensive, reliable, and high quality technical assistance in situ during the life of the project (LOP). Implement Demonstration Horticulture/Small Animal Husbandry Plots Through a participatory process, PREP built demonstration horticulture plots, pigsties, and mobile chicken coops, in schools and homes in each of the 10 target communities in Caazapá, where residents, students, and teachers learned the new technologies. Promoting Risk and Emergency Preparedness (PREP) Final Program Results Report November 2018 13 The 24-square-meter horticulture plots are designed to provide sufficient vegetables throughout the year for a family of five to six people. Some of the best practices implemented in the horticulture plot included the use of half￾shade meshes to protect crops from intense sun, wind breaks, exudation irrigation systems, raised beds, and compost and organic groundcover to improve soil aeration, fertility, and moisture retention during drought. To complement the infrastructure itself, PREP conducted training sessions to demonstrate the production cycles, crop protection measures and best practices. Small animal husbandry structures for pigs and chickens complemented the horticulture plots, which included structural solutions to the common problems identified by the communities during the baseline survey that most often caused the loss of animals. PREP also provided a training plan to teach beneficiaries to properly care for animals, ultimately contributing to increased quantity and improved quality of healthy animals. PREP and SENACSA trained beneficiaries on poultry and swine feeding practices, management, pest and disease prevention, vaccination, and proper soil and environmental management to protect animal health. The pigsty model incorporates a potable water supply system with a sucker for pigs, allowing farmers to mitigate the diseases that normally affect pigs that consume contaminated water. It also has a system to prevent the mother from lying down on newborn piglets and causing their accidental death, and it features a solid floor with sufficient slope to facilitate the process of cleaning the pig’s environment and contributing to preventing diseases. The chicken coop model is mobile in case families must evacuate their homes. They contain feeders and drinking buckets that facilitate the application of liquid vaccines, and they feature raised beds to avoid animal contact with ground humidity. Once smallholder producers tested the demonstration plots and approved their final design, ACDI/VOCA’s technicians developed easy-to-read graphic brochures and manuals with step-by-step processes to assemble standardized structures. PREP also developed and distributed an agro-calendar that demonstrated the correct times for sowing, growing, and harvesting, and a pest and disease control guide showing the most comment insect types that affect crops and organic/natural control measures. PREP distributed copies of the assembly and best practices guides to beneficiaries, USAID/Paraguay and PREP’s most innovative contribution to vegetable production was the implementation of an exudation irrigation system as part of the home garden. In an exudation irrigation system, clay pots are half-dug into the garden plot and filled with water. The porous quality of the clay pot enables water to slowly seep into the soil to irrigate fruits and vegetables in the home garden. This method rations the use of water and extends the productive season even through periods of intense heat. An important extra benefit of this irrigation system is the reduced time burden on females in the household, who typically water the garden plots. To replicate the use of this irrigation system at a local level, PREP trained local artisans to make the pots with local materials, thus diversifying their production beyond manufacturing bricks. The artisans accepted this training and already acquired a pottery wheel to manufacture clay pots. During the LOP, the whole country experienced an intense drought, including Caazapá, where it lasted more than a month and despite these harsh conditions, PREP beneficiaries continued to produce vegetables during that period due to the proposed irrigation system. Promoting Risk and Emergency Preparedness (PREP) Final Program Results Report November 2018 14 its local implementing partner FECOPROD, MAG, government offices in Caazapá, and the National University of Asunción’s Faculty of Agronomy (FCA/UNA) in Caazapá for distribution to its students. Replicate Horticulture/Small Animal Husbandry Plots in Homes Once smallholder producers validated the proposed models, PREP’s agricultural expert supervised the replication of similar plots, standardized pigsties, and mobile chicken coops in 500 households in the Department of Caazapá with the assistance of local technicians. Beneficiaries contributed labor to assemble the horticulture/small animal husbandry plots. PREP provided them with inputs and, by following the assembly guides, the beneficiaries built the structures by themselves, under the supervision of PREP staff. PREP’s beneficiaries distributed the guides to other neighbors to promote the replication of the infrastructure. PREP granted kits to assemble 346 horticulture plots, 76 standardized pigsties, and 78 mobile chicken coops in Caazapá. All 500 beneficiary households received seeds to grow spring/summer and autumn/winter vegetables. According to quantitative data collected in October 2018, 431 households reported harvesting at least three spring/summer species (lettuce, tomato, bell pepper, pumpkins, cabbage, or beets) during the previous season. In addition, all the plots reported producing at least three autumn/winter species (carrots, chard, and parsley). Moreover, 70 percent of the plots maintained by smallholder producers are currently in a second spring/summer productive stage. PREP illustrated best practices to protect animals from climate-induced stresses and shocks with the construction of a pilot pigsty and a mobile chicken coop in coordination with the National Animal Health and Quality Service (SENACSA). The program connected beneficiaries with DEAG’s extension agents, who assisted the program to distribute agricultural inputs and participated, with SENACSA, in training sessions with beneficiaries. During final surveys, 70 percent of the smallholder producers reported increased animal production. Students from the FCA/UNA in Caazapá participated in agricultural DRR workshops and received copies of the materials developed by PREP to disseminate best practices. A principal factor contributing to the success of the horticulture and small husbandry plots is the in-depth and high-quality technical assistance provided by locally hired technicians in Guaraní language. The technicians conducted house-by-house follow-up visits on frequent basis, to verify the implementation of the plots, monitor the results, and adjust/correct issues that arose during the implementation process. Promoting Risk and Emergency Preparedness (PREP) Final Program Results Report November 2018 15 Train Households in Food Preservation and Planning PREP implemented methodologies focused on training beneficiaries on crop cycles and planting schedules to guide them to plan vegetable production throughout the year. Technical assistance oriented the beneficiaries in early planting to produce several harvests during the year for family consumption, while also allowing beneficiaries to sell surplus vegetables at the local market. To aid in the planning, PREP developed and distributed an agro￾calendar to help smallholder producers better plan the production of their horticulture plots, such as sowing vegetables at the ideal time, monitoring plant development phases, and verifying the association of species to achieve year-round production. The calendars were distributed widely to public and private institutions for use in training and replication in other areas. Once beneficiaries harvested their vegetables, PREP collaborated with a DEAG food technologist to train the beneficiary communities in processing vegetables to prepare sauces, jams, and canned vegetables. Food preservation extends the shelf life of these products for up to a year, enabling the families to utilize preserved vegetables long after they have been harvested and to increase the availability of food during times of scarcity. Replicate Floating Garden Pilot ACDI/VOCA designed pilot floating gardens during previous OFDA programming in the department of San Pedro. To replicate the model in an area similarly affected by flooding, PREP brought the floating gardens to Nanawa. This innovative technology allows farmers to grow food on flooded land. If properly cared for, the gardens can produce enough food for five to six people per harvest, and the system is suitable for several types of vegetables, especially leafy greens. To replicate the model, PREP implemented a learning-by-doing technique using a farmer-to-farmer methodology, provided in situ technical assistance, and granted in-kind inputs and seeds to replicate the floating gardens. PREP promoted interdepartmental exchange by bringing the local leaders responsible During a monitoring visit, beneficiaries of horticulture plots reported that they had produced enough vegetables to be self-sufficient for periods of up to four months. During this time, they did not need to purchase vegetables given their own successful production. Beneficiaries reported that they highly valued the traceability of their production, as they are growing better quality products than they had previously purchased and now know what they are feeding their families. Beneficiaries also mentioned that they have adopted and are practicing the preservation techniques from PREP’s trainings to preserve their vegetables. Beneficiaries learned how to cook and preserve the vegetables from their gardens. Promoting Risk and Emergency Preparedness (PREP) Final Program Results Report November 2018 16 for the construction and implementation of the San Pedro floating gardens to Nanawa to train and provide technical assistance to the local community. Together with PREP, the beneficiaries built 36 floating gardens. PREP technicians trained beneficiaries on the garden maintenance and best practices. During the final survey, 88 percent of those interviewed responded that they acquired new knowledge on how to assemble a floating garden, how to prepare the substrate, how to take care of the floating garden, and the best periods for production. This indicates the efficiency of the PREP training processes. For those that applied the best practices, they were able to vegetables and herbs for family consumption. Of those interviewed, 43 percent achieved a relatively good production of chard, cucumbers, lettuce, pumpkins, coriander, and parsley, while bell peppers and tomatoes were not harvested, or the production was insignificant. After the first harvest, PREP noted that several gardens were not receiving the proper maintenance and care to be fully productive. Therefore, PREP conducted a barrier analysis through in-depth interviews to identify the limitations that prevented the beneficiaries from achieving the expected results. Of those interviewed, 26 percent attributed the gardens’ lack of productivity to river contamination, 19 percent to birds that ate seedlings, 12 percent to their insufficient maintenance and care of their gardens, and 4 percent to poor quality substrate. 2 The analysis verified beneficiaries’ lack of time and dedication to care for the gardens, related to the fact that the primary livelihood for the majority of the beneficiaries in Nanawa is not agricultural but instead commercial (given Nanawa’s location as a border city). In addition to this opportunity cost, the need for a floating garden was not internalized by most, given that during PREP’s implementation period, Nanawa was not affected by severe floods and dryland was available for regular gardening activities. To address this situation, PREP developed strategies to promote social behavior change with beneficiaries and encouraged them to adopt this technology as a mitigation measure for food security during floods. PREP reconditioned 33 floating gardens, replanted a second cycle of crops with beneficiaries, and reminded them of best practices to properly care for their gardens. During the final survey, 69 percent of those interviewed responded affirmatively when asked whether they would dedicate time to produce in their floating garden during the next season. For future programs, the social, economic, and livelihood context of the beneficiaries must be fully examined to determine the probability of adoption and sustainability of the floating garden. 2 The remaining 39 percent did not respond with a specific cause/limitation. A beneficiary of a floating garden in Nanawa showing his pumpkins harvest. Promoting Risk and Emergency Preparedness (PREP) Final Program Results Report November 2018 17 GENDER-SENSITIVE APPROACHES Based on ACDI/VOCA’s previous experience in DRR programming, PREP staff understand that to enhance community resilience, both women and men must receive DRR training, have access to information around ways to reduce risks related to natural hazards, and share in DRR leadership roles. In PREP’s targeted areas, men and women take on different and gendered roles and responsibilities, and often have differential access to and control over productive assets, services, information, and income￾earning opportunities. These differences often lead to significant gender gaps, including gaps that disproportionately affect the abilities of male and female community members to plan for and respond to disasters. While women’s vulnerability is frequently highlighted in DRR programming, their contribution to building disaster resilience is often overlooked. Women are often responsible for non-remunerative work, including unpaid care work and voluntary community work. While this work is a very important contribution to households and the broader community, it is often not made visible or seen as valuable, primarily because it is unpaid. Furthermore, while women take on most of this work in their households and communities, they are often not included in leadership roles that make decisions or set priorities. The result of this can further exacerbate gendered gaps that marginalize women from the development of DRR policies and decision-making processes. Furthermore, gender-blind programming in the DRR space can often reinforce and reproduce harmful gender stereotypes about women’s roles and responsibilities, especially related to unpaid care work. To address this, PREP developed a gender strategy to integrate gender considerations in program activities, and where necessary, developed targeted gender activities to address important gaps. These included: • At the beginning of the program, ACDI/VOCA’s headquarters specialists trained the field staff on important gender concepts, familiarizing them with ACDI/VOCA’s gender policy and improving their ability to recognize and address gender gaps in their programming. • During the training process, the PREP team recognized that risk and vulnerability have fundamental social and gender dimension, including that men’s and women’s livelihoods are different and their views on natural and disaster-related hazards differ. Furthermore, the PREP team identified the different ways that men and women participate in DRR programming and the different gendered roles they take on. These gendered roles included more participation of women in voluntary work to plan and prepare for disaster-related risks, and less participation of women in leadership positions. • PREP adjusted activities to include women and men equitably in leadership positions, particularly in the MCRMRs and CRMTs in the targeted intervention areas of Caazapá, by increasing outreach to women to serve in leadership functions, in decision-making processes, and in action implementation, and by sensitizing men to make more room for women leaders. Female beneficiary of horticultural plot. Promoting Risk and Emergency Preparedness (PREP) Final Program Results Report November 2018 18 • PREP promoted women’s participation in OFDA trainings to increase their DRR capacities, by adjusting training schedule to better accommodate women’s schedules. Likewise, PREP promoted the equitable selection of participants through the MCRMRs. • PREP identified that women’s roles consist of productive activities such as home gardens, chicken coops, or pigsties, in addition to unpaid care work within their households such as taking care of and feeding their families. PREP introduced agricultural best practices that allowed them to improve food productivity, such as improved planning/calendaring, shade mesh, irrigation pots. • Technicians with knowledge of the local Guaraní language conducted all trainings, making it accessible for men and women from varied educational backgrounds. • When planning training schedules, PREP considered the existing workloads of men and women to avoid creating additional work burden, particularly for women. This included consulting women about the best times and locations for training to ensure that it didn’t conflict with their other responsibilities. • The primary beneficiaries of PREP’s horticulture plots, pigsties, and mobile chicken coops were elderly women. Under this scenario, PREP proposed and built models for horticulture and small animal husbandry considering aspects such as weight carrying, transportation, watering needs, and time saving for other household activities and family time. PREP worked on revealing the unequal division of labor and women’s unpaid care work through an interactive activity. This exercise implemented in Caazapá made women’s care work more visible and showed the value of that work as a means towards promoting more equitable workloads within households and communities. As a result, during the final evaluation survey many female beneficiaries reported an increased sense of self-esteem, since they and their families now recognize and appreciate their contributions to the household. Gender workshops were implemented in four target districts of Caazapá. Promoting Risk and Emergency Preparedness (PREP) Final Program Results Report November 2018 19 QUANTITATIVE ANALYSIS OF INDICATORS The following chapter analyzes PREP's achievement of indicators. This evaluation compares final achievements with baseline values, while assessing challenges faced in achieving targets. The chapter also includes comments about data quality and management, from data collection to analysis and interpretation. The chart below describes PREP’s progress on its seven indicators, as well as the achieved results and the percentage of achievement at the end of the program. Table 2. PREP’s Indicators List of Results and Indicators LOP Target Baseline Results Achieved Percentage of Achievement Sector 1: Risk Management Policy and Practice Objective 1: Provide the national government and local actors (local government and communities) with tools and capacities to proactively monitor and manage risks. Indicator 1: Number of agroclimatic risk and socioeconomic vulnerability maps developed 6 0 6 100% Indicator 2: Number of people trained in disaster preparedness, mitigation, and management 940 0 2,151 (1033 M/1118 F) 229% Indicator 3: Number of people passing final exams or receiving certificates 200 0 210 (114 M/96 F) 105% Indicator 4: Percentage of people trained who retain skills and knowledge after two months 60% 0% 73% 122% Sector 2: Agriculture and Food Security Objective 2: Increase sustainable food production for households vulnerable to cyclical climatic events. Indicator 5: Number of people benefiting from seed systems/agricultural input activities 2,680 0 3,129 (1604 M/1525 F) 116.8% Indicator 6**: Number of project households with increased access to targeted vegetables produced 300 0 431 143.7% Indicator 7: Number of floating gardens implemented in the target area as a result of program technical assistance 36 0 36 100% ** Using baseline findings, we requested a modification to indicator #6 on August 30, 2017. AOR Raquel Salinas acknowledged receipt of suggested changes on October 6, 2017. Indicator 1: Number of agroclimatic risk and socioeconomic vulnerability maps developed Under the risk management policy and practice sector, PREP developed five agroclimatic risk maps and one socioeconomic vulnerability and food security map with information from 11 municipalities in Caazapá. These important tools provide stakeholders with a valuable source of analysis, to integrate information regarding climatic events and to increase the GOP’s ability to engage in data-driven decision-making and develop appropriate DRR programs. The maps contribute to strengthening institutions’ ability to provide support to vulnerable communities, as well to use rural socioeconomic vulnerability information systems to guide development programs for the Department of Caazapá. The map developing process included training seasons executed by ACDI/VOCA and IICA. More than 400 national-level technicians from the private and public sector learned to develop, use, and update Promoting Risk and Emergency Preparedness (PREP) Final Program Results Report November 2018 20 agroclimatic risk maps. PREP achieved 100 percent of the LOP target for Indicator 1, thereby contributing to increasing GOP ability to proactively monitor and manage risks. Indicator 2: Number of people trained in disaster preparedness, mitigation, and management PREP held training and educational events in disaster preparedness, mitigation, and management for 2,151 participants. PREP achieved a balance between men and women participants (52 percent women, 48 percent men). The training process included topics related to DRR planning, the roles and functions of MCRMRs and CRMTs, emergency plan elaboration, institutional strengthening, and OFDA’s forest firefighters, first aid, and DANA courses. Other important training topics included those related to the water balance model and risk maps in the agriculture sector. PREP surpassed the target for Indicator 2 and built skilled human resources, increasing institutional and community knowledge of DRR planning, management, preparedness, and mitigation. The following graphic shows the total trainees by gender and municipality. Indicator 3: Number of people passing final exams or receiving certificates PREP developed a strategy to strengthen national disaster risk management institutional capacity by providing technical assistance and training at the local and municipal levels. PREP achieved and surpassed this indicator’s LOP target by implementing key trainings from the USAID/OFDA RDAP portfolio. In total, 210 trainees received their certification, in first aid (129 participants), damage assessment and need analysis (23 participants), disaster risk reduction management (5 participants), and forest firefighter (53 participants) courses. RED’s certified instructors conducted all trainings from OFDA’s RDAP, coordinated with SEN’s head of training. PREP certified a total of 96 women and 114 men. Indicator 4: Percentage of people trained who retain skills and knowledge after two months To verify knowledge retention, PREP requested RED’s instructors of first aid, DANA, and forest firefighter courses, to prepare a test to measure knowledge about the fundamental aspects of the courses they implemented. PREP’s staff then probed the retention of knowledge and skills by giving these tests to a random sample of trainees (45), of whom 33 (73 percent) passed. PREP surpassed this indicator target by 122 percent. Of the participants who retained skills and knowledge, 42 percent were women. 60% 43% 45% 45% 47% 61% 45% 30% 53% 67% 0% 10% 20% 30% 40% 50% 60% 70% 80% Total trainees by gender and municipality F M Promoting Risk and Emergency Preparedness (PREP) Final Program Results Report November 2018 21 Indicator 5: Number of people benefiting from seed systems/agricultural input activities A total of 3,129 people (49% female, 51% male) benefited from agricultural inputs, trainings, and technical assistance. PREP developed agricultural activities that included the distribution of inputs for the replication of home gardens, seeds, pigsties, and mobile chicken coops. This indicator also reports on training and workshop participants, including those related to agricultural production, care of small animals, and beneficiaries who learned to use agricultural sector and water balance risk maps. PREP monitored vegetable production in beneficiary households, and once they were able to harvest, family members who consumed these products were also reported as beneficiaries. The indicator goal has been surpassed, reaching 117 percent. Indicator 6: Number of project households with increased access to targeted vegetables produced Exceeding PREP’s goal of 300 households, 431 households (of 500 surveyed) reported increased access to at least three of six targeted vegetables.3 A combination of structural (inputs for home gardens and seeds) and non-structural activities (training and technical assistance) allowed 431 households of 500 beneficiaries, to increase production of the vegetables they most commonly consume (bell pepper, lettuce, tomatoes, pumpkins, cabbage, and beet). Indicator 7: Number of floating gardens implemented in the target area as a result of program technical assistance Through technical assistance, in-kind grants, interdepartmental exchanges, and several training and workshops events, PREP achieved the LOP goal of implementing 36 floating gardens. The floating gardens were built in four urban neighborhoods (barrios) in Nanawa and the neighboring agricultural community of Beterete Cué in Villa Hayes. A key important finding in this activity shows that 53 percent of the total beneficiary households selected were led by women. PREP has understood the significant importance of implementing the floating gardens for small-scale home vegetables production with women as head of families. In terms of sustainability, an analysis made 3 To the time of survey, 67 families reported between one and two species harvested and two families have not reported any production. 51% 43% 44% 45% 47% 53% 63% 30% 54% 56% 49% 57% 56% 55% 53% 47% 38% 70% 46% 44% 0% 10% 20% 30% 40% 50% 60% 70% 80% Total people benefiting from seed systems/agricultural input activities by gender and municipality Total: 3,129 people F M Promoting Risk and Emergency Preparedness (PREP) Final Program Results Report November 2018 22 with home gardens beneficiaries shows that the technical knowledge on the technology proposed has been transmitted by PREP to beneficiaries through the training plan and the “learning by doing” technique developed, people know what a floating garden is and how to build it and how to take care of it. The social dimension is covered by the active presence and participation of the beneficiaries in all project stages. The low cost to replicate it, repair the home garden, or replace seeds and some material for the next productive season allows the continuous reproduction of this innovative production technique. 60% 100% 67% 25% 45% 40% 0% 33% 75% 55% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Independiente San Miguel Sportivo Oriental Beterete Cue Nanawa Villa Hayes Total home gardens implemented by gender and district F - Floating Garden M - Floating Garden Promoting Risk and Emergency Preparedness (PREP) Final Program Results Report November 2018 23 CONCLUSIONS AND RECOMMENDATINS After having collected, analyzed, and systematized the quantitative and qualitative information, in general terms, the evaluator reached the following conclusions. Risk Management Policy and Practice After having collected, analyzed, and systematized the information, the evaluator reached the conclusion that PREP has: • Strengthened the GOP’s multilevel linkages and abilities for effective disaster risk reduction (DRR) preparedness and response at all levels. GOP institutions are now in better condition to manage risks and respond to disasters in the target regions. • Contributed to decentralizing DRR to departmental and local levels. • Strengthened vertical linkages among DRR institutions, in accordance with the National Policy on Disaster Risk Management and Reduction (NPDRMR). • Developed tools, procedures, guidelines, protocols, and best practices that have been made available to the GOP and stakeholders and can be replicated in other regions in Paraguay. • Improved knowledge and understanding of risk management options and strengthened monitoring capacities of risk management initiatives. • Contributed to strengthening the ability of vulnerable households to prepare for, mitigate, and adapt to shocks and stresses caused by natural hazards through a people-centered, multisector approach. In addition, PREP created and incorporated a culture of prevention and resilience to natural disaster events in communities and families. • Created a social platform for emergency preparedness and response at the local level, with an important relevance through the involvement of municipal authorities. • Provided, through a training process, the concepts of disaster risk reduction and management to decision makers. The program positively impacted the knowledge, attitudes, and practices of target communities. MCRMRs and CRMTs are continuously looking to increase their capacity as social institutions for emergency preparedness and response at the local level. Their members understand the importance of conducting frequent meetings, the application of internal procedures and statutes, and the need for continuous dialogue with municipal and departmental authorities as the most feasible way to support families within their communities. Ninety seven percent of interviewed households reported feeling more prepared for emergency and disaster events. The program led to a substantial increase in knowledge and awareness about the hazards faced by communities and how to reduce or mitigate the associated risks. This increase in knowledge and awareness extends across the population in the community and local institutions. In terms of perceived impact, 88 percent of those interviewed reporting knowing the existence, roles, and importance of CRMTs. In the same vein, 100 percent of MCRMR representatives reported knowing the specific roles and responsibilities of general or operative coordinators. Sustainable Food Production for the Victims of Floods After having collected, analyzed, and systematized the information, the evaluator reached the conclusion that PREP: Promoting Risk and Emergency Preparedness (PREP) Final Program Results Report November 2018 24 • Contributed to increased sustainable food production for households vulnerable to cyclical climatic events in target intervention areas, decreasing the probability of an acute decline in food access or consumption due to weather disturbances through a prevention and mitigation strategy. • Diversified the food available to beneficiary families, improving overall household food security. The year-round availability of lettuce, tomatoes, cabbage, bell peppers, and beets provide new sources of vitamins and minerals in beneficiaries’ diets. • Created links between institutions such as the MCRMR, the FCA/UNA, the DEAG and small agricultural producers, generating a space where they can discuss the problems that affect their food security and local technical knowledge to propose solutions. • The easy-to-read graphic brochures and manuals with step-by-step processes to assemble standardized structures, the agro-calendar that demonstrated the correct times for sowing, growing, and harvesting, and a pest and disease control guide showing the most comment insect types that affect crops and organic/natural control measures are very important printed reference materials that remain with the producers to replicate this new knowledge. • Promoted the active participation of national and local government institutions and academia in training and technical assistance for sector-level activities for household beneficiaries. • Improved human assets through technical capacity building that allowed access to agricultural sector benefits to more than 3,100 smallholder producers in the program’s intervention areas. Based on the survey, and comparing it with the baseline report, PREP increased the number of households that have a home garden with improved technology like wire mesh to protect crops from animals, shade cloths, and materials for irrigation systems from 14 percent at baseline to 67 percent at endline. The endline survey indicates that 94 percent of beneficiary households consume their home garden production. Fifty eight percent of surveyed households reported that PREP’s implemented demonstration horticulture plots and training process increased sustainable food production by improving knowledge and low-cost techniques. More than 90 percent of those interviewed declared improved knowledge of agricultural techniques such as vegetable production cycles, or how to prepare a better substrate to increase production. PREP’s pigsties and mobile chicken coops in Caazapá contributed to a 70 percent increase in animal production among beneficiaries in Caazapá. Recommendations The following recommendations aim to contribute to the sustainability of PREP activities: • The National Emergency Secretariat (SEN) and local governments should schedule new training processes and allocate resources to reinforce newly acquired knowledge, support the newly formed institutional structures, and create long-lasting changes in practices and attitudes within MCRMRs and CRMTs. • Municipal and departmental authorities must follow up by providing political support to MCRMRs, CRMTs, and the DBRMR to maintain, increase, and expand the activities and goods generated by PREP. • Future DRR programming should build on this program’s successful development of structural and non-structural solutions to address DRR management. Promoting Risk and Emergency Preparedness (PREP) Final Program Results Report November 2018 25 ANNEXES ANNEX A. EVALUATION METHODOLOGY Because the PREP Program was designed as a program to strengthen the capacity of the GoP and households that are vulnerable, as a way to prepare for, mitigate and adapt to the impacts caused by natural disasters, using structural and non-structural strategies that can be replicated in other communities, the evaluation process reviewed the development hypothesis of the Program and the performance measurement plan to determine how the expected results have been achieved. The evaluation focused on "testing with tangible evidence" the results and goals achieved, to help understand the performance of the Program. The evaluation included: Qualitative analysis For the qualitative data collection, focus groups, in-depth interviews, and observation techniques of field work were carried out to learn about the institutional and local organizations supported / established by PREP. The local actors were asked about the experiences, perceptions, opinions, feelings and knowledge about PREP interventions. A questionnaire was applied and through the answers obtained it was possible to determine: • The influence of PREP in the adoption of new technologies / practices and the adoption of models in the preparation, mitigation and adaptation to the impacts caused by natural disasters. • Behavioral changes between institutional and local actors that led to the successful adoption of new risk management practices. • How successful the risk management models and actions implemented by PREP have been. • How likely are the actions promoted by the PREP to be continued after the end of the program. Sampling The sample was limited to the representatives of local institutions and social leaders who have worked with the PREP program (beneficiaries) and other interested parties. Following the usual methodology used for qualitative studies with local actors, a non-probabilistic method for sampling was used; it’s called expert sampling. This type of sampling, also known as stakeholder sampling (Brown, 2002), is a type of intentional sampling where the researcher previously defines the type of individuals to be interviewed based on their experience in a particular approach or more experience over other experts. People with relevant knowledge of risk management systems and the regular functioning of their organizations were defined as experts to be sampled. For this purpose, a documentary research was carried out to enumerate and review the institutional and local organizations that have worked with the PREP program. In coordination with the ACDI/VOCA Paraguay staff, the interviewees were selected by community and organization. The sample size was determined considering the time constraints of the evaluation and the theoretical saturation as the appropriate parameters to define the size of the sample in qualitative studies. Once the key informants were selected and, to gather rich and illuminating information, at least four focus groups or in-depth interviews were carried out. In addition, secondary sources of information also provided useful data to complete the evaluation. Data collection tools The focus for group discussions, interviews with key informants, as well as the development of the questionnaires were carried out in coordination with the ACDI/VOCA Paraguay team. The questionnaire was based on inquiring about the knowledge, behaviors and practices carried out, with particular focus to Promoting Risk and Emergency Preparedness (PREP) Final Program Results Report November 2018 26 understand the motivations and the barriers to behavior change. The interviews lasted approximately 2 hours. Data analysis For the analysis of the qualitative data, the interviews were transcribed and then imported into the qualitative analysis software NVIVO 11 for data coding and analysis (thematic analysis). Following the thematic analysis approach, the data from the interviews were coded in different nodes to identify their subjects. A node contains all the qualitative data assigned to that code; and it constitutes the main unit of analysis of the methodology. In the same way, the characterization variables of the interviewees and their organizations were grouped to analyze how the topics are composed within different categories. The thematic analysis facilitated the synthesis of the data and the construction of categories, where it was possible to estimate the simple qualitative analysis of the frequencies to understand how the perceived factors that affect a topic are prioritized among the different groups of respondents. The result was expressed on the radar and other visual tables using Excel graphics. Quantitative analysis A statistical design (survey) was used for the quantitative analysis to identify changes (before and after) in the quantitative indicators of the PREP program and their related indicators. The evaluator reviewed the reference study (baseline) as the starting point of the program and used the same instruments designed at that time for this evaluation at end line. The most relevant variables that should be collected in the field for the final evaluation of the program were defined in coordination with ACDI/VOCA staff. With this method, the study found the correlation between the different variables, which allows to quantify the influence of multiple causes in a single effect. Sampling The sampled population were the beneficiaries represented by the head of household. The beneficiary survey was conducted over a period of three days in ten communities of four districts in the departments of Caazapá. The sample sites were limited to the area of intervention of the PREP program and only covered those beneficiaries who have worked with ACDI/VOCA Paraguay in the PREP program. The sampling was accurate and representative of the PREP target. During this stage, the evaluator ensured gender balance among those surveyed. For the quantitative analysis, the evaluation consultant used a Probability Sampling method called Two￾stage Sampling. First, a sample of the communities was carried out (first stage) and then the beneficiaries of those selected communities (second stage). Stratified random sampling was used for the second stratum because it provides more accurate information than simple random sampling for a given sample size. The information is disaggregated by sex to have a gender perspective on the benefits and impacts of the program. The expectation was to gather information from an estimated 150 beneficiaries. The sample was representative at the regional and municipal levels for the rural sectors and the sampling error is 5 percent with a confidence interval of 95 percent. To comply with the requirements of the sample design, the following statistical formula was used to calculate the sample size for finite populations: 𝑛 = 𝛽 2 × 𝜗 2 × ∑ 𝑁 𝑛 𝑖=0 𝜀 2(𝑁 − 1) + 𝛽 2 × 𝜗 2 Where: 𝑛: Sample size for a finite population β: Estimator of the confidence level of the sample at 95% = 1.96 Promoting Risk and Emergency Preparedness (PREP) Final Program Results Report November 2018 27 𝜗: Variance of the observed variable, equal to 𝜗𝑛 = (1−𝑁) 𝑁𝑖/ ∑ 𝑁 𝑛 𝑖=0 ε: Sample error allowed of 5% N: Total population For the selection of the samples, at statistical level the following criteria were used: • Selection of statistically representative samples by district within each municipality that can be added in larger units to those of the municipality (area or region) to be compared with other studies. • Sample size calculated based on the percentage variations of the estimated number of inhabitants per district based on estimated secondary information. • With a maximum allowed sampling error of +/- 5 percent per district. • With enough power to represent with statistical criteria the population heterogeneity within each district. The following table shows the results of the Sample Size selected for the target communities and determines the number of surveys that are carried out at the population level in each community of both departments4: Departament Municipality Community Estimated beneficiary population Estimated number of PREP beneficiaries Sample estimation Families according to references of the ACDI/VOCA team Number of beneficiaries (average of people/families) Number of beneficiaries (individuals) Number of beneficiaries (families) Surveys total CAAZAP Á Maciel Zanja pyta 110 440 220 57 18 Caracaraí 80 320 160 53 16 3 de Mayo Argaña cue 55 220 132 27 11 Dolores 64 256 102 37 13 Cerrito 90 360 180 79 32 Yegros Gral. Colman￾Mbarigui 90 360 252 57 16 Isla Saca 80 320 256 51 19 Caazapá Isla apuá-Yeroa￾Santa Catalina 112 448 269 54 19 Tajy 115 460 230 50 17 Ñaumby-Isla Guazu 100 400 240 43 21 Total General 896 3.584 2.041 508 182 Data collection tools In coordination with the ACDI/VOCA Paraguay staff, the evaluator defined a complete set of questionnaires based on the baseline questionnaires designed at that time. 4 In Nanawa PREP conducted surveys and focus groups with the MCRMRs, not the population itself. Promoting Risk and Emergency Preparedness (PREP) Final Program Results Report November 2018 28 For the data collection, in coordination with the evaluator and the ACDI/VOCA Paraguay team, six local enumerators were selected. According to the terms of reference, they were selected from the districts / communities of the program and were trained by the evaluator and then deployed to collect data in the field. The importance of using local enumerators is that they speak the Guarani language and are familiar with the areas of research. ACDI/VOCA Paraguay provided per diem and a pre-defined amount to cover local transportation, especially for distant areas. The evaluator trained the enumerators to familiarize themselves with the data collection methods and questions, how to use and fill out the survey form, the solution to possible problems, etc. The evaluator and the team, in coordination and with the support of ACDI/VOCA supervised the quality of the data collected by each enumerator and the data of the field was synchronized with a Dropbox server. Cleaning, classification and data analysis At the end of the fieldwork, the enumerators submitted the completed survey forms. The evaluator transcribed the data obtained and made the transfer of data from paper data sheets to an electronic database, eliminating possible errors introduced during the manual process. Excel was used for data entry. The statistical analysis and the administration of the data sets were made through IBM Statistical Software SPSS 23 (predictive analysis program) to verify the consistency of the data. For a more precise analysis, the database was cleaned and standardized following the criteria stipulated by the baseline study and ACDI/VOCA. The data entry errors and deviations of the response code were corrected, and all outliers were discarded with a standard deviation ± 4 of the means in all the metric variables. The analysis was made for each group of indicators and indices of the PREP program in order to construct the final values for each indicator and compare the objectives and general results of the final program with the baseline. Descriptive statistics and frequency analysis of variables were carried out for the characterization of the sample in the different strata and the complete sample. Once the qualitative and quantitative data were processed and the evaluator finished with the analysis to compare or relate the findings, he proceeded with the interpretation and, finally, communicated the findings of the evaluation, the limitations of the methodology, and recommendations to this report. Quality Control The evaluator adopted different strategies to ensure high standards of the data collection process and the maximum quality of the data collected. First, the work of the local enumerators was supervised in the field to ensure that the quality of the data is not compromised in any way. Secondly, the use of a specific application of predictive computer models ensured greater control over the quality of the data through the automatic omission pattern and the validation of the response, minimizing data entry errors. Promoting Risk and Emergency Preparedness (PREP) Final Program Results Report November 2018 29 ANNEX B. QUESTIONNAIRE TO EXPERTS Promoting Risk and Emergency Preparedness (PREP) Final Program Results Report November 2018 30