Dan PERFORMANCE EVALUATION OF THE PARTNERING FOR ADAPTATION AND RESILIENCE-AGUA (PARA￾AGUA) ACTIVITY Final Evaluation Report JUNE 2015 This document was produced for review by the United States Agency for International Development. It was prepared by The Cloudburst Group for the Evaluation, Research, and Communication (ERC) Task Order under the Strengthening Tenure and Resource Rights (STARR) IQC. Prepared for the United States Agency for International Development, USAID Contract Number AID￾OAA-TO-13-00019, Evaluation, Research and Communication (ERC) Task Order under Strengthening Tenure and Resource Rights (STARR) IQC No. AID-OAA-I-12-00030. Implemented by: The Cloudburst Group 8400 Corporate Drive, Suite 550 Landover, MD 20785-2238 Performance Evaluation of the Partnering for Adaptation and Resilience-Agua (PARA-Agua) Activity Final Evaluation Report JUNE 2015 DISCLAIMER The authors' views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. CONTENTS EXECUTIVE SUMMARY.............................................................................................................. 1 EVALUATION PURPOSE AND EVALUATION QUESTIONS ........................................................................ 1 PROJECT BACKGROUND ....................................................................................................................................... 1 EVALUATION APPROACH: DESIGN, METHODS AND LIMITATIONS .................................................... 2 FINDINGS ...................................................................................................................................................................... 1 CONCLUSIONS AND RECOMMENDATIONS ................................................................................................. 3 1.0 BACKGROUND ...................................................................................................................... 5 2.0 EVALUATION PURPOSE AND OBJECTIVES ................................................................... 8 3.0 METHODOLOGIES AND LIMITATIONS ........................................................................... 9 DESK REVIEW ............................................................................................................................................................... 9 FIELD VISITS AND DATA COLLECTION .......................................................................................................... 10 LIMITATIONS .............................................................................................................................................................. 12 4.0 FINDINGS ............................................................................................................................. 13 4.1 EVALUATION QUESTION #1 ........................................................................................................................ 13 4.2 EVALUATION QUESTION #2 ........................................................................................................................ 15 4.3 EVALUATION QUESTION #3 ........................................................................................................................ 19 4.4 EVALUATION QUESTION #4: ....................................................................................................................... 22 4.5 EVALUATION QUESTION #5 ........................................................................................................................ 24 5.0 CONCLUSIONS ................................................................................................................... 25 6.0 RECOMMENDATIONS ........................................................................................................ 27 ANNEX 1. LIST OF SOURCES ................................................................................................. 30 ANNEX 2. STAKEHOLDERS INTERVIEWED ....................................................................... 32 ANNEX 3. KII INTERVIEW MATERIALS ................................................................................ 35 INFORMED CONSENT ............................................................................................................................................ 35 GROUP 1: KII QUESTIONNAIRE FORM ............................................................................................................ 36 GROUP 2: KII QUESTIONNAIRE FORM ............................................................................................................ 43 GROUP 3: KII QUESTIONNAIRE FORM ............................................................................................................ 48 ANNEX 4. SCOPE OF WORK .................................................................................................. 54 ANNEX 5. APPROVED EVALUATION PLAN AND TIMELINE .......................................... 55 ACRONYMS AND ABBREVIATIONS ANA Autoridad Nacional del Agua del Perú (National Water Authority of Peru) CHEC Central Hidroeléctrica de Caldas (Hydroelectric Plant of Caldas) AECOM AECOM International Development CIIFEN Centro Internacional para la Investigación del Fenómeno El Niño (International Center for Research of the El Niño Phenomenon) Cloudburst The Cloudburst Group CO Contracting Officer CONDESAN Consorcio para el Desarrollo de la Ecorregión Andina (Consortium for the Development of the Andean Ecoregion) CORPOCALDAS Corporación Autónoma Regional de Caldas (Autonomous Regional Corporation of Caldas) CoP Community of Practice FEMUCAM Federación de Mujeres Campesinas de Cambita GCI Global Climate Initiative GIS Geographic Information System IQC Indefinite Quantity Contract IDEAM Instituto de Hidrología, Meteorología y Estudios Ambientales de Colombia (Institute of Hydrology, Meteorology and Environmental Studies of Colombia) IPCC Intergovernmental Panel on Climate Change IR Intermediate Results KII Key informant interviews LAC Latin America and the Caribbean LAC/RSD Latin America and the Caribbean Office for Regional Sustainable Development MoE Ministry of the Environment MoU Memorandum of Understanding M&E Monitoring and Evaluation NCAR United States National Center of Atmospheric Research NGO Non-Governmental Organization PARA-Agua Partnering for Adaptation and Resilience—Agua PARA-Agua PE Partnering for Adaptation and Resilience—Agua Performance Evaluation PE Performance Evaluation PGHR Plan de Gestión de Recursos Hídricos (Plan of Water Resources Management) PIRS Performance Indicator Reference Sheet PMP Performance Management Plan POMCA Plan de Ordenamiento y Manejo de la Cuenca (Plan of Basin Development and Planning) PROFONANPE Fondo de Promoción de las Areas Naturales Protegidas del Perú (Fund for the Promotion of Natural Protected Areas of Peru) RFTOP Request for Task Order Proposal SEI Stockholm Environment Institute SENAMHI Servicio Nacional de Meteorología e Hidrología del Perú (National Meteorology and Hydrology Service of Peru) STARR Strengthening Tenure and Resource Rights TMI The Mountain Institute TOC Theory of Change WEAP Water Evaluation and Planning UNESCO United Nations Educational, Scientific and Cultural Organization USAID United States Agency for International Development USG United States Government WB World Bank WEAP Water Evaluation and Planning MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Final Evaluation Report 1 EXECUTIVE SUMMARY EVALUATION PURPOSE AND EVALUATION QUESTIONS The purpose of the Partnering for Adaptation and Resilience—Agua (PARA-Agua) Performance Evaluation (PE) is to provide insight into activity performance and effectiveness beyond what has been captured by the implementer’s Performance and Management Plan (PMP). The evaluation questions that are the focus of this PE reflect PARA-Agua’s operations and the relevant watershed management institutions that they are working with – both at the national and sub-national levels – are as follows [1], [2]: 1. In what ways has PARA-Agua affected the policy capacity of national hydro-meteorological centers (Institute of Hydrology, Meteorology and Environmental Studies of Colombia (IDEAM), National Meteorology and Hydrology Service of Peru (SENAMHI)) and the national agencies with authority to approve watershed-level official planning instruments in the focal countries? 2. To what extent has PARA-Agua built new, potentially sustainable links between researchers at hydro-meteorological centers, national agencies with watershed level planning responsibilities, and watershed management councils and authorities? 3. To what extent have the planning systems that watershed management councils or other water managers use been strengthened as a result of PARA-Agua’s work? 4. What new tools, if any, has PARA-Agua developed for water managers in the focal areas? Which have been used most often thus far? 5. If the evaluation recommends continuing with the activity, what changes would enhance the activity’s impact in the final years of implementation? PROJECT BACKGROUND The PARA-Agua project aims to strengthen the capacity of organizations in the Andean region that generate data related to climate change and water resources as well as develop better watershed management tools. These activities are carried out in an effort to improve the resilience of people, places and livelihoods in the Latin America and Caribbean (LAC) region through investments in adaptation. The contract for implementing the PARA-Agua Project was awarded to AECOM International Development (AECOM) on September 13, 2013, with a period of performance of two years with two one-year extension options ending on September 12, 2017. Specific objectives of PARA-Agua include [1]: 1. Strengthening the capacity of the research community to generate policy-oriented data on watershed management and climate change adaptation; 2. Mainstreaming and integrating climate data into decision-making related to watershed management; and, 3. Strengthening planning systems that optimize water use over the whole length of watersheds in the context of climate change adaptation. MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Final Evaluation Report 2 EVALUATION APPROACH: DESIGN, METHODS AND LIMITATIONS The evaluation design discussed and agreed to by United States Agency for International Development (USAID) involved a mixed-methods approach involving two major components. The first involved an in-depth desk review of PARA-Agua documents and materials. The second component involved conducting a series of key informant interviews (KII) and participating in PARA-Agua activities as part of country visits to Colombia and Peru. The overall strategy for conducting the KIIs and the approach taken (i.e. the stakeholders being targeted and the types of questions asked) were informed by the desk review. All interview materials and questionnaires were approved by USAID prior to their being used in the field. In total the team met with and gathered feedback from over 50 individuals from national hydrological research and planning institutions, local watershed planning authorities and members of Watershed Councils in addition to PARA-Agua and subcontractor staff. Evaluation limitations include being unable to meet with a wider range of program stakeholders given limitations in time and resources for completing this evaluation. However, the Evaluation Team feels that the results and conclusions obtained represent a fair and balanced assessment of PARA-Agua to date. FINDINGS As a result of a desk review of PARA-Agua documents and field visits to PARA-Agua sites in Peru and Colombia, the Performance Evaluation Team finds that:  National and regional governments and watershed councils with advisory or executive water planning mandates underscored the consistent and timely contributions of the PARA-Agua project with regards to enhancing their policy making capacities. The goal of developing a cadre of managers and technical staff within key watershed organizations who are capable of employing the Water Evaluation and Planning (WEAP) model for support planning and decision-making pertinent for climate change adaptation is well underway.  United States National Center of Atmospheric Research (NCAR) is widely recognized as a scientific institution of prestige. Colombian and Peruvian experts and researchers of hydro￾meteorological institutions appreciate opportunities to engage and exchange experiences with NCAR scientists and to build their own capacity. With NCAR as part of the PARA￾Agua team, the Project appears to be in a strong position to strengthen national capacities to generate relevant hydro-meteorological data and the linkages with water planning bodies at various levels so these data can be used effectively.  While PARA-Agua has certainly made in-roads supporting SENAMHI in Peru and IDEAM in Colombia, the sentiment is that these relationships have been somewhat slow to materialize. For example, despite an agreement, in principle, for a scientific twinning partnership between the SENAMHI, IDEAM, and NCAR [3], to date, agreements with IDEAM and SENAMHI have yet to be formally entered into. Interview respondents from hydro-meteorological institutes in both countries expressed interest in reaching such an agreement. At least one hydro￾meteorological research center representative stated that its scientists had provided feedback on the scope of work, confirming the participatory nature of the partnership agreement preparation although such an agreement has not yet been reached.  Elements of the PARA-Agua overall gender strategy are at an initial stage and not yet fully operational. Several of the experts consulted at the hydro-meteorological centers consider MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Final Evaluation Report 3 demographic concerns, such as gender, outside their scope of work. This perception could represent a limitation for later adaptation and resilience strategies that respond to the interests and needs of both men and women. It is important that PARA-Agua undertakes an awareness effort with the research community in order to portray success stories that demonstrate the incorporation of gender mainstreaming, as proposed in the Gender Analysis and Plan of Action.  While still new, project participants appreciate the Community of Practice (CoP) virtual platform, but do not necessarily understand its objective.  Cooperation and relationship building within and between research institutions has progressed, but distance between research institutions and stakeholders based in watershed regions remains. CONCLUSIONS AND RECOMMENDATIONS The KII feedback on the Project was positive overall and confirmed several of the desk review’s preliminary findings. In some cases several PARA-Agua activities unanimously were considered as providing ‘first of its kind’ contributions to watershed planning at the regional level, such as the incorporation of climate change adaptation variables in the WEAP tool. PARA-Agua has played an important role in establishing linkages between the national agencies and the regional counterparts. Support has allowed stakeholders at hydro-meteorological institutions and with watershed planning and management responsibilities to better understand the needs of the different parties involved and building a common language between the various stakeholders in the water climate change adaptation context. As a result of the analysis, it is the recommendation of the Evaluation Team that the PARA￾Agua project be funded for the two additional option years. Priority recommendations to increase PARA-Agua’s impact in the final years include:  Supporting efforts to secure funding for adaptation investments: Securing funding to implement water management adaptations at the watershed level represents a significant challenge. Stakeholders have high expectations on the delivery of this PARA-Agua service. PARA-Agua staff needs to prioritize the delivery of this type of high-level technical advice to partners.  Strengthen relationships with national hydro-meteorological institutions: Reaching agreements on Memorandums of Understanding (MoU) that define the scope of work and interest synergies between PARA-Agua, and IDEAM and SENAMHI, should be prioritized to ensure project activities are aligned with national counterparts. Agreeing on a defined scope of work with these institutions could, for instance, contribute to the design of pertinent and enforceable protocols to incorporate public and private organizations into national information systems for climate change adaptation in watersheds.  Enhancing and expanding the incorporation of social and economic priorities in the tools, instruments and overall activities supported by PARA-Agua: The Project is not exclusively working in the scientific area, but is also strengthening the relevance of climate change adaptation data for water planning and management instruments at various levels. Such link with natural resources planning and management implies that pertinent economic and social considerations should be considered and water planning and management instruments tailored, as required. MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Final Evaluation Report 4 Additional recommendations important to enhancing PARA-Agua’s performance during the remaining implementation period include:  Further link stakeholders across the spectrum: Opportunities arising as a result of the enhanced dialogue between national and regional counterparts facilitated by PARA-Agua should be seized upon and further pursued.  Ensure that legacy institutions continue to prioritize national level capacity building, and where possible entrust such legacy responsibilities to national and regional level stakeholders: PARA-Agua should ensure that the legacy institution alliance between CIIFEN, TMI and CONDESAN prioritizes the strengthening of national capacities. In addition, in order to strengthen national capacity, whenever possible, roles and responsibilities should be under the purview of the national level PARA-Agua partners.  Support efforts to integrate research institutions broadly: This involves successfully and inclusively developing the toolkit aiming to develop protocols integrating public and private research organizations into national information systems. PARA-Agua should also enhance its contributions towards strengthening national processes to develop or strengthen environmental, climate change and/or water information systems.  Implement gender mainstreaming and awareness activities: It is important that the Project fully implements its Gender Action Plan and specific watershed Action Plans. Priority gender￾oriented actions include undertaking an awareness effort with the research community using success stories that illustrate how the incorporation of a gender perspective, incorporating gender and climate change issues to the Water Culture Component of Water Management Plans and adjusting local level Water Users’ Boards regulations to promote women’s participation mechanisms all matter to achieving overall PARA-Agua goals.  Maintain a dynamic and relevant virtual platform: A solid CoP team, with access to materials and forum discussion themes which are updated on a regular basis by PARA-Agua project staff and/or subcontractors is central to keeping interest in the virtual platform alive. The platform can contribute to establishing and strengthening links between PARA-Agua stakeholders at various levels, if Project partners better understand the objectives and intent of the platform.  Further tailor tools and activities to the watershed level: In consultation with stakeholders, undertake the adjustments required to further tailor workshops and capacity building activities to stakeholder needs. Even if the workshops’ feedback was positive, interviewees suggested areas of improvement including additional tailoring of workshops to participant profiles and enhancing the adaptation of methodologies and tools to their water planning priorities.  Monitor capacity building of Watershed Councils: Sub-national water management and planning structures frequently have new members, including new Watershed Councils members, regional directors of government offices, and technical staff. In order to make sure that new relevant members are brought up to speed with regards to PARA-Agua capacity building activities, the Project should monitor and suggest relevant training activities. MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Final Evaluation Report 5 1.0 BACKGROUND The Cloudburst Group, henceforth Cloudburst, was awarded a three-month contract (April 6 – July 5, 2015) to conduct a Performance Evaluation of the PARA-Agua project under the Strengthening Tenure and Resource Rights (STARR) Indefinite Quantity Contract (IQC). PARA-Agua forms part of USAID’s work to support adaptation to climate change impacts in the LAC Region. This PE will assess PARA￾Agua’s performance and effectiveness beyond that captured in the activity implementer’s PMP. Policymakers, researchers and water resources managers across the LAC region urgently need assistance in adapting to the impacts of climate change, in order to help reduce vulnerability, strengthen resilience, and build adaptive capacity. A key objective of USAID’s Global Climate Change and Development Strategy is helping people, places and livelihoods adapt to climate change [4]. By promoting investments in adaptation in the LAC region, the PARA-Agua project embraces the Global Climate Initiative (GCI) mechanism and aims to strengthen resilience to unavoidable climate impacts and enhance the ability of people and ecosystems to withstand the negative impacts of climate change. GCI advances the United Stated Government’s (USG) global development policy in several ways. Of particular relevance in the context of PARA-Agua activities is LAC countries’ ownership of climate plans, climate solutions that spur economic growth, sustainability of economic growth gains through actions that protect investments, governance strengthening and inclusive planning processes for climate resilience, and game-changing investments by pioneering advances in the application of science and technology (such as earth observations and information communication technologies) [5]. The overarching goal of PARA-Agua is to support scientists and the research community throughout the LAC region, especially in Colombia, Ecuador and Peru, to work together and develop better scientific information, programs and decision-making tools to help manage water supplies and other resources. The PARA-Agua project seeks to strengthen the capacity of organizations in the Andean region that generate data related to climate change and water resources as well as strengthening water planning in the context of climate change in the LAC region through investments in adaptation. Ultimately, PARA￾Agua aims to enhance climate change-related resilience and minimize climate change disruption to key economic activities in the region, advancing USAID’s Climate Change and Development Strategy [4]. Stakeholders include national level hydro-meteorological institutions, watershed planning councils, municipal and local governments, civil society, the private sector and other local community constituencies. Although PARA-Agua initially involved work in Ecuador, work in the country has been scaled back as there is no longer a USAID mission present and the focus is on watersheds in Colombia and Peru [1]. Objectives of PARA-Agua fall principally in the following general areas [1], [2]: 1. Strengthening the ability of national hydro-meteorological centers (IDEAM in Colombia and SENAMHI in Peru) and the national agencies with authority to approve watershed-level official planning instruments in the region to generate policy-oriented scientific data on hydro meteorological systems.1 1 This objective has been modified to reflect the relevant watershed management institutions where PARA-Agua is operating [2]. MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Final Evaluation Report 6 2. Ensuring that new climate data gets integrated regularly into decision-making processes by water policymakers community representatives (including women) and private sector representatives who are making decisions on water allocation and use through watershed management councils in three to five watersheds. 3. Strengthening planning systems that optimize water use over the whole length of watersheds glaciers through paramos to coastal lowlands -- in the context of climate change. The Intermediate Results for PARA-Agua correspond to the three main project tasks [4, 5] and are paraphrased as follows: 1. Intermediate Result 1: Capacities of the research community strengthened so as to generate policy-oriented data on watershed management and climate change adaptation; 2. Intermediate Result II: Climate data related to watershed management is mainstreamed and integrated into relevant decision-making; and 3. Intermediate Result III: Basin-wide participatory decision support systems in the context of climate change adaptation are strengthened. The project outcomes as a result of achieving these Intermediate Results include:  Improved climate data integrated into public and private investments in adaptation interventions;  New climate adaptation tools, technologies and methodologies developed and used by the research community, policymakers and representatives from local communities and watershed councils;  New networks and mechanisms for data sharing between researchers, policymakers, and watershed stakeholders on climate change adaptation;  Participation by local communities, including women and vulnerable populations, in developing climate change adaptation and watershed management solutions;  New investment in structural and non-structural adaptation interventions; and  Establishment of legacy organizations to continue project activities. The pathway to achieving these outcomes, the development hypotheses and critical assumptions involved are outlined in the PARA-Agua Results Framework. Key elements of the PARA-Agua strategy for achieving results include [8]:  An approach that is gender-focused and enables both women and men to influence policy and decision-making, employs strategies that respond to gender-based vulnerabilities, and promotes inclusion;  A fully integrated participatory approach that is grounded in focal watersheds and links all project components in a continuous cycle creating outcomes on the ground and bringing about project efficiencies;  Collaboration with USAID bilateral missions and donor partners to leverage capabilities and resources;  Close collaboration and consultation with counterparts at the national, regional, municipal and local levels. AECOM’s strategy and conceptual approach has been strengthened in FY 2015’s PMP in terms of gender mainstreaming and collaboration with counterparts at national, regional, municipal and local levels [5, 6] . The PARA-Agua contract has a performance period covering FY 2014 through FY 2015, with two one￾year options that would extend the period of performance through FY 2017. As of the end of FY 2014 MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Final Evaluation Report 7 all major tasks and activities are in progress both at the regional level in the Andean region of South America and with PARA-Agua currently working in four target watersheds [1], [2]. The decision to extend PARA-Agua will be based in part on an independent program evaluation. MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Final Evaluation Report 8 2.0 EVALUATION PURPOSE AND OBJECTIVES The purpose of the PARA-Agua PE is to provide insight into PARA-Agua activity performance and effectiveness beyond what has been captured in the PARA-Agua PMPs. To guide this PE USAID posed five key evaluation questions. The questions were adapted to reflect PARA-Agua’s operations and the relevant watershed management institutions where the project operates [1], [2]. The key evaluation questions are: 1. In what ways has PARA-Agua affected the policy capacity of national hydro-meteorological centers (IDEAM, SENAMHI) and the national agencies with authority to approve watershed-level official planning instruments in the focal countries? 2. To what extent has PARA-Agua built new, potentially sustainable links between researchers at hydro-meteorological centers, national agencies with watershed level planning responsibilities, and watershed management councils and authorities? 3. To what extent have the planning systems that watershed management councils or other water managers use been strengthened as a result of PARA-Agua’s work? 4. What new tools, if any, has PARA-Agua developed for water managers in the focal areas? Which have been used most often thus far? 5. If the evaluation recommends continuing with the activity, what changes would enhance the activity’s impact in the final years of implementation? In the process of addressing these evaluation questions, this PE will:  Document successes and shortcomings of the PARA-Agua project to determine its effectiveness in achieving the main project goals of enhancing Andean region organizations’ capacities to generate data and support the development of better watershed management planning systems whilst sustaining economic growth and protecting investments;  Provide information on project implementation including how it is perceived and valued by key stakeholders, notably water managers and policymakers throughout the Latin America region, specifically Colombia and Peru;  Determine whether program design, management and operational decisions have had an impact on the achievement of PARA-Agua performance and recommend course corrections;  Provide a recommendation regarding the continuation of PARA-Agua’s work beyond the current award and, if so, what steps or actions might be taken to improve the project’s effectiveness during the remaining implementation period. The intended audience of the PARA-Agua-PE includes Latin America and the Caribbean Office for Regional Sustainable Development (LAC/RSD), USAID/LAC Missions and USAID’s Global Climate Change office. These findings and information generated as a result of this evaluation will be used to both inform current programming and affect future programming in the climate change adaptation arena. MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Final Evaluation Report 9 3.0 METHODOLOGIES AND LIMITATIONS Data collection methods have been selected to provide high quality and rigor in answering the performance evaluation questions subject to existing time and resource constraints. The evaluation plan and strategies adopted in the course of carrying out this PE were discussed with and approved by USAID. To assess the performance of the PARA-Agua project the PARA-Agua-PE team employed a PE methodology that included a desk review of PARA-Agua documents followed by country visits to Peru and Colombia to conduct a series of KIIs and to participate in PARA-Agua field activities. DESK REVIEW The desk review entailed an in-depth assessment of PARA-Agua documents made available to the PE team by USAID and PARA-Agua project staff, such as annual work plans, performance management frameworks, quarterly reports, workshop and event summaries, subcontractor reports, training and outreach materials, diagnostic and baseline studies, research organizations and project partners commissioned papers and research, technical reports, meeting agendas and minutes, and the CoP virtual platform. These documents created a rich dataset for content analysis by the PARA-Agua PE team and achieved the following objectives:  Tracked progress toward expected project outcomes  Linked project activities to performance evaluation questions  Prioritized activities for PARA-Agua-PE analysis  Identified key substantive issues and data gaps to be focused on during country visits Amongst the most relevant sources used for the desk review and guidance for the field visits were:  PARA-Agua Quarterly Reports  PARA-Agua Work Plans and Performance Management Plans, including Performance Indicator Reference Sheets (PIRS)  Results of PARA-Agua diagnostic and baselines studies (Consultation Workshop Reports, Gender Analysis and Action Plans, Methodology to Define Prioritization Criteria of Climate Change Adaptation Measures)  MoUs and twinning partnerships agreements  Published articles from institutes and organizations associated with PARA-Agua  Presentations given as part of PARA-Agua project activities at local, regional, national and international workshops  Capacity building exercises reports/documents/tools developed by PARA-Agua subcontractors with researchers and watershed policymakers at international, national, regional, municipal and local levels  Technical reports and tools, for instance, Climate Scenarios development guidance and the toolkit titled Hydro-climatological Information for Decision Making MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Final Evaluation Report 10 FIELD VISITS AND DATA COLLECTION Following the desk review, the PARA-Agua PE Team identified three focal areas and used these in developing an effective strategy for structuring the KIIs and developing questionnaires to collect data and information in an efficient manner that aligns with the key evaluation questions. The three focal areas include: 1. Impact on Capacities of National Hydro-Meteorological Centers to Generate Relevant Climate Adaptation Data 2. Strengthening of watershed planning systems 3. Building or strengthening sustainable links between hydro-meteorological centers and/or national authorities at various levels with water planning responsibilities These focal areas inform the key evaluation questions (PE questions 1 to 5) and focus on:  Determining whether policy capacity of national hydro-meteorological centers and national agencies with authority to approve watershed-level official planning instruments in the focal countries has been improved (PE question #1);  Verifying to what extent PARA-Agua has built potentially sustainable links between researchers at hydro-meteorological centers, national agencies with watershed level planning responsibilities, and watershed management councils and authorities (PE question #2);  Establishing to what extent planning systems of watershed management councils or other water managers have been strengthened as a result of PARA-Agua’s work (PE question #3);  Verifying whether new tools are relevant and used by water managers (PE question #4). This question has been subsumed in the strengthening of water planning systems’ review area.  Asking for recommendations/adjustments in review areas that would enhance the activity’s impact (PE question #5). These contributions will provide relevant insights to the set of recommendations that will be developed by the Performance Evaluation Team. The key informant interviews used a semi-structured questionnaire that guided interviews with project participants in order to explore their perspectives on a wide range of service delivery activities undertaken by the PARA-Agua project. The main goal of the KIIs was to obtain more detailed and updated information than was available in the desk review and to better understand the PARA-Agua implementation process from the perspective of project participants themselves and in their own words. In selecting individuals to participate in the KIIs, the PE team made every effort to ensure the pool of respondents was sufficiently diverse. In addition to ensuring a balance of female and male respondents, the overall strategy for carrying out the KIIs involved purposefully sampling individuals from different institutions to obtain a wide range of perspectives. In particular, and in order to triangulate results and findings, the PE team obtained input from a range of individuals across the following groups: Group 1: Stakeholders with a role in watershed planning, management or research at the national or sub-national level. This group includes:  Specialists of hydro-meteorological centers;  Representatives of water or environmental government agencies holding policymaking responsibilities;  Members of Watershed Councils; and  Researchers at universities or international water/climate change adaptation programs. MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Final Evaluation Report 11 Group 2: Stakeholders with a role or interest in watershed planning and management at the local level. This group includes:  Local water board representatives with participation or interest in watershed management;  Women’s groups; and  Private sector representatives. Group 3: PARA-Agua project staff and subcontractors. This group includes:  PARA-Agua project staff; and  PARA-Agua contractors (i.e. Stockholm Environment Institute (SEI), The Mountain Institute (TMI), NCAR). The types of questions asked were tailored to prospective interviewees from each of the three groups. In order to use respondents’ time as efficiently and productively as possible and to collect information under the three focal areas so as to answer the evaluation questions, the KII questionnaires and interview materials were submitted to and approved by USAID in advance and prior to implementation. Copies of these materials are provided in Annex 3. With assistance from the PARA-Agua project staff, the evaluation team arranged a number of meetings and conducted interviews with individuals from national and sub-national institutions representing a variety of interests including:  PARA-Agua project staff  USAID Mission’s Environment Teams in Bogota, Colombia, and Lima, Peru  Representatives, scientists and specialists from SEI, TMI, and NCAR  Watershed Councils’ managers of four watersheds, Watershed Councils’ members and Watershed Councils’ Technical Secretariat staff  National hydro-meteorological institutions specialists  National Water Agency of Peru representatives  Women at various levels who have participated in PARA-Agua activities, including watershed-related planning and research  Peruvian Irrigation Board representatives  Private companies providing water and water-related services (Colombia) The evaluation team undertook country visits to Peru (May 10-16, 2015) and Colombia (May 17-23) to conduct a series of interviews with persons representing a range of institutions and whose interests link with different functions and activities associated with the PARA-Agua project. The timing of the country visits also allowed the evaluation team the opportunity to witness PARA-Agua activities first-hand. The PE team used the occasion to participate in and observe the PARA-Agua workshop in Arequipa, Peru and twinning partnership activities in Manizales, Colombia to gather information and gain insight into project activities, in particular, those relating to capacity building at the sub-national level, developing sustainable linkages, and the effectiveness of the watershed management tools developed to date. In total the evaluation team met with and gathered input from approximately 50 individuals – with women accounting for 35 percent of the respondent pool. Roughly one third of those interviewed were most closely linked with PARA-Agua activities in Colombia with the remainder being in Peru, which is where most of the PARA-Agua staff are based. MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Final Evaluation Report 12 TABLE 1. DISTRIBUTION OF KIIS AND PERSONS INTERVIEWED Colombia Peru Total Total # Female Total # Female Total # Female Group 1 9 3 20 5 29 8 Group 2 3 1 3 1 6 2 Group 3 6 3 10 5 16 8 Total* 18 7 33 11 51 18 Source: Evaluation team tabulations. Notes: Due to overlapping roles, members of Group 2 were interviewed in both their role of representatives of their respective constituencies and Watershed Councils’ members. As a result, during the course of the KIIs there were opportunities to ask questions and collect information across groups. However, individuals were counted only once in this table. LIMITATIONS The PARA-Agua PE, like all evaluations, faced limitations with respect to time and resources which are described below. However, despite these, authors believe the evaluation represents a fair and balanced assessment of PARA-Agua to date. Given the short time frame for completing this performance evaluation and the subsequent practical limitations that existed, the PE team was limited in the amount of data and information which could be collected. While the country visits were well organized and the evaluation meeting schedule was full, the week spent by the evaluation team in each country did not allow meetings with all possible stakeholders at the various institutional and regional levels (including stakeholders in rural areas). Also, Watershed Councils in both countries have financial limitations and as a result are not always able to cover their members’ participation in meetings, making it difficult to engage them in activities (e.g., Chira Piura (Peru) and Chinchiná (Colombia)). Nevertheless, the PE team had the opportunity to meet with four Quilca Chili (Peru) Watershed Council members who self-financed their own travel to attend an interview scheduled in Arequipa, which allowed the PE team to speak with two rural constituents. During the field visits, the distribution of interviewees weighed more heavily towards the first and third groups with less coverage of second group of interviewees than what was originally planned. In particular, the evaluation team was not able to meet with any member of women’s groups (i.e., the Regional Council of Women (COREM) in Chira-Piura, Organization of Women in Morrison, Regional Federation of Rural Communities in Quilca Chili, Federación de Mujeres Campesinas de Cambita (FEMUCAM) of Quilca Chili, and Women’s Coffee Organization in Chinchiná). However, the PE team discussed gender-mainstreaming progress in Peru when Watershed Councils’ stakeholders, including one woman representing the Water Irrigation Boards who made reference to Action Plan activities tackling related specific activities that would be undertaken with PARA-Agua. Watershed Councils’ interviewees stressed that women and climate change issues with PARA-Agua will be addressed in Water Culture components of Water Management Plans. Finally, while many of the interviewees and meetings were recommended and organized by PARA-Agua, members of the Evaluation Team were also able to seek out and interview other stakeholders on a number of occasions during the course of the country visits. As a result, the team was able to reduce the potential for selection bias, further strengthening the overall approach and leads to greater confidence in the results obtained. MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Final Evaluation Report 13 4.0 FINDINGS This section highlights the findings based on the desk review of the PARA-Agua documents and project materials in combination with the data gathered during field visits such as from key informant interview transcripts and notes taken while participating in workshops or attending formal meetings with project participants who work in PARA-Agua participating organizations and institutions. In order to evaluate progress, this section is organized by evaluation question. All quotations, unless stated otherwise, are translated responses of comments collected during the course of carrying out the KIIs. 4.1 EVALUATION QUESTION #1 IN WHAT WAYS HAS PARA-AGUA AFFECTED THE POLICY CAPACITY OF NATIONAL HYDRO-METEOROLOGICAL CENTERS (IDEAM, SENAMHI) AND THE NATIONAL AGENCIES WITH AUTHORITY TO APPROVE WATERSHED-LEVEL OFFICIAL PLANNING INSTRUMENTS IN THE FOCAL COUNTRIES? STRENGTHENING CAPACITY OF NATIONAL HYDRO-METEOROLOGICAL CENTERS One of the findings of Gap Analysis [10] was that in order to generate climate information relevant to policy formulation and to leverage investment funds for adaptation, it was essential for PARA-Agua project staff members to work with the national hydro-meteorological centers (IDEAM, SENAMHI) and other national agencies with authority to approve watershed-level official planning instruments. The national hydro-meteorological institutions, IDEAM and SENAMHI, face demands from stakeholders at various levels, including regional and local. However, a significant bottleneck is the limited staff and financial capacity that national agencies have to provide timely and effective information for watershed planning at the local and regional levels. PARA-Agua engages at the national as well as sub-national level and has the difficult task of creating linkages between these actors while at the same providing cutting edge technical support and state-of-the-art tools for climate adaptation modeling. Colombian and Peruvian experts and researchers of hydro-meteorological institutions appear eager to build capacity and to engage with experts from NCAR who are widely respected in the climate modeling and research community. With NCAR as part of PARA-Agua team, the Project appears to be in a strong position to strengthen national capacities to generate relevant hydro-meteorological data and strengthen the linkages with water planning bodies at various levels so these data can be used effectively. Indeed, the perception of government agencies with water planning mandates underscored the consistent contributions of the PARA-Agua project with regards to enhancing their policy capacities. One respondent in Peru indicated: “We have subscribed around 30 or 40 inter-institutional agreements. Out of these 30 or 40 agreements, three or four are working fine; one of them is PARA-Agua.” This sentiment was shared by a large number of respondents who view PARA-Agua as being effective and having the ability to bring about tangible results while operating in a difficult environment and where they are building capacity with innovative methodologies and tools and having recourse to climate data and technical expertise not necessarily available beforehand. MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Final Evaluation Report 14 While PARA-Agua has certainly made in-roads supporting SENAMHI in Peru and IDEAM in Colombia, the consensus is that these relationships have been somewhat slow to materialize, particularly in Peru. For example, despite an agreement in principle for a scientific twinning partnership between the SENAMHI, IDEAM, and NCAR [3], to date, agreements with IDEAM and SENAMHI have yet to be formally entered into. Interview respondents from hydro-meteorological institutes in both countries expressed interest in reaching such an agreement. At least one hydro-meteorological research center representative stated that its scientists had provided feedback on the scope of work, confirming the participatory nature of the partnership agreement preparation although such an agreement has not yet been reached. In discussing hydro-meteorological downscaling modeling techniques with NCAR and SEI [11], a number of respondents indicated that the appropriate counterparts from SENAMHI were not adequately consulted about the methods and technical details. Without SENAMHI endorsing the climate scenarios’ downscaling, results proposed by PARA-Agua will not be approved. This contrasts with how PARA￾Agua Project staff and subcontractors perceive the situation. They indicated PARA-Agua and SENAMHI collectively generated information (for instance, forecasting of rainfall), with the purpose of contributing to the national environmental information system led by ANA, which would also collect hydro￾meteorological information generated by SENAMHI. For IDEAM in Colombia, in contrast, it appears the approach was more inclusive and that they felt they were adequately consulted by PARA-Agua in the process of selecting the appropriate climate scenario modeling framework. KIIs with other stakeholders in Colombia highlighted progress in the validation of information, and the discussion and validation of hydro climatic models. Discussion forums provided by PARA-Agua have allowed a dialog between IDEAM and Chinchiná Basin stakeholders about the possibility of establishing a Hydro-Climatological Forecast Center for the Colombian coffee region that would be established in the Chinchiná Basin. This achievement would represent a landmark for PARA￾Agua as per the various stakeholders interviewed. PROTOCOLS TO INTEGRATE PUBLIC AND PRIVATE RESEARCH ORGANIZATIONS INTO NATIONAL INFORMATION SYSTEMS Government institutions working on hydro-meteorological information management, such as IDEAM and SENAMHI, identified the need to develop protocols and standards for incorporating climate research groups of universities, and thus meet the research demands of watershed planners. In some cases, Peruvian regional governments may cover the costs for the development of planning tools that include analysis of climate change, however SENAMHI does not have the staff to handle a large demand. Moreover, this activity was identified as a project priority at a multi-stakeholder consultation workshop held in Lima, with representatives of relevant government institutions [11]. The PARA-Agua proposed initiative would streamline institutional responses to address this situation and would be conducted in close coordination with national environmental authorities. One of the toolkits under consideration by PARA-Agua seeks to establish protocols to incorporate public and private organizations into national information systems for climate change adaptation in watersheds [3], would be developed in the first quarter of 2015 and would also be updated with inputs from stakeholders. However, during KIIs, some respondents noted that there is no consistent understanding and/or undertaking of this task and stressed they have not worked with PARA-Agua on developing these guidelines. Others, on the other hand, highlighted Protocols development, as a PARA￾Agua work in progress. One Watershed Council Stakeholder in Peru noted: MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Final Evaluation Report 15 “We have not had the opportunity to work on the protocols [on integrating public and private organizations into national information systems for climate change adaptation in watersheds] yet, nevertheless there was a workshop in Lima targeting protocols. We would appreciate a specific workshop for our watershed. The National Climate Change Strategy does have a protocol, as does the Water Resources Management Plan”. Another Watershed Council stakeholder in Colombia stated: “An instrument that would be helpful is to formalize information protocols. This would be an exercise that could provide sustainability to communications between hydro-meteorological centers and decision makers. The protocols are being designed [with PARA-Agua]”. As well, it was also noted that systems providing the information on which the modeling and adaptation strategies are based, do not fully collect socio-economic and cultural variables, including gender[12]. Several of the experts consulted at the hydro-meteorological centers consider socio-economic concerns outside their scope of work. This perception could represent a limitation for later adaptation and resilience strategies that respond to the interests and needs of both men and women. 4.2 EVALUATION QUESTION #2 TO WHAT EXTENT HAS PARA-AGUA BUILT NEW, POTENTIALLY SUSTAINABLE LINKS BETWEEN RESEARCHERS AT HYDRO-METEOROLOGICAL CENTERS, NATIONAL AGENCIES WITH WATERSHED LEVEL PLANNING RESPONSIBILITIES, AND WATERSHED MANAGEMENT COUNCILS AND AUTHORITIES? LINKAGES WITH NATIONAL AND SUB-NATIONAL AUTHORITIES During the KIIs, the majority of interviewees underscored how the various PARA-Agua activities built sustainable links between researchers at hydro-meteorological centers and stakeholders’ water planners. For example, one respondent in Peru indicated: “The added value of the WEAP capacity building supported by PARA-Agua is the climate change component. With PARA-Agua we work on strengthening capacities of climate scenarios in modeling groups and having forums for debates with other institutions.” In Colombia, the efforts to develop climate tools and undergo climate scenario modeling at the local level are evident as indicated by a respondent who stressed: “WEAP is just what we needed. There is a post WEAP situation. When we were undertaking the diagnostic phase the [WEAP] model arrived. It is a technical tool. The juncture was timely and valuable: We knew what to do with the information of the POMCA (Plan of Basin Development and Planning) diagnosis”. However, while the tools and models developed under PARA-Agua for the individual watersheds draw heavily on the information generated from national institutions and the researchers at those institutions, there appears to be a disconnect between these national institutions and those at the regional and local level tasked with implanting water policies. As one respondent in Peru noted: “Technology is being brought to institutions [as SENAMHI]. The challenge is to bring these institutions closer to the watersheds. Institutions as SENAMHI tend to generate data for high-level scientists, not for MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Final Evaluation Report 16 stakeholders at the watershed level. SENAMHI is the official source of information and they certify it [the information]”. Similarly, a respondent in Colombia indicated that: “From our regional perspective, we feel IDEAM is a distant institution. It has been difficult to generate possibilities to have access and contact with them for planning purposes. But the PARA-Agua project has created spaces where we exchange with each other”. An additional issue noted by several of the KIIs is the inability to obtain relevant data from the national authorities – both SENAMHI and IDEAM - in a timely and efficient manner. One possible reason for this could be resource constraints limitations of these national institutions to provide timely and effective information services to watershed and other territorial planning instruments, as determined in several PARA-Agua reports [10], [11]. Still, prior to PARA-Agua there appears that there was very little discussion or interaction at the various levels and that the project has been key in facilitating that discussion. For example, a Watershed Council stakeholder in Peru stated: “There was definitely a need for a dialogue with SENAMHI and watershed councils. PARA-Agua facilitated these types of data exchanges with SENAMHI”. A regional government stakeholder in Colombia noted: “The latest evolvement is a proposal to create a Regional IDEAM Forecast Center. This would be one of the milestones of PARA-Agua. In the different spaces facilitated by PARA-Agua we work with the same stakeholders. The Project has facilitated knowing a little more about strategies that arise at the national level and that sometimes have no answer in the region. Initially IDEAM thought there was no need for regional level interactions before installing the Forecast Center. Now we have a common language regarding the need to link the various meteorological networks”. A Watershed Council stakeholder in Colombia said: “Training workshops have facilitated recognition between stakeholders at different levels. The fact that IDEAM and other regional entities involved participate has allowed us to recognize and validate what each one is doing, generating information”. Interviews conducted in both countries highlighted that the evolving environmental, climate change and water information systems are complex with regards to the different institutions and levels involved. In Peru, for instance, there are several information systems being designed including the National System of Environmental Information of the Ministry of the Environment (MoE), the National System of Regional Environmental Information, and the National Information System of Water Resources (agreement between National Water Authority of Peru (ANA) and SENAMHI). At the municipal level, some municipalities are beginning to develop Local Systems of Environmental Information. In Colombia, interviewees underscored that they are also discussing how to generate a single information system at the regional level that would bring together the information generated by different bodies. In both Colombia and Peru, the fact that information systems are being developed and strengthened provides PARA-Agua with an opportunity to provide timely support to the strengthening of one or several of these national processes. MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Final Evaluation Report 17 CHALLENGES TO BUILDING INSTITUTIONAL LINKAGES Sub-national water management and planning structures frequently have new members, including, new Watershed Council members, regional directors of government offices, and technical staff. Training accompanies each new appointment. Interviews with representatives from Watershed Councils identified basin understanding (dynamics, basin components and challenges, etc.) as the most relevant capacity building activity to be undertaken for new Watershed Councils members. New Watershed Councils members also need training on climate change adaptation approaches, instruments and tools provided by PARA-Agua. As stressed by several interviewees, the purpose is not to make experts on climate change modeling scenarios, but to help the Watershed Councils members to better understand the basics, relevance and implications of tools as the WEAP model. Such knowledge would allow them to participate in Watershed Councils activities related to water planning and climate change adaptation. For now it seems this activity is not included in the PARA-Agua activities’ portfolio. PARA-Agua could monitor to ensure that this type of capacity building activity is undertaken by Watershed Councils Technical Secretariats themselves in Peru, for example. A wrong assumption on the part of PARA-Agua staff would be that their work is mainly associated with Councils members who are representatives of professional bodies and universities. Watershed Councils have a number of technical staff, including the Technical Secretariat, and in the case of Peru hydrologists, economists, Geographic Information Systems (GIS) technicians, and communications experts, though smaller watersheds may not have recourse to this entire staff. The technical staff remains in the watersheds. In Peru, interviewees noted that the Project has provided them with relevant and updated information, tools and methodologies to enhance their water planning and management capacities. As an illustration, one interviewee highlighted the fact that, as a result of a PARA-Agua training held in Lima, a Watershed Councils Technical Secretariat staff person could reformulate his or her work plan to assess potential eutrophication in water reservoirs, a key problem in the watershed. TWINNING PARTNERSHIPS Twinning partnerships are a central part of building sustainable links between researchers at hydro￾meteorological centers, national agencies with watershed level planning responsibilities, and watershed management councils and authorities. PARA-Agua has established MoUs between the Chira Piura and the Chinchiná basins[13] and the Yolo County Flood Control and Water Conservation District (USA) and the National Water Authority (Peru)[14]. In the case of the twinning partnership between the Yolo County Flood Control and Water Conservation District and ANA, KIIs confirmed that Watershed Councils and ANA staff would like to learn about the items defined in the MoU, including watershed management, how water fees are charged, and how infrastructure maintenance is done. The PE Team attended twinning partnership activities between the Chinchiná Watershed stakeholders and Chira Piura and Quilca Chili Watershed Councils members and Technical Secretariat staff held in Manizales, May 19-21, 2015. Interviewees highlighted the relevance of these activities not only for experiences’ exchange benefits, but also for strengthening Watershed Councils as relevant bodies in water planning and management in the context of climate change adaptation. Hence twinning partnership activities contribute to strengthen the legitimacy of Watershed Councils as relevant MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Final Evaluation Report 18 institutions in the context of natural resources planning and management. Further, these meetings also allowed for the exchanging of experiences and learning about the mission and mandates of Watershed Councils, and facilitated understanding of the scope of water planning instruments, and issues surrounding regional and national information systems for managing a variety of water, climate, and environmental data in each country. COMMUNITY OF PRACTICE The CoP virtual platform has the potential of, among other objectives, strengthening institutional links. Developed through a consultative process [15], many of the interviewees expressed having participated in the consultations leading to its establishment though not many of them had navigated the virtual platform, which was launched when the KIIs were being held. Interviewees who did navigate the platform considered the materials relevant and were looking forward to participating in the forums to exchange their experiences. The PE Team cannot issue an opinion right now on the pertinence of the CoP since it is still new. However, the central findings of the CoP Consultation Report underscored that overall, respondents do not use virtual platforms for experience exchange. Those who use them look more for information or tools needed immediately for a project that they are currently working on. Hence, the time spent on the virtual platform will be minimal, unless timely information is provided [15]. These findings should be carefully considered so that the CoP virtual platform meets users’ expectations. As per the KIIs with the PARA-Agua Team, the CoP went through several delays due notably to branding requirements. The CoP is currently operational in the sense that users can register and have access to posted news, information and sources. The different fora for discussion are not working yet. The draft version of the Hydro-climatological Information for Decision Making is available in the CoP. The preliminary version has been posted in the CoP under the tab 'Under Construction' ('Construir' in Spanish). The CoP presents the toolkit in a user-friendly way (varying colors, concise text, use of diagrams, easy navigation between the different sections). CoP users can post comments on the bottom. During the KIIs with the PARA-Agua team, interviewees underscored that the CoP will have a moderator and an IT person. A stable CoP team, with access to relevant materials and topical forum discussion themes (among others) provided on a regular basis by PARA-Agua staff and subcontractors, is central to keep interest in the platform alive. The platform has the potential of contributing to the establishment of sustainable links between researchers at hydro-meteorological centers, national agencies with watershed level planning responsibilities, and watershed management councils and authorities. Moreover, the CoP website has been designed to capture the gender of users [3]. LEGACY INSTITUTIONS Legacy institutions and agreements refer to the continuity of activities initiated under PARA-Agua and passing these on to other organizations and institutions who will continue the legacy. CIIFEN and TMI have been identified to take over CoP-related activities. It is expected that CIIFEN’s involvement will reinforce CoP legacy efforts. The legacy institution alliance between CIIFEN and TMI was expanded by PARA-Agua to include the CONDESAN. Upon the completion of PARA-Agua activities, the alliance will implement a sustainability strategy, with defined roles and responsibilities that will continue to advance the goals of PARA-Agua over the long-term[3]. In this regard, one of PARA-Agua’s key objectives is to MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Final Evaluation Report 19 strengthen national capacities to generate policy-oriented data for watershed management and climate change adaptation and water use planning systems. One of the expected final results at the end of PARA-Agua is the sustainability and persistence of capacities developed at national and sub-national levels in each of the intervention countries. The emphasis on international institutes and alliances should not disregard the fact that, whenever possible, roles and responsibilities should come back to the main stakeholders of the Project, (i.e. researchers at universities and specialists at hydro-meteorological centers; staff working at national agencies with watershed level planning responsibilities; members, technical staff and watershed management councils; and local authorities). GENDER PARA-Agua staff and its subcontractors are consciously pursuing an inclusive approach by actively engaging qualified women, including country coordinators and scientific and technical specialists. This is particularly evident with SEI staff, which currently includes two female specialists, one in each of the two Peruvian watersheds, who are working in a rural environment where women’s participation in water and hydrological activities is traditionally limited. Interviews with watershed stakeholders underscored the high-level and quality support provided by these two women, in addition to that provided by PARA￾Agua staff and subcontractors in general. Several respondents indicated that PARA-Agua has been very successful in their capacity building and the input and expertise provided by the female experts has been of very high quality. 4.3 EVALUATION QUESTION #3 TO WHAT EXTENT HAVE THE PLANNING SYSTEMS THAT WATERSHED MANAGEMENT COUNCILS OR OTHER WATER MANAGERS USE BEEN STRENGTHENED AS A RESULT OF PARA-AGUA’S WORK? The Gap Analysis identified lack of experience in conducting cost-benefit analysis of climate change impacts, including social and economic factors, and climate change impacts on water availability in the water-planning arena. PE findings (desk review and KIIs) corroborate the fact that PARA-Agua activities have supported the strengthening of climate information use in planning systems at the sub-national watershed level. For this purpose, PARA-Agua signed MoUs with the ANA [16] and the Autonomous Regional Corporation of Caldas (CORPOCALDAS) in Colombia[17]. WATERSHED MANAGEMENT COUNCILS Watershed Councils recently have been introduced in both countries, bringing together representatives from diverse sectors with frequently divergent interests on watershed planning and management. In Peru, Watershed Councils are partially decentralized bodies working within sub-national jurisdictional units, known as the ‘regions’. This contrasts with the Chinchiná Basin in Colombia, where watershed management is more of a hybrid structure where public institutions (including the regional corporation and the watershed council), the private sector, and academia converge in the jurisdictional region. In both countries Watershed Councils are still fragile bodies, with no or limited funding, whose members participate supported by their constituencies, or with their own means, e.g., community or Non￾Governmental Organization (NGO) representatives. All respondents acknowledged PARA-Agua’s significant technical support at the watershed level and that stakeholders at this level have enhanced technical capacities as a result. Besides Watershed Councils, in MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Final Evaluation Report 20 Peru, the Autonomous Water Authority, Local Water Authority, and Irrigation Boards have benefited from PARA-Agua assistance as well. This is also the case of CORPOCALDAS in Colombia. The application of watershed climate models in the formulation of public investment for climate change adaptation was identified as a relevant priority in the Regional Consultation Workshop[11]. Watershed Councils’ members and Technical Secretariats staff in both countries appreciated the climate modeling scenarios development, facilitated by PARA-Agua. NCAR completed data sets for FY2014 target watersheds in the Northern Andean Region based on the latest results from the Intergovernmental Panel on Climate Change (IPCC). Work also has begun on new datasets for Quilca-Chili[8]. PARA-Agua partners appreciate the intensive, hands-on training for decision making in water management institutions provided by the PARA-Agua team and subcontractors. The goal of developing a cadre of managers and technical staff within key watershed organizations who are capable of employing the WEAP model for support planning and decision-making is well underway. Working with trained stakeholders, PARA-Agua has generated a series of ‘model runs’ under the various scenarios identified earlier to produce information on the performance of different management strategies in the face of various levels of climate uncertainties. In addition, the capacity and technical expertise of Watershed Councils that result from the PARA-Agua activity also appears to reinforce the legitimacy of these relatively new bodies as authorities in the context of climate change adaptation planning and management. The Improved and relevant water planning and management tools help to leverage the political positioning of Councils’ decision-making processes. REGIONAL WORKSHOPS Chira Piura, Quilca Chili and Chinchiná watersheds interviewees underscored the high regard for tools developed by PARA-Agua. Replication of PARA-Agua’s capacity building has already taken place. The Chira Piura Watershed Councils Technical Secretariat is using its own resources to support staff in order to provide training and technical assistance that draws heavily from PARA-Agua materials to neighboring watersheds (Tumbes and Chancay-Lambayeque). Interviewees praised PARA-Agua’s methodologies regarding climate change adaptation initiatives prioritization, climate scenarios development, interpretation of hydro-meteorological info, and project development. The PE Team attended a workshop in Arequipa (Quilca Chili watershed) involving the use of the ‘R’ tool (targeting Models in the XLRM analysis). Workshop attendance allowed the evaluation team to gain insight into the pedagogical approach employed while seeing firsthand how PARA-Agua staff and their subcontractors (SEI female and male experts and a climate change expert from NCAR) engage and involve participants during workshop activities. Stakeholders whose priority is their water planning and management instruments stressed that PARA￾Agua methodologies and tools should correspond to their water planning priorities and appreciated PARA-Agua’s willingness and openness to take on board their main concerns. Nevertheless, several stakeholders stressed several further adjustments are still required to fully distill their planning priorities using PARA-Agua’s tools and instruments. For instance, some of them stated that the current WEAP model allows a limited assessment of which economic and social indicators the Water Management Plan is impacting and could benefit from providing better coverage of socio-economic aspects such as gender or considering the role of specific sectors like agriculture and the diversity of water users associated with this sector. MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Final Evaluation Report 21 Moreover, in some workshops using evaluation sheets and the KIIs, participants stressed that the time devoted to developing workshop themes is limited. Adding an afternoon or a morning to the workshop was proposed as an option to solve this issue. FINANCIAL INVESTMENT IN AND FUNDING FOR ADAPTATION Adaptation involves not only applying climate-modeling scenarios but also supporting Watershed Councils to implement prioritized climate change adaptation projects. From the outset, Project stakeholders considered that PARA-Agua results should lead to specific actions in national public investment systems to influence climate based decision-making regarding investments in infrastructure or technical assistance[18]. In this regard, national hydro-meteorological centers (IDEAM, SENAMHI) and national agencies with authority to approve watershed-level official planning instruments provide the legal framework for the investment of public funds in watershed management projects[10]. Numerous interviewees noted that they need PARA-Agua support to find funding for climate change adaptation projects and interventions at the basin level. KIIs confirmed that expectations are high for the technical advice being or to be provided by PARA-Agua in this regard. Given the challenges and difficulties associated with navigating the world of donor funding, proposal writing, technical reporting and documentation required, etc., the PARA-Agua project has recruited consultants in Colombia and Peru (one in Colombia and two in Peru), along with an international expert to assist Watershed Councils, regional authorities and watershed partners with this process. In Peru, this type of work will also be linked to the other element of this task, i.e. working on the Regional Water Fund (FORASAN) financial and legal structure. The regional government would manage the Fund since Watershed Councils do not have legal entity status to do so. Therefore, two consultants have been recruited for the two different kinds of support foreseen for this country, (i) identifying and presenting proposals for watershed level climate change adaptation projects and interventions, and (ii) supporting the creation and establishment of the Regional Water Fund of Chira Piura, the first of its nature in Peru. The PE resulted in no evidence that the establishment of a Water Fund would also be undertaken in Colombia. If this is not the case, it would be useful to explain the rationale of this decision, particularly considering Watershed Councils in Colombia do not have a legal entity status either. In spite of some delays, this activity is progressing in Chira Piura. Until now a preliminary structure and organization for the Water Fund have been designed, involving the public water company (EPS Grau), the potential fund manager Fondo de Promoción de las Areas Naturales Protegidas del Perú (Fund for the Promotion of Natural Protected Areas of Peru), and the private sector. Further coordination meetings are still needed to finalize the fund structure and assure its sustainability[3]. GENDER Mainstreaming activities associated with PARA-Agua overall gender strategy is largely just beginning. The recruitment of a gender specialist in January 2015 should accelerate gender mainstreaming across the PARA-Agua project activities. PARA-Agua has two main gender components: gender awareness and highlighting the role of women in data collection. Based on field visits and the Specific Gender Work Plan for Peru, Gender Action Plans adapted to each watershed are being finalized, which would allow tailoring gender actions to watersheds’ contexts and priorities. In Colombia, the Gender Action Plan for Chinchiná has not yet been finalized. In Manizales, the ‘Guardianas de la Ladera’ (Hillsides’ Guardians – MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Final Evaluation Report 22 female substantive in Spanish) program links payments for environmental services schemes and gender issues in the coffee sector. A visit to ‘Guardianas de la Ladera’ program site was organized during the KIIs, but travel to Bogota did not allow the PE team to participate. In Colombia it is possible to strengthen women’s participation in the Watershed Councils via the coffee agroindustry. Documentation from workshops highlight PARA-Agua’s attention to women’s participation [19], [20] and this was also noted by the interviewees during the KIIs. Gender-related activities during the second quarter included the identification of new women’s groups within the target watersheds. Five new women’s groups have been brought into the project and soon will take part in project-sponsored trainings [3]. The PE team was not able to meet with women’s groups because they did not attend PARA-Agua activities during PE country visits; however a representative of a rural organization indicated that the PARA-Agua Project had initiated contact with rural women of the watershed. The respondent added they agreed to jointly work the ‘Water Culture’ component of the Water Management Plan with PARA-Agua so that it includes gender and climate change issues. As part of PARA-Agua efforts, watershed stakeholders also stressed a trained existing regional network of communicators will support gender and climate change themes in the ‘Water Culture’ component of the Water Management Plan. In the course of conducting interviews one issue raised is that in some areas local practices may limit or exclude women’s involvement. This particular issue is being addressed by PARA-Agua under its Gender Action Plan where in working with the Chira-Piura Board of Irrigation Users to adjust their regulations, which currently limit women’s involvement, and instead incorporate women’s participation mechanisms [21]. However, at the time of the country visits this had yet to be implemented. Interviews conducted with Watershed Councils members and technical staff also revealed that there is a growing awareness of social interest and signaled an interest in tracking gender issues in particular given the differential effects of climate change on men versus women. 4.4 EVALUATION QUESTION #4: WHAT NEW TOOLS, IF ANY, HAS PARA-AGUA DEVELOPED FOR WATER MANAGERS IN THE FOCAL AREAS? WHICH HAVE BEEN MOST USED MOST OFTEN THUS FAR? Up to now the most relevant toolkits, tools and instruments developed by PARA-Agua that target water managers include:  The Draft Toolkit 1 – Hydro-climatological Information for Decision Making  Modeling of climate scenarios and robust adaptation options tools developed, adapted and refined by NCAR (such as adapting WEAP to the key watersheds)  Hybrid dynamical statistical downscaling has been used to generate regional climate models results from Weather Research and Forecasting Model – the latter is a flexible, state of the art Regional Climate Model. To model, prepare and run these tools, NCAR assembles information on regional climate data and projections from Global Climate Model results archived through the Inter-Governmental Panel on Climate Change Assessment Report-5 (IPCC). The use of this information was highly appreciated by participants. NCAR report[22] indicates Activity 1, Activity 2.1, and Activity 3.1 and 3.2 have been completed. NCAR Activity 4, mainly developing WEAP databases, would be pending. MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Final Evaluation Report 23 PARA-Agua’s first toolkit, Hydro-climatological Information for Decision Making, is a draft instrument, currently available in the virtual platform (Community of Practice) for comments. The toolkit is a well￾developed instrument, aiming to strengthen the generation of scientific information that is relevant for decision-maker at different levels. The toolkit identifies several steps, including the identification of information users, understanding of the technical and legal framework of the decision-making space, evaluation of information needs, uncertainty degree determination, and results sharing. Insights from practitioners, scientists and experts will be central to refine the tool and enhance its pertinence [23]. The toolkit, based on input received from regional training workshops, was completed in Q4 FY2014. The final draft version of the toolkit is being shared via the CoP. At the time of the country visits, interviewees indicating they had participated in consultations in the development stage, but were not aware that it had been finalized. Since the toolkit had not been widely disseminated it is not possible to comment on the effectiveness and usefulness of this particular tool to date. However, there was interest among Watershed Councils members and technical staff in both countries and it was suggested that the toolkit be shared more broadly with the appropriate regional counterparts as it is potentially useful for informing policy. A second toolkit, aiming to establish protocols to incorporate public and private organizations into national information systems for climate change adaptation in watersheds, would be developed in the first quarter of 2015 and should also be updated with inputs from project stakeholders. However, several respondents did not have a clear or unanimous understanding of this task. A draft of the second toolkit was not available at the time of this evaluation. Feedback from PARA-Agua workshops and training has been very positive based on feedback from the KIIs and reviewing workshop evaluation sheets. Interviewees highlighted the high quality of the workshop, the hands-on experience and relevance of the topics, which frequently are adapted to watershed planning and management priorities. Stakeholders nevertheless requested further refined adaptation of tools to watershed contexts and location-specific concerns. Modeling of climate change scenarios is consistently highlighted as pertinent for adapting to the impacts of climate variability, particularly by watershed stakeholders, but also by the national hydro-meteorology institute of Colombia. Interview respondents were impressed with the scientific and technical rigor used in developing the watershed modeling tools. To model potential climate change impacts and evaluate possible robust adaptation options, NCAR assembled information on regional climate data and made projections from Global Climate Model results archived through the IPCC Assessment Report-5. NCAR compared these results with historical climate observations. Using Bias Correction and Spatial Disaggregation methods, NCAR also developed a WEAP compatible dataset from this archive. NCAR has continued to refine their climate scenario tools developed for the Year 1 basins (Chira-Piura in Peru and Chichina in Colombia), including WEAP, global climate projections from the IPCC Climate Model Intercomparison Version 5, and from the newly created, regional climate projections made by NCAR under the Year I scope of work [8]. Key informants who participated actively in PARA-Agua capacity building activities were familiar with the diverse tools and methodologies offered by the Project and were pleased to see the project was using the most current data and state-of-the-art techniques being applied to their regions. Several interviewees stated that the WEAP model already existed and that they were familiar with it, however the model they knew did not include variables such as climate change adaptation or water MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Final Evaluation Report 24 resources’ availability. The incorporation of these types of variables in the WEAP model, introduced by PARA-Agua, was highlighted as one of the most important project contributions. Project stakeholders, particularly at the watershed level, emphasized a substantial improvement in the WEAP model, particularly regarding water rationing and demand planning (reservoirs, industrial centers, agroindustry, among others), model calibration, and uncertainty models, including the consideration of population increase and decrease, and forestation and reforestation programming. Stakeholders in both countries highlighted the aspiration and need to work on other WEAP modules that are relevant for them, for instance, water quality, underground water, and flooding. This type of analysis has not been undertaken yet, but it is a possibility, as the WEAP tool can be adapted to watersheds priorities. However, in order to do so requires additional model verification in addition to supplementary information, for instance, data on sub-surface water. In the case of Peru, although SENAMHI does not collect data on underground water, hydrological studies conducted by ANA and some private companies have gathered sub-surface water data. However, such data needs to be accessible by ANA or publically available (in the case of data collected by private companies) in order to supplement the climate modeling at the local level. 4.5 EVALUATION QUESTION #5 IF THE EVALUATION RECOMMENDS CONTINUING WITH THE ACTIVITY, WHAT CHANGES WOULD ENHANCE THE ACTIVITY’S IMPACT IN THE FINAL YEARS OF IMPLEMENTATION? This question is addressed in the recommendations section, 6.0. MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Final Evaluation Report 25 5.0 CONCLUSIONS The PARA-Agua project is a timely intervention in strengthening watershed planning and management in both countries. Peru and Colombia are countries with different policy contexts on water planning and management. Peru has a more centralized structure with a team of technical staff supporting pilot Watershed Councils at the regional level under the ANA’s jurisdiction. This contrasts with Colombia where watershed management is more of a hybrid structure where public institutions, the private sector, and academia converge in jurisdictional regions. In both countries efforts are underway to improve watershed planning and management in the context of climate change. In support of this, Watershed Councils have recently been introduced in both countries, bringing together representatives from diverse sectors with frequently divergent interests on watershed planning and management. The scientific tools and support provided by PARA-Agua has provided Watershed Councils with sound scientific data and arguments to base policy recommendations. Moreover, as stressed during the KIIs, council members felt that working with PARA-Agua has provided technical support while at the same time providing access to the tools and skills necessary to effectively contribute to watershed planning and management efforts. As a result, their own work is seen as more legitimate and is gaining recognition more broadly at the political level. PARA-Agua is well positioned to further support Watershed Councils. In both countries Watershed Councils are still fragile bodies, with no or limited funding, whose members participate through support of their constituencies or with their own means, e.g., community or NGO representatives. To ensure sustainability and provide a steady source of funding to support these Watershed Councils and their advisory or executive mandates, PARA-Agua is helping to secure external funding for investments in climate change adaptation. The KIIs confirmed that expectations are high for PARA-Agua to support these efforts. PARA-Agua is well positioned to assist in this regard and has hired consultants to assist with securing funding. If successful, these efforts would strengthen the position of the Watershed Councils, contributing to their sustainability. Feedback on the PARA-Agua project from a diverse set of stakeholders was highly positive. The KII feedback on the Project was positive overall and confirmed several of the desk review’s preliminary findings. In some cases several PARA-Agua activities unanimously were considered as providing ‘first of its kind’ contributions to watershed planning at the regional level, such as the incorporation of climate change adaptation variables in the WEAP tool. PARA-Agua has played an important role in establishing linkages between the national agencies and the regional counterparts. Support has allowed stakeholders at hydro-meteorological institutions and with watershed planning and management responsibilities to better understand the needs of the different parties involved and building a common language between the various stakeholders in the water climate change adaptation context. The PARA-Agua project is working with subcontractors who are highly regarded by partners and relevant stakeholders. Subcontractors NCAR, SEI and TMI are highly regarded by the scientific and water-planning experts’ community and support the PARA-Agua project. The quality and MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Final Evaluation Report 26 integrity of the work conducted by these subcontractors was noted by interviewees during the KIIs at various levels and has been a key factor in establishing credibility and high regard for project activities. The Project needs to work on closing the implementation gap between activities in Peru and Colombia. From the outset, a challenge for PARA-Agua has been closing the implementation gap of key subcontractors, SEI and NCAR, with regards to watershed planning and management in Peru and Colombia. By mid-2014 PARA-Agua work in the Chira Piura, Peru watershed was on schedule, but this was not the case of PARA-Agua activities in the Chinchiná, Colombia watershed [24]. Since then, the gap has been reduced, but a lag in progress exists. The PARA-Agua staff devoted exclusively to Colombian activities is small (one person supported by a part-time financial expert) and this may have an impact on certain project undertakings. MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Final Evaluation Report 27 6.0 RECOMMENDATIONS The PARA-Agua PE team recommends extending project implementation for the remaining period. The Project endured constraints at the beginning in terms of staffing, office space, etc. PARA-Agua has overcome these limitations and has reached a point where key results are being achieved with regard to building capacities and developing innovative tools, with positive feedback from numerous stakeholders. The PE team also expects that several of its central activities will begin delivering results, particularly gender mainstreaming, development of protocols to incorporate public and private organizations into national information systems for climate change adaptation in watersheds, the CoP, toolkits’ validation, and twinning partnerships between Peruvian and Colombian watersheds and between Peruvian watersheds and the U.S. Yolo County in the coming quarters. The Project’s support for fundraising for climate change adaptation projects and interventions, and Water Fund’s establishment is one of the activities for which PARA-Agua partners’ expectations are high. This activity should clearly be prioritized in the second implementation phase. In light of recommending an extension of PARA-Agua for the remaining period, the following set of priority recommendations are strategic to enhance the Project’s impact during the final years of implementation:  Supporting efforts to secure funding for adaptation investments: Securing funding to implement water management adaptations at the watershed level represents a significant challenge. Although not originally envisioned to be a major component of PARA-Agua, numerous interviewees indicated they would benefit greatly from continuing that relationship with PARA-Agua in assisting them in securing funding for climate change adaptation projects and interventions at the basin level. The recommendation is that PARA-Agua assist stakeholders at the watershed level in identifying and securing the funding necessary for implementing specific adaptation activities. This recommendation does not suggest or advocate PARA-Agua fund such activities directly. Where funding is secured, it should be accompanied by project implementation assistance in the early phases. In order to reach such an investment or implementation phase for climate change adaptation interventions during the second PARA-Agua Project phase, PARA-Agua staff needs to prioritize the delivery of this type of high-level technical advice to partners.  Strengthening relationships with national hydro-meteorological institutions: Reaching agreements on MoUs that define the scope of work and create synergies among PARA-Agua, IDEAM and SENAMHI, should be prioritized to ensure project activities are aligned with national counterparts. Support for the scientific twinning partnership proposed in the updated Work Plan and Management Plan for FY2015 between IDEAM, SENAMHI and NCAR [8] was unanimous during the KIIs and should be pursued. Agreeing on a defined scope of work with these institutions could contribute to, for instance, design pertinent and enforceable protocols to incorporate public and private organizations into national information systems for climate change adaptation in watersheds.  Incorporating social and economic priorities in the tools, instruments and overall activities supported by PARA-Agua: The Project is not exclusively working in the scientific arena, but is also strengthening the relevance of climate change adaptation data for water planning and management instruments at various levels. Given the diversity of relationships between natural resources and livelihoods across a MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Final Evaluation Report 28 wide range individuals, water planning and management instruments should pay more attention to social and economic considerations. Several consulted stakeholders consider socio-economic considerations like gender outside of their scope of work. This perception could represent a limitation for later adaptation and resilience strategies that respond to the interests and needs of men and women. In this respect, it is important that PARA-Agua undertakes an awareness effort with the research community. For example, using success stories that illustrate how the incorporation of the gender is important to achieving PARA-Agua goals such as proposed in the Gender Analysis and Plan of Action [12]. At the watershed level, PARA-Agua should explore enhancements to planning tools that better incorporate socio-economic factors. For example, taking into account a wider range of social and economic variables that are not currently supported by WEAP in addition to hydrological factors like quality and sub-surface water characteristics. Other recommendations to enhance PARA-Agua’s performance during the remaining implementation period include:  Further link stakeholders across the spectrum: PARA-Agua is in a unique position to foster linkages and communications between stakeholders at various levels (i.e., researchers and specialists at hydro￾meteorological centers, staff working at national agencies with watershed level planning responsibilities, members and technical staff at watershed management councils, and local authorities). In the roughly two years since its inception, PARA-Agua has been able to develop a unique understanding of the relevant institutions at the national and regional levels and has used this knowledge to promote dialogue between the various interests where previously there had been little communication. For example, dialog generated as a result of PARA-Agua has led to discussions between IDEAM and local stakeholders on establishing a Hydro-Climatological Forecast Center in the Chinchiná Basin. Opportunities arising as a result of the dialogue between national and regional counterparts should be seized upon and further pursued.  Ensure that legacy institutions continue to prioritize national level capacity building, and where possible entrust such legacy responsibilities to national and regional level stakeholders: PARA-Agua should ensure that the legacy institution alliance between CIIFEN, TMI and CONDESAN prioritizes the strengthening of national capacities, particularly those already installed in the regions and at the national level. Currently all legacy institutions are international institutes and alliances. However, in order to develop national capacity, whenever possible roles and responsibilities should come back to the main stakeholders (i.e. researchers at universities and specialists at hydro-meteorological centers; staff working at national agencies with watershed level planning responsibilities; members, technical staff and watershed management councils; and local authorities).  Support efforts to integrate research institutions broadly: This involves successfully and inclusively developing the toolkit aiming to develop protocols integrating public and private research organizations into national information systems. In both countries the environmental, climate change and/or water information systems are rapidly evolving, which provides an opportunity for PARA￾Agua to enhance its contributions towards strengthening these national processes while building linkages between other sub-national stakeholders and research organizations.  Implement gender mainstreaming and awareness activities: Implementation of the PARA-Agua Gender Action Plan and addressing gender in specific watershed Action Plans is critical. Other pertinent gender actions involve incorporating gender and climate change issues to the Water Culture Component of Water Management Plans and adjusting local level Water Users’ Boards regulations to promote women’s participation mechanisms. If the new Gender Consultant follows MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Final Evaluation Report 29 the Gender Analysis and Plan of Action, gender issues should be better incorporated into capacities to address climate change issues moving forward.  Maintain a dynamic and relevant virtual platform: A solid CoP team, with access to materials and forum discussion themes which are updated on a regular basis by PARA-Agua project staff and/or subcontractors is central to keeping interest in the virtual platform alive. The platform can contribute to establishing and strengthening links between researchers at hydro-meteorological centers, national agencies with watershed level planning responsibilities, and watershed management councils and authorities, if stakeholders better understand the objectives and intent of the platform.  Further tailor tools and activities to the watershed level: In consultation with stakeholders, undertake the adjustments required to further tailor workshops and capacity building activities to their needs. Even if the workshops’ feedback was highly positive, interviewees suggested areas of improvement including additional tailoring of workshops to participant profiles and enhancing the adaptation of methodologies and tools to their water planning priorities.  Monitor capacity building of Watershed Councils: Sub-national water management and planning structures frequently have new members, including new Watershed Councils members, regional directors of government offices, and technical staff. In order to make sure that new relevant members are brought up to speed with regards to PARA-Agua capacity building activities, the Project should monitor and suggest relevant training activities that could be undertaken by Watershed Councils Technical Secretariats. As stressed in interviews, the purpose is not to make Watershed Councils members experts on climate change modeling, but to help them to better understand the relevance and implications of the WEAP model, which would contribute to enhanced water governance and stakeholders’ engagement and participation at the sub-national level. MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Performance Evaluation Final Report ANNEX 1. LIST OF SOURCES [1] USAID, “Request for Task Order Proposal (RFTOP) for PARA-Agua Performance Evaluation - # SOL-OAA-15-000027.” USAID, Washington D.C., 2015. [2] USAID PARA-Agua COR, “Email and Phone Personal Communication - April 22.” 2015. [3] AECOM, “PARA-Agua) Project - Quarterly Report 5 (October to December 2014),” 2015. [4] USAID, “Climate Change and Development Strategy,” Washingrton, DC, 2012. [5] “President’s Obama Development Policy and the Global Climate Change Initiative,” Washington, D.C., 2012. [6] AECOM, “Partnering for Adaptation and Resilience—Agua (PARA-Agua) Project - Work Plan and Performance Management Plan - Fiscal Year 2015,” 2015. [7] AECOM, “Partnering for Adaptation and Resilience—Agua (PARA-Agua) Project - Work Plan and Performance Management Plan - FY 2014,” Washington, D.C., 2014. [8] AECOM, “PARA-Agua Project - Work Plan and Performance Management Plan - Updated - FY 2015,” 2015. [9] AECOM, “Partnering for Adaptation and Resilience—Agua (PARA-Agua) Project - Work Plan and Performance Management Plan - Fiscal Year 2014,” 2014. [10] The Mountain Institute (TMI), “PARA-Agua Project - Task Order 02 - Deliverable 05 - Gap Analysis,” 2014. [11] The Mountain Institute (TMI), “PARA-Agua Project - Task 02 - Consultation Workshop Report,” 2014. [12] M. del P. R. Queneche, “PARA-Agua Project - Análisis de Género,” 2015. [13] “Memorando de Entendimiento para la Alianza Estratégica entre el Consejo de Recursos Hídricos de Cuenca Chira Piura y el Consejo de Cuenca de Chinchiná.pdf.” 2014. [14] “Letter of Intent to Establish Twinning Arrangements between Yolo County Flood Control and the National Water Authority (Peru).” 2015. [15] The Mountain Institute (TMI), “PARA-Agua Task Order 02 - Deliverable 7 - Consultation on Virtual Community of Practice Report,” 2014. [16] “Acuerdo de Entendimiento entre la Autoridad Nacional del Agua de Perú y el Proyecto para la Adaptación y Resilidencia (PARA Agua).” Lima, 2014. MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Performance Evaluation Final Report [17] “Memorando de Entendimiento entre la Corporación Autónoma Regional de Caldas (CORPOCALDAS) y Proyecto para la Adaptación y la Resiliencia Agua (PARA-Agua).” . [18] “PARA-Agua Project -Project Inception and Consultation Workshop - Summary.” Lima, 2013. [19] The Stockholm Environmental Institute (SEI), “PARA-Agua Project - Task Order 02 - Deliverable 3 - Peru - Chira Piura - Workshop Agenda and XLRM Problem Formulation Report,” 2014. [20] The Stockholm Environmental Institute (SEI), “PARA-Agua Project - Task Order 03 - Deliverable 6.a - Colombia - Chinchiná Basin - Training Report,” 2014. [21] AECOM, “Plan de Género Especifico - Perú.” 2015. [22] D. Yates and H. Angarita, “Partnering for Adaptation and Resilience - Agua (PARA-Agua Project) Developing Climate Scenarios,” 2014. [23] The Mountain Institute (TMI), “PARA-Agua - Task Order 02 - Deliverable 10 - Toolkit: Climate Information for Decision Making,” 2015. [24] AECOM, “PARA-Agua Project - Quarterly Report 3 (April to June 2014),” 2014. MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Performance Evaluation Final Report ANNEX 2. STAKEHOLDERS INTERVIEWED Name Gender Institution, Organization or Constituency Interview Technique Interview Country Interview Date 1 Cromwell Alba Male Director, Water Modernization Program, National Water Authority Face to face Peru 05-11-2015 2 Luis Chinchay Male Water Modernization Program, National Water Authority Face to face Peru 05-11-2015 3 Ezequiel Villegas Male Scientific Director, SENAMHI Face to face Peru 05-11-2015 4 Gabriela Rojas Female Applied Meteorology Director, SENAMHI Face to face Peru 05-11-2015 5 Wilson Suárez Male Hydrology Specialist, SENAMHI Face to face Peru 05-11-2015 6 Clara Oria Female Meteorology Specialist, SENAMHI Face to face Peru 05-11-2015 7 Grinia Avalos Female Climatology Specialist, SENAMHI Face to face Peru 05-11-2015 8 Ronald Díaz Male President, Chira Piura Watershed Council Face to face Peru 05-12-2015 9 Fausto Asencio Díaz Male Technical Secretariat Staff, Chira Piura Watershed Council Face to face Peru 05-12-2015 10 Nilton Buguña Male Technical Secretariat Staff, Chira Piura Watershed Council Face to face Peru 05-12-2015 11 Lorena Lisboa Female Technical Secretariat Staff, Chira Piura Watershed Council Face to face Peru 05-12-2015 12 Yvonne Sotela Female SEI - Peru Face to face Peru 05-12-2015 13 Sergio Claure Male Director, PARA-Agua Project Face to face Peru 05-12-2015 / 05-15- 2015 14 José Narváez Male Operations Director, PARA-Agua Face to face Peru 05-12-2015 / 05-15- 2015 15 Cristina Portocarrero Female Peru Country Director, PARA-Agua Face to face Peru 05-12-2015 / 05-15- 2015 16 Mirella Gallardo Female Task Team Leader, TMI Face to face Peru 05-12-2015 / 05-15- 2015 17 Jorge Recharte Male Task Team Leader, TMI Face to face Peru 05-12-2015 / 05-15- 2015 18 Mariela Rodríguez Female Gender Specialist, PARA-Agua Face to face Peru 05-12-2015 / 05-14- 2015 19 Marco Aspilcueta Male Monitoring and Evaluation Specialist, PARA-Agua Face to face Peru 05-12-2015 / 05-15- 2015 20 David Yates Male Scientist, NCAR Face to face Peru 05-12-2015 MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Performance Evaluation Final Report Name Gender Institution, Organization or Constituency Interview Technique Interview Country Interview Date 21 Zacarías Maradiaga Male President, Quilca Chili Watershed Council Face to face Peru 05-14-2015 22 Carmen Málaga Female Representative of Water Irrigation Boards, Quilca Chili Watershed Council Member Face to face Peru 05-14-2015 23 Sócrates Choque Male Representative of Communities, Quilca Chili Watershed Council Member Face to face Peru 05-14-2015 24 Giovanni Salas Male Representative of Professional Bodies, Quilca Chili Watershed Council Member Face to face Peru 05-14-2015 25 Mario Salas Male Representative of Universities, Quilca Chili Watershed Council Member Face to face Peru 05-14-2015 26 Ronáld Fernández Male ANA - Quilca Chili Watershed Council Face to face Peru 05-14-2015 27 Roland Valencia Male Technical Secretariat Staff, Quilca Chili Watershed Council Face to face Peru 05-14-2015 28 Javier Segovia Male Technical Secretariat Staff, Quilca Chili Watershed Council Face to face Peru 05-14-2015 29 Lourdes Escobar Female Technical Secretariat Staff, Quilca Chili Watershed Council Face to face Peru 05-14-2015 30 Erick Mercado Male Technical Secretariat Staff, Quilca Chili Watershed Council Face to face Peru 05-14-2015 31 Jaime Gutiérrez Male Technical Secretariat Staff, Quilca Chili Watershed Council Face to face Peru 05-14-2015 32 Duberly Limacle Male Technical Secretariat Staff, Quilca Chili Watershed Council Face to face Peru 05-14-2015 33 Emily Waytoti Female USAID-Peru Face to face Peru 05-15-2015 34 Marisa Escobar Female Task Team Leader, SEI Skype Colombia 05-18-2015 35 David Purkey Male SEI Skype Colombia 05-18-2015 36 Juliana Valencia Female Colombia Country Director, PARA￾Agua Face to face Colombia 05-18-2015 37 Olga Jeannette Galindo Female Technical Secretariat, Pactos por la Cuenca Face to face Colombia 05-20-2015 38 Andrés Felipe Betancourt Male Technical Secretariat, Pactos por la Cuenca Face to face Colombia 05-20-2015 39 Carlos Arturo Franco Male President and representative of CHEC, Chinchiná Watershed Council Face to face Colombia 05-20-2015 40 Constanza Mejía Female Technical Secretary, Chinchiná Watershed Council Face to face Colombia 05-20-2015 41 Raúl Jiménez Director General, CORPOCALDAS Face to face Colombia 05-21-2015 42 Carlos Julio Castillo Male Sustainable Production and Climate Change Professional, CORPOCALDAS Face to face Colombia 05-21-2015 43 Patricia García Female Land Planning Professional, CORPOCALDAS Face to face Colombia 05-21-2015 44 Juan Carlos Bastidas Male Water Resources, CORPOCALDAS Face to face Colombia 05-21-2015 45 Nilo Lima Male SEI – Colombia Face to face Colombia 05-21-2015 46 Carlos Ayape Male PARA-Agua – Financial Specialist for Colombia Telephone Colombia 05-21-2015 MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Performance Evaluation Final Report Name Gender Institution, Organization or Constituency Interview Technique Interview Country Interview Date 47 Carolina Figueroa Female USAID - Colombia Face to face Colombia 05-22-2015 48 Guillermo Armenta Male Modeling Lead, IDEAM Face to face Colombia 05-22-2015 49 Franklin Ruiz Male Climate Scenarios, IDEAM Face to face Colombia 05-22-2015 50 Yolanda Calderón Female Risk Management, Ministry of the Environment Face to face Colombia 05-22-2015 51 Kaleth Villalobos Male Director, CORPOCESAR Skype Colombia 05-22-2015 MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Performance Evaluation Final Report ANNEX 3. KII INTERVIEW MATERIALS INFORMED CONSENT PARA-Agua forms part of the United States Agency for International Development’s (USAID) work to support adaptation to climate change impacts in the Latin American and the Caribbean (LAC) Region. This PE will assess PARA-Agua’s performance and effectiveness beyond that captured in the activity implementer’s Performance and Management Plan. The overall purpose of the PARA-Agua Performance Evaluation (PARA-Agua PE) is to document the extent to which PARA-Agua has is progressing in achieving its goals and objectives to date. The focus of this PE is the work being conducted by PARA￾Agua with water managers and policy makers in the Latin America region, with an emphasis on Colombia and Peru. To gather additional information necessary to conduct this Performance Evaluation and complete the report, the evaluation team would like to gather additional information from people such as yourself. Any information provided during this interview will be treated as confidential by the evaluation team. Your individual responses will never be discussed with anyone outside the evaluation team. Only the evaluation team will have access to your individual interview records. In reporting on the findings of this evaluation, your responses will combined with those from other respondents to protect your identity. In some instances the evaluation team may be joined by a USAID representative. In these cases the evaluation team will arrange to contact you following the interview in order to give you an opportunity to further discuss any questions or issues raised during the meeting which you would like to share separately with the evaluation team. All responses – both those provided during the meeting as well as part of the follow-up discussion - will be considered as confidential by the evaluation team and treated as such. If you have questions relating to this interview, you may contact the Performance Evaluation Lead, Dr. Daniel Monchuk. He can be reached by phone at +001.240.582.3624 or email at Daniel.monchuk@cloudburstgroup.com. Do you wish to participate in this interview? ______ Yes _______ No Participant Signature: _____________________________ Date: _________ MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Performance Evaluation Plan and Timeline 36 GROUP 1: KII QUESTIONNAIRE FORM  Specialists of hydro-meteorological centers  Representatives of water or environmental government agencies holding policy-making responsibilities  Members of watershed councils  Researchers at universities or international water/climate change adaptation programs Impact on Capacities of National Hydro-Meteorological Centers to Generate Relevant Climate Adaptation Data Questions Spanish Version Questions English version Comments / Remarks 1 ¿Considera que las actividades de PARA￾Agua han tenido un impacto positivo en las capacidades de los centros hidrológicos (IDEAM, SENAMHI) para generar datos sobre adaptación al cambio climático que sean pertinentes para los organismos e instituciones de planificación de los recursos hídricos? Si así fuera, ¿podría brindar un ejemplo del tipo de impacto positivo que ha percibido? O ¿Qué canales utiliza habitualmente para tener acceso a información y datos de adaptación al cambio climático? ¿Cuáles son apoyados por PARA-Agua de forma directa o indirecta? Do you consider PARA-Agua activities have had a positive impact on the capacities of hydrological centers (IDEAM, SENAMHI) to generate watershed climate adaptation relevant data for water planning agencies? If so, can you give me an example of what positive impact that you have seen? OR What channels do you routinely use for accessing information and data? Which are supported PARA-Agua (directly or indirectly)? Questions will be adjusted to the context of interviewees, i.e. responsible of mainstreaming climate change data, or responsible of generating water planning instruments incorporating climate change adaptation actions. Relevant to underscore tools and methodologies supported by PARA-Agua, for instance, National Center of Atmospheric Research of the United States (NCAR) regional climate change data. 2 ¿Han adoptado los talleres un lenguaje didáctico para transmitir la información científica? Si no, ¿cómo podrían mejorarse las metodologías de los talleres para incrementar la comprensión de la información científica? ¿Considera que los talleres se han concentrado en Have the workshops adopted a didactic language to convey scientific information? If not, how could workshop methodologies be enhanced to improve understanding of scientific information? Many of the interviewees have attended different capacity building activities. MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Performance Evaluation Plan and Timeline 37 cuestiones de política pública pertinentes para los actores nacionales concernidos? Si así fuera, ¿qué preocupaciones de política pública se han abordado? Have workshops focused on public policy concerns of relevant national stakeholders? If so, what public policy concerns have been addressed? 3 ¿Está familiarizado con la primera serie de herramientas (toolkit) desarrollada por PARA-Agua y que se compartirá en la Comunidad de Práctica (CdP)? Si así fuera, ¿considera que este conjunto de herramientas abordará las necesidades de los planificadores de recursos hídricos en términos de los datos necesarios para desarrollar o ajustar las políticas correspondientes? O Si así fuera, ¿considera que este conjunto de herramientas mejorará sus capacidades para generar datos importantes de adaptación al cambio climático? Are you familiar with the first toolkit developed by PARA-Agua and that will be shared in the forthcoming Community of Practice (CoP)? If so, do you believe this toolkit will address the needs of water planning managers in terms of the data required to develop or adjust relevant policy? OR If so, do you believe this toolkit will enhance your capacities to mainstream relevant climate change adaptation data? 4 ¿Considera que las actividades de fortalecimiento de capacidades conllevaron a la priorización exitosa de los escenarios climáticos utilizados con WEAP? ¿Fue útil esta información para la toma de decisiones y proceso de diseño de políticas? Si así fuera, ¿cómo fortalece esta información la adopción de decisiones y proceso de diseño de políticas? O Si así fuera, ¿cómo esta serie de herramientas mejora sus capacidades para integrar los datos pertinentes de adaptación al cambio climático? Do you consider capacity building activities led to the successful establishment of the climate scenarios prioritization used with WEAP? Was this information useful for your policy decision-making and design process? If so, how does this information strengthen the policy decision-making and design process? OR If so, how will this toolkit enhance your capacities to mainstream relevant climate change adaptation data? 5 ¿Considera que el proyecto PARA-Agua ha llevado a cabo un esfuerzo sostenido en la construcción de Do you consider the PAR A-Agua Project has engaged in a sustained effort into building MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Performance Evaluation Plan and Timeline 38 comunicaciones durables entre centros hidrológicos y agencias gubernamentales de planificación de agua? Si es así, ¿qué tipos de mecanismos / herramientas / actividades se han utilizado en la construcción de dichas comunicaciones? De los anteriores, ¿cuál ha sido el más eficaz? sustainable communications between hydrological centers and water planning government agencies? If so, what types of mechanisms/tools/activities have been used into building sustainable communications? Of the above which has been the most effective? 7 ¿Qué procesos de intercambio de conocimientos e investigación podrían ser mejor implementado y cómo? O ¿Qué se puede hacer para aumentar el interés gestores de políticas de agua en la incorporación de nuevos datos generados por los centros hidro￾meteorológicos? What research and knowledge sharing processes could be better implemented and how? OR What can be done to enhance water policy managers’ interest on incorporating new data generated by hydro-meteorological centers? Strengthening of watershed systems Questions Spanish Version Questions English version Comments / Remarks 8 Después de participar en una o varias de las actividades de fortalecimiento de capacidades de PARA-Agua, ¿tiene usted una mejor comprensión del desarrollo de protocolos y estándares para integrar información sobre adaptación al cambio climático en las cuencas? En caso afirmativo, por favor señale un momento en el que usted u otra persona que haya participado en las actividades haya utilizado estos protocolos. After one or several of the PARA-Agua capacity building activities in which you have participated, do you have an improved understanding of the development of protocols and standards to integrate information on climate change adaptation in watersheds? If yes, please describe a time when you or someone else who has participated used these Particularly relevant for interviewees with water planning responsibilities It is important to underscore how many activities the interviewee has attended. MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Performance Evaluation Plan and Timeline 39 protocols. 9 ¿Cree usted que el proyecto PAR A-Agua ha ajustado sus actividades a los instrumentos de planificación de cuencas existentes? Si no fuera así, ¿podría brindar un ejemplo de una actividad que podría ser mejor adaptada? Do you think that the PARA-Agua Project has adjusted its activities to existing watershed planning instruments? If no, can you give me an example of an activity that could be better adjusted? 10 ¿Cuál de las herramientas y / o metodologías propuestas por el proyecto PARA Agua considera que son más importantes para mejorar la formulación de proyectos de inversión pública para la adaptación al cambio climático? Which of the tools and/or methodologies proposed by the PARA-Agua Project you consider are more relevant for enhancing the formulation of public investment projects for climate change adaptation? 11 ¿Considera que las herramientas y metodologías propuestas por el proyecto PARA-Agua permitirán el monitoreo de los efectos de las medidas de adaptación a nivel de cuenca, no sólo a nivel de indicadores ambientales, sino también de indicadores socioeconómicos incluyendo las cuestiones de género? Do you consider the tools and methodologies proposed by the PARA-Agua Project allow monitoring the effects of adaptation measures at the basin level not only on environmental indicators, but also on socio-economic indicators, including gender concerns? 12 El proyecto PARA-Agua ha hecho un esfuerzo consistente por invitar a mujeres con responsabilidades relevantes a las actividades de consulta y fortalecimiento de capacidades. Por otro lado, también es importante abordar la limitada consideración de cuestiones socio-culturales en el contexto de los recursos hídricos y la gestión del cambio climático, como se destaca en el Análisis de Género de PARA-Agua (enero de 2015). Al respecto, ¿ha habido actividades de sensibilización sobre cuestiones socio￾culturales o el impacto en las dinámicas de población y las relaciones de género en los nuevos contextos de The PAR A-Agua Project has made a consistent effort to target and invite women with relevant responsibilities to the consultation and capacity building activities. On the other hand, it is also important to address the limited consideration of socio￾cultural issues in the context of water resources and climate change management, as underscored in PARA-Agua’s Gender Analysis (January 2015). Have there been awareness raising activities on socio cultural issues or the impact on population dynamics and gender relations in the new MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Performance Evaluation Plan and Timeline 40 adaptación al cambio climático? Más allá de las actividades iniciales, ¿ha habido actividades sobre cómo aumentar la participación de las mujeres en las actividades de generación de datos o de planificación del agua? Si así fuera, ¿puede dar un ejemplo, si alguna de estas discusiones se hubieran ya aplicado? contexts of adaptation to climate change? Beyond these initial activities, have there been activities on how to increase women’s participation in targeted data generation or water planning activities? If yes, can you give me an example if any of these discussions have been implemented? 13 ¿Participó en las consultas asociadas al diseño e implementación de la Comunidad de Práctica (CdP)? • Si es así, ¿considera que las consultas resultaron en un marco para la CdP que proporcionará un espacio adecuado para la mejora de las capacidades, el intercambio de información y buenas prácticas? • A pesar de la CdP es un trabajo en progreso, ¿la ha utilizado ya? ¿Ha recurrido a la información, recursos en línea, enlaces y contactos proporcionados por la CdP? • ¿Tiene usted o tuvo ya la oportunidad de intercambiar experiencias de planificación de cuencas con otros participantes en la CdP? • Tal y como está diseñada, ¿cuáles son los retos clave para el funcionamiento eficaz de la CdP? Did you participate in the consultations leading to the Community of Practice (CoP) design and implementation framework? If so, do you consider the consultations resulted in a CoP framework that will provide an appropriate space for enhancing capacities, sharing information and best practices? Though the CoP is a work in progress, have you used it already? Have you had recourse to the information, on-line resources, links and contacts provided by the CoP? Have you already had the opportunity to exchange watershed planning experiences with other participants in the CoP? As designed, what are the key challenges for the CoP effective functioning? 14 ¿Cree usted que hay otros actores clave que también se beneficiarían del proyecto PARA-Agua en su cuenca y que actualmente no participan? Por favor explique. Do you believe there are other key stakeholders who would also benefit of the PARA-Agua Project in your basin and who currently do not participate? Please explain. 15 ¿Cómo se podrían mejorar los enfoques y estrategias de participación para asegurar que los datos generados sean pertinentes para los actores interesados? How could participatory approaches and strategies be enhanced to ensure data generated is relevant for targeted stakeholders? MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Performance Evaluation Plan and Timeline 41 Building or strengthening sustainable links between hydro-meteorological centers and/or national authorities at various levels with water planning responsibilities Questions Spanish version Questions English version Comments / Remarks 16 ¿Cuánto éxito ha tenido el proyecto PARA-Agua en la promoción de alianzas estratégicas a nivel local, nacional y regional para el intercambio de información, metodologías, entre otros? ¿Considera que la CdP tiene el potencial de fortalecer esta actividad? ¿Por qué? How successful has the PAR A-Agua Project been in promoting strategic partnerships at the local, national and regional levels to exchange information, methodologies, among others? Do you consider the CoP has the potential of strengthening this activity? Why? 17 ¿Podría indicar dos actividades de planificación del agua que se han fortalecido como resultado de la Alianza Estratégica con el Distrito de Control de Inundaciones y de Conservación del Agua del Condado de Yolo (California, EE.UU.) y el Consejo de Cuenca de Chira Piura? Would you please indicate two water planning activities that have been strengthened as a result of the twinning partnership with the Yolo County (California, U.S.A) Flood Control and Water Conservation District and the Chira Piura Watershed Council? Particularly relevant for members of the Chira Piura Watershed Council 18 ¿Podría indicar dos actividades de planificación del agua que se han fortalecido como resultado de la Alianza Estratégica entre el Consejo de Chira Piura (Perú) y el Consejo de Recursos Hídricos de Chinchiná (Colombia)? Would you please indicate two water planning activities that have been strengthened as a result of the twinning partnership between the Council of Chira Piura (Peru) and the Water Resources Council of Chinchiná (Colombia)? Particularly relevant for members of the Council of Chira Piura (Peru) and members of the Water Resources Council of Chinchiná (Colombia) 19 ¿Están las cuestiones socio-culturales, incluyendo el impacto de la adaptación al cambio climático en las relaciones de género en los contextos de cuencas, siendo consideradas en las actividades previstas en Are socio cultural issues, including the impact of climate change adaptation on gender relations in the watershed contexts, being considered in the activities under the twinning agreements? If so, how? Please provide some MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Performance Evaluation Plan and Timeline 42 los acuerdos de Alianzas Estratégicas? Si es así, ¿cómo? Por favor, proporcione algunos ejemplos. examples. MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Performance Evaluation Plan and Timeline 43 GROUP 2: KII QUESTIONNAIRE FORM  Local watershed representatives with participation or interest in watershed management  Women’s groups  Private sector representatives Impact on Capacities of National Hydro-Meteorological Centers to Generate Relevant Climate Adaptation Data Questions Spanish version Questions English version Comments / Remarks 1 ¿Está familiarizado con los datos y proyecciones de cambio climático a nivel regional del Centro Nacional de Investigación Atmosférica de Estados Unidos (NCAR), incluyendo en particular el módulo de Evaluación y Planificación del Agua (WEAP) que NCAR desarrolló? Are you familiar with the National Center of Atmospheric Research of the United States (NCAR) regional climate change data and projections, including notably the Water Evaluation and Planning (WEAP) module that NCAR developed? 2 ¿Ha asistido a las actividades de fortalecimiento de capacidades de PARA-Agua? Por favor, indique cuáles y cuántas veces. Si es así, ¿han adoptado los talleres un lenguaje didáctico para transmitir la información científica? Si no, ¿cómo podrían mejorarse las metodologías de talleres para mejorar la comprensión de la información científica? ¿Han abordado los talleres las cuestiones pertinentes a nivel de cuenca o a nivel local? Si es así, ¿qué preocupaciones de política pública se han abordado? Have you attended PARA-Agua capacity building activities? Please indicate which and how many times. If so, have the workshops adopted a didactic language to convey scientific information? If not, how could workshop methodologies be enhanced to improve understanding of scientific information? Have workshops focused on concerns relevant at the watershed or local level? If so, what public policy concerns have been addressed? Many of the interviewees have attended different capacity building activities. 3 ¿Está familiarizado con la primera serie de herramientas Are you familiar with the first toolkit MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Performance Evaluation Plan and Timeline 44 (toolkit) desarrollada por PAR A-Agua y que se compartirá en la Comunidad de Práctica (CdP)? Si así fuera, ¿considera que este conjunto de herramientas abordará las necesidades de los planificadores de recursos hídricos en términos de los datos necesarios para desarrollar o ajustar las políticas correspondientes? developed by PAR A-Agua and that will be shared in the Community of Practice (CoP)? Do you believe this toolkit will address the needs of water planning managers in terms of the data required to develop or adjust relevant policy? 4 ¿Cree usted que las actividades de fortalecimiento de capacidades han tenido éxito en el cumplimiento de las expectativas de aprendizaje relacionadas con el instrumento Evaluación y Planificación del Agua (WEAP)? Por favor explique. Do you believe capacity building activities have been successful in accomplishing learning expectations related to the Water Evaluation and Planning (WEAP) model? Please explain. 5 ¿Qué procesos de intercambio de conocimientos e investigación podría ser mejor implementados y cómo? What research and knowledge sharing processes could be better implemented and how? Strengthening of watershed systems Questions English version Comments / Remarks 7 Después de participar en una o varias de las actividades de fortalecimiento de capacidades de PARA-Agua, ¿tiene usted una mejor comprensión del desarrollo de protocolos y estándares para integrar información sobre adaptación al cambio climático en las cuencas? En caso afirmativo, por favor señale un momento en el que usted u otra persona que haya participado en After one or several of the PARA-Agua capacity building activities in which you have participated, do you have an improved understanding of the development of protocols and standards to integrate information on climate change adaptation in watersheds? If yes, please describe a time when you or someone else who has participated used these protocols. Particularly relevant for interviewees with water planning responsibilities It is important to underscore how many activities the interviewee has attended. MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Performance Evaluation Plan and Timeline 45 las actividades haya utilizado estos protocolos. 8 ¿Ha incorporado o discutido el Proyecto PAR A-Agua temas de creación de conciencia sobre las cuestiones de género y el cambio climático con los miembros del consejo? Si es así, ¿qué cuestiones relacionadas con el género se han incorporado? Si es así, ¿en qué contexto se ha discutido el cambio climático? Has the PAR A-Agua Project incorporated or discussed council members’ awareness of gender and climate change issues? If so, what gender related issues have been incorporated? If so, in what context has climate change been discussed? 9 El proyecto PARA-Agua ha hecho un esfuerzo consistente por invitar a mujeres con responsabilidades relevantes a las actividades de consulta y fortalecimiento de capacidades. Por otro lado, también es importante abordar la limitada consideración de cuestiones socio-culturales en el contexto de los recursos hídricos y la gestión del cambio climático, como se destaca en el Análisis de Género de PARA-Agua (enero de 2015). Al respecto, ¿ha habido actividades de sensibilización sobre cuestiones socio-culturales o el impacto en las dinámicas de población y las relaciones de género en los nuevos contextos de adaptación al cambio climático? Más allá de las actividades iniciales, ¿ha habido actividades sobre cómo aumentar la participación de las mujeres en las actividades de generación de datos o de planificación del agua? Si así fuera, ¿puede dar un ejemplo, si alguna de estas discusiones se hubieran ya aplicado? The PAR A-Agua Project has made a consistent effort to target and invite women with relevant responsibilities to the consultation and capacity building activities. On the other hand, it is also important to address the limited consideration of socio-cultural issues in the context of water resources and climate change management, as underscored in PARA-Agua’s Gender Analysis (January 2015). Have there been awareness raising activities on socio cultural issues or the impact on population dynamics and gender relations in the new contexts of adaptation to climate change? Beyond these initial activities, have there been activities on how to increase women’s participation in targeted data generation or water planning activities? If yes, can you give me an example if any of these discussions have been implemented? 10 ¿Participó en las consultas asociadas al diseño e implementación de la Comunidad de Práctica (CdP)? • Si es así, ¿considera que las consultas resultaron en un marco para la CdP que proporcionará un espacio Did you participate in the consultations leading to the Community of Practice (CoP) design and implementation framework? If so, do you consider the consultations resulted in MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Performance Evaluation Plan and Timeline 46 adecuado para la mejora de las capacidades, el intercambio de información y buenas prácticas? • A pesar de la CdP es un trabajo en progreso, ¿la ha utilizado ya? ¿Ha recurrido a la información, recursos en línea, enlaces y contactos proporcionados por la CdP? • ¿Tiene usted o tuvo ya la oportunidad de intercambiar experiencias de planificación de cuencas con otros participantes en la CdP? • Tal y como está diseñada, ¿cuáles son los retos clave para el funcionamiento eficaz de la CdP? a CoP framework that will provide an appropriate space for enhancing capacities, sharing information and best practices? Though the CoP is a work in progress, have you used it already? Have you had recourse to the information, on-line resources, links and contacts provided by the CoP? Have you already had the opportunity to exchange watershed planning experiences with other participants in the CoP? As designed, what are the key challenges for the CoP effective functioning? 11 ¿Cree usted que hay otros actores clave que también se beneficiarían del proyecto PARA-Agua en su cuenca y que actualmente no participan? Por favor explique. Do you believe there are other key stakeholders who would also benefit of the PARA-Agua Project in your basin and who currently do not participate? Please explain. 12 ¿Cómo se podrían mejorar los enfoques y estrategias de participación para asegurar que los datos generados sean pertinentes para los actores interesados? How could participatory approaches and strategies be enhanced to ensure data generated is relevant for targeted stakeholders? Building or strengthening sustainable links between hydro-meteorological centers and/or national authorities at various levels with water planning responsibilities Questions Spanish version Questions English version Comments / Remarks 13 ¿Sabe usted si las cuestiones socio-culturales, incluyendo el impacto de la adaptación al cambio Do you know whether socio cultural issues, including the impact of climate change adaptation on gender relations MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Performance Evaluation Plan and Timeline 47 climático en las relaciones de género en los contextos de cuencas, están siendo consideradas en las actividades previstas en los acuerdos de Alianzas Estratégicas? Si es así, ¿cómo? Por favor, proporcione algunos ejemplos. in the watershed contexts, being considered in the activities under the twinning agreements? If so, how? Please provide some examples. MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Performance Evaluation Plan and Timeline 48 GROUP 3: KII QUESTIONNAIRE FORM  PARA-Agua project staff; and  PARA-Agua contractors (i.e. SEI, TMI, NCAR). Impact on Capacities of National Hydro-Meteorological Centers to Generate Relevant Climate Adaptation Data Questions English version Comments / Remarks 1 ¿Considera que las actividades de PARA￾Agua han tenido un impacto positivo en las capacidades de los centros hidrológicos (IDEAM, SENAMHI) para generar datos sobre adaptación al cambio climático que sean pertinentes para los organismos e instituciones de planificación de los recursos hídricos? Si así fuera, ¿podría brindar un ejemplo del tipo de impacto positivo que ha percibido? Do you consider PARA-Agua activities have had a positive impact on the capacities of hydrological centers (IDEAM, SENAMHI) to generate watershed climate adaptation relevant data for water planning agencies? If so, can you give me an example of what positive impact that you have seen? 2 ¿Me puede dar un ejemplo de una actividad PARA￾Agua que ha fomentado la interacción entre especialistas en los centros hidrológicos (IDEAM, SENAMHI) y tomadores y / o ejecutores de decisiones de política pública? Can you give me an example of a PARA-Agua activit y that has fostered interactions between specialists at hydrological centers (IDEAM, SENAMHI) and public policy decision-makers and/or implementers? 4 Han adoptado los talleres un lenguaje didáctico para transmitir la información científica? Si no, ¿cómo podrían mejorarse las metodologías de los talleres para incrementar la comprensión de la información científica? ¿Considera que los talleres se han concentrado en cuestiones de política pública pertinentes para los actores nacionales concernidos? Have the workshops adopted a didactic language to convey scientific information? If not, how could workshop methodologies be enhanced to improve understanding of scientific information? Have workshops focused on public policy concerns of relevant national stakeholders? If so, what public policy concerns have been Many of the interviewees have attended different capacity building activities. MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Performance Evaluation Plan and Timeline 49 Si así fuera, ¿qué preocupaciones de política pública se han abordado? addressed? ¿Considera que la serie de herramientas (toolkit) que se compartirá en la Comunidad de Práctica (CdP) las necesidades de los planificadores de recursos hídricos en términos de los datos necesarios para desarrollar o ajustar las políticas correspondientes? Por favor explique. Do you believe the toolkit that will be shared in the Community of Practice (CoP) will address the needs of water planning managers in terms of the data required to develop or adjust relevant policy? Please explain. ¿Considera que las actividades de fortalecimiento de capacidades han sido exitosas al cumplir las expectativas de aprendizaje asociados con el instrumento de Evaluación y Planificación del Agua (WEAP)? Por favor explique. ¿Considera que las actividades de fortalecimiento de capacidades conllevaron a la priorización exitosa de los escenarios climáticos utilizados con WEAP? ¿Fue útil esta información para la toma de decisiones y proceso de diseño de políticas? Si así fuera, ¿cómo fortalece esta información la adopción de decisiones y proceso de diseño de políticas? Do you believe capacity building activities have been successful in accomplishing learning expectations related to the Water Evaluation and Planning (WEAP) model? Please explain. Do you consider capacity building activities led to the successful establishment of the climate scenarios prioritization used with WEAP? Was this information useful for policy decision-making and design process? If so, how does this information strengthen the policy decision-making and design process? 5 ¿Considera que el proyecto PARA-Agua ha llevado a cabo un esfuerzo sostenido en la construcción de comunicaciones durables entre centros hidrológicos y agencias gubernamentales de planificación de agua? Si es así, ¿qué tipos de mecanismos / herramientas / actividades se han utilizado en la construcción de dichas comunicaciones? De los anteriores, ¿cuál ha sido el más eficaz? Do you consider the PAR A-Agua Project has engaged in a sustained effort into building sustainable communications between hydrological centers and water planning government agencies? If so, what types of mechanisms/tools/activities have been used into building sustainable communications? If the above which has been the most effective? 6 ¿Qué procesos de intercambio de conocimientos e investigación podrían ser mejor implementado y cómo? ¿Qué se puede hacer para aumentar el interés What research and knowledge sharing processes could be better implemented and how? What can be done to enhance water policy managers’ interest on incorporating new data generated by MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Performance Evaluation Plan and Timeline 50 gestores de políticas de agua en la incorporación de nuevos datos generados por los centros hidrometeorológicos? hydro-meteorological centers? 7 ¿Qué se puede hacer para superar los retos operativos que han afectando el rendimiento del proyecto? ¿Son las definiciones y los términos de los PIRS apropiados para los indicadores existentes? ¿Son estas demasiado amplio o restringidos? ¿Cómo podrían ser mejorados? What can be done to overcome the operational challenges that have been impacting project’s performance? Are the definitions and terms in the PIRS appropriate for the given indicator? Are these too broad/narrow? How might these be improved upon? Strengthening of watershed systems Questions Spanish version Questions English version Comments / Remarks 8 ¿Cree usted que el proyecto PAR A-Agua ha ajustado sus actividades a los instrumentos de planificación de cuencas existentes? Si no, ¿me puede dar un ejemplo de una actividad que podría ser mejor ajustada? Do you think that the PARA-Agua Project has adjusted its activities to existing watershed planning instruments? If no, can you give me an example of an activity that could be better adjusted? 9 ¿Cuál de las herramientas y / o metodologías propuestas por el proyecto PARA Agua considera que son más importantes para mejorar la formulación de proyectos de inversión pública para la adaptación al cambio climático? Which of the tools and/or methodologies proposed by the PARA-Agua Project you consider are more relevant for enhancing the formulation of public investment projects for climate change adaptation? 10 ¿Considera que las herramientas y metodologías propuestas por el proyecto PARA-Agua permitirán el monitoreo de los efectos de las medidas de adaptación a nivel de cuenca, no sólo a nivel de indicadores ambientales, sino también de indicadores Do you consider the tools and methodologies proposed by the PARA-Agua Project allow monitoring the effects of adaptation measures at the basin level not only on environmental indicators, but also on socio-economic indicators, including gender MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Performance Evaluation Plan and Timeline 51 socioeconómicos incluyendo las cuestiones de género? concerns? 11 ¿Ha incorporado o discutido el Proyecto PAR A-Agua temas de creación de conciencia sobre las cuestiones de género y el cambio climático con los miembros del consejo? Si es así, ¿qué cuestiones relacionadas con el género se han incorporado? Si es así, ¿en qué contexto se ha discutido el cambio climático? Has the PAR A-Agua Project incorporated or discussed council members’ awareness of gender and climate change issues? If so, what gender related issues have been incorporated? If so, in what context has climate change been discussed? 12 El proyecto PARA-Agua ha hecho un esfuerzo consistente por invitar a mujeres con responsabilidades relevantes a las actividades de consulta y fortalecimiento de capacidades. Por otro lado, también es importante abordar la limitada consideración de cuestiones socio-culturales en el contexto de los recursos hídricos y la gestión del cambio climático, como se destaca en el Análisis de Género de PARA-Agua (enero de 2015). Al respecto, ¿ha habido actividades de sensibilización sobre cuestiones socio-culturales o el impacto en las dinámicas de población y las relaciones de género en los nuevos contextos de adaptación al cambio climático? Más allá de las actividades iniciales, ¿ha habido actividades sobre cómo aumentar la participación de las mujeres en las actividades de generación de datos o de planificación del agua? Si así fuera, ¿puede dar un ejemplo, si alguna de estas discusiones se hubieran ya aplicado? The PAR A-Agua Project has made a consistent effort to target and invite women with relevant responsibilities to the consultation and capacity building activities. On the other hand, it is also important to address the limited consideration of socio-cultural issues in the context of water resources and climate change management, as underscored in PARA-Agua’s Gender Analysis (January 2015). Have there been awareness raising activities on socio cultural issues or the impact on population dynamics and gender relations in the new contexts of adaptation to climate change? Beyond these initial activities, have there been activities on how to increase women’s participation in targeted data generation or water planning activities? If yes, can you give me an example if any of these discussions have been implemented? 13 ¿Considera las consultas que llevaron a la CdP resultaron en un marco que proporcionará un espacio adecuado para la mejora de las capacidades, Do you consider the consultations leading to the CoP resulted in a framework that will provide an appropriate space for enhancing capacities, sharing MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Performance Evaluation Plan and Timeline 52 el intercambio de información y buenas prácticas? Tal y como está diseñada, ¿cuáles son los retos clave para el funcionamiento eficaz de la CdP? information and best practices? As designed, what are the key challenges for the CoP effective functioning? 14 ¿Cree usted que hay otros actores clave que también se beneficiarían del proyecto PARA-Agua en su cuenca y que actualmente no participan? Por favor explique. Do you believe there are other key stakeholders who would also benefit of the PARA-Agua Project in your basin and who currently do not participate? Please explain. 15 ¿Cómo se podrían mejorar los enfoques y estrategias de participación para asegurar que los datos generados sean pertinentes para los actores interesados? How could participatory approaches and strategies be enhanced to ensure data generated is relevant for targeted stakeholders? Building or strengthening sustainable links between hydro-meteorological centers and/or national authorities at various levels with water planning responsibilities Questions Spanish version Questions English version Comments / Remarks 14 ¿Cuánto éxito tiene el proyecto PARA￾Agua en la promoción de alianzas estratégicas a nivel local, nacional y regional para el intercambio de información, metodologías, entre otros? ¿Considera la CdP tiene el potencial de fortalecer esta actividad? How successful has the PAR A-Agua Project been in promoting strategic partnerships at the local, national and regional levels to exchange information, methodologies, among others? Do you consider the CoP has the potential of strengthening this activity? 15 ¿Podría indicar dos actividades de planificación del agua que se han fortalecido como resultado de la Alianza Estratégica con el Distrito de Control de Inundaciones y de Conservación del Agua del Condado de Yolo (California, EE.UU.) y el Consejo Would you please indicate two water planning activities that have been strengthened as a result of the twinning partnership with the Yolo County (California, U.S.A) Flood Control and Water Conservation District and the Chira Piura Watershed Not necessarily applicable to subcontractors MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Performance Evaluation Plan and Timeline 53 de Cuenca de Chira Piura? Council? 16 ¿Podría indicar dos actividades de planificación del agua que se han fortalecido como resultado de la Alianza Estratégica entre el Consejo de Chira Piura (Perú) y el Consejo de Recursos Hídricos de Chinchiná (Colombia)? Would you please indicate two water planning activities that have been strengthened as a result of the twinning partnership between the Council of Chira Piura (Peru) and the Water Resources Council of Chinchiná (Colombia)? Not necessarily applicable to subcontractors 17 ¿Están las cuestiones socio-culturales, incluyendo el impacto de la adaptación al cambio climático en las relaciones de género en los contextos de cuencas, siendo consideradas en las actividades previstas en los acuerdos de Alianzas Estratégicas? Si es así, ¿cómo? Por favor, proporcione algunos ejemplos. Are socio cultural issues, including the impact of climate change adaptation on gender relations in the watershed contexts, being considered in the activities under the twinning agreements? If so, how? Please provide some examples. MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Final Evaluation Report 54 ANNEX 4. SCOPE OF WORK The scope of work for contract AID-OAA-TO-15-00005 can be found in the file titled “Annex-4-AID￾OAA-TO-15-00005_Cloudburst_SOW.pdf” MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Final Evaluation Report 55 ANNEX 5. APPROVED EVALUATION PLAN AND TIMELINE The approved performance evaluation plan and timeline can be found in the file titled “Annex-5-PARA￾Agua Evaluation Plan_23 April 2015_APPROVED.pdf” MONITORING AND EVALUATION OF THE PARA-Agua ACTIVITY: Final Evaluation Report 1 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523 Tel: (202) 712-0000 Fax: (202) 216-3524 www.usaid.gov