EVALUATION Mid-term Evaluation of the Technical Assistance Program for the Ministry of Environment - Peru June 2015 This study has been prepared at the request of the United States Agency for International Development (USAID). It has been developed by a team of evaluators led by Marta Tostes of Consulting Center and Integrated Services (INNOVA-PUCP) subcontractor of Partners for Global Research and Development LLC (PGRD) under Contract No. AID-527-C-13-00002. ii iii MID-TERM PERFORMANCE EVALUATION OF THE TECHNICAL ASSISTANCE PROGRAM (TAP) FOR THE MINISTRY OF ENVIRONMENT (MINAM) Date of translation: June 2016 Team Consulting Center and Integrated Services of the Pontifical Catholic University of Peru (INNOVA-PUCP) Tostes Vieira, Marta Lucia (Coordinator) Ph.D. Carpio Castro, Augusto José Antonio Ph.D. Dall'Orso Soria, Carlos Antonio Martin Ph.D. Beteta Obreros, Edmundo Mae. Espinosa Winder, Diego MSc. Hernandez Zuniga, Daniel Escalante Barrantes, Gonzalo Contreras Moreno, Julie The views expressed herein belong to the work team that prepared the study and do not necessarily reflect the views of USAID or the Government of the United States. iv Table of Contents EXECUTIVE SUMMARY .......................................................................................................................7 I. BACKGROUND AND PROGRAM DESCRIPTION ................................................................................10 II. PURPOSE AND EVALUATION QUESTIONS ......................................................................................14 FRAMEWORK OF THE MID-TERM EVALUATION................................................................................................. 14 METHODOLOGY ..........................................................................................................................................16 III. FINDINGS, CONCLUSIONS AND RECOMMENDATIONS ...................................................................20 AREA 1: THE COOPERATION MODEL IN THE NATIONAL AND INTERNATIONAL CONTEXT............................................20 T1.A1: The model in the context of international cooperation ...........................................................21 T2.A1: The model in a changing national context...............................................................................24 AREA 2: PROGRAM DESIGN RELEVANCE.........................................................................................................28 T3.A2: Initial Design ............................................................................................................................28 T4.A2: Performance measurement .....................................................................................................30 T5.A2: Management of results framework changes ...........................................................................35 AREA 3: PROGRAM IMPLEMENTATION MANAGEMENT .....................................................................................39 T6.A3: Planning Standards ..................................................................................................................39 T7.A3: Communications and quality management.............................................................................40 T8.A3: Cost Management ...................................................................................................................42 T9.A3 Risks and Changes Management ..............................................................................................43 T10.A3 Human Resource Management ..............................................................................................44 AREA 4: TAP EFFECTIVENESS AND MONITORING.............................................................................................49 T11.A4: Achievement of Results ..........................................................................................................49 T12 A4: Quality of indicators and lessons learned ..............................................................................55 IV. CONCLUSIONS AND RECOMMENDATIONS ...................................................................................59 ANNEXES .........................................................................................................................................63 Annex I: Statement of Work (SOW) .....................................................................................................64 Annex III: Secondary information sources ...........................................................................................95 Annex IV. Primary information sources ...............................................................................................97 Annex VI: Statement of Differences .................................................................................................. 100 Annex VII: TAP Organizational chart ................................................................................................. 101 Annex VIII: Disclosure of Conflict of Interest ..................................................................................... 102 v ACRONYMS ACC Adaptation to Climate Change ADS Automated Directives System AOP Annual Operating Plan (Plan Operativo Anual) BID Interamerican Development Bank (Banco Interamericano de Desarrollo) CEPLAN National Center of Strategic Planning (Centro Nacional de Planteamiento Estratégico) CITES Convention on International Trade in Endangered Species DGDB Directorate General of Biodiversity (Dirección General de Diversidad Biológica) DGVEFPN Directorate General of Valuation, Evaluation and Financing of the Natural Heritage (Dirección General de Valoración, Evaluación y Financiamiento del Patrimonio Natural) DGPNIGA Environmental Management Standards and Tools (Normas e Instrumentos de Gestión Ambiental) DGCCDRH Directorate General of Climate Change, Desertification and Water Resources (Dirección General de Cambio Climático, Desertificación y Recursos Hídricos) FEMA Specialized Environmental Prosecution Service (Fiscalía Especializada en Materia Ambiental) GORE Regional government (Gobierno Regional) GIZ German Agency for International Cooperation (Cooperación Alemana del Desarrollo) G2G Government-to-Government ICAA Initiative for Conservation in the Andean Amazon IR Intermediate Results MEF Ministry of Economy and Finance MINAM Ministry of Environment (Ministerio del Ambiente) M&E Monitoring and Evaluation NAMAS Nationally Appropriate Mitigations Actions OEFA Agency for Environmental Assessment and Enforcement (Organismo de Evaluación y Fiscalización Ambiental) OGA General AdministrationOffice (Oficina General de Administración) OSCE Supervisory Agency of Government Procurement (Organismo Supervisor de las Contrataciones del Estado) TAP Technical Assistance Program PIP Public Investment Project PIRS Performance Indicator Reference Sheets PNCB National Program for Forest Conservation and Climate Change Mitigation (Programa Nacional de Conservación de Bosques para la Mitigación del Cambio Climático) PESEM Multi Annual Strategic Plan by Sector (Plan Estratégico Sectorial Multianual) PEI Institutional Strategic Plan (Plan Estratégico Institucional) PFSI Peruvian Forest Sector Initiative PM&E Monitoring and Evaluation Plan (Plan de Monitoreo y Evaluación) PNAA National Environmental Action Plan (Plan Nacional de Acción Ambiental) REDD + Reducing Emissions from Deforestation and forest Degradation (Reducción de Emisiones por Deforestación y Degradación de bosques) SASE Monitoring, Analysis and Evaluation for Institutional Development (Seguimiento Análisis y Evaluación para el Desarrollo Institucional) SEIA National System of Environmental Impact Assessment (Sistema Nacional de Evaluación para el Desarrollo Institucional) vi SENAMHI National Meteorology and Hydrology Service of Peru (Servicio Nacional de Metereología e Hidrología del Perú) SNGA National Environmental Management System (Sistema Nacional de Gestión Ambiental) SNIP National Public Investment System (Sistema Nacional de Inversión Pública) SOW Statement of Work or Terms of Reference for the evaluation TLC Free Trade Agreement with the United States of America (Tratado de Libre Comercio con los Estados Unidos de América) TOR Terms of reference (Términos de referencia) UE Program Implementation Unit (Unidad Ejecutora del Programa) USAID United States Agency for International Development 7 EXECUTIVE SUMMARY PURPOSE This mid-term evaluation makes strategic and operative recommendations based on a performance analysis of the Technical Assistance Program (TAP) for the Ministry of the Environment (MINAM). These recommendations are intended to support the project to accomplish its objectives by its completion in 2016. The study seeks to identify obstacles that may be hampering the effectiveness of the TAP by identifying the reasons that the objectives embodied in the project’s results framework have been achieved or not. BACKGROUND The Technical Assistance Program is implemented under the Assistance Agreements No. 527-0423 and No. 527-0426 between the Peruvian Government and United States Agency for International Development (USAID). The Program has as its expected result: “to contribute to the sustainable use of natural resources through strengthening the Ministry of the Environment as the lead agency in the National Environmental Management System.” The Program has an implementation period of four years (2012-2016) and a budget of US$12 million. The TAP has five components – activities –, each one of them connected to a priority MINAM theme and to a lead agency for the respective theme:  The CITES TAP seeks to advise the MINAM Bureau for Biological Diversity (“Dirección General de Diversidad Biológica”, DGDB) regarding the effective application of the Convention on International Trade in Endangered Species (CITES);  The Valuation TAP seeks to support the Bureau for the Valuation, Assessment and Funding of the Natural Heritage (“Dirección General de Valoración, Evaluación y Financiamiento del Patrimonio Nacional”, DGVEFPN) in ensuring that the ecosystems offering environmental resources such as providing water and scenic beauty will be operated in a sustainable manner;  The SEIA TAP has as its counterpart the Bureau for Environmental Management Policies, Standards and Instruments (“Dirección General de Política, Normas e Instrumentos de Gestión Ambiental”, DGPNIGA), with which it generates tools, technical procedures and training in order to strengthen the preventive character of the National Environmental Impact Assessment System ("Sistema Nacional de Evaluación de Impacto Ambiental”, SEIA);  The Forests TAP seeks to contribute to the management, conservation and sustainable use of Amazon forests, including the financial mechanisms for their conservation. Its counterpart is the National Program for the Conservation of Forests to Mitigate Climate Change (“Programa Nacional de Conservación de Bosques para la Mitigación del Cambio Climático”, PNCB);  The primary purpose of the ACC TAP is to incorporate an Adaptation to Climate Change (ACC) approach into the National Public Investment System (“Sistema Nacional de Inversión Pública”, SNIP). It does this by developing methodological guides, regulatory proposals and technical assistance in coordination with the Bureau for Climate Change, Desertification and 8 Water Resources (“Dirección General de Cambio Climático, Desertificación y Recursos Hídricos”, DGCCDRH) of MINAM. The TAP functions as an Implementation Unit led by a Coordinator that is supported by a small administrative team and five technical coordinators. Each technical coordinator leads one of the five projects that make up the Program. The Annual Operating Plan provides the framework for the execution of TAP activities. The Implementation Unit coordinates this plan with the four Bureaus and the National Program, which function as counterparts representing MINAM. USAID funding for this Program enters the public budget and is executed in accordance with public sector administrative regulations. As such, the Program constitutes a modality for Government-to-Government (G2G) cooperation that is oriented towards increasing counterpart capacity and the sustainability of results beyond the period of USAID funding. EVALUATION QUESTIONS AND METHODOLOGY The mid-term assessment of the TAP seeks to answer the following questions: Q1. In terms of the national and international context: What risks and opportunities does USAID’s Government-to-Government model present in the context of the MINAM TAP with changes in the national agenda? Q2. How appropriate is the design of the program given trends and dynamics at USAID and the evolution of MINAM? Q3. What strengths and weaknesses exist vis-à-vis the implementation process of the MINAM TAP? Q4. How effective is the TAP in this intermediate stage and what critical factors have to be addressed in order to improve it? These questions provide four focus areas for the evaluation: USAID’s G2G model, the TAP design, project management, and project results. To explore these focus areas, the evaluation uses twelve (12) themes and forty (40) specific evaluation questions. The evaluation employes a qualitative assessment methodology complemented by a review of secondary information to apply the evaluation themes and questions. This approach resulted in sixtythree (63) interviews, four workshops and one case study. The evaluation team used Atlas.ti software to support the analysis of secondary information. FINDINGS, CONCLUSIONS AND RECOMMENDATIONS The principal conclusions and recommendations are captured in the following nine points: 1. Relevance of objectives. The priorities defined in the environmental sector’s national policies are presented genericly in various documents. These require a disaggregated analysis to orient the TAP investment. For this reason, USAID should continue supporting MINAM to define specific objectives without interfering in its autonomy. 2. Relevance of objectives. Project implementation and performance (both financial and achievement of results) are not meeting expectations. The announcement of changes to the results frameworks is a positive adjustment. The productive relationship that has been achieved between MINAM and USAID, i.e. the development of the coordination mechanisms required to carry out activities and actions, gives rise to the recommendation to extend the TAP implementation period in order to provide time for the activity to build on this cooperation, to achieve project objectives, and to fulfill the commitments defined 9 in the Free Trade Agreement with regard to environmental priorities. 3. Management tools. The participatory modification of the TAP performance objectives is positive and necessary. This will require taking up the challenge of updating the performance monitoring system and addressing the absence of a Program Description. The Program Description should describe and update the project’s management tools in accordance with USAID Forward. The majority of respondents recognize the absence of an adequate Program Description as a project weakness. 4. Actions common to all five projects. One of the most important conclusions is that the project is challenged by its dependence on the public administrative system, which is challenged in its ability to carry out joint or crosscutting actions. TAP should support MINAM to strengthen its administrative functions in order to reduce the time and cost of carrying out actions that are common to the five activities. 5. Governance. TAP support for MINAM prioritization at the central level is positive. In order to continue with this process, TAP should establish closer links with the subnational governments to develop local capacities, promote consensus and prioritize investments. 6. Gender focus. The identification of women’s participation in courses, seminars, workshops, etc., helps to demonstrate project interest in developing a gender focus in its activities. TAP should develop a plan that specifies how the project and its activities reflect the principles of gender equity. 7. Coordination with USAID. USAID sector-specific experts have facilitated communication between the front-line departments of MINAM and the TAP. USAID should enhance its role by strengthening USAID representatives’ communication and coordination with the highest levels of MINAM. 8. Coordination measures with the TAP. Internally, TAP does not offer the appropriate management tools to its leadership. These tools should include a Program Description, an appropriate detailed position description for the technical coordinators, and a clear chain of command. The project should quickly develop the appropriate instruments and management capabilities to address this limitation. 9. Communications. The project’s knowledge management is weak. The project should develop a communication plan (in the context of the monitoring and evaluation plan) and develop formats that facilitate follow-up on agreed-upon actions. The TAP has made an important achievement by incorporating a number of resources originating from international cooperation into the Peruvian public sector budget. These can be used to strengthen the capacity of MINAM as the lead agency of the National Environmental Management System (SNGA). However, delays in the initial implementation process and in establishing an understanding of how a program of this nature should function now require rethinking MINAM needs and adapting to the process of consolidation at MINAM. The TAP will need to define a set of specific objectives and performance results that reflect this process and the accompanying changes to optimize results over the remainder of the project. 10 I. BACKGROUND AND PROGRAM DESCRIPTION The Technical Assistance Program (TAP) of the United States Agency for International Development (USAID) to the Ministry of Environment of Peru (MINAM) originates in the framework of Grant Agreement No. 527-0423, signed in 2008 between the United States of America and the Republic of Peru. Through this agreement, USAID provides technical assistance on key themes for the sustainable development of the country, among which the environmental aspect is particularly important. Its objectives included: "Providing assistance to increase public and private investment in conservation; create opportunities to improve livelihoods based on the sustainable management of resources; and improve selected sectors’ sustainable environmental management, strengthening the capacity of public and private entities to improve environmental management.” In this context, in 2010 MINAM solicited consulting services for the “Development of the Technical Assistance Program for MINAM". In November 2010, the consulting firm, Monitoring, Analysis and Evaluation for Development (SASE) presented a proposal. The objective of the consultancy was to design and plan four technical assistance projects with a four-year horizon to form the basis for the current TAP MINAM. The change of government in 2011 precipitated a reorientation of priorities of the Peruvian State on environmental issues, which led to the suspension of the TAP implementation until 2012. In that year, USAID approved the Country Development Cooperation Strategy, 2012-2016, and signed new Grant Agreement No. 527-0426, currently in effect, which has as one of its three goals the development and sustainable management of natural resources in the Amazon and glacial mountains. The priorities of the agreement, and in particular its geographical areas of action, goals and targeted results were incorporated into the TAP. Also, the team hired for its implementation updated the initial design to adapt it to the processes being developed for the country's environmental management. Thus, the TAP started its implementation in August 2012, for a four-year period (2012-2016) with a twelve million dollar budget (US$12, 000,000) to "Contribute to the sustainable use of natural resources by strengthening the Ministry of Environment as the governing body of the National Environmental Management System (SNGA)." The target population includes government officials involved in the national, regional and local levels of government. This covers both MINAM and regional, provincial and local governments officials. Also, TAP prioritizes the Amazon and the glacial mountains as intervention areas of the following regions: Madre de Dios, Loreto, Ucayali, San Martin, Amazonas, Junin, Cusco, Puno, Pasco, La Libertad and Cajamarca. The program implementation unit (UE) is attached to the General Secretariat of MINAM and is funded under the “Budget for Results” budget category. This funding is implemented through five projects: 1. CITES TAP: Its purpose is to strengthen MINAM capabilities for the implementation of the Convention on International Trade in Endangered Species (CITES), in order to ensure that specified species of wild fauna and flora are provided with rulings that ensure their exportation does not prejudice their survival. To achieve this, it is necessary to generate information that allows authorities to take into account recommendations for the management of priority species, as well as keeping scientific experts 11 and regional enforcement authorities informed. These should promote decision-making in project areas. The CITES project provides technical assistance to its counterpart, the Directorate General of Biodiversity (DGDB) of MINAM, responsible for training the authorities and national experts, and provides updated information on the flora and fauna registered in CITES to strengthen the protection of priority species, among other activities. 2. VALUATION TAP: Seeks to establish the sustainable use of ecosystems that provide clean water and scenic beauty. The project promotes the development of tools and studies that allow the establishment of economic and financial mechanisms to leverage income from the production or services originating from sustainable ecosystems. The project also generates specialized training courses to improve and, in turn, generate new studies based on the lessons learned and the experience of the actors in target areas, scientific specialists, relevant authorities, and private companies, among others. It is worth mentioning that studies produced by this project served as the principle input for the elaboration of the official guides for evaluating flora and fauna, the environmental patrimony guide and the environmental impacts guide.. Furthermore, the economic assessment tools are essential in the regions for evaluating local resources. The MINAM counterpart is the Directorate General of Valuation, Evaluation and Financing of Natural Patrimony (DGVEFPN). 3. SEIA TAP: Strengthens the preventive character of the National System of Environmental Impact Assessment (SEIA), through the identification, assessment, mitigation and early correction of significant negative environmental impacts on biodiversity and its related components caused by investment projects. This involves improving the system’s technical and administrative procedures and developing tools to evaluate and protect ecosystems prioritized for their biodiversity and fragility. Currently, the group has provided technical assistance in the process of adapting environmental regulations for the mining, hydrocarbon and industry sectors, as well as working with the Environmental Management Corrective Instrument (IGAC) to formalize small-scale and artisanal mining. The TAP uses technical assistance, tools development, training, networking between relevant authorities at the regional and SEIA level, and others tactics to improve the regulatory system’s results. Its counterpart is the Directorate General of Policy, Standards and Tools of Environmental Management (DGPNIGA) of MINAM. 4. FORESTS TAP: Supports MINAM to implement a forest conservation management model that can be replicated at the national level. The project works on strengthening the administrative, technical and operational capabilities of the National Forest Conservation Program, the regional governments, the provincial municipalities and communities through technical assistance to the relevant forest conservation actors, the development of tools such as guides and forest plans, forestry studies that visualize informal sectors operating in implementation areas, etc. These activities aim at improving the management, conservation and sustainable use of the Amazonian forests, and to design and implement appropriate financial mechanisms permitting their conservation, in coordination with its stakeholders: the National Forest Conservation Program for Climate Change Mitigation (PNCB), the Ministry of Economy and Finance (MEF) and the private sector. 5. ACC TAP: Its main goal is to incorporate the approach for Adaptation to Climate Change (ACC) to the National Public Investment System (SNIP). It has promoted the development of methodological documents that provide guidelines for the relevant actors in the production and service sectors at a national level. In this context, the project develops technical assistance and a relationship between MINAM and MEF in order to establish the General Guide for Developing Public Investment Projects. This guide incorporates aspects of risk management in the context of climate change, with the 12 aim of establishing the foundation for working with other sectors at the nation level. In order to implement these modifications, it is important to undertake specialized studies that can be validated by specialists from each sector. In addition, the project offers trainings to relevant authorities at the national level regarding adaptation to climate change. Its counterpart is the Directorate General for Climate Change, Desertification and Water Resources (DGCCDRH) of MINAM. The TAP development hypotheses is embodied in the program’s results framework, which is presented in the monitoring and evaluation plan (PM&E) from August 2013, and includes logical frameworks for its four initial projects. These frameworks establish, in addition to the purpose of the program, strategic objectives and intermediate results, indicators used to measure these results, targets to be met, and means of verification and underlying assumptions. There are a total of nine standard USAID indicators that cut across the four projects that contribute to the objectives of the Convention 527-0426. The TAP organizational structure is comprised of technical coordinators for each project. These are responsible for coordinating with the four line Ministry directorates and the PNCB, in order to define and implement activities described in the Annual Operating Plan (AOP) for each project. The core decision-making team also includes one monitoring and evaluation specialist, responsible for progress reporting; one specialist an administration and finance specialist, and an executive coordinator responsible for program implementation and for reporting to USAID. Since its beginning, the most important changes to the program have been (i) the addition of a fifth project, the Adaptation to Climate Change (ACC) TAP in 2014, and (ii) the reformulation of the results frameworks for each project, with the intention of adapting to changes in national environmental management priorities. In the case of SEIA TAP, the project adopted new results in response to requests from the DGVEFPN and changes in the PNCB after it had achieved the implementation of the National Service of Environmental Certification for Sustainable Investment (SENACE). The Annual Operating Plan (AOP) of the TAP is defined by the technical coordinator of each project, their counterpart Ministry directorates and the PNCB in the last quarter of the calendar year. Usually, the plans are produced in a series of meetings, after which the TAP team and USAID conduct a review of the plans. USAID cooperation funds for TAP are provided through the MINAM budget and are subject to the regulations of the Peruvian public sector. If necessary administrative processes have not been completed, the first quarter is a period of low budget execution. During this period, coordinators develop the terms of reference (TOR) for the services to be procured in coordination with personnel designated by the line director of MINAM. Each TOR is reviewed by the TAP executive coordinator, the TAP M&E specialist, the administration and finance specialist and USAID before the solicitation process begins. Once the procurement requests are submitted, the TAP administrative area monitors procurement progress in coordination with the General Administration Office (OGA) and the Office of Planning and Budget (OPP) of MINAM. In turn, the coordinators of each project monitor the processes that correspond to them. During the process of implementation, the technical coordinators are responsible for supervising the consultants so that products are delivered on time and are of expected quality. According to the terms established in the terms of reference, coordinators are expected to maintain constant communication with the consultants to avoid problems with deliverables. Products have to be reviewed by the counterpart assignee of each directorate and TAP technical coordinators, who issue technical reviews 13 separately. Both reports are necessary to approve the deliverable or to request adjustments. The TAP coordinators have to follow-up with Directorate counterparts to ensure that reports are generated. Technical coordinators support line management, since the vast majority of the TAP activities require collaboration with these counterparts or the monitoring of activities that counterparts carry out as part of their annual plans. In this context, the technical coordinators need to achieve fluid communication, empathy and synergy with counterpart officials as well as build communication channels that are essential to the success of each TAP project. 14 II. PURPOSE AND EVALUATION QUESTIONS The purpose of this midterm evaluation is to analyze and make recommendations for strategic and operational management, based on evidence in order to improve the implementation of the TAP for the fulfillment of its goals by 2016. The evaluation is focused on identifying obstacles to the effectiveness of the TAP and understanding the reasons why the objectives of the results framework were achieved or not. The Statement of Work for the evaluation (SOW) is found in Annex I. The specific objectives of the midterm evaluation are as follows: 1. Learning about the risks and opportunities posed by the government-to-government model (G2G) applied to MINAM TAP, in the contextual framework of national and international cooperation. 2. Evaluating the relevance of the initial design of the program, with regards to the trends and dynamics of USAID and the evolution of MINAM. 3. Document the strengths and weaknesses that exist in the implementation process and management of MINAM TAP. 4. Understanding the critical factors that determine the effectiveness of the TAP at its midpoint. Framework of the Mid-Term Evaluation The specific objectives described above correspond to initial indications that signaled the need for an evaluation. These indications refer to gaps in the effectiveness and efficiency of the TAP (see Methodology section) corresponding to four areas: challenges in the program environment, relevance of the project design, TAP team management difficulties, and project effectiveness and results monitoring. The following tables represent the different levels of analysis that were carried out during the evaluation: the general level corresponding to evaluation areas and overarching questions, the intermediate level with evaluation themes and the direct level through the evaluation questions. Table 1 shows the general level with the evaluation areas and the general questions by area. Table 2 presents themes for intermediate analysis and the evaluation questions for direct analysis, defined in the SOW, disaggregated into 40 questions. 15 Table 1: Areas and questions of the midterm evaluation Areas General question A1. Program model in national and international context Q1. Depending on the national and international context, what risks and opportunities does the model of USAID government-to-government international cooperation have, applied to the context of MINAM TAP, with changes in the national agenda? A2. Relevance of the initial design of Program Q2. What is the relevance of the program design in terms of trends and dynamics at USAID and the evolution of MINAM? A3. Program management Q3. What strengths and weaknesses exist in the process of implementation of MINAM TAP? A4. Program efficiency and monitoring Q4. What is the effectiveness of the TAP in this intermediate stage and what critical factors need to be addressed in order to improve it? Table 2: Levels of analysis of the mid-term review Areas Topics Questions (*) A1. Program model in the national and international context T1.A1. Context of international cooperation 6 T2.A1. Changes in the national agenda 3 A2. Relevance of the initial program design. T3.A2. Initial Design 3 T4.A2. Performance measurement 4 T5.A2. Changes management in the results framework 4 A3. Program management T6.A3. Planning standards 2 T7.A3. Communications and quality management 5 T8.A3. Cost control 2 T9.A3. Risk and change management 2 T10.A3. Human capital management of the program 3 A4. Program efficiency and monitoring T11.A4. Achieving Results 2 T12.A4. Quality indicators and lessons learned 4 Total of areas: 4 Total of topics: 12 Total of questions: 40 (*) Group number of fieldwork questions, described in the SOW, Annex 1. 16 Methodology The evaluation team has gone into great detail to ensure the methodological quality of the evaluation, since it represents a performance evaluation of a program consisting of five projects, each with specific dynamics and unique challenges. In order to provide details regarding the evaluation process, Annex II1 presents a Field Work Report that provides a detailed description of the methodology and lessons learned during the evaluation. The methodological diagram shown below (Diagram 1) summarizes the process of gathering evidence, and reaching findings, conclusions and recommendations for the evaluation: Diagram 1: Steps used in the methodology of the midterm evaluation The development of the evaluation begins with the signs previously noticed by the USAID team, consisting of factual evidence, and then moves to analysis in the assessment methodology that allows identification of empirically verifiable findings. The aim is to distinguish patterns and trends regarding TAP performance that support short and mid-term conclusions and recommendations. Furthermore, these signs served as triggers, through which the evaluators were able to identify analytical themes, which in turn, allowed the structuring of evaluation results. These signs are shown below: Initial signs  Bottlenecks in the implementation of the TAP;  Deficiencies in the implementation unit;  Lack of evidence supporting the selection of activity areas and beneficiaries;  Lack of, or unclear, coordination among the five projects, which makes it difficult to understand them as a program;  Difficulty in coordinating between the three levels of government; 1 Annex II can be found in the original Spanish versión of this evaluation. STEP 1: SIGNS: The evaluation methodology is guided by USAID initial perceptions to the TAP development. STEP 2 METHODOLOGY: Definition of evaluation, design areas and topics, and applications of methods of information gathering. STEP 3 FINDINGS: Analysis of tactical evidence that answers the evaluation questions and is verifiable. STEP 4 CONCLUSIONS: Synthesis from thematic and general findings, defining general patterns and trends in the TAP development. SET 5 RECOMMENDATIONS: Technical assistance in strategic and operational decision-making of the TAP goals by 2016. 17  Low execution of funds and administrative management problems; and,  Lack of definition regarding what constitutes completion of the results framework objectives. The SOW requires an emphasis on evaluation of the Valuation TAP and Forests TAP, which was reflected in the development of methodological tools and the preparation of this report. The findings were identified by the evaluation team from the data collected during the evaluation process. Sources included document reviews, interviews and workshops. The team formulated general conclusions and recommendations to support decision-making in the TAP for both the short and medium term following the structure presented in Diagram 1. Based on an analysis of the evaluation questions, the team determined that answers needed to be derived from data from different sources of information. This approach included four equally important and complementary processes of information gathering, primarily qualitative: one review of secondary information and three primary information-gathering processes. Secondary data review included the results monitoring data for each project according to the results framework, as well as the quarterly and annual reports of the TAP, and event registers and documents resulting from program activities, such as designs, studies, methodologies, and manuals, among others. One of the central objectives of the review was to provide a first estimate regarding the verification of results and performance of the TAP in general, which would determine the focus of other data gathering approaches. The review of secondary information also included the means of verification of the TAP achievements, for which, a continuous comparison mechanism was established. Annex III presents a bibliography of secondary information sources. Workshops, structured interviews and a case study were carried out to collect primary data. The fieldwork tools were refined in order to track indicators and analyze them to compare and contrast different perspectives, opinions and findings. A process of standardization and improvement of tools was carried out during fieldwork development, which included the use of pilot interviews and meetings, as well as interviews with the TAP team to compare the findings at the end of the process. While the field tools underwent a process of adaptation, the data collection process maintained consistency in terms of the topics and structure of the interviews, as well as the general interview management protocol. Diagram 2 presents the sequence of information gathering methods of the evaluation. 18 Diagram 2: Sequence of information gathering methods for the midterm evaluation The directory of the actors interviewed can be found in Annex V, Primary Information Sources. The classification is below in Diagram 3: Diagram 3: Prioritization of key informants and effectiveness of interviews 19 The analysis of information from fieldwork was carried out with the help of Atlas.ti, software facilitating the systematization and analysis of qualitative information. This allowed the organization of qualitative information into hermeneutical units and their classification according to 12 topics and 40 evaluation questions. It is important to note that the analysis was performed according to the three levels described above: areas, topics and evaluation questions. The sequence of this process is presented in Scheme 4. Diagram 4: Sequence of analysis of the information on the midterm review 20 III. FINDINGS, CONCLUSIONS AND RECOMMENDATIONS As stated in the SOW methodology, presented in Annex I, the findings, conclusions and recommendations correspond to different stages of analysis. The findings are empirically verifiable and have been identified through methods of primary and secondary information gathering, as described in the methodology. In this sense, the findings are comparable and verifiable using secondary information provided to the evaluators or transcripts of interviews. Because of the extent of the evaluation framework, the level of detail in the specific evaluation questions and the amount of information collected, through both interviews and review of secondary information, this report presents a synthesis of the most important and prevalent findings. Annex V2 includes tables by topic that presents the findings, conclusions and recommendations at the level of specific evaluation questions. The conclusions are deductions made by the evaluators as a result of an analysis and interpretation of findings. They seek to assess the issues evaluated (national context, program design, human capital management, performance measurement, cost control, etc.). They also seek to answer the evaluation questions associated with each evaluation area. The recommendations are tasks formulated with a view of overcoming obstacles and taking advantage of opportunities in order to obtain the expected program results by 2016. The following section analyzes the most relevant findings for each evaluation area. AREA 1: The cooperation model in the national and international context Area 1 seeks to answer the following question: Based on the national and international environment, what are the risks and opportunities of the USAID model for government-to-government cooperation when it is applied through TAP MINAM in context of changes in the national agenda? As evidenced below - from interviews and documents reviewed - the G2G model is a contribution and an important opportunity in terms of international cooperation because it is oriented towards meeting the institutional objectives of MINAM, as a national environmental authority. This is reflected in the prioritization of issues on the national agenda and the improvement of results-based management in pursuit of institutional sustainability. However, there is a major challenge in terms of the MINAM capacity to expend the due to difficulties in the recruitment processes. These processes reflect the requirements of the Peruvian public sector and USAID restrictions on the use of funds. In order to answer this question in more detail, two evaluation issues have been established within Area 1: the effectiveness of the adopted model in the context of international cooperation, and the effectiveness of the model facing changes in the national context. Also, as stated in the evaluation design, each of these topics includes specific questions that have guided information collection. The findings are 2 Annex IV can be found in the original Spanish versión of this evaluation. 21 then developed referencing the responses of the various actors to answer the general question of Area 1. T1.A1: The model in the context of international cooperation 1.1 The USAID G2G cooperation model: Context and actors. First, we describe the central characteristics of the G2G model and its type of program. The G2G model allows MINAM to incorporate funds directly to its budget and assign them according to its priorities. This is demonstrated by the fact that the funds are incorporated into the administrative system of MINAM through implementation letters, responding to the annual work plans (AOP) of the TAP. In this sense, MINAM perceives the USAID model as useful in that, by incorporating funds directly into its administrative system, MINAM is better able to systematically monitor and supervise its priority efforts to contribute to the achievement of the MINAM strategic results. The TAP result frameworks adopt the objectives of the MINAM Directorates-General AOP, insofar as these objectives aim towards the improvement of its institutional capacity, especially with regards to the implementation of results-based management. This occurs to the extent that the TAP team provides technical assistance to the DG to elaborate the AOP and to reinforce the need to define the performance measures in each area. Through this collaboration, MINAM assumes greater ownership of the project because it is related to their own institutional objectives, as well as the very definition of these objectives, making it more consistent with its long-term integrated vision. In this way, the G2G model responds to commitments made in the Paris Declaration regarding the effectiveness of development aid, and the Accra Agenda for Action (2005), which states that funding should be linked to the fulfillment of conditions or indicators derived from the national development strategy, with the goal of strengthening the capabilities of national governments, not forcing them to accept priorities of other countries. The senior management of MINAM and USAID recognize that such cooperation is positive because it facilitates a focus on MINAM priorities. However, USAID operational policies included in the document Use of Reliable Partner Country Systems for Management and Implementation of Direct Assistance (ADS 220), stress that the decision to use the G2G model should result from a strategic planning and design process that includes a rigorous analysis of risks and mitigation strategies. ADS 220 was published in August 2014, long after the program initiated activities in 2012. Therefore, these requirements were not clear when MINAM TAP was designed, or when implementation began, or when its PM&E entered into force (the PM&E was last updated in August 2013). 1.2 Program opportunities and risks regarding the use of specific tools for the implementation of the cooperation agreement No. 527-0426. The USAID grant agreement is formalized and becomes operational through implementation letters, which commit funds for the implementation of program activities. This mechanism allows the use of implementation letters to approve management documents, such as the AOP, and to register changes and formalize commitments. They establish general conditions with little detail, referencing or attaching documents that complement these conditions. The low level of detail allows flexibility in making operational decisions. However, the delay in the approval of these documents by USAID, as noted by the TAP team, resulted in delays in the initiation of some activities. In general, it can be noted that the main opportunities and risks in this type of model are as follows: 22 Opportunities:  The model allows MINAM to incorporate funds directly into their priorities, making activities more consistent with its long-term integrated vision. This model is consistent with international agreements to strengthen the capabilities of national governments. This generates institutional sustainability by strengthening the environmental authority.  The mechanism promotes the definition of priorities, and through this process improves MINAM institutional capacity, especially in terms of the implementation of results-based management, which is a national policy. Risks:  The area of International Cooperation, the General Secretary and the TAP team properly understand this kind of model of cooperation. In the case of the Directorates and the PNCB, this familiarity is limited and becomes an obstacle to their taking advantage of its benefits and financing their priorities.  There is no program management document (Program Description) to establish project timelines and tools (TAP and USAID). The absence of key management documents will present a challenge even for the best development model. 1.3 Benchmarking of models of international cooperation in the context of MINAM cooperation. MINAM has many donors that use different mechanism to channel their support. Some models, such as the German cooperation (GIZ), do not include direct funding for MINAM, so they are not vulnerable to administrative challenges of the public sector. They are perceived as "more flexible" in terms of pursuing their goals, which are not necessarily MINAM goals. According to the TAP team, the project’s achievements are not comparable with those of other donors because its objectives and methodologies are different. In this regard, other cooperation models do not necessarily address the priorities of the ministry itself, but perform activities according to the objectives of the partners and counterparts. These models include funding for the fulfillment of goals (European Union), of bilateral cooperation with rapid reaction capability (e.g. GIZ and other USAID initiatives) or standardized multilateral cooperation (Interamerican Development Bank). It is important to note that there is no study that shows the advantages and disadvantages of each model, nor channels that disseminate the benefits of the G2G USAID model. There is a perception that other models of cooperation are more agile, but there is not a specific study to support this conclusion. Senior MINAM management considers USAID cooperation as fundamental in pursuing its institutional objectives. However, according to some interviewees, the dynamics of MINAM, in terms of both its internal consolidation and policy towards other sectors, often focuses efforts on activities to address immediate situations, which results in an emphasis on other donor cooperation that allows for flexible implementation. Also, the perception of flexibility is increased by the fact that some donor funding exceeds TAP funds, and it seems that these sources with "greater flexibility" respond better to certain short-term priorities of MINAM. 23 The MINAM project management unit tries to focus donor support in different regions of the country, which is the case for TAP. However, the TAP does not have a clear alignment with other MINAM partners that meet through the Green Table (Mesa Verde). This framework is not used to design TAP activities, which is evident in the lack of synergies with similar programs or projects at the subnational level. Moreover, the presence of various types of donor cooperation requires greater coordination between projects of USAID and other sources, to avoid duplicating efforts and leaving gaps unattended. This is a fundamental aspect, especially in the case of the Forests TAP. 1.4 Other initiatives, projects, programs funded by USAID and its counterparts: synergies and complementarities with the TAP. MINAM does not have an area that coordinates all of the cooperation-funded initiatives that are related to TAP, nor of the projects funded by USAID on the subject. USAID interviewees reported concerns on this subject, but an overview of objectives and stakeholders has not been adequately disseminated among the various counterparts. The team is only aware of one event that sought to align the project objectives with the objectives of USAID Development Objective 3 (DO3). There are several projects or programs that share themes and lines of action with TAP. There is no evidence of a clear alignment among these projects, since they duplicate activities and have no link to each other. Among the best-known initiatives that have initiatives similar to the activities of TAP, there are projects related to forest conservation, such as Peru Forest Sector Initiative (PFSI) and Peru Bosques and the Initiative for the Conservation of the Andean Amazon (ICAA). In these cases, the presence of USAID technical officers allows a link between other initiatives and the Governing Body of the National Environmental Management System (SNGA) in Peru, which occurs through the TAP platform. 1.5 Procedures for the implementation of MINAM TAP Peruvian public sector processes apply to the implementation of TAP to define priorities, budget execution and carry out recruitment. In addition, USAID must approve a variety of TAP instruments, such as letters of implementation, the APO or the terms of reference (TOR) of consultancies. Also, USAID places conditions on implementation based on its policies and guidelines regarding eligible expenses. Finally, TAP must report on project performance to USAID according to its requirements for monitoring and evaluation. Interviewees reported that there are processes and requirements for administrative approval of USAID that were not known initially since they did not have previous experience with this donor.3 MINAM officials had to learn them during implementation. This had an impact on the implementation of activities and consequently on financial performance, as happened in the case of the purchase of a server that the PNCB requested for the Forests TAP. The change of the USAID coordination team also had impact on the agility of implementation processes. Additionally there is evidence that, while the MINAM General Administration Office (OGA) is 3 MINAM has a work experience with USAID on Strengthened Environmental Management project to address priority problems (STEM). It is likely that the lessons learned from this experience have been poorly communicated. 24 committed to the Directorates-General to support the financed activities, contracting in support of TAP implementation was described by some interviewees as atomized and unarticulated. In some cases, implementation was compromised by changes in the political priorities of the DG or by errors in TORs, resulting in the annulment of recruitments, cases in which responsibility also fell to the technical elaboration of the TORs. On the other hand, the implementation of results-based management promoted by USAID with its tools generated great interest, not only among the involved DGs, but also by other areas of MINAM. In this sense, there is a process of uptake by MINAM around results-based management, including the results framework tool, which is establishing itself as an instrument for establishing consensus and visualizing the main objectives that guide the DGs in the preparation of the AOP. 1.6 Opportunities and risks associated with modern USAID management tools (USAID Forward). USAID Forward represents a policy of modernizing the management of development projects using results-based management (http://www.usaid.gov/usaidforward). This includes new standardized management tools, protocols, and procedures, and through the use of a logical and clear structure, presents performance indicators that are aligned with project objectives. In this context, the emphasis on monitoring is important because of its relevance to achieving the implementation results-based management. Initially, there was a knowledge gap among the technical coordinators of TAP regarding USAID procedures, besides Pervian public sector procedures. Addressing this gap requires an ongoing training for performance monitoring. For example, the requirements for standardization of means of verification demand time to implement, but must be understood as necessary to establish accountability and for the project evaluation. T2.A1: The model in a changing national context The following analysis considers changes in the national and international context. These have implications for the TAP implementation since they include factors that fall outside the direct responsibility of the team, but which present risks and opportunities for the program. 2.1 Opportunities and risks posed by new international agreements on environmental topics from the COP204 The COP20, held in Lima in December 2014, provided a greater nationwide dissemination of the meaning and importance of climate change, which may imply a greater inflow of resources for adaptation and mitigation efforts. This is relevant for TAP to the extent that climate change will always be horizontal to all its projects. In addition, one of the TAP projects is directly linked to this topic. The COP20 showcased the importance of communities in forest management, as well as the mechanism of Reducing Emissions from Deforestation and Forest Degradation (REDD +), central topics in the performance of Forests TAP. In addition, there are the Nationally Appropriate Mitigations Actions (NAMAS) and the potential of products such as coffee and cacao, especially from organic markets, as well as the entry of new cooperation programs such as the one provided by Norway (estimated to be 4 COP is the Conference of the Parties of the United Nations Framework Convention on Climate Change (UNFCCC) 25 300 million euros.) In the case of Valuation, Forests, and ACC TAP, the COP20 increased awareness that supports the development of more projects related to forest conservation and valuation for carbon capture. It is important to highlight that TAP has been flexible about the structure of its objectives in terms of incorporating the ACC project, and also by considering the incorporation of requests from the Directorate General of MINAM during the AOP elaboration process. Since the COP20, but not necessarily as a result of the event, the results framework of the five TAP projects has been reviewed in response to MINAM requests. The results of the COP20 are still being interpreted, and dissemination of related environmental issues has declined since the event. This affects TAP performance as the environmental issues lose sway on the government political agenda. It is difficult to talk about consolidation in terms of national policies related to the climate change topics, since the COP20 has been carried out in the context of easing environmental legislation. Respondents claim that the COP20 did not mean a significant change on the national agenda, especially regarding TAP topics. For example, regarding SEIA TAP, Environmental Impact Assessments (EIA) have not been emphasized since the COP20. Another example is the website of COP20, which does not present relevant thematic conclusions about the event or the negotiations that took place. 2.2 Opportunities and risks presented by new modernization trends in the Peruvian public sector TAP has introduced a results-based management approach, resulting in a greater emphasis being placed on the planning process to define objectives, such as those established for the budgetary programs and the ‘Budget-for-Results’. In addition, respondents expect the new government procurement law to effect positive changes, for example, by increasing the ceiling for direct assignment service orders. While the State modernization process promises medium-term positive repercussions, in the short-term it may negatively affect TAP implementation. Some respondents point out that the public sector tends to interrupt its processes when appointed officials and administrations change. However, TAP is aligned with many of the pillars of the public administration modernization policy, such as strategic and operational planning, results-based budgeting, and budgetary programs. Another factor to consider is the lack of coordination in implementing policies at the level of regional governments, which is critical in the environmental sector and has an impact on the TAP. An example of this situation is the Amazon Region, where few resources are allocated towards the exploitation of the forest’s economic potential and where it is difficult to access conservation credits. This creates a perceptible, vacuum in terms of the decentralization of resources for environmental conservation and governance. 2.3 Critical factors of the USAID cooperation model in MINAM consolidation as a national authority and its links with other public administration sectors. As MINAM is in the process of institution strengthening, the TAP seeks to support its consolidation as the public environmental authority. In that sense, the strengthening of SENACE (by its Spanish acronym; National Service of Environmental Certification) by SEIA TAP provides some of the best evidence of the contribution of TAP to the consolidation process of the Ministry as an environmental authority over other sectors. This process has made it easier to coordinate with other ministries and levels of 26 government using appropriate management instruments. The creation of SENACE was achieved through an autonomous budget from MINAM, and therefore cannot be funded by TAP. The public budget allocation to MINAM in 2014 (New Soles 550.6 Millions) demonstrates the importance of MINAM six years after its creation in 2008 (New Soles 117.8 Millions in 2009). Because of the complexity of the administrative and operational systems, which are part of the public sector modernization, the TAP project investment creates difficulties in terms of the ability of the MINAM administration to respond effectively in the face of changes and immediate requirements. This means that TAP achievements largely depend on the achievements of MINAM, in the same way that the coordination of the Directorate General affects the internal coordination between TAP projects. MINAM is a consolidating institution in the process of organizing the new funding from international cooperation, including the TAP. The challenge, in this case, is linked to the commitment of the highest political levels and their coordination with other levels of government, as well as the need to strengthen the relationship between MINAM and other sectors and subnational governments. All of this should be reflected in appropriate guidance for the program coordinators. On the other hand, it is essential that TAP continues supporting MINAM to specifically define their priorities in a framework of autonomy and respect for their decisions. This will lead to better-informed decision-making starting from studies, meetings, and workshops. The summary of the main findings of Area 1, with their corresponding conclusions and recommendations related to the international cooperation context is displayed in Table 4. 27 Table 4: Findings, conclusions and recommendations on the USAID cooperation model FINDINGS BY AREAS CONCLUSIONS RECOMMENDATIONS Q1-According to the national and international context, what risks and opportunities exist with the MINAM TAP G2G cooperation model in the context of changes in the national agenda? 1.1 It allows for MINAM to apply funds directly to their priorities. Such cooperation is positive because it facilitates assistance for MINAM priorities; this advantage is not understood by all actors in the MINAM environment. Develop and disseminate a brief study of the G2G cooperation model that promotes the visibility of the advantages of the model compared to other international cooperation models. 1.2 There are other cooperation models considered more agile. This is a perception, as there is no study on the subject. Non-assessed agility of the cooperation models leads to perceptions that compromise the image of the TAP. 1.3 MINAM is encouraged to define its priorities. The COP20 has not resulted in profound changes at the national policy level. The Institutional Strategic Plan (PEI) defines the national policy priorities of the environmental sector; these are defined with an insufficient level of specificity to guide investment in the sector. TAP must continue to support MINAM to better define its priorities in a framework of autonomy and respect for its decisions, supporting this process with studies, meetings and training workshops. 1.4 Documentation on the selection of the G2G model is limited to ADS document 220. The lack of documentation on the choice of G2G model could cause difficulties, which worries the TAP administrative area. Develop a Program Description of the TAP to guide the management of the program (activities, reports, administration, etc.) Furthermore, a letter from USAID and MINAM should formalize the fact that TAP is implemented under the framework of the G2G model. 1.5 There is not a management document (Program Description) to guide activities, reports and program implementation. There is only the PM&E and implementation letters, which do not meet this need. The TAP coordination team recognizes the difficulty they have in implementing and managing the Program in the absence of a Program Description, which has implications for the responsibilities of the team involved in the TAP. 1.6 New laws and regulations for the Peruvian public sector have an impact on the TAP implementation ability. The implementation of new tools to modernize the Peruvian public sector management is a positive process that generates gaps in organizational administrative capabilities to invest in MINAM. Perform joint actions to develop MINAM administrative capacities with TAP support, which will strengthen the position of the overall program. Source: Own elaboration 28 AREA 2: Program Design Relevance The question that guides Area 2 is: How relevant is the TAP program design in terms of USAID trends and dynamics and the evolution of MINAM? The program was designed in a participatory manner to assist the Directorate General of MINAM in implementing its objectives as a national environmental authority. Therefore, the TAP seeks to support the achievement of the expected impacts of environmental public policies, as well as contributing to defining the objectives of the same. Currently, the results framework of TAP is being reformulated, particularly with regards to the Valuation, SEIA, and Forests projects. This review process was initiated in response to mounting evidence, recognized within TAP and the DG, that it is essential to make adjustments that address priorities for the coming years as well as the fact that these will not be met on time, nor with the initial objectives included in the results frameworks. T3.A2: Initial Design 3.1 Procedure for TAP design. In 2010, a consultant from SASE designed TAP with four projects aimed at providing technical assistance to three line directorates and one MINAM program. As explained in the background section, the current TAP structure was established in 2012 and is being improved at based on requests from the Directorates General. The design was based on the requirements of agreements signed between USAID and the Peruvian government. The parties agreed to changes through the issuance of the implementation letters. However, the design did include the flexibility needed to adapt to the evolution of MINAM over the implementation period. It is important to highlight that the program design did not include a situation analysis. Since the initiation took more than two years, the performance targets had to be adapted to a new program context. In this sense, the objectives set in 2012 were unrealistic, particularly two corresponding to the Valuation TAP project, "six million soles generated by the mechanisms and implemented economic instruments", and the Forest TAP, "250,000 hectares of affiliated community forests... have developed production and conservation activities with economic and biological sustainability criteria." The results framework design was made directly with the technical areas of MINAM Directorate General in a participatory manner with each area, but without project integration or a joint perspective. 3.2. TAP sustainability at the national, regional and local levels in the current sector context. According to the 2013 PM&E, the TAP objective is: "Contributing to sustainable use of natural resources by strengthening the Ministry of Environment as the governing entity of the National Environmental Management System, implementing activities in three Line Directorates, the Directorate General of Biodiversity (DGDB), the Directorate General of Valuation, Evaluation and Financing of the Natural Heritage (DGEVFPN), the Directorate General of Policies, Environmental Management Standards and Tools (DGPNIGA), and the National Program for Forest Conservation and Climate Change Mitigation (PNCBPMCC) of the Ministry of Environment."The TAP results framework was derived from the priorities of the Directorate General and, therefore, it sought to support the expected impacts of environmental public policies. Each project has its intermediate results defined, some of which are not appropriate for the time period given and the resources programed for the corresponding activities, 29 such as: mechanisms that are only recently being defined, with long-term implications, which must be implemented and generate millions in revenue (Valuation TAP), or 250,000 hectares of productive forests implemented (Forests TAP). By assuming the priorities and objectives of MINAM, TAP also assumes its administrative weaknesses and its ability to achieve these impacts. This is a topic being considered in the revision of the TAP results frameworks. As mentioned earlier, there is not much clarity in the definition of objectives to guide TAP investment, nor of the expected impacts of MINAM policies supported by TAP projects. The weaknesses that affect TAP planning are rooted in the difficulties MINAM has in consolidating its strategic planning, to the extent that indications of the beginning of the current national strategic planning process are only recently being given through the CEPLAN guidelines (Directive 001-2014). Currently, the TAP contributes to defining the objectives of MINAM Directorate General so the expected results of their projects are clearly established within a framework of respect for the autonomy of the environmental authority. This participation is particularly focused on helping TAP DG counterparts to define their priorities in the framework of improving their institutional sustainability. While the program addresses the issue of organizational and individual capacity strengthening in different areas, TAP does not incorporate a line of capacity building or training for the MINAM administrative area. However, TAP is generating unexpected positive results in this area, such as the design of processes, specifically for the OGA, which provides support functions for the entire ministry. On the other hand, one of the future challenges will be the generation of a strategy for scaling-up investment, with integrated actions that include larger contracts that strengthen strategic alliances with private and public organizations, and universities, among others, that possess the ability and potential to implement activities in TAP action areas, thus leveraging greater impact. Regarding the sustainability of the Valuation TAP project, the training courses on valuation of ecosystem services are provided to national and regional officials. This capacity development strategy targets actors from the entire stakeholder system, linking activities to universities that will have a vision for how to manage project knowledge to improve the system’s ability to replicate and scale up implementation. Meanwhile, to ensure the sustainability of SEIA TAP, the project is considering its connections with land management and looking to SENACE to integrate aspects of the management of hydrological and forest resources. To this end, SEIA TAP is supporting efforts to acquire information required regarding environmental performance as well as comprehensive and integrated perspectives on environmental impact assessments. TAP is working to articulate its projects. For example, the SEIA TAP should give a favorable opinion on the decisions made for DGEVFPN investments, for which it should work hand in hand with the Valuation TAP. 3.3 Alternatives to structuring a MINAM Capacity Strengthening Program with a results-based approach. To obtain better results, Interviews identified the desire of regional governments to establish inter￾agency working groups with different levels of government around a core group to determine the goals, indicators, and responsibilities for capacity strengthening, which is understood as essential for results￾based management. In this regard, the regional response highlights the importance of strengthening 30 regional authorities and their link to MINAM. TAP is also fomenting links between MINAM and the private sector in an effort to achieve greater stakeholder awareness and involvement in the State’s proposals. A strengthening program seeks to develop external and internal abilities, not only in their role as an environmental authority, but also in terms of the administrative management of resources. Externally, TAP has ongoing efforts to strengthen MINAM relations with other pubic sector entities, such as with MEF on the topic of SNIP, so that environmental issues are incorporated systematically. T4.A2: Performance measurement 4.1 Consistency of the TAP intermediate results with the results framework of the Development Goal 3 of USAID, and with the objectives of the environmental clauses of the Free Trade Agreement with the United States of America (FTA). The TAP M&E area has developed a Monitoring and Evaluation Plan (PM&E), approved in August 2013, to adjust to new recommendations regarding how conduct perform measurements of standard USAID indicators. Diagram 5 presents the Development Goal 3 (OD3) results framework and the contribution of the TAP projects to its intermediate results. While the revised PM&E sought to align indicators, it did consider the scope, time, and resources available, nor the real contribution of TAP to this process. This will affect the final evaluation of the program, when these results will be measured against programmed targets. Diagram 5: Contribution of TAP projects to OD3 of the USAID strategy 31 The 2009 trade agreement with the US contains environmental issues in Chapter 18. Negotiation analysts identified three sensitive issues in environmental matters: effective compliance of environmental laws, sovereignty to adopt and modify environmental standards, and mechanisms for resolving disputes. In this sense, TAP has been contributing to the implementation of the trade agreement and, as noted above, it directly addresses the most sensitive issues by strengthening MINAM as governing entity in environmental issues at a national level. 4.2 Consistency of the TAP expected intermediate results with MINAM plans (PESEM, PEI, National Plans, etc.). Table 5 presents the expected goals of intermediate results for 2016 TAP projects based on the August 2013 PM&E. These results are linked in a general way with MINAM planning documents. In terms of the planning documents, the results are consistent; despite the fact that the TAP expected results are dependent on major processes related to the General Directorates. As always, it is important to keep in mind that TAP priorities are defined with the DG and must meet goals linked to MINAM objectives. Table 5: 2016 goals planned for TAP projects Project 1: MINAM Capacity Strengthening as a scientific authority of the Convention on International Trade Agreement on Endangered Species of Wild Fauna and Flora - CITES IR1: MINAM has consolidated key functions such as CITES Scientific Authority by generating scientific information of wild flora and fauna in Amazon Regions*, in order to issue opinions that contribute to preventing exportation from prejudicing their conservation. Goals for 2016: 100% of prioritized flora and fauna species have information to support the issuance of opinions 15 scientific studies have been generated, published and used by the Administrative Authority (MINAG and PRODUCE) Project 2: Strengthening of Abilities in Biodiversity and Ecosystem Services Economic Valuation - Economic Valuation IR2: The Public sector has established mechanisms and types of Economic Valuation and Payment for National Heritage Ecosystem Services for 05 regions: Ucayali, Amazonas, Madre de Dios, San Martin and Junin. Goals for 2016: 05 Economic Valuation and Payment Mechanisms for Ecosystem Services designed, validated and implemented. 06 million soles generated by the economic mechanisms and instruments implemented. 32 Project 3: Environmental Management Strengthening in the framework of the National System of Environmental Impact Assessment, in order to contribute to the conservation of biological diversity - SEIA IR3: The National Environmental Impact Assessment System (SEIA) has been improved in its political and organizational aspects, as well as its technical and administrative instruments and procedures, preferably in geographic areas prioritized for the conservation of biological diversity or fragile ecosystems. Goals for 2016: The National Service for the Environmental Certification of Sustainable Investments (SENACE) has developed policies, procedures, instruments, technologies and capabilities for the appropriate management of detailed-EIA for five (05) priority regions. As the sector governing body, MINAM has developed instruments and abilities for the operation of the National Environmental Impact Assessment System (SEIA), which includes the Strategic Environmental Assessment (SEA) in five prioritized (05) sectors and five (05) regions. Project 4: Capacity Strengthening for the Management and Conservation of Forests - Forests IR4: The National Program for Forest Conservation and Climate Change Mitigation has improved its implementation of Amazon community forests interventions in the regions of Loreto, Ucayali, Madre de Dios, San Martin, Amazonas, Junin, Pasco, and Cusco. Goals for 2016: A forest management and conservation model is implemented. 250,000 hectares of community forests affiliated with PNC8-MCC of Amazonas, Junin and Pasco have developed production and conservation activities with economic and biological sustainability criteria. Source: USAID TAP / MINAM, PM&E, August 2013 Note: The expected intermediate results of the ACC TAP were not included because they were approved in 2014 The TAP results frameworks adopted the goals and objectives of the Directorate General, which correspond to MINAM planning documents. In this sense, the program results are directly linked to institutional priorities. The TAP products, however, are not relevant to the corresponding institutional objectives. In fact, they are often very small and compartmentalized when compared with the larger goals established for the program and ministry. It is important to highlight that the priorities identified in TAP management documents do not mention the priorities of the regional and local plans, which represent a bridge for the ministry to these subnational government levels. The current MINAM Multi-Annual Sector Strategic Plan (PESEM) is valid from 2013 to 2016 and is in the midst of being updated in response to the CEPLAN 001-2014 directive, which includes a prospective analysis and new strategic planning processes that will affect the design of the MINAM Institutional Strategic Plan (PEI) 2014-2016. The PEI biannual assessments show the evolution of the 62 proposed indicators (18 of overall strategic objective and 44 of specific objectives). The relevance of the 2010-2021 National Environmental Action Plan (PNAA), approved in October 2010, must also be considered. The plan includes strategic actions in seven areas: water, solid waste, air quality, climate change, biodiversity, mining and energy, and environmental governance. While all of the issues are important, the most relevant issues for TAP are biodiversity, climate change and environmental governance. TAP should negotiate what activities it will perform with the DG, and these should be incorporated into the plans linked to the TAP projects. In reality, respondents asserted that PNAA guidance was general 33 and did not provide detailed guidance regarding TAP investments. 4.3 Impact of products expected from MINAM TAP in terms of establishing or strengthening environmental governance. TAP products are broadly useful because they allow the strengthening of MINAM as the environmental authority, but they are too granular and unlinked to provide a comprehensive perspective on the projects mid-term results within the context of the results framework. A symptom of this problem is the multi-annual programming of intermediate results goals (tracking table.) Each year, the operational objectives (deliverables) are redefined at the AOP level to make them consistent with the time remaining to achieve the intermediate results targets, as well as to allocate the resources from USAID to MINAM. This means that the planning that commits resources is annual and the assessment made by M & E areas and the administrative area, therefore, is also annual. The planning is done by responding to the requests of the Peruvian public sector, but without a global vision of the progress required in subsequent years to achieve performance targets, which are required to obtain the project’s intermediate results. Therefore, the products are necessary, but not sufficient to achieve the TAP intermediate results, since regular reprogramming has broken their link to the original schedule required to meet project results. For example, regarding the Valuation TAP, the methodology, materials, and course studies serve to direct valuation proposals in the different development centers of Satipo, such as Somontonari or Betania; but they have not considered how long the mechanisms will be implemented and what resources they will generate, which form part of the expected intermediate results. 4.4 Consistency of the indicators proposed in the projects’ logical frameworks with TAP expected results and standard USAID indicators. It is first important to note that, in addition to the overall TAP institutional objective, the expected results are divided among the logical frameworks of the four projects, which are reflected in the August 2013 PM&E. In principle, the objectives of these projects should be aligned with the priorities of the DG and MINAM strategic plans, but not necessarily with USAID standard indicators. Table 6 describes the USAID standard indicators associated with each TAP. The following is an analysis of the consistency of each project’s performance indicators as documented in the PM&E: 1- CITES TAP: The strategic objective level indicator is linked to the species for which there are scientific studies supporting opinions. The intermediate expected results, such as procedures, people trained, scientific studies, and a virtual platform, are well aligned with the high level result. 2- Economic Valuation TAP: The strategic objective level is measured by the amount of revenue generated by the mechanisms and instruments implemented in five Amazonian regions, and the number of mechanisms established for economic valuation and payment for ecosystem services. The second indicator is not linked to implemented mechanisms, but rather general mechanisms, which is far removed from the intermediate results related to methodologies, training, studies, mechanisms, and awareness. The evaluation team was informed that the result framework is being updated and that this should lead to a new version of the logical framework. 3- SEIA TAP: The strategic objective of this project is measured by the percentage of approved EIA that are compliant with established standards, compared to a baseline. Intermediate results are linked to the number of formulated instruments and people trained, as well as the involvement of regional 34 governments. The top level of performance measurement is aligned with the expected deliverables. 4- Forests TAP: In the case of technical assistance related to Forests, it is assumed that the performance measurement at the strategic objective level is the number of hectares affiliated with the National Program for Forest Conservation and Climate Change Mitigation (PNCB-MCC) that are conserved using biological and economic sustainability criteria. The products that should support this process are linked to staff trained, management instruments, investment plans, a monitoring system, reports, and guidelines for developing Public Investment Programs (PIP), etc.; which in principle are ambitious and appropriate. However, the delay in achieving the objectives planned for the first few years requires a complete reformulation of the objectives set and a planning review. Therefore, the logical framework is aligned with the priorities of MINAM Directorate General more than the USAID priorities, as it is linked to the environmental policy of the Peruvian state and focused on the issues targeted by each project. The Economic Valuation and Forests TAP need a performance indicator review. Table 6: MINAM TAP: USAID Standard Indicators Used TAP Projects USAID-MINAM 4.8 4.8 ENVIRONMENT TAP: FORESTS 4.8-7 Amount of greenhouse gas emissions Amount (GHG), measured in metric tons of CO2e, reduced or trapped as a result of the US government assistance. 4.8.1 4.8.1. Natural Resources and Biodiversity TAP: FORESTS 4.8.1-26 Number of hectares of biological significant and / or natural resources under improved management, as a result of the US Government assistance. TAP: CITES, ECONOMIC VALUATION, FORESTS 4.8.1-27 Number of people trained in natural resources management and / or biodiversity conservation with the US Government support TAP: CITES, ECONOMIC VALUATION, FORESTS 4.8.1-28 Number of technical assistance days in natural resource management and / or biodiversity provided to interested counterparts and parties with the US Government support 4.8.2 4.8.2 Clean Productive Environment TAP: FORESTS 4.8.2-6 Training hours on climate change, conducted with the support of the US Government TAP: FORESTS 4.8.2-14 Number of institutions with greater ability to address issues related to climate change as a result of the US Government assistance. TAP: CITES, SEIA, FORESTS 4.8.2-28 Number of laws, policies, strategies, plans, agreements or regulations that address climate change (mitigation and adaptation) and / or biodiversity conservation officially proposed, adopted or implemented as a result of the US Government assistance. MINAM TAP GNDR1 Number of laws, policies or preliminary procedures, proposed or adopted to promote gender equality at local, regional and, national levels. Source: USAID TAP / MINAM. PM&E Note: the ACC TAP is not included as it was not in force at that time 35 T5.A2: Management of results framework changes 5.1 Level of satisfaction with the results framework among those involved in TAP projects and the repercussions of including a fifth project. Of the 63 interviews performed, this topic was included in 23 interviews for those directly involved in the project. Of these respondents, 43 percent expressed good and very good satisfaction levels. They explained that in many cases this satisfaction level does not refer to the approved results framework, but to the results framework that was under revision. These changes during the evaluation process make the importance of the interview results relative, but they do provide a general perspective of the level of satisfaction. A third of those directly involved (officials linked to Directorate General and TAP coordinators) had an ‘intermediate’ level of satisfaction, and one quarter was not satisfied, primarily regarding the Valuation and Forests projects. Responses demonstrated a good level of satisfaction in MINAM with the flexibility demonstrated in terms of the requested adjustments to better define national priorities and recent changes that were negotiated regarding the results framework. However, respondents did not known how these changes would be formalized internally. Meanwhile, TAP team members responded that they had learned to negotiate change management during program implementation. This process begins when past implementation is assessed and the new AOP is developed, for which new financing for activities is requested. In these cases, they consider the relevance of requested activities to TAP objectives and then they negotiate the allocation of the corresponding budget. As negotiations with the DG take place at the operational level, the TAP technical coordinators, along with the M&E and administrative staff, assume responsibility for negotiating changes in management documents (results and logical frameworks), as needed. The incorporation of ACC TAP project deals with a specific topic in MINAM’s overall policy, i.e the theme of SNIP funding, and uses the well-known mechanisms of guidelines and training. The project reinforces the link between the sector and MEF, and consequently with other sectors, which must estimate the impacts of ACC in their proposals. The inclusion of ACC TAP did not affect the other projects, since each technical coordinator works independently. 5.2 Inclusion of a gender approach in the expected results and the mechanisms of monitoring / evaluation. A review of TAP consultant terms of reference reveals that they included some gender focus. Of the eight TORs analyzed, the gender focus is reduced to registering training participants by gender, but in no case do the TORs address the role of women in the processes that were being analyzed. Thus, the performance measurement of training by gender is the only one properly addressed, but no progress was made in looking at the role of women in significant issues such as forest conservation or climate change adaptation and mitigation. 5.3 The orientation of MINAM TAP towards the improvement of environmental governance In 2010-2021 PNAA, the subject of environmental governance is mentioned among the imperative strategic activities, with the goal of one hundred percent of SNGA entities implementing the National Environmental Policy and its environmental management instruments. This framework includes 18 strategic actions, of which the following are the closest to TAP objectives: 36  Consolidate the National Environmental Management System-SNGA (which is being done by strengthening SENACE).  Implement Eco-efficiency in the public sector and promote it in the private sector (with guidelines for developing the PIP).  Consolidate the National Environmental Impact Assessment System (SEIA) and the effective implementation of its environmental management instruments (SEIA TAP).  Inventory, evaluate and value the natural resource heritage for its conservation, sustainable management, and articulation in national accounts (CITES and Valuation TAP).  Implement land-use planning on the basis of ecological and economic zoning in support of the sustainable use of natural resources and the organized occupation of the country (Forests TAP). Environmental management also generates opportunities for participation, but these are not central to the activities requested by the DG. The following strategic activities included under PNAA are well removed from the TAP scope:  Develop imperative environmental research programs and CITES.  Provide free and effective access to environmental information.  Strengthen the exercise of environmental enforcement and mechanisms of citizen participation in environmental monitoring at the national level (while not of central importance, activities were also held with the Agency for Environmental Assessment and Enforcement-OEFA).  Guide the student and civic education towards education for sustainable development. Taking these aspects into consideration, opportunities arise to expand TAP activities associated with the improvement of environmental governance. Those responsible for the activities, besides MINAM, would be the ministries of Housing, Construction and Sanitation, Health, Energy and Mines, Agriculture, Production, and Transport and Communications; the Directorate of Port Captaincy; the Autonomous Water Authority (ANA); OEFA; the National Superintendence of Sanitation Services (SUNASS), Health Provider Companies (EPS by its Spanish acronym), the National Meteorology and Hydrology Service of Peru (SENAMHI), regional governments, local governments and society as a whole. 5.4 Level of flexibility from USAID to redefine TAP objectives The Peruvian government defines TAP objectives, specifically the DG beneficiaries when developing the AOP. USAID documents TAP results towards in terms of its own performance measures, but does not interfere with national priorities. This relationship has guided the change of results frameworks for three of the five projects (Economic Valuation, SEIS, and Forests.) This shows that the TAP is flexible to changing needs, as changes have been effected at the level of activities (AOP) and in terms of modifying intermediate results (results framework), as well as to add a new project (ACC TAP.) During the evaluation process, changes were negotiated within the programs result frameworks to update the priorities according to the requirements of the DGs. These changes are reflected in implementation letter agreements, as well as lessons learned documented in monitoring and evaluation area reports. It is important to note that there is no document that formalizes this process. This topic should be addressed in the Program Description. Table 7 below summarizes the findings, conclusions and recommendations relating to Area 2. 37 Table 7: AREA 2: Findings, conclusions and recommendations on the Program's Initial Design FINDINGS BY AREAS CONCLUSIONS RECOMMENDATIONS Q2-What is the relevance of the program design based on USAID trends and dynamics and the evolution of MINAM? 2.1 TAP structure was designed and is being improved in a participatory manner based on DG requests. It is positive that changes are made to the objectives of MINAM TAP in a participatory manner, but these are not yet reflected in PM&E revisions. Continue with a participatory model and incorporate a clear changes management model in the Program, which must be defined in the Program Description. 2.2 The M&E area has developed a PM&E to be adapted to USAID Forward recommendations. The Peruvian public sector has not developed a clear model for a monitoring and evaluation system to guide TAP; but the TAP M&E specialist knows techniques for developing a PM&E according to the requirements of USAID Forward and the results-based management requirements of the Peruvian public sector. Use the technical ability of the M&E specialist on USAID Forward management instruments to update M&E tools when required. 2.3 TAP does not interfere with national priorities, but it pushes for clear priorities so that the project can clearly define its activities. It is positive that the TAP promotes the definition of national priorities within an autonomous framework of MINAM, because this has a positive impact on the definition of lines of investment for the sector. The TAP must continue supporting MINAM to better define its priorities in a framework of autonomy and respect for its decisions, supporting this process with studies, meetings and training workshops. 2.4 The strengthening of governance is prioritized. It is positive to prioritize the strengthening of MINAM governance, which is having more impact on its position related to other national sectors than on stimulating its relationship with sub-national governments. It must continue to prioritize the strengthening of MINAM governance, both in the context of other national sectors as well as of sub-national governments, with capacity strengthening programs on environmental matters among various actors, studies, and supporting consultation processes that conclude with specific recommendations for investment in the environmental sector. 2.5 The program start was delayed more than two years (2010-2011), thus the goals had to be adapted to a new context, where MINAM is being pressured to consolidate as the environmental authority. The contexts that determine MINAM priorities are changing constantly, but there is no adequate change management in the program, which results in outdated management instruments. TAP must update its management instruments to the extent that MINAM priorities are adjusted, and it must register these changes. These changes should be described in the program description. 38 FINDINGS BY AREAS CONCLUSIONS RECOMMENDATIONS 2.6 Changes are being processed in the results-based framework for 3 of the 5 TAP projects: Economic valuation and SEIA, along with Forest which is initiating its review. This will affect the TAP indicators. It is positive to perform a participatory review of the TAP objectives, but this will impact its M&E system, which will have to update its performance indicators. Clearly define which document will form the basis for the TAP final evaluation in order to facilitate an adequate monitoring system structure, with an updated PM&E. 2.7 There is a poor planning of the gender approach in PM&E, because there are no tools for its application. In the TORs of TAP consultants, the issue of promoting gender equality is specified only in a generic manner; trainees by gender are also registered, but not women’s role in the activities performed. In the design and promotion of activities, women’s participation in seminars and other workshop courses is identified; this does not mean that there is a gender perspective in the activities performed. There is a lack of gender perspective and training of those involved in this issue, which complicates the inclusion and monitoring of this perspective in products. Develop a plan for the introduction of a gender perspective in TAP and its projects, as well as a preparation of those responsible for what a gender approach means in the environmental arena, responding to the National Plan for equal opportunities. Source: Own elaboration 39 AREA 3: Program Implementation Management The question that guides Area 3 is: What strengths and weaknesses exist in the TAP implemention process? TAP has a Monitoring and Evaluation Plan developed and approved in August 2013, which incorporates the USAID guidance with regards to results management, including the detailed definition of indicators and the use of standard indicators. The previously discussed shortcomings in the implementation of the Peruvian public sector process and policies restricting allowable expenses with USAID funds are reflected in lower than anticipated budget execution. Between August 2012 and November 2014, expended funds represented 42 percent of the budget allocated in the AOP (not on the total obligated by USAID to the Program, which is performed on an incremental basis hat takes into account actual expenditures). TAP takes into account MINAM priorities, but does not register the agreements reached to support program planning, and does not have updated management documents. The absence of a Program Description, generates a lack of clarity regarding the roles of those involved. It is often the USAID technical officers that make the necessary arrangements to advance the implementation of planned activities. In addition, technical coordinators have an inadequate contractual model that does not ensure the completion of the functions required for each position. Currently, the results framework of three of the five projects of the program are being modified in order to better define objectives, to avoid further delays in achieving the expected results, and to increase budget execution. T6.A3: Planning Standards 6.1 Elaboration of Monitoring and Evaluation and Operating Plans. The TAP monitoring and evaluation area has developed an M & E Plan that follows the recommendations of the 527-0426 framework agreement, in which the commitment of the governments of Peru and the United States to conduct a comprehensive monitoring of the proposed activities is mentioned. For this purpose, TAP prepared performance indicator reference sheets (PIRS) that respond to activities proposed by the Peruvian government, and in a second stage were aligned with USAID standard indicators. Based on these concepts, TAP develops quarterly reports and an annual report regarding its progress. The AOP development follows the guidelines of the public sector for the DG. The final program AOP incorporates the activities identified in the project AOPs, plus the management activities. USAID approves the final AOP. The evaluation found that the projects agree on AOP changes based on DG requests and then coordinate their approval with the M&E area and the administrative area. Hence, there is no automatic mechanism that registers the synchronization of the various changes to the program. Such a mechanism would allow for an analysis of efficiency by comparing the achievement of the products and intermediate results with budget execution and the activity schedule. 6.2 Processes of implementation of operational plans and procurement plans required by the public administration and control system. During the two years of the program implementation, the administrative procedures of the Peruvian public sector and its control system have been followed. However, there are delays in recruitment and 40 administrative procedures due to various factors, from the lack of precision in the production of the terms of reference (TOR) to compliance issues with suppliers, which leads to incomplete hiring processes. On one hand, the technical coordinators feel they do not have administrative support within the Program, which would allow them to spend more time on direct technical assistance. In contrast, MINAM officials feel that some coordinators do not pay attention to the required administrative procedures, especially in the case of Forests TAP. All contracts are issued by the General Administration Office (OGA), which ensures compliance with the processes required by the Peruvian Hiring Law. The M&E area improved the design of the process that TAP carries out with OGA to contract services (see Diagram 6). The OGA, in turn, feels that this process allowed them to improve their implementation capacity. This TAP support to the OGA is an unexpected result that is highly appreciated at MINAM. Diagram 6: Services procurement process T7.A3: Communications and quality management 7.1 Distribution of responsibilities and mechanisms for communication and coordination for the development of program reports. Germane to this topic is the absence of a TAP Program Description, which should clearly define the roles of those involved and the responsibilities regarding the monitoring reports or the type of communication that should exist. Currently there is no formal regularity and record of meetings between TAP technical coordinators and their counterparts in the Directorate General. The coordination, supervision and participation of stakeholders are not registered and this could lead to unclear responsibilities. Additionally, TAP has no communication mechanisms, such as newsletters, web pages, among others, which affects the visibility of the Program. TAP team members expressed that visibility is achieved through 41 the DG and not directly as a program, which must be explicit in any official document or management instrument. The evaluation process coincided with the hiring of a communication specialist to improve the visibility of the Program, although respondents stated that it is not TAP that should be visible, but the work of MINAM with different stakeholders. Only some TAP technical coordinators are recognized by the DG for generating more fluid communication with their counterparts, as in the case of SEIA TAP. An important contribution to the communication process is the participation of USAID technical officers, who are recognized for facilitating communication between the environmental authorities and TAP. 7.2 Consulting call processes. The OGA uses Peruvian public sector solicitation announcement mechanisms provided through the website of the Government Procurement Supervisory Agency (OSCE.) TAP technical coordinators and MINAM officials also directly distribute these solicitations to recognized specialists. In many cases, there are no professionals specialized in environmental issues available in Peru, and consulting positions remain vacant. 7.3 Organization of the TAP planning / programming process, The TAP planning process is participatory and takes into account MINAM priorities, as defined with DG and PNCB. As previously discussed, agreements made in coordination meetings are not properly recorded and there are no documents explaining each step of the participatory process leading to agreements. Also, management documents are not updated and these functions are not defined in a Program Description. Therefore, there is no formal process for the supervision and tracking of the agreements reached during planning. Only at the end of the process, when the DG develops its requests, does the Planning Office incorporate the agreements reflected in the AOP. 7.4 Areas of coordination between the five program projects The TAP program works as the sum of its five component projects. The management unit, led by the executive Coordinator and the M&E, is insufficient to achieve the program’s integration. This deficit is related to the lack of appropriate management instruments to guide the work of the coordinator, including a Program Description, updated PM&E, and position descriptions for the technical coordinators specifying their technical assistance and management functions, among others. It is important to note that the role of sponsor5 within MINAM is not clear within the program. Having a MINAM sponsor (here understood as a closely engaged and influential member of MINAM) is an essential support for project management. The sponsor should serve as a liaison between the technical coordinators and the DG (including PNCB), especially when the project seeks agile solutions to implementation problems. The lack of close assistance from the promoter hinders program communication channels, the implementation of activities defined in the AOP, and the search for solutions to implementation obstacles. This may again be rooted in the weaknesses of the program management instruments. From November 2013, USAID supervision of the Program has consisted of a technical officer responsible for each project. Each technical officer’s team has the responsibility to improve internal communication, as well as that of TAP technical coordinators communication with representatives of the 5 According to the standards of the Project Management Institute (PMI), one of the roles of the "promoter" is to ensure the Insertion of a project within the host organization. 42 DG and the PNCB. However, technical coordinators still communicate with their counterparts independently. The physical location of TAP hinders effective communication and coordination with the Directorates General, the General Secretary and the OGA, since separated locations prevent a continuous and direct relationship between the TAP representatives and MINAM. This also hinders the coordination and closer monitoring of agreements, which are necessary to jointly perform activities according to the G2G guidelines. The physical distance of the office does not provide any compensatory advantages with regard the implementation of activities or communication. 7.5 Distribution of responsibilities in supervising the quality of the contracted services. Both DG and TAP technical coordinators must approve the quality of contracted services. They must reach a consensus in the event of any disagreement, but the opinion of the DG is ultimately more important. However, those responsible for product review often do not spend the time required by administrative procedures to review product quality, claiming that they have too heavy a workload to comply with this requirement. T8.A3: Cost Management 8.1 Convergence between the budget execution and disbursement processes. The ceiling amount for the program is twelve million dollars ($12´000,000) incorporated into the budget of MINAM under donations and transfers. For the period August 2012 to November 2014, the budget allocated to the program by the Peruvian public sector through the AOP exceeded thirteen million soles (S / .13'629,473) of which only five million seven hundred thousand (S /. 5'758,021) were expended through November 2014, i.e. just 42 percent of the allocated budget. As presented in Table 8, through November 2014, the projects achieved a low level of budget execution. The level of implementation of CITES TAP at 42 percent, Valuation TAP at 40 percent, SEIA at 36 percent and ACC TAP at 36 percent. The case of Forests TAP is an outlier, having only executed 18% of the total budget assigned between 2012 and 2014. It is important to note that in 2013 a reprograming of the budget made a reduction of 60 percent. The decrease was applied as follows: the SEIA TAP declined by 83 percent, the Valuation TAP by 72 percent, the Forests TAP by 66 percent and CITES TAP by 37 percent. This reprogramming affected the implementation of activities established in the AOP, based on the original budget. Between 2012 and 2014, TAP management expenses accounted for 44 percent of the expended budget and the projects themselves expended the balance of 56 percent. Expenditures for technical activities are low. Among the individual project expenses, internal coordination or operation expenses represent a significant portion. In the case of Forests TAP for 2014, these represented 58 percent of budget execution; a figure that exceeds the 33 percent of CITES TAP, the 29 percent of Valuation TAP and 36 percent of SEIA TAP. The percentage of interal coordination and operation expenses of Forests TAP is only surpassed by the ACC TAP (66 percent), a project that, unlike the others, began operations in 2014. 43 Table 8: TAP budget execution: from August 2012 to November 2014 Project 2012 2013 2014** 2012-2014** Budget Expended Amount % Budget Expended Amount % Budget Expended Amount % Budget Expended Amount % CITES TAP: 53,469 17,960 34 1,573,704 660,324 42 1,220,012 526,409 43 2,847,185 1,204,694 42 Valuation TAP 50,804 19,917 39 678,216 224,186 33 1,122,075 494,034 44 1,851,095 738,138 40 SEIA TAP: 48,430 524 1 366,960 279,900 76 1,375,678 372,104 27 1,791,068 652,529 36 Forests TAP: 93,203 7,084 8 733,128 115,933 16 1,339,746 268,249 20 2,166,077 391,267 18 ACC TAP 721,790 256,513 36 721,790 256,513 36 TAP Function and Expenses 549,094 359,336 65 1,474,968 1,258,395 85 2,228,196 897,147 40 4,252,258 2,514,878 59 Annual Implementation795,000 404,823 51 4,826,976 2,538,739 53 8,007,497 2,814,458 35 13,629,473 5,758,021 42 Source: USAID TAP/ MINAM. * Information based on 2012 budget execution matrix through November 2014. ** Amounts reported in November 2014. A significant number of sub-activities, tasks, and scheduled activities for 2014 projects have not been budgeted or have a low budget execution (which, in many cases, is none). This means that many activities will be postponed or prolonged into 2015. These represent 57 percent of the CITES TAP sub activities, 65 percent of the Valuation TAP sub activities, 80 percent of the SEIA TAP sub activities, 78 percent of the TAP Forests sub activities, and 56 percent of ACC sub-activities. There are different variables that hinder disbursement for the implementation of activities, such as the approval process for the incorporation of funds by the Planning and Budget Office (OPP). In practice, this results in a delayed activity implementation schedule beginning in April, which generates coordination challenges between OGA and TAP in the development of TORs, as well as in the supervision of consultants delivery of products, among others. 8.2 Reaction to the delays in implementation of planned activities and the program budget. There is not a defined strategy to manage delays in activity implementation and budget execution. TAP seeks to document lessons learned in its M&E reports and train the responsible parties. The main response to delays was to design procurement processes with the OGA and estimate completion times, for which each process and its bottlenecks was taken into account to arrive at realistic scheduling for the following process. T9.A3 Risks and Changes Management 9.1 Project Change Management The most relevant changes at the Program level are recorded in implementation letters, while at the activities level they are recorded in the AOP and its amendments. The main lessons learned are recorded in TAP monitoring reports. However, there is no document that identifies the roles and procedures to be carried out in response to a requested change, whether small, medium or large. In addition, there are no documents that show how the request a change to the AOP or the Program (e.g. 44 results framework). Such a documentation should include processes to be carried out to update the AOP and other management documents (PM&E). Again, a Program Description would facilitate these TAP management processes. Operational aspects of activities, specifically the definition and content of the TORs, are flexible within the scope of DG requirements. 9.2 Identification of Program risks and contingency plans. Currently, there is no program implementation risk management. The only identification of risks is implied through the assumptions of the logical framework, which are detailed in the initial TAP design and in the logical frameworks presented in the PM&E. Respondents identified the possibility of losing the opportunity to strengthen MINAM operations , its officers and its systems as an important external risk at a moment of great importance for environmental issues in Peru, which is recognized for its vulnerability as a mega-diverse country facing climate change. The main internal risks mentioned during interviews - reinforced by the monitoring reports - are linked to the way responsible parties manage falling short of expected results,, low budget execution, the disconnection between the technical and administrative functions, inefficient capacity strengthening and difficulty integrating the programs. T10.A3 Human Resource Management 10.1 Analysis of the TAP organizational chart regarding the positions needed to facilitate coordination and the TAP hierarchy The expanded organizational chart (see Annex VII) has all the positions needed to ensure coordination between the various organizations involved. In addition, it visualizes the coordination between USAID technical officers, TAP coordinators and MINAM general managers. USAID technical officers play an important role in generating synergies with other donor programs and in linking them to MINAM. On the other hand, the relationship between the TAP executive coordinator and the technical coordinators is horizontal, which is positive for communication, but which hinders his leadership in the absence of clear position descriptions and appropriate management tools. This is due, in part, to the fact that all coordinators were hired at the same time, and therefore, the Executive Coordinator did not participate in the selection process. The type of contract held by the technical coordinators is generating the primary obstacle to effective human resource management. The coordinators have external consultant contracts. As such, they are not responsible for the Program’s performance, even though are responsible for the technical assistance under each of the projects. 10.2 Program team member profiles, functions and coordination levels TAP has defined profiles for its professionals. In the case of the technical coordinators, the qualifications are technical, but their job functions are primarily directed to contract administration. That is to say, TAP is a program of technical assistance, but its coordinators manage contracts, leaving little time to provide direct technical assistance. This frustrates the technical coordinators. It is important to point out that the technical coordinators state that they have to sign documents in the contracting processes that have them assume responsibilities that do not correspond to their status as external consultants. This inconsistency in contracting creates confusion in coordination with MINAM staff. 45 10.3 Incentives for the best team performance. There are no incentives of any kind for program staff, mainly due to the inadequate external consultant contract used for the technical coordinators. This contributes to their feeling limited in the performance of their duties. In this conext, assuming responsibilities depends more on each individual’s personal characteristics and commitment than on the systematic incentives that should contribute to improving team performance. As a result, there are some coordinators that are proactive and committed in dealing with obstacles (especially administrative), while in other cases, responsibility for resolving the difficulties is shifted to other personnel or areas. Table 9 presents Area 3 findings, conclusions and recommendations. 46 Table 9. Area 3: Findings, Conclusions and Recommendations about the Planning Standards FINDINGS BY AREAS CONCLUSIONS RECOMMENDATIONS Q3-What strengths and weaknesses exist in the USAID/MINAM TAP implementation process? 3.1 M&E Plan has been developed following USAID recommendations. This introduces elements of results management, measuring baselines and it improves the indicator definitions. The recognition of the importance of baselines and Monitoring Plans by the DG (including CBPP) has led to the uptake of these instruments by some directorates, such as DGPNIGA. TAP use of M&E tools is positively influencing DG management and the Project Management Secretariat, as well as the TAP team in monitoring results and implementation. The team knows the techniques for developing a PM&E. Continue strengthening the use of M&E tools at the level of DGs and the Project Secretariat, and use technical knowledge of USAID Forward requirements to update M&E tools when required. 3.2 In the last year TAP benefited from political leadership and USAID accompaniment. The presence of USAID technical officers facilitates communication between MINAM and TAP. The technical officers are coordinators and managers of other USAID Portfolio projects. The presence of USAID technical officers facilitates communication between MINAM and TAP, which is helps overcome implementation problems. The participation of the USAID technical officers strengthens the relationship between USAID Portfolio Projects and MINAM. Strengthen USAID technical officers’ role to facilitate communication between the parties (MINAM and TAP). Promote greater involvement of TAP projects with USAID Portfolio Projects through USAID technical officers. 3.3 The TAP is absorbed within MINAM administrative and political system, according to its original design. It is good for the objectives of this type of strengthening program for TAP to be part of the MINAM administrative and political system. It creates a major challenge and opportunity for the Program in terms of influencing the complex bureaucratic processes. Encourage the Program to generate assessments and proposals in order to improve MINAM processes. 3.4 Progress has been made in inter-sectoral coordination with mechanisms like SENACE, but there are still weaknesses in the definition of plans and policies in the environmental sector, especially for the orientation of investment and links with other government sectors. The strengthening of MINAM governance is positive, but this is having more impact on its positioning with regard other national institutions than encouraging its link with subnational governments. Continue prioritizing the strengthening of MINAM governance, both in the context of other national sectors and subnational governments, with capacity development programs among various actors, studies, and supporting consensus-building processes that conclude with specific recommendations for investment in the environmental sector. 3.5 TAP technical coordinators perceive a lack of administrative support that would allow them to work more in their technical area. Gaps occur between the TAP technical coordinators qualifications and the requirements for public administration and management. Update the technical coordinator qualifications to include knowledge on public management or maintain a technical profile supporting enhanced direct technical assistance, but with substantially increased administrative support. 47 FINDINGS BY AREAS CONCLUSIONS RECOMMENDATIONS 3.6 Coordination meetings are not registered, nor are agreements documented in any formal or regularized manner The knowledge management is uncertain, as reflected in various areas, especially the lack of a format to monitor and facilitate communication between TAP, MINAM and USAID. Develop formats to record meetings minutes, workshops and general agreements that facilitate the monitoring of agreed upon actions. 3.7 In the period August 2012 to November 2014, the budget allocated to the program amounted to S/.13'629,473, of which S/. 5'758,021 (42%) was expended. The program expenditure level is low, but it may reflect the low implementation level of the entire public sector and MINAM especially. Extend the TAP implementation period and rethink the actual periods required for the contracting processes that produce the products that influence the expected results. 3.8 The projects have a low level of budget execution through November 2014, especially the Forests TAP with 18%. The Forests TAP did not achieve any of the expected results in the Monitoring Table. This raises questions about its relevance. If there are no achievements in 2015, MINAM/USAID should assess whether the resources allocated for the Forests TAP could be more useful in another MINAM area. 3.9 Until November 2014, 44% of expenses went to TAP management and operating expenses. The level of operating and management expenses is relatively high given that the project implementation is low. It is positive that there is support for common actions to strengthen the MINAN and TAP administrative capacity, however, these are being counted together with administrative expenses, which increases the share of this category. Reconsider how to account for activities that contribute to the strengthening of MINAM and TAP administrative processes. 3.10 The expanded organizational chart is effective. It is positive that the expanded organizational chart reflects the levels of coordination between the TAP, MINAM and USAID. However, lack of clarity of coordination in the PM&E continues to challenge TAP implementation. Establish a communications plan in the PM&E framework that specifies the type and frequency of coordination/communication required. 3.11 Given TAP characteristics, and considering the coordination model with MINAM, findings indicate communication problems challenged the joint definition of objectives and implementation strategies with the DG (including PNCB) and OGA. There are communication channels between the parties, but they are only used to resolve ad hoc issues or problems, which raises the question of whether the program coordinator has the right tools to exercise leadership. 48 FINDINGS BY AREAS CONCLUSIONS RECOMMENDATIONS 3.12 Due to the absence of a centralized program, rather five projects, initiatives are taken individually in order to overcome the obstacles. The coordination difficulties between MINAM DGss are reflected in coordination problems between TAP projects, which share common topics that require an integral perspective. Integrate TAP projects using common activities that strengthen MINAM governance both with regards to other national institutions and subnational governments. Undertake common actions to strengthen MINAM administrative capacities, which will strengthen the vision of the program as a whole and not the sum of its parts. 3.13 The sponsor role is not clear. This leads to limitations in agreeing on implementation problems and obtaining political support to generate solutions. Again, the lack of a Program Description fails to deliver clarity on the roles of the people involved. The sponsor figure is essential for TAP as a link between technical coordinators and the DGs (including PNCB), especially when they must find flexible solutions for implementation problems. Include the sponsor role in the Program Description in the key role of supporting problem solving for TAP implementation, from the AOP production, the monitoring and evaluation of projects and joint actions, and facilitating communication channels between the TAP and DG technical coordinators. 3.14 The relationship between the TAP Executive Coordinator and technical coordinators is horizontal. This is positive because it creates a democratic relationship between the coordinators, but it also hinders leadership, as the Executive Coordinator did not participate in the selection of technical coordinators. It has been questioned if the TAP executive coordination unit has the right tools to exercise its leadership. Review the technical coordinators job descriptions, the decision on the types of contract and possible incentives for them, under the leadership of Executive Coordinator. 3.15 There is no assessment of the impact of the external consultant-type contract on the responsibilities of the technical coordinator to support their prompt reformulation based on their actual responsibilities. 3.16 The roles of the technical coordinators are mainly linked to contract administration, although their qualifications are more technical and TAP is a technical assistance Program. Gaps occur between the TAP technical coordinators qualifications and the positions’ demand for knowledge on public administration and management. Update the technical coordinators to include technical knowledge on public management; or choose a technical profile to deliver enhanced technical assistance, but with increased administrative support for operational functions. Source: Own elaboration. 49 AREA 4: TAP Effectiveness and Monitoring This section attempts to answer the question: How effective is TAP at this intermediate stage, and what critical factors need to be addressed to improve its effectiveness? TAP professionals have elaborated documentation providing evidence of TAP performance and results. This documentation establishes the progress made by the different projects, as in the case of quarterly and annual performance reports, and allows a comparison between these results and the current results framework, the AOP of each project, and the current monitoring table. Based on these tools, we observe progress in the SEIA and CITES projects, as well as minor achievements in the Valuation and Forest projects, which should be a cause for reflection vis-à-vis project targets. Previous sections have explored the link between products and expected results. The prioritization of monitoring systems at an operational level has also been described, i.e. they are tightly linked to the AOP by the administrative processes required by the Peruvian public sector. The monitoring of the multi-year programming through the monitoring table was undertaken in response to a USAID requirement. In order to answer this question in more detail, Area 4 discusses two evaluation topics: performance results and lessons learned. T11.A4: Achievement of Results 11.1 Contribution of products achieved during the first stage of implementation to achieve the TAP expected results. While respondents indicate that DGs, except the PNCB, are satisfied with project results, officials are only recently understanding the potential of having additional resources directly in their budget. Regarding the contribution of the products to achieving expected results, respondents recognize that products have been completed, but that they are very fragmented. (See item 4.4). In general terms, there are inefficiencies in the administration and documentation of completed activities. Table 10 summarizes the products completed by the CITES TAP according to the 2013 and 2014 AOP. 50 Table 10: Achievements of the activities proposed in the TAP CITES: 2013 and October 2014 2013 October 2014 Proposed Achieved Proposed Achieved Objective 1 5 Management Documents. 2 Trainings. 5 Guidelines. 1 CITES project baseline. 3 Management Documents (Proposed regulation, Incentive Plan and CITES focal point designation). 1 Training Design. 1 Training. 4 Methodological guidelines (orchids, anatomical and Dendrological of cedrela and crocodilia). 1 Forestry Technical Guideline. 1 Monitoring of CITES management instruments. 1 Update of flora and fauna species CITES. 1 Publication of CITES management tools. 1 Advisory report to CITES authority. 2 CITES Technical Committee Reports. 5 Events (workshops, trainings). 2 M&E Guidelines. 2 Publications (manual and guideline). I CITES project baseline. 1 Forestry Technical Guideline. 1 Update of flora and fauna species (nomenclature). 2 Events (Workshop updating threatened flora; Meeting of contributions to the orchid’s guideline). 2 Evaluation guidelines: (birds and alligators). 1 Presentation of studies of orchids, mahogany and cedar. Objective 2 3 Assessments of flora and fauna species. 8 Situational assessments. 1 Virtual Platform 1 Evaluation of species existence. 2 Diagnostics (Orchids and birds). 4 Scientific studies. 4 Situational assessments. 1 Publication of assessment. 30 percent implementation of the virtual platform. 4 Studies (arapaima genre, orchids, mahogany and cedar). 2 Situational assessments. Source: MINAM TAP / USAID. Own elaboration. The contribution of the SEIA TAP is apparent in the SENACE consolidation and in the approval procedures for environmental impact studies in the agriculture, industry, fishery, and construction sectors, among others. Table 11 presents the results and the proposed activities based on the 2013 and 2014 AOP. 51 Table 11: Achievements of the activities proposed in the TAP SEIA AOP: 2013 and October 2014 2013 October 2014 Proposed Achieved Proposed Achieved Objective 1 7 Assessments. 1 Strategic Plan Proposal 1 Proposal for PIP. 1 Platform Proposal. 1 Single Window Proposal. 9 Workshops. 5 Monitoring Reports. 4 Courses. 2 Guidelines. 2 Systematizations. 5 Assessments. 1 Monitoring report template. 1 Baseline criteria proposal. 1 Manual. 1 PIP characterization proposal 1 Single Window Proposal. 1 PE Proposal. 1 Workshop. 1 Platform Proposal. 1 Assessment of Workshop supply and demand. Objective 2 1 Program Proposal. I Budget Proposal. 5 Reports. 4000 Print Publication. 1 Connection Proposal 1 Process Proposal. 1 Monitoring proposal. 1 IT platform. 2 Workshops. 1 National Event. 3 Reports. 1 National Event. 1 SEIA Program Workshop I Budget Proposal. 1 SEIA Report. 1 Monitoring Proposal. 1 Computer platform. 4000 Print Publication. 1 Guideline. 2 Normative Proposals. 2 Certification Records. 2 Trainings. 1 National Event. 1 Baseline Study. 1 Baseline Study. 1 National Event 2 Certification Records. 2 Normative Proposals: (legal and process) 1 Update of IT module. 1 Monitoring Plan. 1 SEIA Report. 1 SEIA Program Workshop. Objective 3 4 Technical Assistance activities. 2 Workshops. 2 PCs. 4 Macro regional courses. 2 Technical Assistance activities 2 Trainings. 2 Systematizations. 2 PCs. 1 Technical Assistance activity 1 Training Proposal. Source: MINAM TAP/USAID. Own elaboration. While the achievements of the Valuation TAP are limited, its recent performance includes the development of a validated methodology for economic valuation courses in Satipo and Pucallpa. Also, it made progress in facilitating flora and fauna population studies. Project capacity development involves regional governments and stakeholders, such as universities, civil society organizations and private companies. However, respondents note that this training is still at an early stage, and it remains necessary to develop the connection between the capacities the course will generate and the eventual empowerment of local population (Table 12). 52 Table 12: Achievements of the activities proposed in the Valuation TAP AOP: 2013 and October 2014 2013 201 4 Proposed Achieve d Proposed Achieved Objective 1 1 Scientific Inventory. 5 Valuation cases (flora, fauna, water, soil and air). 3 Strengthened Regions: Valuation Courses. 1 Guideline of Economic Valuation 3 Reports on ssistance in VE. 2 Interns attend international courses. 3 PSE Assistance Reports 2 Guidelines for Flora Inventory 3 Guidelines for Fauna inventory. 1 Analysis of important factors for the predictive model. 5 Valuation Cases. 1 Course of economic valuation 1 Scientific Inventory. 3 Strengthened Regions: Courses of valuation and PSE, Studies of economic valuation, Courses for specialists. 2 Hydrological valuation studies (Amazon and Junin). 2 Hydrological valuation studies (Amazon and Junin). 4 Curricula. 1 Support theme for the valuation course. Objective 2 1 Instrument guidelines. 1 Economic instrument. 1 Evaluation of instrument efficiency. 1 Synthesis of business experiences for the design of economic instruments. 1 Economic Instrument. Objective 3 1 Systematized Guideline for PIP. 1 Methodology for PIP formulation. I Guideline for PIP methodology. Objective 4 5 Valuation Studies. 3 Studies in PSE. 1 Supporting theme for the Valuation Course 1 Supporting theme for the Valuation Course. Source: MINAM TAP/USAID. Own elaboration. As for the Forests TAP, there was support for PNCB in the organization of a national event on political and forestry public management in Santa Maria de Nieva, where the Specialized Environmental Prosecution Service (FEMA) and the local native communities met. The project also supported training workshops on forest plantations in the central rainforest in coordination with the respective local offices. The Forests TAP has also collaborated with the Biodiversity and Companies Initiative, where resopndents recognized positive results. These achievements, however, did not represent significant progress in relation to established objectives and targets. There is no assessment of the compliance, in quantity or quality, of the products contracted through third parties to reach project targets. Monitoring reports only mention whether the projectus were completed. There is also no monitoring or analysis of the quality of expenditures in terms of efficiency or the unit costs of activities. In particular, it is not clear what level of contribution is expected from Forests TAP activities to the results or outcomes for 2016, especially in the terms of the number of hectares. There are also documents, consultancies and products developed or requested by the Forests TAP that do not meet the precise requirements established in the TORs. In this regard, the people involved have not thought about how to ensure that such problems do not recur in the future. Finally, there is a critical delay in the purchase of a server to process satellite imagery for PNCB that has not been overcome, and which affects the image of TAP. Table 13 presents the gaps between the programed results and the achievements up to October 2014. 53 Table 13. MINAM TAP: Achievements of the Proposed Activities in the Forests TAP AOP from October 2013 to October 2014 201 3 2014 Proposed Achieved Proposed Achieve Objective d 1 11 Assessments. 6 Workshops. 18 Trainings. 1 National Event. 15 Investment Plan Reports. 3 Investment Plan Reports 1 National Event. 1 Impact Evaluation 1 Monitoring System. 100 Trained Officials. 35 trained Communities 3 Plans: Forest Conservation, Investment, and Technical Assistance. 2 Consultancies: Legal assessments of investment plans. 1 Monitoring Plan. 3 Technical Assistance activities. 1 Conservation Proposal. 2 Curriculum designs. Objective 2 3 Workshops. 3 Monitoring systems. 3 Packs of Satellite images. 3 Packs of technological equipment. 3 Coverage maps 6 Trainings. 1 PIP Guidelines for forest conservation. 30 Trained Officials. I Budget Proposal. 1 Financial Mechanism. 50 Trained entrepreneurs. 89 Trained Officials. 13 Trained entrepreneurs. Objective 3 8 Workshops. 1 Budget Proposal. 1 Financial Mechanisms. 1 PIP Forests Guideline. 3 Trainings. Source: MINAM TAP/USAID. Own elaboration. As for the ACC TAP, which joined TAP in 2014, as of yet there is no analysis of the vulnerability of the Junin region to water and sewage in the face of climate change, as there is still no forecast for climate change. According to interviewees, this requires an appropriate scale to establish the relative probability of different environmental risks. In relation to the SNIP regarding water and sewage topics, the delays are attributed to the fact that it took eight months to sign new formats, and therefore the consultant's contract was not available in September 2014. Table 14 presents planned activities and results according to the 2014 POA. Table 14: Achievements of the activities proposed in the ACC TAP: October 2014 October 2014 Proposed Achieved Objective 1 6 Workshops (210 people). 1 Guideline. 1 Technical Proposal. Objective 2 9 Studies. 4 Courses (200 people). 5 Studies. 4 Courses (172 people). Source: MINAM TAP/USAID. Own elaboration. For those interviewed, the project had not yet delivered products that would generate substantial impact. As part of its monitoring and evaluation function, TAP has not completed updating the 54 documention that will serve as the basis for the final evaluation of the program; as the results framework (and baselines) are in the process of being updated. Changes in the results frameworks are positive but hinder the evaluation process that should have a clear and validated foundation to use as a reference. The evaluation process needs a document that would mitigate the risks of the changing the results frameworks. 11.2 Achievement of PM&E performance indicator targets: TAP successes and failures As shown in Table 15, the achievement of expected results in terms of the indicators proposed in the monitoring table is slightly higher in the case of CITES, as compared with the other projects.. Table 15: Achievement of PM&E performance indicator targets: September 2014 Projects Number of proposed indicator targets Number of targets achieved ACC 7 2 Forests 10 1 Valuation 7 2 CITES 6 4 SEIA 10 4 Source: MINAM TAP/USAID. Own elaboration. The development of methodological guidelines for the prioritized species under CITES is positive. Under the Valuation TAP, the valuation courses were well structured, continuous and consistent with DGEVFPN priorities. Also, the final coursework met the quality needed to contribute to achieving the intermediate results targeted in the Project results framework in relation to the training. TheValuation TAP Project results framework reformulation process, is progressing well, having incorporated realistic objectives and appropriate indicators for the type of assistance provided by the TAP to the DGEVFPN. The valuation of the Catarata de San Andrés has been an important positive advance, and the courses held in Junin and Pucallpa have benefited from participants from a variety of regional and local organizations and disciplines. As a case study, the evaluation included a more detailed analysis of the economic valuation training performed in Pucallpa, which resulted in a very interesting experience. According evaluation survey responses, participants perceive that they are able to replicate the knowledge gained in the course, although it remains necessary to develop the connection between the new information and capabilities and the empowerment of the local population. The activity included the identification of gaps in the prior knowledge of students about the topics covered in the courses, particularly statistics and economics. Respondents perceive the need to continue the capacity strengthening of participants with practical activities that will consolidate their knowledge, such as certification programs or the review and practice of case studies. Some course participants expected to become experts in the field, which was not the case given the complexity of the information. It is good to have generated a awareness of the knowledge gaps regarding the appropriate impact valuation methodologies to use in negotiations with big oil companies in the area, such as REPSOL, Lot 57; PLUSPETROL, Lot 108, and PERUPETROL, Lot 157. Respondents identified difficulties with workshop logistics with regards the transfer of teachers from Lima to the city of Satipo. 55 There are also opportunities in the Amazon that have not been considered in the project, such as the Condorcanqui Virgin Forests. In the central rainforest, there is the case of the Valle de los Rios Apurimac, Ene and Mantaro (VRAEM), which are challenged with the chemical pollution of water and soil, as well as the conversion of forest to coca cultivation. The products that developed as part of Valuation TAP Project don't show a clear causal relationship with the goal of generating six million soles through payment schemes for environmental services, which is needed to attribute these results to the activities developed by the project. The achievements of SEIA TAP have exceeded those of other projects. The project has also been more efficient in the sense that progress has relied on only four professionals. The project succeeded in developing the standards for the four most important sectors. The ACC TAP identified problems in developing standards for the water and waste management project, which has been delayed eight months and is still waiting for TAP and MINAM internal management to complete the process. As for the Forests TAP, there was no significant progress with respect to the achievement of performance targets set, according to the different tables and evaluation elements provided by MINAM and TAP. The project has not achieved the proposed targets for the different performance indicators associated with the Forests TAP. In most cases, performance is well behind scheduled levels, and in some cases there is no progress (0%). The evaluation was unable to identify any objective responses explaining the delays in achieving proposed objectives. Progress was made in ordering zoning maps in coordination with PNCB; however, the project could not acquire the server required to uses these maps because the OGA had issues with the corresponding TORs. T12 A4: Quality of indicators and lessons learned 12.1 The PIRS according to USAID Forward guidance: Quality of indicators As mentioned before, in August 2013, the TAP M&E area developed its M&E Plan following USAID recommendations. This included the introduction of new results-based management elements and the development of technical specifications for the performance indicators (Performance Indicators Reference Sheets, PIRS), which include indicator quality assessments. This document needs to be updated to address the introduction of the new ACC TAP project, among other reasons. Although there is no document that describes the requirements for program management instruments (i.e. Program Description) tools and administration processes, TAP has produced the technical progress reports required for both the Peruvian public sector and for USAID. The M&E system is based mainly on the AOP without a multi-year life of program plan that includes targets for products and outcomes associated with financial planning. Overall, the breakdown of activities managed by the technical and M&E team did not match all of the activities included in information from the financial area. Also, the process of updating the results frameworks of the TAP Valuation, SEIA and Forests will require a review of the performance monitoring system. This will generate a discontinuity in program performance measurement due to changes in the results frameworks. 56 12.2 TAP monitoring and evaluation contribution to the state of art and program progress The TAP M&E area created a design of the OGA administrative processes that facilitated the improvement of TAP implementation, an achievement that MINAM recognizes as an important program contribution. This unexpected result is valued highly by other MINAM areas that now better understand administrative processes and can use this product to reduce the negative impact of high staff turnover by reducing the time required to orient new staff to MINAM administrative processes. Also, the M&E team has completed a system for assigning value to each AOP activity milestone, which could become an important tool for measuring the project’s actual progress implementing its plans. 12.3 Systematization of the information produced by TAP, dissemination of results among key stakeholders and the dissemination of lessons learned Without a Program Description, there is no formal description of program management tools and there is no document that specifies how the work managed by the technical coordinators should be systematized or documented in terms of lessons learned. Project reporting included monthly reports that specify the difficulties that the coordinators had in carrying out their activities. However, these difficulties are not systematized in a compilation document and they do not specify corrective actions that should be implemented to overcome obstacles. There is no communication plan that includes dissemination strategies for successes or for the publication of products that would provide relevant information to TAP internal and external actors. 12.4 Lessons learned in terms of the development of the program objectives Although lessons learned are reported in monitoring reports, there is no organized process for systematizing these lessons. As a result, actors are continuously adapting, with little organization, to constant changes in the development of the program. The evaluation did not find any documents or other medium that comment, explain or convey the benefits of the program and its impact on the target beneficiary or other actors. One can also point out some lessons learned vis-a-vis program management tools. First, interviewees highlighted that program management documents (Program Description, results framework, logical framework, baseline, PM&E, monitoring reports, etc.) are out of date with regards to an adequate change and risk management for the project. Respondents also indicate that the program has no communication management and internal knowledge management tools, which should be developed. Finally, monitoring reports identify the need for flexible mechanisms to facilitate relationships between technical coordinators, the MINAM administrative area and line directorates, including the PCNB. These would help to identify best practices and disseminate them. Table 16 summarizes Area 4 findings, conclusions and recommendations. 57 Table 16. Area 4: Achievement of Results: Findings, conclusions and recommendations FINDINGS BY AREAS CONCLUSIONS RECOMMENDATIONS Q4: What is the effectiveness of the TAP at its mid-point, and what critical factors need to be addressed to improve performance? 4.1 Valuation: It is positive to have developed the valuation courses and to have progressed in population studies of flora and fauna. This capacity development involves regional governments and their stakeholders. Initiating an implementation methodology for a economic valuation course with regional actors that supports environmental governance led by MINAM is an important TAP achievement, but this is a long process. As presented in the Law on Compensation Mechanisms for Ecosystem Services, this process has six steps, of which two are complete. Extend the TAP implementation period and consider realistic periods required for contracting processes for products that are relevant to the expected results. 4.2 Valuation: It is necessary to develop a connection between the capacities generated through training and the empowerment of local populations. This is still at an early stage. 4.3 Forests: Partial support to the organization of a national event in politics and public forestry management. The Forests TAP did not achieve the expected results established in the TAP monitoring table. If there are no achievements in 2015, TAP/MINAM/USAID should assess whether the resources allocated for the Forests TAP could be more useful in other MINAM areas. 4.4 Forests: There is no significant progress with respect to the achievement of set targets. There is a major problem with the purchase of a server that affected the reputation of USAID 4.5 TAP. The products are very fragmented to have a clear causal effect at the results level. This requires an increased effort in the administration of contracts. The fragmentation of administrative procedures is negative; it diffuses the efforts of the technical coordinators and damages the TAP image with the DGs. Integrate the contracting processes in order to optimize administrative efforts and facilitate the supervision of products and the monitoring of results. 4.6 There are inefficiencies in the administration and documentation of completed activities in each project. The technical coordinators, the administrative managers and TAP monitoring use versions of the AOP whose codification does not match, which complicates M&E. These add to the difficulties of an outdated PM&E and the lack of a multi-year plan and budget. Design administrative processes and standardize the AOP codification, disseminating updates between all members in order to standardize criteria. 58 FINDINGS BY AREAS CONCLUSIONS RECOMMENDATIONS 4.7 The breakdown of activities managed by the technical and M&E areas does not match that of the financial area. The M&E system is essentially based on the AOP without a multi-year programming of investment, with products and results goals not associated to a financial plan. It is especially problematic that the breakdown of activities managed by the M&E technical part does not coincide with the financial area, and that there is not a multi-year programming of the investment to be developed with the TAP. Establish a multi-year programming for TAP resources with a breakdown of targets for products and results (monitoring table) and financial goals (costing and budget). This should seek to establish monitoring according to the technique of earned value and allow for an estimated time for a TAP extension period. 4.8 TAP designed an administrative process to facilitate the improvement of TAP implementation. This was an unexpected result highly valued by other MINAM areas. It is positive to have supported OGA in the design of improved processes that contribute to the capacity strengthening objectives of G2G cooperation. Perform similar capacity development activities for MINAM administrative functions, which will strengthen the perspective of TAP as a whole. 4.9 There is some consistency between the TAP indicators and some USAID standard indicators. The M&E Plan indicators are built to be consistent with the results frameworks, which reflect DG requirements (and PNCB). It is positive to build an M&E system based on MINAM priorities that also provides USAID reports that include standardized indicators. Use the technical capacity of M&E area, in terms of complying with USAID Forward, to upgrade M&E tools when required. 4.10 The updating of the results frameworks will require a review of the indicator system. It is positive to make changes in MINAM TAP objectives in a participatory manner, and is the case of the PM&E reviews 4.11 The change in the results frameworks will generate discontinuity in measuring program performance. Changes in the results frameworks are positive, but they will hinder the evaluation process, which should have an clearly defined basis for measuring performance. 4.12 There is no document that clearly establishes the way the program performance indicators were planned. The TAP coordination team has difficulty in implementing its system of monitoring and evaluation with the absence of a Program Description, which has consequences on the responsibilities of the team. 4.13 Some DGs have a political role that depends on the political status, and these seek to establish their area as a ruling authority, which causes problems in the definition of priorities in the medium term and with TAP coordination. The contexts that determine MINAM priorities are constantly changing, but the program does not have a proper change management processes, which results in outdated management tools. TAP should continue supporting MINAM in better defining its priorities, and should update its management tools when needed. 59 IV. CONCLUSIONS AND RECOMMENDATIONS It is an important achievement for TAP to have introduced international cooperation resources to the Peruvian public sector budget. These can be aimed directly at strengthening the capacity of MINAM as the SNGA governing body in Peru. However, delays in the initial implementation process and a lack of understanding regarding how a program of this type functions has precipitated a need to rethink MINAM needs and adapt to the MINAM consolidation process. As a result, TAP objectives will evolve over the next few years, and its stakeholders should seek concrete results that reflect this process. The main conclusions and recommendations divided into 9 items are found below: 1. Relevance of objectives: The priorities of the national policies for the environment sector are generically presented in various documents (National Environmental Action Plan, Environmental Agenda, among others), a situation that requires a disaggregated analysis to guide the TAP investment. Moreover, it is important to note that some General Directorates and the PNCB have a political role and look to position their area as a ruling authority over select priorities, which leaves TAP needing to coordinate the definition of medium term priorities. Recommendation: TAP should continue supporting the Ministry of Environment to define its priorities at a more specific level, within a framework of autonomy and respect state decisions. TAP should support this process with studies, meetings and training workshops. 2. Alignment of objectives to realistic activity timelines to improve implementation: The implementation level of the program, both financial and obtaining results, is a weakness, while the announcement of changes to the results frameworks is a positive reaction. In the context of unrealistic objectives, changes in the results frameworks are positive, but they pose a challenge to the final evaluation process, which should have a clear and validated basis from which to evaluate the program. In particular, the Forests TAP did not achieve any of its expected results, raising questions about its relevance. On the other hand, the start of a course implementation methodology about economic valuation with regional actors that help the environmental governance led by MINAM has represented an important TAP achievement; however this is a long process of which only two out of the six steps identified in legislation on the subject have been undertaken. Recommendation: The TAP implementation period should be extended in order to achieve the proposed objectives for achieving the sustainable use of natural resources, and thus fulfill the initial commitments included in the FTA, and strengthen the capacity of the environmental sector. It is important that there is the flexibility to propose a new result framework, and the need to do so is evidence that the cost of implementing a program of this nature is very complex. It virtually involves the ending of an entire program and the start a new one, which would require a complete fiscal period. In the short term, the parties can negotiate a change in the results framework or the inclusion of a new project (as was the case of ACC TAP). To succeed, TAP must reconsider the actual time periods needed for the recruitment processes that affect on the delivery of products and thus the achievement of results. 60 3. Management Tools: The participatory change of TAP specific objectives is a positive development. This involves assuming the challenge of upgrading the monitoring and evaluation system and addressing the absence of the Program Description. This should be a management document describing the responsibilities of the main actors in the TAP implementation and establishing accountability. Its absence is a weakness recognized by most of the program actors. The TAP coordination team has had difficulty in the implementation of its monitoring and evaluation system, which is negative and has consequences on the responsibilities of the involved team. Recommendations: It is important to develop a TAP Program Description in the short-term that incorporates select USAID best practices for the organization of its programs at the national and international level, i.e. those that can guide program management (activities, reports, administration, among others). Also, the Program Description must specify clear processes for managing changes to the program. Furthermore, TAP should regularize the 2G2 model framework through a letter between USAID and MINAM. To support this process, the TAP M&E technical capacity with regards USAID Forward management tools can be applied to update these tools when required, and expand their use at the level of DGs and the Project Secretariat. This will contribute to the structuring of a proper monitoring system with an updated PM&E that clearly defines which document will be used as the basis for performing the TAP final evaluation. 4. Actions common to the five projects: One of the most important conclusions is linked to the problems caused by the weakness of systematic actions common to all of the projects linked to the administrative field. The fact that TAP is housed within the MINAM administrative and political system is consistent with its objectives as a G2G program, but it requires an adjustment to the Peruvian public sector, which includes long administrative processes and complex procedures. The implementation of new modernization tools in the Peruvian public sector represents a positive development, but, in the short term, it exacerbates gaps in MINAM organizational administrative capacities to carry out its investments. For TAP, there are serious gaps between the technical qualifications required of the TAP technical coordinators and the on-the-job need for public administration and management skills. Also, the level of operating and management expenses is relatively high compared to the low level of project implementation. It is positive that there is TAP support for common actions that strengthen MINAN and TAP administrative capacities, however, the related expenses are being accounted as administrative costs, which increases the share of this category. To these challenges are added the coordination difficulties between MINAM DGs, which are paralleled in coordination problems between the TAP projects. These projects share common issues that require a more integrated perspective. Recommendation: Implement shared activities to strengthen MINAM administrative capabilities, and which will strengthen the program’s holistic perspective. For this, TAP must reformulate how it accounts for activities linked to strengthening MINAM and TAP administrative processes and strengthen TAP management tools so that they impact the General Directions. 61 5. Governance: Regarding the governance process, the prioritization of MINAM governance strengthening is positive, but this has had more impact on its positioning vis-à-vis other national sectors than in strengthening its links with subnational governments. Recommendation: Continue prioritizing MINAM governance strengthening both in the context of other national sectors and of subnational governments through capacity strengthening activities among various actors, conducting studies, and supporting agreement processes that conclude with specific recommendations for investment in the environmental sector. It is also important to support a strategy of communication and research that generates new scientific knowledge in priority areas that facilitate MINAM’s effective leadership as the nation’s lead environmental agency. 6. Gender Approach The participation of women in the courses, seminars, workshops and other activities is identified in the design and promotion of activities; however, this does not mean that there is a gender approach in the activities themselves. On the contrary, activities lack a gender approach and there is a lack of preparation on gender issues among those involved in activities, which makes it difficult to monitor project products inclusion of a gender approach. Recommendation: Develop a plan for the introduction of a gender approach in TAP and its projects, as well as the preparation of those involved regarding the relevance of a gender approach in environmental issues. This approach should respond to the National Equality Opportunity Plan. 7. Coordination with USAID: The presence of USAID technical officers facilitates communication between the DG and TAP, which has contributed positively to overcoming implementation problems. The participation of USAID technical officers strengthens the relationship between the USAID Portfolio Projects and MINAM. However, the responsible parties did not agree on a Program Description, which hindered program implementation. There was a change of responsibilities that led to delays in TAP implementation. Recommendation: Enhance the role of USAID technical officers to further facilitate communication between the DG and TAP and to promote a greater involvement of TAP projects with USAID Portfolio projects. However, as long as the TAP Coordinator lacks the tools to exercise his leadership, this will only help temporarily. 8. Coordination with the TAP: For the internal management of the program, the evaluation has noticed that the TAP executive coordinator does not have the appropriate tools to exercise leadership, such as having a Program Description, a suitable profile for the technical coordinators and the definition of the objectives of the General Directorates. Recommendations: In the short-term, revise the qualifications and responsibilities of the technical coordinators, revise the type of contract and review possible incentives for the coordinators under the executive coordinator’s leadership. It is important to update the technical coordinator profiles to include qualifications regarding public administration and management, while strengthening the coordinators technical responsibilities. 9. Communications: There is much to achieve in terms of the project’s knowledge management. This is reflected in various ways, especially in the lack of an integrated internal supervision system to facilitate communication between the parties (TAP, MINAM, USAID). The project has communication channels, but these respond only ‘as needed’ to ad hoc issues or 62 problems. Recommendations: The project should elaborate and deploy formats and protocol for documenting meeting minutes, workshops and agreements that will facilitate the monitoring of agreed actions. In addition, MINAM/USAID/TAP should establish a management communication plan under the PM&E framework that specifies types of coordination and their frequency. 63 Annexes 64 Annex I: Statement of Work (SOW) 1. PROGRAMA A EVALUAR Identificando Información El proyecto tiene por título «Programa de Asistencia Técnica PAT USAID/MINAM» y ha sido aprobado para un período de 4 años a desarrollarse entre el 2012-2016. Contexto de Desarrollo El PAT se inscribe en la Estrategia de Cooperación Internacional de los EE.UU. El año 2008, como parte de esta estrategia, la República del Perú y los Estados Unidos de América, a través de la Agencia de los Estados Unidos para el Desarrollo Internacional (USAID), suscribieron en el Convenio de Donación Nº 527-0423, que se canalizaría a través de la Unidad Ejecutora del Proyecto STEM (UEP). De acuerdo con dicho convenio, EEUU brindaría asistencia técnica en los siguientes temas: crecimiento económico, inversión en la población, gobernabilidad justa y democracia, paz, seguridad y medio ambiente; siendo la donación para realzar dicha asistencia, de aproximadamente de 59 millones de dólares. Estas metas se constituyen teniendo como referencia los objetivos de desarrollo del TLC como marco de relacionamiento comercial y económico entre ambos países. Dentro de dicho Convenio, el tema ambiental fue de particular importancia. Entre sus objetivos, se contempla “Prestar asistencia para aumentar la inversión pública y privada en conservación; proporcionar oportunidades para mejores formas de vida, basadas en el manejo sostenible de los recursos y mejorar la gestión ambiental sostenible en sectores seleccionados, fortaleciendo la capacidad de las entidades públicas y privadas para mejorar la gestión ambiental”. En este contexto, el Ministerio del Ambiente (MINAM) convoca en el año 2010 una consultoría denominada “Servicio de consultoría para la Formulación de Programas de Asistencia Técnica para el MINAM”, presentada a USAID en noviembre del 2010. El objetivo del mismo fue formular cuatro fichas de proyectos de asistencia técnica con un horizonte de planificación de cuatro años, expresados en: Una matriz de identificación de actores clave, un esquema de marco de resultados, matriz de marco lógico, una matriz de presupuesto detallado por actividad y un cronograma general de implementación. A todo ello se adjuntaba una propuesta de estructura organizacional para la Unidad Ejecutora del Programa. Dicho producto fue base para el actual PAT, pues identificó las necesidades de las direcciones del MINAM en términos de asistencia técnica para su fortalecimiento. Sin embargo, en el año 2012, se estableció un nuevo Convenio de Donación Nº 527-0426. Este tuvo por finalidad el fortalecimiento de la democracia y estabilidad del Perú, mediante una mayor inclusión social y económica; la reducción del cultivo ilícito de coca y de la explotación ilegal de recursos naturales; y se estableció como uno de sus tres objetivos de desarrollo el manejo sostenible de los recursos naturales en la Amazonía y la Sierra Glacial. Según el Convenio, “esta actividad proporcionará asistencia financiera a MINAM para reclutar líderes…para implementar proyectos ambientales”, lo cual implica un gran esfuerzo institucional desde diversos sectores. De esta manera, solo después de dos años se viabilizó el PAT y al que se incorporó especial interés de los ámbitos de acción geográfica vinculados al Convenio de Donación Nº 527-0426 y 65 sus objetivos. Asimismo, es importante indicar que dentro de los lineamientos que el Convenio señala para la ejecución de actividades es que estas, además, tengan un enfoque de resultados y una atención prioritaria a la participación por género. En el siguiente Gráfico (Gráfico 1), se puede visualizar la línea de tiempo de los antecedentes del PAT: Gráfico 1: Línea de tiempo de los antecedentes en lo que se enmarca el PAT Área y grupos meta El PAT USAID/MINAM tiene como población objetivo a funcionarios públicos involucrados en los tres niveles de gobierno nacional, regional y local. Ello incluye tanto a los funcionarios del MINAM, como funcionarios de los Gobiernos Regionales, provinciales y locales. Asimismo, su área de intervención prioriza la región amazónica y la sierra glacial, considerando las siguientes regiones: Madre de Dios, Loreto, Ucayali, San Martín, Amazonas, Junín. Cusco, Puno, Pasco, La Libertad y Cajamarca. Resultados esperados Tal como se establece en el Marco de Resultados del PAT, este tiene como resultado general esperado es “Contribuir al uso sostenible de los recursos naturales a través del Fortalecimiento del Ministerio del Ambiente como ente rector del Sistema Nacional de Gestión Ambiental” Enfoque e implementación El enfoque del PAT consiste en la asignación de presupuestos por resultados directamente al MINAM y está adscrito a la Secretaría General de dicho Ministerio. Esta asignación se viene implementando, a través de cinco Proyectos: 1. Proyecto de fortalecimiento de las capacidades del MINAM como autoridad Científica de la Convención Sobre el Comercio Internacional de Especies Amenazadas de Fauna y Flora silvestres (CITES), cuyo propósito o resultados esperados es lograr que las capacidades del MINAM se fortalezcan para la aplicación de la 66 Convención CITES, a fin de asegurar que las especies de fauna y flora silvestre incluidas en la CITES son exportadas bajo dictámenes que garanticen que dicha exportación no perjudique su supervivencia” 2. Proyecto fortalecimiento de capacidades nacionales para la valoración económica de la biodiversidad y los servicios ambientales, cuyo propósito es que los ecosistemas que suministran servicios ambientales de provisión de agua y belleza escénica se aprovechen de manera sostenible. 3. Proyecto para implementar mecanismos para la aplicación de los instrumentos de gestión ambiental y desarrollo de instrumentos complementarios, en el marco del Sistema Nacional de Evaluación de Impacto Ambiental (SEIA), cuyo propósito esperado es fortalecer el carácter preventivo del SEIA, a través de la identificación, evaluación, mitigación y corrección anticipada de los impactos ambientales negativos significativos derivados de los proyectos de inversión sobre la diversidad biológica y sus componentes relacionados. 4. Proyecto de Iniciativa de Conservación de Bosques para la mitigación del cambio climático, cuyo propósito esperado es que el MINAM implemente un Modelo de Gestión de Conservación de Bosques que sea capaz de replicar a nivel nacional. 5. Proyecto de Fortalecimiento de capacidades para la incorporación del enfoque de adaptación al cambio climático en los proyectos de inversión pública, cuyo propósito esperado se centra en incorporar en el Sistema Nacional de Inversión Pública (SNIP) el enfoque de Adaptación al Cambio Climático (ACC) para los Proyectos de Inversión Pública (PIPs). Cada uno de estos Proyectos está dirigido por un Coordinador Técnico, asimismo se incluye un Coordinador Técnico de Monitoreo y Evaluación, coordinando con el Departamento de Administración y Finanza, reportando los avances al Coordinador Ejecutivo, que es responsable de la ejecución y la implementación de los cinco Proyectos del PAT USAID/MINAM, quien a su vez reporta las actividades al USAID. La hipótesis de desarrollo del PAT se encuentra en el marco de resultados del programa, que incluye un marco lógico de cada uno de sus Proyectos. En este marco se establece, además de la finalidad del programa y de cada proyecto, los objetivos estratégicos de cada uno de ellos y los objetivos específicos o resultados intermedios de cada proyecto. También se establecen los indicadores que sirven para medir el alcance de estos resultados y las metas que deben ser alcanzadas, así como los medios de verificación de las mismas y los supuestos que se asume para que sean alcanzables. A continuación se encuentra el Marco de Resultados en el que se enmarcan estos proyectos. El marco lógico, a su vez ha sido derivado del marco de resultados, y es su complemento, en el cual se busca un diseño que se pueda monitorear y medir, donde los objetivos están claramente enunciados, la hipótesis general del proyecto claramente descrita, y los indicadores se definen claramente a cada nivel de jerarquía del proyecto. El esquema a continuación, describe el marco de resultado global del Convenio MINAM-USAID actualizado en el documento de Plan de Monitoreo y Evaluación del PAT-USAID/MINAM. Los resultados principales e intermedios se diseñaron en base a objetivos estratégicos y operacionales (objetivos específicos) correspondientes. Se consideró por cada resultado 67 intermedio un aproximado de 2 a 3 indicadores. Siendo los principales: número de procedimientos y leyes, numero de guías, pautas metodológicas, estudios, número de personas capacitadas o sensibilizadas, número de organizaciones y entidades capacitadas u organizadas, cantidad de dinero generado, así como indicadores de cuantificación de recursos naturales y biofísicos (Tabla 5). En el Gráfico 3. Puede observar que el que goza de un mayor porcentaje son el número de guías, metodologías y estudios. Cabe señalar que según el PMP, hay un total de nueve indicadores estándar del PAT USAID/MINAM, que son transversales entre los Proyectos y que a la vez contribuyen a los objetivos del Convenio Perú-EEUU. 68 Esquema: PAT-USAID/MINAM, Marco de Resultados, 2013 Fuente: PAT-USAID/MINAM (2013). Plan de Monitoreo y Evaluación. 69 Tabla 5. PAT – USAID/MINAM – INDICADORES RESULTADO PRINCIPAL Y RESULTADOS INTERMEDIOS INDICADORES 1. Número de procedimientos para la gestion del Convenio CITES. 2. Número de personas capacitadas en la implementación de la Convención CITES 3.- Número de horas-hombre capacitadas. 1. Número de estudios científicos 2. Porcentaje de implementación de plataforma virtual IR1. El MINAM ha sido consolidado en funciones claves como Autoridad Científica CITES, mediante la generación de información científica de especies de flora y fauna silvestre priorizadas en regiones amazónicas para la emisión de dictámenes; contribuyendo a que la exportación no 1. .- Porcentaje de especies seleccionadas que cuentan con estudios científicos para los dictámenes 1. Numero de Metodologías de Valoración aprobadas 2. Número de funcionarios capacitados en Valoración Económica 1. Número de Estudios de Valoración Económica del Patrimonio Natural 2. Número de Mecanismos de Manejo Sostenible 3. Número de Personas sensibilizadas 1. Número de guías metodológicas para la formulación de PIP en Servicios Ecosistémicos 2. Número de Gobiernos Regionales han sido capacitados en las guías metodológicas mediante el SNIP verd 1. Monto de dinero generado por los mecanismos e instrumentos implementados en las 05 regiones amazónicas 2. Número de Mecanismos de Valoración Económica y PSE 1. Número de instrumentos formulados para la gestión de EIA detallados, a cargo del SENACE 2. Número de personas capacitadas en evaluación y elaboración de EIA-d. 1. Número de instrumentos formulados para administrar el SEIA 2. Número de personas capacitadas en la función rectora del SEIA 1. Número de gobiernos regionales que cuentan con normativa o directivas sobre SEIA 2. Número de personas de las regiones capacitadas en el proceso de evaluación de impacto ambiental. 3. Número de regiones que cuentan con información sistematizada para el proceso de evaluación de impacto ambiental IR3. El Sistema Nacional de Evaluación de Impacto Ambiental (SEIA) ha sido mejorado en sus aspectos políticos y organizativos, así como en sus instrumentos y procedimientos técnicos y administrativos, preferentemente en zonas prioritarias para la conservación de la diversidad biológica o ecosistemas frágiles. 1. Porcentaje de Estudios de Impacto Ambiental (EIA) aprobados que cumplen con lo establecido en la norma, en comparación a la línea base IR3.3 El proceso de evaluación de impacto ambiental de cinco (05) sectores ha sido articulado efectivamente con el SEIA, en cinco (05) regiones de la amazonia, en el marco de sus competencias. IR2. El Sector Público ha establecido mecanismos e instrumentos tipos de Valoración Económica y Pagos por Servicios Ecosistémicos del Patrimonio Natural para 05 regiones: Ucayali, Amazonas, Loreto, Madre de Dios, San Martin y Junín. S E I A IR3.1 Políticas, procedimientos, instrumentos, tecnologías y capacidades del SENACE han sido desarrollados para la adecuada gestión de los EIA￾detallados para cinco (05) sectores. IR3.1 Políticas, procedimientos, instrumentos, tecnologías y capacidades del SENACE han sido desarrollados para la adecuada gestión de los EIA￾detallados para cinco (05) sectores. V A L O R A C I O N IR2.1 Se han aprobado las metodologías de valoración económica. IR2.2 Se ha diseñado, validado e implementado mecanismos tipos para la conservación del Patrimonio Natural, en el MINA IR2.3 Se han incorporado metodologías de Valoración Económica de Pagos por Servicios Ecosistémicos en la formulación de Proyectos de Inversión Pública entre el MINAM y el Ministerio de Economía y Finanzas. C I T E S IR1.1 Se han generado instrumentos para la gestión eficiente de la Autoridad Científica CITES IR1.2 Se ha generado y/o sistematizado información científica para la emisión de dictámenes por el MINAM y es utilizada por las Autoridades CITES (MINAG y otros). 70 Para alcanzar los objetivos de cada proyecto se prioriza, como mecanismo de trabajo, la permanente coordinación con las Direcciones de Línea y el Programa Nacional de Conservación de Bosques, además de estar coordinando permanentemente con los proyectos, programas e Instituciones, que estén vinculadas al trabajo y objetivos del PAT. El ámbito de intervención del programa es la región amazónica y sierra glacial, considerando a las regiones de Loreto, Ucayali, Madre de Dios, San Martín, Amazonas, Cusco y Junín, Pasco y Puno, con la mayoría se ha iniciado contacto y se viene estableciendo un mecanismo de coordinación para el trabajo futuro. Data existente Durante la elaboración del SOW se entregaron diversos documentos desde los más generales vinculados al entorno del Programa, hasta los más específicos. En especial, habría que destacar que como parte del sistema de monitoreo y evaluación, se lleva a cabo diversas actividades de seguimiento, que incluyen los reportes trimestrales y anuales, visitas de campo, así como evaluaciones parcial y final, a lo largo del Programa. A continuación, se clasifica el listado de documentos relevantes sugeridos, según temáticas: 1-Contexto nacional e internacional de la cooperación:  GUEVARRA, Susana. Mapeo de Proyectos del MINAM. Evaluation, Junio de 2014.  MINAM. OFICIO 070-2010, con la manifestación de interés en cuatro Proyectos de Asistencia Técnica.  MINAM. Reporte de monitoreo. Proyectos de Cooperación ejecutados por el Ministerio del Ambiente al primer semestre de 2013. Lima, octubre de 2013.  PAT USAID/MINAM. Antecedentes y Organización del Programa. s/a.  PAT USAID/MINAM. Actores Clave en la Gestión de Proyectos del PAT USAID/MINAM. Lima, setiembre de 2014.  USAID. Convenios de donación 0423 y 0426 entre el Estado Peruano y el Gobierno de Estados Unidos.  USAID. Convenio de Donación de USAID SOAG No. 527-0423. 2010. Y sus siete enmiendas. RESULTADO PRINCIPAL Y RESULTADOS INTERMEDIOS INDICADORES 1. No. de personas capacitadas en gestión y conservación de bosques. 2. No. de instrumentos de gestión para conservación de bosques en uso. 3. No. de planes de inversión articulados con Municipios Provinciales y/o Gobierno Regional. 1. No. de Gobiernos Regionales con sistemas de monitoreo en funcionamiento. 2. No. de reportes públicos difundidos 1. No. de Guías de PIPs elaboradas. 2. No. de perfiles de Proyectos de Inversión Pública aprobados. 3. No. de empresas financiando actividades de conservación de bosques. IR4. El Programa Nacional de Conservación de Bosques para la Mitigación del Cambio Climático ha mejorado su implementación e intervención en los bosques comunales amazónicos de las regiones de Loreto, Ucayali, Madre de Dios, San Martin, Amazonas, Junín, Pasco y Cuzco. 1. No. hectáreas afiliadas al PNCB-MCC conservadas con criterios de sostenibilidad biológica y económica. IR4.1 Gobiernos Regionales, Municipalidades Provinciales y Comunidades Nativas han desarrollado capacidades para la conservación y gestión de bosques. IR4.2 Gobiernos Regionales han implementado sistemas de monitoreo para la conservación de bosques en coordinación con MINAM, MINAG y MEF. IR4.3 Mecanismos financieros públicos y privados para la conservación de bosques han sido implementados, en coordinación con MEF, PNCB-MCC y sector privado empresarial. B O S Q U E S 71  USAID. Considerations for the Monitoring and Evaluation of G2G Activities. PPL/LER Discussion Note. Julio de 2014.  USAID. ADS Chapter 220; Use and Strengthening of Reliable Partner Government Systems for Implementation of Direct Assistance. Julio de 2014. Algunos documentos que faltan investigar:  Justificación de la selección del tipo de cooperación. 2-Cambios en la agenda nacional  CEPLAN. Directiva General del Proceso de Planeamiento Estratégico. SINAPLAN. Directiva 001-2014-CEPLAN, Abril 2014.  MEF. Directiva N° 0001-2014-EF/50.01. “Directiva para los programas presupuestales en el marco de la programación y formulación del presupuesto del sector público para el año fiscal 2015”. Enero 2014.  MINAM. Plan Estratégico Institucional 2014-2016.  MINAM. Planes Nacionales vinculados a las Direcciones beneficiadas con el PAT USAID/MINAM.  MINAM. Política Nacional del Ambiente.  MINAM. Plan Nacional de Acción Ambiental 2011-2021.  MINAM. Ejes Estratégicos de la gestión Ambiental.  MINAM. Agenda Nacional Ambiental 2013-2014.  OSCE. Ley 30225. Ley de Contrataciones del Estado. El Peruano, Julio 2014. Decreto supremo 080-2014. Dificultad:  Se debería hacer mención más desglosada sobre los documentos que definieron en los últimos años la agenda nacional ambiental.  Se requerirá más entrevistas para identificar otros ámbitos de modernización del sector público peruano que requiera ser incorporado en el análisis de cambios en el entorno. 3-Diseño inicial  MINAM. Decreto Legislativo N° 1013, Ley de creación del Ministerio del Ambiente.  MINAM. Plan Estratégico Sectorial Multianual, 2013-2016.  MINAM. Cartera Estratégicas Multianual de Proyectos de Inversión Pública. Noviembre de 2013.  MINAM. Plan Estratégico Institucional 2014-2016.  MINAM. Planes Nacionales vinculados a las direcciones beneficiarias del PAT USAID/MINAM.  MINAM. Política Nacional del Ambiente.  MINAM. Plan Nacional de Acción Ambiental 2011-2021.  MINAM. Ejes Estratégicos de la gestión Ambiental.  MINAM. Agenda Nacional Ambiental 2013-2014.  MINAM. Reporte de Seguimiento y Evaluación de la Agenda Nacional de Acción Ambiental – AgendAmbiente Perú 2013-2014.  PAT USAID/MINAM. Plan de Monitoreo y Evaluación. Lima, agosto de 2013.  SASE. Formulación de Programas de Asistencia Técnica para el MINAM. 2010 (no fue entregue todavía). Dificultad:  Hay indicios de que el documento de SASE con el diseño inicial no necesariamente presenta un buen conocimiento entre las personas entrevistadas. 72  No se encontró los criterios para la focalización geográfica y de los beneficiarios de las actividades que se vienen llevando a cabo. 4-Medición de desempeño  PAT USAID/MINAM. Plan de Monitoreo y Evaluación. Lima, agosto de 2013.  PAT USAID/MINAM. Reporte Trimestral, agosto a diciembre 2012. Lima, marzo de 2013.  PAT USAID/MINAM. Reporte de Gestión Trimestral, Enero a Marzo de 2013. Lima, 2013.  PAT USAID/MINAM. Reporte de Gestión Semestral, Enero a Junio de 2013. Lima, 2013.  PAT USAID/MINAM. Reporte de Gestión, Enero a Setiembre de 2013. Lima, 2013.  PAT USAID/MINAM. Presentación PAT para USAID. Diapositivas. Lima, agosto 2014. Dificultad:  La medición del desempeño se vincula mucho a cada proyecto. Sería importante verificar el enlace entre estos indicadores por proyecto, los señalados por el programa como los de más alto nivel, en especial los indicadores estándar. Para ello, entre otros documentos, revisar las fichas de los indicadores (PIRS) y analizar la evaluación de su calidad.  La línea de base se incorpora directamente al tracking table y no existe un documento con claridad sobre la forma cómo se planteó la primera medición de los indicadores de desempeño del programa. 5-Gestión de cambios del marco de resultados  PAT USAID/MINAM. Plan de Monitoreo y Evaluación. Lima, agosto de 2013. Dificultad:  No está claro cómo se inicia la solicitud y la aprobación de un cambio, ni cuáles serían considerados cambios pequeños, medianos o hasta llegar a una nueva versión del marco lógico o de la matriz de resultados. 6-Estándares de la Planificación  PAT USAID/MINAM. Plan de Monitoreo y Evaluación. Lima, agosto de 2013.  PAT USAID/MINAM. Marco Lógico del PAT USAID/MINAM. Lima, Mayo de 2013  USAID. Evaluation Policy. Enero de 2011.  USAID. ADS Chapter 200. Introduction to Programming Policy. Octubre de 2012.  USAID. ADS Chapter 201. Planning. Marzo de 2012.  USAID. ADS Chapter 203. Assessing and Learning. Febrero de 2012.  USAID.  USAID/Perú. 2012 “Estrategia de la Cooperación de los EE.UU. para el Desarrollo del Perú, 2012-2016”. Lima, Perú: USAID/Perú. (CDCS)  USAID. Performance Management Plan (PMP): Toolkit A Guide for Missions on Planning for, Developing, Updating, and Actively Using a PMP. October 29, 2013. Dificultad:  Sería importante coordinar con Evaluation para saber si faltan documentos clave para la comprensión del marco de monitoreo y evaluación de USAID. 7-Gestión de las comunicaciones y de la calidad  PAT USAID/MINAM. CARTA N" 027 - 2013-MINAM-SEG-PAT-USAID/MINAM; de entrega del Plan de Monitoreo y Evaluación. Lima, agosto de 2013. 73 Dificultad:  No se encontró página web, boletines, o herramientas de comunicación que faciliten la transparencia y rendición de cuentas del programa, lo cual puede llevar a dificultades de comprensión de las oportunidades que puede generar inclusive entre los responsables de las direcciones de línea beneficiarias. 8-Control de costos  PAT USAID/MINAM. Ejecución Mensualizada, 2012-2013, Julio de 2013.  PAT USAID/MINAM. Proyección de Gastos al 2014, julio de 2013.  PAT USAID/MINAM. Plan Operativo Anual, 2013, Enero de 2013.  PAT USAID/MINAM. Plan Operativo Anual, Enero a Diciembre del 2014, Enero de 2014.  PAT USAID/MINAM. Reporte Trimestral, agosto a diciembre 2012. Lima, marzo de 2013.  PAT USAID/MINAM. Reporte de Gestión Trimestral, Enero a Marzo de 2013. Lima, 2013.  PAT USAID/MINAM. Reporte de Gestión Semestral, Enero a Junio de 2013. Lima, 2013.  PAT USAID/MINAM. Reporte de Gestión, Enero a Setiembre de 2013. Lima, 2013. Dificultad: Existen diversos documentos Excel que se pusieron a disposición de la evaluación, pero se requiere hacer una cronología de la documentación para poder entender la secuencia y los criterios para la elaboración de las proyecciones y de los reportes. 9-Riesgos y gestión de cambios  PAT USAID/MINAM. Plan de Monitoreo y Evaluación. Lima, agosto de 2013.  PAT USAID/MINAM. Reporte Trimestral, agosto a diciembre 2012. Lima, marzo de 2013.  PAT USAID/MINAM. Reporte de Gestión Trimestral, Enero a Marzo de 2013. Lima, 2013.  PAT USAID/MINAM. Reporte de Gestión Semestral, Enero a Junio de 2013. Lima, 2013.  PAT USAID/MINAM. Reporte de Gestión, Enero a Setiembre de 2013. Lima, 2013.  PAT USAID/MINAM. Presentación PAT para USAID. Diapositivas. Lima, agosto 2014. Dificultad:  No se identifican los riesgos del proyecto, más allá de los supuestos de los marco lógicos que son muy genérico y no permite establecer planes de contingencia. Ello dificulta la ejecución especialmente frente a los cambios constantes en los contextos internacional, nacional y de necesidades de los beneficiarios del Programa. No se encontró un documento que explique gestión de calidad en el sentido de dejar explícito la gestión de las necesidades de las direcciones y cómo ello se vincula con los cambios del proyecto. 10-Gestión del capital humano del programa  PAT USAID/MINAM. Antecedentes y Organización del Programa. s/a.  PAT USAID/MINAM. Organigrama PAT en el Ministerio. Diapositivas. Lima, agosto 2014. Dificultad:  No se encontró documentos que contengan incentivos al desempeño del personal, ni actividades de desarrollo del equipo del PAT USAID/MINAM. 11-Logro de resultados  PAT USAID/MINAM. Plan de Monitoreo y Evaluación. Lima, agosto de 2013.  PAT USAID/MINAM. Reporte Trimestral, agosto a diciembre 2012. Lima, marzo de 2013. 74  PAT USAID/MINAM. Reporte de Gestión Trimestral, Enero a Marzo de 2013. Lima, 2013.  PAT USAID/MINAM. Reporte de Gestión Semestral, Enero a Junio de 2013. Lima, 2013.  PAT USAID/MINAM. Reporte de Gestión, Enero a Setiembre de 2013. Lima, 2013.  PAT USAID/MINAM. Presentación PAT para USAID. Diapositivas. Lima, agosto 2014. Dificultad:  La línea de base se incorpora directamente al tracking table y no existe un documento con claridad sobre la forma cómo se planteó la primera medición de los indicadores de desempeño del programa. 12-Calidad de indicadores y lecciones aprendidas  PAT USAID/MINAM. Plan de Monitoreo y Evaluación. Lima, agosto de 2013.  PAT USAID/MINAM. Reporte Trimestral, agosto a diciembre 2012. Lima, marzo de 2013.  PAT USAID/MINAM. Reporte de Gestión Trimestral, Enero a Marzo de 2013. Lima, 2013.  PAT USAID/MINAM. Reporte de Gestión Semestral, Enero a Junio de 2013. Lima, 2013.  PAT USAID/MINAM. Reporte de Gestión, Enero a Setiembre de 2013. Lima, 2013. Dificultad:  Las lecciones aprendidas están dispersas por cada uno de los reportes y no se encontró un documento con la sistematización inicial de los procesos de aprendizaje que se llevan a cabo. 2. RATIONALE DE LA EVALUACIÓN DE MEDIO TÉRMINO Estos términos de referencia han sido elaborados de forma participativa, involucrando, mediante reuniones de planeamiento, a los responsables de USAID que sugirieron una serie de temas que facilitaron la definición de los enfoques de la evaluación; a los responsables máximos de la contraparte demandante, que son en este caso los viceministros del MINAM involucrados; la dirección del equipo vinculado a la ejecución del programa; representantes de USAID Evaluation; y los consultores responsables de la elaboración de este SOW. En estas reuniones, se abordó los posibles indicios que deberían llevar a definir el objetivo general de la investigación que oriente las actividades de evaluación que podrían generar hallazgos, conclusiones y recomendaciones específicas con el fin de incrementar la relevancia de los resultados esperados de la evaluación, así como las posibilidades de que se lleve a cabo las recomendaciones planteadas a partir de dichos resultados. En este sentido, se buscó asegurar que las cuestiones planteadas reflejaran las prioridades de los involucrados sobre temas pertinentes para el propósito de la evaluación intermedia, hecha con independencia y con profesionales externos, que ayudarán a la reflexión de las actividades a realizar. Los ejes, los temas generales y las preguntas de evaluación también han sido elaboradas mediante un proceso participativo, en el que se incluyeron las opiniones y sugerencias de representantes de USAID, así como de la alta dirección del MINAM y los miembros del equipo del PAT USAID/MINAM; pues como usuarios de la evaluación, se deben canalizar los esfuerzos para que las recomendaciones tengan una mayor incidencia sobre su toma de decisiones. De esta manera, se aseguró que estas preguntas tuvieran concordancia con las necesidades del equipo del PAT respecto a qué información concreta se requiere para mejorar la implementación del mismo; así como la eliminación de aspectos que pueden ser poco relevantes o profundamente conocidos, con lo que su evaluación implicaría esfuerzos innecesarios. 75 Indicios que sustentan la necesidad de una evaluación A lo largo de la implementación del PAT, hubo algunos cuellos de botella que se ha observado en la misma implementación del PAT USAID/MINAM, así como las deficiencias en la organización ejecutora. En primer lugar, se debe tener en cuenta que el PAT USAID/MINAM ha venido implementándose desde el 2012, cumpliendo con las actividades bajo el marco de resultados. Sin embargo, incluso con los avances y los cambios continuos en base a las coordinaciones interinstitucionales, se presentan diversos obstáculos para su implementación, lo cual se puede visualizar en los reportes trimestrales del 2012 al 2013. Según las primeras evidencias, dicha situación responde a tres ejes: problemas del contexto externo, pertinencia del diseño del proyecto y dificultades en la gestión del equipo del PAT USAID/MINAM. Esta problemática se agudiza por la escasa evidencia que sostenga la focalización para la selección de los ámbitos de acción, así como de los beneficiarios. Cada PAT tiene un promedio de 3 a 6 Regiones como beneficiarios, no habiendo una clara descripción de la selección del ámbito o del beneficiario final, siendo funcionarios, comunidades e instituciones los beneficiarios indirectos y directos de cada Proyecto. Esta definición de ejecución de acciones transversales no coincide en los cinco Proyectos, lo cual se suma a que existen indicios de que se ha considerado implementarlo bajo una estrategia vertical cuando existen acciones que pueden realizarse de manera conjunta. Otro de los problemas que acoge el PAT USAID/MINAM ha sido la dificultad de coordinación en los tres niveles de gobierno para la viabilización de acciones en las regiones beneficiarias. Se ha tenido indicios de que existe una buena una gestión intersectorial, pero en un entorno cambiante de modernización del sector público peruano, lo cual dificultó también llegar a las regiones y a las localidades en eficaz relevancia. Todo ello se refleja en la deficiente ejecución de los fondos y la gestión administrativa. Si bien el MINAM, es el encargado de concertar los esfuerzos necesarios entre las entidades del sector público y privado, para lograrlo, debe reforzar la institucionalidad ambiental y contar con instrumentos de gestión, para lo cual sería necesario el fortalecimiento de capacidades así como el soporte logístico y humano mínimos. Además de que el PAT USAID/MINAM fuera concebido como proyectos independientes, se han planteado muchas actividades y acciones, de las cuales hay indicios que no necesariamente estarían respondiendo al cumplimiento de los resultados ni la hipótesis planteada. Por ejemplo, se debería evaluar si el marco de resultados se ajusta a lo que significa en términos conceptuales el “fortalecimiento de capacidades”. Esta última consideración es la pieza fundamental del PAT USAID/MINAM, pero se han considerados diferentes objetivos, si una vinculación estrecha a los propósitos del PAT USAID/MINAM. Es necesaria la implementación de una estrategia de coordinación y apoyo a las Direcciones para la continuidad del Programa por parte de USAID. Asimismo, cabe señalar que MINAM, es un ministerio de reciente constitución, pues no tiene más de seis años de creación. Las Direcciones Generales que ejecutan el PAT USAID/MINAM tienen el mismo tiempo de funcionamiento, a excepción del Programa Nacional de Conservación de Bosques para la Mitigación del Cambio Climático, que fue creado el año 2010. Dicha particularidad 76 es una ventaja para la implementación del PAT USAID/MINAM pero también una desventaja. Entre las dificultades señaladas en las entrevistas iniciales que tiene Ministerio del Ambiente podríamos mencionar: - Debilidades Institucionales para coordinaciones transversales - Direcciones Generales han implementado recién sus estrategias con los Planes nacionales y otras no tienen. - Direcciones Generales podrían tener dificultades para vincularse con el PAT USAID/MINAM, especialmente en términos de agilidad para la atención a nuevos requerimientos o requerimientos más específicos. - Procesos Administrativos dificultoso para viabilizar la ejecución de los fondos a los Proyectos PAT USAID/MINAM Otras de las dificultades señaladas durante las entrevistas se refieren a la asignación de los Fondos directamente a la institución y su dilato nivel de ejecución en el ejercicio anual. Asimismo de criterios y requerimientos del USAID al MINAM, que en los siguientes apartados se van a detallar. 3. PROPÓSITO GENERAL DE LA EVALUACIÓN DE MEDIO TÉRMINO El propósito de la evaluación intermedia se centra en identificar los obstáculos para la ejecución del PAT USAID/MINAM, de tal forma a realizar recomendaciones fundamentadas que se reflejen en una asistencia técnica efectiva que sirvan para mejorar la toma de decisiones en la implementación de lo que queda de ejecución del Programa Asistencia Técnica. Para ello, se tendrá en consideración una investigación acerca del contexto de cooperación en el cual se desarrolla, el diseño del programa y su medición de desempeño, las herramientas de gestión y la eficacia del Programa. Se dará énfasis en encontrar hallazgos acerca de las razones por las que se logró o no cada una de metas del marco de resultados y en miras al cumplimiento de los objetivos finales del programa. Para ello, se evaluará los aspectos relacionados con las oportunidades y riesgos vinculados con el contexto nacional e internacional del PAT; las fortalezas y debilidades en los procesos internos del PAT, incluyendo tanto el diseño del Programa y su pertinencia respecto del contexto y finalidad, como las capacidades de gestión, reporte, verificación, coordinación, entre otros aspectos propios del equipo de trabajo del PAT; que facilitaron o dificultaron su eficacia. De esta manera, del propósito general de la evaluación se desprenden objetivos específicos, que se refieren a los diferentes aspectos que se busca abordar para lograr entender las diferentes razones que pueden influenciar el desempeño del Programa: A. Conocer los riesgos y oportunidades que tiene el modelo de cooperación internacional Gobierno a Gobierno de USAID aplicado al PAT-MINAM, en el marco del contexto internacional de cooperación El primer objetivo específico está relacionado con las oportunidades y riesgos que presenta el contexto nacional e internacional dentro del cual se lleva a cabo el PAT USAID/MINAM. Este contexto incluye el modelo de cooperación internacional de USAID, los convenios de donación 0423 y 0426 entre el Estado Peruano y el Gobierno de Estados Unidos, los estándares y acuerdos internacionales en relación con la cooperación internacional y el medio ambiente, entre otros. El 77 rápido proceso de modernización reciente del sector público peruano también implica un reto para la ejecución presupuestales en todos los niveles del sector público, marco en el cual se encuentra la ejecución directa que debe tener este Programa. B. Evaluar la pertinencia del diseño programa, en función de las tendencias y dinámicas de USAID y de la evolución del MINAM En esta parte se analizará el diseño del PAT en relación con el marco de resultados y en cuanto a su capacidad de adaptación a dichas dinámicas. Asimismo, se analizarán la relación entre el diseño del PAT y las políticas y procesos tanto de USAID como del Gobierno Peruano, como el Objetivo de Desarrollo 3 de USAID para el Perú, el TLC o los planes de desarrollo y dinámicas de los ministerios. C. Conocer las fortalezas y debilidades que existen en el proceso de implementación del Programa PAT-MINAM Para cumplir con este objetivo, se evaluarán aspectos organizacionales internos al PAT, como su cumplimiento de recomendaciones, planes de acción, delegación de responsabilidades, canales de comunicación, reporte, verificación, ejecución presupuestal, así como los incentivos para cumplir con dichos aspectos. D. Entender los factores críticos que determinan la eficacia del PAT en su etapa intermedia El cuarto objetivo está relacionado con las metas obtenidas por el PAT, dentro del marco de resultados, que se encuentran en el tracking table y los PIRS del PAT. Para ello, se analizará hasta qué punto los productos entregados contribuyen con el logro de los resultados, cómo funciona el sistema de monitoreo y adaptación, entre otros aspectos que permitirán profundizar en las causas por las cuales se dieron dichos logros. 4. USUARIOS DE LA EVALUACIÓN DE MEDIO TÉRMINO 1. Los usuarios primeros y directos de la evaluación son los miembros del equipo del PAT USAID MINAM, quienes usarán las recomendaciones en el proceso de toma de decisiones, en miras al cumplimiento de los objetivos finales del Programa. 2. También se debe destacar la alta dirección del MINAM que requiere de conocer los obstáculos para la ejecución del Programa de tal forma a superar las dificultades y poder utilizar los recursos para cumplir con los objetivos de fortalecimiento del sector. 3. Asimismo, la evaluación será utilizada por parte de USAID con el fin de mejorar sus capacidades de monitoreo y evaluación de desempeño, así como las de sus aliados y contrapartes gubernamentales. 4. Finalmente, la evaluación será utilizada por instituciones locales, a quienes les permitirá mejorar su capacidad de llevar a cabo evaluaciones de acuerdo con estándares internacionales. 5. PREGUNTAS DE LA EVALUACIÓN DE MEDIO TÉRMINO Las preguntas de evaluación intermedia han sido formuladas con el fin de lograr su objetivo vinculado a “Identificar los obstáculos para la ejecución del PAT USAID/MINAM, de tal forma a realizar recomendaciones fundamentadas y asistencia técnica efectiva que sirvan para mejorar la toma de decisiones en la implementación del resto del Programa Asistencia Técnica”., que puedan ser respondidas en base a evidencia empírica concreta, y no en las opiniones subjetivas que pueda tener el equipo de evaluación. Asimismo, se 78 ha considerado las limitaciones de tiempo y presupuesto que determinan la posibilidad de responder a cada una de ellas de forma satisfactoria. Estas preguntas ha sido elaborada a partir de los objetivos específicos de la evaluación intermedia, considerando cómo cada uno de ellos va a permitir llegar al propósito general de la misma, de entender las razones que determinan la eficacia del PAT e informar al equipo de implementación del mismo acerca de la toma de decisiones para alcanzar los objetivos finales del Programa. Si bien algunas de estas preguntas se pueden responder mediante la utilización de los datos plasmados en los reportes trimestrales y anuales del programa, la gran mayoría de ellas implicará la implementación de metodologías de levantamiento de información primaria, tal como se explicará en la Sección 3. Se identificaron cuatro ejes para cumplir con el objetivo de la evaluación y se vinculan a: contexto internacional y nacional; pertinencia del diseño; gestión del Programa y eficacia en el logro de los objetivos. A cada uno de estos temas se asignó una pregunta general, temas de reflexión y preguntas específicas. A continuación, en la Tabla, se presentan las preguntas que se requiere responder para alcanzar los objetivos específicos de la evaluación intermedia. 79 TABLA. Evaluación de Medio término del PAT USAID/MINAM: Preguntas de Investigación Objetivo de la evaluación: Identificar los obstáculos para la ejecución del PAT USAID/MINAM, de tal forma a realizar recomendaciones fundamentadas y asistencia técnica efectiva que sirvan para mejorar la toma de decisiones en la implementación del resto del Programa Asistencia Técnica. EJES PREGUNTA GENERAL TEMAS PREGUNTAS ESPECÍFICAS Modelo (Contexto nacional e internacional) ¿En función del contexto nacional e internacional, qué riesgos y oportunidades tiene el modelo de cooperación internacional Gobierno a Gobierno de USAID aplicado a un contexto del PAT-MINAM con cambios en la agenda nacional? 1-Contexto de la cooperación internacional ¿En qué consiste y cuál fue el contexto de selección del modelo de cooperación G2G de USAID? ¿Qué relación hay entre este modelo de USAID y los acuerdos o estándares internacionales en materia de cooperación internacional? ¿Las contrapartes conocen sobre esta forma de cooperación? ¿Qué oportunidades y riesgos implican para el Programa el tipo de instrumentos específicos (como cartas de ejecución) a partir de los cuales se ejecutan los convenios de cooperación 423 y 426? ¿Cómo influye la existencia de otros modelos de cooperación? ¿Cuál es su ventaja y desventaja con otros modelos de cooperación internacional? ¿Cuál es la prioridad del PAT frente al contexto de cooperación del MINAM? ¿Qué nuevas oportunidades y riesgos implican las nuevas tendencias de modernización de las herramientas de gestión de USAID (USAID forward) para este tipo de modelo de cooperación de USAID? 2-Cambios en la agenda nacional ¿Qué oportunidades y riesgos implican los nuevos acuerdos internacionales sobre temas ambientales que se consolidarán en la COP20 que será en Lima en diciembre? ¿Qué oportunidades y riesgos implican las nuevas tendencias de modernización del sector público peruano que pueden afectar el desarrollo de programas enmarcados en este modelo de cooperación de USAID? (ley de contrataciones, CEPLAN, PPR, etc.) Como modelo puede tener ventajas ¿Qué factores críticos existen para este tipo de modelo de cooperación de USAID en la consolidación del MINAM como autoridad nacional y en su vínculo con otros sectores de la administración pública y del entorno nacional? 80 EJES PREGUNTA GENERAL TEMAS PREGUNTAS ESPECÍFICAS Pertinencia (Diseño del programa) ¿Cuál es la pertinencia del diseño del programa en función de las tendencias y dinámicas de USAID y de la evolución del MINAM? 3-Diseño inicial ¿El procedimiento de diseño del PAT siguió las recomendaciones de USAID y del SNIP? ¿Qué tipo de escenarios sobre el contexto utilizaron para su implementación? ¿Qué tan realistas fueron los objetivos y metas en el diseño del PAT y por qué? ¿Se conoce el documento del diseño del PAT elaborada por el SASE? 4-Medición de desempeño ¿Son consistentes los resultados intermedios esperados del PAT MINAM con el nuevo marco de resultados del Objetivo de Desarrollo 3 de USAID para el Perú, y con los objetivos de las cláusulas ambientales del TLC? ¿Son consistentes los resultados intermedios esperados del PAT MINAM con los planes del MINAM (PESEM, PEI, Planes Nacionales, entre otros? ¿Los indicadores planteados en el marco lógico de los proyectos son consistentes con los objetivos esperados para el programa PAT y para los indicadores estándares de USAID? 5-Gestión de cambios del marco de resultados ¿Qué nivel de satisfacción sobre el marco de resultados presentan los involucrados en los proyectos del PAT? ¿Cuáles fueron los principales cambios en los requerimientos de los demandantes y cómo fueron incorporados al diseño del programa? ¿Qué nivel de flexibilidad existe por parte de USAID para redefinir los objetivos en función de los cambios en los requerimientos de los demandantes? Gestión (Implementación del programa) ¿Qué fortalezas y debilidades existen en el proceso de implementación del Programa PAT-MINAM? 6-Estándares de la Planificación ¿El Plan de Monitoreo y Evaluación y los Planes operativos siguen las recomendaciones de USAID forward? ¿Se cumplen con los procesos de definición de los planes operativos y los planes de contratación exigidos por la administración pública y su sistema de control? 7-Gestión de las comunicaciones y de la calidad ¿Cómo se distribuyen las responsabilidades para el desarrollo de los informes de monitoreo? ¿Qué tipo de comunicación (informal-formal, escrita-oral) existen con los interesados del proyecto? ¿Cómo se distribuyen las responsabilidades en la revisión de la calidad de los servicios contratados? 8-Control de costos ¿Cómo se encuentran los procesos de ejecución presupuestal y los procesos de desembolso de fondos? ¿Cómo se reacciona frente a los retrasos en la ejecución de las actividades previstas y del presupuesto del programa? 81 EJES PREGUNTA GENERAL TEMAS PREGUNTAS ESPECÍFICAS 9-Riesgos y gestión de cambios ¿Cómo se gestionan los cambios del proyecto? ¿Cuáles cambios serían considerados pequeños, medianos y grandes? ¿Se han identificado los riesgos del Programa y se tiene planes de contingencia previstos? 10-Gestión del capital humano del programa ¿El organigrama del Programa contiene todos los cargos que se requieren para facilitar la coordinación entre los diferentes niveles? ¿El perfil y las funciones de los miembros del equipo del programa están claramente definidos según los resultados/entregables que se buscan? ¿Ello facilita los niveles de coordinación con los demandantes? ¿Cuáles son los incentivos para el mejor desempeño del equipo? ¿Qué estrategias y actividades de desarrollo del equipo existen y que faciliten superar las debilidades y amenazas? Eficacia (Indicadores) ¿Cuál es la eficacia del PAT en esta etapa intermedia y qué factores criticos deben ser abordados para mejorarla? 11-Logro de resultados ¿Los productos logrados durante la primera etapa de ejecución contribuyen para lograr los resultados esperados del PAT? ¿Cuál es el nivel de logros de los indicadores señalados en el Tracking Table del PM&E? ¿Qué exitos y fracasos se revelan en relación con el PAT y su implementación? 12-Calidad de indicadores y lecciones aprendidas ¿Los PIRS están completos y se han revisado según las orientaciones de USAID forward? ¿Se revisa la calidad de los indicadores utilizados? ¿Cómo se registran las lecciones aprendidas frente al desarrollo de las metas del Programa? Fuente: Elaboración propia. 82 6. DISEÑO DE LA EVALUACIÓN Y METODOLOGÍA Respecto al diseño de la evaluación y la metodología, el PAT USAID/MINAM requiere que la evaluación genere evidencia creíble e información de alta calidad, que corresponda a las preguntas de evaluación6. Considerando que la evaluación se basa en preguntas formuladas con el fin de que sean verificables empíricamente y que están orientadas a conocer aspectos relacionados con la eficacia de la implementación del programa, la estrategia general para responder estas preguntas consiste en la revisión y análisis de datos existente y evidencia fáctica respecto al desempeño del mismo. Diseño de la evaluación La metodología de la evaluación intermedia considera dos procesos igualmente importantes y complementarios: la revisión de información secundaria referente a los logros de cada proyecto y la obtención de información primaria, a través de entrevistas con los diferentes actores relacionados con el Programa. Esta información será analizará en función de las preguntas planteadas y se harán recomendaciones orientadas a mejorar la toma de decisiones interna del PAT. En esta sección se detalla la metodología y los pasos a seguir para lograr responder a las preguntas pertinentes a una evaluación intermedia planteadas en la sección anterior. Para ello, se requiere analizar los principales grupos de actores identificados:  USAID.  USAID, Evaluation  Equipo del PAT.  Viceministros involucrados con el PAT.  Direcciones Generales del MINAM.  Gobiernos regionales.  Municipalidades.  Universidades.  MEF.  SENAMHI.  Otros actores que consideren relevante en el Plan de trabajo. Para entender el rol de los actores más cercanos al PAT USAID/MINAM revisar el anexo 7.1. 6 PGRD, p. 7 83 MÉTODOS DE OBTENCIÓN DE INFORMACIÓN Se enfatiza a continuación los tipos de fuentes que se tendrán para responder cada una de las preguntas, así como los actores involucrados en este ámbito. PREGUNTAS ESPECÍFICAS USAID PAT MINAM GORES Munici￾palidades Univer￾sidades MEF SENAMHI ¿En qué consiste y cuál fue el contexto de selección del modelo de cooperación G2G de USAID? ¿Qué relación hay entre este modelo de USAID y los acuerdos o estándares internacionales en materia de cooperación internacional? ¿Las contrapartes conocen sobre esta forma de cooperación? Entrev/Doc Entrev x x x x x x ¿Qué oportunidades y riesgos implican para el Programa el tipo de instrumentos específicos (como cartas de ejecución) a partir de los cuales se ejecutan los convenios de cooperación 423 y 426? Entrev/Doc Entrev x x x x x x ¿Cómo influye la existencia de otros modelos de cooperación? ¿Cuál es su ventaja y desventaja con otros modelos de cooperación internacional? ¿Cuál es la prioridad del PAT frente al contexto de cooperación del MINAM? Doc Entrev Entrev/Do c x x x x x ¿Qué nuevas oportunidades y riesgos implican las nuevas tendencias de modernización de las herramientas de gestión de USAID (USAID forward) para este tipo de modelo de cooperación de USAID? Doc Entrev x x x x x x ¿Qué oportunidades y riesgos implican los nuevos acuerdos internacionales sobre temas ambientales que se consolidarán en la COP20 que será en Lima en diciembre? x Entrev Entrev/Do c x x Entrev/D oc x x 84 PREGUNTAS ESPECÍFICAS USAID PAT MINAM GORES Munici￾palidades Univer￾sidades MEF SENAMHI ¿Qué oportunidades y riesgos implican las nuevas tendencias de modernización del sector público peruano que pueden afectar el desarrollo de programas enmarcados en este modelo de cooperación de USAID? (ley de contrataciones, CEPLAN, PPR, etc.) Como modelo puede tener ventajas x Entrev Entrev/Do c x x x Doc x ¿Qué factores críticos existen para este tipo de modelo de cooperación de USAID en la consolidación del MINAM como autoridad nacional y en su vínculo con otros sectores de la administración pública y del entorno nacional? x Entrev Entrev x x x x x ¿El procedimiento de diseño del PAT siguió las recomendaciones de USAID y del SNIP? ¿Qué tipo de escenarios sobre el contexto utilizaron para su implementación? ¿Qué tan realistas fueron los objetivos y metas en el diseño del PAT y por qué? ¿Se conoce el documento del diseño del PAT elaborada por el SASE? Entrev/Doc Entrev/ Doc Entrev/ Doc x x x x x ¿Son consistentes los resultados intermedios esperados del PAT MINAM con el nuevo marco de resultados del Objetivo de Desarrollo 3 de USAID para el Perú, y con los objetivos de las cláusulas ambientales del TLC? Doc Entrev/ Doc x x x x x x ¿Son consistentes los resultados intermedios esperados del PAT MINAM con los planes del MINAM (PESEM, PEI, Planes Nacionales, entre otros? Doc x Entrev x x x x x ¿Los indicadores planteados en el marco lógico de los proyectos son consistentes con los objetivos esperados para el programa PAT y para los indicadores estándares de USAID? Doc x x x x x x x 85 PREGUNTAS ESPECÍFICAS USAID PAT MINAM GORES Munici￾palidades Univer￾sidades MEF SENAMHI ¿Qué nivel de satisfacción sobre el marco de resultados presentan los involucrados en los proyectos del PAT? ¿Cuáles fueron los principales cambios en los requerimientos de los demandantes y cómo fueron incorporados al diseño del programa? x Entrev x x x x x x ¿Qué nivel de flexibilidad existe por parte de USAID para redefinir los objetivos en función de los cambios en los requerimientos de los demandantes? Entrev Entrev Entrev x x x x x ¿El Plan de Monitoreo y Evaluación y los Planes operativos siguen las recomendaciones de USAID forward? Doc Entrev/ Doc x x x x x x ¿Se cumplen con los procesos de definición de los planes operativos y los planes de contratación exigidos por la administración pública y su sistema de control? x x x Entrev Entrev x x x ¿Cómo se distribuyen las responsabilidades para el desarrollo de los informes de monitoreo? ¿Qué tipo de comunicación (informal-formal, escrita-oral) existen con los interesados del proyecto? x Entrev x Entrev Entrev Entrev Entrev Entrev ¿Cómo se distribuyen las responsabilidades en la revisión de la calidad de los servicios contratados? x Entrev x Entrev Entrev Entrev Entrev Entrev ¿Cómo se encuentran los procesos de ejecución presupuestal y los procesos de desembolso de fondos? x Entrev/ Doc x Entrev Entrev x x x ¿Cómo se reacciona frente a los retrasos en la ejecución de las actividades previstas y del presupuesto del programa? x Entrev x Entrev Entrev x x x ¿Cómo se gestionan los cambios del proyecto? ¿Cuáles cambios serían considerados pequeños, medianos y grandes? x Entrev x Entrev Entrev x x x ¿Se han identificado los riesgos del Programa y se tiene planes de contingencia previstos? x Entrev/ Doc x x x x x x 86 PREGUNTAS ESPECÍFICAS USAID PAT MINAM GORES Munici￾palidades Univer￾sidades MEF SENAMHI ¿El organigrama del Programa contiene todos los cargos que se requieren para facilitar la coordinación entre los diferentes niveles? x Entrev/ Doc x Entrev Entrev Entrev Entrev Entrev ¿El perfil y las funciones de los miembros del equipo del programa están claramente definidos según los resultados/entregables que se buscan? ¿Ello facilita los niveles de coordinación con los demandantes? x Entrev/ Doc x Entrev Entrev x x x ¿Cuáles son los incentivos para el mejor desempeño del equipo? ¿Qué estrategias y actividades de desarrollo del equipo existen y que faciliten superar las debilidades y amenazas? x Entrev x Entrev Entrev x x x ¿Los productos logrados durante la primera etapa de ejecución contribuyen para lograr los resultados esperados del PAT? x Entrev/ Doc Entrev Entrev Entrev Entrev Entrev Entrev ¿Cuál es el nivel de logros de los indicadores señalados en el Tracking Table del PM&E? ¿Qué exitos y fracasos se revelan en relación con el PAT y su implementación? x Entrev/ Doc x Entrev Entrev Entrev Entrev Entrev ¿Los PIRS están completos y se han revisado según las orientaciones de USAID forward? ¿Se revisa la calidad de los indicadores utilizados? x Entrev/ Doc x Entrev Entrev x x x ¿Cómo se registran las lecciones aprendidas frente al desarrollo de las metas del Programa? x Entrev x Entrev Entrev x x x Nota: x significa que no es pertinente. 87 A. Revisión de información secundaria En relación con la revisión de información secundaria, por un lado, según se ha señalado en la sección 1.7, esta consiste en la información generada por los procesos de evaluación del Programa, los cuales incluyen datos de seguimiento de las metas esperadas para cada Proyecto, de acuerdo con el Marco de Resultados. Esta información se plasma en los reportes trimestrales y anuales del PAT, los registros de eventos, los documentos relacionados con visitas in situ, así como en los documentos que son los resultados de las actividades del programa, como diseños, estudios, metodologías, guías, entre otros. Es importante enfatizar que el único indicio de línea de base está en el tracking table. Por otro lado, se revisará información secundaria relacionada a factores que podrían explicar las razones por las cuales se dificultó la ejecución del Programa y el logro de los resultados obtenidos. Esta información se refiere tanto al contexto en el cual está el Programa, como al diseño y a las operaciones del PAT, lo cual se plasma en documentos como el de Cooperación Gobierno a Gobierno (G2G), los Convenios de Donación 0423 y 0426, bibliografía relacionada con otros estándares y convenciones sobre cooperación internacional, así como documentos internos del diseño y las operaciones del programa. La información secundaria se deberá ir complementando a lo largo de la evaluación, cuando se debe consultar la nueva documentación que los actores provean al equipo de trabajo. Es importante cuidar para evaluar la calidad de la evaluación y contrastar esta información con los resultados de las entrevistas. B. Obtención de información primaria Esta tarea se centra en entrevistas semi-estructuradas a representantes de los diferentes grupos de interés del PAT, como USAID, las diferentes direcciones del MINAM, el MEF, así como representantes de los diferentes Gobiernos Regionales involucrados, como Ucayali, Madre de Dios, Loreto, Amazonas y Cusco, entre otras instancias públicas e instituciones involucradas en la implementación del Programa y sus diferentes Proyectos. Estos actores han sido seleccionados en base a un mapeo de actores relevantes para el PAT, tomando en cuenta sus roles, posiciones e intereses respecto al Programa (Ver Anexo 2). Las entrevistas estarán orientadas a buscar los elementos que permitan identificar los obstáculos a la ejecución del programa, entendiendo la situación problemática relacionada con el cumplimiento de los resultados esperados en el marco de resultados del Programa y en conocer las razones que determinan el desempeño del Programa. En este sentido, se desarrollarán entrevistas específicas para cada grupo de interés, en función de las preguntas señaladas en el punto 3.2 y su tipo de relacionamiento con el PAT. Estas entrevistas deben ser registradas en la medida de lo posible con fotos y grabaciones, desde que se tenga el consentimiento informado del interlocutor. La información primaria se obtendrá siguiendo la estructura del modelo estandarizado para cada grupo de interés, pero buscarán también profundizar en lo que sea más relevante para el objetivo de la evaluación, esto es, sin que se pierda la estructura de la información que se busca levantar. Cabe resaltar que estas entrevistas serán validadas con representantes del PAT USAID/MINAM. Por ello es importante que sean grabadas en audio, con consentimiento del entrevistado, y se transcribirán (las más relevantes) para su posterior anexo al informe de la evaluación. Para llevar a 88 cabo el trabajo de campo se requerirá el apoyo del PAT y de USAID con el fin de realizar las coordinaciones con los entrevistados y acordar reuniones con ellos dentro los lapsos requeridos por el cronograma. Asimismo, se definirán los viajes necesarios para reuniones con representantes en las distintas regiones del país. Cabe resaltar que las entrevistas se realizarán con el fin de obtener evidencia respecto a las preguntas planteadas, en un espacio de cordialidad, apertura y objetividad, evitando actitudes inquisitivas. Es importante informar que el anonimato y la grabación de audio son opcionales, pero se requiere consentimiento informado (tiene un formato) y se recomienda el registro fotográfico. Métodos de análisis de la información Para la sistematización y análisis de información cualitativa, se utilizará el software Atlas Ti, que permite ordenar la información cualitativa en unidades hermenéuticas y clasificarlas según los 12 temas sobre los cuales se buscarán hallazgos que lleven a conclusiones y sostengan recomendaciones para la mejor efectividad del Programa. Esta herramienta permitirá sistematizar los datos de manera que sea posible analizar más fácilmente la información en miras a encontrar resultados que respondan a las preguntas de investigación. Para alcanzar estos resultados, se utilizará una matriz de análisis en la que se categorizará la información encontrada de acuerdo con las preguntas de evaluación establecidas para alcanzar los objetivos específicos y con ellos el resultado general de la evaluación. Fortalezas y debilidades metodológicas Se requiere señalar las fortalezas y debilidades del proceso de levantamiento de información primaria y secundaria que se llevará a cabo durante la evaluación de medio término del PAT USAID/MINAM. En relación a las fortalezas, se puede señalar:  Se ha definido las preguntas de la evaluación, tomando en consideración el objetivo de la evaluación, los ejes pertinentes que dan origen a las preguntas generales, las temáticas más relevantes para encontrar hallazgos, conclusiones y recomendaciones, así como las preguntas específicas de la evaluación.  La definición se hizo de manera participativa con los actores clave al cual va dirigida las recomendaciones de la evaluación, esto es, USAID, la dirección del PAT USAID/MINAM y la alta dirección del MINAM.  Para cada pregunta específica se detalla los actores involucrados a ser entrevistado (fuente primaria), mientras para cada tema general se señaló las referencias pertinentes (fuentes secuendarias). Sobre las debilidades, se identifica que:  Las entrevistas puede ser conducidas con cierta subjetividad, lo cual implicaría sesgar las respuestas. Aunque es importante dar espacio para la reflexión adicional y el acercamiento entre los interlocutores, se debe enfatizar lograr con objetividad la finalidad de los cuestionarios. En este sentido, todo el equipo que realice el trabajo de cambio debe pasar por una inducción para mantener esta objetividad.  Como el estudio se concentra en el fortalecimiento de capacidades del MINAM, y este tema tiene muchos tipos de interpretaciones, especialmente en lo que a logros de más alto nivel 89 concierne. Es importante tener bien clara la secuencia de objetivos y el flujograma de causalidad que condiciona los indicadores de medición del desempeño.  Hubieron cambios en los protagonistas vinculados al PAT, tanto por parte de USAID como por parte del MINAM, lo cual puede complicar entender las etapas iniciales del programa. En este sentido, se buscarán algunas contingencias para superar los problemas encontrados durante la revisión de la información secundaria y la aplicación de las entrevistas. Durante la elaboración del informe final, se dará énfasis a la comparación entre las reflexiones obtenidas de la documentación y aquellas que se pudieran recabar con las entrevistas. 7. PRODUCTOS Entregables A. Plan de trabajo. B. Informe del trabajo de campo. C. Informe Final preliminar. D. Informe Final. N° Entregables Tiempo estimado 1 Plan de trabajo con mayor detalle de las fuentes de información, herramientas de recolección de información (cuestionarios por grupos de interés), revisión con los actores del propósito de la evaluación para su retroalimentación 9 días 2 Informe del trabajo de campo, con presentación preliminar de los resultados, retroalimentación de los destinatarios de la evaluación 42 días 3 Informe final preliminar, que debe ser presentado a los destinatarios con conclusiones y recomendaciones, las cuales pasarán por revisión que deben reflejarse en sugerencias para la mejora del documento. 25 días 4 Informe final, con la incorporación de las sugerencias y que debe ser aprobado. 14 días 5 Informe final en inglés 10 días Plan de trabajo El plan de trabajo tiene un plazo de 80 días efectivos distribuidos en cinco etapas de actividades: 1. Elaboración del Plan de Trabajo: En esta fase se tendrán las reuniones preliminares y la preparación de herramientas para el trabajo de campo, lo cual requerirá la aprobación de USAID. Esta etapa se prevé que dure nueve días útiles de todo el equipo. 2. Revisión y análisis de información secundaria: En esta etapa se listará la información recibida y se categorizará en el programa Atlas ti, según las temáticas a partir de las cuales se deben encontrar hallazgos, que sentarán las bases de las conclusiones y de las recomendaciones fundamentadas. Esta etapa se prevé que dure diez días útiles de todo el equipo. 3. Trabajo de campo: Primero se empezará con la revisión de la información secundaria, que es trabajo de gabinete -no trabajo de campo-, pero que afinará el conocimiento para el 90 desarrollo de las entrevistas a los diversos actores involucrados. Se prevé la entrega de un informe de trabajo de campo que debe ser revisado por los involucrados directos en la evaluación de medio término para que se tenga retroalimentación. Esta etapa se prevé que dure cuarenta y dos días útiles de todo el equipo. 4. Elaboración del informe final: Primero se hará la comparación de la información secundaria, obtenida con la documentación revisada, con la realización del trabajo de campo, donde se recabará información primaria. Primero se entregará un informe final preliminar, que debe recibir retroalimentación, para después poder elaborar el informe final revisado. Esta etapa se prevé que dure treinta y nueve días útiles de todo el equipo más 10 días de traducción del documento al inglés. 5. Gestión del equipo: también es importante contabilizar el esfuerzo que el equipo debe realizar para las reuniones de coordinación, especialmente para la aprobación de los entregables A continuación se presenta el listado de las actividades y de los entregables de la evaluación de medio término del PAT USAID/MINAM que representa el cronograma del Plan de trabajo, suponiendo el inicio de las actividades el 13 de octubre y con la entrega del informe final revisado el 30 de enero. Tabla. Evaluación de medio término del PAT USAID/MINAM: Tareas para la implementación 1 Plan de Trabajo 1.1 Reuniones preliminares 1.2 Preparación de herramientas de campo 1.3 Elaboración y aprobación del Plan de trabajo E1- Plan de Trabajo 2 Revisión de información secundaria 2.1 Revisión de información secundaria 2.2 Clasificación de información por temas y categorías 3 Trabajo de campo 3.1 Coordinaciones para trabajo de campo 3.2 Realización de entrevistas PAT 3.3 Realización de entrevistas USAID 3.4 Realización de entrevistas Vice ministros MINAM 3.5 Realización de entrevistas Directores MINAM 3.6 Realización de entrevistas GORE y Municipalidades 91 3.7 Realización de entrevistas Universidades 3.8 Realización de entrevistas otros actores del Sector público peruano 3.9 Elaboración de informe del trabajo de campo 3.10 Presentación del informe del trabajo de campo para su retroalimentación E2- Informe del Trabajo de campo 4 Elaboración del informe final 4.1 Sistematización y análisis de información primaria en contraste con la secundaria 4.2 Modelamiento de resultados 4.3 Elaboración del informe final preliminar 4.4 Presentación del informe final preliminar E3- Informe final preliminar 4.5 Elaboración de informe final revisado E4- Informe final revisado en español 4.6 Traducción del informe final E5- Informe final en inglés 5 Gestión de la Consultoría 5.1 Reunión de inicio y cierre 5.2 Reuniones del equipo 5.3 Reuniones de coordinación con Evaluation para aprobación de entregables 5.4 Reuniones de coordinación con USAID para aprobación de entregables Informe del trabajo de campo Al final del proceso de levantamiento de información primaria a partir de las entrevistas se elaborará una presentación en Power Point con una presentación con el informe del trabajo de campo para ser presentada a las partes y recibir retroalimentación. Este representa un hito muy relevante para la consultoría en la medida que se espera que la retroalimentación facilite no solo la reflexión sobre los hallazgos y las conclusiones, sino aporte también en las ideas para las recomendaciones para incrementar la efectividad de la ejecución del Programa. El contenido del informe del trabajo de campo debe contar con los siguientes elementos: 1. Presentación con antecedentes. 2. Metodología del trabajo de campo. 92 3. Por cada uno de los 12 temas los principales hallazgos (que incentiven una reflexión durante la presentación sobre las conclusiones y las recomendaciones) 4. Conclusiones generales. Informe final Tal como se puede apreciar en el cronograma de la evaluación (sección 6.2), a lo largo de las diferentes etapas se irá elaborando el informe final de evaluación. Este incluirá en detalle todo el proceso de evaluación, el procesamiento de la información del trabajo de campo, contrastado con el levantamiento de información con lo cual se tendrá el sustento de los resultados alcanzados, con los aspectos saltantes durante la evaluación. De este modo, se busca encontrar hallazgos, que sustente las respectivas conclusiones y facilite el marco para proponer recomendaciones para mejorar la implementación del PAT USAID/MINAM. En relación con las recomendaciones, estas se orientarán a responder las preguntas: 1. ¿qué se recomienda? 2. ¿por qué? 3. ¿cómo se debe llevar a cabo? El informe final contará con información sistematizada ilustrada con esquemas, gráficos y tablas que permitan lectura sintética y objetiva del mismo. A continuación se encuentra la estructura propuesta del informe final: 1. Resumen Ejecutivo. 2. Antecedentes. 3. Metodología utilizada. 4. Por temas: 4.1. Hallazgos. 4.2. Conclusiones. 4.3. Recomendaciones. 5. Conclusiones y recomendaciones generales. 6. Anexos. El informe final deberá contar con una presentación en Power Point que facilite la retroalimentación con los involucrados del informe final preliminar. El informe final revisado debe seguir una estructura similar al preliminar y responder a las sugerencias que hayan enriquecido el documento inicial. Tal como se aprecia en el cronograma, el informe se entregará en la semana 16 de la consultoría, en idioma castellano, en versión borrador, formato digital (PDF); para luego levantar las observaciones y comentarios de USAID y proceder a elaborar la versión final del informe, a entregarse en la semana 18. Se debe entregar el informe en 3 copias en formato impreso y en CD. El archivo Word no debe sobrepasar las 60 páginas sin anexos, mientras la presentación debe ser calculada para 30 minutos. Las entrevistas más relevantes deben ser transcritas y consideradas en anexos, lo cual se puede definir preliminarmente en el plan de trabajo, pero retroalimentar con la presentación del informe del trabajo de campo. 93 8. COMPOSICIÓN DEL EQUIPO La metodología propuesta condiciona la definición de los especialistas, revisores temáticos y personal de apoyo a ser contratados. En primer lugar, se requiere un equipo multidisciplinario con una coordinación de la consultoría que lo haga trabajar de manera interdisciplinar. En este sentido, se requerirán especialistas en las siguientes temáticas: Evaluadores principales i. Coordinación: Especialista en evaluación (Marta Tostes) ii. Gestión social y ambiental (Diego Espinosa). iii. Gestión de políticas públicas en temas ambientales (Augusto Castro). Revisores temáticos iv. Políticas públicas (a definir). v. Servicios ambientales y valoración económica (Fernando León a.c.). vi. Conservación de bosques (Daniel Hernández a.c.). Asistentes del trabajo de campo vii. Asistente de trabajo de campo 1(a definir). viii. Asistente de trabajo de campo 2(a definir). 9. GESTIÓN DE LA EVALUACIÓN Requerimientos Se tiene una serie de requerimientos para la optimización de la elaboración del informe final: 1. Que se faciliten los documentos del Programa y cualquier otro documento que sea de relevancia para el desarrollo de la consultoría, dada la relevancia de la información secundaria en la metodología de búsqueda de hallazgos con indicios te temas que puedan obstaculizar la ejecución del proyecto. 2. Que el equipo de USAID revise adecuadamente los informes parcial en un plazo razonable de 5 días, para que no afecte el desarrollo de las actividades previstas y el cumplimiento de los compromisos respecto a los plazos. 3. Que se facilite la conexión con los interlocutores, con cartas de presentación del equipo, así como recomendación para que en las fechas señaladas en el plan de trabajo sean realizadas las entrevistas. 4. Para las entrevistas y para las presentaciones del informe del trabajo de campo e informe final preliminar se requerirán espacios apropiados para su realización. En el caso de las presentaciones, se requerirá un espacio con multimedia. Cronograma A continuación se presenta el diagrama de Gantt de la evaluación de medio término del PAT USAID/MINAM que representa el cronograma del Plan de trabajo, suponiendo el inicio de las actividades el 13 de octubre y con la entrega del informe final revisado en español el 30 de enero, mientras la traducción revisada el 13 de febrero. Es importante resaltar las siguientes restricciones respecto al cronograma: 94 1. Realización de la COP 20 en Lima, que será del 2 al 12 de diciembre, con lo cual los equipos del MINAM estarán volcados a su preparación a partir de mediados de noviembre. 2. El período vacacional del sector público entre Navidad y Año nuevo. Presupuesto El presupuesto se presentará tomando en consideración: horas-hombre de trabajo de los miembros del equipo, así como los viáticos y traslados de los viajes previstos a Amazonas, Junín, Ucayali. En primer lugar el cálculo del presupuesto variará según la experiencia de los especialistas a contratar, pero se estima su dedicación por días. 95 Annex III: Secondary information sources A continuación, se lista los documentos que han sido revisados para la evaluación intermedia: 1-Contexto nacional e internacional de la cooperación:  GUEVARA, Susana. Mapeo de Proyectos del MINAM. Evaluation, Junio de 2014.  MINAM. Reporte de monitoreo. Proyectos de Cooperación ejecutados por el Ministerio del Ambiente al primer semestre de 2013. Lima, octubre de 2013.  Resolución Ministerial N° 2022012-MINAM.Lima 13 Agosto 2012.  Resolución Ministerial N° 181-2014-MINAM. Lima 23 Junio 2014. 2-Cambios en la agenda nacional  CEPLAN. Directiva General del Proceso de Planeamiento Estratégico. SINAPLAN. Directiva 001-2014- CEPLAN, Abril 2014.  MEF. Directiva N° 0001-2014-EF/50.01. “Directiva para los programas presupuestales en el marco de la programación y formulación del presupuesto del sector público para el año fiscal 2015”. Enero 2014.  SISTEMA DE NACIONES UNIDAS. La ONU Y LA COP 20. Boletín. 1 edición-Abril 2014.  MINAM. Plan Nacional de Acción Ambiental 2011-2021. 3-Diseño inicial  MINAM. Decreto Legislativo N° 1013, Ley de creación del Ministerio del Ambiente.  MINAM. Plan Estratégico Sectorial Multianual, 2013-2016.  MINAM. Cartera Estratégicas Multianual de Proyectos de Inversión Pública. Noviembre de 2013. 4-Medición de desempeño  PAT USAID/MINAM. Reporte de Gestión Trimestral, Enero a Marzo de 2013. Lima, 2013.  PAT USAID/MINAM. Reporte de Gestión Semestral, Enero a Junio de 2013. Lima, 2013.  PAT USAID/MINAM. Reporte de Gestión, Enero a Setiembre de 2013. Lima, 2013.  PAT USAID/MINAM. Reporte de Gestión Semestral. Enero a Marzo 2014, Lima 2014.  PAT USAID/MINAM. Reporte de Gestión Semestral. Enero a Junio 2014, Lima 2014. 5-Gestión de cambios del marco de resultados  PAT USAID/MINAM. Plan de Monitoreo y Evaluación. Lima, Agosto de 2013. 6-Estándares de la Planificación  PAT USAID/MINAM. Plan de Monitoreo y Evaluación. Lima, Agosto de 2013. 7-Gestión de las comunicaciones y de la calidad  PAT USAID/MINAM. CARTA N" 027 - 2013-MINAM-SEG-PAT-USAID/MINAM; de entrega del Plan de Monitoreo y Evaluación. Lima, Agosto de 2013. 8-Control de costos  PAT USAID/MINAM. Reporte de Gestión Trimestral, Enero a Marzo de 2013. Lima, 2013.  PAT USAID/MINAM. Reporte de Gestión Semestral, Enero a Junio de 2013. Lima, 2013.  PAT USAID/MINAM. Reporte de Gestión, Enero a Setiembre de 2013. Lima, 2013.  PAT USAID/MINAM. Reporte de Gestión Semestral. Enero a Marzo 2014, Lima 2014.  PAT USAID/MINAM. Reporte de Gestión Semestral. Enero a Junio 2014, Lima 2014. 9-Riesgos y gestión de cambios 96  PAT USAID/MINAM. Plan de Monitoreo y Evaluación. Lima, Agosto de 2013.  PAT USAID/MINAM. Reporte de Gestión Trimestral, Enero a Marzo de 2013. Lima, 2013.  PAT USAID/MINAM. Reporte de Gestión Semestral, Enero a Junio de 2013. Lima, 2013.  PAT USAID/MINAM. Reporte de Gestión, Enero a Setiembre de 2013. Lima, 2013.  PAT USAID/MINAM. Presentación PAT para USAID. Diapositivas. Lima, Agosto 2014.  PAT USAID/MINAM. Reporte de Gestión Semestral. Enero a Marzo 2014, Lima 2014.  PAT USAID/MINAM. Reporte de Gestión Semestral. Enero a Junio 2014, Lima 2014. 10-Gestión del capital humano del programa  PAT USAID/MINAM. Organigrama PAT en el Ministerio. Diapositivas. Lima, Agosto 2014. 11-Logro de resultados  PAT USAID/MINAM. Plan de Monitoreo y Evaluación. Lima, Agosto de 2013.  PAT USAID/MINAM. Reporte de Gestión Trimestral, Enero a Marzo de 2013. Lima, 2013.  PAT USAID/MINAM. Reporte de Gestión Semestral, Enero a Junio de 2013. Lima, 2013.  PAT USAID/MINAM. Reporte de Gestión, Enero a Setiembre de 2013. Lima, 2013.  PAT USAID/MINAM. Reporte de Gestión Semestral. Enero a Marzo 2014, Lima 2014.  PAT USAID/MINAM. Reporte de Gestión Semestral. Enero a Junio 2014, Lima 2014. 12-Calidad de indicadores y lecciones aprendidas  PAT USAID/MINAM. Plan de Monitoreo y Evaluación. Lima, Agosto de 2013.  PAT USAID/MINAM. Reporte de Gestión Trimestral, Enero a Marzo de 2013. Lima, 2013.  PAT USAID/MINAM. Reporte de Gestión Semestral, Enero a Junio de 2013. Lima, 2013.  PAT USAID/MINAM. Reporte de Gestión, Enero a Setiembre de 2013. Lima, 2013.  PAT USAID/MINAM. Reporte de Gestión Semestral. Enero a Marzo 2014, Lima 2014.  PAT USAID/MINAM. Reporte de Gestión Semestral. Enero a Junio 2014, Lima 2014. 97 Annex IV. Primary information sources Grupo de Interés Actor Cargo Institución USAID Bernardo Espinoza Analista Financiero USAID Fernando Chávez Coordinador técnico USAID USAID Juan Robles Jefe Adjunto de Oficina y Coord. Proyecto PAT ACC USAID Álvaro Gaillour Administrador Proyecto PAT USAID/MINAM USAID Christopher Moore Coordinador SEIA y Bosques USAID PAT MINAM Juan Carlos Flores Coordinador Ejecutivo PAT PAT USAID/ MINAN Luis Albán Coordinador PAT Bosques PAT USAID/ MINAN Raquel Soto Coordinador PAT - SEIA PAT USAID/ MINAN Parlos Palomares Coordinador PAT - Valoración PAT USAID/ MINAN Fabiola Carreño Coordinación PAT CITES PAT USAID/ MINAN Rosa Salas Coordinador PAT ACC PAT USAID/ MINAN Rosario Ponce Cubias Especialista Administración y Finanzas - PAT PAT USAID/ MINAM Madeleine Obando Especialista en Monitoreo y Evaluación PAT USAID/ MINAM MINAM Alta Dirección Ruperto Taboada SG - MINAM MINAM - Alta dirección Mariano Castro Viceministro de Gestión Ambiental MINAM - DGPNIGA José González Oficina de Cooperación y Negociación Internacional MINAM - Alta dirección Juana Muñoz OGA MINAM - Alta dirección Raquel Angulo Oficina de Planeamiento y Presupuesto MINAM - Alta dirección Katherine Pardo Asesora Secretaria General MINAM - Alta dirección Gabriel Quijandría Acosta Viceministro de Desarrollo Estratégico de los RRNN MINAM-Alta Dirección PAT CITES Lucas Benites Perú Bosques Perú Bosques Victor Miyakawa PFSI PFSI José Alvarez Alonso Director General de DGDB MINAM - DGDB Pedro Vasquez Ruesta Jefe del CDC Centro de Datos para la Conservación - UNALM Letty Salinas Jefa del departamento de Ornitología del MHN Museo de Historia Natural UNMSM Carlos Reynel Rodriguez Jefe del Herbario MOL de la UNALM Herbario - UNALM PAT Valoración Juan Moncada Alvites Grte. Autoridad Regional Ambiental GORE Amazonas Pedro Baras MINAM GORE Amazonas 98 Grupo de Interés Actor Cargo Institución Oscar Rubén Raqui Especialista de la Gerencia de Medio Ambiente Muni. Provincial de Satipo Juan Pablo Ferreyros Coordinador Técnico de Regencia Forestal ONG AIDER Franz Tang Jara Grnt. RRNNy Gest. Ambiente GORE Ucayali Danny Perez Supervisor de limpieza pública Muni. Dist.de Manantay Christy Mendez Poma Gerente Regional de RRNN y Ges Amb. GORE Junín Lidia Berrocal Asistente de proyecto DEVIDA Muni. Dist. de Rio Negro (Satipo) Wilfredo León Docente de la Facultad de Agronomía UNCP Nilda Huari Pastrana Sub Gerente de Desarrollo Económico Productivo Muni. Dist. de Mazamari Yoseph Ojeda Docente de la especilidad de Ing. Ambiental Universidad Nacional Daniel Alcides Carrión Rita Girón Especialista en Recursos Geneticos INIA José Gonzales Especialista en RRNN GORE Ucayali Carlos Fachín Mattos Decano de Ciencias Forestales Universidad Nacional de Ucayali Carlos Orihuela Romero Especialista en Valoración Económica Universdad Nacional Agraria de la Molina Fresia Dávila Ramírez Asistente de Asesoramiento en CCNN GORE Ucayali - DEF Sonia Braga Vásquez Asistente de Protección Ambiental y RRCC PERUPETRO Milagros Dominguez Asesora REFOREST Concesionaria Ivonne Rose Maldonado Área de planificación en Gerencia de RRNN GORE Ucayali Roger Loyola Gonzales Director General MINAM - DGEVFPN Eduardo García Zamora Especialista en Valoración Económica MINAM - DGEVFPN PAT SEIA Amalia Cuba Salerno Directora General de DPNIGA MINAM - DGPNIGA Ángel Chávez Mendoza Director DGAAM MINEM -DGAAM Raúl Rabelo Salva Coordinador del SEIA MINAM - DGPNIGA Efrain Samochuallpa Solis Gerente Regional de RRNN y Gestión del Medio Ambiente GORE Cusco PAT Bosques Jaime Semizo Jefe de la Oficina Zonal de Selva Central Satipo PNCB 99 Grupo de Interés Actor Cargo Institución Rubén Martinez Especilista Oficina Zonal de Selva Central Satipo PNCB Gustavo Huamani Coordinador Técnico PNCB Mapeo y Monitoreo de Bosques Rolando Vivanco Especialista SIG PNCB Mapeo y Monitoreo de Bosques Daniel Castillo Especialista en Monitoreo PNCB Mapeo y Monitoreo de Bosques Yessica Armas Especialista en Inversiones MINAM – Oficina de Proyectos de Inversión Patricia Juarez Asistente Técnico MINAM VM Desarrollo Estratégico de RRNN Elizabeht Cardenas Especialista en Diversidad Biológica MINAM - DGDB María Luisa del Río Asesora Iniciativa Peruana Biodiversidad y Empresas MINAM VM Desarrollo Estratégico de RRNN Gustavo Suarez de Freitas Coordinador Ejecutivo PNCB Carlos Ynami Coordinador Adjunto PNCB Ruth Escarate Merino Coordinador Técnico PNCB Fortalecimiento de Capacidades Fernando Canchanya Coordinador Técnico PNCB Sistemas Productivos Sostenibles Claudio Manuín Jefe de la Oficina Zonal Condorcanqui PNCB Oficina Zonal Amazonas Wilfredo Valencia Coordinador Oficina Pachis Oxapampa PNCB Oficina Zonal Satipo PAT ACC Eduardo Durand Director de la DGCCDRH MINAM - DGCCDRH Laura Avellaneda Coord. Gestión de Riesgos del Cambio Climático MINAM - DGCCDRH Luzidnya Cerrón Esp. Gestión de Riesgos del Cambio Climático MINAM - DGCCDRH Gabriela Rosas Directora de Meteorología Aplicada SENAMHI Oscar Obando Director de Hidrología SENAMHI Coop. Intern. Fernando León Especialista en financiamiento GIZ Pro Ambiente 100 Annex VI: Statement of Differences Partners for Global Research and Development (PGRD), como responsable final de la evaluación, ha retirado una de las conclusiones y recomendaciones del Informe de Evaluación, por indicación USAID. Coordinaciones con MINAM: En el caso de las coordinaciones con el ministerio, no se tiene clara la figura del promotor (sponsor), que es fundamental para el PAT como enlace entre los coordinadores técnicos y las DG, especialmente cuando se deben buscar soluciones ágiles para los problemas de ejecución. Si no hay una atención cercana del promotor, se entorpecen los canales de comunicación, se dificulta la ejecución de las actividades previstas en el POA y la búsqueda de soluciones de problemas del PAT. Ello nuevamente puede tener su origen en las debilidades de las herramientas de gestión del PAT como la falta del Program Description. En este sentido, la recomendación se centra en comprometer en el Program Description al Secretario General, como promotor, en la búsqueda activa de soluciones de problemas de ejecución del PAT, desde la elaboración del POA, así como el seguimiento y evaluación de los proyectos y acciones comunes, facilitando los canales de comunicación entre los coordinadores técnicos del PAT y las DG. Así se asumiría el liderazgo en las coordinaciones y la mejora de las condiciones para el trabajo de los sectoristas de USAID 101 Annex VII: TAP Organizational chart 102 Annex VIII: Disclosure of Conflict of Interest U.S. Agency for International Development Av. La Encalada s/n, Santiago de Surco Lima, Perú