1 Final Report Performance Evaluation of PRODECENTRALIZATION II Client: USAID November 2012 2 Index Página Executive Summary 03 Introduction 13 Chapter 1: Peruvian Context 15 Chapter 2: The national focus: From policy to practice 20 2.1 The regulatory framework 20 2.2.1. Objectives of the project 20 2.1.2. The program’s contribution on legislative matters 20 2.1.3. Aspects that limited or facilitated the program’s action on that issue 23 2.2 Social Conflicts 24 2.2.1. Objectives of the project 24 2.2.2. The program’s contribution on the issue of social conflicts 25 2.2.3. Aspects limiting or facilitating the program’s action on this point 27 2.3 The contribution from a gender perspective 28 Chapter 3: Capacity Building 30 3.1 Objectives and indicators of the capacity-building program 30 3.2 Quantitative research 30 3.3 Qualitative evaluation of capacity building 41 3.4 Main findings 45 Chapter4 Evaluation of the program characteristics 47 Chapter 5 Conclusions 53 Chapter 6 Lessons learned 56 Appendix Theoretical framework of the study 59 3 Executive Summary The USAID/Peru ProDecentralization (PRODES) program supports the efforts of the Peruvian government in the decentralization process through the improvement of public policy and the regulatory framework, as well as strengthening the capacity of sub￾national governments. The purpose of this study is to assess the performance of ProDecentralization II during the period 2008 and 2012. The evaluation objectives were as follows:  Assess the effectiveness of ProDecentralization results to date at the national and sub-national (regional and local) levels.  Identify factors that facilitate or create challenges for the achievement of ProDecentralization objectives.  Identify lessons learned and make strategic recommendations for how USAID can better provide technical assistance to the GOP to strengthen the decentralization process. The study considered the following evaluation questions:  To what extent (quantitative and qualitative) has ProDecentralization II achieved overall results/outcomes with respect to: 1) improving the decentralization regulatory and policy framework; and 2) strengthening sub-national governments’ capacity to govern?  What were the primary obstacles (internal to the project and external) to achieving ProDecentralization’s expected outcomes?  Which are the main lessons learned and recommendations for USAID and the GOP for possible future interventions to improve decentralization? The study was carried out in the departments of Ucayali, San Martin and Ayacucho. In these regions, we focused on doing an assessment at the regional, provincial and district levels. With this framework in mind, we designed an assessment instrument that includes a qualitative component (in depth interviews) and quantitative research based on secondary data. In the case of the quantitative assessment, we employed a panel data approach. With USAID’s consent, the Assessment Team proposed that quantitative assessment be centered on two indicators (Y variables) to study the performance of the ProDecentralization Program in the regional governments of Ayacucho, San Martin, and Ucayali and in each of their respective provincial municipalities. The indicators were as follows: 4 a. Efficient investment budget execution; and b. Number of public investment projects approved. In addition, a qualitative study was conducted to identify limitations, bottlenecks and opportunities to achieving ProDecentralization’s objectives. Interviews were conducted at each level of government: national (29 interviews), regional (10 interviews per region) and local (8 interviews per region). The guidelines applied to these interviews can be found in the appendixes. The evaluation results are grouped below by evaluation question. ProDecentralization II Results Findings Regulatory Framework and Public Policy  ProDecentralization’s support on regulatory framework and public policy matters has been necessary– in the perception of the stakeholders themselves- but insufficient to promote key reforms. Political will on the part ofdecision makers is also necessary improve the decentralization process.  Establishing a working relationship with the Assembly of Regional Presidents was a positive aspect of ProDecentralization that will help drive further progress on regulatory and policy issues.  The Decentralization Compass is valued by stakeholders because it has opened up a space for interaction and coordination that was previously lacking.  While the quality of the Decentralization Compass was consistently highlighted by respondents, it is noteworthy that it failed to generate a consensus for an agenda to strengthen the decentralization process. Reaching this type of consensus is necessary to influence legislation.  The documents or contributions that were submitted to implement regulations, such as the Unified Administrative Procedures Text (TUPA) model and regulations governing associations were not mentioned by interviewed stakeholders. High turnover of personnel reduced the visibility and level of awareness of these contributions.  The Decentralization Map, intended to provide the capacity to monitor the decentralization process, was not mentioned as a reference document that clarifies policy reform for the Executive and Legislative Branches of the Peruvian Government.  Technical assistance to develop legislation related to the decentralization process was valued by stakeholders.  The assistance provided to the regional governments received greater recognition than that given to provincial and district municipalities; these more local 5 governments wanted greater input during discussions on the decentralization process. Social Conflicts  While national guidelines for conflict prevention have been developed and the organizational structure of the relevant government offices has been strengthened, these advances do not resolve fundamental underlying problems.  In the absence of a shared vision about how to achieve a system that responds well to dealing with conflicts, assessments by stakeholders of accomplishments to date and demands for future action are highly diverse.  The establishment of an office devoted to managing social conflicts should be supplemented by an effective reorganization, driven by a deeper analysis of conflict situations, of every relevant level of government.  The project is linked to capacity building in seven regionsassociated with the handling of social conflicts. Gender  The gender approach achieved positive results on personal and institutional (i.e. programs oriented at gender gap reduction) levels and also increased awareness of women’s and children’s issues.  Fourteen projects prepared by recipients of ProDecentralization’s public investment projects diploma included a gender approach (4 done in Ucayali, 3 in Ayacucho and 4 in San Martin), for example, the “Sexual and Reproductive Health for teenagers Capacity Strengthening in the districts of Callería, Yarinacochas and Mamantay, Coronel Portillo Province in Ucayali.”  Fifteen draft ordinances used a gender approach, including issues such as maternal health, children, and teenagers. One from Maynas Province, District of San Juan Bautista, referred to the strengthening of the services to raise awareness and prevent domestic violence. As of June 2012, 01 was approved in this report’s scope of evaluation1 , which declares as a priority and of public need the attention of early childhood in Apurimac. Capacity Building  The PRODES proposal aligned with the needs and priorities of regional and municipal governments.  Training had better results in places where it was accompanied by technical assistance. 1 On August 20th, 2011, the Regional Council of Ayacucho, approved Regional Ordinance No. 021- 2011-GRA/cr under which the improvement of neonatal maternal health is declared public policy. 6  Training on the topic of TUPA was judged to be more effective to the extent that it was treated as a technical subject and directly associated with revenue generation.  Respondents were less positive about training on the topics of budgets, contract hiring and procurement. It was pointed out that political forces often prevented projects, hiring and procurement processes from being carried out within the established regulatory framework. The problem is not limited to the lack of familiarity with the regulations or procedures, but extends to political pressure to act contrary to these same regulations or procedures.  Those interviewed at the sub national levels highlighted the quality of the training sessions, the methodology, the teachers selected and the designed materials.  In addition to the significant number of government staff and officials that received training, the materials and methods have also been incorporated by other institutions working within the country and, consequently, have been reaching more beneficiaries.  This evaluation shows empirically that ProDecentralization increased the number of local projects approved by the GOP project approval process.  Training local elected officials (such as city council members) appears to have had a positive impact on their productivity since those that were trained are performing their job functions better than their predecessors.  The quantitative analysis indicates that the program has had a positive impact on the number of public investment projects that have been approved. According to the qualitative data gathered, this effect is due to innovative training supplemented by good materials. All those interviewed recognize the positive attributes of the training.  With respect to the effects on the indicator “execution of the investment budget”, the quantitative study failed to provide conclusive results. One possible explanation for this may be that the impact of investment budget execution is harder to measure, since it does not rely on the governments’ capacities alone. Moreover, this variable does not include other actions taken by PRODES or other projects sponsored by USAID, the MEF and various other donors who support the strengthening of public administration in this same scope of reference. Characteristics of the PRODES II Projects:  The demand-based approach, allowing the stakeholder demand to drive activities, improved versatility and allowed PRODES to address social conflicts, an issue that was not initially included in the project. However, the demand based approach made it difficult to measure outcomes and led to a dispersion of efforts. 7  The gender based approach that has been part of the training and activities implemented by PRODES has encouraged actions aimed at reducing the gender gap or are otherwise beneficial to women.  The gender approach has enabled female elected local officials to identify and take action on gender issues.  Building alliances has improved achievement of objectives and optimization of resources.  There was no overlap with other foreign assistance agencies or other North American foreign assistance projects.  The perception is that the project is completely aligned with the characteristic that is often attributed to North American foreign assistance: The interest in strengthening the governing bodies of the nation. Conclusions Regulatory Framework  The decentralization process is currently mired in stagnation and inactivity due to a lack of focused direction from the Central Government. This situation will affect the execution of the next phase of PRODES as well as expectations for the Project.  The progress toward decentralization that has occurred over the past few years has been largely driven by regional governments. Their role as catalysts of this process could be supported and encouraged with technical assistance.  In order to ensure that the decentralization process moves forward on legislation, political spokespersons are needed. It is essential that regional and local governments possess a greater capacity to develop proposals and take a leadership role.  The diagnosis performed using the Decentralization Map and the Decentralization Compass is still valid which would indicate that the data collection process was adequate.  The technical assistance used to prepare legislation has proved to be effective in overcoming the limitations faced by international foreign assistance when it wants to influence the legal framework. Resistance from multiple sources is reduced using this approach.  Fostering the creation of spaces for dialogue between the different levels of government was one of PRODES’ most effective interventions.  Lack of clarity as to the responsibilities of the different levels of government generates conflict, which has an impact both on the management of the decentralization process and on conflict resolution.  Political dynamics in the decentralization process carry a good deal of weight. Because of this, an approach which is exclusively technical in nature, with a 8 central approach aimed at building capacities is insufficient. This is particularly clear when dealing with capacity building or creating a structure to resolve social conflicts.  While the submission of proposals for ordinances that involve gender has increased, the rate of approval was significantly lower for gender related ordinances (of the 15 ordinances submitted; only 2 were eventually approved as of June 2012). Sub-national Government Capacity building  The intervention by PRODES in the presentation of projects is seen as positive. We have also identified government administrative actions that demonstrate a notably greater level of efficiency as a result of activities implemented by PRODES.  Capacity building to improve government efficiency requires greater emphasis on technical assistance. Technical assistance increases the likelihood that individual knowledge translates to institutional change.  The strengths of the PRODES approach includes the use of alliances, its development of innovative methods, designing quality materials and meeting the needs identified and defined by the beneficiaries.  The weaknesses of the PRODES model are 1) the heterogeneity of interventions which makes it difficult to measure impact, 2) The governments who expressed their demands have not always possessed much clarity about their needs, 3) There were difficulties to achieving the participation of the highest level public officers in trainings, and 4) the high rate of staff turnover at the levels of government where PRODES worked.  Capacity building is still in demand at different levels of government.  Encouraging the participation of women in capacity building courses was an appropriate measure given that women usually have a lower level of education and experience than their peers. Characteristics of the PRODES II Projects  The demand based approach is a unique aspect of PRODES. However, there is no evidence to suggest that the program’s achievements owe themselves to this aspect.  The demand based approach succeeded in winning the trust and commitment of the beneficiaries.  The demand based approach does not have comparable effects at the various starting points of each zone. The quantitative and qualitative data reflect differences from one intervention location to another.  The demand based approach requires a complimentary in-depth institutional analysis to improve the effectiveness of the interventions. 9  The work done with many regional governments, provincial municipalities and district municipalities created valuable spaces for intergovernmental dialogue. However, the resulting dispersion of activities and resources reduced the direct impact of the activities.  PRODES has carried out activities in areas where there are other USAID sponsored programs and where other international foreign assistance agencies operate, without incurring any overlap. There was a case with a North American foreign assistance agency where efforts were coordinated to complement each other: ProParticipation (aimed at the population) and ProDecentralization (aimed at the authorities).  PRODES strengthens the North American foreign assistance approach, which is strongly concerned with institutional strengthening. Bottlenecks  The problems that the Project deals with are technical in nature; the decentralization process is political and creates discrepancies between different levels of government, particularly regarding issues related to transfers of power and resources.  The lack of consensus prevails across different levels of government. There is no consensus about what needs to be done on the topics of decentralization, handling social conflicts or administrative systems like SNIP. This lack of a shared vision produces both progress and setbacks which reduces the likelihood of obtaining results.  High staff turnover is detrimental since it impedes the accumulation of knowledge and experience. In this regard, the training provided loses impact and sustainability.  The current level of political instability in the regions where PRODES worked limited the probability of the success of these interventions.  Finally, one of the principal limiting factors to the decentralization process has been the absence of mechanisms for intergovernmental coordination, the difficulties in forming consensus and the lack of leadership and political vision on the part of the executive and other key stakeholders. The project’s role in creating spaces for intergovernmental dialogue is duly noted. Lessons Learned:  It is essential that we maintain and even accentuate the interview process carried out by PRODES (up-down and bottom-up). This type of process, which received high marks from the stakeholders, becomes necessary when seeking greater participation on the part of the regional, provincial and district levels of government in determining the regulatory framework.  A project’s resources can be optimized by using an approach of alliance building. 10  Particular attention must continue to be paid to the selection of trainers and getting government institutions involved in the preparation of materials.  Projects such as PRODES should place more emphasis on technical assistance. It improves results and gives the work a long term perspective.  The timing of changes in government administrations must be taken into account when designing the programs. Training programs that are started during this phase weaken the program intervention’s overall impact due to inevitable staff turnover and because complications often arise during the transition period.  Having a tailor made program to offer (in this case a demand based approach) serves to build bonds of trust with the different bodies of government with whom the project will work and makes it easier to adapt to the situation. But a prior assessment of the government bodies that will be participating in this program is essential in order to determine if they have the capacity to clearly identify their needs.  We recommend that the gender based approach be maintained and even given more emphasis since this ensures that gender issues are given appropriate consideration. Recommendations  We recommend to continue providing neutral spaces for dialogue between key players in the National Government (Executive and Legislative Branches) with a view toward forming networks and developing synergies that carry the potential to build consensus about the direction of the decentralization process and the direction of related legislation.  It is important to continue to develop consensus and coordinate existing proposals about guidelines for the decentralization process at the sub-national levels of government. This must be done in a way that promotes a Bottom–up dynamic to form coalitions for policies that move the process forward. Along these lines, the Alliance with the Assembly of Regional Governments and associations of municipal government should be strengthened.  The Decentralization Compass which was recently transferred to the Congressional Committee on Decentralization should be monitored to ensure its continuity.  More emphasis should be given to technical assistance focusing efforts at the regional level.  It is recommended that PRODES continues its work and even increase its assistance to initiatives such as the association of townships and regional alliances. Social Conflicts a. Capacity building must consider the unique characteristics of each place where we implement our program. The majority of these conflicts are 11 associated with a particular scope of influence such as the existence of mines, forests, etc. b. Capacity building should proceed when directly related to the powers conferred to the sub-national governments. Doing it before these powers and authority are clearly defined would be ineffective. c. Indicators are needed to enable the measurement of the capacity of sub￾national governments in conflict prevention and management. d. While it is true that the most obvious social conflicts are associated with mining operations and large scale infrastructure projects, there are a number of totally different conflicts which cannot be ignored because doing so would increase unrest in populations that have been neglected. e. It is necessary to examine ways to strengthen political representation since its absence is often a cause of the emergence of unrest and conflict. f. More coordination needs to be encouraged between the National Government and sub-national governments on matters of conflict resolution. Moreover, greater clarity is needed about the definition of the powers and jurisdiction held by each body and level of government.  With regard to the gender issue, we recommend that the approach be enhanced since the presentation of gender related initiatives by itself does not guarantee subsequent approval. This requires the formation of alliances to exert the necessary political and social pressure to achieve their approval. Recommendations to USAID:  With regard to legislative reforms that require political willingness and a high level consensus, we recommend that USAID search for other types of alliances that effectively enable the key players to incorporate their proposals and vision into the debate. It must be taken into account that regional governments lack the resources and access to experts to be able to submit proposals that modify the current regulatory framework.  One relevant challenge consists of improving the availability of data required to conduct a rigorous assessment prior to program implementation. In this sense, the barometer should already have information in its data base about the number of PIPs (Public Investment Projects) that have been prepared in order to establish the ratio, N° of Investment Projects approved/prepared, which is a better indicator of efficiency.  We suggest that certain aspects of the demand based approach be revisited. Beyond its inherent advantages, the approach is only effective where governments correctly identify their needs. In this sense, the demand based approach could be supplemented with an assessment to ensure that training content is adapted to the needs of the regional and municipal governments and not solely based on an open ended request for training. This point becomes more relevant in the case of a variable which is difficult to model such as “investment budget execution.” 12  In the future, program design, implementation, and data collection should be conducted with a view toward performing a rigorous evaluation. Data collection should be consistent with the evaluation method proposed and defined from the beginning of the program. In this regard: 1) we suggest reducing the heterogeneity of the training, and 2) keeping more detailed and computerized records on the profile of each trainee and of participating government entities. Recommendations to the Peruvian government  The Government should recognize that, in practice, the decentralization process has been moving forward with different degrees of efficiency between sectors and regions. This creates the need to have a clear diagnosis of the project’s status and an investment that should be made in design and monitoring.  It is essential to clearly convey the direction that the decentralization process might be headed or where stakeholders want it to go. This involves clearly defining the functions, powers and mechanisms of interaction between the different levels of government.  Encourage Regional Government Presidents to assume a leadership role in their areas of jurisdiction (provincial and district municipalities).  Place more emphasis on technical assistance in the regions and municipal governments; initiatives such as those in CONECTAMEF can contribute to this objective.  It is essential to create new spaces where people are able to interact and coordinate with the different government bodies.  Look at developing legal frameworks that can deal with the issues behind social conflicts. Understand that the Central Government also needs to adapt its procedures and training to learn how to operate in the scenario of a decentralized nation.  Place more emphasis on developing indicators that enable all stakeholders to more clearly read the status of the decentralization process. 13 Introduction Since its inception in 2002, the decentralization process has been a fundamental aspect of the reform of the Peruvian state and participatory democracy, with emphasis placed on it during the administrations of Alejandro Toledo and Alan Garcia. In the last ten years, significant progress has been made on the decentralization process with the support of different levels of government and civil society. The main goal of the Prodescentralization project was to support efforts to go forward with the decentralization process, focusing on two main results: 1) Improvement in the legal and public policy framework (national level) 2) Building local governments’ capacity to govern effectively (sub-national level) Activities at the national level were designed to improve the legal and public policy framework for decentralization. The main activities were the following: 1) Evaluation of the legal and public policy framework 2) Design of a tool for prioritizing actions related to the legal and public policy framework 3) An annual status report on decentralization 4) Technical assistance on critical legal reform matters and the implementation of regulations, procedures and public policies At the sub-national level, the focus was on building local and regional governments’ capacity to govern effectively, with the following as the main intervention areas: 1) Reinforcement of Good Government Practices such as participatory budgeting and planning, including public accountability processes. 2) Improvement in the implementation of National Administrative Systems. Efforts were focused on strengthening the National Public Investment System, procurement processes, state contracting and results-based budgeting. 3) Improvement in the efficiency of selected municipal governments, including administrative simplification and results-based budgeting. 4) Promotion of local economic development. 5) Implementation of innovative strategies for tax collection at the local level. 6) Support for municipal associations. 7) Support for Inter-regional Coordination Boards. 8) Formation of public / private alliances. The project used the following implementation strategies:  Demand-based project: Taking into account the priorities and needs stated by national public policy makers and regional and provincial authorities. At the beginning of the project, an assessment of regional and provincial governments’ needs and priorities was prepared. Based on this information, the project was 14 developed. During the second year, an instrument called a “Governance Barometer” was designed for following up on "measurable" improvements, along with other monitoring and evaluation tools. The assessment was updated annually to ensure that the technical assistance met Peruvian government entities’ needs and priorities.  From policy to practice: While the activities were organized under the aforementioned headings, every activity executed by Prodescentralization had applied components; that is, the project provided support on the development of national policies and/or procedures for implementing these policies, and sought their application in practice with selected regional and provincial governments.  Alignment with Peruvian government policy and promotion of ownership: Every initiative implemented by the program (except for the annual decentralization analysis), was executed in collaboration with the responsible Peruvian government entities according to the subject matter or intervention area. All training materials were approved and “co-branded” with the corresponding supervisory entity.  Results-based approach: sub-national interventions sought improvement in public administration. Regional and municipal governments demonstrated their commitment by applying the knowledge, techniques, tools and systems developed during the project.  Alliances and agreements were developed to maximize the impact of the project. The list of actors that worked with the project includes other USAID projects, other donors, civil society organizations, Peruvian government entities, associations at the regional and municipal levels and private entities. The information presented below analyzes all of the aspects detailed above, in an effort to establish the extent to which the strategies established initially were maintained and how this contributed (or not) to the achievement of the established objectives. Chapter 1 is a contextualization of the decentralization process, which frames the activities carried out by the project. The second chapter covers the work performed in relation with the regulatory framework, while the third chapter presents an analysis of what has been achieved in terms of capacity building. The last chapter analyzes general characteristics of the program. The performance evaluation of this program was performed by a four-member team whose members were from Imasen and the Center for Social, Economic, Political and Anthropological Research?(CISEPA). The team, whose members were Edmundo Beteta (Team Leader), Giovanna Peñaflor (Qualitative Analyst), Jessica Benza and Cristian Alvan (Senior Analysts), reviewed the materials developed by the project, gathered secondary information, conducted interviews and analyzed the foregoing in order to prepare this document. 15 The Peruvian context The decentralization process in Peru began with the Reform Act on Chapter XIV of Title IV of the 1993 Constitution (Law 27680) and the passage of the Framework Law on Decentralization (Law 27783) (LBD) in 2002. Within this framework, the decentralization policy was defined as a top-down process that would take place through an orderly, progressive transfer of powers and functions of the national government to regional and local governments in phases. The purpose of the process was to "achieve harmonious and sustainable integral development of the country through the separation of powers and functions, and the balanced exercise of power by the three levels of government for the benefit of the population" (LBD: art. 3). The basic regulatory framework (Constitutional Reform Law, Framework Law on Decentralization, Organic Law on Regional Governments, Organic Law on Municipalities, Law on the National Accreditation System and its Regulations, Law and Regulations on Incentives for the Formation of Regions, and the Fiscal Decentralization Law). During this period, investment and social programs were transferred to regional and local governments. In 2005 there was an unsuccessful attempt to form regions. Then, starting with the change of administration in 2006, there was a “decentralization shock" in which transfers of functions and assets, including the regional directorates of ministries, were accelerated and 95% of the transfers were completed. During this period, efforts were made to optimize the decentralized operation of administrative systems, particularly the National Public Investment System. The National Council for Decentralization, which was the joint technical body (with the involvement of the national government and regional governments) in charge of coordinating and supervising the decentralization process, was disbanded and replaced by the Decentralization Secretariat within the Office of the President of the Council of Ministers (PCM), as well as the newly created Intergovernmental Coordination Council (ICC) as an advisory body, with the participation of representatives of regional and local governments. From 2007 to 2008, progress was made on the passage of the Organic Law on the Executive Branch (LOPE) and the issuance of 100 legislative decrees 2 within the framework of the approval of the Free Trade Agreement with the United States. Among these, the following merit mention: Civil Service reform, the optional modernization of regional and local governments, and national strategic planning. (Prodes: 2008, p. 12). In addition, the implementation of the Results-Based Budget began and Congress passed new legislation to increase the development of municipal associations and to establish 2 Among them D.L. No. 1023, which created the National Civil Service Authority, the supervisory body for the Administrative System for Human Resource Management; D.L. No. 1024, which created and regulated the Public Manager Corps; D.L. No. 025, which approved training and performance regulations for the public sector; D.L. No. 1026, which established a special optional system for regional and local governments wishing to implement integral institutional modernization processes; and D.L. No. 1088, Law on the National Strategic Planning System and the National Center for Strategic Planning (PRODES:2008). Chapter 1 16 the division of powers between the regional executive and legislative branches (or Regional Council) within Regional Governments. It also passed the Law on Equal Opportunities for Men and Women, which establishes guidelines for the different levels of government including regional and local governments, in order to ensure the principle of equality within the framework of their authority (PRODES: 2009, p. 10). In 2009, a debate and review of the proposed Law on the Organization and Functions of Ministries of the Executive Branch, which details shared and exclusive powers and functions, began. Through Law 29341, municipal associations were granted legal status, allowing them to execute projects and resources jointly. Similarly, Law 29379 reinforced the Interregional Coordination Boards (JCI) as a mechanism to ensure continued progress on the regionalization process, which had stalled in 2005. As the decentralization process had not concluded, the qualitative research sought to delve deeper into the actors’ perception of the decentralization process. For this reason, questions were included to determine their opinion on the progress made and limitations of the process, as well as the main bottlenecks encountered. The following is a summary of some of the prevailing opinions of key actors on the context and current status of the decentralization process, since we consider them to be an important element to take into account in framing the intervention of PRODES-II, as well in reviewing assumptions for future intervention:  A recurring theme mentioned by respondents was the lack of proper distribution of powers and responsibilities among the different levels of government. Although decentralization was conceived at its outset as an orderly transfer process based on capacity-building in sub-national governments, respondents believe that, in practice, the process may have been influenced by political negotiation among the various actors, which resulted in a rather disorderly process in which powers have not been well defined. Although the different levels of government coexist peacefully, the existence of overlapping functions that hinder policy development and prevent proper use of public resources is still mentioned.  As the Framework Law on Decentralization (LBD) (Law 27783 of 2002) does not precisely define exclusive, delegated and shared powers or delineate functions, these powers have been defined by a multitude of pieces of legislation passed throughout the process, such as the Organic Law on Regional Governments (LOGR) (Law 27867 of 2002), the Organic Law on Municipalities (LOM) (Law 27972 of 2003), the Organic Law on the Executive Branch (LOPE ) (Law 29158 of 2007) and even different laws on the organization and functions of ministries. Respondents agree that, in practice, timing differences in the passage and/or amendment of these laws has resulted in a disorderly process. According to functionaries of the Commission for the Decentralization of Congress and other experts, the fact that the laws on the organization and functions of ministries determine and define a set of functions shared by the three levels of government has given rise to sectoral logic in the process that is inconsistent as a whole.  If the main objective of decentralization is to "achieve integral and harmonious development of the territory through proper allocation of powers for the benefit of the population," respondents coincide that one of the main difficulties in 17 moving the process forward is precisely the lack of a clear division of powers and the difficulty of dealing with the problem of the diversity of sub-national governments, especially municipalities, with technical criteria when defining powers, functions and the allocation of resources.  The lack of clarity on the matter of powers limits citizens’ possibilities of identifying which level of government is responsible for policies, thereby hindering citizen oversight and the ability to hold authorities accountable.  Regarding the regulatory framework for sub-national governments, respondents agreed on the need to determine the role of the Regional Coordination Councils, as they did not have a clear understanding of their role with regard to coordination with mayors and the participation of civil society.  In addition, several respondents indicated that the organizational structure and institutional design of sub-national governments should be revised to ensure that they are able to operate effectively.  Another problem hindering the decentralization process is stagnation on the matter of regionalization and difficulties in creating macro-regions. However, rather than a regulatory problem, they consider this situation to be a political problem and they mentioned the correct decision by PRODES-II to contribute to the ongoing articulation process through the Interregional Coordination Boards (JCI) and to support the creation of municipal associations.  Respondents repeatedly mentioned that the stagnation of the regionalization process has had an impact on the lack of progress on the fiscal decentralization process accompanying the transfers, as it was originally tied to the regionalization process that stalled in 2005. The passage of a new law on fiscal decentralization is a pending issue that depends on political will on the part of actors in the Peruvian government.  Respondents also noted the absence of intergovernmental coordination mechanisms, the difficulty of reaching consensus and the lack of leadership and political foresight on the part of the Executive Branch and other key actors as a major constraint in the execution of the decentralization process. Decentralization has resulted in a proliferation of politically empowered actors, but the main problem is to answer the following question: How can a decentralized state with three levels of government be built? In the opinion of some respondents, it will be a long-term process that could take up to thirty years and will require a shared strategic vision of the process and the type of country and regions or municipal governments we want to have.  Regarding supervision of the process, respondents agree on the weakness of the former National Council for Decentralization (CND) and its failure to promote regionalization. However, they also indicate the limitations of the current Decentralization Secretariat (SD-PCM) created to replace it, as it is a third-level body with difficulties negotiating on equal terms with the ministries. Regarding the Intergovernmental Coordination Council (ICC), they mentioned that it was not 18 working; consequently, bilateral relations between the Executive Branch and regional/local actors were given priority.  These difficulties in intergovernmental coordination are repeated at the regional level, where respondents indicated that there is no capacity to articulate the relations of provincial and district governments. The ill-defined role of the Regional Coordination Councils as spaces that confuse intergovernmental coordination functions with those of the participation entity also hampers this process.  In addition, respondents indicated that there are also difficulties in intersectoral coordination within the Executive Branch, which hinders the coordinated execution of the decentralization process. This problem takes on additional importance when we consider that many of the functions of the powers shared by all three levels of government are determined by the laws on the organization and functions of each ministry.  Another element of the context that was mentioned in the interviews and forms part of the assessment on which foreign assistance interventions are conditioned is the lack of guidelines and basic consensus among the main actors. In this case, the need to coordinate and articulate proposals made by regional governments, as well as to give more prominence to the National Assembly of Regional Governments as the space for reaching consensus at the sub-national level and as a link with the Executive Branch at the national level, was also mentioned. The elements characterizing the abovementioned decentralization process are summarized in the following chart: 19 The national focus: From policy to practice In this chapter of the report, matters related to the work of PRODES II regarding legislation and the implementation of polices, what it has done in relation with social conflicts and its contribution with regard to gender are analyzed. 2.1 The regulatory framework 2.1.1 Objectives of the project The intervention of PRODES-II in the development of the regulatory framework for decentralization and policy implementation has been defined with a "from policy to practice" approach, through which it seeks to collaborate with key actors in the Peruvian government (GOP), such as the Executive Branch (PCM, MEF and other sectors) and the Congressional Commission on Decentralization, to facilitate the development of regulations and the implementation of policies (policy) that contribute to improved performance of regional and local governments (practice) and ultimately, to progress on the decentralization process. Consequently, with regard to the regulatory framework for decentralization, the objectives of PRODES-II were geared toward: a. Providing information to key decision makers in the central government (GOP), thereby facilitating informed decision-making. b. Helping to build consensus among key decision makers in the Peruvian government (GOP). c. Promoting public debate on decentralization policies in order to identify needs for regulatory reform and to reach consensus on the general outlines of the decentralization 2.1.2 The program’s contribution on legislative matters As shown in the PMP table (see Annex) in the preceding section, the project produced the annual “Decentralization Compass”, the Map of Legislation and Policies on Decentralization and an annual report on the decentralization process. The "Decentralization Compass" was developed in collaboration with key actors in the Executive and Legislative branches, regional and local governments and other relevant actors. In it, policies that should be promoted in order to achieve progress on the decentralization process were identified, providing information to policy makers that will improve the quality of the policy formulation process and build consensus. For example, for the last year of the project, with the help of the key actors mentioned above, the compass identified the following policies that should be promoted: Chapter 2 20 a. Political articulation and coordination in the management of various government agencies; that is, alignment of policies, guidelines, plans and actions at both the national and decentralized levels. b. Define the steering role of the National Government and its implementation and operational mechanisms, taking into account the autonomy of decentralized governments. c. Decrease institutional parallelism. It would be necessary not only to transfer functions and powers from the National Government, but also to avert the emergence of parallel institutionalism which would give rise to the execution of national programs that overlap the programs run by decentralized governments. d. Incorporation of the notion that decentralization not only involves the reform of the organizational structure of the Executive Branch, but also its modus operandi, decision making and management systems, among other reforms. It should be noted that the comparison between these proposed topics and the answers obtained from the different actors interviewed for this report confirms the need to implement them. Regarding the need to incorporate changes in the Executive branch’s way of operating, positions such as that of the director of an NGO were found: "I think there are weak points in the regulatory framework, which are the laws that should apply to the ministries, the laws on the organization of the ministries’ functions, which should apply starting in the year 2007 when the Organic Law on the Executive Branch was passed. It states, ‘From now on, every ministry must be restructured, with new laws in line with decentralization.’ That took place in some ministries, but not in others. Therefore, there are ministries that still operate according to laws passed prior to decentralization," which indicates the need for the central government to review what they are and are not doing. In addition, the lack of leadership of the process on the part of the Executive branch was mentioned: "The central government has not taken on the task of strategic leadership in achieving articulation of the three levels of government. Articulation not only implies the operation of the Intergovernmental Coordination Council; it also means that in a specific territory, the three levels of government must be articulated, and the national government should take the initiative." (Congressional functionary) Although his remarks place greater emphasis on the operational aspect than the regulatory aspect, they show that he perceives the lack of political articulation and coordination, which is a problem related to the disappearance of the CND, as it left a vacuum that was not filled by the creation of the Decentralization Secretariat. This last situation leads some to think that the model used needs to be completely overhauled ("We already had a faulty framework because it states that the entity in charge of fostering the decentralization process was the National Council for Decentralization / CND, which no longer exists because it was merged with the PCM’s Decentralization Secretariat, turning it into an administrative entity." (Municipal organization) "The CND failed, that model failed and the PCM is insufficient. There is a pending agenda." (Legislative branch) 21 Lastly, respondents constantly refer to institutional parallelism: "There are four types of organizations within the regional governments. One is from before the times of the CETARES, another is from the times of the CETARES, and then there are the regional directorates and regional management, so it’s very difficult to govern there. The regional health director reports to the minister of health. The social development manager does not have authority over the director of education, the Ministry of Education deposits money in the account of the Regional Directorate of Education so that it can prepare early childhood education profiles, the regional director of education doesn’t know what the money is for and the regional general manager doesn’t know that the regional government has received a deposit. That design doesn’t lead to good management." (Central government official). The annual report on decentralization, a tool that has been given periodically to key actors in the Peruvian government, sub-national government and civil society, has contributed to the debate and informed decision making. With regard to technical assistance on the issue of approval or reform of decentralization legislation, PRODES-II collaborated on the drafting of 11 bills, provided technical support on 8 initiatives on administrative modernization of regional and local governments, and supported 14 initiatives on decentralization by collaborating on the organization of debates and public forums. It should be noted that PRODES-II also identified some policies and regulations related to the decentralization process on which a high level of political will and consensus would be required:  Transfer of sectoral functions to regional governments.  Delineation of functions of different government levels.  Intergovernmental articulation and guidance of the decentralization process at the national level.  Concerted execution of the decentralization process.  Completion of the development of decentralized administrative management of sectoral functions transferred to Regional Governments.  Promotion of the definition of territorial boundaries.  Development of a classification for municipalities.  Promotion of the regionalization process  Promotion of national strategic planning as a national system.  Redesign of the fiscal decentralization model according to equity and partnership criteria.  Institutional reform and modernization of Regional Governments.  Development of a system for monitoring and evaluating results of the decentralization process.  Production and dissemination of relevant information by the Executive Branch. As in the preceding case, in this case the interviews indicate an on-target assessment regarding the needs of the decentralization process that should be taken into account in the new phase of the project. However, it should be noted that the respondents did not agree on the priorities of these pending tasks. On some points, there are conflicting positions, such as in the case of fiscal decentralization on which some analysts expressed their doubts: "With regard to fiscal decentralization, I think we should proceed calmly. 22 What we are discussing now is the weakness of regional governments for executing it, isn’t it? So I don’t see that there’s any hurry for regional governments do obtain their own resources." It can be concluded that there is a vacuum where a contribution to consensus building is needed among public policy developers, among the different actors involved in the project. A member of the Executive branch commented, "Yes, I believe that PRODES has made a very clear assessment; now it just needs someone to act on it." A similar position was stated by a Congressional functionary who said, "We have the information provided by PRODES.... the problem is that Congress needs to pay attention to it and take it into account." On this matter, a way of encouraging them to take action on it needs to be determined. In the qualitative performance evaluation of PRODES-II, questions were included to determine respondents’ perception of the project’s intervention in the development of the regulatory framework for decentralization. The findings considered most relevant are the following:  There is agreement among respondents at the national level when evaluating the materials produced by the program, especially the Decentralization Compass: "In general, I greatly appreciate Prodes’ work. I think they do good work; they’re very well informed." (Political analyst). However, it should be noted that the level of follow-up on the document is usually lower than the importance attached to it: "We’ve recently received a number of PRODES publications. Since it was just completed recently, we just found out about many of their books a short time ago. They sent us a complete collection of everything they’ve produced over the last 3 or 4 years. That’s how we found out about a set of documents that the mayors surely must have received directly. Apart from the mayors, we don’t know, but we do know that it contributed to capacity building and to making up￾to-date information available.”  Regarding the Decentralization Compass, the fact that its development had been coordinated with different actors in the Peruvian government, regional governments, local governments and other actors was particularly highlighted. Also assessed as positive was the recent transfer of the Congressional Commission on the Decentralization Compass, although the need to expand consultations made for its preparation to other regions of the country was mentioned. "The situation is that PRODES is over and we in Congress have to invent a new way to make the compass. We don’t know exactly how we’ll do it yet, but we have to make some inquiries outside Lima. The PRODES inquiries are concentrated on the departments where the project has worked, such as the high jungle and coca growing areas. We are going to have to do something more extensive, but we need to obtain financing and find different mechanisms to create the instrument in Congress." (Functionary of the Congressional Commission on Regionalization) 23  In addition, the Congressional Commission on Decentralization highlighted the collaboration of PRODES-II, which provided technical assistance on the preparation of different laws relevant to the process that have been mentioned in performance indicators. In other words, both the documents contributed by PRODES-II and the process followed in their preparation were appreciated: The methodology is a consultation methodology and the compass was built based on consultations with various actors. Many inquiries were also made in the territories where the project intervened. We were part of the consultation made in Lima, but the result is an important tool to guide the work of the commission or members of Congress, a reference for the pending agenda, although it will not necessarily be dealt with." (Congressional functionary)  Respondents particularly highlighted the interaction workshops held with the participation of key actors, as spaces that offered a neutral platform where representatives of different areas of the Executive Branch (PCM, MEF, etc.) and the Legislative Branch (Congressional Commission on Decentralization), as well as sub-national governments and other relevant actors, could make contact and exchange views. Thus, one of the strengths of the project highlighted by respondents was the creation of informal coordination spaces. PRODES-II had to operate in a context marked by the absence of intergovernmental coordination mechanisms, overcoming the difficulty of reaching consensus and a lack of political foresight and leadership on the part of the Executive Branch and other key actors, due to which the on-target decision by PRODES-II to contribute to the ongoing process of articulation through the Interregional Coordination Boards (JCI) and to support the creation of municipal associations was highlighted. 2.1.3 Aspects that limited or facilitated the program’s action on this point The aspects that limited the program’s action in this area were mostly inherent to the Peruvian political process, because as detailed in the chapter on the decentralization context, many of the regulatory reforms required depend on the political will of different actors, the development of a hegemonic vision and leadership of the process by the Executive Branch, and the ability to reach consensus and foster intergovernmental coordination processes that do not currently exist. In this regard, the accompaniment of PRODES-II on the development of the regulatory and policy framework had to be adapted to these limitations, collaborating only on aspects where there was consensus to move forward, in a context in which several respondents indicated that there was stagnation of the process: "The decentralization process is static; there is a lack of political leadership to move forward with the process, as well as a lack of technical tools. What one can see is that after the National Council for Decentralization was disbanded, the governing body, which is the Decentralization Secretariat, lost political weight. In addition, I think that from the point of view of the central government, the technical capacities to monitor the process are not adequate yet." (Former MEF official) 24 An important element to highlight is the low visibility of PRODES-II interventions in assisting with the development of the regulatory framework and information for policy development at the regional and local levels, where little is known about the Decentralization Compass and the Map of Regulations and Policies on Decentralization. This is due to the manner in which PRODES-II was designed; it should have involved regional actors more. Despite the lack of recognition of the work of PRODES-II in this area at the regional and local levels, the project’s on-target decision to contribute to the ongoing process of articulation through the Interregional Coordination Boards (JCI) and to support the creation of municipal associations was noted. Also mentioned was the foreign assistance provided to the National Assembly of Regional Governments (ANGR), although respondents felt that greater support was needed to coordinate and articulate proposals from regional governments and to give more prominence to the National Assembly of Regional Governments as a space for reaching consensus at the sub-national level and a link with the Executive Branch at the national level. 2.2 Social Conflicts 2.2.1 Objectives of the project In order to analyze the information obtained, it is important to note that PRODES II established the following as a general objective in conflict resolution: "To assist the Peruvian government in its efforts to design and implement a more effective strategy to prevent and handle conflicts through democratic channels and institutions. The assistance should increase public confidence in the government's commitment to resolving conflicts." The specific objectives of PRODES II regarding conflict resolution were: a. To assist the PCM on the development of a coherent, integrated vision and planning for conflict management, response and resolution, including contributions from key ministries and local officials in the areas where conflicts take place. b. To design an appropriate and effective communication strategy geared toward having representatives of the national and local governments, as well as civil society organizations, communicate actions related to the conflict resolution policy. c. To provide technical assistance to promote key changes in laws and policies thatfacilitates the implementation of a national conflict management strategy. d. To build institutional capacity in conflict management, prevention and resolution by training line ministries, regional governments and sub-national actors. Efforts should take place in the departments of San Martin, Amazonas and Madre de Dios, as well as other areas directly affected by conflicts. 25 Specifically at the national level, its work was related to the following: Legal and policy framework: a. Provide the PCM and the ministries’ most important conflict management offices with a more coherent and effective conflict resolution strategy that can be applied at the national and sub-national levels. b. Technical assistance on the preparation of a legal framework for conflict management in a joint effort with the PCM and key members of Congress committed to the issues related to conflicts (indigenous matters, the Amazon region, energy and mines, women, etc.), involving civil society in the debate. Institution building: a. High technical level meetings with the PCM’s conflict unit, the Prime Minister and key ministers, in order to develop a comprehensive outline for conflict prevention and administration. b. Design an information and communication strategy that includes mapping tools which will serve as a mechanism for analyzing and identifying conflicts. c. Preparation of management tools to raise the level of the Coordination Unit in the hierarchy to that of Conflict Management Office. (The ROF, MOF and CAP were approved.) At the sub-national level, the program’s intervention was to take place in the following manner: a. Prepare a training package on conflict management and resolution to be validated at the intervention sites. These packages should include tools and methodologies for conflict management and prevention. b. Provide training and technical assistance on capacity building for conflict management and prevention at the regional level that includes participation at the municipal level, community leaders, civil society and representatives of commercial sectors. c. Assistance on the implementation of a national policy that makes it possible to formalize the artisan and informal mining sector. 2.2.2 The program’s contribution on the issue of social conflicts According to the Office of the Public Ombudsman, during the current administration’s term in office, as of July 2012 there were 247 on-going social conflicts, of which 150 were related to environmental issues. According to information from the PCM, as of July of this year, the Social Conflict Management Office (OGCS) was working on more than 60 social conflicts. 26 Regarding the context in which PRODES conducted its activities and will conduct its activities in the next phase, concern about social conflict is high among respondents; it is obvious that it is one of the issues in relation with which there is greater demand: "The state and companies have realized that they can no longer operate in the same way because when they build a road, install a transmission line, lay pipe or whatever, in some way it directly or indirectly affects a private or public interest. The level of awareness regarding participation and the capacity to express discontent is higher because of the media and education. Companies and public institutions are taking courses on how to assess conflicts and in my classes, because I teach at the Catholic University, I have many people from the Ministry of Transportation, the Ministry of Agriculture and the Ministry of Mines taking courses. They realize that they can no longer give the order that 'the highway goes there’ without having conducted a consultation process. By law - because that was another law that was passed - every government sector, of the state, must have a conflict management team.” (Specialist in conflict resolution) This confirms the importance of having included the issue and the need to incorporate it in the new phase of PRODES. From the point of view of a key actor, the work was performed in different areas: "…we took the capacity of this organization to support us into account in two lines of work: the first line is the design of the office of social conflict management. With the contribution of PRODES and the consultant they hired, what was to be a new office with a new approach to social conflict management was designed. Work on this output was performed with the PCM’s office of public management. The result was a proposal for the reorganization of this office. The second line of work was training on social conflicts for regional government officials and representatives of institutions in the sectors involved in managing them." As usually occurs with project interventions, the training component received recognition: "PRODES has been holding training workshops. Training materials were developed for three modules and 200 to 300 people were trained on conflict resolution. Participation was excellent, but more work is still needed.” (Specialist in conflict resolution) From the information gathered through interviews, the following can be established:  PRODES’ contribution to having seven regions with greater insight on the subject of conflicts is acknowledged. This step requires accompaniment to consolidate its achievements: "There are seven or eight governments that now have a broader perspective regarding conflicts and the possibility of working on them, but how to go about it, the matter of establishing conditions, relations, all these sensitive issues are not being dealt with yet.” (Specialist in conflict solution)  The importance of having provided training is acknowledged. However, it was also mentioned that the effects of this training are reduced when the way in which each level of government involved should intervene is unclear. "I believe 27 that Regional Governments need on-going training. A workshop is not enough. They need more support. "(Specialist in conflict resolution)  Training is limited by staff turnover, as mentioned before. There are cases, such as Ayacucho, where officials have a low level of knowledge of the training programs held.  PRODES II not only provided training; among its achievements are the approval of the Guidelines and Strategies for the Management of Social Conflicts (2011) by means of a Ministerial Resolution. However, the effect of this progress is not yet clearly perceived: "On the issue of conflicts, legislation was passed, specific legislation on conflicts, in which we proposed functions and conflict competencies for the national government. This conflict management office that I mentioned, and also for the regional governments with more specific capacities, competencies and functions for regional governments. The problem was that the administration in office at the time, the APRA administration, cut several of those functions. In the end, it seems that in the last period before the PRODES decentralization program ended, work was done on legislation on the competencies of regional governments in relation with the issue of conflicts, so that draft legislation was prepared and presented to the office. I don’t know what its status is now. (specialist on conflicts), "What I know about is the design of the Directorate of Social Conflicts that served to place it on the agenda of the PCM and the office in charge of PCM programs, and it ended up in a proposal that’s being discussed.” (former central government official)  The Geo-referential Information System and the website of the Conflict Office have been developed and monthly newsletters have been published, although they were not mentioned by the actors consulted, which may have to do with the fact that pertinent decisions have not been taken by the central government.  The National Plan for Formalization of Artisanal and Informal Mining, which is currently in force as the framework for the process of formalizing illegal mining, is in line with a position indicated repeatedly by those consulted. Dealing with the issue of social conflicts not only requires working on consultation processes, but also focusing on the root problems, which requires an analysis of the situations that lead to conflicts. This type of intervention should be extended to other issues that give rise to conflicts in each region, such as water, natural resource management in general, etc. 2.2.3 Aspects that limited or facilitated the program’s action on this point Intervention on the issue of conflict involves working on an extremely broad and complex matter, where it is necessary to clarify the agenda and work at the nation and regional levels to find an appropriate way to analyze social conflicts, design intervention processes, and manage early identification, monitoring and evaluation of social conflicts. Similarly, a system for monitoring progress on these matters is needed. The first matter to consider is that there is no clear assessment of the reasons why conflicts arise or why they spread. According to some, "The problem of social conflict 28 is not due solely to a lack of skills or a strategy for preventing and managing conflicts; it also has a significant political component.”(Congressman) Others attribute the problem to a lack of political parties and organizations: "I see that the actors in confrontations have carte blanche to spark crises. I think one of the most serious problems existing in the country is the lack of democratic leadership, the lack of organizations." (Former head of the Conflict Office) It was mentioned that the lack of political parties leads to local and regional authorities being "abducted" by unsatisfied social demands. They explain that in the absence of a political party that supports the authorities, they can only opt to take a stand in favor of these demands or lose their legitimacy. As a result, their role as mediators or conflict resolvers is diminished. Along the same line, local authorities are concerned about conflicts motivated by the pursuit of power: "We have many social conflicts. We have conflicts due to interest in gaining power, the desire to become part of the local government. There are those who do not run for mayor or participate in the elections, but want to be mayors. Sometimes one does not define those accompanying one, and it turns out that the deputy mayor wants to be mayor and organizes the population, generating instability through social conflicts." (District mayor) In short, the issue has a political component that PRODES is not responsible for dealing with, but that it must take into account in the development of institutional redesign strategies directly linked to the decentralization process. When evaluating the position of the central government on the issue of conflict, respondents were critical: "It is not well prepared; its awareness is very general. It’s sensitive when its projects are stopped, when it costs it votes and popularity, but many of these conflicts need a real commitment to change, to taking another approach. You can have these conflict offices, but they must be implemented. There is still a lot to be done, but apparently the government does not yet see it as a priority. I think progress is being made on some matters; for example, there is talk of a dialogue office and it looks like they are going to reorganize the conflict management office, but I still don’t see a change in the approach promoted by these different sectors. I don’t see it yet.” (Specialist in conflicts) “We have a state that is not capable of preventing or foreseeing conflicts. I think that in general, we have a state that does not have this issue among its priorities.” (Specialist in decentralization) From this point of view, despite the fact that the central government is requesting that work be done on the issue of conflicts, it does not appear to have a clear idea of what is needed, which may cause complications in the logic of a demand￾based intervention. The necessary link between the central government and regional governments is extremely fragile. For this reason, one person commented, "What I do know is that the when the National Government has a conflict with a Regional Government, it freezes its accounts. They fight each other and articulation goes down the drain. As this is a very political issue, any type of articulation or coordination depends heavily on political will, the context, the current situation and whether or not they 29 have a shared vision" (specialist in decentralization). In this context, training or capacity building does not appear as a central theme. Efficient coordination mechanisms must be developed first. In addition, it was repeatedly stated that the role of regional governments is not yet defined. Currently, they are unable to make decisions about many of the matters that generate conflicts (the SUTEP strike was given as an example), a fact that contrasts with their impression that mechanisms to control the performance of presidents of regional governments are lacking. From this perspective, we are far from being able to take a position such as the one stated by a specialist: "To the extent that decentralization takes place, primarily the decentralization of conflict management, officials at the local, regional or provincial level will have more confidence to be able to intervene in these conflicts, without saying, ‘It’s not my job’. They will clearly understand that it is their responsibility to prevent conflicts. " We can conclude that the intervention is marked by the importance of the issue, but consensus must still be reached on the role of each actor and how actors should interact at different levels of government. 2.3 The contribution from a gender perspective With regard to the implementation of the gender approach, it can be found in the application of the activities executed by PRODES-II. Training on was held on female candidates in decentralized public management, in addition to training programs for male and female council and assembly members. A review of the attendance lists for the training programs shows that the objective of having more than 30% female participation was achieved. These workshops were intended for participants to take gender into account, as well as to build capacities for serving in the positions to which they were elected or for which they were running. According to the data collected from the program itself, those participating in the training program prepared 53 projects during the period between 2009 and 2011, 35 of which included a gender approach. The above mentioned figure covers some constituencies that were not involved in this study; in Ucayali, Ayacucho and San Martin, 11 such projects were recorded.However, this was not the only achievement of the project, as indicated by respondents. From the information provided, it can be noted that the content of the training programs reinforced the gender issue and how it has helped the women who participated in them to develop a clear perspective on the issue and incorporate it in their projects. "I’ve seen that all of the training programs organized by PRODES taught that women have the same capacities as men, both to become involved in politics and to serve in a public function in municipalities or any other public institution. They teach participants that women can hold any position they want and have the right to do so, provided they are properly trained. I personally feel that PRODES has given me primarily academic training, and this training has made me more sure of myself and more aware that I can do anything I set out to achieve, 30 both politically and in the pursuit of development and prosperity for my community.” (Municipal functionary in Puquio) "What we have seen in PRODES’ activities is that equality between men and women in the civil service has not been achieved. It does not exist, so we must work toward equality. Based on what we have learned in PRODES, we have developed a small intervention program in our own regional government, consisting of training both men and women to improve on the execution of the specific functions they perform and civil service itself. With this, we want to ensure that the women currently working in our regional government do not feel displaced in favor of male staff. We want women to become aware of the importance of their role in both public administration and politics in general." (Council Member, San Martin) "Well, in its training sessions and in every program supported by PRODES, a bigger role for women has always been sought. PRODES has accompanied us in the implementation of participatory budgeting. Through its participation, the participation of committees of female entrepreneurs, female board members of wawawasis (day care centers) and most women's organizations based in Ucayali has been encouraged. PRODES believes that the participation of women in politics and in the development of their community should be promoted, which is why it is committed to building their capacities to achieve both objectives." (Municipality of Padre Abad) A finding worth mentioning is that awareness of the gender perspective is raised not only by organizing activities exclusively for women; good results can be obtained in training sessions held for male and female officials at the same time. It should be noted that the training program participants prepared 38 draft proposals for Regional Ordinances. Of the total number of Ordinances prepared, 15 included the gender approach, and 02 of the latter had been approved as of June 2012. If we focus only to the area of the influence of the current research; the regional government of Ayacucho approved the Regional Ordinance N°021-2011 GRA/CR, which declares as a public policy the improvement of public maternal health, being pending of approval: “The internship program for elementary school students from Ayacucho rural areas” and other refer to address the issue of school dropouts in the rural areas of elementary school students. 31 Capacity Building Two approaches are presented in this part of the report. The first one is the quantitative evaluation of the project’s performance and the second one is the analysis derived from the interviews conducted. A third point refers to the main findings derived from the integration of both perspectives. 3.1 Objectives and indicators of the capacity-building program As part of the overall objective of the PRODES program, capacity building for more effective governance was planned for selected Regional and Local Governments (sub￾national strengthening). To this end, training was provided on diverse matters to support the implementation of national policy at the regional and local level, as well as to develop their capacities further. The program focused on providing training and technical assistance to four regional governments and about 300 municipalities. According to the Catalogue of Resources and Materials for the USAID / Peru PRODES Project2, training was provided for public authorities and officials of regional and local governments for the purpose of contributing to improved performance of their functions, and civil society capacities were strengthened for effective participation in participatory public management. The capacity building programs included a training program for male and female regional council members, a training program for male and female municipal assembly members, a certificate program on the formulation of public investment projects with a human development and child welfare approach, a training program for investment promoters, a training program for community leaders, and workshops on results-based budgeting and management, contracting, procurement and strategic planning, among other public management topics. Technical assistance was provided on specialized topics such as the implementation of the Unified Administrative Procedures Text (TUPA) model, municipal tax collection and participatory budgeting, among others, based strictly on each government entity’s needs. 3.2 Quantitative research As the PRODES Project did not include the possibility of rigorous assessment of the project’s impact, USAID included both the qualitative and quantitative approaches in the design of the final evaluation. In this regard, the quantitative approach will be limited to some factors of the program’s intervention on which the data has made rigorous analysis possible. Therefore, it was possible to review secondary information from reliable and GOP official sources. With the consent of USAID, the Evaluation Team proposed focusing the quantitative evaluation on two indicators (Y variables) to analyze the PRODES Program’s performance in the regional governments of Ayacucho, San Martin, Ucayali and their respective provincial municipalities. These indicators were the following: a. Effectiveness in the execution of the investment budget b. Number of public investment projects approved Chapter 3 32 First, the impact of the presence of the program as a whole is evaluated, and then only the effect of the number of training courses and the number of beneficiaries is considered. Given the heterogeneity of the technical assistance provided and the heterogeneity of this type of intervention, they will not be considered in the second phase. These indicators were selected due to the fact that they coherently show whether the expected outcome of the program at the sub-national level, which was strengthening of the capacities of sub-national governments to govern3 , has been achieved. In addition, they are indicators of the outcome established by PRODES in the Performance Management Plan, in addition to being those best adapted to the quantitative methodology to be used. The number of public investment projects approved by the SNIP (National Public Investment System) is an indicator of effectiveness in their formulation and approval. The series that show the projects approved for government entities reveal dissimilar results over the period of analysis. In regional governments, there appears to be a drop in the number of projects, with the exception of Ucayali, while in the case of provincial municipalities, the series appears to have a positive slope. In general, no results can be inferred from this descriptive analysis when comparing the regions intervened with a similar region (control group). 3 Terms of reference, outcome 2 (p. 7). 33 However, the reduction in the number of projects approved does not imply that the amount invested in each one was lower, due to which the indicator is only referential. For example, in the case of the Regional Government of Ayacucho, the number of feasible projects from 01/01/2009 to 12/31/2009 was 157, with an investment amount of 274.9 million nuevos soles, whereas from 01/01/2011 to 12/31/2011, the number of feasible projects was 62 and the amount invested was 338.8 million. The projects in which the largest amount was invested were agricultural sector projects in both years. Source: Governance Barometer Preparation: In‐house Graph No. 1: Approved Public Investment Projects 2008-2011 34 In the case of the Regional Government of Ucayali, the number of feasible projects from 01/01/2009 to 31/12/2009 was 71, with an investment amount of 188.5 million nuevos soles, whereas from 01/01/2011 to 31/12/2011, the number of feasible projects was 67 and the amount invested was 213.3 million. The projects in which the largest amount was invested were in education, culture, and sanitation4 . It should be noted that the fact that a project is approved does not necessarily mean it is executed. In addition, there are criteria for establishing a project’s priority in relation with others. In this regard, there is also a trend among regional governments to approve large-scale strategic projects. With regard to the effectiveness of budget execution, the indicator measures the investment budget executed (accrued) over the total modified institutional budget (PIM) for investments for each year of analysis. In general, trends show a drop for all government entities in 2011, whether intervened or not. It should be noted that 2011 was the newly elected regional and local authorities’ first year in office, and presidential elections took place in July. As it was an election year, according to law5 , non-financial government spending executed during the first seven months of the year was not to exceed 60% of the non-financial expenses budgeted. However, it appears that that in the government entities in which the program intervened, the decrease in the budget executed is less than in the control group, especially for provincial municipalities. 4 Source: SNIP Project Bank. At constant 2009 prices (INEI). For some years, the number of feasible projects in the Project Bank does not tie to the number of feasible projects according to the Governance Barometer, due to which the number from the Project Bank is being used only in this case.   5 Law No. 27245, Art.4. Graph No. 2: Level of budget execution 2008-2011 35 3.3 Training and beneficiaries The total number of training sessions and beneficiaries is shown in Graphic No. 3 for every topic selected for capacity building through the program. For the Ayacucho region, the largest number of training sessions is in the area of institutional reform and modernization with 29 training sessions. However, there are greater numbers of beneficiaries in the areas of management and results-based budgeting and results-based participatory budgeting with 570 and 562 individuals respectively. For the San Martin region, the largest number of training sessions is in results-based participatory budgeting with 43 training sessions, and strategic planning with 28 training sessions. The largest number of beneficiaries is in the area of results-based participatory budgeting with 3109 beneficiaries, followed by strategic planning with 1123 beneficiaries. Lastly, in the Ucayali region, results-based participatory budgeting leads with 33 training sessions and 1242 beneficiaries, followed by strategic planning with 573 beneficiaries and access to public information with 18 training sessions. In general, the greatest demand corresponding to regional governments is concentrated in the area of results-based participatory budgeting, with a total of 2927 beneficiaries trained and 105 training sessions. Fuente: Barómetro de Gobernabilidad Elaboración: Propia 36 Graph No. 3: Number of trainings and number of beneficiaries 2009 - 2012 It should be noted that the number of training sessions and beneficiaries largely corresponds to the demand from each government entity and the target public. For example, training sessions and technical assistance on topics related to the SNIP are geared toward very specific personnel, due to which the number of beneficiaries does not determine the full impact of the proposed indicators. Data estimation methodology, treatment description and management The methodology used is the panel modeling approach, given the complexity of the observation units and the availability of data for each government entity, following Angrist and Krueger (1998). According to Benavente (2004), a set of panel or longitudinal 37 data is that which follows individuals (firms, governments or countries) over time and, consequently, provides multiple observations for each individual. Thus, panel data combine time series and cross-sectional data. One of the advantages of this model is that it allows for correcting important econometric problems associated with omitted variables or unobservable effects, which lead to biased and inconsistent results. That is to say, the panel makes it possible to control for unobservable factors that vary across individuals but remain constant over time or vary over time but remain constant across individuals. A specification of the panel data model is the fixed effects model, which takes the following form: Yiti TRitXitdtDteit  (a) Where subindex i represents each entity and t each year analyzed, Yit corresponds to indicators to be assessed (investment budget execution and number of PIP approved), Xit variables correspond to observable characteristics of the units analyzed, TRit corresponds to the variable and its coefficient, is the parameter to be estimated, which represents the impact of treatment, i denotes unobserved cross-sectional heterogeneity, Dt are dichotomous variables for each year analyzed that serve to introduce temporary, unobserved heterogeneity, and eit is a random variable that represents error in estimation. Thus, in the specification (a) we are assuming that the unobservable cross-sectional and time-varying factors are absorbed by the constants i and y dt respectively, and not by the error (i.e., we assume that they are not random, but rather fixed). This model assumes that individual effects are deterministic parameters which must be found. This is the case where N represents a "population" of government entities, and the inferences relate only to the N available. Thus, the inference is conditional on the individual N and not other sets. Firstly, the treatment has been assumed to be the effect of the program as a whole on the regions intervened without directly differentiating between training and technical assistance. Therefore, a dichotomous variable with a 0 value is specified if the government entity did not receive the program and 1 if it did. The second manner of describing the treatment is to assign the total number of training sessions or total number of beneficiaries to this variable: ‐ Number of training sessions (t): Number of training events held during the year. ‐ Number of beneficiaries (t): Total number of people participating in training events per year. TRi 38 These variables have been calculated at the departmental level (including provincial municipalities and regional governments), as the information available has not allowed adding provincial-level municipalities and regional governments independently. Therefore, aggregation is still perfectible. In addition, five training topics were selected from the total list of topics6 , which can influence the indicators to be evaluated in a more direct way, thereby obtaining two more variables: ‐ Number of training sessions (5): number of training events per year, corresponding only to the five topics selected ‐ Number of beneficiaries (5): Total number of people who participated in training events, corresponding only to the five topics selected As mentioned previously, only the training sessions will be considered in this phase of the evaluation. As technical assistance services are of a very specific nature, they are much more heterogeneous in terms of their type, length and frequency, not only from one intervened region to another, but also from one government entity to another in the same department. This means that the proposed methodology does not show credible results. That said, the total number of technical assistance services is shown in Chart No.A-2 from the appendix. However, despite being more homogeneous than technical assistance services, training sessions have a set of characteristics that should be mentioned. It should be noted that, due to the project’s demand-based approach, the frequency of training sessions also varies; in addition, there is a certain relation with the number of provinces, the size of the population in each region intervened and the type of training each one requires. Adding training courses lasting different lengths of time and on different topics is not a futile effort. In addition, the database prevents determining whether the same individual participated in more than one training program. Therefore, the aggregations are only indicative and the results are can only be interpreted in the sense of the sign, of the statistical correlation between the intervention variable and the outcome variable, and not the magnitude of the coefficient. (A description is presented in Chart No.A-1 in the appendices). The control group for each government entity comes from a baseline study commissioned by the program in 2009.7 For the control group of regional governments, the variables used were total population, number of provinces, percentage of rural population, percentage of urban population, per capita budget, and natural region (coast, highlands or jungle). For provincial municipalities, the variables used were total population, percentage of rural population and per capita budget. This group was useful for a preliminary exploration of quantitative analysis in which the dichotomous variable was used as the treatment variable (whether or not the entity 6 These five topics are: Key aspects of public management; training on results‐based management and budgeting; contracting, procurement and public works; formulation and evaluation of Public Investment Projects; and administrative simplification. 7 Baseline of the ProDescentralization program, governance barometer and composition of the control group.Final report. Patricia Zárate, September  2009. Detailed in Table No.A‐1 in the appendices. 39 received the program). However, in the case of regions, the amount of data is quite small; it covers only 24 observations (three beneficiary regions with their respective controls, for 4 years). In addition, a test of means was made (Table No. A-2 in the appendices), which shows that in most cases, the null hypothesis of no difference between the means of the treatment group and the control group is not rejected. For the second treatment description, where the dichotomous variable was replaced with the number of training sessions and number of beneficiaries, the database was modified to make it consistent with the methodology. The treatment variables were calculated at the departmental level, without making a distinction between regional governments and provincial municipalities, since the data did not allow differentiation. The X variables (population, percentage of urban population, percentage of households with water, percentage of households with electricity, percentage of illiterate population) are also at the departmental level. The number of public investment projects approved is the average of all the provinces in the department. Similarly, the budget execution percentage corresponds to the execution of all provincial municipalities in the department. This may have implications for the accuracy of the results because, on the one hand, there is the treatment variable at the aggregate level of both regional governments and provincial municipalities (total of all training sessions and beneficiaries in every geographical space) and, on the other, the Y variables correspond only to provincial municipalities. Lastly, a test was performed on this second treatment description with a more homogeneous sample, eliminating from the treatment group the departments with the greatest differences in certain observable characteristics from the previous sample. For this purpose, the Propensity Score technique, which calculates the probability of belonging to the treatment group, was used, building a Common Support and automatically eliminating departments outside the distribution. The variables used in the Propensity Score were population, percentage of urban population, and percentage of households with at least one unmet basic need. The Common Support made comparing the groups possible, but at a cost of reducing the sample; only 10 departments remained. For this reason, the sample was reduced from 96 to 40 observations (10 departments for 4 years).8 The data used in this study is from various sources for the 2008-2011 analysis periods. The number of public investment projects approved and the execution of the investment budget (Y variables) were obtained from the Governance Barometer Management Indicators Database. The list of training sessions and the number of beneficiaries was also provided by PRODES (treatment variables). The National Household Survey (ENAHO) for the 2008-2011 period provides the percentage of households with at least one unmet basic need, the percentage of households connected to the public water network, the percentage of households with electricity, the percentage of urban population and the number of years of formal education for each geographical area corresponding to the regional governments and provincial municipalities. Data on mining royalties, transfers received by regional and local governments from the central government from mining tax payments from 2008 to 2011, were obtained from the query module on transfers to regional and local 8 The balancing of Common Support is shown in Appendix 3. 40 governments on the website of the Ministry of Economy and Finance. Lastly, data was obtained from the Population and Housing Census of 2007, from which the total population in 2007 and the percentage of urban population in 2007 were used. Limitations of quantitative analysis The main limitation of quantitative analysis is that this project was not designed for an impact assessment, due to which there is no rigorous baseline or monitoring of indicators suited to this methodology. In addition, there are limitations as to the specifications of the model. It should be noted that what is being modeled are government entities and not individuals; there are non-observable variables that are characteristic of each region. Also, there are limitations to evaluation of the demand-based approach. While it is believed that it is more efficient to provide training to the regions according to their needs, this makes the evaluation very endogenous. It is the same government entity requesting the intervention that ultimately is not consistent across the sample. This heterogeneity in the treatment makes it more difficult to evaluate by causing potential statistical problems such as endogeneity. In it, explanatory variables could be correlated with the error term; there is a bias when determining the causal relationship between Y variables and X variables. Also, it is necessary to detail the reasons why the regions selected were intervened, in order to make it possible to control for these effects and extrapolate the results to other similar regions. For example, it was not possible to observe the quality of the training, which certainly has an impact on the outcomes to be measured (however, according to the information gathered in interviews, the quality of these training sessions was very good). When aggregating by number of training sessions, their number in each geographic region was taken into account, measuring the impact of all of the training sessions separated in 5 topics. It should be noted that the presence of more training programs held by the state or other foreign assistance organizations has not been modeled; nor has the effect the elections may have had on the outcome variables. Similarly, it was not possible to determine whether the same individuals participated in more than one training program or if they stopped working for public entities as a result of their improved capacities. Additionally, there is a lack of information on the characteristics of each regional government and provincial municipality. With such information, it would be possible to build a more homogeneous control group and therefore control for more variables, such as percentage of professional employees, total number of employees, equipment, 9 number of computers, etc. It should be noted that, while the increase in approved investment projects has an expected correlation with the capacities of a limited set of officials, the result regarding the execution of the investment budget involves the capacities of a large group of officials (those in charge of spending, procurement, logistics, etc.) and other institutional variables that the model does not capture. 9 Information contained in the RENAMU survey, available only up to 2009. 41 The quantitative study refers only to variables susceptible to an impact evaluation, which do not reflect the full scope of the intervention of PRODES. In turn, the treatment variables used do not reflect every action taken by the program. Technical assistance is not being taken into account and it is known that the number of training sessions and the number of beneficiaries correspond to the demands of each government entity. Despite these limitations, the econometric estimates give indications of the program's impact on the indicators evaluated. Estimation results Chart No. 1 shows a summary of the estimates, mainly the estimated parameter of the program’s impact on the proposed indicators. The number in parentheses represents statistic t, which indicates whether or not the parameter is statistically significant. Over 1.28, it indicates that the parameter is statistically significant at 20%, 1.64 or over at 10%, 1.96 or over at 5%, and 2.57 or over at 1%. R2 is a coefficient between 0 and 1 that measures whether both proposed X variables explain the model as a whole. Lastly, N is the number of observations in the regression. Approved Investment projects In a preliminary approach, taking the group established in the baseline study as the control group and the intervened regions as the treatment, the results show a positive impact on provincial municipalities for approved public investment projects at a statistical significance level within 20%. In the case of regional governments, the impact is statistically significant. It should be noted that the number of observations is very small in this case. (See also Table No. A-4 in the appendices) In a second approach, when using the number of training sessions as the treatment variable10 and considering the total number of departments, the impact on the number of approved projects indicator is positive and has a significance level of about 15% (see Table No.A-5 in the appendices). These results are consistent with those obtained when replacing the treatment variable with the number of training sessions on only 5 topics (those most closely related to the indicator evaluated), where a positive impact is also observed, and there is a higher significance level of 10% (see Table No.A-7 in the appendices). The results are similar for the number of beneficiaries. Lastly when restricting the previous analysis to observations within Common Support,11 a positive impact is observed at a significance level of 1% (See also Table No.A-9 in the appendices). The total number of observations in this case is 40. The impact and statistical significance level improve when the treatment variable is changed to training sessions covering only the five topics most closely related to the indicator to be evaluated, as occurs when the sample is restricted to a more homogenous one. 10All of the variables at the regional level have been aggregated in this case. 11As explained in the Quantitative evaluation of the project’s contribution section, the sample was restricted to a more homogenous one, in order to make the treatment group more comparable to the rest of the departments. 42 Budget execution In a preliminary approach, taking the group established in the baseline study as the control group and a dichotomous variable related to the presence of the program as the treatment variable, the results show an impact close to zero, but statistically insignificant for provincial municipalities, and for the regional governments, a similar impact at a significance level of 10%. However, the number of observations is very small for the regions. (See also Table A-4 in the appendices.) In the second approach, when the number of training sessions was used as the treatment variable, the impact of the program on the budget execution indicator was almost zero and statistically insignificant. (See also Table A-5 in the appendices.) These results remain the same when the number of training sessions on only 5 topics is used and when the sample is restricted to those of common support (see also Tables No 7 and 9 in the appendices). These results are similar for the number of beneficiaries. Chart No. 1: Summary of the Main Regressions Sample: Treatment and Control Group (Baseline) % Budget Executed (Regions) % Budget Executed (Provinces) No. of PIP approved (Regions) No. of PIP approved (provinces) Treatment (dichotomous) 0.227 0.014 -23,029 7,105 T (1.65) (0.33) (1.21) (1.43) R2 0.32 0.03 0.78 0.04 N 24 200 24 200 Sample: All Departments in the Country % Budget Executed (Provincial Average for the Dept.) No. of PIP approved (Provincial Average for the Dept.) # Training sessions -0.001 0.344 T (0.76) (1.85) R2 0.14 0.08 N 96 96 # Beneficiaries 0 0.008 T (0.85) (1.74) R2 0.14 0.07 N 96 96 # Training sessions (5 topics) -0.004 1.373 t (0.77) (2.06)* R2 0.14 13 N 96 96 # Beneficiaries (5 topics) 0 0.024 t (0.71) (1.22) R2 0.14 09 N 96 96 43 Sample: Departments within Common Support % Budget Executed (Provincial Average for the Dept.) No. of PIP approved (Provincial Average for the Dept.) # Training sessions -0.001 0.543 t (1.22) (3.11)** R2 0.6 0.57 N 40 40 # Beneficiaries 0 0.014 t (1.66) (3.27)** R2 0.62 0.58 N 40 40 # Training (5 topics) -0.008 1.804 t (2.28)* (2.93)** R2 0.66 0.55 N 40 40 # Beneficiaries (5 topics) 0 0.038 T (3.42)** (1.92) R2 0.72 0.46 N 40 40 * p<0.05; ** p<0.01 The different estimates made show signs of a positive effect of the program on the number of public investment projects approved at a statistical significance level between 1% and 20%, which is acceptable considering the number of data. Parameter  is positive and statistically different from zero. The statistical significance level improves as the treatment group becomes more comparable with the control group, as observed for estimates using a common support. Regarding the impact on the investment budget performance indicator, the methodology does not yield conclusive results. One possible explanation may be that the effect on the budget is more difficult to measure, as it not only depends on the capacity of governments, but also on other factors such as the complexity of the National Public Investment System (SNIP), the mechanisms, difficulties and delays in contracting and procurement processes, eventual delays in government coordination in making resources available to the executing units, and other logistical problems in general. In addition, the variable does not capture other interventions through which PRODES and other USAID, MEF and other donors’ projects support improvement in public management in these same environments. It should also be noted that the majority of regressions only include the number of sessions, and not the number of technical assistance services, which were an important part of the intervention. 3.3 Qualitative evaluation of capacity building Training is the component of PRODES best recognized by respondents, not only in terms of recollection and identification, but also due to how it is valued. One of the aspects that stands out is that this attitude is found among those who have been allies of PRODES in conducting training sessions (such as UNICEF or MEF), as well as opinion leaders, authorities and beneficiaries (regional council members, for example). 44 The menu of options developed by the entity evaluated is quite extensive in terms of target publics and topics. The training of council and assembly members, which had the added advantage of being carried out in coordination with government and civil society entities, as well as other donors, was mentioned. Also mentioned was training on project development and the Unified Administrative Procedures Text (TUPA). This forms the profile of an entity that ranges from fairly generic to specialized topics. What is the specificity of PRODES in such a wide-ranging offering? Respondents clearly recognize that the it has developed a very efficient working methodology that has even been adopted for other training processes (this is mentioned especially by PRODES allies; i.e. entities that participated in the design of the training programs). Allies of the program see it the way a person interviewed by the MEF does. This person commented that PRODES has "a well-known training model that has been disseminated by international foreign assistance. Thanks to it, we had successful programs." With regard to entities that worked with PRODES, they repeatedly mention how the alliance enabled them to achieve their objectives. This is the point of view of the Office of the Public Ombudsman ("Our main experience with the Prodescentralization project was the training program for regional council members”) and non￾governmental organizations such as Prisma (“PRODES’ work with the governance and transparency project we executed enabled us to reach 19 regions with the course for council and assembly members”). The same is true for UNICEF, whose objective of offering regions a different type of training was easily aligned with PRODES’ objectives. According to comments received, PRODES appears to have contributed to the experience of others, as in the case of UNICEF; to them, PRODES’ contribution to the human development component is clear. "We were just looking at this matter of capacity building among officials at the regional and local levels. Then we said, ‘Well, we have to do something to help them formulate public investment projects,’ because that was one of the major weaknesses. They did not have all the skills required to formulate public investment projects, particularly on social issues, with an approach centered on individuals. What we did do was incorporate the human development approach that we call person-centered. So we reviewed everything they had done, because we said we were not going to reinvent the wheel; there is already something that has been done successfully. They have a methodology geared primarily toward adults that is not the methodology where you sit in front of the class like a traditional teacher, but rather it is learning by doing.” In cases such as the foregoing, it is clear that there was a synergy that helped build public officials’ capacities and established a different way of conducting training workshops. When analyzing characteristics of the training activities, the quality of the professionals who led the training sessions and the quality of the materials are basically what respondents mentioned. With regard to the latter, there is fairly broad consensus. Some even consider it as one of the program’s most important contributions, as shown by the answers given by the Prisma representative who said, "I acknowledge PRODE’s ability to 45 produce materials; they have produced many materials that have been important in local capacity building.” From the beneficiaries’ point of view, those who received training acknowledge the relevance of the course, the appropriateness of the topic and especially the results obtained from it. In the case of training for council and assembly members, there are areas where differences with regard to previous changes of administration are mentioned, as in the case of Ucayali, where representatives of the regional government said, "Well, I think that members of the regional council have become aware of what decision making means. As a result, the work of the Regional Council has improved. Unlike last year, some assembly members have formulated draft ordinances, and that contributes to governance." Similarly, in Ayacucho it was noted that "What has made an impact is the training program for male and female assembly members ... because the invitation was open and many female assembly members who had never had access to any information on the decentralization process, their legislative functions, representation or oversight, participated. Before that, they were doing the best they could." While these are not results that can be verified quantitatively, it should be understood that, as indicated by the respondents themselves, capacity building requires time to take full effect. This would indicate that this type of training is effective, but its impact depends on the level of training elected officials had beforehand (the lower it is, the greater the potential for a perceptible difference). With regard to training on the TUPA, significant results were also reported in the provincial municipality of Huamanga: "For example, we established our TUPA in a traditional manner, with a cost. A birth certificate would cost ten soles for no reason. We’ve been trained on that; there must be technical support if it costs 8 or 10 soles. For this purpose, a cost study is needed. Then each procedure has its support. That is the contribution we received. Another contribution is that we submitted our first draft to them, in order for them to analyze, evaluate and make observations on it.” This kind of testimonial clearly shows that the type of training provided and the topics selected were appropriate and have produced some degree of change in institutions. Similarly, in Ucayali the presence of more qualified personnel in the regional government due to PRODES programs was mentioned: "In our region we have very solid groups of project developers. In addition, we have project evaluators, such as the head of our Planning and Investment Office (OPI), who completed this certificate program, as did the head of the regional OPI. Having this fine group, we can understand that the quality of projects has improved." Lastly, in San Martin there is a tendency to think that PRODES training sessions in the area, along with other activities they organized (which implies acknowledgment that there is alignment in the selection of the actions carried out), produced changes in organizations, as emphasized by the following testimonial: "At least in San Martin, goodwork is being done, and I think that is the result of the technical support PRODES has provided to some officials.” It should be emphasized that in many cases, it is acknowledged that the activities carried out by PRODES were not limited to training, but also included technical assistance. When this is recalled, the evaluation of the organization’s activities improves. According to testimonials gathered, technical assistance was provided with regard to citizen 46 participation. In any case, it is clear that technical assistance or accompaniment was provided at different levels. “- We were always given roles or commitments that we had to fulfill one way or another. That was your commitment. I’ll train you, and in return, you have to do your part. One of the impacts is the application of the results-based participatory budget methodology and everything that can come from it, such as accountability. That was PRODES’ work and it has been institutionalized." (District municipality of San Martín) The results obtained clearly show that even if the assessment is directly related to the position of the respondent, it can be stated that the training activities have had a positive impact on different geographical areas and on different levels. This means that testimonials reveal changes in the way they operate and their ability to meet the demands of their work both in regional and municipal governments. Positive comments were received from functionaries as well as elected officials (as in the case of assembly members). This reveals not only the project’s concern regarding capacity building in local and regional governments, but also specific contributions that are clearly differentiated from other initiatives. The information obtained also clearly indicates that PRODES expanded its activity beyond the scope of this study (this occurred with both UNICEF and PRISMA; the latter noted that by joining efforts, they were able to reach 14 regions). While this situation is positive, it is also another factor limiting the possibility of conducting a quantitative impact evaluation, as mentioned previously. That those consulted consider PRODES’ performance regarding capacity building to be good does not mean they have no objections or suggestions for improving the effect achieved through training. The most important of these is related to the need to migrate more intensively toward technical assistance. This is because the problems faced by public authorities and officials are often very complex and require follow-up, and also because there is criticism of those who think that a course is sufficient: "- Fortalecer capacidades, resolviendo casos, dejando de lado ir con el cursito o diplomado donde la capacidad de retención es bien poca y la gente después de trabajar duro, tiene que tomarse un fin de semana para entender cosas. Después el que enseñó se va, y la capacidad de retención o aprendizaje es pobre, el enfoque debe cambiar a darse asistencia técnica. Trabajar con ellos, conversar directamente con las autoridades. E:¿PRODES tuvo este enfoque? ‐ Si un poco. Piensa en la Secretaría de Descentralización, tiene un plan de desarrollo de capacidades que son un grupo puntual de diplomados o cursos. Entiendo que políticamente tienen que atender sus necesidades, pero también en una oportunidad les mencioné que tenían que trabajar con ellos en el campo, con los consultores metidos en la zona. Tener el apoyo a nivel de universidades locales para que empiece a florecer el tema." (Former conflict officer) Rather than a criticism of the way in which PRODES has been operating, this type of perception should be taken as information aimed at reinforcing aspects that differentiate this initiative from those existing previously. As noted before, technical assistance is present and must be emphasized. 47 One factor in favor of focusing on technical assistance or giving more importance to this part of the component is the possibility of ensuring that what is learned remains in the institution. This concern is based on high staff turnover, especially mid-level functionaries (although turnover is high among authorities, considering that only two regional presidents were re-elected), which gives rise to the need to continue with training. Respondents see this constant turnover as one of the major limitations of training. Some even mentioned that the persons trained end up as consultants that do not even work in the locality. Although one wants to believe that the effort invested in training is not lost because it ultimately enhances the capacities of those who sooner or later are in a public position, the fact is that the environment in which it operates limits the institutionalization of good practices or of improving processes in beneficiary entities in a sustainable manner. In addition, not everyone has the same learning capacity and not every functionary is willing to follow the program. Therefore, an official of the district municipality of Ucayali remarked, “I would suggest to PRODES that assistance should be more continuous and should be accompanied by monitoring." Along the same line, there is concern about the timing of training. Training during the transition of one administration to another is remembered as an error of PRODES-I. It is considered positive that this time the project is being executed well before the change of authorities. This requires taking into account the electoral timetable for such actions. Respondents believe that training on other topics or tasks that sub-national governments have pending should continue. Among these are everything related to administrative simplification, which is an important topic, considering that a lack of it hinders project execution, according to information obtained from the MEF respondent. In some cases, what is requested is that training on certain topics be extended to other levels of government or other areas. This occurs in the case of the TUPA, a topic that appears to be less of a concern among populations in the most remote areas. "E: How would you evaluate the training? - Well, I've participated in one of the Supervisory Agencies for State Contracting (OSCE) through PRODES. It’s new to me, because it is not, shall we say, the area in which I work, but the one time I participated I found it very interesting, as you can see many things. I've learned many things, but now I find certain details complicated. However, I think it’s very important to continue with the issue of the TUPA. There are many municipalities in our country, in both rural areas and urban areas. The coast contributes more, but in our jungle and in the mountains, people are still not used to contributing and many of our municipalities lose a lot of income for their communities because they are not knowledgeable about what to charge and what not to charge. Work on the matter of capacity building on the topic of these administrative instruments, basically, should be continued.” (Municipality of San Martin) Others believe that continued training on the SNIP is needed and would be justified, given the large number of municipalities that have not had the opportunity to learn how to fulfill this institution’s requirements. Lastly, PRODES’ allies request focusing on matters that are less operational than those covered so far: "PRODES 2 thoroughly covered the content of the original plan that we made in the National Council for Decentralization with MEF and USAID. It appeared to be a simple topic; planning was here, projects were here and budgets were here. Then what PRODES 1 did very well was to improve project management, but there are other topics that were not handled in the same way. One of these is planning, in which there 48 is little interest. That would be a good framework for foreign assistance in general. We shouldn’t neglect the other eight quadrants that are so important.” An infrequent but no less important criticism has to do with the so-called “theoretical” approach of the training courses, referring to the development of proposals that respond to or assume ideal models of relationships between institutions and/or situations in the process, without taking into account, among other matters, the political pressure that public opinion or demands from the population could exert (or including information on how to handle it). This weakness could be eliminated with more practical learning activities, including technical assistance and/or increased joint work on projects geared more toward the population, such as ProParticipación. 3.4 Main findings  The capacity building program has had a positive impact that is demonstrated in the quantitative research by the greater number of public investment projects approved in the regions where the intervention took place than in those where it did not.  According to the qualitative information gathered, the positive effect is due to providing innovative training with adequate materials. Every respondent acknowledged the positive features of the training.  Not being able to make more significant impacts in statistical terms for the number of projects indicator is apparently related to the turnover of personnel that occurs in governments, particularly at election time. Therefore, San Martin and Ucayali show better results than Ayacucho (where there were no re￾elections).  The fact that the training sessions were appreciated had to do with adequate selection of instructors, the development of materials, the methodology and every other aspect related to the logistics of the events. This indicates that it is necessary to continue with this planning process in the training programs.  The topics covered in the training sessions met the demands of regional and municipal governments, meaning that the process for determining them worked, which is vital for the demand-based approach.  In the opinion of participants, when training was accompanied by technical assistance, better results were obtained.  Technical assistance would be more appropriate in high-turnover contexts to the extent that change is institutionalized, based on comments made by those who had the opportunity to participate in these experiences.  Training requiring continuous attendance in face-to-face classes is not always best for senior staff. Technical assistance may be better suited to their needs.  Achieving good performance of regional and municipal governments is not only related to technical matters. Respondents indicate the importance of political considerations, which ultimately have to do with each authority’s style with regard to follow-up on processes. 49  The alliance with UNICEF and introducing the human development component in public investment projects helped to draw attention to issues related to women and vulnerable populations.  The training of council members (male and female) received positive evaluations.  According to the information received, those who participated in these training sessions showed better performance than before.  TUPA training appears to be more clearly related to changes occurring in the institution. This is related to the matter of technical assistance, as well as the fact that in this case, it is easier to follow the technical recommendations.  When reference is made to training on procurement processes, difficulties due to the trend of constant changes in the legislation are mentioned, as this situation creates a need for continuous training.  District municipalities are demanding a longer intervention. It should be noted that they are the ones that place the most emphasis on the fact that the people who received training are no longer there; that is, apparently there is greater turnover of personnel at the municipal level.  Training creates important opportunities for interaction by including personnel from different areas.  The pending demand continues to be diverse, which coincides with one of the aspects of the tailor made option (flexibility), although the problem is some governments’ inability to clearly establish their priorities. 50 Evaluation of the program characteristics In this final part of the document, some of the general characteristics of the program are analyzed in depth. The first thing to consider is that it is confined to respondents' perception of “foreign assistance”, and more specifically, international foreign assistance. There is a positive perception of international foreign assistance in the areas surveyed, where respondents identify it as a critical element in achieving good governance. Along that line, in the case of the San Martín Region, its president’s capacity to obtain international foreign assistance in the region was mentioned. Similar comments were made regarding some districts or provincial municipalities. Beyond the existence of a demonstrable positive correlation or lack thereof, what is important is that those interviewed feel that it exists. Hence, the importance of cooperation as a guiding element of government action: although not explicitly mentioned, it is assumed that one must have a good relationship with it, which implies openness to considering what it proposes. The main findings regarding international foreign assistance are shown in the following graphic. Graphic No. 4 Perception of international foreign assistance When it is mentioned that appreciation of foreign assistance varies from one area to another, it is not a range of positive to negative impressions, but rather of degrees of enthusiasm about it regarding whether or not it is in line with the needs of the area where an intervention is taking place. National entities have an outlook of continuity with greater impact than smaller municipal governments usually envision, although the latter feel they are in great need of foreign The perception on cooperation depends on respondents’ place of residence.It is best in Lima and San Martin. The perception on cooperation is more positive at the regional level than at the municipal level.   It is better in provincesthan in the districts. Cooperation is differentiated between that focused  on strengthening civil society and that having to do with providing support to authorities. Cooperation is differentiated according to whether it is directed toward financing works or improving services or toward a more institutional matter. Cooperation is perceived as necessary. In many casesit performs a function that is abandoned after it is withdrawn. There is a perception that the capacity to coordinate with cooperation is an elementthat has to do with governments’ capacity to do things. Chapter 4 51 assistance. This is probably because they have a different perspective. Central government authorities think of foreign assistance entities as allies (they help them to do a better job), while in other cases, recipients see themselves as beneficiaries (the smaller the size of the government receiving foreign assistance, the more they see themselves this way). Respondents see foreign assistance as having to do with aid to governments on three specific matters: financing of civil works, institutional development / improved administration and the strengthening of civil society. Often, governments’ greatest expectations are related to the financing of civil works. While international foreign assistance is perceived as being necessary, this does not mean it should not be improved. In some cases the need for better coordination between organizations working in the same area or with similar objectives was mentioned. In this regard, it is important to analyze how PRODES fits in with other international foreign assistance efforts taking place in the area. The answer, according to indications in previous sections herein, is that there is no impression of overlapping. In general terms, U.S. foreign assistance is positioned with more institutional issues, while other countries are more concerned with the economic environment (economic development programs). This does not mean that the impact of programs such as crop substitution is not acknowledged, but there is a feeling that it comes with a more general proposal, immersed in the promotion of institutionalization and democracy (a political analyst expressed his opinion along these lines). On the less positive side (the extent to which people see intervention as having to do with very specific interests), it is more closely associated with intervention in areas where drug trafficking is taking place. When referring to the limits of international foreign assistance, respondents allude to excessive orientation of training. With regard to this situation, they question its potential impact due to lack of follow-up (which has more to do with technical assistance than training), high staff turnover, lack of time among high-level officials (implying that training opportunities come late when they are already authorities, due to which developing leadership schools for those planning to run for office at some point was proposed). One aspect that goes beyond the focus on training is the heterogeneity of the areas intervened. According to comments received, the actual situation of the different regional and local governments in which intervention is being planned is not always taken into account. Discussion of international foreign assistance involves referring to different entities, different sources of financing. Basically, German, Canadian, Spanish and American foreign assistance are distinguished. Regarding American foreign assistance, mainly USAID and AID are mentioned. Although respondents noted that they did not perceive any overlapping between cooperating entities during the process being evaluated, the possibility of this occurring should not be dismissed, given that GIZ has also decided to become involved in management issues. 52 GráPH N° 5 La percepción sobre la cooperación norteamericana When referring to U.S. foreign assistance in the area, Chemonics is mentioned (more so in the case of San Martin), as well as PRODES and ProParticipación. Between the last two (both linked to USAID) respondents see a clear difference. While the former is associated with work among authorities, the second one is seen as strengthening civil society, although at some point, people felt that PRODES also carried out actions that benefited civil society. In general terms, they perceived that the programs complement each other rather than overlap. Some respondents perceive that PRODES and ProParticipación represent not only a choice of different target publics, but also different ways of presenting USAID’s image, with the PRODES approach being a limiting factor in influencing the regulatoryagenda: ”PRODES is a project which AID gives to a contractor; therefore it is an AID project. AID and this institution are the official foreign assistance of the U.S. government and consequently, their link has to be with Peruvian government entities. In a different case, foreign assistance could collaborate with civil society, so it would not be an AID project, but rather a group of civil society institutions that use other existing mechanisms in their foreign assistance. The Propuesta Ciudadana (Citizen Proposal) people can appear on television with a clearer, more political position because they are not committing AID, but rather a citizen proposal. " (Assembly of Regional Governments official) As indicated in the preceding point, PRODES established alliances with a number of entities. However, there are demands for greater coordination, which tend to come from entities related to citizen participation: "PRODES should seek greater articulation in its intervention. Even though it articulated with other USAID programs because USAID required it, with other projects there was insufficient articulation with parallel organizations that are working on similar things to improve the quality of the intervention. I think an effort was made to do that last year, but this aspect was neglected in the first two or three years.” (Mesa de la Concertación de Lucha contra la Pobreza San Martin / San Martin Round Table on the Fight Against Poverty) Work has been done for  many years on topics related to Decentralization . It has two differentiated branches: PRODES geared toward authorities and ProParticipación toward civil society. Its programs are perceived as complementary rather than   overlapping. There is a clear differentiation with cooperation from other countries.   There is a positive impression among those that have had contact with it. This does notmean they see no roomfor improvement. There is a positive atmosphere for its presence, including a demand for continuity. 53 Focusing on PRODES also entails discussing the demand-based approach. This concept is not always clearly identified by respondents, although flexibility is recognized as a feature of the program. For example, in the assembly interviews, it was mentioned that the demand-based approach is relative, because the main concerns of the cooperating parties are also present, in this case the war on drug trafficking: "In theory, it is the right approach. However, there is a catch, which is that I identify the needs of the people and, as I'm the one with the offer based on the needs of people, they’ll come to me because I’m the one with the offer and it’s free. I’d say that cooperation is a more reasonable approach. It’s an approach based on relative demand. Sometimes the approach is conditional on the crop substitution policy." Suspicions aside, and taking into account the information provided by the people who were in charge of the program, there has been an orientation toward working with the beneficiaries identified, for whom the intervention was designed. Has the demand-based approach been positive in the achievement of the project’s objectives? There are arguments both for and against this approach. On the positive side, there is the ability to adapt to an extremely heterogeneous situation such as that of sub-national governments and even at the regional level. Another positive aspect is working on what the authorities are willing to prioritize, which assumes greater commitment on the part of governments. On the negative side, there is the possibility of losing sight of the program’s objectives, due to having a perspective based on the context and depending on the beneficiaries’ capacity to establish their needs adequately. For this reason, there are critical views of this process in institutions that could emulate the approach if they consider it appropriate: "The logic of the PRODES process was demand. They had defined areas or regions, and its analysts were those who could identify local demands and act on those demands. The advantage of this is that it guarantees commitments for joint action, but the disadvantage is that one cannot reach everyone because not everyone has the same demand. There are others have not established the same need and they are the ones in greatest need. Unfortunately, those who make the fewest demands are those with the greatest needs, but they lack the ability to make demands." (Prisma). Regarding this point, it should be noted that improvements can be achieved in project formulation, but not on proposal of broader development plans whose preparation is not always valued in the short term due to pressure to obtain results. The validity of this information is evident among those who insist on following the line of providing training for council and assembly members first. The interviewee from ProParticipación said, "The advantage is that you go where you are needed and there you see possibilities that your intervention will be beneficial. The disadvantage is that if you are constantly running from one place to the next, you don’t establish long-term processes." The results obtained from quantitative analysis lead us to believe that while the demand-based approach led to programs that were differentiated from one region to another, this situation did not lead to similar results. This is not due to errors in the identification of demands, but rather to differences inherent to dealing with regional governments with varying degrees of institutionalization and development. PRODES is seen as a program focused on improving the capacity of regional and municipal governments. To a lesser degree, its involvement in the regulatory field and the development of tools to help governments and opinion leaders to better evaluate progress on the decentralization process and to evaluate the performance at different levels of government are acknowledged. 54 This occurs because the focus on training and technical assistance is clearer. Those who have worked for the PCM state that "PRODES had its role, providing support through publications and training that were of great help in clarifying the picture, because unfortunately changes of authorities always bring that sort of problems." The problem with training is that it must deal with the political situation in each location, an aspect which is not taken into account when selecting and designing interventions. In order for training and technical assistance to have a positive impact, minimum conditions are required on the following matters (according to information gathered in interviews): institutional stability, existence of leadership, continuity in government (which reduces turnover somewhat), articulated civil society, an economic development orientation, good budget execution, the ability to consider investment quality and foresight regarding development. These conditions (existing or non existing, in addition to the level attained) are the minimum conditions that should exist when intervention areas are selected, even more so in the case of the demand-based approach). Otherwise, the project will run the risk of having to deal with situations such as the following: "...In the case of some departments, clashes and the lack of consensus and dialogue hinder progress; it’s complicated....I’ve talked with Cesar Villanueva quite a bit to work on the matter of strengthening regional governments, since there are negative perceptions that spread and result in a number of problems ranging from recalls to internal conflicts.” (MEF official) This is where the need to design interventions that help balance the distance between one area and another arises. That situation leads many to doubt whether the decentralization process is something that should progress equally for everyone. ("Before getting concerned about how the transfer process is going, let’s be concerned about whether they are ready to assume new functions. It seems to me that transferring education to them is only going to increase educational differences among Peruvians, because there are great distances between one regional government and another.” (Analyst and researcher) The heterogeneity of contexts must be taken into account when analyzing the results for each of the regions evaluated: San Martin and Ucayali have re-elected administrations that differentiate them from Ayacucho. In turn, the dynamics in San Martin appear to be different from those of Ucayali, mainly because their views on corruption are markedly different. In addition to this heterogeneity, there is that caused by serving regions, provinces and districts. Is it appropriate to work on all these levels simultaneously? The answer has two dimensions. On the one hand, it is true that it is difficult to measure the impact obtained with the different actions carried out, while on the other, it is undeniable that benefits were obtained. PRODES’ way of working has brought about articulation (an unexpected benefits of the intervention), which is extremely necessary in light of the loss of spaces that should produce it (National Council for Decentralization/CND) and the low level of efficiency in achieving this objective in bodies such as the Decentralization Secretariat of the PCM: "The National Council for Decentralization had several problems, but a great advantage was that in that space, the three levels of government were trying to work toward decentralization together. What the Decentralization Secretariat did was concentrate all decision-making and lines of action in a second floor office of the PCM and end coordination among the three levels of government." (Assembly of Regional Governments) In light of this situation, it is important not to lose sight of articulation as a means of creating a climate conducive to decentralization. From the discussion in this section, the following findings can be indicated: 55  PRODES II is a program that fills an important vacuum in terms of articulation among central government entities, nongovernmental organizations and the different levels of government. Its absence raises the need to establish who will assume that role.  PRODES II is generally better known by respondents for its capacity-building activities than those related to regulation, which does not mean it did not have a major role in this matter. It is not easy for international foreign assistance to influence the regulatory framework, much less to ensure that certain political processes take a given direction. The decentralization process is a political process, not just a technical process. This fact should be taken into account more in decisions to be made on the next part of the project.  The demand-based approach is controversial. The main concern is that the beneficiaries themselves do not know how to identify their priorities. The main advantage is greater commitment to take action on the part of beneficiaries.  To optimize the demand-based approach, an institutional assessment of the governments with which the project will be working is necessary.  There is no statistical evidence that the demand-based approach has been decisive in the positive results achieved by the program.  Authorities and technicians must be aware of the processes that public administration demands. Therefore, both levels must be trained. The problem is that training authorities is extremely complex.  The program’s best performance on the matter of capacity building has to do with methodological innovations and the quality of the activities carried out (including materials). For better achievement of the objectives, the following recommendations should be taken into account:  Strengthen alliances with all kinds of entities. This will ensure that the experiences gained are reproduced by other actors in other settings and contribute to institutionalization of the practices.  Coordinate with other sources of international foreign assistance to ensure that there are no overlapping efforts.  Ensure that PRODES and ProParticipacion have similar intervention areas and align their work better.  Articulation should be a more specific objective. This implies having the objective of fostering more explicit regional alliances and associations. 56 Conclusions From the data gathered over the course of this study, we submit the following conclusions: In relation to the regulatory framework of decentralization:  Currently, the decentralization process is going through a period of stagnation, one cause of which is the lack of a clear definition on the part of the central government about what they want to do with this process. There is no consensus about which direction decentralization should take, there is no strategic vision, or at least none has been made public and this hinders efforts at the national level. This situation will affect the execution of PRODES’ next phase and the expectations that develop around the project.  The progress that decentralization process has made in recent years owes itself primarily to the momentum coming from the country’s interior and fundamentally, due to the advances achieved by regional governments. They could become the catalysts of this process, a role which will require continued support and particularly more technical assistance.  In order to ensure that the decentralization process moves forward with the regulatory framework, policy spokespersons are needed. It is essential that regional and local governments have a greater capacity to develop proposals and exercise influence and leadership in order to have bearing on decision making.  The materials developed by PRODES are appreciated for their quality. However, these need be given more visibility in order to become a real benchmark or a common starting point in other areas where the project currently has no presence.  The diagnosis performed using the Decentralization Map and the Decentralization Compass is still valid and the aspects identified as needing more work to be able to move forward with the decentralization process are still pending.  The task that still needs to be carried out is building a consensus, mainly in relation to priority actions. Insofar as this occurs progress can be made in preparing a regulatory framework that fundamentally resolves the definition of the powers and jurisdiction of each level of government and the ways in which they should interact and coordinate.  The Congress of the Republic of Peru lacks the resources to replicate the processes involved in producing the Decentralization Compass, which they consider necessary in order to update the document. The document itself is Chapter 5 57 perceived as too heavily focused on the PRODES II project areas of intervention and Congress may open up the debate at the national level.  The technical assistance used to prepare legislation has been an effective way to overcome the inherent limitations of being part of international foreign assistance in helping to enhance the regulatory framework.  The creation of spaces for coordination between the different levels of government was one of PRODES correct choices for minimizing social conflict.  Social conflicts represent a complex set of problems where, as with decentralization, the authorities and responsibilities of each level of government have not been defined. Confrontational situations are created due to the lack of clarity in these roles, which makes conflict resolution and coordination efforts difficult.  Politics carries a good deal of weight when trying to deal with social conflicts. Because of this situation, an approach that is exclusively technical, an approach primarily dedicated to building capacities through the sharing of methods that are useful in social conflict management will rarely succeed.  There is no consensus about what the causes of these social conflicts are, a situation that hinders the application of the demand based approach. This, insofar as government authorities have not necessarily been able to identify what their needs actually are.  Adequate conflict prevention/resolution management requires unique approaches for each region or township; since there are underlying problems that need to be addressed, similar to what occurred with the artisanal and small-scale mining law. About the gender approach  We have observed positive impacts on the lives or professional development of the participants in program activities and those in their environment.  The activities carried out by PRODES have allowed initiatives to be carried out that are aimed at reducing the gaps between men and women and laws or actions aimed, for example, at giving women more visibility whether as government employees or as beneficiaries of concrete actions on the part of the government.  The training workshops on the subject of budgeting have helped to raise aspects or issues that are more valued by women and were not being taken into account by the authorities. This refers to aspects related to the development of projects related to human development or related to childhood, which were submitted by those who participated in the training programs for councilmen and council women, assembly men and assembly women, also participated in drafting ordinances that included the gender approach.  Those who participate in the council men/women, participate in the development of ordinances that included the gender focus. 58  Gender topics were included in all activities directed toward both men and women and not only those organized exclusively for women. About capacity building  The PRODES intervention generated a positive impact with its presentation of projects (in a quantitative format) and actions have been identified which have shown governments to be more efficient as a result of activities implemented by PRODES.  Technical assistance would be more appropriate in contexts of high rotation, insofar that change is institutionalized based on what has been expressed by persons who had the opportunity to take part in these experiences.  To become more efficient, capacity building needs to incorporate both training and technical assistance, with more emphasis on the latter. Technical assistance is what enables us to be more likely to be successful in having individual knowledge transformed into institutional change.  The PRODES approach strengths include that it forms alliances, develops innovative methods, designs quality materials and responds to the needs defined by the beneficiaries.  The PRODES model weaknesses include the heterogeneity of the intervention, the fact that the users are not always clearly aware of what their needs, the difficulties in attracting the most senior public officials to participate in the training along with the high staff turnover rates at the levels of government where we have been working.  Capacity building is still in demand at different levels of government. However, we need to be more diligent when identifying if what they really need are the abilities to apply established procedures or if there are other more fundamental problems (the lack of the ability of political leaders to make correct decisions).  Training assembly persons and council members (male and female) is a way to assist political leaders to develop a clearer vision about their responsibilities and how to help satisfy the demands of the population.  That the training programs for assemblymen/ assembly women and councilmen /council women incorporate a vision of gender that should be accentuated in order to generate a greater number of approved regulations on this issue. Program characteristics  The demand based approach has been a source of differentiation for this project. However, there is no evidence to suggest that the program’s achievements owe themselves to this aspect, or that this approach is the only way to gain beneficiary trust and commitment. 59  The demand based approach has lead to a dispersion and heterogeneity that does not allow comparison of the results obtained in one place or another  The application of the demand based approach should include an in depth analysis of the institutional situation of the municipalities subjects of program intervention.  The work carried out with the regional governments, provincial municipalities and district municipalities has resulted in the benefit of creating spaces for interaction and coordination. This work method can be counterproductive in terms of the amount of impact it generates because it leads to the dispersion of activities and resources.  PRODES has carried out its program in locations where other USAID sponsored programs and international foreign assistance agencies operate, without overlapping efforts. There was a case with a US foreign assistance agency where efforts were coordinated to complement each other, between Pro Participation (aimed at the population) and Pro Decentralization (aimed at the authorities).  PRODES continues the trajectory of US foreign assistance, which is strongly devoted to the aspect of institutional strengthening. 60 Lessons learned In relation to the PRODES II Project, we list the following lessons learned andrecommendations:  While as an agency of international foreign assistance, USAID does not take a leadership role in promoting the institutionalization of intergovernmental coordination as a part of the decentralization process, it is recommended that it continue to offer neutral spaces of interaction between the key players in the National Government (Executive and Legislative Branches), the Regional Governments and Local Governments, with a view toward paving the way for the formation of networks and the development of synergies that could potentially help to build a new consensus as to which direction the decentralization process and its applicable legislation should take. Any space that is conducive to the development of intergovernmental relations based on trust and cooperation between the actors of these three levels of government can contribute to overcome this process’s present stagnation and move forward with developing the key policies needed for successful decentralization.  Although since its inception decentralization has been conceived in legislation as a top down type of process, it would be meaningful to reinforce USAID interventions by supporting the development of consensus and articulating existing proposals on the guidelines of the decentralization process at the sub national levels of government. The processes of horizontal coordination should be enhanced since this can lead to a Bottom- Up type of dynamic or the formation of coalitions dedicated to encouraging policies that contribute to resolving the problems that currently have the process at a standstill. Along this line of thought, a deeper alliance should be achieved with the Assembly of Regional Governments, as well as with the entities that assemble the municipalities.  With regard to the Decentralization Compass that was recently transferred to the Congressional Committee on Decentralization, it is advisable that it be monitored to ensure its continuity and particularly to ensure that it remains an instrument used to gather and articulate the points of view voiced by the different national actors.  As for regionalization, we recommend continuing to work with initiatives related to organizing town associations and with the Interregional Coordination Committee as processes which seek the development of a dynamic and consensus that enables the process to move ahead.  In relation to regulatory reforms which require political will and a high degree of consensus, it is recommended that USAID continue to maintain a low profile, offering technical assistance and foreign assistance as a function of what key decision makers require and opportunities the context offers. Capítulo 6 61  It is recommended that emphasis be place on working through strategic alliances, given that they allow one to join forces and obtain resources, and they generate a broader impact from the activities carried out it is likely that activities continue to replicate themselves beyond the PRODES PROJECT scope of intervention.  The beneficiary entities perceive and appreciate project activities more positively when there is greater presence of technical assistance. More emphasis must be placed on technical assistance. Migrating more intensively toward technical assistance must be accompanied by continuous supervision.  The electoral calendar should be taken into account when scheduling program activities, particularly training activities. This also relates to the need to evaluate the existing level of institutionalism in the area of intervention.  In regard to the future of capacity building activities, we need to consider the following: a. Training strategies need to be established that consider the high government staff turnover that takes place following an electoral process. b. When training high level government officials, we need to provide them with elements that help them understand the importance of building and maintaining a career in public service, and to work as career officers. c. Mechanisms should be found to ensure that training becomes more institutionalized and beneficial to the governments selected for program participation. One possibility is to establish that government officials provide their commitment to remain with the entity and with those who receive training identifying the expected results or goals. In this manner they would utilize foreign assistance agency and government resources more effectively. d. More emphasis must be placed on encouraging the adoption of good practices, which means being able to clearly identify which regional, provincial or district governments are implementing good practices and with what results and then to findthe mechanisms to provide visibility and replicate them. e. We must continue to ensure that the training strategy considers the quality of the professionals, methods and materials. This factor was the reason that motivated everyone who received the training to recognize the relevance of the courses, the appropriateness of the subject matter and above all the existence of results that are a consequence the training itself.  One relevant challenge consists of improving the availability of data in order to conduct a rigorous assessment at the time of a program’s inception. In this regard, the barometer should have the information on the number of PIPs prepared in its data base so that we can determine the ratio of the No. of PIPs approved/PIPs prepared, which is better indicator of efficiency.  In relation to the above, while we have access to figures regarding the total amounts of approved Public Investment Projects at the end of each year, it is 62 worthwhile to have available data on the amounts involved in each investment project or other relevant indicator for each and every project in every government entity over the course of the entire sampling period.  In the future we should seek a stronger relationship between the program design, its implementation and data collection, with a view toward increasing the possibility of performing our assessment with rigorous methods. Data collection should be consistent with the proposed assessment method and defined from the beginning of the program. In this regard: i) we suggest reviewing a reduction of the heterogeneity in the training, and ii) keeping more detailed and computerized records on the parties who receive training and of participating government entities.  We suggest that certain aspects of the demand based approach be revised. Beyond its advantages the approach can differentiate its effectiveness depending on the relative capabilities of governments to correctly identify their needs. In this sense, the demand based approach employed by the program could be supplemented with an evaluation of the corresponding specialist so that training content is tailored to the needs of the regional and municipal governments and not solely because these governments request it. This point becomes more relevant in the case of a variable difficult to model such as the Execution of the investment budget.  Given that there are signs of a positive effect in the number of approved public investment projects, the program should make recommendations to the governments about the importance the role that job stability of government staff and employees who work in these relevant areas represents, or ensure that the capacities acquired as a result of the program are not lost due to government officials and staff turnover.  In the case of training workshops on the subject of budget execution, the program should meet with those involved (government entities, experts, etc) to look at performing a diagnosis based on the determining factors of budget execution, such as the professional profiles of human resources, staff stability and turnover, compensation policies, the quality of procurement and logistics processes, among others. The analysis should seek to distinguish whether these aspects are more relevant and if this is the case, determine whether it is economically viable to multiply training efforts in terms of quality or quantity.  The gender approach should be maintained and even enhanced in order to achieve that the projects, ordinances and regulatory designs always include these principles. The target objective should always be to increase the number of approvals. 63 Theoretical framework of the study The proposed evaluation design comprises two approaches: one is quantitative and the other qualitative. Below we present a methodological description of the implementation of both phases. 1.1. QUANTITATIVE STAGE This section will provide only a few ideas that summarize many theoretical and empirical developments. In principle, we will employ two approaches to evaluate the project: the conventional impact assessment approach (quasi-experimental) and different versions of the panel data model. If there is insufficient data to conduct a rigorous impact assessment, we will make our best effort to construct simple correlations and descriptive statistics of the primary outcome indicators, taking into account the unique characteristics of the sub national governments covered by the project. 1.1.1 Quasi-experimental approach In order to know the impact or benefits of a program on its beneficiary population, one must identify the cause and effect relationships between the components (goods and services) that the program produces and the expected outcomes or variables of interest upon which the program’s benefits are defined. To achieve this, it is essential to isolate the observed benefits (specifically the intermediate and final outcomes), all those impacts on the beneficiary population derived from factors outside the program 12 , and which would have occurred anyway without the intervention, as a consequence of the normal course of events in the environment or the actions of other public or private programs. Therefore, to quantify the program’s impact, it does not suffice to compare the “before and after” status of program beneficiaries, but instead, respond as accurately as possible to the question: What would the situation of these beneficiaries be today if they had not participated in the program? This is called a counterfactual conditional situation. In methodological terms, the above involves comparing the program’s final outcomes (and the intermediate results as applicable) on the beneficiaries relative to a group of non beneficiaries with similar characteristics (called “control group”), and in so doing, isolate the effects of other factors. A control group is basically a segment of non beneficiaries who are part of the target population, i.e., a group that shares the social, economic, educational, etc. characteristics of the beneficiary 12 To be precise it is worthwhile to isolate the program’s causal effect only from those elements that are correlated with the program assignment (factors that are unique to the group of beneficiaries or related to the level or magnitude of the benefit provided). Those elements which affect the final outcome but which have the same effect on the beneficiaries and about whom there is no relation to the benefit do not need to be included in the measurement of the impact.       Appendix 1 64 group. The only difference between the individuals in the control group and the eligible beneficiaries should be their participation in the program. The evaluation of the ProDecentralization project proposes a unique challenge: the treatment units are sub national (regional, provincial and local) government entities, and so it is not a trivial matter to find units that are strictly comparable on observable and unobservable aspects. For the same reason, to the extent this is possible, we shall probe for data that enables us to at least construct alternative methods to control the most diverse set of characteristics and the differences between the treatment units and the control units and to construct the best counterfactual scenario. In summary, an impact study requires prior knowledge or assumptions about the process that generates the data and enables us to distinguish causality of association. The statistical methods for estimating impact parameters are selected according to each program’s characteristics: objectives, design (benefits, eligible population, mechanism of assignment to treatment, etc), secondary data or those that are possible to obtain through some method of (primary) data collection. As a minimum, the data contains the characteristics of the individuals (or other prominent unit of analysis), variables of the intermediate and final outcomes, other variables capable of influencing the outcomes and those originating from surveys and administrative records). Below, we provide notes about some relevant topics a) Causality and the problem of identification The impact assessment of a given program requires making an estimate of the causal effect, attributable exclusively to participation in the program (treatment D) in outcome variables (Y). The key problem is the association between variables D and Y, which can be due to: ‐ Effect of D on Y (causality) ‐ Effect of Y on D (inverse causality) ‐ Effect of a third variable X on Y and D. (“X’s effect becomes confused with D’s effect...”) For example, one can ask him or herself if the training or reinforcement activities have achieved any effects on the execution of the budget or on the increased level of tax revenue collection of a local government. In this case, it would not be enough to compare the results of districts that received training or technical assistance and the others if for example the best results are more common in the regions or districts that have greater resources (technological, human, etc). In this case, the effect of training or technical assistance becomes confused with the effect generated by other inputs: if other inputs have a positive effect on achievements, we will observe a positive association between some districts that were received treatment and others, even when there cannot be any direct effect from the ProDecentralization Project. The program’s impact corresponds to the difference in the outcome variable (Y) recorded by individual i (or the sub national government) with or without the program’s intervention.   YY 01 iii (1) Satisfying the condition of independence requires the random selection of the individuals who benefit from the program, known as an experimental or random design. As previously stated, the ProDecentralization project requires a non experimental evaluation approach as long as the data is available. 65  10   /, XDYY b) Quasi experiments Matching Method For various reasons, a randomized control group is often unobtainable, so this methodology will attempt to recreate the conditions of an experiment when it is not available. By aligning the distribution of the characteristics of the untreated population and the beneficiary population, the method replicates a feature of the random (experimental) data (Blundell and Costa Dias 2002; Heckman, Ichimura and Todd 1997) The “Matching” method, attempts to estimate what the outcome of the treatment group would be if it had not participated in the program, starting with the pairing of each participant from the sample with members of the non treatment group. Once each beneficiary is paired with his/her/their respective control group member(s), the only difference between both groups is their participation in the Program. The quasi-experimental groups have a problem in common that arises in practice for the majority of programs. It is very difficult to obtain an identical control subject like that obtained in randomization. The Matching method rests on the assumption that the assignment to treatment is independent of the potential outcomes; once it is controlled by a set of observable characteristics (called “confounders”), it should satisfy the condition of conditional Independence: The main problem with matching estimation (in cross sections) is that if there are unobservable elements between the treatment and control groups that affect the final outcome, the indicator measuring the impact of the program will be biased. Differences in differences (double difference) One condition necessary when developing a quasi-experimental design is the existence of a data base (or several bases that enable us to form a common database of beneficiaries and non beneficiaries that has information for both groups about program eligibility criteria (for example: socio economic status, population size, average level of education in the sub national government, geographic location or whatever other characteristic that enable the measurement of the final outcomes or expected impacts of the intervention (for example, achievements in tax revenue collection). Once the control and treatment groups are formed and having obtained the observations of the outcome variables and the variables that characterize the individual and his/her environment in two moments in time (before and after the program’s intervention), the impact of the program will be quantified through econometric estimation in the following model, where the estimator of parameter α (ˆ ) corresponds to the impact of the program: Y i3i21it itii *X)Pe*TR(PeTR it        (6) Where, Yit = outcome variable for individual 1 en period t. TR i = variable equal to 1 is individual i belongs to the treatment group and 0 if he or she belongs to the control group. 66 Pe i = variable equal to 1 if the data from the outcome variable for individual i belongs to the post program period and 0 if not. Xit = set of variables which characterize the individual and his/her environment. it  = random error term The estimation of the model of equation (6) is known as the double difference method. If outcome data is available for both groups (treatment and control) before and after Program implementation, it is possible to compare the differential evolution of such results and obtain a credible assessment, provided that the differences in unobservable aspects have remained constant over time; that is, if the average outcomes of the beneficiaries and control group members had followed a parallel path in the absence of the treatment, which is not always true (Abadie 2004). Instrumental variables and regression discontinuity The bias from unobservable characteristics is a problem that leads to developing other methods such as the instrumental variables method. This bias is derived from the existence of factors that affect participation in program (Pi) and the researcher is unaware, for example of the disposition to participate in the training program on the part of officials or the more competent public employees. In statistical terms this implies that in the equation: Yi  21 XTR ii *   i (7) The error term i contains unspecified variables in matrix Xi which affect participation in program (Pi). To correct this bias you need to use the estimated value of P, according to the formula 7, but including a variable Z that determines participation in the program, but has no effect on the results. That is, a variable that is not in X and is not correlated with the error term. This variable is called instrumental variable and its effect is that it allows Pi to vary independently, eliminating the cause of the bias (Angrist et al., 1996). Finally, another method that is widespread internationally is regression discontinuity, which, in a quasi-experimental context, seeks to use the discontinuities and delays of program implementation. For example, a cutoff score could be established for determining eligibility that can be used to obtain estimates in the neighborhood that can serve to make those that are found around the threshold comparable, beneficiaries and non-beneficiaries. The method has objectives similar to those of instrumental variables, because it uses an exogenous variable that has a strong correlation with participation, although it is dissimilar (Imbens and Lemieux, 2007; Khandker et al. 2010). 1.1.2 Panel data model approach In the event it becomes too complex to construct a counterfactual scenario for each beneficiary and if data permits, one can put together a panel data model for the impact assessment, following Angrist and Krueger (1998). According Benavente (2004), a set of panel or longitudinal data is one that follows individuals (companies, governments, or countries) over time and consequently provides multiple observations for each individual. In this manner, the panel data combines data in a time series and a cross section. This model assumes a specification such as the following: 67 Y  iit  it XTR   itit ݐ݅ݒ ൅ μi ൌ ݐ݅ݑ :Where Note that μi denotes specific unobservable individual effects that are time-invariant. These account for any type of individual effect not included in the regression. In regard to the term vit, this refers to the so called idiosyncratic effects. Panel data models Fixed Effects: This model assumes that the individual effects μi are deterministic parameters should be found. This would be the case if N represents the “population” such as a series of sub national governments and the inferences are only related to the N available. Thus, the inference is conditional on the N individuals and it is not making an inference for other sets. Panel model with Random Effects: Este model assumes that the individual effects are not deterministic but that each one is a random variable. This is a reasonable assumption in the event that the data were a genuine sample used to make an inference about the population as a whole. Hausman Test: When there are doubts about the most suitable model, we use the Hausman test, which simply asks if there are significant differences between the estimators for Fixed Effects and Random Effects. Dynamic panel models: These models enable us to estimate ratios of a dynamic nature and which is impossible to estimate using cross sectional data. For example if the tax revenue collections of a sub national government depends on revenue collections from the previous year. 1.1.3 Variables to use, potential sources of information and constraints In principal, it is essential to have indicators that can demonstrate whether a project is effective or not, both for the quasi-experimental approach and the panel data models. To the extent practicable, we will attempt to use the two above mentioned approaches depending on the variability and availability of the data (for example, the frequency of the series for the panel data models or the availability of variables to control the differences between the different government entities). As variables to explain (endogenous variables Y) we will use the impact variables proposed in the document PERU PRODECENTRALIZATION PROJECT: ACTIVITY AND PERFORMANCE PLAN MAY 2009. It must be noted that it will make a difference when measuring the impacts between the government units that received technical assistance and those that received training. The most important results are: 1. Increase in the percentage of public investment projects declared as "viable", according to the SNIP. In particular, this is the average percentage by which the sub national treatment group exceeds the number of public investment projects declared as viable. 2. Increased budget execution per government entity. This is the average at which the sub-national governments in the treatment group exceeds the percentage of execution of public investment in the non treatment group (control group). 3. Improved administrative efficiency and simplified registration and licensing procedures for new companies. This refers to the percentage of sub-national governments from the treatment group that has adopted the unified (simplified) administrative procedures TUPA. The units that received technical assistance will be distinguished from those who received training. 68 4. Improved tax collection. This is measured as the percentage of municipalities in the treatment group who have increased their tax revenues by at least 10% above those of the control group. As in the previous case, those units that received technical assistance will be distinguished from those who received training. In addition, we will attempt to conduct the evaluation with the mandatory indicators proposed by USAID. In any case, ideally we would want to have constant variables for specifying Y. Another critical aspect in the panel data estimation method is that one can build data based on the TR intervention for every geographic unit examined and for every year, ideally from the period prior to the project startup year. In this sense, having data available on the budget executed by the project, or the number of public workers or government officials trained would allow to take advantage of an eventual graduated progression of the treatment "dose" for each geographic unit. Thus, one could make an approximate estimate of the "marginal effect of one Peruvian Nuevo Sol spent by the program” or "one additional trained staff member." In relation to the above, the estimate would only be one level below the project impact for an outcome Y, because an intervention that does not reflect the diverse set of activities executed by the project in building the capacities of the sub national governments would only be quantified as TR. According to the methodology, thoroughness is required in choosing control groups, as they are being selected from among the sub national governments who are not in the treatment group. When applying the quasi-experimental method, it is necessary to monitor the effects of variables such as the ones mentioned below: ‐ Population ‐ Average level of education of the population ‐ Proxy variables of socioeconomic indicators ‐ Demographics: age groups, gender, etc.. ‐ Availability of basic infrastructure ‐ Number of employees working in the public entity ‐ Budget size   ‐ Quality of human resources, among others ‐ Programs pertaining to other institutions and to the National Government In the case of panel models, if the data allows it, a series of each indicator for each sub national government will be constructed that will make it possible to apply the methodology. In principle, the main sources of information for the quantitative part are listed below: ‐ Data provided by officials from each one of the sub‐national governments. ‐ Barometer prepared by Peru USAID / ProDecentralización. ‐ Project baseline data gathered by the Institute of Peruvian Studies (IEP) to develop the barometer. ‐ Data from the Economic Transparency Portal of the Ministry of Economy and Finance (MEF), where both the revenues and expenditures of each regional and district 69 government from 2009 to date are provided along with the progress completion of budget execution and the investment projects of the regional governments. ‐ Data from the Human Development Index. ‐ Outlook forecasts from the national census. ‐ National Household Survey In total, there are 192 units to observe adding the district, provincial and regional governments of Ayacucho, San Martin and Ucayali. The breakdown of the analysis will be done according to the availability of data, since it is not only essential to have the data from every target variable (Y), but also to have a series of variables to control the differences in each region (the control variables). If it is not possible to accomplish this at the district level, we will begin moving up to the provincial level and so on, paying careful attention to the thoroughness of the study process. To accomplish this it will be essential to use the data from the evaluation midpoint, as this will provide us with an idea of how the project was performing as of that moment. Similarly, we will need to evaluate whether or not there was a significant changes in the evaluation policies of USAID or ProDes, as this could lead to biased estimates. It is important take into account where the intervention applied the demand based approach when comparing the units, since statistically this factor can lead to biased estimates. This approach brings selection bias in unobservables, as determined by the needs of each unit intervened. However, this has been done at the regional level, so the units within each of them have been intervened in the same manner. Methodologically we should also eliminate bias related to some other intervention with similar objectives in these areas, whether or not they are sponsored by USAID or international organizations. Finally, it will be important to review the years where interventions began in each unit. For example, intervention at the provincial level took place the first year, while the intervention for district governments has been in phases.