POLICY REFORM LESSONS LEARNED: IN DEVELOPING COUNTRIES TEN YEARS OF ENVIRONMENTALLY RELATED POLICY REFORM ACTIVITIES EXECUTIVE SUMMARY MAY 2005 This publication was produced by EPIQ II, United States Agency for International Development. EPIQ II Prime Contractors: Chemonics International, International Resources Group, PA Consulting Group (2003–2009) POLICY REFORM LESSONS LEARNED: TEN YEARS OF ENVIRONMENTALLY RELATED POLICY REFORM ACTIVITIES IN DEVELOPING COUNTRIES EXECUTIVE SUMMARY May 2005 DISCLAIMER The views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. POLICY REFORM LESSONS LEARNED CONTENTS INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7 INVOLVE THE PLAYERS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9 Lesson 1: Find a policy champion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9 Lesson 2: Involve key stakeholders throughout the policy process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10 Lesson 3: Coordinate with other donors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11 CONDUCT OPEN AND FLEXIBLE POLICY DIALOGUE . . .13 Lesson 4: Policy dialogue must adapt to specific political, economic, and cultural contexts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13 Lesson 5: Promote a flexible approach to policy dialogue . . . . . . . . . . . . . . . . .14 GET THE TIMING RIGHT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .15 Lesson 6:Approach policy change as a continuing process . . . . . . . . . . . . . . . . .15 Lesson 7:Time policy: work strategically . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16 COMMUNICATE EFFECTIVELY . . . . . . . . . . . . . . . . . . . . . . . .17 Lesson 8: Know the context in managing and communicating information . . . . .17 Lesson 9: Communication legitimizes the policy process and reinforces change .18 Lesson 10: Be sure analyses are solid and credible . . . . . . . . . . . . . . . . . . . . . . .19 DESIGN THE POLICY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21 Lesson 11: Help donor partners to support policy efforts . . . . . . . . . . . . . . . . .21 Lesson 12:Account for a variety of factors when helping design policy . . . . . . .22 Lesson 13:Analyze the incentive structures of policy instruments . . . . . . . . . . .23 Lesson 14:Anticipate barriers to policy implementation . . . . . . . . . . . . . . . . . . .24 IMPLEMENT THE POLICY . . . . . . . . . . . . . . . . . . . . . . . . . . . . .25 Lesson 15: Be alert to policy implementation realities . . . . . . . . . . . . . . . . . . . .25 Lesson 16: Develop an implementation strategy . . . . . . . . . . . . . . . . . . . . . . . . .26 Lesson 17: Identify financing sources for short- and long term implementation .27 Lesson 18: Plan to help build the capacity of stakeholders . . . . . . . . . . . . . . . . .28 Lesson 19: Delegate implementation responsibility to local authorities . . . . . . .29 EVALUATE THE POLICY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .31 Lesson 20: Evaluate environmental policy performance . . . . . . . . . . . . . . . . . . .31 Lesson 21:Tell a story – process and results . . . . . . . . . . . . . . . . . . . . . . . . . . .32 Lesson 22: Learn to track what is important . . . . . . . . . . . . . . . . . . . . . . . . . . .33 LIST OF CASES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .35 POLICY REFORM LESSONS LEARNED CONTENTS 5 POLICY REFORM LESSONS LEARNED INTRODUCTION For many years, development assistance to foster sustainable use and conservation of natural resources, and to improve environmental quality in developing nations, focused mainly on field projects, demonstrations, infrastructure investments, and capacity-building programs for various civic organizations (NGOs) and government agencies. More recently there has been growing recognition that policy reform offers greater and more enduring program results. Over the last decade, USAID, in cooperation with the EPIQ I and EPIQ II consortia, has reviewed the experience gained from creating and executing programs focused on designing, implementing, and assessing environmental and natural resource related policy reforms and strengthening the organizations engaged in policy work. This review and compilation of supporting case studies culminated in the preparation of Policy Reform Lessons Learned: Ten Years of Environmentally Related Activities in Developing Countries (PRLL)1 by the EPIQ II consortia in 2004. PRLL was also intended as a practical guide for individuals with operational responsibilities for policy reform, including their counterparts in government agencies, local as well as international NGOs, various affected parties, and donors and donor-funded policy advisors. This Executive Summary of PRLL is designed to provide readers with a quick reference book that includes a brief introduction to policy reform fundamentals and shortened presentations of the twenty-two lessons learned that appear in the full report. The CD-ROM provided in the pocket on the inside back cover of this report includes the full text of PRLL. The 66 case studies that appear in PRLL at the end of the lessons have been consolidated into a single PDF file on the CD-ROM for added convenience. Readers may view the PRLL on the EPIQ II Web site (www.epiq2.com) or obtain a paper copy from USAID. POLICY FUNDAMENTALS The challenge in assisting donor partners is to learn how policy reform might bring about actions that will then lead to improved and sustainable environmental outcomes. The focus on policy and policy reform concerns institutional change—new rules, new incentives, and new sanctions. Various public policies can impact environmental quality or natural resource use, though this may not be their primary intent. Two types of policies typically have environmental ramifications: Environmentally related policies. Designed expressly to improve environmental quality or natural resource use, they may focus on reducing pollution discharged into the environment by setting (or changing) policy—the “rules of the game” that guide the behavior of businesses and households related to pollution, waste, or natural resource use. Sector and macroeconomic policies. Designed to meet economic development or stabilization objectives, these policies can have positive or negative environmental impacts. Examples include policies to restructure the energy sector which reduce both fuel use and air pollution; fiscal reforms that reduce official debt, lower interest rates, and increase incentives for environmental investment; and trade policies and agreements that may provide inducements for better— or worse—corporate environmental performance. Coherent and effective programs to improve environmental outcomes will require careful attention to all development assistance activities, not just those directly focused on environmental policy. A necessary starting point is to agree on a common development framework. 1. PRLL encompassed and updated the material of three predecessors: Environmental Policy Dialogue: Lessons Learned (February 1998); Environmental Policy Implementation: Lessons Learned II (April 200 I); and Environmental Policy Lessons Learned (August 2002). POLICY REFORM LESSONS LEARNED INTRODUCTION 7 A COMMON DEVELOPMENT FRAMEWORK:THE POLICY PROCESS Policy effectiveness is enhanced to the extent that there is a concerted and coordinated sequence of stages that start with an assessment of the policy opportunities—this is often thought of as diagnosis, or as problem definition. Another way to consider this task is that it concerns the quest for policy reform opportunities. This second stage, concerned with design of a policy solution, entails: (1) an analysis of the opportunities to improve policy outcomes; (2) an assessment of probable impediments to policy reform; and (3) the formulation of feasible policy innovations. The third stage is implementation of those innovations developed in the design phase of the Figure 1 program. Finally, there is the essential work of monitoring and evaluation of the outcomes of the reform process. These stages can be portrayed as a wheel (see Figure 1). Notice in Figure 1 that this ongoing process is coordinated and facilitated—“managed”—by individuals from a variety of settings and circumstances who nonetheless agree to maintain a continual and constructive dialogue about the process. It is important that these individuals be sensitive to the importance of timing and timeliness and that they make a commitment to open, ongoing and frank communication. The arrows in Figure 1 capture critical flows and sequences. Appropriate and lasting policy reform will be enhanced to the extent that the general spirit of Figure 1 is central to the policy process. The lessons learned are grouped into those related to the hub of the wheel, which is a set of actions referred to collectively as “Managing the Process” and those that concern the four individual stages of the policy reform process. MANAGING THE POLICY PROCESS The focus of the first nine lessons is on a set of actions that will generally produce a constructive and effective process of coordination and facilitation (here called “managing the process”). Lessons from these actions focus on: (1) the roles of the participants in the policy process; (2) the importance of communication (dialogue) for exchanging information and reaching agreement on the policy opportunities and how best to pursue those opportunities; and (3) the sequencing and temporal dimensions of the process. A short introduction to these actions is provided below and each action is linked to the corresponding lessons in Table 1. Involve the players. The first three lessons pertain to the players who participate in the policy process—decision￾makers, civil society, implementing agencies, affected firms and industries, and pertinent donors. The lessons illustrate why effective and respectful participation is important, and describe ways of involving the players and coordinating their constructive interactions to increase the likelihood of successful policy reforms. Conduct open and flexible policy dialogue. Lessons 4 and 5 focus on the importance of open communication—the policy dialogue—and the mechanisms by which players exchange information and work out agreements on specific policy opportunities (problem definition), and how those policy opportunities might best be pursued. Open communication must occur throughout the policy process and, if this happens, then those responsible for making decisions at each step along the way will be well informed and able to offer coherent reasons for decisions and actions undertaken. Get the timing right. Lessons 6 and 7 concern the temporal aspects of the policy process. These lessons illustrate how attention to timing (and timeliness) make it possible for all participants to be more effective in moving the process forward. Communicate effectively. Lessons 8 and 9 concern the importance of effective communication. All participants must have access to the same information, they must trust that information, and they must see how that information contributes to the effectiveness of the policy reform process. Honest communication is essential in assuring 8 INTRODUCTION POLICY REFORM LESSONS LEARNED Table 1 Management Actions and Associated Lessons Learned problem are identified, analyzed, assessed, and developed into a plausible and coherent program of future work (and donor assistance). The process of M A ANAGEMENT CTIONS LESSONS* Lesson 1: Find a policy champion design also entails a review and Involve the Players Lesson 2: Involve key stakeholders throughout the assessment of: (1) possible entry points: policy process (2) appropriate and cost-effective policy Lesson 3: Coordinate with other donors instruments; and (3) implementation Lesson 4: Adapt policy dialogue to specific political, strategies. For example, if improper Conduct Open and economic, and cultural contexts disposal of used batteries is posing a Flexible Policy Dialogue Lesson 5: Promote a flexible approach to policy health risk from water pollution, the dialogue policy design process may reveal several entry points—manufacture, sale, use, Lesson 6: Approach policy change as a continuing or disposal. The policy instruments Get the Timing Right process might include taxes to discourage use Lesson 7: Time policy:Work Strategically of particular offending types of batteries, a system of deposits and Lesson 8: Know the context in managing and communicating information refunds, or separation of batteries at Communicate Effectively landfills. Once the list of entry points Lesson 9: Communication legitimizes the policy and policy instruments has been process and reinforces policy change formulated, additional analysis and information might be developed and accountability, transparency, and legitimacy of the process presented to decision-makers for further refinement of the and the resulting policy reforms. best approach to be pursued. Implement the policy. Policy implementation entails STAGES OF THE POLICY PROCESS introduction of new legislation, possible restructuring and There are four stages in the policy process, from which reform of specific government organizations, pursuit of thirteen lessons can be derived. The four stages are judicial decrees if existing institutional arrangements (laws comprised of several steps. Table 2 summarizes the actions and administrative rules) are being ignored, or training of and steps of the policy process and the related lessons. staff to make them more effective in carrying out existing responsibilities. Diagnose the problem. During problem diagnosis, activities are undertaken to identify those policy opportunities that Evaluate the policy. Evaluation entails a careful review and emerge from the identification of particular environmental analysis of data and pertinent information concerning the problems. It is here that attention is focused on the plausible performance and implementation of the policy. Although causes of—and reasons for—particular policy relevant evaluation is often paired with monitoring (referred to as problems. It is here that impediments to policy reform will “M&E”) the term “monitoring” is more appropriately used be identified. If recent policy reforms are in place that have to describe the type of analysis carried out as a routine nonetheless failed to solve particular policy relevant implementation activity. problems, then an evaluation of those existing policies will blend into a process of re-defining the problem and starting anew to seek solutions to this persistent problem. Design the policy. Policy design (formulation) is a process in which feasible options for solving a particular policy relevant POLICY REFORM LESSONS LEARNED INTRODUCTION 9 Table 2 Actions and Steps in the Policy Process H ? STEPS LESSONS* z Identify Issues: z What z Identify policy options: z Assess policy choices: z Select the best policy: choices and selection of the policy when helping design policy policy instruments implementation D DESIGN OW DO YOU DO IT What are the problems to be addressed by new policies? Identify barriers to addressing them: are the constraints and other factors that might impede new policies? What are the appropriate intervention points, tools, and approaches? Formulate pros and cons, analyze costs and benefits, and elaborate cultural preferences Establish the process, venues, and participants for the vetting of policy Lesson 11: Help donor partners to support policy efforts Lesson 12: Account for a variety of factors Lesson 13: Analyze the incentive structures of Lesson 14: Anticipate barriers to policy IAGNOSE Lesson 10: Problem diagnosis depends on solid analytics and a credible analytical process z Legalize the selected policy (legislation, decrees, and regulations): What legislative decrees or regulations need to be enacted? Lesson 15: Be alert to policy implementation z Develop an implementation strategy: Clarify, realities assign, and formalize institutional roles, Lesson 16: Develop an implementation strategy responsibilities, and relationships z Strengthen institutional capacity: Identify and Lesson 17: Identify financing sources for short￾IMPLEMENT recruit staff, provide training and and long-term implementation awareness/education to staff and stakeholders Lesson 18: Plan to help build the capacity of z Mobilize resources for management: What stakeholders are the staff and resource costs; what are the financing sources? Lesson 19: Delegate implementation z Implement policy and monitor for responsibility to local authorities effectiveness: Set benchmarks, establish indicators, and begin implementation EVALUATE z Frame the Evaluation: Establish performance criteria, select methods and data requirements, and determine when evaluations will be conducted z Evaluate implementation performance and capacity: Use selected evaluation methods to conduct evaluations z Evaluate environmental impacts: Determine the environmental benefits achieved by the policy Lesson 20: Evaluate policy performance Lesson 21: Tell a story—Process and results Lesson 22: Learn to track what is important *Many lessons cut across and are designed to encompass the steps in each stage. 10 INTRODUCTION POLICY REFORM LESSONS LEARNED INVOLVE THE PLAYERS LESSON 1: FIND A POLICY CHAMPION WHAT HAS BEEN LEARNED Policy reform requires effective leadership at each stage of the process. This policy "champion" (or champions) brings political insight, charisma, and good sense to the difficult task of institutional change. Policy champions exist in national or local government agencies, NGOs, or various interest groups. It may happen that the specific policy champion is less important than the particular position that individual occupies in an agency or NGO. Problems arise when environmental reforms are controversial and result in significant economic tradeoffs or large groups of winners and losers. In these circumstances, it may be futile to expect a policy champion to overcome serious opposition. This reminds us that “good” policies and reforms must be judged not merely by their technical and economic merits, but also by their political feasibility. If Herculean efforts are necessary to get new policies introduced, such policies will not endure, and they will thus be ineffective. KEY UNDERLYING ISSUES Leadership in the policy process. At every stage of the process, champions contribute familiarity with the issues and solutions, have the personal and political skills to move the policy reform agenda, and perhaps command respect among interest groups and government officials. These champions typically will come from the official ranks of government; however, it is not unusual for local NGOs to champion policy reform at various stages in the process. Identifying champions. It may happen that policy leaders are the product of prior programs intended to strengthen technical skills, general knowledge, and the analytical foundation needed to improve the policy credentials of an agency. Placement of advisors, and the development of policy analyses, will be ineffective forms of assistance unless there is a policy champion to utilize these contributions. Assessing commitment. A first step in determining whether there will be adequate support for a particular stage of the policy process is to assess the government’s motivation for leading or participating in the process and discussions. A government ministry or agency may be participating because of political pressure, fulfillment of campaign promises, or because an economic assistance package offered by a donor is contingent on the country adopting certain policy reforms. Elevating the process to include high-level policymakers. Interested parties that are effective champions at an early stage of the policy process may be unable to remove key obstacles to reform once later stages of the policy process— design or implementation—are reached. This is common when environmental and economic policies are in sharp conflict, or when new policies engender significant budgetary resources. At times, it may be necessary to involve decision-makers at a higher level of government—for example, in the cabinet rather than the environmental ministry—to ensure that a broader perspective of benefits and costs are considered. PROGRAMMATIC IMPLICATIONS 1. Conduct an analysis of the degree and extent of political commitment to policy reform measures, including identifying factors in the lack of commitment and the feasible options for increasing commitment. 2. Engage higher level policymakers when the support of an implementing agency is not adequate to promote the policy. The danger here, however, is that once the pressure is gone, implementation may suffer. 3. Monitor progress of the reform process, including assessing the effectiveness and commitment of champions, participating agencies, and organizations. 4. Help create a pool of potential leaders and policy champions by building the capacity of agency staff, policy analysts, and local academic or research institutes. 5. Engage key players at the level closest to where the changes will be felt most, as well as at higher decision￾making levels, so as to bridge potentially conflicting interests early in the policy process. POLICY REFORM LESSONS LEARNED INVOLVE THE PLAYERS 11 LESSON 2: INVOLVE KEY STAKEHOLDERS THROUGHOUT THE POLICY PROCESS WHAT HAS BEEN LEARNED As countries adopt democratic processes, they also learn to involve key interested parties in important policy dialogue. Participatory processes provide opportunities and mechanisms for discussing, formulating and debating policy reform. Reforms emerging from this process may be perceived as more legitimate by all interested parties, thus enhancing the prospects for successful implementation. Consensus building at each stage always involves individuals and groups with different motivations, visions, and expectations about policy reform. KEY UNDERLYING ISSUES Consensus building in the policy process. Effective policy reform formalizes an emerging political consensus to modify institutions (rules) that now inform and guide the choices of individuals, households, informal groups, and firms. To build an understanding of incentives, constraints, and technical relationships and dependencies, all interested parties and decision makers should review and endorse analyses of current policies and their problems. These individuals must then participate in the determination of feasible policy options, appropriate time frames, and implementation approaches. Also, effective policy reform depends on clarity regarding compliance mechanisms - markets, regulatory systems, informal social enforcement, and judicial processes. Participatory decision-making. As participatory decision making evolves and strengthens, there will be better opportunities to alter and refine policy initiatives. In contrast, interested parties are excluded from the policy process with top-down approaches to policy reform. In these settings, reform is likely to fail. While early success may give the impression of effectiveness in adopting new institutions (laws and rules), ultimate success is invariably reduced because of a lack of participation. And even if one governmental regime manages to implement successful policy reforms, if resentment and opposition linger, a new government may face abrupt policy reversals. Accounting for various interests. The policy process is more effective if key participants represent a range of interests: z Government. If local officials are required to play a pivotal role in implementing policies, their early involvement in the reform process can help to identify implementation options, constraints, and resource needs. z Private sector. The private sector, through trade organizations and chambers of commerce, often plays a key role in policy dialogue. These private entities may be able to influence governments to adopt flexible and cost￾saving environmental policies, rather than relying solely on command-and-control approaches. Also, elected representatives often join with industries to support policies that have economic and environmental benefits. Many environmental problems arise from self-interested actions that shift costs to others: private-sector firms will not necessarily find common ground to undertake reform. z NGOs. These groups can represent the interests of a wide range of different "publics," drawing domestic and international attention to issues, communicating new policies to their constituencies, and serving as watchdogs once policies are adopted and implemented. Accommodating special interests. While consultation with all key parties is integral to successful policy dialogue, unanimous support is not required for all reform; some accommodations can be made for some affected groups if this will help to get reforms moving in the right direction. PROGRAMMATIC IMPLICATIONS 1. Ensure that key parties are present in the policy process to provide support to those organizations with limited capacity to participate in policy reform. 2. Recognize the trade-off between the number of participants and prospects for reaching agreement on the most contentious elements of policy reform. 3. Nurture a broad-based constituency within government, various NGOs, the private sector, and other organizations to ensure exchange of ideas, especially where there are crosscutting issues, as well as positive and negative impacts, to consider. 12 INVOLVE THE PLAYERS POLICY REFORM LESSONS LEARNED LESSON 3: COORDINATE WITH OTHER DONORS WHAT HAS BEEN LEARNED Special problems arise when several donors are working in a particular country to solve related policy problems. These parallel activities are often based on differing assumptions, goals, and approaches that emerge from bilateral dialogue. Multilateral policy dialogue requires greater donor coordination than do other types of assistance activities. Policy reform initiatives will be more successful if they send consistent signals concerning the need for modified choices and actions with respect to environmental outcomes. KEY UNDERLYING ISSUES National strategies versus donor strategies. Because effective local leadership is essential for successful policy reform, donors must unite around a single coherent national environmental strategy. This is preferred to a situation in which line ministries participate in specific but isolated donor agendas. Unfortunately, the policy priorities and organizational imperatives of the various donor agencies often make this integrated approach difficult. A donor’s willingness to fund particular policy reforms may be tied to the adoption of specific institutional change, and this tends to inhibit openness and it pre-empts national or local leadership in the reform process. More seriously, when donors have different visions of the appropriate reform goals and strategies, resulting pressures on host-country policymakers are generally counterproductive. Cross-sector consistency. The need for donor coordination is further heightened by the importance of policy consistency across sectors. For reasons of efficiency and national priorities, donors may allocate priority sectors among themselves, with one donor taking the lead in health and another in the environment. However, policy measures across all sectors must be linked by the underlying principles on which the policies are based. For example, market-based approaches for private enterprise development may be undercut if the government simultaneously resorts to price controls or rationing in the energy or water sectors. If these instances arise, great care will be required to explain and justify the reasons for conflicting approaches to policy reform. Proactive donor coordination. Policy reform (institutional change) in any sector will often coincide with broader institutional changes across sectors—and will likely involve multiple donors. In such circumstances, donor coordination creates opportunities for more deep-seated and lasting reforms. Often, without proactive donor coordination, no single donor can engage the government in a serious dialogue about the full set of institutional changes needed to bring about coherent reform. Breaking bad habits. Both donors and host governments have engrained habits that reduce prospects for enhanced donor coordination. Paramount among these bad habits is the project-driven approach to solving environmental problems. This approach can undermine the evolution of collaborative relationships among donors. In fairness to donors, many governments prefer to deal with each donor separately. Nevertheless, a number of important and promising innovations in the last five to ten years have increased the prospects for effective donor coordination as part of the policy reform process: z The structural adjustment process—driven by fiscal and economic crises in host countries—places a greater premium on program and project consistency; z Tight donor budgets have forced many donors to tie individual program components to an integrated package jointly funded by multiple organizations; z Increased domestic pressure for tangible results attributable to individual donors and donor programs forces explicit consideration of “who is doing what”; and z Increased emphasis on policy change—of getting the basics in order-has improved the nature and extent of the policy dialogue among donors and host governments. PROGRAMMATIC IMPLICATIONS 1. Employ informal and formal mechanisms that facilitate the coordination among donor programs, and the exchange of information, to ensure that assistance resources are used effectively and strategically with minimal duplication of effort. POLICY REFORM LESSONS LEARNED INVOLVE THE PLAYERS 13 POLICY REFORM LESSONS LEARNED CONDUCT OPEN AND FLEXIBLE POLICY DIALOGUE LESSON 4: POLICY DIALOGUE MUST ADAPT TO SPECIFIC POLITICAL, ECONOMIC,AND CULTURAL CONTEXTS WHAT HAS BEEN LEARNED An open and flexible policy dialogue is essential for exchanging ideas, information, and alternative visions at each stage of the policy reform process. To be effective, dialogue must take place in a wide variety of settings, and it must include a diversity of parties. Policymakers must be encouraged to engage these diverse interests at all steps in the process. Effective donor involvement in this process requires donor familiarity with the quite specific political, economic, and cultural contexts within which policy reform will occur. Such knowledge can be gained by forging respectful partnerships and alliances with key members of local organizations and other independent experts. By strengthening these partnerships in the mechanics and processes of policy dialogue, by sharing and learning about appropriate analytical (diagnostic) methods, and by informed interaction in evaluating policy options, donors can play a more effective role in this process. KEY UNDERLYING ISSUES Understanding local interests. Diverse and broad-based factors—government goals and objectives, private-sector interests, specific political agendas, and grassroots concerns—drive the political process in every country. Given this background, donors must explore opportunities to collaborate with key individuals or small groups to enhance the policy process. The turnover in political leadership—and thus centers of influence—makes it prudent to explore diverse constituencies for reform. Adapting dialogue to changes in context. Changes in the political, economic, or cultural contexts can lead to important changes in the nature and scope of the policy dialogue. Changes in economic factors such as market prices, new uses of natural resource products, or changes in inflation or exchange rates alter the value of land and related natural resources and, in the process, also change the level of interest in the process of policy reform. All participants in policy dialogue, from donors to host counterparts must understand how these changes affect the climate for policy reform. Informal information exchange. In addition to customary channels for accessing government officials, other avenues of dialogue such as round-table discussions with governmental officials can build consensus on various elements of legislative proposals before final drafts emerge. Exchange programs among countries have had great success allowing policymakers to view a proposed policy in action. In recent years, several forestry exchange programs involving India, China, Nepal, and Kenya, among others, have resulted in increased policy dialogue and publications that had important influence on policymakers. Using local settings. Cooperation with local universities and policy analysis institutes that advise key government decision makers can benefit the process of policy reform. This interaction often takes place through informal opportunities for policy dialogue in the context of training courses and other settings. An academic setting can provide a less intimidating and more open exchange of ideas. Innovation in community involvement. Local communities and NGOs can contribute important insights and leadership to policy dialogue. A variety of communication techniques can be used to overcome literacy barriers and raise awareness or to bring together interest groups who are often in conflict over the direction of policy reform. Nurturing local expertise. Donors often assist in strengthening local agencies and institutes, a challenge that can entail years of support, training, technical assistance, and financing. Once established, these local entities can be highly effective and credible agents of policy change. For example, they can carry out studies that bear on policy issues, and if they have done sound work local policymakers and key interest groups will have an easier time extending trust and credibility. PROGRAMMATIC IMPLICATIONS 1. Employ participation panels and focus groups to reflect the views of a diverse set of interests in policy dialogue. 2. Forge partnerships with local universities and research organizations, senior advisors, NGOs, and local experts to articulate the various local contexts and help local organizations to build their capacity. POLICY REFORM LESSONS LEARNED CONDUCT OPEN AND FLEXIBLE POLICY DIALOGUE 15 LESSON 5: PROMOTE A FLEXIBLE APPROACH TO POLICY DIALOGUE WHAT HAS BEEN LEARNED The policy process can too easily become linear and somewhat inflexible. The policy reform process is vulnerable to stalemate and conflict if participants are unable to reach agreement on specific issues. Most often this occurs at the design stage, but it may also occur at other stages throughout the policy process. For example, it is difficult, perhaps impossible, to initiate creative policy formulation in the absence of a shared and consistent view of the precise nature and extent of problems to be addressed. Policy dialogue can advance the policy reform process only if the venues, processes, and mechanisms employed in that dialogue are flexible. KEY UNDERLYING ISSUES The nature of dialogue. Open and candid dialogue is the key to successful policy reform. The plausible components of this dialogue can include formal hearings, facilitated negotiations, informal meetings, observational study tours, workshops, conferences, and, more recently, the use of electronic forms of dialogue (list serves, web pages, chat rooms, etc.). Dialogue may involve multiple simultaneous activities. For example, while general issues might be discussed and debated in large meetings with many interests represented, policymakers may need to resolve the most controversial issues in one-on-one meetings with key participants—including other government agencies. Selecting the form of dialogue. A key challenge is selecting the best form of dialogue for the particular issues under discussion. Different issues and mooted solutions suggest specific procedural guidelines and venues for effective dialogue. Issues to consider are the type of venue, who should organize the dialogue activity, whether there is a need for professional facilitation, and who should participate. The answers to these questions will depend on anticipated obstacles and opportunities in the process of policy reform. Involving neutral parties. In some cases, the success of policy dialogue depends on who organizes or facilitates the activities. A neutral party such as a university institute, an NGO, or even a donor may be better positioned to host meetings or workshops that bring together all those with an interest in particular environmental problems and their plausible solution. If the goal of the dialogue is to reach agreement on specific provisions of policy reforms, it may be beneficial to involve an independent facilitator or arbiter to convene such activities. Drawing on international experience. In many instances, proposed policy reforms involve approaches or instruments that have not been previously considered or adopted in the host country, but for which there is considerable international experience. Study tours to engage host-country policymakers and policy dialogue may provide an appropriate forum for sharing international experience and anticipating problematic issues in the immediate context. Preparing for dialogue. Policy dialogue can be used to introduce new instruments or analytical and monitoring tools for evaluating policies and policy impacts. To improve the effectiveness of dialogue in such cases, preparatory activities may be necessary to adapt policy instruments to local conditions, or to provide an analytical framework that can be used to assess proposed policies. Such preparatory work is often necessary to focus discussions and to anticipate potentially controversial issues such as the probable impacts of particular reforms. PROGRAMMATIC IMPLICATIONS 1. Maintain flexibility in the mode, pace, and level of dialogue to ensure that local circumstances and lessons are taken into account. 2. Clearly articulate the goals of policy dialogue and the specific problems or obstacles to be addressed and resolved. 3. Ensure relevance and effectiveness by remaining flexible in both process and in content of technical assistance programs. 16 CONDUCT OPEN AND FLEXIBLE POLICY DIALOGUE POLICY REFORM LESSONS LEARNED GET THE TIMING RIGHT LESSON 6: APPROACH POLICY CHANGE AS A CONTINUING PROCESS WHAT HAS BEEN LEARNED Policy reform is an evolving, iterative, multistage process that requires patience, mutual respect, and enduring commitment. When one difficulty or problem has been solved, another will likely surface. Quick and easy progress on one or several fronts does not ensure similar success on other fronts. Nor can we assume that early success ensures durability and long-run success in environmental outcomes. Similarly, the lack of immediate institutional change (reform) does not necessarily imply failure. The policy process is one of incremental changes in ideas, in visions, in goals, and in objectives of diverse and widely scattered individual interests. To support and facilitate such a process, the government and donors must make a long-term commitment to work with and sustain the entities that must participate in policy formulation and implementation. Working with the host-country government, donors must develop comprehensive and integrated plans and programs employing a range of targeted delivery mechanisms to anticipate and address barriers to policy reform. Periodic assessment of progress is essential to target resources and make adjustments in the assistance program. KEY UNDERLYING ISSUES The pace of reform. In developed and developing countries alike, the policy reform process can take years. The complexity and interdependency of biologic, geologic, and atmospheric systems, combined with technological, economic and political issues mean that policy reform cannot be seen as some quick fix. Similarly, the identification and nurturing of the appropriate implementing entities can take time. Chronic problems in these implementing entities can include frequent restructuring; budget shortfalls; staff turnover; and lack of resources for training, equipment, and the staff necessary to formulate, implement and enforce new institutional arrangements. Long-Term Engagement. Policy analysis and dialogue must be sustained throughout the policy process. Often, issues emerge at one stage that were not foreseen at the previous stage. One Step at a Time. When working in developing and transition economies, USAID must carefully evaluate political, economic and social circumstances as a precursor to recommending policy reform. Enduring political and cultural traditions naturally influence government receptiveness to new policy proposals. Outside advisors, although prepared and well-intended, cannot always assess all of these circumstances. The “jump-right-in” approach has a track record of failure, whereas a step-by-step approach is often greeted with greater receptivity. Working simultaneously at different stages of the process. Policy analysis and dialogue must be sustained throughout the process. Issues often emerge at one stage that were not foreseen at the previous stage of the process, while in other cases it may be necessary to return to a previous stage to address a critical deficiency that had not been foreseen. PROGRAMMATIC IMPLICATIONS 1. Allow enough time for key constituencies to participate in the policy dialogue, to evaluate options, and to become fully engaged and invested in the policy process. 2. Recognize that the policy process involves a step-by-step approach to institutional change. At the same time, institutional reform is a dynamic and iterative process involving continuous assessment, adjustment of policies, and refinement of implementation approaches that requires some flexibility of approach. 3. Build the capacity for sustained policy change into technical assistance programs to increase their effectiveness, ensure their sustainability, and elevate the policy analysis and dialogue through increased credibility. 4. Look for opportunities to reach consensus fairly quickly on some policy reforms, because small victories can provide the foundation for building consensus on more controversial reforms later. POLICY REFORM LESSONS LEARNED GET THE TIMING RIGHT 17 LESSON 7: TIME POLICY:WORK STRATEGICALLY WHAT HAS BEEN LEARNED Effective policy reform is demand-driven, and the most effective reforms tend to emerge in response to host-country interests, needs, and schedules. Natural or human-caused events can create an immediate interest in institutional change. Such windows of opportunity can arise quickly and drive the policy reform process in promising directions. In the absence of such events, it may be difficult to engage either policymakers or various interests in serious discussions of policy change. On the other hand, if donors or their assistance partners are engaged with host-country leaders, it will be easier to anticipate possible opportunities and pitfalls of the reform process. KEY UNDERLYING ISSUES Unforeseen events. Targets (or windows) of opportunity dramatically increase the interest of policymakers and interested parties in policy reform and can substantially enhance the reform’s success. Examples of events that may create these opportunities include media exposure of a high￾profile environmental problem, a man-made calamity, or a natural disaster. Government changes. Changes of government or senior personnel changes within a government—local, regional, or national—frequently provide new opportunities for launching the policy dialogue. These changes often correlate with a shift in relationships among interest groups that can remove or greatly modify previous barriers to policy change. Early involvement in the process. In many cases the need to develop a consensus on policy reform can be anticipated early in the process. Donors can take advantage of targets of opportunity when they have already established relationships of trust with their counterparts even if significant and observable results in policy development and reform have not yet emerged. Being “at the right place at the right time” often results from an effort in building relationships over time and bear fruit most visibly at times of crisis or change. Linking environmental and market reforms. Other windows of opportunity can result from a confluence of changes. In one instance (Central Asia), market reforms and efforts by a municipality to control industrial pollution created an opportunity to introduce a pilot emissions-trading program. Challenges of maintaining flexibility. Workloads and resource requirements can increase dramatically when targets of opportunity lie outside the parameters of an existing work plan or when they challenge previously planned activities. This can make rapid redeployment and reallocation of USAID resources necessary, placing a premium on swift and flexible administrative actions that facilitate effective response. Administrative challenges may also require agreement from counterparts to disengage from activities that suddenly have a lower priority. PROGRAMMATIC IMPLICATIONS 1. Track events in the political, social, economic, and environmental arenas to enhance quick response to targets of opportunity for policy reform. 2. Plan to take advantage of certain foreseeable events, such as elections or cyclical events (droughts and floods), that constitute targets of opportunity for promoting change. 3. Make long-term commitments to assist and strengthen the technical, managerial, and governance skills of local partners; establish good working relationships; and develop a record of responsive support. This will enhance chances to take advantage of targets of opportunity. Note, however, that this also requires flexibility to deviate from accepted work plans and to reallocate resources on short notice. 4. Allow sufficient time in policy-assistance efforts for key constituencies to become involved, to evaluate options, and to become participants who feel they are being brought into the dialogue—and that their concerns are being considered. 18 GET THE TIMING RIGHT POLICY REFORM LESSONS LEARNED COMMUNICATE EFFECTIVELY LESSON 8: KNOW THE CONTEXT IN MANAGING AND COMMUNICATING INFORMATION WHAT HAS BEEN LEARNED Donors and their partners need to understand the political and social context in managing and communicating information central to the policy reform process. The data and information to be collected, processed, and communicated should not only contribute to—and inform—the policy process, but should also reflect the needs expressed by various participants in the process. In designing information systems, it is critical to ask the right questions about the scope of information-gathering efforts, information￾disseminating methods, and resource and time requirements. Also, criteria should be applied to ensure that information gathered and offered for use meets the goals and objectives elaborated by the designers and implementing entities. KEY UNDERLYING ISSUES Information needs in the policy process. In many countries, government agencies are legally required to provide certain non-proprietary information to the public and to respond to requests for information. Implementing agencies, particularly, collect, process, and informally communicate a great deal of information. Using this information effectively will facilitate the policy-reform process. Dimensions of context. The context for managing information involves a range of interrelated factors. Foremost are the specific elements of the policy process and details of their development: What does the policy call for in terms of changes in the behavior of industry, groups, and individuals; who are the probable winners and losers; how did the policy come into existence—did it result from government-initiated analyses or were changes tied to pressures from donors and others? While the policy process provides the primary context for the collection of information, information systems must also respond to a variety of interrelated and changeable non-environmental factors such as economic and financial conditions, the political situation, and social and cultural settings. The value of information. Information collection and analysis is often under funded and under appreciated. In part, this results from the difficulty of valuing information for its benefits to the environment or in determining how its POLICY REFORM LESSONS LEARNED availability influences the pace and outcome of the policy process. This arises because inadequate attention had been paid to the essential role of data and information in the reform process. Donors' needs for information will often relate to accountability; they must demonstrate what has been accomplished as a result of their assistance programs, and information in the form of performance indicators will often be required. Designing information systems. The design of the information system must address a number of questions: z What types of information are needed? z How will the information be used in the policy process? z Who needs the information and how will it be transferred to users? z What are the constraints on funding, staffing, and access to data to undertake information collection and analysis? z How will information be collected? z Where and how often will it be collected? z How will its quality be controlled and assured? z How it will be archived? PROGRAMMATIC IMPLICATIONS 1. Invest appropriate resources in designing information systems to analyze the context and to establish credible baselines for the policy and the expectations of the users of the results. 2. Establish a regular process to evaluate the need for changes in information management in response to policy shifts and exogenous factors. For information management, establish clear goals and objectives; identify, analyze and select indicators; and prepare a detailed plan for collecting, evaluating, and disseminating information. (USAID's Center for Development Information and Evaluation, CDIE, provides a number of useful tips for monitoring and evaluation of assistance programs (see www.dec.org/usaid_eval/). COMMUNICATE EFFECTIVELY 19 LESSON 9: COMMUNICATION LEGITIMIZES THE POLICY PROCESS AND REINFORCES CHANGE WHAT HAS BEEN LEARNED A system of clear and open communication using a variety of instruments will simultaneously promote transparency and improve accountability of policymakers and implementing agencies. Effective communications may also help improve environmental performance of facilities. KEY UNDERLYING ISSUES Scope of communications. Communications within the government, among policymakers and implementing agencies, and between the government and various interest groups is essential at each stage in developing and implementing policy reforms. In addition, information on the nature of reforms and progress in implementing them will have value in legitimizing the policy process to the general public. A variety of formal and informal mechanisms are used to communicate information. Some are mandated by statute and regulation, such as “state of the environment” reports and announcements of rulemaking, public hearings, and public disclosure. Accountability. Communication is important in demonstrating policymakers’ accountability for the policies they develop and for the manner in which they are implemented. Where policy reforms are controversial, communication of the results or impacts to those favorably or adversely impacted by the reforms can help build broader support for the reforms, foster dialogue, and lay the foundation for subsequent policy reforms. Transparency. Transparency refers to the openness and clarity of the process of policy reform. Transparency and accountability are mutually reinforcing: the more transparent the flow of information, the easier the task of government in accounting for and enhancing perceived legitimacy of the reforms. However, for individual interested parties (and the general public) to use this information effectively, there must be provisions for access to the policy process including involvement in dialogue, public hearings, and opportunities to provide written and oral comments on policies throughout the process. One danger of a commitment to transparency is that some participants may wish to provide information that only supports their positions. To overcome this, tendency, the general public and the various interested parties must be firmly grounded in understanding policies. NGOs and trade associations can play a role in monitoring government on behalf of these groups, conducting independent assessments of policy performance, and by educating their members so they can effectively confront their officials, raise these issues, and hold government accountable. Improving environmental performance through public awareness and education. The communication of information can be an important component of policy implementation. If the public understands the implications of information that is provided, these shared perceptions can pressure industries within the various sectors to redouble their efforts to improve their public image both locally and abroad. Public disclosure laws, such as toxic release inventories, can pressure factories to improve their environmental performance and, in some cases, avoid international bans and embargoes. PROGRAMMATIC IMPLICATIONS 1. Encourage policy partners to support communication of information with resource and staff and to ensure the public’s access to the policy process, extolling the potential benefits to the government of legitimizing the policy process, and garnering support for reforms. 2. Support capacity-building efforts among NGOs, local research and training institutes, and relevant interest groups to ensure these groups can analyze information effectively and educate the public. 3. Make information available through public disclosure documents and lists, environmental quality indexes, and environmental labeling campaigns; good information can help increase policy compliance in the absence of 20 COMMUNICATE EFFECTIVELY POLICY REFORM LESSONS LEARNED DIAGNOSE THE PROBLEM LESSON 10: BE SURE ANALYSES ARE SOLID AND CREDIBLE WHAT HAS BEEN LEARNED Accurate and comprehensive problem diagnosis is essential in developing a shared understanding that existing institutional arrangements are the plausible cause of unacceptable environmental outcomes. From this understanding, it is then possible to begin initial efforts at developing policy options to rectify the identified problems. Careful analysis of problems and their possible solutions are central aspects of establishing essential credibility for policy reform. KEY UNDERLYING ISSUES Having strong analytics. While good analysis does not always lead to good policy, constructive policy is a rarity in the absence of reliable analysis about the reasons for particular problems, assessment of feasible solutions to those problems, and assessment of the most plausible reform strategies. Drawing on local expertise. Involving local experts in diagnosing problems can help donors identify and gather information that is not easily collected by foreigners, serves as an important capacity-building technique, and offers important long-term results. Further, it can help persuade local interests and policymakers of the usefulness of the analysis. Conventional wisdom. Local decision makers and interested parties may have previously reached conclusions based on anecdotal or incomplete information. In cases where highly visible problems create pressures to find quick solutions, interested parties and policymakers might need convincing that analysis is valid and necessary. Time and resource commitments. Although “quick and dirty” studies are easier, they can lead to misleading conclusions when the issues are complex and dynamic. Time and money need to be committed to discover and document the central issues and underlying causes. Gaining the support of in-country counterparts for this endeavor, though often challenging, is the key to successful policy reform. Drawing on experiences in other regions and countries. Where experience with a certain type of policy is limited or POLICY REFORM LESSONS LEARNED does not exist, important insights may be gained by drawing on analyses and experiences in other regions or countries. The introduction of new policy instruments or the creation of new institutional mechanisms, such as environmental funds, will often benefit from analyses from other countries. Multidisciplinary perspectives. Frequently, multidisciplinary analysis is required—from physical and biological sciences, engineering, economics, institutional analysis, or other social sciences—to identify and include all of the relevant facets of environmental problems and the likely impacts of possible approaches and solutions and their applicability to the local context. Information dissemination. Effectively communicating analytical results is as important as the analysis itself. If results are poorly communicated and not understood by other participants in the policy dialogue, the probability of acceptance and desired impact are greatly diminished. Where evaluation ends and problem diagnosis begins. In many cases, evaluation and problem diagnosis are part of the same analytical effort. This may occur when there is widespread agreement that current policies are performing poorly, even if this agreement is based on anecdotal or qualitative evidence. The analysis then might include a rigorous quantitative evaluation to: (1) reinforce or affirm the conventional wisdom related to current policies; (2) be a catalyst for serious discussions with policymakers and interested parties on policy reforms and the problems or obstacles that have to be overcome; and (3) identify critical information gaps. PROGRAMMATIC IMPLICATIONS 1. Include solid analyses in the policy dialogue process with the support of many interested parties, even when there is intense pressure for rapid solutions, or where a shared consensus on causes and impacts already exists. 2. Work collaboratively with local experts to test new theories and approaches, e.g., market instruments, and frequently brief policymakers and interested individuals. This also is important for capacity building. DIAGNOSE THE PROBLEM 21 POLICY REFORM LESSONS LEARNED DESIGN THE POLICY LESSON 11: HELP DONOR PARTNERS TO SUPPORT POLICY EFFORTS WHAT HAS BEEN LEARNED USAID’s assistance partners are often protective of their role in policymaking. While they welcome project assistance, they may resist activities aimed at shaping or implementing policy. Helping with policy design may be perceived as too intrusive a role for donor organizations, signifying the recipient’s weakness and diminishing the credibility of the government’s policymakers. Donors must break down barriers to policy support and seize opportunities for intervention. KEY UNDERLYING ISSUES Sovereignty. As steward of the public’s environmental and natural resources trust, governments are accountable for decisions that contribute to the quality and value of these resources. By their nature, policies (institutions or rules) establish rights of access to resources, condition resource use, and determine the benefits that may be obtained by holders of these rights. USAID’s partners are often concerned about compromising their sovereignty if they permit foreign parties to put new institutional arrangements in place, even when a compelling case can be made that the country will benefit from the institutional changes. Policymakers must establish their credibility both within the government and among parties affected by their policies. This credibility is easily undermined if donors are perceived to be “leading” rather than “supporting” policy reform and implementation. Resistance to change. Donor organizations often encounter bureaucratic resistance to rapid change in institutional arrangements. Protocols for revising legislation and regulations must be respected, even if the process is slow and uncertain. At times, government officials may want to slow down reforms because of opposition within the government or within the agency that will be responsible for implementation. Projects versus programs. “Brick and mortar” projects yield tangible, visible, and immediate benefits and may be preferred by donors’ partners to assistance in reforming policies. In these circumstances, governments can readily translate visible project results into political credibility and leverage. Policy reform—institutional change—is a slower and more uncertain process that may create losers as well as winners and yield benefits that may only be realized over several years. Barriers to policy assistance. USAID can help to promote the legitimacy of its assistance in several ways. Most importantly, existing policies that appear to be the plausible reason for unwanted environmental outcomes must be carefully analyzed, and feasible reforms need to be identified and assessed. Partners will be more receptive to outside involvement if the proposed institutional changes seem to yield important benefits relative to current policies and the assistance is viewed as a partnership. Tying assistance to performance. Recognizing that the policy process may move very slowly as a result of changes in government and shifts in policy champions from one stage to the next, it may be useful to introduce incentives for reaching policy milestones. Such milestones may include completion of background reports, development of draft legislation, or achieving different levels of compliance. PROGRAMMATIC IMPLICATIONS 1. Encourage donor partners to establish their credentials and make a commitment as early as possible in the policy process. Donor partners can get involved initially in policy dialogue by supporting analyses of existing policies and by participating with other donors in workshops and roundtables, which can help identify how the donor can become more involved and supportive of the policy process. 2. Encourage donor partners to avoid “pushing” the policy process: the host government’s leadership role in setting the schedule and the agenda should be respected; donors should expect to play a supporting role. 3. Emphasize to donor partners the win-win nature of the environmental policy assistance effort. POLICY REFORM LESSONS LEARNED DESIGN THE POLICY 23 LESSON 12: ACCOUNT FOR A VARIETY OF FACTORS WHEN HELPING DESIGN POLICY WHAT HAS BEEN LEARNED Given pressing financial needs for a variety of social services, many developing countries and economies in transition will suggest that they are unable (or unwilling) to bear much economic sacrifice to enhance environmental quality. Thus, environmental goals are more easily advanced if they contribute to the attainment of other pressing policy goals or to perceptions of economic progress. For instance, improvements in water quality can benefit urban water users as well as enhance aquatic life. Even so, certain new environmental policies may entail trade-offs. Thus the creation of new institutional arrangements will require careful analysis of the economic and social costs and the development of actions that will reduce these costs as much as practicable. KEY UNDERLYING ISSUES Market failures. Feasible environmental policy options depend on a society’s broader economic, political, and governance conditions. The integrative perspective of environmental economics focuses on trade-offs between environmental quality and demands for conventional goods and services. “Market failures” result when prices reflect only private costs, ignoring damages from pollution or natural resource exploitation, and consumers neglect external environmental impacts of production, consumption, and other activities. An integrated environmental-economic perspective includes an examination of taxes, regulations, and other policy instruments that a government can apply to rectify environmental externalities (market failures). Policy failures. While externalities are often pervasive in developing countries, excessive pollution and resource depletion will also result from distortion of market forces, scarce human capital, and weak institutional underpinnings of a market economy. Policy reform can address such policy failures by focusing on increased awareness of flawed incentives in the realms of both production and consumption. Appropriate resource pricing and cost recovery. Opportunities exist to increase efficiency, promote equity, and conserve natural resources through appropriate pricing and cost recovery. For example, higher prices (or reduced subsidies) can induce water users (both consumers and irrigation farmers) to use water more judiciously. Macroeconomic linkages and incentives. Micro-level incentives must reflect the macro-level options available. For example, if a policy change aims to slow the rate of agricultural expansion into forested lands, macro-economic policies must be revised to encourage alternative income sources for farm households. Access to fertilizer and credit, market infrastructure, imported or domestic availability of agricultural machinery, and foreign exchange policies may have as much influence on deforestation rates as forestry sector policies. Spatial dimensions of environmental problems. Today’s major global and local environmental problems—global climate change, loss of biological diversity, unsustainable land use, natural resource depletion, air and water pollution, the demographic resource imbalance—all require action that cuts across national, sector, and organizational boundaries. PROGRAMMATIC IMPLICATIONS 1. The key in policy analysis is to search for opportunities for policy reform that will improve environmental quality and simultaneously contribute to economic development. 2. Carefully appraise the potential effects of environmental policy change on natural systems given the increasing pressures resulting from population growth rates in many developing countries. 3. Account for the underlying causes of environmental problems and the role of individual action when helping to prepare national environmental action plans or other higher-level strategies, policies, and institutional reforms. Policy reform requires a strong consensus that the social costs of mismanagement, or ignoring environmental problems, may be large indeed. 24 DESIGN THE POLICY POLICY REFORM LESSONS LEARNED LESSON 13: ANALYZE THE INCENTIVE STRUCTURES OF POLICY INSTRUMENTS WHAT HAS BEEN LEARNED It is important to assess the structure of existing incentives and the factors that may reduce a new policy’s effectiveness. Ideally, new institutional arrangements will result in certain and predictable directional changes in individual choice and action, such as reductions in pollution or adoption of more environmentally sound technologies. Flaws in the design of new institutional arrangements (new rules) or weak implementation will most certainly undermine the goals and objectives of the policy process. Some of these problems can be anticipated and corrected at the design stage, while others will require adjustments during implementation to overcome design flaws. KEY UNDERLYING ISSUES The nature of incentives. Policy reforms alter the incentive structure of firms and individuals by compelling behavioral changes or by establishing new prices for inputs, outputs, or pollution that induce new behaviors. For some environmental problems, altered institutions (rules) giving rise to new prices may be adequate to bring about new behaviors. In other settings, the new rules may need to compel new behaviors. Economic and market assumptions. Designers of policies must be aware of uncertain economic and market conditions that make it difficult to predict how and when firms and individuals will respond to policy changes. Conflicting policies. Trade policies, general fiscal policies, and sector policies must be considered when attempting to predict the individual responses to sanctions and incentives embedded in new environmental policies. Policy reform will need to be aware of these confounding influences. Benefits and costs. The presumption of environmental policy reform is that overall beneficial impacts will be positive, but there will always be a distribution of gains and losses from policy reform that will engender concern and perhaps opposition. Some key characteristics of this structure of benefits and costs must be recognized in formulating policy: z Most new environmental policies benefit a relatively large group for which benefits per beneficiary are small. On the other hand, the financial and other costs of new policies can fall on particular industries or facilities. z Benefits from environmental policy reform occur over long periods of time into the future, while the costs implied by such policies are often borne soon, and may continue into the immediate future until industry has a chance to respond with suitable technical change. z Benefits of environmental policy are often non-monetary (clean water, clean air, less soil erosion), while costs are monetary. z Powerful interests can be expected to exert pressure in the reform process to alter incentive structures in their favor, dilute enforcement, or derail the policy entirely. z Such interested parties who will be negatively affected by a policy are usually better organized than those likely to benefit from change because the beneficiaries of environmental reform are often the population at large, or are yet unborn members of future generations. PROGRAMMATIC IMPLICATIONS 1. Carry out analysis that identifies key assumptions linked to how interested parties will likely respond to incentives so that the impacts of policy reform can be assessed. 2. Identify potential conflicts with other policies early in the design process to determine the extent to which other policies will adversely affect the proposed reform and to determine how the proposed environmental policy affects sector policies. 3. Ensure that the new policy is clearly and formally articulated so that the impacts on all members of society can be assessed. 4. Anticipate and be ready to address the political pressures that aggrieved and powerful interests may exert on the policy process. This will require careful design of policies and implementation strategies, possibly using phased requirements. POLICY REFORM LESSONS LEARNED DESIGN THE POLICY 25 LESSON 14: ANTICIPATE BARRIERS TO POLICY IMPLEMENTATION WHAT HAS BEEN LEARNED Experience shows that the best design can fail unless implementation problems are anticipated and addressed before the final design is approved. Two keys to better design are early, preemptive analysis of potential implementation barriers and accommodations for implementers to participate in finalizing the design of policies. KEY UNDERLYING ISSUES Taxonomy of implementation barriers. Generally, three groups have a key role in policy implementation, and each has the ability to reduce the pace and effectiveness of policy reform. They are: z Governmental power structures. Once a policy has been approved, the executive, legislative, and judicial branches of government all have the potential to derail the policy. Government officials, even if they are not directly involved in implementation, can undermine policy through allocating budgets to implementing agencies, enacting legislation to complement policy reforms, or adjudicating legal challenges to policy or specific attempts to enforce the policy. In addition, courts can stall implementation by enjoining new legislation. Policy reforms that feature enforcement sanctions as a deterrent to noncompliance often depend heavily on the support of the judicial branch of government to prosecute violations and to levy sanctions commensurate with the violation. In many poor countries, officials are unwilling to arrest the poor for crimes they commit to eke out a subsistence existence, in part because judges are often lenient and may reduce or dismiss damages. z Agency reluctance. After legislative approval, responsibility for implementation shifts to government agencies or, in some cases, to NGOs that may be ill-equipped to carry out the functions of the policy reform. Problems include unclear delineation of roles and responsibilities, limited resources or capacity to carry out the new responsibilities, and lack of commitment to the new policy on the part of the implementing agency. This is particularly true in countries in which policies are decided upon by small groups of policy elites in distant capitals. z Interested parties. Beneficiary groups are often not empowered to assert their interests in policy reforms. To ensure that implementers are accountable to the public for policy reform, appropriate information must be made available to the public, and the public must be able to assess this information. In some cases, NGOs may have to play this role. Ex-ante assessment requires early involvement of donors. Ideally, potential implementation barriers are evaluated and discussed during the stage of policy dialogue. This requires the participation of implementing agencies and interested parties who are in the best position to identify potential problems and to help find solutions. In some cases, implementation barriers may be so difficult to overcome that an alternative policy, perhaps less complex or relying to a greater extent on voluntary compliance, must be considered. PROGRAMMATIC IMPLICATIONS 1. Carry out an analysis of the implementing agencies and ensure that important players in implementation are adequately engaged during diagnosis and design. 2. Determine likely policy winners and losers, recognizing that powerful interests may have supporters in sector ministries and agencies that may be able to exert indirect influence on the policy through budgeting of financial resources for the implementing agency, or by reducing political support for enforcement. 3. Strengthen linkages between central and local authorities in the implementation process as early as possible, convening workshops to present new policy to local implementers, and targeting local officials and staff for training programs that support effective implementation. 26 DESIGN THE POLICY POLICY REFORM LESSONS LEARNED IMPLEMENT THE POLICY LESSON 15: BE ALERT TO POLICY IMPLEMENTATION REALITIES WHAT HAS BEEN LEARNED A number of factors can impede the implementation of the policy as originally designed. Before implementation assistance is provided, it is useful to analyze the policy to identify barriers that may reduce its effectiveness and assess options for overcoming these barriers. This is especially important when the donor has not been involved in earlier stages of the policy reform process, or donor support is requested for implementation of an existing policy. The assessment may indicate the need for changes in the policy or changes in the implementation strategy to overcome weaknesses in the policy and aid donors in determining the likely success of the proposed assistance programs, and deploying these resources strategically. KEY UNDERLYING ISSUES To assist or assess? USAID and other donors are often asked to provide assistance with the implementation of policies. This may involve training for staff in environmental agencies or technical assistance for key interested parties. Possible impediments include weaknesses in agency capacity to implement a policy, or the inability of affected parties to respond to incentives because of their limited financial situation, or poor information. The scope of assessment will depend on the nature and timing of the request for assistance as well as the resources the donor can mobilize for the assessment. At one extreme, the scope of assessment may be quite broad and take several months to complete. Where the request for assistance focuses on a single policy, the scope of analysis may be more limited than that described here. Changing the policy. If there is agreement that a new or revised policy is flawed, the next step is to determine whether the policy can be changed. The donor and its assistance partners must examine the design process, how long it took to approve the policy, major problems that had to be overcome before consensus was reached, and the receptiveness of policymakers to changes in the original policy. If the policy has been in place for several years, changes are likely to entail a comprehensive assessment and vetting of this analysis in the stage of problem diagnosis. Again, if the donor has participated in the design process, the prospects for changing the policy prior to implementation may be greater. Overcoming bad policy with good implementation. As a rule, it is easier to address policy design flaws during the implementation stage if they relate to institutional capacity issues for implementers and affected parties rather than to poorly designed incentive structures. Even if the success of the implementation options is uncertain, the donor’s effort may enhance the donor organization’s credibility and commitment to assisting the partner, and provide an opportunity to revise the policy in the future. PROGRAMMATIC IMPLICATIONS 1. Make a careful review of the problem, the process, and the policy goals before embarking on policy implementation in order to gauge the host government’s responsiveness to making additional changes. 2. Plan to implement policy in phases, establishing intermediate goals or benchmarks so that performance can be evaluated. When skepticism is prevalent that policy change is needed, new policies can be simplified, broken into manageable components, or piloted in limited forms to assess implementation barriers. 3. Take steps to ensure USAID remains positioned to assist policy change by targeting some assistance to initial implementation tasks, helping design protocols for monitoring implementation progress, maintaining open dialogue with both policy formulators and implementers. POLICY REFORM LESSONS LEARNED IMPLEMENT THE POLICY 27 LESSON 16: DEVELOP AN IMPLEMENTATION STRATEGY WHAT HAS BEEN LEARNED An implementation strategy must be developed early in the policy reform process. If that implementation plan reveals possible barriers then specifics of the policy may be adjusted to mitigate implementation problems. Such plans serve a useful role in establishing compliance or implementation schedules and helping implementers identify key gaps in agency capabilities and funding, thus enabling donors to target their assistance resources more effectively. KEY UNDERLYING ISSUES Capitalizing on the momentum of design. The design stage leading up to approval of a policy may be characterized by intense last-minute negotiations, fine-tuning of laws or regulations, and mobilization of legislative support. Practical issues relating to implementation budgets, compliance planning, and capacity building are put aside to concentrate efforts on approval of the policy. Once the policy is approved, stewardship for the policy shifts to implementing agencies. The development of a comprehensive strategy for implementing the policy may aid in focusing attention on resource needs and agency strengthening requirements. Strategy basics. Some problems that need to be addressed in developing implementation strategies include the following: z Implementer incentives. One of the major difficulties in developing countries is providing appropriate incentives for implementing agencies and staff. In addition, understaffing may lead to overworked staff or create significant lags in key implementation activities such as permitting, inspections, and enforcement. Mechanisms such as self-monitoring and reporting by regulated firms (with appropriate sanctions for non-compliance) may reduce pressures on ministries and agencies with limited staffs and budgets. z Opportunity costs of donor projects for agency staff. Most donors are prohibited from compensating government staff for the time they spend on coordination of donor projects although these activities detract from their regular duties. In some countries, however, staff receives salary supplements when they participate in donor-financed activities such as training, workshops, and advisory or steering committees. Such incentives can lead staff to pursue these financial rewards to the exclusion of other work. In some cases, training and study tours to other countries can partially compensate for lower salaries. Ultimately, problems of staff incentives and high salary differentials can be addressed only through comprehensive civil service reforms. z Lack of transparency and accountability. The public can be an important ally of agencies in their implementation efforts provided the public has access to information and is allowed to participate in the review of environmental management actions. Bilateral assistance programs often feature democratization as a key objective, with the environment sector offering significant opportunities for changing the relationships with the public. By conducting implementation activities openly and transparently and providing public access to information on the performance of the regulated firms, implementers can increase their accountability with the public and legitimize their enforcement role with interested parties. PROGRAMMATIC IMPLICATIONS 1. Support counterparts by providing financing experts outside the agency and arranging twinning relationships with countries that have experience in developing implementation strategies. 2. Encourage counterparts to prepare implementation strategies by promoting strategies as mechanisms that help all donors identify and prioritize their assistance resources. 28 IMPLEMENT THE POLICY POLICY REFORM LESSONS LEARNED LESSON 17: IDENTIFY FINANCING SOURCES FOR SHORT- AND LONG-TERM IMPLEMENTATION WHAT HAS BEEN LEARNED Donor financing is often key during the start-up phase of policy implementation, but it may hinder the development of domestic funding if support is viewed as a permanent substitute for a strong commitment from government to support financing when the donor funding phases out. Sustained financing is not limited to expenditures by implementing agencies. Governments are not expected to shoulder the responsibility for financing in the private or municipal sector, but they can play an important role in creating new laws and rules, and adopting and enforcing policies that will solidify the commitment of affected parties. KEY UNDERLYING ISSUES Devolution of management responsibility without revenue authority. Central authorities have increasingly delegated responsibility for environmental protection and natural resource management to local governments, community￾based organizations (CBOs), or NGOs. Such delegation may be motivated by a desire to reduce demands on central budgets or recognition of the comparative advantages of local management. In many instances, however, the transfer of these responsibilities has not been accompanied by funding to cover costs or authorization to implement revenue mechanisms. As donor support is withdrawn, other sources will be inadequate for fully covering management costs. Investments versus recurring costs. Donors often provide assistance in financing or co-financing investments in environmental infrastructure or equipment related to environmental services (such as garbage trucks and trash compactors) or environmental management (such as computers and monitoring and laboratory equipment). Investment financing is often more attractive to donors than support for recurring costs since the benefits are easier to quantify and such support provides opportunities for vendors in the donor country to enter the market in the recipient country. The benefits of these investments will be diminished without a commitment to fund recurring costs such as maintenance, spare parts, fuel and electricity, staff and training. Planning for sunset. Part of the reason for unsustainable financing can be traced to the design of donor programs. Project assistance often is not contingent on financial contributions from the recipients. More recently, USAID has gained a greater appreciation for the use of counterpart incentives, graduation provisions, and diversifying assistance to help partners develop financing to sustain projects and programs. PROGRAMMATIC IMPLICATIONS 1. Review and assess the national and local budgeting process, identify barriers that limit budget increases, and determine the stability of earmarks, the nature of intergovernmental transfers, and any legal constraints on local governments' ability to raise revenue, so as to pinpoint the support most needed. 2. Frame assistance programs focused on goals that are regionally or globally important (such as global warming, and biodiversity protection) within the context of local or national development goals in order to sustain commitment to policy implementation. 3. Create conditionality for assistance programs through means such as: – Increase awareness of sustained financing issues by introducing this component to implementation strategies. – Tie assistance efforts to related and mutually reinforcing policy measures that will promote sustained financing. – Incorporate financing indicators into the evaluation process. – Sequence assistance efforts and requiring assistance partners to reach benchmarks before “graduating” to the next phase of assistance. – Collaborate with other donors and IFIs on tied (or coordinated) assistance strategies to increase counterpart incentives for sustained financing. POLICY REFORM LESSONS LEARNED IMPLEMENT THE POLICY 29 LESSON 18: PLAN TO HELP BUILD THE CAPACITY OF STAKEHOLDERS WHAT HAS BEEN LEARNED questions, review compliance action plans, and assist firms Often the major assistance priorities conveyed to donors are in accessing information and equipment vendors. activities that strengthen the management capacity of implementing agencies. While such capacity building is z Limited capacity to develop projects. A major impediment to important, the need to help a variety of affected parties to investment has been limited skills in preparing investment participate constructively in the policy process and respond projects, including assessing cost-minimizing technologies, to policies is often overlooked—even though it may make a developing financial plans, and identifying available greater contribution to sound policymaking and compliance financing sources. Several donor efforts to help investors build capacity have been organized through a network of rates. KEY UNDERLYING ISSUES The increasing complexity of policy. Policy reform often involves the introduction of management approaches or tools that have not previously been employed in the partner country. These approaches may necessitate the recruitment of more specialized and skilled staff as well as training for both existing and new staff. Stricter environmental standards translate into higher investment costs and place a premium on the design of compliance plans to minimize costs. donors and international financial institutions. In addition, USAID has worked closely with environmental funds, ministries, and investors to prepare projects, train applicants in the preparation of applications to environmental funds, and strengthen institutions. z Matching technical assistance to educational levels. Assistance programs have often been under subscribed because target groups lack the education and experience to utilize fully the assistance offered. In all cases, assistance programs that provide training to affected parties should Affected parties also may be expected to self-monitor and include both a needs assessment and a skills assessment. report air emissions and water discharges. Assistance programs that have proven effective elsewhere may need to be tailored to enhance their effectiveness in a Weaknesses in capacity. A potentially weak link in the new country or region. implementation process is the limited capacity of affected parties to respond to incentive structures associated with the PROGRAMMATIC IMPLICATIONS new policy. Issues that may account for weak capacity 1. Evaluate policies in terms of how affected parties are include limited access to information or skills to evaluate financial alternatives, and select a cost-effective strategy or likely to respond to incentives and determine if their investment. Other weaknesses in capacity include: response could be enhanced through capacity building, as a step in programming assistance resources. z Limited access to information. In many countries where Western environmental standards have been adopted and 2. Identify barriers that limit the ability of affected parties regulated firms have been required to develop compliance to use assistance as a step in programming capacity- strategies, it is difficult to obtain information on suitable building resources (for example, people with limited abatement technologies. For small businesses, acquisition education, living in remote locations or with limited of computers for Internet access and the associated access to various media forms may not be able to take training may be beyond their means. Implementing advantage of the assistance offered). agencies and donors can play a constructive role in 3. Take account of cultural differences in programming developing information bases and helping firms access this policy implementation assets. information. At the same time, implementing agencies must develop the capacity to respond to compliance 30 IMPLEMENT THE POLICY POLICY REFORM LESSONS LEARNED LESSON 19: DELEGATE IMPLEMENTATION RESPONSIBILITY TO LOCAL AUTHORITIES WHAT HAS BEEN LEARNED as a national park or watershed), where the creation of Local actors often can implement changes in the most cost- new organizations or the devolution of authority to effective and long-lasting basis. Local incentives are the NGOs is more effective. Such decisions must consider the driving force for widespread adoption of improved methods, time frame for implementation and barriers to transfer of and removing policy obstacles to sound investment management authority. systematically helps to improve incentives. Delegating implementation responsibility to those who are closer to the z Provision of adequate funding for local staff. In many problem improves effectiveness because these local parties countries, local governments have limited authority to tend to be more motivated to carry out the policy as generate revenue to cover staff salaries and related costs. designed, regularly monitor compliance, and have greater They must often depend on transfers from central budgets credibility and accountability with affected parties. to support these activities. Local entities cannot implement policies without the financial means to do so. KEY UNDERLYING ISSUES z Local capacity. Lack of local capacity is often the most Decentralization. Throughout the developing world, donors difficult problem to overcome. Monitoring and have encouraged governments to decentralize many of the enforcement activities require a level of education and functions of government. In part, this strategy has been training among staff that may be in short supply in local motivated by a political agenda to democratize government, jurisdictions. In addition, salaries are often low relative to spread power more widely, and lay the foundation for greater salaries in the private sector. Thus, recruitment and participation in the election process. A key issue is whether retention problems may make it difficult to implement national policies can be effectively implemented by central activities in local agencies. authorities. In case after case, devolving responsibilities to z Delineation of responsibilities. Local responsibilities are local authorities increases the effectiveness of policy implementation. Setting the roles for central and local implementers. often not clearly defined in legislation or regulations. Various agencies may have overlapping responsibilities, or the responsibilities are not clearly communicated to local Central agencies often contribute to the implementation officials or the central government fails to help local effort, taking a lead role in interpreting legislation by writing officials develop the needed capacity. regulations or more informal procedures. Central agencies may also be better positioned to secure financing needed at PROGRAMMATIC IMPLICATIONS the local level, resolve disputes, and monitor and evaluate the 1. Involve and support local partners and create local effectiveness of implementation. Local roles in demand for policy implementation, by working with implementation usually involve direct contact with affected local NGOs to monitor implementation of policies with parties in such activities as development and review of distinct local benefits. permits, monitoring and review of compliance activities, and enforcement. Key challenges: 2. Determine weak links in the implementation process related to local agencies and encourage national agencies z Selection of local implementers. The selection of the local implementation partner may depend on what skills and resources the local agency can apply to the task, as well as the degree to which the local authority's traditional jurisdiction relates to the area covered by the policy. Devolution to local agencies may be difficult if the focus of a policy is a cross-jurisdictional geographical area (such to address legal or organizational impediments (such as failure of the courts to prosecute violations) and help co￾finance capacity-building activities. 3. Demonstrate the benefits of local implementation through pilot programs and demonstrations coupled with public awareness and outreach. POLICY REFORM LESSONS LEARNED IMPLEMENT THE POLICY 31 POLICY REFORM LESSONS LEARNED EVALUATE THE POLICY LESSON 20: EVALUATE ENVIRONMENTAL POLICY PERFORMANCE WHAT HAS BEEN LEARNED Donors and their partners often are reluctant to undertake evaluations of policy reforms because it is difficult to unambiguously determine if the policy is working and measure its impact. The potential benefits of evaluation as a strategic stage of the policy process are not always appreciated by donors or their partners. Evaluation is not simply a document to satisfy a donor’s reporting requirements but can foster accountability and transparency while providing continuous information on the implementation of the policy. Environmental policy can be evaluated if planned in a careful and timely manner, using a well-conceived participatory process. KEY UNDERLYING ISSUES Evaluation: a stage of the policy process. Evaluation is often treated as a discrete, one-time analytical exercise rather than a multi-faceted stage of the policy reform process involving a number of analytical activities. This misperception of evaluation as simply a reporting requirement may account for the lack of interest among donor partners. A single evaluation may be of limited use in tracking implementation success or analyzing impacts of policy reforms. In addition, if evaluation has not been planned from the beginning of policy implementation, the information needed to track progress may be limited, making it more difficult still to prepare the evaluation report. While the report may be the culmination of this stage of the policy process, evaluation should include many intermediate activities to track and analyze progress. The benefits of evaluation. Donors and their partners do not always appreciate the potential benefits of evaluation. For donors, evaluation may be viewed as the report required to account for assistance resources devoted to projects and programs. Unless donors are explicitly required to analyze the results of the policy reforms, it is less costly to limit the focus of their evaluations to the inputs and outputs of the assistance program. Also, the policy reform’s success may occur long after the donor’s assistance program is completed. POLICY REFORM LESSONS LEARNED Thus, donors’ partners will need to understand the benefits of evaluation in terms of improved accountability and transparency and press for evaluations to be planned and indicator data collected during the assistance program. Policy reform is “noisy.” A key challenge in evaluation is to “filter out the noise” in order to focus only on the relevant information and analyses that answer the question: Is the policy achieving the desired results? The noise occurs because policy reforms are not implemented in a vacuum but subject to dynamic changes in the political, economic, social and cultural context. The political and economic setting for implementation is similarly dynamic, with agencies balancing priorities and budgets for a variety of policy reforms. Need a chorus, not one voice singing the praises of the policy. Many voices will often be more convincing than a single voice and will lend credibility to the policy as well as the policymakers. Evaluation results should be generated at frequent intervals and a dissemination strategy developed to ensure the results are shared widely among implementing agencies, the general public and affected parties. PROGRAMMATIC IMPLICATIONS 1. Reach agreement with all relevant parties on the expected results of the policy reform, when they are expected to occur, how the results might change over time, the potential obstacles to success, and how to account for these in evaluation. 2. Utilize the work-planning process to engage counterparts in discussions about what will be achieved by the assistance program and the policy reforms, decide how to measure and evaluate success, discuss the timing and frequency of evaluation activities. 3. Build progress reporting into the implementation process so that local partners can observe whether or not the reforms are accomplishing the goals that were set during the design stage and if not, to make mid-course corrections in the policy or its implementation. EVALUATE THE POLICY 33 LESSON 21: TELL A STORY—PROCESS AND RESULTS WHAT HAS BEEN LEARNED Evaluation must tell a story—in other words, it should provide feedback to policymakers and managers that can guide diagnosis and changes in policy design and its implementation. Evaluation will be more useful if care is taken to carefully present the problem statement, establish the baseline for comparing the “before” and “after” policy scenarios, identify criteria and indicators to monitor change, identify information gaps and plan how to collect data for the analysis, and establish the monitoring schedule to answer the various questions that comprise the evaluation. KEY UNDERLYING ISSUES Answering the basic policy question. The role of evaluation in the policy process is to answer the question: Is the policy working as well as it should? To answer this, an evaluation needs to assess both the effectiveness of implementation as a process and the outcome or results of the policy. The implementation effort can be evaluated earlier in the process than results, given the time required for affected parties to respond to the policy change, and for their behavioral responses to result in improvements. Planning ahead. An important consideration is to characterize the baseline so that evaluators can distinguish the “before” from the “after.” In cases where the policy reform process has involved diagnosis and extensive dialogue between policymakers and affected parties, some of the required baseline information may already be available. Less than everything you need to know. An evaluation is costly to conduct, and it may not be possible to carry out a comprehensive analysis of all relevant policy questions. In addition to cost and resource considerations, there are spatial and temporal dimensions to the policy reform that must be considered in prioritizing the policy questions to be evaluated. Setting the evaluation hypothesis. The goals of the policy reform are often stated in qualitative as opposed to quantitative terms, requiring the analysts to determine what constitutes success or effectiveness. Often this reflects the early stages of our scientific understanding, especially of the natural systems. It may be necessary to discuss different specifications of the hypothesis before undertaking the evaluation. Attributing change. External factors may have both positive and negative influences on the performances of policies. It may be necessary to analyze these factors and attribute results to each of the most important factors. For example, periods of recession, high rates of inflation, or unfavorable costs of capital may discourage facilities or other affected parties from undertaking investments in pollution controls, alternative activities, or new methods. PROGRAMMATIC IMPLICATIONS 1. Formulate a set of questions early in the process that will help assess the use of assistance resources and document the results of policy reforms. These questions may include the following: a. Who will use the results of the evaluation? b. What types of information and analyses will best enable users to judge if the policy is successful? c. When will the evaluation results be needed? d. How should evaluation results be communicated to various audiences? e. Who will establish performance criteria, develop baselines, collect trend data, conduct the analyses, and disseminate the results? f. Do the individuals and agencies involved in conducting the evaluation have the appropriate skills and necessary resources? 2. Revisit the goals of the policy in the course of the evaluation to ensure that the process and results associated with the assistance effort and policy reforms are being assessed accurately. 34 EVALUATE THE POLICY POLICY REFORM LESSONS LEARNED LESSON 22: LEARN TO TRACK WHAT IS IMPORTANT WHAT HAS BEEN LEARNED An evaluation is tailored to answer the set of questions that donors, their partners, and other counterparts and affected parties have identified as being appropriate to measure progress in achieving policy goals. Just as evaluations do not analyze every question that could be asked about the policy and its implementation, not all types of information should be monitored and evaluated. The selection of indicators is a critical step in planning and conducting evaluations. KEY UNDERLYING ISSUES Tracking for accountability. For donors or their partners, what is important to track often depends on what their respective governments, the general public, or affected parties expect will be achieved. For donors, accountability often focuses on whether assistance resources are used effectively, which may involve monitoring of indicators that describe inputs and outputs to determine how the resources were employed. For counterparts, accountability requires tracking of both process indicators and results because of their need to demonstrate progress as well as the impact of policy reforms. Different audiences. For a given evaluation question, different audiences may expect the question to be answered in different ways. For example, if a policy is designed to reduce nutrient loading, farmer groups will want to know what impact the policy has on farm profits while the public’s interest in the policy’s impact may well concern water quality. Thus, multiple indicators may be needed to answer a single evaluation question. Complex processes. Policy implementation will often occur in many steps, taken by numerous agencies and affected parties, and involving a variety of activities. While it is possible to develop indicators to track all of these activities, it may be useful to construct composite indicators or indexes to provide a sense of overall progress in implementing the reform. Results versus impacts. Indicators may not be sufficient to measure impacts and additional analysis may be required. For example, air quality can be monitored but does not enable policymakers to place a value on the benefits of improved quality in terms of health effects. While designing monitoring plans, the acceptance of results indicators as surrogates for impact measures should be discussed and, if not acceptable, plans should be made to conduct supplemental analysis. Unintended successes. Sometimes the policy reforms result in different types of successes than was originally envisioned, either for the target reform or affected group. For example, a donor-led dialogue related to a specific policy reform may lead assistance partners to adopt and make wider use of participatory processes. Unintended failures. A policy may be soundly designed and implemented yet still fail to yield desired results due to exogenous shocks or policy decisions unrelated to the reform. It may be necessary then to document, ex ante, the potential of these factors to undermine the policy. Some of these external factors such as inflation, the strength of capital markets, or trends in weather can be monitored along with indicators more directly related to the policy reform. PROGRAMMATIC IMPLICATIONS 1. Employ USAID’s series of guidance documents including the Performance Monitoring and Evaluation Tips to support development and implementation of performance indicators. 2. Carefully define indicators that involve threshold terms used to characterize a degree of completion such as “operational,” “fully implemented,” or “innovation adopted.” 3. The ideal or most accurate indicator may not be affordable or practical to track on a sustained basis, particularly if a donor will later need to shift responsibility for tracking the indicator to assistance partners that lack the required financial or human resources or commitment. POLICY REFORM LESSONS LEARNED EVALUATE THE POLICY 35 POLICY REFORM LESSONS LEARNED LIST OF CASES LESSON CASE REGION DESCRIPTION Lesson 1 Case 1.1 LAC Being Prepared to face down the opposition: air pollution control in Quito, Ecuador Case 1.2 ANE Private sector power development in Pakistan proves difficult without “champions” Case 1.3 ANE Reform of water policies in Jordan requires willing and committed partners Case 1.4 ANE Engaging high-level forums to champion policy change: the US–Egyptian partnership for economic growth and development Lesson 2 Case 2.1 ANE Involving key stakeholders in policy dialogue: USAID’s Natural Resources Project Case 2.2 LAC The importance of building consensus for policy change: Ecuador’s Agrarian Development Law of 1994 Case 2.3 EE Building consensus on regional initiatives to improve water quality in Central Asia Case 2.4 ANE Monitoring and evaluation of the forest policy reform process: Global Witness as a watchdog in Cambodia Lesson 3 Case 3.1 ANE Donor coordination in Indonesia’s forestry crisis Case 3.2 EE Coordination and cooperation improve donor assistance to Central and Eastern Europe Lesson 4 Case 4.1 ANE FAO’s efforts for involving and better addressing concerns of all stakeholders: Gender and the policy process in natural resource management in Asia Case 4.2 AFR Opportunistic policy dialogue: shifting the focus to capitalize on contextual change and structural transformation in Niger Case 4.3 ANE, AFR Forest policy dialogue in a variety of forms and settings Case 4.4 LAC The importance of local institutions: IMAZON in Brazil Lesson 5 Case 5.1 EE Sharing of international experience enhances policy dialogue to improve water quality Case 5.2 ANE Interactive water policy dialogue in Egypt Lesson 6 Case 6.1 ANE South Korea’s environment improves after two decades after legislation introduced Case 6.2 LAC Reforming institutions often takes longer than finding technical solutions: solid waste collection in Machala, Ecuador Case 6.3 EE Small start to win early successes: a bite-sized approach to tradable permits in the Czech Republic Lesson 7 Case 7.1 ANE Political change provides stimulus for environmental policy and institutional reform in Sri Lanka Case 7.2 ANE Officials in 10th of Ramadan City, Egypt involve stakeholders at early stage in developing a municipal environmental management system (EMS) Case 7.3 EE Window of opportunity opens to test market-based air pollution policies in Almaty, Kazakhstan Lesson 8 Case 8.1 AFR Successful policy reform requires understanding of complex multidisciplinary contexts: wildlife management in Tanzania Case 8.2 AFR Investments in information systems helps Madagascar build capacity to monitor and assess development projects Case 8.3 ANE Use of analytical approaches in the Philippines Environmental and Natural Resources Accounting Project Lesson 9 Case 9.1 ANE Egypt builds support for agricultural policy reforms by showcasing stakeholder benefits Case 9.2 ANE Publicity of China's system of urban environmental indicators leads to improved performance Case 9.3 ANE PROPER: innovative use of facility information to generate support for policy reforms in Indonesia Lesson 10 Case 10.1 EE Solid analysis furthers diagnosis of timber auctions in Romania Case 10.2 EE Building analytical capacity for problem diagnosis may needed to ensure that investment decisions are sound: the case of Novokuznetsk, Russia Case 10.3 LAC Development of a comprehensive forest policy in El Salvador Case 10.4 ANE Evaluating and learning from experience in designing policies and programs to protect Egypt’s POLICY REFORM LESSONS LEARNED LIST OF CASES 37 LESSON CASE REGION DESCRIPTION Red Sea coastal resources Lesson 11 Case 11.1 ANE Cash transfers are catalysts for Egyptian policy reform program Case 11.2 ANE External pressure results in limited forestry sector reforms in Indonesia Lesson 12 Case 12.1 EE Policy design in the period of economic transition: overcoming the poor climate for environmental investments in CEE and NIS countries Case 12.2 AFR Conserving Madagascar’s unique biological diversity through rural development Case 12.3 AFR Regional policy design to preserve wildlife habitats in Botswana Case 12.4 EE Managing Central Asia's water resources: setting an appropriate spatial scale for policy design Lesson 13 Case 13.1 ANE, AFR Conflicting forestry policies create disincentives for sustainable management strategies Case 13.2 LAC Agreement for higher tourism operator fees in the Galapagos Islands reached by extolling benefits to economic interests of protecting ecological resources Case 13.3 AFR Challenges of structuring incentives for reform in Zambia Lesson 14 Case 14.1 ANE Environmental monitoring and enforcement authorities devolve to local governments in Indonesia Case 14.2 ANE Prosecution constraints limit effectiveness of system of fines for marine damages in Egypt Case 14.3 ANE Powerful interests and regulatory reform of the electric power sector in India Lesson 15 Case 15.1 ANE Changes in policy design to improve management of Bunaken National Park in North Sulawesi, Indonesia Case 15.2 EE Unrealistic environmental standards undermine implementation in the former Soviet Union Lesson 16 Case 16.1 EE CEE countries prepare implementation strategies for approximating with European Union environmental legislation Case 16.2 LAC Designing monitoring plans to evaluate Panama's progress in protecting the Canal's watershed Case 16.3 AFR Monitoring the results of Forest Code reforms - tracking the process and measuring the impact Case 16.4 EE Developing environmental financing strategies to support compliance schedules among CEE countries Lesson 17 Case 17.1 EE Anticipating the challenge of sustaining environmental investments in Eastern European countries Case 17.2 LAC Graduation requirements provide stimulus for Jamaican NGOs to sustainably finance their operations Case 17.3 ANE Revenue enhancement to support Red Sea protection Lesson 18 Case 18.1 AFR Strengthening stakeholder capacity: popularization of the Rural Code in Niger Case 18.2 EE Increasing stakeholder capacity to identify and mobilize financing for environmental investments in Poland Case 18.3 LAC Building capacity to address regional environmental issues in Central America Lesson 19 Case 19.1 AFR Prospects for improved resource management in Niger enhanced by reforms that generate local benefits Case 19.2 AFR Capacity building in communities for improved wildlife management in Tanzania Case 19.3 ANE Support for civil society-led decentralization initiatives for natural resource management in Indonesia Lesson 20 Case 20.1 ANE FAO—looking for impact beyond the life of the program Lesson 21 Case 21.1 AFR Senegal KAP surveys Case 21.2 LAC Macroeconomic policies affect the success of forestry policies in Guatemala Case 21.3 EE “Best practices” criteria guide environmental fund reviews Lesson 22 Case 22.1 AFR Use of indicators in an M&E framework: Niger Agricultural Sector Development Grant, Phase II Case 22.2 ANE Testing and evaluating arrangements for decentralized policy implementation in Egypt Regions LAC—Latin America and Caribbean EE—Eastern Europe and Eurasia ANE—Asia and Near East 38 LIST OF CASES POLICY REFORM LESSONS LEARNED POLICY REFORM LESSONS LEARNED FUNDAMENTALS 39 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523 Tel: (202) 712-0000 Fax: (202) 216-3524 www.usaid.gov