AN END OF PROGRAM EVALUATION OF THE COOPERATIVE AGREEMENT BETWEEN USAID/EGAT AND THE INTERNATIONAL COUNCIL FOR LOCAL ENVIRONMENTAL INITIATIVES (ICLEI) Submitted to: United States Agency for International Development Bureau for Economic Growth, Agriculture, and Trade Office of Environment and Science Policy Global Climate Change Team (USAID, EGAT/ESP/GCC) Under: Purchase Order No. CZ00005CZ2000/22 Prepared by: Joseph Lieberson, Economist and Evaluator Submitted by: 1730 North Lynn Street Arlington, VA 22209-2023 October 17, 2005 Program Evaluation of the Cooperative Agreement i October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) TABLE OF CONTENTS Page No. Acronyms & Definitions........................................................................................................... ii Executive Summary................................................................................................................. iv A. Background....................................................................................................... 1 1. Why the evaluation was done ............................................................... 1 2. Evaluation research methodology......................................................... 2 3. USAID’s Global Climate Change Program.......................................... 3 4. The International Council for Local Environmental Initiatives (ICLEI)................................................................................. 4 B. Evaluation Findings .......................................................................................... 9 1. The cooperative agreement................................................................... 9 2. Cities for Climate Protection (CCP) ................................................... 11 3. USAID Operational Issues.................................................................. 13 4. Sustainability and Replication ............................................................ 14 5. Other Issues......................................................................................... 15 C. Summary Conclusions .................................................................................... 17 ANNEXES Annex A Questions and Issues Which the Evaluation Examined................................ A-1 Annex B The Country Programs.................................................................................. B-1 Annex C Interviewees for this Evaluation ................................................................... C-1 Annex D Data Collection Interview Protocols for USAID and ICLEI Staff and for Country Participants ................................................................ D-1 Annex E Evaluation Statement of Work.......................................................................E-1 Annex F Evaluation Work plan ....................................................................................F-1 Program Evaluation of the Cooperative Agreement ii October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) ACRONYMS & DEFINITIONS CCP Cities for Climate Protection. A five-step program designed to reduce greenhouse gas emissions at the city level. Climate Change Climate change refers to a statistically significant variation in either the mean state of the climate or in its variability, persisting for an extended period (typically decades or longer).Climate change may be due to natural internal processes or external forcings, or to persistent anthropogenic changes in the composition of the atmosphere or in land use. Note that the United Nations Framework Convention on Climate Change (UNFCCC), in its Article 1, defines “climate change” as: “a change of climate which is attributed directly or indirectly to human activity that alters the composition of the global atmosphere and which is in addition to natural climate variability observed over comparable time periods.” The UNFCCC thus makes a distinction between “climate change” attributable to human activities altering the atmospheric composition, and “climate variability” attributable to natural causes. GHG Greenhouse gases are those gaseous constituents of the atmosphere, both natural and anthropogenic, that absorb and emit radiation at specific wavelengths within the spectrum of infrared radiation emitted by the Earth’s surface, the atmosphere, and clouds. This property causes the greenhouse effect. Water vapor (H2O), carbon dioxide (CO2), nitrous oxide (N2O), methane (CH4), and ozone (O3) are the primary greenhouse gases in the Earth’s atmosphere. Moreover there are a number of entirely human-made greenhouse gases in the atmosphere, such as the halocarbons and other chlorine- and bromine-containing substances, dealt with under the Montreal Protocol. Besides CO2, N2O, and CH4, the Kyoto Protocol deals with the greenhouse gases sulfur hexafluoride (SF6), hydrofluorocarbons (HFCs), and perfluorocarbons (PFCs). HEAT Harmonizing Emission Analysis Tool. An online software program, developed by ICLEI that aims to help local governments consider the costs, benefits and emission reductions from various approaches. ICLEI International Council for Local Environmental Initiatives. An international association of local governments as well as national and regional local government organizations working to improve the global environment while supporting sustainable development at the local level. Program Evaluation of the Cooperative Agreement iii October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) NGO Non-governmental Organization USAID United States Agency for International Development. An independent federal government agency that receives overall foreign policy guidance from the Secretary of State. USAID has been the principal U.S. agency to extend assistance to countries recovering from disaster, trying to escape poverty, and engaging in democratic reforms. Program Evaluation of the Cooperative Agreement iv October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) EXECUTIVE SUMMARY In 1998, USAID and the International Council for Local Environmental Initiatives (ICLEI) launched a cooperative agreement to work on global climate change at the local level. The cooperative agreement gave USAID the opportunity to test ICLEI’s Cities for Climate Protection (CCP) approach as a way to reduce greenhouse gas (GHG) emissions in several countries at the municipal level. A cooperative agreement was used rather than a contract since ICLEI’s bottom￾up approach and its Cities for Climate Protection (CCP) program represented a complete package that USAID was supporting. This report provides the findings from a Washington-based final program evaluation carried out during August-October, 2005. The evaluation found that the partnership was successful since both USAID and ICLEI shared the same objectives of reducing GHG emissions at the local level, while supporting development and economic growth priorities of city governments. ICLEI fully involved USAID/Washington and field staff in the program and provided quarterly reports, which made it possible for USAID to stand back and avoid micro-managing the program. Both ICLEI and USAID achieved their own institutional objectives. The CCP program is a logical approach that provides a clear roadmap on how to set goals, build consensus, choose activities and measure results. There is universal agreement among USAID/Washington staff, USAID Mission staff, and local municipal participants that CCP is a sound program. CCP promotes low cost investments that reduce GHG emissions, generates a quick financial payback and reduces local pollution. As a financial mechanism, the cooperative agreement provides a simple way for a USAID Mission to provide financing for activities in its own country through a buy-in. There is a question on how to operate a longer-term program like CCP when many of the participants have a short time horizon. USAID Missions often have changes in staff, budget priorities and program policy. For example, USAID/India and USAID/Indonesia committed to funding multi-year activities but there was a reversal in the second year as funding was canceled. At the municipal level, when the mayor or commissioner changes, programs often change, too. This is a problem that USAID faces in every sector. While there is no easy solution, USAID might want to require USAID Missions to provide at least two years of forward-funding at the start of an activity. The CCP cities have the capacity to manage and maintain their activities and each city is linked to other cities through an ICLEI developed network, which provides needed support. Since benefits are clear and financial incentives are strong, there is little likelihood that the activities will end. The program is sustainable. A different question is whether CCP is replicable on a national basis. If 10 or even 20 city programs are launched in a country, there is no assurance that CCP will ever affect more than a few percent of a country’s several thousand municipalities. CCP needs to go beyond serving just a few cities to create a larger program. It needs to develop linkages to those working on the same issues at the national, province or state level and to explore prospects for the private sector to finance, build and operate city projects. At the city level, it needs to fully involve budget and Program Evaluation of the Cooperative Agreement v October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) finance offices, since they are the ones that must be convinced that any activity should be funded. The program has developed a set of effective activities that could be adopted by other cities. USAID/Washington could develop a follow-on program with ICLEI to launch additional city programs in other countries with a new cooperative agreement. Alternatively, USAID could develop a new program that uses the success of the pilot activities to move from the past retail, city-by-city approach, to a wholesale national approach that still has cities as its focus but mainstreams CCP programs by involving national government ministries and the private sector to reach a much larger number of cities in several countries. Program Evaluation of the Cooperative Agreement 1 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) COOPERATIVE AGREEMENT BETWEEN USAID/EGAT AND THE INTERNATIONAL COUNCIL FOR LOCAL ENVIRONMENTAL INITIATIVES (ICLEI) A. BACKGROUND 1. WHY THE EVALUATION WAS DONE USAID and the International Council for Local Environmental Initiatives (ICLEI) established a Cooperative Agreement in 1998 to launch new approaches to reduce greenhouse gas (GHG) emissions at the local level. The cooperative agreement is in its seventh year and ends in 2005. At the FY 2004 EGAT Portfolio Review, senior EGAT management decided that it was time to evaluate the effectiveness of the cooperative agreement in achieving program objectives and to help USAID decide what should be done in the future. This end of program evaluation answers those questions by: (1) determining if the partnership between USAID and ICLEI successfully advanced the objectives of both USAID and ICLEI; (2) identifying what has worked well, what has not, and what changes could make the partnership more efficient and effective; and (3) determining if the relationship between USAID and ICLEI should continue, and if so under what arrangements. The cooperative agreement is a centrally funded activity, designed to introduce new ideas and techniques overseas. Like any other pilot effort, if the approach is sound, after a few years the program will take root, be sustained and replicated elsewhere. Future funding will come from the country itself or with some support from a USAID country Mission or another donor. On the other hand, if the country does not see a benefit or fails to give it high priority, it will come to an end. It is important to learn how and why pilot programs succeed or fail by evaluating the experience in participating countries. The evaluation examined five country field programs. The Philippines and Mexico were two early participants followed by India, Indonesia and South Africa. USAID/South Africa funds its own cooperative agreement with ICLEI, separate from the central cooperative agreement managed by EGAT/ESP/GCC. The South African program is not part of the central cooperative agreement, but is examined in this evaluation to provide additional information and data. The next section of this report describes the evaluation methodology including how data was collected and analyzed. It is followed by a brief description of USAID’s Global Climate Change Program. After that comes a discussion of the ICLEI Cities for Climate Protection (CCP) Program and budget information followed by a section on the USAID/ICLEI Cooperative Agreement. Section 3 entitled, ‘Evaluation Findings’ analyzes all of the country programs to determine: how well the cooperative agreement has worked, the performance of the CCP, USAID operational issues, sustainability, and replication. There is a short conclusions section which is followed by six annexes at the end of the report which provide additional information Program Evaluation of the Cooperative Agreement 2 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) on country programs along with information on evaluation objectives, the evaluation work plan, issues examined, and information on data collection. 2. EVALUATION RESEARCH METHODOLOGY This end of program evaluation focuses on the USAID/ICLEI cooperative agreement and the results that were achieved. It analyzes the experiences of five countries that participated in the ICLEI CCP program. Due to time and financial constraints, the evaluation was done from Washington rather than in the field. These are the key evaluation questions that evaluation analyzed: • Was the USAID/ICLEI partnership successful in advancing the objectives of both USAID and ICLEI? • How effective was the cooperative agreement and how well did ICLEI perform? • How and why were intended results achieved or not achieved? Were there other results that were not easily measured or quantified? • Are activities and program results sustainable and replicable? • Should the USAID/ICLEI partnership continue and if so, under what arrangements? The evaluation started with a review of program documentation which included the original cooperative agreement of September 1998, and four modifications to that agreement. ICLEI submitted quarterly progress reports to USAID and those covering October 2003 to March 2005 were available and reviewed by the evaluator. Other program materials from the USAID Climate Change Team and from ICLEI were also reviewed.1 The next step was to analyze the reports and other data to identify key questions and issues to examine (see Annex A). Those questions were then organized into a data collection interview protocol for USAID and ICLEI staff and another protocol for those in the field, working on local level activities supported by the cooperative agreement (see Annex D for the two interview protocols). To assess program operations and results, many evaluations collect field data by interviewing 20- 50 key participants in each country, collecting data on local programs, and checking on program results through site visits. In this case that would have been an expensive and time-consuming approach --- particularly since the cooperative agreement covered 64 cities2 in five countries. Instead, a different approach of interviewing participants outside of Washington by telephone was used. The evaluator personally interviewed four USAID staff in Washington and reviewed USAID/Washington files on the cooperative agreement. Three ICLEI staff in California were interviewed by phone as were overseas participants. For the five countries, interviews covered 11 USAID Mission staff and 7 country participants --- typically the mayor, government managers, engineers and other technical experts who implemented the programs in these developing cities. There are of course drawbacks to this approach. Only 3 to 5 people were 1 Due to the high turnover rate of CTOs, files for the early years of the program do not appear to be complete. 2 The USAID/Washington cooperative agreement covers 52 cities in Mexico, Indonesia, India and the Philippines. USAID/South Africa has its own cooperative agreement with ICLEI covering another 12 cities in South Africa. Program Evaluation of the Cooperative Agreement 3 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) interviewed in each country, which is a limited sample, and it was not possible to physically confirm whether investments and claimed benefits actually occurred. The 18 overseas interviewees provide a wide range of perspectives reflecting differing country conditions, but there is a high degree of agreement on key issues. The use of a standard questionnaire allows a process of “triangulation,” by posing the same set of questions to USAID, ICLEI and participants from a number of different cities. In most cases, similar patterns and experiences are identified for an issue though there are differences in the type of projects that different cities thought to be most important and the way local political concerns affected city programs. 3. USAID’S GLOBAL CLIMATE CHANGE PROGRAM Compounds found in the Earth’s atmosphere act as greenhouse gases (GHGs). They form a natural blanket surrounding the Earth, keeping it warm by retaining some of the sun’s energy in the form of heat. This keeps the planet’s temperature where it needs to be to support life as we know it. Energy production, industry, agriculture and transport systems are contributing greenhouse gases, such as CO2, to the atmosphere at a faster rate than they can be absorbed by ecosystems such as forests. If greenhouse gases continue to accumulate in the atmosphere, scientists expect that global temperatures could increase 2.5 to 10.4 degrees Fahrenheit by 2100, relative to 1990.3 Forecasts of climate-related impacts are much less certain than predicted temperature increases. However, most scientists agree that global climate change will affect the planet and the way we live. Addressing the causes and effects of climate change is an important part of USAID’s development assistance program. USAID environmental programs reduce greenhouse gas emissions while promoting energy efficiency, forest protection, biodiversity conservation, and other development goals. This “multiple benefits” approach helps countries achieve economic development without sacrificing environmental protection. USAID’s Global Climate Change (GCC) Program is active in more than 40 countries and provides about $180 million per year to promote: • Clean energy technology • Sustainable land use and forestry • Adaptation to climate change • Climate science for decision making USAID emphasizes partnerships with the private sector and working with local communities and national authorities, and non-governmental organizations to create alliances that build on the relative strengths of each. Bringing together a diverse range of stakeholders helps avoid unnecessary duplication and lays the foundation for a sustained and integrated approach. The next section describes ICLEI’s Cities for Climate Change Program (CCP), which works with local governments to reduce GHG emissions and which USAID has supported. 3 U.S. Government Climate Change Assistance for the Developing World. Making a Difference in People’s Lives: USAID’s Climate Change Initiative, 1998-2002, EGAT/ESP/GCC. Program Evaluation of the Cooperative Agreement 4 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) 4. THE INTERNATIONAL COUNCIL FOR LOCAL ENVIRONMENTAL INITIATIVES (ICLEI) ICLEI is an international association of local governments as well as national and regional local government organizations working to improve the global environment while supporting sustainable development at the local level. It was organized in 1990 under the sponsorship of the United Nations Environment Program and the International Union of Local Authorities. ICLEI has offices on all major continents and there is a U.S. office. Global warming and climate protection are recognized as important world issues at international conferences and by many national governments. While it is useful to identify and discuss those problems at national and international levels, many of the solutions have to be implemented at the local urban level where approximately half of the world’s population now lives. Many pollutants come from daily energy, transportation, waste and land use practices in communities throughout the world. Local governments can influence those practices --- which means that local governments can affect pollution that directly harms city residents and greenhouse gases that cause global climate change. Cities for Climate Protection (CCP) is a global campaign of ICLEI. CCP engages local governments to adopt policies and implement measures to achieve quantifiable reductions in local GHG emissions, improve air quality, and enhance urban livability and sustainability. Over 650 local governments (including 150 U.S. cities) from 6 continents and over 50 countries participate in the CCP program. They collectively represent approximately 10 percent of the world’s man-made greenhouse gas emissions. It would be an extremely tough job selling cities on the need to take independent action to deal with global warming, which is viewed by many as a world-wide problem beyond the interest and capacity of local governments. But it is possible to convince cities that they can reduce local pollution while reducing municipal costs --- and at the same time help reduce greenhouse gas emissions. The CCP program provides a framework for local governments to integrate climate protection policies with actions that address municipal concerns such as saving money, creating jobs, improving transportation, improving air quality and enhancing community livability. Local cities undertake the CCP program because they see a chance to reduce costs and deliver benefits to local people. ICLEI’s Cities for Climate Protection (CCP) program is based on a five-step process used successfully in the U.S. and in a number of countries. To participate, local governments commit to the CCP five milestones. The milestones and the CCP Software provide a simple, standardized and proven way to calculate greenhouse gas emissions and to monitor, measure and report on performance. To complete the five milestones, a city: 1. Conducts a baseline inventory and forecast of community and municipal energy use, greenhouse gas emissions and waste generation for a target year. 2. Sets a reduction goal along with a timetable, usually as a percentage reduction below the quantity of emissions released in the baseline year. Reaching agreement on a goal fosters political will and creates a framework to guide planning and implementation. 3. Develops a local action plan that outlines how the municipality will reach its goal. Program Evaluation of the Cooperative Agreement 5 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) 4. Implements the local action plan by focusing on the emission sources and reduction measures identified in the local action plan. 5. Monitors progress and reports on results by routine monitoring of emission reduction measures that typically quantify greenhouse gas and energy consumption reductions. CCP milestones include a blend of political consensus building, actions that bring both a local and global benefit, and technical quantification and analysis that support and give credence to each other. Some of the milestones are technical (milestone 1 requires completion of an emissions inventory); while others are political (milestones 2 and 3 require local political debate and consensus building). The hard numbers that result from milestone 1 support the debate and planning activities of milestones 2 (setting goals) and 3 (an action plan), which in turn support milestone 4, the implementation of GHG reduction measures. Before a local municipality can become a CCP participant, its elected council, executive body or chief executive officer must adopt a resolution to undertake the five-milestone process. This key step demonstrates from the outset that community leaders and constituents are aware and support the process. High level support from officials and stakeholders gives credibility and momentum to the campaign. ICLEI provides support and technical assistance to municipalities as they work through the milestones. For example, ICLEI holds training workshops on how to do an emissions inventory and how to select and quantify performance measures. ICLEI draws on the campaign's global network of municipalities to provide case studies and local government guides. To help local governments achieve their CCP goals, ICLEI facilitates city-to-city exchanges and helps cities find project financing. These are opportunities typical municipalities would not have if they were operating without support from the ICLEI network. ICLEI’s “CCP Software” helps municipalities through the first three milestones. It has been used successfully by over 450 local governments worldwide. The CCP software boils down the methodology on the inventory and measures to quantify emission reduction targets into a simple, easy to use package suitable for the average municipal official. For Milestone 1, the software generates the baseline inventory and forecast based upon entry of locally available energy use, transportation and waste generation data. For Milestones 2 and 3, the software will track reductions from specific measures so that a municipality can determine the impact of action plan measures against achievement of the reduction target. To help select optimal measures, the software tracks associated local co-benefits, such as energy and cost savings. With ICLEI's help, local governments have been able to recognize that tackling climate change is good business. For example, if a municipality decides to reduce GHG emissions by implementing energy efficiency measures, the action reduces energy costs (fiscal responsibility), improves local air quality (quality of life), and creates jobs during retrofit projects (economic development). The development and implementation of new technologies can also stimulate local economic activity. When the CCP Campaign was first introduced, many local government managers were able to "sell" the program to their decision makers on the strength of these co-benefits, rather than on the merits of slowing global warming. Program Evaluation of the Cooperative Agreement 6 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) ICLEI recently developed its Harmonized Emissions Analysis Tool (HEAT). It is a software solution supporting ICLEI greenhouse gas and air pollution emission reduction programs. It provides an online multilingual, global database of emissions and action plans covering a broad array of sectors such as energy, waste and transportation. HEAT includes standardized reporting for all cities and easy-to-use emissions calculators. This software assists local governments in developing emissions inventories and making action plans to reduce emissions. It is a database tool that tracks multiple records of energy and emissions data, as well as multiple records of emission reduction activities. The software allows a municipality to: • Build an emissions inventory by having users enter data records of energy use that are translated into emissions by the software. • Conduct a simple forecast with user-input parameters. • Set a target or goal for reducing emissions (e.g., reduce GHG emissions by 10%) • Quantify emission reduction activities. • Develop and report an emissions reduction action plan, which consists of a series quantified measures designed to meet the reduction target as well as progress towards meeting a target. When HEAT calculates benefits, it provides an analysis of both local pollution benefits and greenhouse gas co-benefits. It also contains calculators to compute and compare options for a variety of measures, such as landfill gas capture, a fleet fuel switch, more energy efficient lighting, more efficient electric pumps and other emission reduction options. ICLEI reports strong demand for HEAT training and demonstrations. ICLEI has run training programs in India, Brazil and South Africa and live demonstrations for the World Bank, Asian Development Bank, the Clean Air Initiative, and the US Environmental Protection Agency. After the training in Hyderabad, India ICLEI was asked to provide training for the Indian Andhra Pradesh Pollution Control Board, the South African Department of Environmental Affairs and Tourism and the Ministry for Environment in Indonesia. ICLEI expects the HEAT software to be of use to the wider community including USAID missions, consultants, companies and academics. New technology and new knowledge are important but new attitudes and approaches to city operations are the key results of the CCP program. ICLEI provides training, technology and software, but what makes it effective is the emphasis on capacity building at the local level. ICLEI has an in-country representative and technical experts from ICLEI’s headquarters who work closely with each city to develop local management and technical skills. Training is not generic or abstract. It is directly tied to the CCP milestones and problems cities face as they implement their own CCP program. ICLEI staff are available to guide a city through each of the CCP milestones and any unexpected problems. The emphasis on capacity building and learning by doing helps make the program sustainable. 5. THE USAID/ICLEI COOPERATIVE AGREEMENT By the late 1980’s, USAID had been working for several years on a number of policy reform programs and projects related to energy conservation, pollution and global climate change. Most USAID activities were at the national level. It was clear that something else was needed at the Program Evaluation of the Cooperative Agreement 7 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) local level to include local governments. In 1998, USAID examined the ICLEI program to see if it might provide a useful approach. ICLEI had been working with a growing number of U.S. cities through its Cities for Climate Protection (CCP) program. It recognized that it was slow and difficult to change national policies, but also realized much could be done at the local level. Though global climate change was its objective, ICLEI realized that would be hard to sell to cash-strapped municipal governments. Instead, ICLEI promoted measures that reduced greenhouse gas emissions but also generated cost savings for the local government and improved health and living conditions for local people. It was an effective way to market the program to local governments—an approach which appealed to USAID. ICLEI was an NGO with enthusiasm, a strong mission and a commitment to solve global climate change problems at the local level. CCP was a logical and successful approach to create sustainable programs by developing a local consensus and then building local government capacity to carry out the effort. The only uncertainty was whether the approach could be successfully replicated in developing countries. USAID has a history of funding new ideas and new approaches. It decided that the CCP approach was sound and it was worth the risk to launch pilot CCP programs in two countries. USAID/Washington would fund the first two years and USAID Missions would have the opportunity to “buy-in” to the effort by contributing their own funds to expand the program. If the pilots did well, the approach could be expanded to other cities and to other countries. After several years it should be clear whether the approach would work or not. Next there was the question of what implementation mechanism to use. If the object is to generate immediate results (school buildings constructed, health workers trained, agriculture services delivered to farmers), then USAID usually hires a contractor. In this case, while results would be generated, ICLEI would have to devote much time to explaining and selling its program to local governments. Then it would provide training sessions and technical assistance to help local governments move through the 5-step CCP program. As a development program, ICLEI’s bottom-up approach and its Cities for Climate Protection (CCP) program represented a complete package that USAID was supporting and a cooperative agreement, rather than a contract, was selected.4 In September 1998, USAID/Washington’s Global Bureau, Office of Environment, Urban Programs Division obligated $401,676 to ICLEI under a new Cooperative Agreement. Under this agreement, ICLEI would be charged with launching CCP campaigns in Mexico and the Philippines. Two years later, in 2000, an additional $420,000 was provided by USAID/ Washington to start development of programs in India and Indonesia. In 2001, 2003 and 2004, USAID/India provided additional funding to support ICLEI efforts in India. In 2003, 4 Per ADS 304, a Cooperative Agreement shall be used when…the principal purpose of the relationship is the transfer of money … to the recipient in order to accomplish a public purpose …and substantial involvement is anticipated between USAID and the recipient during the performance of the proposed activity. See http://www.usaid.gov/policy/ads/300/. Program Evaluation of the Cooperative Agreement 8 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) USAID/Indonesia did the same for its programs. By 2003 the Philippines and Mexico programs were completed. For FY 1998 to FY 2005, Chart 1 shows the funding sources and Chart 2 shows the line item budget categories. CHART 1 USAID Funding Sources, FY 1998 - FY 2005 Funding Source USAID/ Washington USAID/ India USAID/ Indonesia Funds Obligated Cumulative Total Funds Original Cooperative Agreement 1998 $ 401,676 0 0 $ 401,676 $ 401,676 Modification #1, 2000 $ 420,000 0 0 $ 420,000 $ 821,676 Modification #2, 2001 $ 191,630 $ 146,250 0 $ 337,880 $1,159,556 Modification #3, 2003 $ 101,939 $ 283,456 $ 200,000 $ 585,395 $1,744,951 Modification #4, 2004 $ 264,287 $ 108,284 0 $ 372,571 $2,117,522 Total Funding by Source $1,379,532 $ 537,990 $ 200,000 $2,117,522 CHART 2 USAID Line Item Budget for Project Activities FY 1998 – FY 2005 Line Item Amount Climate Action Project $ 597,146 India $ 674,459 Indonesia $ 698,043 Translation $ 15,000 South Africa HEAT Software Training $ 40,000 Brazil HEAT Software Training $ 50,000 HEAT Software Users Manual $ 12,638 COP 10-Side Event $ 30,236 TOTAL ESTIMATED COST $2,117,512 Program Evaluation of the Cooperative Agreement 9 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) B. EVALUATION FINDINGS 1. THE COOPERATIVE AGREEMENT USAID usually uses a contract with a private sector firm or non-governmental organization (NGO). The contractor is given a list of tasks that need to be completed and results that must be achieved. USAID prescribes the approach the contractor will follow and closely monitors performance and progress. In contrast, a cooperative agreement is more of a partnership. The cooperating organization has its own program and its own way of achieving results. In effect, USAID buys into that program and approach. A cooperative agreement spells out objectives, anticipated tasks and outcomes in broad terms, but it is not a performance-based contract. USAID is not buying specific results, but is helping implement a program that addresses a development challenge. Under such an arrangement, a question often arises: Does the cooperative agreement meet the objectives of both ICLEI and USAID? ICLEI’s Perspective ICLEI is an NGO noted for its strong enthusiasm for its mission of “helping cities work together for the global common good.” Reducing greenhouse gas emissions with the CCP program is one of its projects. With the cooperative agreement, ICLEI was able to successfully modify the International Cities for Climate Protection campaign to work in developing countries. From ICLEI’s point of view, USAID money allowed ICLEI to have a major impact on greenhouse gas emissions. As one ICLEI staffer said, it was a “perfect partnership.” Another staffer pointed out that ICLEI worked with 156 local governments in the U.S. with a total population of nearly 20 million. Under USAID’s cooperative agreement, ICLEI worked in five USAID countries. In India, for example, there are 23 million people in the 16 cities where ICLEI worked --- a population that represents more than the total population in ICLEI’s U.S. CCP program. The chance to have an impact on the lives of so many people and the environment was a unique opportunity for ICLEI. ICLEI was able to present its approach to local governments, gain their confidence and convince them of the value of the CCP approach. ICLEI worked closely with local governments to develop a program specifically tailored to their individual needs and then guided them through the 5 milestones of the CCP program. ICLEI was very satisfied with how smoothly the cooperative agreement worked with USAID/Washington and the USAID field missions. ICLEI noted that any partnership will have some problems, but in this case, none were serious. Over the 7 year project life, there were at least 4 Washington Cognizant Technical Officers (CTOs), also known as project managers. According to ICLEI, it would have been better to have had more consistent policy direction from USAID/Washington, but the advice and support from USAID/Washington was fine. In the USAID Missions, some local representatives have a more hands-on management approach while others do not. The different styles did not create any problems. ICLEI did run into problems when USAID Mission strategies shifted and no longer considered climate change a priority. To start a country program, up-front costs are high as support is Program Evaluation of the Cooperative Agreement 10 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) developed among local level politicians and technical staff in a number of different cities. Once that support is provided, training sessions, design work and technical support is needed. It may be a year or two before project operations begin; then benefits are generated and quickly increase into the future. But it did not always work that way. USAID/India and USAID/Indonesia both agreed to provide funding for their country programs in 2003. As such, ICLEI hired staff and expanded its efforts. But as things moved forward in 2004, USAID/Indonesia developed a new Strategic Plan, which did not include a focus on climate change, and backed out of its $200,000 pledge in 2004. Furthermore, USAID/India cut its $200,000 pledge in half in 2004. USAID/Washington provided funding for Indonesia to partly close the gap and to ensure that programs could be smoothly phased out ICLEI recognizes that donors do change their programs but a sudden drop in financing for a multi-year program is costly and harms the momentum that takes so long to develop. USAID’s Perspective As one USAID staffer said: “ICLEI is an NGO with strong enthusiasm and a good program to deal with climate change. We bought into a good idea that was successful in the U.S. and were able to take it overseas to support USAID objectives.” Another staffer noted that: “USAID usually works at the national level. By teaming up with ICLEI, USAID was able to achieve its own objectives of working with municipalities that want to protect the environment and promote economic growth. ICLEI works at the local level and when it develops successful programs, it is a good way to help a national government learn what it should support country-wide.” All agreed that ICLEI’s grassroots approach made sense, its staff was highly motivated and the development of networks among cities within a country and between countries was very effective. There was also agreement that both USAID and ICLEI shared the same viewpoint that better economic performance, energy conservation and financial savings are the best way to deal with climate change issues in the developing world. USAID Missions noted how well the ICLEI program meshed with their own energy conservation, urban services or global climate change objectives. Several USAID staff noted that a cooperative agreement means that USAID buys into the ICLEI program and USAID loses control. USAID is used to dealing with contractors and USAID field staff want a “hands on approach.” On the other hand, most staff noted that with a cooperative agreement, USAID cannot micro-manage a program and in this case that is the best approach. Even though a cooperative agreement is not the usual way USAID operates, USAID staff were very pleased with the cooperative agreement. Mission staff also noted that since CCP is a small program with a low dollar value, they are glad they do not have to micro-mange the activity. ICLEI can take a strong lead as long as USAID is a key participant. ICLEI uses local nationals rather than expatriate staff to operate local country offices and builds the capacity of each office to run its own country programs. ICLEI country directors are fully responsible for building relationships with supporters and operating programs on the ground. An ICLEI strength is having highly skilled in-country managers and support from the ICLEI home office. However, communications was a problem in one country. A USAID Mission manager noted that: “All communications and data went from the local ICLEI country representative to ICLEI headquarters in California and then headquarters sent the message to the USAID Mission. Running a local program through headquarters on the other side of the world Program Evaluation of the Cooperative Agreement 11 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) may not be an efficient way to run a program.” This problem was not mentioned by USAID managers in other countries. ICLEI and USAID Agree on the Benefits of the Cooperative Agreement ICLEI staff appreciate how smoothly the cooperative agreement works. A key point is that it offers needed flexibility to respond to local needs and changing conditions. ICLEI introduced the CCP approach to 64 cities which helped ICLEI achieve its objective of reducing GHG emissions. The surprise drop in funding from USAID/ India and USAID/Indonesia was the only thing ICLEI found disappointing --- and the problem would have been the same if it had been a contract rather than a cooperative agreement. Yearly funding is always “subject to availability” and priorities often shift. Everyone agrees that the cooperative agreement is the right approach. That is now, but seven years ago when the program was started, USAID was taking a risk on an experiment. While ICLEI says it would have been happy to have used a contract rather than a cooperative agreement, it might have created problems. USAID was not sure how the program would evolve, which approaches would be most successful and how it would be introduced to other countries. A contract would have had to specify the approach and funding line-items for the life of the project, which would have created problems after the first few years. But there is another side to a cooperative agreement. If USAID is not sure of the specifics of how a program will evolve and how it will operate, there is great potential for disagreements. In this case, both USAID and ICLEI had a shared objective and a shared approach but were not sure how the process would work itself out. USAID was willing to let ICLEI use its CCP program without trying to micromanage it. The cooperative agreement was the right instrument for a pilot program introducing a new approach to deal with GCC issues. 2. CITIES FOR CLIMATE PROTECTION (CCP) Among USAID/Washington staff, USAID field staff and local municipal participants, there is universal agreement that CCP is an excellent program that generates good benefits. All agree that the five CCP milestones are a logical approach that provides a clear roadmap on how to proceed. What is interesting is why CCP works so effectively. The first step in each country is to build awareness of the problems and what can be done at the local level. While ICLEI staff always speak about global climate change, they know that municipal leaders probably think that the national government or international organizations should take care of the problem. While mentioning how local governments can make a contribution to solving climate change, ICLEI wisely promotes the CCP program as way for a municipal government to reap financial savings while improving the health and livability of the city. The program offers low cost investments that generate a quick payback through energy savings. Efforts to reduce greenhouse gas emissions also reduced ground level pollution which is a health hazard. Cost savings and reducing pollution appeal to municipal politicians and Program Evaluation of the Cooperative Agreement 12 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) municipal engineers. It is a win-win approach of saving money and reducing GHG emissions, and an excellent way to sell the program. The next step is making sure the municipality is serious and committed. If a donor comes to a country and offers free resources, everyone wants to participate and receive free training and equipment. The real problem is finding those who are committed and willing to put in their own time, effort and money to make it succeed. ICLEI devotes much of its time in reviewing the capabilities and commitment of a large number of cities before making its selection. Putting in that effort up front helped assure program success. The second CCP milestone is a good test of local government commitment. It requires the municipality to set an emissions reduction target along with a timetable for achievement. This means that the mayor, budget staff and engineers all have to agree on what needs to be done. In most cases the government has to pass a municipal ordinance or resolution committing the government to undertake the action plan. Cities face common problems but many feel isolated with little idea of what other cities are doing. Building a network of cities to share knowledge is an important part of the CCP program and ICLEI has supported in-country networks in all five countries. A city is connected with other cities along with linkages to local and international organizations. ICLEI organizes meetings and web sites that improve the interchange of technical information and lessons from successful projects. The in-country networks help cities learn from each other about how to deal with technical problems, local political issues and how to attract donor, government and private funding. This provides an opportunity for a city to look beyond its own experiences, to learn from other cities and most important it creates a psychological boost that the city is not facing problems alone. There were other benefits and concerns noted by the interviewees: • The twining of cities is an excellent way to help cities share experiences. • The 5 CCP milestones did not have to be sequential. It is possible to start on implementation early in the process. • The emissions targets put the effort on a sound technical basis and help insulate it from politics. • The technology and maintenance are simple and straight-forward. • The program can shift to meet the needs of different cities. In some cases it is energy saving lights and pumps. In others it is waste management, traffic control and tree planting. • In each country there is usually an over-subscription of municipalities wanting to be part of the CCP program. It is popular. • A number of municipal governments were eager to join the international ICLEI organization and paid the $1,000 annual membership fee. • When setting targets for GHG reductions, some local governments are reluctant to enter a national political area handled by the central government. This is an issue in India and the Philippines. Program Evaluation of the Cooperative Agreement 13 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) • Since the process is heavy on up-front work it can take a long time to reach the last CCP milestone (evaluation and verification of results). That makes it difficult to know whether a program is actually delivering expected benefits. • To expand beyond the pilot stage, cities need more money than they can get from local taxes. It would be useful if the project could work to link cities with commercial funding sources. The CCP program has a logical and holistic approach. It provides the right blend of political consensus building, actions that generated both local and global benefits and cost savings that made the whole effort possible. 3. USAID OPERATIONAL ISSUES An issue that emerged repeatedly was the question of continuity or how to operate long-term projects when many of the participants have a short time horizon. At the municipal level, when the mayor or commissioner changes, policies often change too. With a change in senior USAID staff, there is often a change in priorities. There are also changes in budget, policy and politics that can create challenges. For example, USAID/India and USAID/ Indonesia were committed to funding a multi-year activity but there was a reversal in the second year. USAID/South Africa is changing its management approach to reduce the number of contracts and to combine activities. It is not clear how small activities like CCP will fit under the new approach. CCP was a good fit with the USAID/India strategy which emphasized improvements in the urban environment and improved energy efficiency. That was the reason USAID/ India decided to provide its own funds for phase 2, to expand the program beyond what had been done under USAID/Washington funding. However, as phase 2 was heading into its second year, USAID/India decided to cut its funding in half. USAID/Indonesia had agreed to use Mission funds to expand its program and provided $200,000 in the first year (FY 2003) and pledged to provide $200,000 in 2004, the second year. Before the second year, USAID/Indonesia had a new Strategic Plan and climate change was not in the plan. As a result, USAID/Indonesia did not provide its $200,000 for the second year. This created a problem for ICLEI’s budget and personnel planning. In the end, USAID/Washington helped fill part of the gap, but the program was disrupted in Indonesia and it may have sent a negative signal to the Government of Indonesia. In the same year India cut back on its pledge. Having two unexpected reductions at the same time was a double blow. To avoid such problems in the future, it might make sense for a country mission that wants a three year program to pay at least two year’s funding up-front as its buy-in to the Washington cooperative agreement. An alternative would be for a USAID Mission to have its own forward-funded cooperative agreement. USAID/South Africa has its own $2 million cooperative agreement with ICLEI and is very satisfied with the CCP program. But there are management issues within the USAID Mission. USAID/South Africa, like many other USAID Missions, is trying to reduce the number of projects and mechanisms it has to manage. It takes almost as much management effort for a $10 million contract as it does for a $1 million contract --- rather than 20 contracts 5 should be easier to manage. The USAID Mission is considering bundling similar programs together under a Program Evaluation of the Cooperative Agreement 14 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) single contract or creating umbrella contracts to cover a number of different activities. It is not clear if future CCP work will fit neatly under another activity, under an umbrella contract or if it will terminate. There are also problems at the local level. In India, there is agreement that technical issues are not a problem. It is usually political. An enthusiastic commissioner would often be the force behind a CCP program but, when political leaders changed, old programs suffered. It is similar in Mexico, where the municipal administration changes almost every three years. At the beginning of each new period the administration reevaluates the policies and programs of the previous administration. Some end and some are completely changed. There were changes at USAID/Washington that included a least four different project managers (or CTOs) and an organizational shift that moved the cooperative agreement from the Urban Programs Team to the Energy Team to the Global Climate Change Team within EGAT. In addition, the program started in 1998 under a Democratic Administration and in 2001 a Republican Administration took over. When discussing these changes with ICLEI and USAID field staff they all said they were not affected by the changes. They received good services from USAID/Washington. Day-to-day program operations work smoothly. The question of how a long-term program can be managed when USAID Missions and local city leaders have a short time horizon is difficult to resolve. 4. SUSTAINABILITY AND REPLICATION Sustainability is an on-going problem throughout the Agency in all sectors. The critical factors are whether: an activity uses an appropriate technology; local staff can maintain the equipment; management and technical skills are effective; local government supports the activity; benefits are clear and substantial; and financial costs and benefits are favorable. Each of the five countries has slightly different approaches and different strengths and problems. It is still possible to draw general findings. The CCP program reduces GHG emissions by working from an “urban services” approach of encouraging actions like energy efficient lighting and water pumping while improving urban livability by reducing traffic congestion and improving waste management. The cities have seen how well CCP works by reducing local pollution and greenhouse gas emissions while saving money. In most cities the capacity to manage and maintain activities is strong and the cities are linked to other cities, which provide needed support. Since the benefits are clear and the financial incentive is strong, there is little likelihood that activities will end. There will always be questions of whether future local government leaders will see a political benefit in supporting CCP and whether local governments will have the funds to expand program to new areas of their city. On balance it appears that cities have changed the way they operate and the CCP programs are an integral part of operations. Most city programs should be sustainable. Even if program activities are sustained in the original cities, there is the question of whether they will be replicated in other cities. If 10 or even 20 pilot activities are launched in a country, there is no assurance that CCP will ever affect several thousand cities. That is the question of Program Evaluation of the Cooperative Agreement 15 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) whether CCP is replicable on a national basis. It requires mainstreaming the program to include private sector finance and project management and linking those working on the technical issues with city budget offices, since they are the ones that must be convinced that any activity should be funded. There is also a need to involve the National Ministry of Environment since effective enforcement of environmental regulations gives the CCP program a boost. Efforts are needed to engage national governments and donors to recognize the strong benefits of the CCP pilot approach and to encourage them to “mainstream” CCP and to replicate it in more cities. 5. OTHER ISSUES There are a number of questions about CCP program operations and broader policy and program issues. Each issue is listed below, followed by the evaluation findings: A. Cities are usually concerned about local pollution such as Sulfur Oxides (SOX), Nitric Oxides (NOX) and air-born particulate matter. If that is the problem cities face, what has been the success in building awareness and action to deal with another problem--- the challenge of global climate change? While ICLEI talked to cities about the need to reduce greenhouse gas emissions, it did not try to sell its program based on that issue alone. It knew that cities might be concerned about GHG emissions, but they were not going to devote their money and time to deal with it. CCP was promoted as a way to save money on energy conservation and to improve the health and livability of a city through better waste management, methane recovery, reducing traffic congestion and tree plantings. That was clearly the best way to gain adoption of the program. In addition, CO2 emissions are an easy proxy measure of local pollutants such as SOX and NOX which accompany the production of CO2. B. LDC Government policies, particularly energy and pollution, can encourage or discourage energy conservation and global climate change measures. The ICLEI focus is on local governments and not national policies. Its view is that by building success at the local level, it will make an impression on the central government and the central government will adopt local approaches. In the U.S. and Europe that approach took over 20 years to work. It may take longer in developing countries. From USAID’s perspective, it makes sense to support ICLEI’s local-level programs as an alternative approach since the bulk of USAID’s efforts support policy reform programs aimed at national governments. But there is the question of whether USAID might be able to speed up the 20 year process it will take a national government to adopt the CCP approach on a country-wide basis. C. In some countries the central government subsidizes energy and in most countries pollution laws are not effectively enforced. That makes it more difficult to reduce GHG emissions. In Indonesia, energy receives large subsidies. India and other countries also provide subsidies. ICLEI works at the local level and does not deal with the issues of energy subsidies and Program Evaluation of the Cooperative Agreement 16 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) environmental regulations; they are usually the responsibility of the central government. These problems clearly harm the CCP program. CCP cannot cover all problems and in this case, in most countries they are handled by USAID and other donors with other project and policy reform programs. D. Effective institutions can bring together local government officials, equipment suppliers and technical experts such as engineers. With ICLEI support, several cities have sponsored trade fairs that gave equipment suppliers and service companies a chance to meet and talk with government officials and engineers. There have also been pollution fairs designed to educate government staff and the public about pollution and climate change issues. The cities are happy with the results and plan to do more fairs in the future. E. When local governments consider making an investment, the rate of return on that investment often determines whether they adopt new technology. Energy efficient street lighting, traffic signals, electric motors and pumps are the most popular with a financial payback period of 12 to 18 months for street lighting and signals and 18-24 months for electric motors and pumps. That means that an energy efficient street lighting investment of $100 will reduce electricity cost by $100 over a year or a year and a half. With a one year pay back, that is an astonishing 100 percent rate of return. Other investments such as methane capture, improved waste management, tree plantings and efforts to reduce traffic congestion have a longer payback period but several cities are making those investments as a way to clean up their city to make it more livable. In most cases those investments have a 2 to 5 year payback. F. The CCP program is more than just installing new equipment. Once a donor program ends, if maintenance and management are inadequate the project often fails. While there are problems in some countries, on balance the technology is relatively simple for energy efficient lighting, energy efficient motors and traffic control. The city can maintain the equipment or hire private sector firms to handle any problems. Methane capture and waste recovery are two areas where operations and maintenance have proved to be more complicated, though many cities are dealing with those issues. Program Evaluation of the Cooperative Agreement 17 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) C. SUMMARY CONCLUSIONS THE ICLEI APPROACH WORKED VERY WELL. When ICLEI was selected USAID picked a group that promoted CCP projects in the U.S., but had almost no CCP project experience in developing countries. There was a risk that the approach would not work in developing countries since skill levels, politics and capabilities differ greatly from U.S. cities. All participants agree that the technical approach is excellent and ICLEI has done a superb job of getting all stakeholders in each city to buy into the program. The claimed benefits are real and it is not difficult for cities to achieve the benefits. A COOPERATIVE AGREEMENT WAS AN APPROPRIATE DEVELOPMENT INSTRUMENT USAID bought into ICLEI’s program and implementation approach. The cooperative agreement is a perfect way to implement a project that works to educate people on new approaches, builds consensus, and then guides cities through the implementation process. It is hard to define those tasks and quantify expected results in a contract. CITIES FOR CLIMATE PROTECTION CCP offers low cost investments that generate quick payback through energy savings. It is a win-win approach of saving money and reducing greenhouse gas emissions and local pollution. The five milestones are a logical approach that provides a clear roadmap on how to proceed. USAID staff in Washington and the field along with host country participants give CCP high marks. USAID OPERATIONAL ISSUES USAID field staff is in the field because they like to get in close and manage the development process. Some say they micro-mange it. USAID field staff realized that they needed to put aside their usual hands-on approach and let ICLEI implement its program, as long as they were fully informed about current operations and future plans. There are two cases where USAID countries committed to funding multi-year programs, but failed to provide the promised funding in the second year. It was disruptive for ICLEI. USAID money is usually tight and priorities often shift. However, development is a multi-year effort and USAID should be able to plan its programs more than a year in advance. One way to encourage a multi-year commitment would be to require two years of forward-funding. Program Evaluation of the Cooperative Agreement 18 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) SUSTAINABILITY AND REPLICATION The program works well and equipment maintenance is not a problem. The cities like the idea of saving money while reducing pollution and greenhouse gases. The program will likely be sustained in most of the 64 cities. But will it be replicated in other cities? The ICLEI approach requires salesmanship and capacity building. Local leaders have to be educated and consensus must be created. Local engineers need to be trained in the new technology. It is a retail, labor intensive approach that works fine but in most countries it can only reach a small number of new cities each year. There are several hundred to several thousand cities in each developing country. A future program might consider a wholesale rather than a retail approach. It would mean working with national and provincial governments. It could include developing local capacity by training trainers. A large group of local nationals would then work with the cities on the CCP program. To help cities finance their investments, it would be useful to work with the private sector, government finance offices, government development banks and other donors. There is a broader issue that goes beyond this program. Over many years USAID has launched a number of pilot projects in almost every development sector. They demonstrate new technology and approaches which usually generate good benefits. However, all too many of the demonstration programs just end. They are not replicated or scaled-up. In the future, when USAID designs a pilot program it should address that issue. Program Evaluation of the Cooperative Agreement October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) ANNEXES Page No. A. Questions and issues examined in the evaluation........................................................... A-1 B. Country programs --- Philippines, Mexico, India, Indonesia, South Africa, Brazil .......B-1 C. Interviewees for this evaluation ......................................................................................C-1 D. Data collection interview protocols for USAID and ICLEI staff ................................... D-1 and for country participants. E. Evaluation Statement of Work.........................................................................................E-1 F. Evaluation work plan .......................................................................................................F-1 Program Evaluation of the Cooperative Agreement A-1 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) ANNEX A QUESTIONS AND ISSUES WHICH THE EVALUATION EXAMINED Key overall questions for the study: • Has the partnership successfully advanced the objectives of both USAID and ICLEI? What has worked best and what has not. Why? Any need for a change in the partnership? • Lessons-learned from the Cities for Climate Protection (CCP) implementation approach. What elements have been most successful; why? Are changes needed in the approach? • What has been learned about the types of activities that CCP has promoted (e.g., street lighting, water pumping, methane recovery). Which have been most successful and why? • Are cities able to go on by themselves? Is USAID needed? Is ICLEI needed? • Should new country and city programs be started? Why? Where? How? • Should there be a new cooperative agreement or contract between USAID and ICLEI? Sustainability requires sound financial, management, technology and, institutions. • Has the ICLEI CCP five-step program been appropriate and successful? Why? • Can cooperating cities do it on their own? Do they have the needed technology management, financial resources and institutional structure? Did they develop sustainable programs? How? • Can activities be picked-up by other donors? When and why? Examples so far? • What is the likelihood that USAID-supported country programs will continue into the future? Why? • What is the likelihood that cities (and countries) outside the program will adopt the approach (replication)? Monitoring and evaluation ICLEI has a 5-step Cities for Climate Protection (CCP) program: 1. An inventory of baseline greenhouse gas (GHG) emissions and forecast of emissions growth. 2. Establishment of an emission reduction goal. 3. Develop and put in place a local action plan. 4. Measure implementation. 5. Monitor and verify progress. ICLEI CCP is very strong on steps 1-3, but there is little documentation on measuring implementation and monitoring and verifying progress? What has been actually achieved? CCP is very good on process (training, planning, workshops, capacity building), but much less is said about results. What has been achieved? What are the expected pollution and climate change benefits? Have they been achieved? How do they compare to program costs of $2.2 million over a period of 7 years? If possible, identify successful examples and anecdotes that might be used to publicize program results. Program Evaluation of the Cooperative Agreement A-2 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) What can be said about the importance of the following issues? What has the partnership done to deal with them? • Cities are usually more concerned with local, immediate issues of air quality, rather than global climate change. What has been the success in building awareness and action to reduce the impacts of global climate change (GCC)? • Government policies, particularly energy and pollution policies that encourage or discourage energy conservation and global climate change measures. • Role of government regulations (environmental laws, licensing, fees and fines) in support of GCC efforts. • Are effective institutions available to bring together local government officials, equipment suppliers and technical experts (engineers)? • How do the cities view the financial costs and benefits of CCP activities? • How quickly must the investment payback be --- 1, 2, 3, 10 years? • The process is more than just installing new equipment. How are the cities doing with on￾going maintenance and management of the new activities? • Have cities identified new activities independently and without ICLEI support? USAID operational issues What are the benefits and drawbacks of a cooperative agreement vs. a traditional USAID contractor? What about performance-based contracts? What other implementation resources could be used---universities, contractors that specialize in pollution and climate change programs, research institutions, think tanks, LDC institutions, others? What are the benefits and drawbacks of a central project with Mission buy-ins (or task orders) vs. Missions issuing their own contracts or cooperative agreements? Program Evaluation of the Cooperative Agreement B-1 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) ANNEX B THE COUNTRY PROGRAMS The Philippines The program was launched in 1999 and now has 16 local government members with a combined population of 5 million people. Over 60 projects were implemented with a reduction of 40,000 tons of greenhouse gases and $7.5 million in annual cost savings for the local governments. Each country takes a slightly different approach to its CCP program. Most country programs emphasize energy saving measures. In the Philippines energy efficiency was part of the program, but many cities were more interested in improving the quality of city life through improved solid waste management, transport rationalization, air and water quality improvement and tree plantings (which also captured CO2 emissions). In an effort to reduce pollution from vehicles, Baguio City introduced a coding system to limit vehicles into the central district of the city, which sharply reduced air-borne particulate levels. For many cities, trash landfills were filling up and there were health concerns. Naga City introduced a successful trash reduction and materials recovery program. Naga and Baguio Cities were less successful with waste to energy programs. Efforts to reduce liquid wastes flowing into rivers were successful in Naga City. The CCP project was successful at getting local community commitment and participation. An interesting approach used by Naga City was the mobilization of volunteers from NGOs, social organizations, senior citizens and religious groups to report on pollution. For example, “Anti-Smoke Belching” volunteers reported violations but were also good at nagging and shaming polluters. There are plans for city and regional anti￾pollution festivals and trade fairs and even restocking of rivers with fish fingerlings. The central government Departments of Energy and the Department of Environment and Natural Resources supported city efforts. The variety of programs in the Philippines may be due in part to the work ICLEI did in linking cities together to learn from each other. City managers from other countries have also come to the Philippines to learn about new approaches, which is a source of pride to city mangers in the Philippines. Mexico The Mexico CCP program stared in 1999 and now has 9 participating cities. Mexico is a middle￾income country with a per capita GDP of $9,600, which means that its pollution problems and solutions differ from those of a low-income country. The CCP program proved to be well designed and practical in Mexico. Most successful were street lighting and traffic lights that have been switched to energy efficient systems. Traffic control is now being moved to computerized control which will speed up traffic flow and reduce the pollution that comes from cars and trucks stuck in traffic with their engines idling. The Mexican government worked with users, cities, the federal government, donors and manufacturers to improve the efficiency of electrical motors. High efficiency motors represented just 3 percent of sales but with new government regulation they went up to 50 percent and now Program Evaluation of the Cooperative Agreement B-2 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) are at 100 percent of sales. Mexico’s environmental regulations follow U.S. EPA criteria, so Mexican government regulations should have an impact on ground-level pollution from industry and transportation and GHG emissions. Unfortunately that has not been the case. Two reasons cited are strong economic growth and the lack of effective enforcement of environmental regulations. CCP helped by identifying targets and quantifying benefits. Mayors always make decisions based on political impact but they will adopt a good plan that delivers obvious benefits quickly. That is why street lighting and traffic light replacements, with a 12-18 month payback, were accepted and implemented. Municipal administrations seem to change every three years. At the beginning, a new administration reevaluates the policies and program of the previous administration. Some programs end and some programs are completely changed. As a result, continuity and longer-term efforts are difficult. All agree that ICLEI did a good job with its technical assistance, training programs and organizing interchanges among cities. But now, in the post-ICLEI period, it is difficult to get complete engagement between the municipalities. There are delays in sharing information and the meetings of city managers are too far apart. As one interviewee said: “Because of our culture, we need some kind of pressure to work better. When we had a dedicated ICLEI person that checked on progress every week and compared goals to actual progress and set up inter￾municipal meetings, the results were much better. Now everybody is in charge but nobody really is and nobody sees the project as a priority.” Another point was that the CCP approach was technically sound but it did not provide a good link to the private sector --- market information, equipment suppliers and financial sources. Those were not ICLEI responsibilities under the program but times change. The cities have the technical and managerial skills to run the programs but now, new technology and in particular, finance is the critical problem if CCP is to be scaled-up throughout the country. India The CCP Program in India (phase 1) covered 2001 and 2002 and was implemented in 7 cities. Phase 2 covered 2003 to 2005 and brought the total up to 16 cities with another 2 cities ready to join the program, for a total of 18 cities. It is important to put this in context because in India, with a population of just over one billion, everything is on a much bigger scale than in other countries. The 16 cities have a population of 23 million, which is bigger than the total population of many developing countries. The potential impact on greenhouse gas emissions is also great since there are 5,000 Indian municipalities that could benefit from the lessons of the 16 CCP programs. CCP program activities included workshops, training, equipment, energy and water pump efficiency audits as well as technical assistance to support a wide range of projects, including: energy efficient street lighting, power saving electrical timers, high efficiency electric motors and pumps for municipal water supply, efforts to reduce transport pollution, and, roof-top solar collectors for hot water heating. Tree planting and greenery programs improved the livability of the cities while reducing CO2 emissions. Transport pollution is reduced by stiffer regulation efforts. Jabalpur municipality contracted out its street lighting operations and maintenance to a Program Evaluation of the Cooperative Agreement B-3 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) private company in an innovative test to see if the private sector could do a better job at lower cost than the municipality could do on its own. Ten projects had an energy efficiency focus and saved 4 million KWH and reduced carbon emissions by 4,000 tons a year. The projects leveraged domestic investment of $3 million and $1 million of non-USAID foreign assistance.. The technology is not complicated on CCP projects and local technical skills can maintain the equipment. The investment payback from energy savings is 12-18 months, 24 months for water pumping and 36 months for power timers. Since street lighting has been so successful, it will be extended to new areas by the cities if they can find the funds. In Gunter Municipality, the amount of water pumped was doubled while electricity costs were reduced. Some cities were also able to improve water quality which had an important health effect. The solar collectors will have a longer payback and the likelihood of replication is less clear. Rather than technical issues, political support has been a problem. A mayor may be a major supporter of the program, but if a new mayor is elected, he/she often has new and different priorities and continuity of program approach and funding can be a problem. Efforts by cities to have state pollution control boards regulate more aggressively have been disappointing and the central government has been a minor player. The cities had high praise for ICLEI’s technical assistance, the CCP program approach and support for development of a network among municipalities to share experiences. The CCP program fit well with the USAID Mission strategy, which emphasizes improvements in the urban environment and improved energy efficiency. That was the reason USAID/India decided to provide its own funds for phase 2, to expand the program beyond what had been done under USAID/Washington funding. However, as phase 2 was heading into its second year, the USAID Mission decided to cut its funding in half. This made it hard to maintain momentum and to complete all of the planned activities. With the program coming to an end in 2005, some in the USAID Mission are examining the possibility of a Phase 3 program. The Mission is interested replicating and spreading the successful CCP programs to more municipalities. It is also considering better integration of CCP with other Mission environmental and finance projects to make CCP more effective. It should also be possible to link CCP findings and lessons with USAID national level environmental programs and policy issues. Indonesia The program began in 2001 with 5 cities and now has 10 with a total population of more than 6.5 million. Twenty-six projects have been initiated and another 20 are in the pipeline. Energy efficient street lighting was very successful and the projects made it to the last milestone, number 5. Over $4 million in domestic investment has been leveraged and 500,000 tons of CO2 avoided per year, mostly from new composting programs diverting waste from dumps. The annual savings are near $1 million. Future plans include waste-to-biogas projects. Program Evaluation of the Cooperative Agreement B-4 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) ICLEI promoted the CCP program in a highly effective manner. They identified mayors who were interested in environmental protection and tourism and were willing to adopt GHG reduction actions because of its linkage to local pollution and waste management. By working closely with local government staff, ICLEI was able to build support and buy-in. The CCP program was an excellent way to gain agreement from all stakeholders and then move through the CCP process to set clear goals and develop realistic action plan. Street lighting had a payback period of 1-2 years and improved waste management had a much longer payback period. Maintenance is a major weakness for public facilities and may be a threat to future sustainability. USAID/Indonesia had agreed to use Mission funds to expand the program and provided $200,000 in the first year (FY 2003) and pledged to provide $200,000 in 2004, the second year. Indonesia developed a new Strategic Plan, but climate change was not in the new plan. As a result, USAID/Indonesia did not provide its $200,000 for the second year. When USAID/Indonesia cut back on its pledge, it created a problem for ICLEI’s budget and personnel planning. USAID/Washington filled part of the gap but the program was disrupted. This made it hard to maintain momentum and to complete all of the planned activities. South Africa In 2001 South Africa negotiated its own Cooperative Agreement with ICLEI, separate from the USAID/Washington Cooperative Agreement. USAID/South Africa provided $1.6 million to develop CCP programs in 9 cities and total funding has since been increased to nearly $2 million to cover 11 cities. This is an evaluation of the USAID/ Washington cooperative agreement and it concentrates on the countries in that program. The South Africa CCP program is included since it adds additional information to the evaluation. From the Mission’s point of view, the Washington cooperative agreement was complimentary and supportive since it helped ICLEI develop and fine-tune its program overseas, which reduced ICLEI start-up and learning costs in South Africa. The USAID Mission’s program included efforts to reduce greenhouse gas emissions as well as programs to improve municipal service delivery; the CCP program was a good bridge to cover both of those development efforts. In addition, the Mission identified another problem. Most donors were using non-governmental organizations (NGOs) to implement their foreign assistance programs which meant that local municipalities were not being supported and their capabilities were not being developed. The CCP program helped improve local capacity to deliver improved municipal services while reducing greenhouse gas emissions. Global climate change and the need to reduce pollution were not fully understood at the city level in South Africa. Thus, it required an education and training effort so that municipal mayors, managers and technicians fully understood the problems, solutions and financial savings municipalities could achieve. For this effort, a cooperative agreement with an experienced NGO like ICLEI was the ideal way to proceed. A contract with an engineering firm would not have been as successful, since most contractors are only interested in implementing technical solutions. The key was building knowledge, building consensus, building local networks---and then thinking about implementing technical solutions. That is what ICLEI does so well. Program Evaluation of the Cooperative Agreement B-5 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) Energy efficient street lighting and energy efficient lighting in Johannesburg municipal buildings were the most successful programs. Methane conversion from sewage generated good benefits. Most of the programs are at stage 2 or 3 of the CCP five-stage process so actual benefits have not been measured. There are 210 municipalities and it is important to demonstrate that the technology works and saves money so that other municipalities can see the benefits and be convinced to adopt the CCP approach. After four years, CCP implementation is going well and all agree that the CCP approach is logical and works effectively. The cities were pleased with the program and the central government’s Department of Environmental Affairs and Tourism (DEAT) is a strong supporter. The programs concentrated mainly on energy conservation including the first “Green Building” in South Africa. The cities are interested in expanding efforts to new areas to improve air quality, capture methane gas from sewage, reduce volatile organic compound (VOC) emissions and convert vehicles to run on Liquefied Petroleum Gas (LPG). In addition to cleaning the local air, these efforts reduce greenhouse gas emissions. On the operations of the CCP program, the Mission had praise for ICLEI, but cited concern about the turn-over of local ICLEI staff. In addition, the Mission noted that local programs were run through ICLEI headquarters in California rather than in South Africa. The local ICLEI representative would send his communications to California and then headquarters in California would contact the USAID Mission. The Mission viewed this as an awkward and inefficient way to run an operation. Since this was a Mission funded program, the Mission played a more active role than did Missions in countries that used the centrally funded ICLEI cooperative agreement. Looking to the future, there are a number of strengths and several challenges. DEAT, an important player in the climate change issue in South Africa, is a supporter of the CCP program and has worked closely with the project. The South African National Air Quality Management Act requires cities to reduce pollution, which puts pressure on the cities to take action. CCP efforts to improve information flows among the cities and to spread lessons-learned have been successful. For instance, there is a South Africa network of cities which will help to ensure that this process continue in the future. But future sustainability is not assured. In many countries, cities lack the needed funds and that is the main threat to sustainability. But in the view of a USAID manager, in South Africa funds are available at the municipal level but skilled manpower is limited. Skilled manpower exists at the national government level and in the private sector. There is a need to develop technical and managerial capacity at the municipal level if CCP programs are to continue and spread to other municipalities. An ICLEI manager did not agree with the USAID manager and stated that: “The folks I have worked with in South African local governments are at least as skilled as any I’ve worked with in the national government or private sector.” On the USAID side, there are questions about future CCP support. The South Africa Mission, like many other USAID Missions, is trying to reduce the number of projects and financial mechanisms it has to manage, since it takes almost as much management effort for a $10 million project as it does for a $1 million project. As the number of agreements/contracts is reduced, it is not clear if future CCP work will fit neatly under another activity or under an umbrella contract for service delivery. There is a strong possibility that CCP funding will end in South Africa. Program Evaluation of the Cooperative Agreement B-6 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) Brazil The USAID/Brazil program promotes renewable energy and energy efficiency technologies to mitigate global climate change and to expand the socio-economic benefits of clean, reliable energy. The cooperative agreement provided training workshops to build awareness and offered Brazilian institutions a chance to test an innovative approach to saving energy while reducing greenhouse gas emissions---the Harmonized Emissions Analysis Tool (HEAT). HEAT is a CCP software solution that supports greenhouse gas and air pollution emission reduction programs. It provides a global database of emissions and action plans covering a broad array of sectors such as energy, waste and transportation. The cooperative agreement took the existing software that is being used in other CCP countries and created a Brazilian software version. This software will help local governments and NGOs in Brazil consider the costs, benefits and emission reductions of different approaches. Program Evaluation of the Cooperative Agreement C-1 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) ANNEX C INTERVIEWEES FOR THIS EVALUATION USAID/Washington Current and past CTOs of the USAID/ICLEI Cooperative Agreement Duane Muller Senior Climate Change Specialist USAID/EGAT/ESP/GCC Simone Lawaetz Urban Energy Program USAID/EGAT/I&E Robert MacLeod USAID/ANE ICLEI Staff- California Bob Price Former International Director CCP Campaign ICLEI-USA Jim Yienger Program Manger ICLEI-USA Michelle Wyman Executive Director ICLEI-USA India Mr. Subodh Shah Executive Engineer (Electrical), Street Lighting Department Vadodara Municipal Corporation Gujarat, India Mr. Aniruddhe Mukerjee, IAS Ex Commissioner, Jabalpur Madhya Pradesh, India Mr. Chukka Yesuratnam Ex Mayor, Guntur Municipal Corporation Andhra Pradesh, India Program Evaluation of the Cooperative Agreement C-2 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) USAID/India Rebecca Black Program Office USAID, New Delhi John Smith-Sreen Environment Officer Office of Economic Growth USAID, New Delhi Mr. Nabaroon Bhattacharjee Environmental Officer Office of Economic Growth USAID, New Delhi Indonesia USAID/ Indonesia Suzanne Billharz Senior Environmental Officer Office of Economic Growth USAID, Jakarta Edi Setianto Environmental Officer Office of Economic Growth USAID, Jakarta Mexico Arq. Luis Armando Martinez Diaz Head of the Greenhouse Gas Reduction Department Toluca Municipal Government Toluca, Mexico Ing. Carlos Muller Colson Director of Public Lighting Santiago de Queretaro, Qro. Mexico USAID/Mexico Jorge Landa Clean Production and Energy Advisor USAID, Mexico City Program Evaluation of the Cooperative Agreement C-3 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) Philippines Erlinda Bailey, Chief, Socio-Cultural Management Division Environment and Natural Resources Office Naga City Government Naga City, Philippines Mr. Victorino Aquitania, Project Development Officer Office of the City Planning and Development Coordinator City Government of Baguio Baguio City, Philippines USAID/Philippines Daniel Moore Chief, Office of Energy and Environment USAID, Manila Rosario (Chato) Calderon Senior Technical Advisor, Energy and Clean Air Office of Energy and Environment USAID, Manila South Africa USAID/ South Africa Melissa Knight, Team Leader Local Government Services Team USAID, Pretoria Nkosiphambili Ndlovu Senior Environmental Advisor Local Government Services Team USAID, Pretoria BRAZIL USAID/Brazil Alexandre Mancuso Senior Energy Advisor USAID, Brasilia Program Evaluation of the Cooperative Agreement D-1 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) ANNEX D DATA COLLECTION INTERVIEW PROTOCOLS FOR USAID AND ICLEI STAFF AND FOR COUNTRY PARTICIPANTS D-1 Interview Protocol For USAID and ICLEI Staff 1. Introduction Who we are, what we are doing, what was your role I have been asked to do an evaluation of the Cooperative Agreement between USAID and ICLEI (the International Council for Local Environmental Initiatives) which is currently managed by USAID’s Global Climate Change Team. We want to learn how the co-op agreement worked, a bit about the global climate change activities it supported and what might be done in future programs. I have just a few questions that will take about 20 minutes of your time. But maybe first you could tell me a bit about your role with the activity, what years it covered and any general observations you have. What were the most interesting things about the coop agreement? How do you think it worked? Were there any important questions or problems? 2. The partnership A cooperative agreement differs from a USAID contract. Rather than buying specific products, it is bringing together USAID and ICLEI since they both share common goals and want to work together. But they are different organizations: there can never be a complete match of interests. • How well did the partnership advance the objectives of USAID and ICLEI? • Where were there differences? Any major differences? • What worked best and what has not? Why? 3. Cities for Climate Protection (CCP) Let’s talk about how the program works. Are you familiar with the ICLEI 5-step, Cities for Climate Protection (CCP) program? Let me give you a short summary: • An inventory of baseline greenhouse gas (GHG) emissions and a forecast of emissions growth. • Establishment of an emission reduction goal. • Developing and putting in place a local action plan. • Measuring implementation. • Monitoring and verifying progress. Program Evaluation of the Cooperative Agreement D-2 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) • What do you think of the Cities for Climate Protection (CCP) 5-step program? • What are the lessons from the (CCP) implementation approach? What elements have been most successful; why? Are any changes needed in the approach? • CCP has promoted a wide range of activities (e.g., street lighting, water pumping, methane recovery). Which activities have been most successful and why? 4. Monitoring and evaluation The ICLEI 5-step, Cities for Climate Protection (CCP) program is very strong on the first three steps and reports are available on those items. What can be said about the last two steps, measuring implementation and monitoring and verifying results? Is there good data on the reduction of greenhouse gas emissions? 5. The future • Should new country and city programs be started? Why? Where? How? • Some programs have been very successful. Are some cities able to go on by themselves? Is USAID help needed? Is ICLEI help needed? • Should the USAID/ICLEI relationship continue and under what arrangements? • In a future agreement, what things might be changed? • Would it make sense to use a cooperative agreement or another mechanism? 6. USAID operational issues What are the benefits and drawbacks of a cooperative agreement vs. a traditional USAID contract? What other implementation resources could be used---universities, contractors that specialize in pollution and climate change programs, research institutions, think tanks, LDC institutions, others? What are the benefits and drawbacks of a central project with Mission buy-ins or IQC task orders vs. Missions issuing their own contracts or cooperative agreements? How have things worked with changes in the USAID offices managing the coop agreement and changes in the CTO? Are there any problems with financial payments? 7. There are a number of broader issues Let me run through a list of questions and see if you have ideas that could help us. • Cities are usually concerned about local pollution (SOX, NOX and particulates). What has been the success in building awareness and action to reduce the impacts of global climate change? • What about Government policies, particularly energy and pollution policies that encourage or discourage energy conservation and global climate change measures. • Then there is the role of government regulations (environmental laws, licensing, fees and fines) that could support GCC efforts. What impact have the had? Program Evaluation of the Cooperative Agreement D-3 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) • Are effective institutions available to bring together local government officials, equipment suppliers and technical experts such as engineers? • How do the cities view the financial costs and benefits of CCP activities? • How quickly must the investment payback be --- 1, 2, 3, 10 years? • The process is more than just installing new equipment. How are the cities doing with on￾going maintenance and management of the new activities? • Have cities identified new activities independently and without ICLEI support? 8. Sustainability All too often once donor support ends, program benefits end. Will benefits be maintained in this case? • What is the likelihood that the programs started in the four USAID-supported countries will continue into the future? Why? Is support needed from aid donors? • Can cooperating cities do it on their own? Do they have the needed technology management, financial resources and institutional structure? Did they develop sustainable programs? How? • What is needed for other donors to support CCP activities? When and why? Examples so far? • Then there is replication. What is the likelihood that cities (and countries) outside the program will adopt the CCP approaches? Program Evaluation of the Cooperative Agreement D-4 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) D-2 Interview Protocol For those who Participated at the Local Level 1. Introduction I have been asked to do an evaluation of the cooperative agreement between the U.S. Agency for International Development (USAID) and ICLEI (the International Council for Local Environmental Initiatives). The program worked with cities in your country to reduce pollution and greenhouse gas emissions. Are you familiar with it? We want to learn how the Cities for Climate Protection (CCP) program worked in your country. We want to learn from the successes in your country so that we can do a better job with future programs. I have just a few questions that will take about 20 minutes of your time. But maybe first you could tell me a bit about your role with the activity, what you did, what years it covered and any general observations you have. What were the most interesting things about the program? How well do you think it worked? What were the important successes or problems? 2. Cities for Climate Protection (CCP) Let’s talk about how the program works. Are you familiar with the ICLEI 5-step, Cities for Climate Protection (CCP) program? Let me give you a short summary: • An inventory of baseline greenhouse gas (GHG) emissions and a forecast of emissions growth. • Establishment of an emission reduction goal. • Developing and putting in place a local action plan. • Measuring implementation. • Monitoring and verifying progress. • What do you think of the Cities for Climate Protection (CCP) 5-step program? • CCP has promoted a wide range of activities (e.g., street lighting, water pumping, methane recovery). Which activities have been most successful and why? • What are the lessons from the (CCP) implementation approach? What elements have been most successful and why? Are any changes needed to make it better? 3. The future We would like your ideas on what should be done in future programs. • Should new country and city programs be started? Why? Where? How? • Some programs have been very successful. Are some cities able to go on by themselves? When is donor help needed? Program Evaluation of the Cooperative Agreement D-5 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) 4. Monitoring and evaluation Remember when I mentioned the ICLEI 5-step Cities for Climate Protection (CCP) program earlier? The CCP is very strong on the first three steps and reports are available on those items: • A greenhouse gas emission inventory and forecast of emissions growth • Establishment of an emission reduction goal. • Develop and put in place a local action plan. • What can be said about the last two steps, measuring implementation and monitoring and verifying results? Is there good data on the reduction of greenhouse gas emissions? 5. There are also some broader issues Let me run through the list and see if you have any ideas that could help us. A. Cities are usually concerned about local pollution (SOX, NOX, particulates). What has been the success in building awareness and action to reduce the impacts of global climate change? B. What about Government policies, particularly energy and pollution policies that encourage or discourage energy conservation and global climate change measures. C. Then there is the role of government regulations (environmental laws, licensing, fees and fines) that could support GCC efforts. What impact have they had? D. Are effective institutions available to bring together local government officials, equipment suppliers and technical experts such as engineers? E. How do the cities view the financial costs and benefits of CCP activities? F. How quickly must the investment payback be --- 1, 2, 3, 10 years? G. The process is more than just installing new equipment. How are the cities doing with on-going maintenance and management of the new activities? H. Have cities identified new activities independently and without ICLEI support? 6. Sustainability. All too often once donor support ends, program benefits end. Will benefits be maintained in this case? • What is the likelihood that the programs started in your city will continue into the future? Why? Will it need support from aid donors? • Do you have the technology management, financial resources and institutional structure that you need for a sustainable program? • Would it help if donors supported your work? When and why? Have other donors helped out? • Then there is replication. What is the likelihood that cities (and countries) outside the program will adopt the CCP approaches from your city? Program Evaluation of the Cooperative Agreement E-1 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) ANNEX E EVALUATION STATEMENT OF WORK Evaluation of the Cooperative Agreement between USAID/EGAT, Office of Environment and Science Policy, Global Climate Change Team and the International Council for Local Environmental Initiatives (ICLEI) The U.S. Agency for International Development (USAID) requests the services of a qualified consultant to evaluate a Cooperative Agreement between USAID/EGAT, Office of Environment and Science Policy, Global Climate Change (GCC) Team and the International Council for Local Environmental Initiatives (ICLEI), #LAG-A-00-99-00001. BACKGROUND A Cooperative Agreement between USAID and ICLEI provides a partnership and financing mechanism through which ICLEI’s participating cities and USAID Missions and bureaus can fund and implement climate related activities. Addressing the causes and effects of climate change has been a key focus of USAID’s development assistance for over a decade. USAID has funded environmental programs that have reduced greenhouse gas emissions while promoting energy efficiency, forest protection, biodiversity conservation, and other development goals. This “multiple benefits" approach to climate change helps developing and transition countries achieve economic development without sacrificing environmental protection. To help countries address domestic and international climate change priorities, USAID’s GCC Program is active in more than 40 countries and dedicates about $180 million per year to promote: • Clean energy technology • Sustainable land use and forestry • Adaptation to climate change • Climate science for decision-making USAID places particular emphasis on partnerships with the private sector and on working with local and national authorities, communities, and nongovernmental organizations to create alliances that build on the relative strengths of each. Bringing together a diverse range of stakeholders helps avoid unnecessary duplication and lays the foundation for a sustained, integrated approach. Through training, tools, and other means of capacity building, USAID helps developing and transition countries address climate-related concerns as a part of their development goals. USAID’s GCC Program is a Presidential Initiative. As such, the Program reports to Congress, the White House, U.S. government agencies, and the public annually on the climate-related activities of USAID. ICLEI is a membership association of local governments and national and regional local government associations that have made a unique commitment to sustainable development. Program Evaluation of the Cooperative Agreement E-2 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) ICLEI's Cities for Climate Protection™ (CCP) Campaign is ICLEI’s performance￾oriented campaign that offers a framework for local governments to develop a strategic global agenda to reduce the emissions of greenhouse gases (GHGs) from urban areas and improve the local air quality and urban livability. The CCP campaign,works with local governments to improve city management and address economic, environmental and quality of life concerns. These actions simultaneously reduce or avoid air pollution, including GHG emissions. The main aim of ICLEI is to build the capacity of local government to support local-level environmental protection and sustainable development initiatives. From its launch in 1993, the Campaign has grown to include over 600 local governments from more than 25 countries today. The original Cooperative Agreement with ICLEI, initiated on October 1, 1998, seeks to build the capacity of local governments to achieve measurable reductions in greenhouse gas emissions, improve air quality, and enhance urban livability. The program links global climate change with such immediate municipal concerns as air quality, transport, and cost of energy and through that link motivates local government emission reduction actions. The Cooperative Agreement enlists ICLEI expertise to provide technical assistance, training and tools that focus on reducing emissions through energy efficiency, renewable energy, waste management, land use planning, and transportation improvements in developing countries—all which support the goals and objectives of USAID’s Climate Change Program. Specific objectives of the Cooperative Agreement between USAID/EGAT/ESP/GCC and ICLEI include: (1) Provide technical assistance, training and capacity building to select CCP partners (2) Develop and/or disseminate innovative approaches and test pilot approaches (e.g., USAID/Brazil) (3) Provide an efficient, cost-effective means for USAID/EGAT, Regional Bureaus and Missions to obtain assistance from the ICLEI (e.g., USAID/India and USAID/Indonesia) (3) Leverage additional Mission collaboration (e.g., USAID/South Africa) (4) Provide accurate and compelling results reporting to USAID/EGAT, Bureaus and Missions, Congress and other funders of joint activities. (5) Develop and maintain effective partnerships with external organizations and individuals to accomplish the above objectives. To date, ICLEI has helped USAID in meeting the objectives of its Global Climate Change Program by working with municipal governments to undertake the five-milestone process of ICLEI’s CCP campaign, in countries such as Mexico, the Philippines, India, Indonesia and South Africa. A strength of ICLEI's CCP campaign is that it involves all the stakeholders to address the cause. More than 600 local governments worldwide currently participate in the CCP. USAID, through its cooperative agreement with ICLEI, currently supports the participation of 45 of these local governments. Most of these cities have completed inventories of local greenhouse gas emissions and are identifying measures that, when implemented, will reduce the carbon Program Evaluation of the Cooperative Agreement E-3 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) dioxide, methane and criteria air pollutants currently emitted from local buildings, water supply systems, landfills, transportation infrastructure, and street-lighting. Through early implementation, more than a dozen of these local authorities are already realizing a variety of benefits associated with their efforts to mitigate climate change with local action. To promote broader implementation of these and other initiatives at the grassroots level, local policy makers benefit from having tools designed to convey the environmental benefits of projects based upon earlier lessons learned. With its network of CCP participant cities, ICLEI is in a position to mobilize hundreds of local governments around the world to undertake assessments, implement strategies, and utilize tools that reduce air pollution and greenhouse gas emissions. Furthermore, since 1998, ICLEI and its participating cities have met their cost￾sharing goals, based on the provisions contained in the original cooperative agreement, primarily through in-kind contributions, but also through direct contributions. Cities that participate in CCP also have access to a software tool to help local government inventory greenhouse gases and standard air pollutant emissions and evaluate policies and measures to reduce those emissions. With support from ICLEI, the local governments as a group have made progress in completing baseline inventories of their greenhouse gas emissions, identifying measures to reduce/avoid emissions, and implementing the identified measures. PURPOSE The Agreement described above is now its sixth year of existence. USAID’s Climate Change team inherited this cooperatives agreement in 2003 from the energy team based in GAT/I&E. While various USAID Missions had bought into this mechanism (India and Indonesia), the energy office decided it was not a priority for funding or staffing, and thus transferred the agreement to EGAT/ESP/GCC in 2003. During the FY04 EGAT Portfolio Review, USAID senior management within EGAT expressed a desire to evaluate the effectiveness of ICLEI in achieving program objectives. This is the first evaluation of the agreement since its initiation in 1998. The overarching objective of this evaluation is to: (1) determine if the USAID/ICLEI partnership has been successful in advancing the objectives of both USAID and the ICLEI; (2) identify lessons learned regarding what has worked, what has not, and what changes could be made to make the partnership more efficient; and (3) determine if the relationship between USAID and ICLEI should continue, and if so under what arrangements. SPECIFIC TASKS The consultant or contractor will work with both the ICLEI and USAID to conduct the evaluation. Tasks include, but are not limited to, the following: 1) Review of the relevant documents, including the original Agreement and the four amendments to answer basic questions about the Cooperative Agreement, • How and why intended results were or were not achieved • Examination of specific contributions of activities to the results Program Evaluation of the Cooperative Agreement E-4 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) • Assessment of other desired results not easily measured or quantified • Examination of sustainability of activities and their results 2) Review quarterly reports, tools, manuals, and other documents provided by ICLEI to USAID as part of the Cooperative Agreement, and assess the general characteristics of this portfolio, including areas of duplication, successes in increasing efficiencies by using products in more than on project/country (e.g., replicability), consistency in quality, range/methods of distribution, etc. 3) Conduct interviews by phone or e-mail with a broad cross-section of people involved with the administration of the program, and a representative sample of grantees, beneficiaries and recipients of USAID and ICLEI assistance. Questions should include those addressed during a typical program evaluation, in addition to questions specific to Cooperative Agreement objectives of both USAID and ICLEI. The consultant should work with USAID/EGAT/ESP/GCC and ICLEI to develop a set of specific questions and/or subject areas. Potential key people that could be interviewed include: • Duane Muller, USAID/W (current CTO) • Simone Lawaetz, USAID/W (previous CTO) • Bob MacLeod, USAID/W (previous CTO) • CCP cities/mayors (to be provided by ICLEI) • ICLEI staff o Bob Price o Jim Yienger 4) Conduct interviews by phone or e-mail with key USAID Missions to examine how selected programs followed the original Scope of Work, and what the results of these efforts were with respect to administrative efficiency, structure and performance. The consultant should attempt to interview relevant USAID project officers in India, Indonesia, South Africa, Brazil, Philippines and Mexico (see below). • N Bhattacharjee, USAID/India • Edi Setianto, USAID/Indonesia • Suzanne Bilharz, USAID/Indonesia • Melissa Knight, USAID/South Africa • Nkosiphambili Ndlovu, USAID/South Africa • Alexandre Mancuso, USAID/Brazil • Rosario (Chato) Calderon, USAID/Philippines • Daniel Moore, USAID/Philippines • Jorge Landa, USAID/Mexico 5) Conduct analysis of all the data and drafting of a preliminary report, including findings, conclusions and recommendations, including Lessons and recommendations for adjustments in programs and policies to improve results. Program Evaluation of the Cooperative Agreement E-5 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) 6) Present the results of evaluation findings and recommendations in a meeting organized by USAID in Washington, D.C. 7) Complete a final written report incorporating the comments from USAID reviewers. DELIVERABLES The consultant will produce the following documents and deliverables: 1) A summary of methodology and questions used for the evaluation; 2) A list of people and organizations interviewed; 3) A preliminary report for review by USAID, including findings, data analysis, conclusions and recommendations; 4) An oral presentation of evaluation findings and recommendations; and 5) A final report in English, incorporating comments from stakeholders, available in electronic (PDF or MS Word) format and in hardcopy. The main body of the report (excluding country, subject area, and interview lists, etc.) is not to exceed 15 typed pages, including a one-page executive summary of key findings and recommendations. All reports and data produced by the assessment shall become the property of USAID. WORK SCHEDULE It is estimated that the work will take 5 days for preparation, 10 days to carry out interviews and data analysis, and 7 days for final analysis, preliminary report writing, and final report writing, including incorporation of comments from USAID. Work will begin on or about August 1, 2005, the preliminary evaluation report will be delivered on or about October 1, 2005, and the final report will be delivered no later than October 31, 2005. A six-day work week is authorized if needed. LOGISTICAL SUPPORT 1) USAID/EGAT staff will make themselves available to the consultant for interviews and discussions needed to complete the assessment. 2) USAID/EGAT will review all materials, including drafts of the assessment report, in a timely manner. 3) USAID/EGAT and ICLEI will assist the consultant in making contact with individuals for interviews. 4) The consultant will report to Duane Muller, USAID/EGAT, as the principal liaison for the purposes of clarifying assessment objectives, schedule adjustments, and other issues as needed. Program Evaluation of the Cooperative Agreement F-1 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) ANNEX F EVALUATION WORK PLAN Phase I: Preparation Week One --- August 8-12, 2005 Meet with USAID CTO to discuss evaluation parameters and desired outcomes. CTO will provide relevant documents needed for the evaluation. The Evaluator will review the documents to identify key data and issues. The Evaluator will then start preparation of the following evaluation documents: • an evaluation work plan • a list of key evaluation questions • a draft evaluation outline • a template for setting up interviews • interview protocols for USAID, ICLEI and host country participants. Week Two --- August 15-19, 2005 Complete the data review. Complete the preparation of the work plan and other evaluation documents. The documents will be reviewed by the CTO on August 18 and any necessary changes will be completed. Phase II: Interviews and Data Analysis Week Three --- August 22-26, 2005 Final revisions will be made in evaluation documents. Research assistant will contact all participants and set up appointments for telephone interviews for week four. Those in the Washington, DC area will likely be interviewed in person, if available. Week Four --- August 29- September 2, 2005 The interviews begin this week. Start data analysis. Week Five ---September 5-9, 2005 The interviews continue, but since this is the Labor-day holiday it might not be possible to reach all of the interviewees. Continue data analysis. Week Six --- September 12-16, 2005 Interviews are completed and data analysis is completed. Start on preparation of the draft evaluation report. Program Evaluation of the Cooperative Agreement F-2 October 17, 2005 Between USAID/EGAT and the International Council for Local Environmental Initiatives (ICLEI) Phase III: Report Preparation and Submission Week Seven --- September 19-23, 2005 Draft report will be submitted to the USAID CTO. USAID CTO and the Evaluator will agree on a time and place for the Evaluator to present the results of the evaluation at an internal meeting organized by USAID in Washington, D.C. Subsequent Weeks --- September 26 to October 31 USAID CTO and the evaluator will vet the preliminary report with stakeholders, analyze the responses, and then synthesize them. Following receipt of USAID comments, the Evaluator will make necessary additions and amendments to the draft. The evaluator will provide a briefing on the evaluation finding at USAID. Development Associates will then send the final report to USAID in both electronic form and in the requisite number of hard copies. The evaluation is completed at this point. The CTO will send an electronic copy of the final report to CDIE so that it can be included in the Agency’s database. EGAT and ICLEI Report.doc/R113