USAID FROM THE AMERICAN PEOPLE y '- I I . I LII I1 II 3 II -LII Ill URBA'N ENERGY SERVICES u7- 11~11~11~~~~ * FA PROGRAM -- SUSTAINABLE 1': llli+l MUNICIPAL ENERGY SERVICES :k II I "WATERGY", PROGRA I 5 September 2005 This publication was produced for review by the United States Agency for International Development. It was prepared by Nexant, Inc. under Contract No. EPP-I-02-03-00007-00 URBAN ENERGY SERVICES PROGRAM -- SUSTAINABLE MUNICIPAL ENERGY SERVICES EVALUATION OF THE "WATERGY" PROGRAM IN BRAZIL September 2005 This publication was produced for review by the United States Agency for International Development. It was prepared by Nexant, Inc. under Contract No. EPP-1-02-03-00007-00 DISCLAIMER The author's views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Table of Contents EXECUTIVE SUMMARY ......................................................................................................................... 1 SECTION 1 INTRODUCTION AND OVERVIEW .............................................................................. 5 1.1 BACKGROUND AND OBJECTNES ................................................................................................................ 5 1.2 OVERVIEW OF W ATERGY ACTIVITIES INBRAZIL ........................................................................................ 5 1.3 WATER AND ENERGY SECTORS IN BRAZIL .................................................................................... 6 1.3.1 The Brazilian Water / Wastewater Sector ........................ .... ........................................................ 6 1.3.2 The Brazilian Energy Sector ............................................................................................................ 7 1.4 EVALUATION OF OBJECTIVES AND METHODOLOGY ................................................................... 8 1.4.1 Objectives .......... .......................................................................................................................... 8 1.4.2 Methodology ..................................................................................................................................... 9 SECTION 2 PROGRAM OBJECTIVES .............................................................................................. 10 2.1 ORTGTNAL WATERGY PROGRAM OBJECTIVES ............................................................................. 10 2.2 D~SCUSSIONF PROGRAM OBJECTIVES .................................................................................................... 11 2.3 PERFORMANCE MEASUREMENTS .............................................................................................................. 11 SECTION 3 PROGRAM RESULTS AND ACHIEVEMENTS .......................................................... 12 3.1 WATERGY TOOL KIT ................................................................................................................................ 12 3.2 THE ASSOCIATION OFBRAZILIAN WATER AND ENERGY (ABAE) ............................................................ 12 FIGURE 3.1 ABAE's ORGANIZATIONAL STRUCTURE .................................................................................................. 13 3.3 ASTUDYTOURTOTHEU.S ....................................................................................................................... 13 3.4 THE CAGECE PILOT PROJECT .................................................................................................................. 14 3.5 CAPACITY BUILDING EFFORTS .................................................................................................................. 15 SECTION 4 PROGRAM EVALUATION ............................................................................................ 17 INTRODUCT~ON ....................................................................................................................................................... 17 4.1 EFFECTIVENESS OF THE PROGRAM ............................................................................................................ 17 4.1.1 Capacity Building ........................................................................................................................ I7 4.1.2 Resource Mobilization .............................................................................................................. 18 4.1.3 Policy Reform ................................................................................................................................. 18 4.1.4 Information Management- Dissemination to Municipalities ........................................................ 19 4.1.5 Donor / Partner Dissemination ..................................................................................................... 19 4.2 EFFICIENCY OF THE PROGRAM .................................................................................................................. 19 4.3 PROGRAM MANAGEMENT AND OPERATIONS ............................................................................................. 20 4.4 SUSTAINABILITY OF THE OPERATION ........................................................................................................ 20 4.5 ENVIRONMENTAL AND SOCIAL IMPACTS ................................................................................................... 21 SECTION 5 CONCLUSIONS AND RECOMMENDATIONS ........................................................... 22 5.1 PROGRAM SUCCESS .................................................................................................................................. 22 5.2 PROGRAM CHALLENGES ............................................................................................................................ 22 5.3 STAKEHOLDER NEEDS ............................................................................................................................... 23 5.4 RECOMMENDATIONS ................................................................................................................................. 23 5.4.1 Recommendations for the Alliance's ongoing Efforts .................................................................. 24 5.4.2 Recommendations for the ABAE Efforts ................... ................................................................ 24 Appendix A List of Stakeholders Interviewed ............................................................................ 25 Appendix B Acronyms ................................................................................................................ 27 Executive Summary Background The Alliance to Save Energy (the Alliance) has been implementing the Watergy program for almost 6 years in many parts of the world to build local energy management capacity and promote the efficient use of energy and water in municipal water and wastewater systems. The Alliance has focused on capacity building, resource mobilization, policy reform, information management, and donorlpartner coordination. More recently, the Alliance is worlung on developing commercially viable models for financing and implementing Watergy projects. Under the Sustainable Municipal Energy Services (SMES) Task Order, Nexant was requested by USAIDIEGAT to provide an independent evaluation of Watergy in Brazil and offer recommendations to assist USAlD and the Alliance scale-up and commercialize the program. Brazilian Water and Energy Sectors It was necessary to provide some contextual review of the water and energy sectors as well as some key challenges that the Brazilian economy faced in recent years. This background helped increase the understanding of the overall environment within which the program evolved during the past 5 years, and the impact of the surrounding market conditions on its outcome. In summary, the water sector has been in a state of turmoil, the energy sector has undergone its most serious crisis since the 1970s, and the economy has weathered currency devaluation and a major political shft. The introduction section provides an elaboration on this context. Methodology The Nexant evaluation team included a US. consultant from Nexant (Mr. Emad Hassan) assisted by a Brazilian consulting firm with experience in the water and energy sectors (CGILA). The team followed a sequence of steps beginning with the collection of program background materials from USAID and the Alliance staff in Washington D.C. The team then developed a framework defining program evaluation objectives and identifying broad quantitative indicators as well as qualitative measures. In conjunction with such framework, discussion guides were prepared for the interviews with Alliance Washington D.C. representatives, Alliance project team in Brazil, and outside stakeholders. The team then collected the necessary data from the Alliance reports and from interviews with the ASE management team in D.C. and the relevant USAID staff. After identifying a list of target stakeholders, the Nexant team met in Brazil and was accompanied by Ms. Simone Lawaetz, the USAID representative directing the evaluation effort. During the period April 25 and May 3, the team conducted a series of interviews and meetings with 15 different organizations and stakeholders in Sao Paolo, Belo Horizonte, Brasilia, Fortaleza, and Rio de Janeiro (See Appendix A). These meetings covered the ASE team, public and private water and energy utilities, USAID staff, relevant NGOs, financial institutions, and other entities associated with the program activities. Phone interviews were also conducted and some organizations documented their comments on the Nexant-provided interview guide. Evaluation of the WATERGY Program in Brazil Under USAlD Contraqct EPP-I-02-03-00007-00 Page 1 of 27 0 Executive Surnmarv Program Results and Achievements As a result of these interviews and information gathering meetings, the evaluation team was able to capture the results and achievements of the program since its inception. While some of these results are quantitative such as energy savings, audits, etc, qualitative accomplishments have also been noted and documented. In the state of Ceara, in the northeast of Brazil, the Alliance's cooperation with Companhia de Agua e Esgoto do Estado do CearB (CAGECE) led to a successful project, demonstrating water and energy efficiency and achieving 88 gWh of electric energy savings. These savings led to additional 88,000 water connections to new residential customers without increasing the amount of water supply. The program also produced a Toolkit CD-ROM that provided information as well as a technical manual to interested parties. The manual was later translated into Portuguese and distributed with support from Programa Nacional De Conservacao De Energia Eletrica (PROCEL) . The Watergy program coordinated a study tour to the U.S. for a team of relevant utility representatives to promote energy generation of biogas from wastewater treatment. A series of capacity building and training events were held over the course of the program covering different topics and technical areas related to water and energy. Perhaps one of the key achievements of the Watergy program in Brazil was the creation of the Association of Brazilian Water and Energy (ABAE) as an NGO to promote water and energy efficiency. ABAE provides a promising vehicle to achieve sustainability for the Watergy activities in the Brazilian water and energy markets. A new executive director was hired in March 2005 and has already started to engage in dialogues with the market players. Summary of Findings & Recommendations As a result of the literature review, information gathering, and face-to-face as well as phone interviews with key stakeholders, the Nexant evaluation team arrived to thee following findings and recommendations: Program Success Mobilizing highly qualified technical resources to the water sector especially in automation and monitoring. Undertaking a successful capacity building and technical training activities in various states in Brazil. Weathering sector-wide changes in staff and political resistance to change by focusing on technical career bureaucrats within utilities. Creation and wide dissemination of an industry-needed Watergy Toolkit CD-ROM at all workshops, seminars and training programs. Demonstration of the significant potential of energy savings, environmental improvements, and social benefits through a pilot project jointly implemented with CAGECE. Evaluation of the WATERGY Program in Brazil Page 2 of 27 Under USAlD Contraqct EPP-I-02-03-00007-00 0 Executive Summary Establishment of 'Watergy' as an icon denoting energy and water efficiency within the sector. Program Challenges The initial program objectives were very broad, making it difficult to develop targets and indicators and monitor performance. The recent change in program direction towards commercialization added a significant challenge to the existing program structure and the skill set of the operating team. The Alliance's Brazil Team relied on their existing network of technical staff in the sector and did not actively involve other leading players, including those within associations, utilities, governments and financial institutions. Some 'multiplier' effect may have been missed by not more closely engaging them. Relationships with these groups will be important for ABAE to cultivate. ABAE has been created in a sector crowded with associations and other interest groups, making it more difficult to define a unique vision. More project development and financing skills are required to successfdly implement the current 'commercialization' activities. Some stakeholders could not clearly distinguish between the Alliance, SCAI', USAID, and ABAE, particularly given that SCAI, using different affiliations, is the main interface with the sector stakeholders. The Alliance has not developed a comprehensive database of activities, either implemented by themselves or others, and their achievements. Stakeholder expectations need to be better defined upfront and managed during project implementation. Recommendations * A clear exit strategy for the program needs to be developed; in which lessons learned from past efforts are transferred to ABAE and the expectations of program partners for future assistance are managed in line with the exit strategy. The next stage of the program effort requires a focus on the larger market instead of a limited number of utilities, and should seek to develop relationships with strategic players that can provide a multiplier effect. Commercialization efforts need to be carefully orchestrated, objectives need to be better outlined, and the necessary skills developed or "borrowed." Rather than just "financing a deal", activities could be expanded to include a more comprehensive partnership with Associa@o Brasileira das Empresas de Servicos de Concervacao de Energia (ABESCO) to foster pilot activities leveraging the capabilities of its members in project development and finance. It is important that ABAE develop its strategic vision as a market enabler, creating value by leveraging its knowledge as well as that of other players in a market-sensitive manner. The Association should be careful to set itself apart from technical consulting firms. I Consulting firm owned by the ASE consultants in Belo Horizonte. Evaluation of the WATERGY Program in Brazil Page 3 of 27 Under USAlD Contraqct EPP-I-02-03-00007-00 0 Executive Summarv ABAE is being created in a sector already over-populated with associations and trade representation entities. While ABAE's focus is unique, it must forge long-term and lasting relationships with market players such as PROCEL and AESBE to take advantage of program synergies as well as with "market-makers" such as the Caixa Economics Federal (CEF), The Brazilian Development Bank (BNDES) and the Ministry of Cities (Ministerio das Cidades). * The evaluation team considers ABAE's current business plan a good starting point; however, one that should have more market intimacy. The vision for ABAE should be based on an assessment of market needs. ABAE should strongly consider developing a survey instrument prior to finalizing the business plan using the experience of established experts in the sectors. The survey can bring the added benefits of generating awareness of ABAE and of its mission to promote the efficient use of water and energy resources. Evaluation of the WATERGY Program in Brazil Page 4 of 27 Under USAID Contraqct EPP-I-02-03-00007-00 Section 1 Introduction and Overview 1.1 BACKGROUND AND OBJECTIVES Municipal water utilities are struggling to meet their objective of improving the reliability and quality of water supply services to their constituents while maintaining financial viability. In Brazil, with the increasing global trends towards urbanization, this task becomes more complex and challenging. A key issue faced by water utilities is the high amount of energy needed for water supply and delivery. In many cases, water utilities spend over 50% of their operating budget for energy, and rationalization of energy tariffs is leading to increases in these costs. Studies and audits have pointed out the large inefficiencies in the management of energy and water resources by water utilities. The implementation of energy and water efficiency measures can contribute significantly to the reduction of operating costs, increasing financial viability, and enhancing water services. The Alliance to Save Energy has been working for more than six years with municipalities in various parts of the world to build local capacity and promote the efficient use of energy and water in municipal systems. The Alliance to Save Energy (ASE or the Alliance), in cooperation with the U.S. Agency for International Development (USAID) has developed the concept of Watergy to describe the linkage that exists between water and energy in the context of municipal water utilities. Energy plays a major part in conveying water to the end user as well as its role in potable water disinfection and wastewater treatment. When water is wasted in a municipal water system, energy is almost always squandered as well. The Watergy program has attempted to assist water utilities to cost-effectively provide the consumer with the desired services associated with water, while using the least amount of water and energy possible. "Watergy efficiency" encompasses the spectrum of water efficiency activities, energy efficiency activities, and resulting synergies from co-managing water and energy resources. Since its inception, the Watergy program has focused on capacity building, resource mobilization, policy reform, information management, and donorlpartner coordination. More recently, the Alliance is working on developing commercially viable models for financing and implementing Watergy projects. To monitor progress to date and to explore ways to scale-up Watergy efforts and increase implementation of energy and water efficiency projects, USAlD initiated this program evaluation of Watergy activities in Brazil. 1.2 OVERVIEW OF WATERGY ACTIVITIES IN BRAZIL Watergy activities in Brazil have focused on: r Capacity Building: Assisting municipal water utilities either directly or through trade associations, government institutions, and other stakeholders to develop the capacity to address energy and water efficiency opportunities. Resource Mobilization: Assisting in mobilization of funds and technical resources from the international donor community, government institutions, NGOs, trade associations and from the private sector to assist in capacity building and project implementation. Evaluation of the WATERGY Program in Brazil Page 5 of 27 Under USAlD Contraact EPP-I-02-03-00007-00 Section 1 Introduction and Overview Policy Reforms: Recommending policies and regulations that promote the efficient use of water and energy resources by both utilities and consumers. Information Management: Serving as a clearinghouse of information on improving municipal water utility efficiency including case studies, videos of efficiency techniques, software programs, technical overviews, and other pertinent resources. * Donor/Partner Coordination: Developing ties and coordinating municipal water efficiency activities among donors and stakeholders. 1.3 WATER AND ENERGY SECTORS IN BRAZIL It is paramount to understand the overall context of the Brazilian water and energy sector and the very rapid state of change that these sectors have undergone during the establishment of the ASE's activities in Brazil. This background is important for an objective evaluation especially since the ASE established its activities in Brazil, the water sector has been in a state of turmoil, the energy sector has undergone its most serious crisis since the 1970s and the economy has weathered currency devaluation and a major political shiR. The Brazilian economy has been erratic since the 1994 introduction of the Real Plan witnessing a serious devaluation of the currency, which effectively stalled all major long-term investment projects. 1.3.1 The Brazilian Water I Wastewater Sector The Brazilian water and wastewater market requires significant investment, and has been in crisis ever since the establishment of the Alliance's activities in 2001. Overall potable water coverage is only 70 % and the country's sewage collection rate is at approximately 25%. Wastewater treatment among utilities and many industries is virtually nonexistent. The Federal government is aiming for 100% urban water supply coverage and sewage collection rates as well as enforced compliance with environmental standards for wastewater treatment. The need for investment in this sector over the next 5 years is estimated at US$ 12 billion. The Government is taking water very seriously, and is negotiating with the state utilities to maximize investments in water coverage primarily; and secondarily in sewage collection and treatment. The dominant players in Brazil are the 25 state water companies that control 70% of the market and hold over 4,000 municipal water concessions. Most municipal concessions were awarded to the state companies in the 1970's and are now expiring. These water companies have been investing in efficiency gains, as well as increasing coverage ratios since 2000 in an attempt to raise their market value and retain concessions. Investments have been increasingly focused on water loss reductions, wastewater treatment, and expanded sewage and potable water coverage, as well as rural potable water provision. The Brazilian water sector consists of two distinct markets. The first market is the wealthy Southern and Southeast regions characterized by high GDPIcapita, much higher levels of water infrastmcture, and dominant state water utilities. These dominators, although often inefficient, are capable of large efficiency gains and investment. The second market is comprised of the North and Northeast regions characterized by low levels of water infrastmcture, poverty, poor institutional structures and bankrupt state institutions. There is little room for improvement in the short term without multilateral assistance. Evaluation of the WATERGY Program in Brazil Page 6 of 27 Under USAID Contraqct EPP-I-02-03-00007-00 Section 1 Introduction and Overview This administration's strategy has not been to privatize the sector; however to foster PPP (Public Private Partnerships) with Governments and private sources of capital, operators; and particularly service providers and vendors. The legal and institutional framework of the water and sanitation sector in Brazil is incomplete and fragmented, and neither clearly distinguishes the regulatory responsibilities of the federal government, nor those of the states or municipalities. In general, there are overlapping policy, regulatory, and enforcement agencies at all three levels of government, leading to a lack of coherence in policy-making and enforcement. This lack of coherence creates uncertainty in policy-making and instills a high level of risk into the marketplace. Several associations have represented the sector since the 1970s and have had conflicting interests defending public management versus private capital and debating on what sub-national level has the decision making authority on water services. The Brazilian Constitution grants municipalities the responsibility for providing water services. In practice, the major players are the state water companies, which control 70% of municipal concessions. Despite being the conceding power legally, municipalities have little ability to raise finance to invest in system improvements and expansions. The creation of ANA - The National Water Authority - in late 2000 has not had a major impact on the sector's policies. Ths multi-tasked Agency has suffered from political problems given that it has been tasked with oversight of water usage and charges activity based-usage fees for water used in electricity generation, irrigation, navigation and potable water. Average prices of water are relatively high, averaging US $0.80/m3. Wastewater prices are computed as a percentage of the water bill. Low metering and collection ratios, and lower customers affordability coupled with inefficient management have resulted in an inability to finance investment needs. Tariff price disparities are regional, with the state water companies of the South and Southeast having superior metering and collection ratios, and hence, better ability to finance new investments. It is constitutional to cut off supply for non- payment. Investments in Brazil's water and wastewater sector are primarily being financed by state￾owned water companies and through Federal disbursements to city systems through the Caixa Economica Federal (CEF), with limited funding through BNDES and through international (multilateral) financial organizations. These institutions are strong and are the only ones that can effectively make a difference at the policy and operational levels. The Federal government's ability to provide guarantees and resolve institutional and legal issues is critical to increase foreign and domestic private capital in the water sector. Until a Federal decision is reached on legislation defining ownership of concession rights, Brazilian states and municipalities will continue to proceed in individual directions in search of solutions to their investment and operational needs. 1.3.2 The Brazilian Energy Sector During the period in which ASE has established activities in Brazil, the country's energy regulatory structure has been completely overhauled following an energy crisis that resulted in rolling brown-outs and rationing. As a result of various confusing regulatory activities to restructure the sector, generation, transmission and distribution utilities have suffered and Evaluation of the WATERGY Program in Brazil Page 7 of 27 Under USAlD Contraqct EPP-I-02-03-00007-00 Section 1 Introduction and Overview have been weak financially. Energy efficiency initiatives have been extremely difficult to implement without being directly involved with Eletrobras. PROCEL, the Federal Government's efficiency program, spent approximately $400 million on energy efficiency projects, and produced savings of over $3 billion - measured in the avoided cost of power plant and T&D equipment. Procel has received a major impetus from the passage of Law 9,991 in June 2000 requiring all distribution concessionaires to invest 1% of their revenues in energy efficiency research and implementation (approx. $250 rnillion/yr) with 25% of the proceeds must be implemented on demand-side activities or in process audits and changes within large and mid-size consumers. As a result of this promising source of capital funding, almost all Brazilian concessionaires initiated some sort of ESCO type activities. PROCEL's activities were traditionally geared to end-user education, and re-vamping of municipal illumination and energy audits on public sector (mostly federal) buildings. Currently, Procel is actively seeking private partnerships to conduct joint energy audits with utilities, and expects to have continued funding to catalyze the ESCO business in Brazil. PROCEL's long-term goal is to save 77 TWh by 2010, which will be equivalent to approximately 15% of total Brazilian demand. Currently no real ESCO financing mechanism exists in Brazil, mainly due to lack of available funds in local currency at reasonable lending rate rates, for medium to long-term financing (8-12 years). Typical loans for small business have been 18 months - 2 years (renewable) at interest rates in the range of 18-22%, and are usually asset-based. While the Brazilian government's policies and focus don't seem to offer strong support to the existence of ESCOs, other funding sources such as BNDES should be considered. 1.4 EVALUATION OF OBJECTIVES AND METHODOLOGY 1.4.1 Objectives The principal objectives of this task is to provide an independent evaluation of the Alliance's Watergy program in Brazil and to develop a comprehensive set of recommendations to assist USAID and the Alliance improve, scale-up and commercialize Watergy activities in Brazil. The evaluation task addresses the first of 2 phases: Phase 1 - Evaluation of the Watergy activities in Brazil Phase 2 - Development of recommendations for enhancement of Watergy activities. The report addresses the following issues: Measurable program impacts Success relative to defined quantitative indicators Qualitative measures of success Indirect program impacts Organizational and operational issues Effectiveness of program management ("Are they doing the right things?") Evaluation of the WATERGY Program in Brazil Page 8 of 27 Under USAlD Contraqct EPP-I-02-03-00007-00 Section 1 Introduction and Overview Efficiency of program management ("Are they doing things right"?) 1 A.2 Methodology To develop this program evaluation, the Nexant team used the USAID suggested approach, which included the following tasks: Task 1 -Review of Background Watergy Documents The Nexant team reviewed relevant background documents on the Watergy Program, including program statement, stated objectives, program strategies and approaches, annual program descriptions and quarterly progress reports for Brazil. Also a set of program performance indicators identified by USAID and the Alliance were reviewed. Task 2 -Evaluation Framework The evaluation team then developed a framework for the program evaluation. This included definition of the evaluation objectives, identification of broad quantitative indicators for the program including assessment of actual results and changes in market as a result of Watergy, number of successful project models developed and disseminated, and other program impacts, as well as more qualitative measures such as effectiveness of the awareness and capacity building activities, stakeholder satisfaction, effectiveness of communications strategy, and indirect effects of the program. The evaluation framework also included program organization, effectiveness and efficiency of program management, and program sustainability. In conjunction with the evaluation framework, discussion guides were prepared for the interviews with Alliance Washington D.C. representatives, Alliance Brazil project team, and other partners and stakeholders in Brazil. Task 3 - US. Data Collection The Consultant team collected the necessary data required for the evaluation from U.S. sources, including Alliance reports, interviews with Alliance headquarters management and program staff, and discussions with USAlD staff. Task 4 - Site Visits After identifying a list of target stakeholders, the Nexant team met in Brazil and was accompanied by Ms. Simone Lawaetz of USAD who is directing this evaluation effort. During the period April 25 and May 3, the team conducted a series of interviews and meetings with 15 different organizations and stakeholders in Sao Paulo, Belo Horizonte, Brasilia, Fortaleza, and Rio de Janeiro (See Appendix A). These meetings covered the Alliance team, public and private water and energy utilities, USAID staff, relevant NGOs, financial institutions, and other entities associated with the program activities. Phone interviews were also conducted and some organizations documented their comments on the Nexant-provided interview guide. Evaluation of the WATERGY Program in Brazil Under USAlD Contraqct EPP-I-02-03-00007-00 Page 9 of 27 Section 2 Program Obiectives 2.1 ORIGINAL WATERGY PROGRAM OBJECTIVES "Watergy" is the term used by the Alliance to Save Energy and USAID to describe the nexus between water and energy within municipal water systems. Watergy efficiency denotes the interconnections between water and energy savings, combining activities that conserve water, with energy efficiency measures that reduce energy consumption, and synergies resulting from co-managing energy and water resources. The original Watergy program objectives focused on the following activities: Capacity Building: Assist municipal water utilities, directly or through trade associations, government institutions, and other stakeholder to develop the infrastructure to address energy and water efficiency opportunities through good governance. Resource Mobilization: Mobilize funds and other technical resources from the international donor community, government institutions, NGOs, trade associations and from the private sector to assist in capacity building and project implementation. Policy Reforms: Promote policies and regulations for the efficient use of water and energy resources by both utilities and consumers. Information Management: Act as a clearinghouse of information on promoting municipal water utility efficiency including case studies, video of efficiency techniques, software programs, technical overviews, and other pertinent resources. Donor/Partner Coordination: Develop ties and coordinate municipal water efficiency activities among donors and stakeholders including integration with existing activities of the regional development banks and additional World Bank projects. Most recently, and as a natural evolution of the program's past activities, Watergy efforts are now focusing on the development of viable business models for project implementation and the development of a new Brazilian Association for Water and Energy Efficiency (ABAE). The recent objective to "commercialize" Watergy and develop projects aims to bridge current financing barriers in the water sector and lead to the development of business models for potential replication. Specifically, the Alliance would identify barriers to project finance and implementation; create a template for municipal water and energy efficiency performance contracting projects; and initiate momentum through the development of 1 to 2 bankable projects. The creation of ABAE is expected to sustain Watergy efforts in Brazil, independent of US AID funding, and engage stakeholders such as public and private utilities, BNDS, CEF, IADB, the World Bank, and ESCOs as well as other relevant NGO's to promote the improved management of water and energy resources in water and other sectors. Evaluation of the WATERGY Program in Brazil Page 10 of 27 Under USAlD Contraqct EPP-I-02-03-00007-00 Section 2 Program Objectives 2.2 DISCUSSION OF PROGRAM OBJECTIVES The program objectives that were establishet the onset were broad enough to accommodate the rapid changes occurring in the water and energy sector during the course of the program. Their broadness afforded the Watergy team a high degree of flexibility to structure the program based on sector conditions. However, from an evaluation perspective, the objectives raised a number of concerns, such as: It is difficult to develop quantitative performance indicators for the broadly stated objectives. Even if performance indicators were developed, measurement of performance against such indicators would be very difficult without a very elaborate monitoring system. The objectives do not focus on results in terms of project implementation, energy and water savings, and related benefits. Greater efforts to facilitate commercial implementation of Watergy projects, a priority at USAID, are not emphasized in these objectives. 2.3 PERFORMANCE MEASUREMENTS While the program did not develop any quantitative performance indicators, USAID and the Alliance have recently developed a set of program performance indicators that are likely to be used in future evaluation. These indicators are: 1. Percent energy reduced by utilities and municipalities due to the implementation of energy efficiency recommendations identified through Watergy projects. (An average of those projects where energy efficiency recommendations were being implemented in a given year.) 2. Number of state- or national-level policies to promote efficiency in municipal water distribution improved as a result of Alliance assistance. 3. Increase in the number of households with access to clean water. Data collected will include the number of new public standpipes and the estimated number of households served by each pipe. 4. Number of projects to improve municipal or water system efficiency brought to financial closure as a result of assistance from the Alliance team. 5. Number of times performance-based contracting used to finance energy or water efficiency measures associated with Watergy. The recent shift of the Alliance's objectives towards project finance and development provides a more specific and measurable scope of work as well as addresses the need to operationalize the Watergy concept. However, as described in more detail later, such a shift in objective requires that the appropriate skill mix be put in place to meet this objective. Evaluation of the WATERGY Program in Brazil Page 11 of 27 Under USAID Contraqct EPP-I-02-03-00007-00 Section 3 Program Results and Achievements Based on the gathered information and on the outcome of the interviews, the Watergy activities have been extensive over the past 4 years, and have reached a noticeable level of success in the Northeast of Brazil and the state of Minas Gerais. with the exception of the creation of ABAE, the program has been primarily focused on technical activities and guidance to the sectors, including assistance on direct automation, technical supervision, and audits as well training seminars and the publication of manuals that balanced breadth and depth of technical material. The results of these efforts are detailed below: 3.1 WATERGY TOOL KIT The Watergy Toolkit CD-ROM was prepared and distributed to a large number of stakeholders. The Watergy manual, which was first published in 2002, was later translated to Portuguese with the support of PROCEL in a second edition, which allowed for a larger reach in the sector. The lut includes: Videos - Optimization of pumping systems - Leak detection - Audit training Watergy Report and Executive Summary Reports and case studies on Energy audits Water distribution systems optimization Pumps and motors Demand reduction and conservation Leak detection Operations and maintenance practices Energy efficiency in wastewater treatment The kit has been widely disseminated at all of the workshops, seminars and training programs in which the Alliance Watergy staff members have participated. 3.2 THE BRAZILIAN WATER AND ENERGY ASSOCIATION (ABAE) The Alliance to Save Energy initially proposed to work with Brazilian utilities, in partnership with PROCEL, to create the Brazilian Watergy Efficiency Network (BWENIRNEA), a national efficiency network focused on promoting efficiency within the country's water and wastewater sectors. Later, the Alliance decided to re-name the initiative with a more Brazilian-friendly term and supported the development of the Brazilian Water and Energy Association or in Portuguese, The Associaq50 Brasileira ~gua e Energia (ABAE). ABAE serves to link efficiency efforts in over 18 states from across the country. ABAE was approved as an NGO and recognized by the Ministry of Justice in August 2004. In March 2005, ABAE held its annual meeting and appointed its governing council. Mr. Eduardo Nunes De Lima was appointed as Executive Superintendent, Mr. Eusthquio L, de Vasconcelos of SCAI is now the President Director, and Mr. Frederico F. de Vasconcelos of Evaluation of the WATERGY Program in Brazil Page 12 of 27 Under USAID Contraqct EPP-I-02-03-00007-00 Section 3 Program Results and Achievements SCAI heads up the Fiscal Council. The figure below provides its organizational structure (based on the local requirements to achieve and maintain an OSCP status). ABAE's Organizational Structure Figure 3. I ABA E's Organizational Structure Source: ABAE Presentation to the Evaluation Team April 26, 2005 ABAE states its strategic vision as "To be the reference forum for the eflcient and sustainable use of water and energy in Brazil by 2008", and its institutional mission as "To promote energy eflciency and rational water use, contributing to the social, economic and environmental development of Brazilian society". The Alliance also developed a business plan for ABAE outlining the association's objectives, market interaction and implementation plan. A STUDY TOUR TO THE US. In September 2004, the Alliance organized a study tour of Brazilian water utility representatives to the United States, the focus of which was on energy efficiency wastewater treatment technologies, specifically those using anaerobic sludge digestion with methane recovery for combined heat and power as well as other energy efficient applications. Participants visited suppliers and manufacturers of equipment and technology, government agencies and private organizations working with energy efficiency and alternative energy resources. Each of the utilities participating had a different level of experience with wastewater treatment, although none are currently using or recovering the methane gas to generate power. The tour included representatives from COPASA~, CEMIG~, DESO-sE4 and CAGECE and CAESB. Companhia de Saneamento de Minas Gerais Companhia Energktica de Minas Gerais Companhia de Saneamento de Sergipe Evaluation of the WATERGY Program in Brazil Page 13 of 27 Under USAID Contraqct EPP-I-02-03-00007-00 Section 3 Program Results and Achievements 3.4 THE CAGECE PILOT PROJECT The Alliance to Save Energy has worked alongside the Companhia de Agua e Esgoto do Ceara (CAGECE) in the Northeast of Brazil since 2001 in order to develop and implement measures for more efficient use of water and energy. This partnership aimed to improve the distribution of water and the access to sanitation services, while reducing operational costs and environmental impacts. The initiative also aimed at establishing an example for similar projects nationwide, since the water and sanitation sector represents some 2.3 percent of Brazil's energy consumption. The Alliance provided CAGECE with the tools and the know￾how to produce initiatives that result in savings and rational use of energy and distributed water. Activities in this pilot, among other things, included the installation of high efficiency motors, the automation of manual pumping facilities, analysis of electrical systems within administrative buildings, and increasing automation within wastewater treatment systems. The automation of the water supply system in the Fortaleza Metropolitan Region allows for correction of deficiencies in the system, particularly those that are caused by over-design and improved pressure control leaks and loss monitoring. The Alliance's reported activities in 2002 included: Establishing a baseline of energy consumed and water distributed for CAGECE. Implementing efficiency measures that led to a reduction in operational energy consumption. Developing a financing proposal with the Government of Brazil Fight Against Electricity Waste Program (PROCEL) in order to implement energy efficiency projects with CAGECE's operations crew. The technical support provided by the Alliance resulted in the development of energy efficiency projects, costlbenefit analysis, and specifications of equipment that could be financed. Arranging for R$5 million in financing for energy efficiency projects to CAGECE. These projects included rewinding and replacement of motors, maximizing existing pump systems efficiency, and increasing storage capacity to allow the shutdown of pumps during peak hours. Creating an operations procedures manual for CAGECE to serve as a reference for daily performance to operations crews and CAGECE management. Reported Results by the Alliance: The results reported on this pilot project by the Alliance included both energy savings as well as additional residential water connections. Reported results indicated that 88 GWh of energy savings were achieved each year for a period of 4 years. Additionally, the Alliance pointed out that before CAGECE instituted their energy efficiency program, they provided access to 442,400 households. Nearly two years later, 55,597 additional households had potable water access with a negligible change in the amount of water supplied. Growth rate in new connections far surpassed that of the distributed volume of water. By 2002, the utility maintained energy and water consumption levels while providing 88,000 new connections over the original baseline. Four years of official data show savings of over US$2.5 million with an initial investment by CAGECE of only US $1.1 million (R$3 million). Evaluation of the WATERGY Program in Brazil Page 14 of 27 Under USAID Contraqct EPP-I-02-03-00007-00 Seclion 3 Program Results and Achievements Discussions with the CAGECE team provided positive feedback on the project results. In addition to the actual electric savings, CAGECE management highlighted that the program achieved six additional benefits: Standardization of internal processes Hydraulic Pressure controls - benefited the company directly by reducing 9-10 leakslday System-wide reports - the program enabled 50 reports for operations such as reservoir levels, pumping needs, etc which allowed for optimization of individual units Contract renegotiations - the information generated through the automation process allowed for vendor contract negotiations including with the electric utility Operations Management Customer satisfaction - the program enabled faster response time to customer claims and information sharing. The work performed at CAGECE was both detailed and strategic. The first phase of the project lasted 2 years and included the technical audit as well as a detailed engineering and automation plan for the utility's energy savings program. The next phase of the project also lasted 2 years, and included constant monitoring of the project's results and publishing the results and actions to other stakeholders in the sector. According to CAGECE's director, Nonnberto Benevides, "after the implementation of the project, the Alliance team actively evaluated the project's impacts including across 24 remote operations through macro￾metering, pressure controls and pumping station monitoring. The program's impacts were very positive and it resulted in Procel awarding its Efficiency Award to CAGECE." According to CAGECE representatives, approximately 30% of the reported savings is attributable to the Watergy program. It should be noted, however, that the CAGECE representative that was coordinating the pilot during its implementation had been seconded to the Secretaria de Recursos Hidricos do Estado do Ceara (SRH) and was not available to comment on the project despite numerous attempts by the evaluation team. 3.5 CAPACITY BUILDING EFFORTS The Alliance'seminars focused on capacity building has been centered on technical seminars that have been conducted in cooperation with several water utilities and institutions such as COPASA, CAGECE, Companhia de Saneamento do Distrito Federal (CAESB), Companhia de Agua e Esgoto da Paraiba (CAGEPA), Companhia de Saneamento de Sergipe (DESO), PROCEL, and PMSS. The technical seminars included a presentation about the ASE and ABAE followed by 10 to 12 technical and managerial presentations covering the following: System Automation Demand-side management and tariff management Pumping controls Gravity controls and enhancement Evaluation of the WATERGY Program in Brazil Page 15 of 27 Under USAlD Contraqct EPP-I-02-03-00007-00 Section 3 Prwram Results and Achievements Management of peak and off-peak demand and energy consumption Application of variable speed drives. Pressure controls Water Loss Management and measurement Power factor correction techniques Negotiation of demand and energy charges with energy utilities Discussion of Caixa Econornica Federal financing models. Combined, training seminars have had an impact on more than 1,150 stakeholders according to the numbers published by the ASE on attendance of the organized seminars. Table 3.1 Watergy Capacity Building Seminars # of Attendees 1 CAGECE 1 2002, Ceara 1 150 2002, Distrito Gerais I CEGEPA 1 2003, Paraiba 1 160 I DESO 1 2004, Sergipe 1 150 1 CEMIG 1 2004, Minas I 5 0 I COSANPA 1 2004, Para I 150 Besides the technical seminars, the ASE was very active in distributing the Watergy manual. Over 1,000 copies of the Portuguese manual were distributed during the technical seminars, and serve as a reference to utility engineers to-date. Evaluation of the WATERGY Program in Brazil Page 16 of 27 Under USAlD Contraqct EPP-I-02-03-00007-00 Section 4 Program Evaluation INTRODUCTION This section evaluates the Watergy program in Brazil using the following 5 key criteria: Effectiveness of the program -How effective was the Watergy program in meeting the original objectives? Efficiency of the program - How efficiently were the Watergy program resources used? Program management and operations - How well did the Watergy program staff function and what were their strengths and limitations? Sustainability of the operation - To what extent will the activities initiated by Watergy continue and accrue future savings? Environmental and social impacts - To what extent did the Watergy program contribute to environmental and social benefits? Each of these criteria is discussed below. 4.1 EFFECTIVENESS OF THE PROGRAM A summary of the key findings of the evaluation, related to each of the Watergy objectives, can be found below: 4.1 .I Capacity Building Seminars and technical assistance programs have been extensive and well structured and executed. Almost all the interviewed players have praised the technical value of the Watergy initiative, and the knowledge and technical skills that were demonstrated by the Alliance team during the seminars and workshops. Stakeholders interviewed commented that the "level of technical expertise was very useful" (COSANPA). A further qualitative comment is the fact that several of the stakeholders stated that "the ASE was able to structure technical seminars to suit the particular utilities' technical levels" (COPASA). Besides technical capacity building, the Alliance catalyzed a stronger understanding on how to negotiate with energy utilities and to approach issues related to project financing. Testimony to this came from a former director of CAGECE, who credited the Watergy seminars with hls ability to discuss financing issues related to small-scale water efficiency projects with financial institutions such as CEF. I "the seminars were an excellent venue to explain the restrictions for CEF Jinancing and how to overcome the barriers in obtaining small-scale CEF finance for automation and eficiency projects" - Renato Rolim, CAGECE's contact during the pilot. Evaluation of the WATERGY Program in Brazil Page 17 of 27 Under USAlD Contraqct EPP-I-02-03-00007-00 Section 4 Program Evalualion 1 "the ASE's activities were instrumental for our internal evaluation process and to the long-term value creation within the utility." Samir Abud Mauad, Director at COPASA, Capacity building efforts primarily focused on technical staff within the water utilities mores than on senior managers or decision-makers that may change with each election since they are political appointees. While this afforded the program the benefit of working with a core and more stable group of utility staff, opportunities may have been missed to engage leading players in the utilities, as well as in the financial, government, and NGO community. In addition, key institutions that could have played a multiplier effect in spreading the word of 'Watergy' were not fully engaged. One association representative stated that joint trainings with existing institutions could have increased the effectiveness of the Watergy trainings, especially given the strong technical and engineering base in Brazil. Feedback on the study tour to the US, on wastewater treatment technologies were positive and indicated that the participants have all benefited from the visit. 4.1.2 Resource Mobilization The Alliance's resource mobilization efforts have been limited to leveraging resources from PROCEL for information dissemination as well as significant in-kind contributions for technical seminars. The Alliance's attempts to obtain funding first through PROCEL and then later through third-party resources for CAGECE's program met with little success. This is not surprising given that establishing partnerships and obtaining local funding, particularly for public sector organizations, is a major challenge in Brazil. The fact remains that there have been no private water efficiency deals executed in Brazil with the exception of a handful of vendor-funded performance contracts. Bridging existing financing barriers in the water sector and developing business models for potential replication, as stipulated in recent Watergy 'commercialization' objectives, could be a challenging task for the current Alliance team. Achieving these new objectives will require a different skill mix than that existing among the current ASEIABAE team. Carrying the Watergy concept from the capacity building and training mode to market practice will require a broad range of project development capabilities that should have been recognized when setting the new objectives. 4.1.3 Policy Reform The Alliance's program in Brazil did not focus on the development of new policies, legislation or decrees to support water efficiency. This was likely a smart move given the failures of several well-funded initiatives by the World Bank, JBIC and the IDB to foster policy reform in the water sector in Brazil over the last decade. However, a dialogue with the Brazilian institutions in the sectors such as ANA and ANEEL might have helped in aligning Watergy with parallel initiatives, such as those conducted by PROCEL and SABESP (Companhia de Saneamento Bisico do Estado de SBo Paulo). Most policy discussions remained among the circle of utilities with which the Alliance team has had direct interactions. Less frequent communication with the sector's leading institutions, such as the Ministry of Cities (Ministerio das Cidades), has limited the Alliance's efforts to create a stronger national presence. Ths should be taken into consideration in ABAE's business planning. Evaluation of the WATERGY Program in Brazil Page 18 of 27 Under USAlD Contraqct EPP-I-02-03-00007-00 Section 4 Program Evaluation 4.1.4 Information Management- Dissemination to Municipalities The Alliance employed a great deal of effort to disseminate information to the market, which was met by a positive response to products such as the Toolkit CD-ROM and the Watergy manual. The manual and the Toolkit have been widely disseminated at all workshops, seminars and training programs. While the evaluation team's perspective was that the Alliance's approach to the market was narrowly focused on the technical circle, it is worthy to note that such approach paid-off under the massive managerial change in utilities and municipalities staff as it kept the continuity of the initiative. Information was adequately generated and documented as demonstrated by the Alliance's quarterly reports that were managed by the Alliance management team in Washington D.C. However, and despite the major effort to disseminate and document information and management element of the program, the program did not develop a comprehensive database for all activities and contacts. Additionally, most relationships with utilities were strongly focused on the technical staff with less emphasis on management levels. Another observation on information dissemination was the limited engagement of the sector's many multipliers such as the associations and Federal agencies and Ministries. The communication efforts of the program missed a key landmark by not engaging the flagship water utilities such as SABESP, SANEPAR and CEDAE. It was rather surprising that a senior management staff at SABESP, the largest water utility in Brazil had never heard of Watergy. 4.1.5 Donor I Partner Dissemination In terms of market interaction with other potential partners, the evaluation team believes that the effectiveness of past Watergy activities on leveraging other donors and market partners was limited. A more proactive effort should have been allocated to engage the sector's key partners such as ABES (AssociaqZo Brasileira de Engenheiros Sanitarios, the Brazilian Association of Sanitary Engineers, a powerful association that comprises several thousand water sector engineers), AESBE (AssociaqZo das Empresas de Saneamento Bisico Estaduais, the association of state-owned water utilities in Brazil representing public interest in the water sector), and later ABESCO (AssociaqZo Brasileira das Empresas de Servicos de ConcervaqZo de Energia, the Brazilian equivalent of NAESCO) and ABCON (AssociaqZo Brasileira das Concessionarias de Servicos Publicos de Agua e Esgoto, the Brazilian Private Water Concession Association). 4.2 EFFICIENCY OF THE PROGRAM The Alliance's program in Brazil was undertaken during a period that included major currency devaluation. The budget allocated to Brazil was adequate and that the Alliance's performance was commensurate to the budget given this currency volatility. However, it's important to note that the evaluation team has not had access to the financial reports of the Alliance, nor that it was able to breakdown the program budget against the various activities of the program. Evaluation of the WATERGY Program in Brazil Page 19 of 27 Under USAID Contraqct EPP-I-02-03-00007-00 Section 4 Program Evaluation 4.3 PROGRAM MANAGEMENT AND OPERATIONS The program has been well managed and regular communication has been established between headquarters and the field office. In some cases, there was confusion amongst the stakeholders regarding the various roles and 'faces' of the Alliance, SCAI, ABAE, and USAID. As ABAE matures, greater efforts will need to be made to clarify its identity and role, vis-a-vis the other institutions. During interviews, it became apparent that 'Watergy' management needed to better manage the expectations of program beneficiaries. For example, during interviews, CAGECE management commented that the scope of Alliance assistance was not clearly defined upfront. As a result, assistance stopped before all their expectations, particularly those related to project financing, were met. It is worthy to mention that such criticism is typical in donor￾funded projects, where the recipient of the assistance is always assuming that financial resources will ultimately be available. The following table lists both the strength and limitations of the Alliance team in Brazil based on this evaluation exercise: STRENGTHS Strong technical competence in water automation techniques and operation Good reputation within the technical circle of the water industry Proven technical training skills Enthusiastic and dynamic staff Good understanding of the needs of water utilities and their technical priorities Ability to operate with low budget Willingness and ability to customize programs to local needs Access to U.S expertise SUSTAINABILITY OF THE OPERATION LIMITATIONS No formal M&V program to measure & demonstrate implementation results Lack of financial structuring expertise Limited knowledge of project development Lack of familiarity with ESCO operation Lack of policy reform vision No proper documentation on effectiveness of workshopsltraining Difficulty in separating business interest from development activities Limited network with management staff of utilities The Watergy program has significant potential for sustainability as efforts were focused on training and capacity building for technical staff in the water sector. For example, the pilot project implemented at CAGECE will continue to accrue benefits through accumulated energy and cost savings and replication of energy saving measures by staff trained under the program. To enhance the sustainability of the program and eliminate its dependency on donor Evaluation of the WATERGY Program in Brazil Page 20 of 27 Under USAID Contraqct EPP-I-02-03-00007-00 Section 4 Program Evaluation funding, ABAE was created. ABAE aims to attract independent sources of revenue, earned through services to members. The recent emphasis on project development and finance will also assist in identifying new business opportunities for ABAE. For ABAE to be sustainable, more in-depth business planning and identification of more market and business-oriented resources needs to be undertaken. An effective communication strategy and the identification of a clear value-added role that could attract revenue are important elements for the survival of the association. The newly appointed Executive Director seems to offer many of the skills needed for ABAE to operate effectively. However, ABAE's future will depend on the development of an appropriate market vision and being able to identify and offer revenue￾generating services. For ABAE to develop into a mature and respected institution, it will require the support of relevant Government institutions such as the Ministry of Cities (Ministerio das Cidades). Some stakeholders have expressed concern that ABAE has not yet sought such approval and is instead being developed as a "parallel initiative without any formal support by the Government" in a sector already crowded with associations and interest groups. In the future, efforts should be made to garner such support and dispel such perceptions. Moreover, where organizations are implementing similar programs, such as PROCEL's "SANEAR~, ABAE will need to work closely with them to take advantage of any synergies between the programs to prevent duplication. 4.5 ENVIRONMENTAL AND SOCIAL IMPACTS Environmental Impacts The primary environmental impact of the program has been the measurable energy loss and water loss reduction within CAGECE, which resulted in 88 mWh of savings. Moreover, there are likely to be significant unmeasured positive environmental impact from additional measures adopted by utility engineers that have attended the numerous seminars and who have had access to the Watergy toolkit. The associated reduction in Greenhouse Gase (GHG) emissions provides significant benefits to improving environmental conditions as well as public health. Social Impacts The program has significant social impacts on the society at large especially where water supply may be challenging or could potentially be a challenge in the future. The ASE reported that the CAGECE pilot resulted in over 55,000 additional households with potable water access with a negligible change in the amount of water supplied as a result of the increased system efficiency. SANEAR is the Prograrna Nacional de EficiCncia EnergCtica no Saneamento Ambientala program created by PROCEL in June 2004 to cover efficiency issues in both energy and sanitation. Evaluation of the WATERGY Program in Brazil Under USAID Contraqct EPP-I-02-03-00007-00 Page 21 of 27 Section 5 Conclusions and Recommendations 5.1 PROGRAM SUCCESS Since its inception, the Watergy program in Brazil has made a significant contribution to raising the awareness of water and energy efficiency within private and municipal utilities. Watergy's success is particularly notable as it began operation during a period in which the water and energy sectors were operating in a state of crisis and in a struggling economy. Within this dynamic and challenging context, the Alliance has managed to create a successful presence for Watergy, with name recognition and market acceptance. Key successes and achievements credited to the program include: Mobilizing highly qualified technical resources to the water sector especially in automation and monitoring. Undertaking a successful capacity building and technical training activities in various states in Brazil. Weathering sector-wide changes in staff and political resistance to change by focusing on technical career bureaucrats within utilities. Creation and wide dissemination of an industry-needed Watergy Toolkit at all workshops, seminars and training programs. Demonstration of the significant potential of energy savings, environmental improvements, and social benefits through a pilot project jointly implemented with CAGECE. Establishment of 'Watergy' as an icon denoting energy and water efficiency within the sector. PROGRAM CHALLENGES The program was challenged by several issues related to program design and direction and to the skill sets of the Alliance staff. These are summarized below. The initial program objectives were very broad, making it difficult to develop targets and indicators and monitor performance. The recent change in program direction towards commercialization added a significant challenge to the existing program structure and the skill set of the operating team. Evaluation of the WATERGY Program in Brazil Page 22 of 27 Under USAlD Contraqct EPP-I-02-03-00007-00 Section 5 Conclusions and Recommendations The Alliance's Brazil Team relied on their existing network of technical staff in the sector and did not actively involve other leading players, including those within associations, utilities, governments and financial institutions. Some 'multiplier' effect may have been missed by not more closely engaging them. Relationships with these groups will be important for ABAE to cultivate. ABAE has been created in a sector crowded with associations and other interest groups, making it more difficult to define a unique vision. More project development and financing skills are required to successfully implement the current 'commercialization' activities. Some stakeholders could not clearly distinguish between the Alliance, SCAI, USAID, and ABAE, particularly given that SCAI, using different affiliations, is the main interface with the sector stakeholders. The Alliance has not developed a comprehensive database of activities, either implemented by themselves or others, and their achievements. Stakeholder expectations need to be better defined upfront and managed during project implementation. STAKEHOLDER NEEDS Stakeholders in the water and energy sectors were generally pleased with the cross-sectoral approach of the Watergy program and are enthusiastic about its implementation. However, most recognize that financing remains one of the biggest obstacles to its implementation. To address this, the Alliance should consider initiating or expanding on some of the following activities to help stakeholders move from awareness to implementation: Identify sources of project finance compatible with efficiency projects Provide capacity building for project development Create attractive incentives for private equity funding to invest in efficiency projects Develop a "Best Practices" guidebook, and standardized replicable "models" (and related documents) for project financing and implementation Develop easy-to-use monitoring systems Mobilize additional resources from domestic and international sources 5.4 RECOMMENDATIONS As expected from this evaluation effort, the Nexant team developed 2 sets of general directional recommendations to help scale up the program and broaden its impact. One set was for the Alliance program's ongoing efforts in general and the other is for the further development and growth of ABAE. During the second phase of this evaluation, more specific recommendations should be developed. Evaluation of the WATERGY Program in Brazil Page 23 of 27 Under USAlD Contraqct EPP-I-02-03-00007-00 Section 5 Conclusions and Recommendations 5.4.1 Recommendations for the Alliance's ongoing Efforts A clear exit strategy for the program needs to be developed; in which lessons learned from past efforts are transferred to ABAE and the expectations of program partners for future assistance are managed in line with the exit strategy. The next stage of the program effort requires a focus on the larger market instead of a limited number of utilities, and should seek to develop relationships with strategic players that can provide a multiplier effect. Commercialization efforts need to be carefully orchestrated, objectives need to be better outlined, and the necessary skills developed or "borrowed." Rather than just "financing a deal", activities could be expanded to include a more comprehensive partnership with ABESCO to foster pilot activities leveraging the capabilities of its members in project development and finance. 5.4.2 Recommendations for the ABAE Efforts It is important that ABAE develop its strategic vision as a market enabler, creating value by leveraging its knowledge as well as that of other players in a market-sensitive manner. The Association should be careful to set itself apart from technical consulting firms. * ABAE is being created in a sector already over-populated with associations and trade representation entities. While ABAE's focus is unique, it must forge long-term and lasting relationships with market players such as PROCEL and AESBE to take advantage of program synergies as well as with "market-makers" such as CEF, BNDES and the Ministry of Cities (Ministerio das Cidades). The evaluation team considers ABAE's current business plan a good starting point; however, one that should have more market intimacy. The vision for ABAE should be based on an assessment of market needs. ABAE should strongly consider developing a survey instrument prior to finalizing the business plan using the experience of established experts in the sectors. The survey can bring the added benefits of generating awareness of ABAE and of its mission to promote the efficient use of water and energy resources. Evaluation of the WATERGY Program in Brazil Under USAID Contraqct EPP-I-02-03-00007-00 Page 24 of 27 Appendix A List of Stakeholders Interviewed lrganization SABESP - Coinpanhia de Saneamento Bisico do