Urban Energy Services Program Sustainable Municipal Energy Services Evaluation of the ”Watergy” Program in India Prepared for United States Agency for International Development by Nexant Inc. Under Contract No. EPP-I-02-03-00007-00 September 2005 Note: Title page added to electronic version by USAID Development Experience Clearinghouse Table of Contents Page EXECUTIVE SUMMARY ........................................................................................................................1 BACKGROUND...........................................................................................................................................1 OBJECTIVES ..............................................................................................................................................1 METHODOLOGY ........................................................................................................................................1 PROGRAM RESULTS AND ACHIEVEMENTS................................................................................................2 MAJOR FINDINGS AND RECOMMENDATIONS ..............................................................................6 SECTION 1 INTRODUCTION.............................................................................................................8 1.1 BACKGROUND..............................................................................................................................8 1.2 OVERVIEW OF WATERGY ACTIVITIES IN INDIA...........................................................................8 1.3 Objectives of This Program Evaluation ..................................................................................9 1.4 Methodology..........................................................................................................................10 SECTION 2 ORIGINAL WATERGY PROGRAM OBJECTIVES................................................11 2.1 DEFINITION OF WATERGY .........................................................................................................11 2.2 ORIGINAL WATERGY PROGRAM OBJECTIVES .....................................................................11 2.3 DISCUSSION OF PROGRAM OBJECTIVES ....................................................................................11 SECTION 3 PROGRAM RESULTS/ACHIEVEMENTS ................................................................13 3.1 INTRODUCTION ..........................................................................................................................13 3.2 KEY STAKEHOLDERS .................................................................................................................13 3.3 ENERGY AUDITS ........................................................................................................................16 3.4 SEMINARS AND WORKSHOPS.....................................................................................................18 3.5 TRAINING AND CAPACITY BUILDING ........................................................................................19 3.6 PREPARATION AND DISSEMINATION OF TOOLS AND RESOURCES.............................................21 3.7 POLICY REFORMS ......................................................................................................................21 3.8 ASSISTANCE IN DEVELOPMENT OF ENERGY EFFICIENCY ACTION PLANS ................................22 3.9 KEY PERFORMANCE INDICATORS..............................................................................................23 3.10 ESTIMATED AND ACHIEVED ENERGY SAVINGS ........................................................................24 SECTION 4 QUALITATIVE PROGRAM EVALUATION............................................................26 4.1 INTRODUCTION ..........................................................................................................................26 4.2 EFFECTIVENESS OF THE PROGRAM............................................................................................26 4.3 EFFICIENCY OF THE PROGRAM ..................................................................................................30 4.4 PROGRAM MANAGEMENT AND OPERATIONS ............................................................................30 4.5 SUSTAINABILITY........................................................................................................................32 4.6 ENVIRONMENTAL AND SOCIAL IMPACTS ..................................................................................33 SECTION 5 CONCLUSIONS AND RECOMMENDATIONS........................................................35 5.1 MAJOR SUCCESSES ....................................................................................................................35 5.2 MAJOR LIMITATIONS .................................................................................................................35 5.3 STAKEHOLDER NEEDS ...............................................................................................................36 5.4 SUMMARY OF FINDINGS ............................................................................................................37 5.5 RECOMMENDATIONS .................................................................................................................38 LIST OF APPENDICES .......................................................................................................................................... Evaluation of the WATERGY Program in India Under USAID Contract EPP-I-02-03-00007-00 Page 1 of 38 Executive Summary BACKGROUND The Alliance to Save Energy has been working for more than six years with municipalities in various parts of the world to build local capacity and promote the efficient use of energy and water in municipal systems. The Alliance to Save Energy (“Alliance”), in cooperation with the U.S. Agency for International Development (USAID) has developed the concept of Watergy to describe the linkage that exists between water and energy in the context of municipal water utilities. “Watergy efficiency” encompasses the spectrum of water efficiency activities, energy efficiency activities, and resulting synergies from co-managing water and energy resources. Under the Watergy program the Alliance is currently developing commercially viable models that can be used to apply water and energy efficiency concepts to achieve financial sustainability and independence from USAID funding. Under the Sustainable Municipal Energy Services (SMES) Task Order, Nexant was requested by USAID/EGAT to provide an independent evaluation of Watergy in India as well as offer recommendations to assist USAID and the Alliance improve, scale-up and commercialize the program. OBJECTIVES The principal objectives of the program evaluation are to provide an independent evaluation of the Alliance’s Watergy Program in India and to develop a comprehensive set of recommendations to assist USAID and the Alliance improve, scale-up and commercialize Watergy activities in India. The project is being conducted in two Phases: Phase 1 - Evaluation of the Watergy activities in India Phase 2 – Development of recommendations for enhancement of Watergy activities. This report provides the results of the Phase 1 evaluation. Phase 1 addressed the following: ƒ Measurable program impacts ƒ Success relative to defined quantitative indicators ƒ Qualitative measures of success ƒ Indirect program impacts ƒ Organizational and operational issues ƒ Effectiveness of program management (“Are they doing the right things?”) ƒ Efficiency of program management (“Are they doing things right”?) METHODOLOGY The major activities carried out in the Phase 1 evaluation are summarized below: ƒ Review of background Watergy documents Evaluation of the WATERGY Program in India Under USAID Contract EPP-I-02-03-00007-00 Page 2 of 38 ƒ Development of the evaluation framework and discussion guides for interviews with Alliance headquarters staff, India field staff and major stakeholders and partners. Samples of discussion guides are provided in Appendix D. ƒ Collection of data from U.S. sources including Alliance reports, interviews with Alliance headquarters management and program staff, and discussions with USAID staff ƒ Site visits to India to meet with alliance staff, partners, and stakeholders. Two site visits were conducted by a member of the evaluation team, with Ms. Simone Lawaetz accompanying on one of the visits. ƒ Review of findings and preparation of evaluation reports. PROGRAM RESULTS AND ACHIEVEMENTS Activities The Alliance Watergy activities in India have been conducted in four states, Karnataka, Andhra Pradesh, Maharashtra and Delhi. The activities include: ƒ Energy audits - A major focus of the Watergy activities has been on providing technical assistance to municipalities by conducting energy audits. The Alliance has worked with The Energy and Resources Institute (TERI) as the strategic partner for conducting energy audits. ƒ Seminars and workshops – The Watergy program has conducted 11 workshops and seminars for a wide range of audiences. ƒ Training and capacity building – Four formal training courses have been conducted by the Alliance. In addition, Energy Management Cells have been established in all four states. ƒ Preparation and dissemination of tools and resources – The Alliance has developed and widely disseminated the Watergy Toolkit and the results of energy audits. ƒ Policy reforms – The Alliance has contributed significantly to major policy reform in Karnataka that when approved will substantially facilitate the implementation of energy efficiency programs. The Alliance has also contributed to the establishment of the Maharashtra Road Map for water and energy efficiency projects. ƒ Development of energy efficiency action plans - The Alliance has assisted in the development of energy efficiency action plans in Karnataka, Andhra Pradesh and Maharashtra. Evaluation of the WATERGY Program in India Under USAID Contract EPP-I-02-03-00007-00 Page 3 of 38 Performance Indicators In the original Watergy program design, no quantitative program performance indicators were specified. USAID and the Alliance have recently developed a set of program performance indicators that are likely to be used in the future for program evaluation. Table ES-1 lists some of the program output indicators and presents the findings of this evaluation against these indicators. Table ES-1 – Performance Indicators -Program Outputs Output Indicator Results Comments No. of project implementation models developed 3 Alliance focus has not been on alternative implementation models. The planned Karnataka pilot projects will use 3 to 4 different implementation models. Number of projects implemented 5 Projects have been implemented in Visakhapatnam, Vijayanagaram, Karimnagar, Hubli-Dharwad, and Bellary. Planned projects include Pune and Delhi. Percent reduction in energy use/costs See comments Preliminary results of monitoring activities indicate 5.4% savings in Visakhapatnam and 17.3% in Vijayanagaram. No. of government policies adopted/improved See comments Alliance has assisted in a major policy reform initiative in Karnataka, but the draft Government Order (G.O.) has not yet been approved and issued. Alliance has also contributed to the establishment of the Maharashtra Road Map. No. of actions taken by municipalities N/A Many low-cost/no-cost actions have been undertaken by the five municipalities where projects have been implemented. No documentation on no. of actions. Increase in no. of households with access to water See comments Alliance estimates for Karnataka indicate 295,000 households may be potentially served with water with the energy that will be saved from the projects in the 13 ULBs included in the G.O. No. of projects with private sector participation 0 Private sector implementation has not been a major focus of Watergy in India Estimated Potential and Achieved Energy Savings The Alliance Watergy audits have identifies substantial potential energy savings. Since the actual implementation of the audit recommendations has only been initiated recently, the achieved energy savings are much less than the potential estimated savings. Table ES-2 provides a summary of the estimated potential energy savings and the actual achieved savings based on the measured results of the implementation of the audit recommendations. (It should be noted that none of the projects have a formal monitoring and verification plan in place and the “achieved savings” are as reported by the municipalities. Independent verification of these achieved savings was not conducted by the evaluation team.) Evaluation of the WATERGY Program in India Under USAID Contract EPP-I-02-03-00007-00 Page 4 of 38 Table ES-2 - Estimated Potential and Achieved Energy Savings kVA MWH Rs. Lakhs kVA MWH Rs. Lakhs Tiptur/Arasikere 148 364 21 N/A 110 4 Hubli/Dharwad 604 4,102 156 N/A N/A N/A Mysore 879 2,449 106 N/A 384 14 Bellary 473 1,294 65 40 N/A 1 Belgaum 325 2,307 94 N/A N/A N/A Gulbarga 350 1,304 62 N/A N/A N/A 7 KUDCEMP ULBs Visakhapatnam* 440 2,357 73 150 1,436 26 Vijayanagaram* 822 594 44 930 101 28 Karimnagar 180 104 20 180 55 7 Pune Municipal Corp. Delhi Jal Board** 3,159 10,718 534 2,159 2,658 165 Total 7,380 25,593 1,174 3,459 4,744 245 Sources: Data provided by Alliance India staff, Watergy Fact Sheet and Watergy Audit Reports Notes - * Based on results through April 2005. ** Delhi results are estimates based on initial implementation of some meaasures. N/A - Not available Maharashtra Delhi N/A Estimated Potential Savings Achieved Savings from Implementation Karnataka Andhra Pradesh Municipality N/A Program Effectiveness The Watergy program has been effective in increasing the awareness and interest on the part of state government and municipal officials with respect to Watergy efficiency in all four states. Watergy has contributed significantly to the establishment of energy efficiency action plans and strategies. Significant contributions have been made by Watergy to capacity building in municipalities. The Alliance has also been effective in targeting its awareness and outreach activities at senior executives (officers of the Indian Administrative Service or IAS) who are responsible for many of the important decisions relative to program implementation. However, With respect to mobilizing funds and resources, the Alliance efforts have not demonstrated much success, particularly as related to the private sector. Also, while the Alliance has widely distributed the Watergy Toolkit, the Watergy program has not achieved the objective of being a clearinghouse for information on Watergy efficiency. While the Alliance has worked closely with DfID in Andhra Pradesh, there appears to be limited formal activity on their part to coordinate Watergy efficiency activities among other donors and stakeholders and efforts to involve ESCOs and private sector FIs have to date not been successful. Evaluation of the WATERGY Program in India Under USAID Contract EPP-I-02-03-00007-00 Page 5 of 38 Table ES-3 Summarizes the performance indicators for program outcomes. Table ES-3 – Performance Indicators – Program Outcomes Outcome Indicator Outcome Achieved? Comments Increase awareness of State government and municipal officials regarding EE Yes Alliance Watergy efforts have led to substantial increase in awareness and motivation related to energy efficiency in the focus states Establish integrated energy and water management strategy Yes In Karnataka, Maharashtra and Andhra, the Alliance Watergy efforts have contributed to the development of energy efficiency action plans, and energy management cells have been established with Alliance support in Karnataka, Maharashtra, AP and DJB. Increase capacity of municipalities for defining and implementing EE projects Yes Significant contributions to capacity building have occurred in PMC, DJB, and selected ULBs in Karnataka and Andhra Pradesh Promote policies and regulations for Watergy efficiency ?? Major initiative has been drafted in Karnataka but the suggested policy reforms have not been implemented yet. Alliance has contributed to the development of the Maharashtra Road Map. Mobilize funds and resources from donors, government agencies, NGOs, private sector Limited Some resources have been mobilized from government agencies but not much from donors, NGOs and private sector Act as a clearinghouse for information on Watergy efficiency ?? While the Alliance has widely disseminated the Watergy Toolkit and the Watergy Report, they have not been a clearinghouse of information from other sources and projects Coordinate Watergy efficiency activities among donors and stakeholders Limited Alliance has coordinated with a number of partners and stakeholders but not with many of the donors who are funding energy efficiency activities in the water sector. Program Efficiency The Watergy India budgets have been limited and the staff have made good use of the budgets to promote Watergy in four states. The activities in support of the stakeholders in the four target states have generally been carried out quite efficiently. However, since there has been very Evaluation of the WATERGY Program in India Under USAID Contract EPP-I-02-03-00007-00 Page 6 of 38 limited implementation, it is difficult to assess the efficiency of the activities in terms of the dollars spent relative to programs implemented or savings achieved. Some of the stakeholders mentioned that the actual direct interaction with the Alliance staff had been limited. Discussions with the Alliance staff indicated that the available travel budget limited their ability to spend more “face time” with the stakeholders. The Alliance staff appear to have been stretched to their capacity in attempting to work in four states simultaneously. The Watergy India staff have not been very efficient at documenting some of their activities and results, and in measuring the effectiveness of the outreach efforts. Strengths and Weaknesses The strengths and weaknesses of the Alliance India staff are summarized in Table ES-4 below: Table ES-4 – Strengths and Limitations STRENGTHS LIMITATIONS ƒ Good skills in management, coordination and communication ƒ Specialization in municipal EE ƒ Access to and communication with IAS officers ƒ Enthusiastic and dynamic staff ƒ Good understanding of institutional issues ƒ Willingness and ability to customize programs to local needs ƒ Access to U.S expertise ƒ Skills in leveraging and adapting international experience to local needs ƒ No formal M&V program to measure & demonstrate implementation results ƒ Limited technical capabilities ƒ Lack of financial structuring expertise ƒ Limited knowledge and experience with ESCOs and equipment manufacturers ƒ Heavy reliance on TERI ƒ Limited documentation of successes ƒ No documentation of the effectiveness of the workshops/training programs ƒ Lack of hands-on experience in project implementation MAJOR FINDINGS AND RECOMMENDATIONS Major Findings ƒ The Alliance has done a very good job with limited resources to create awareness, increase the motivation and lay the groundwork for implementation of water sector EE projects. ƒ The audits conducted with Alliance support have led to some implementation activities and are likely to lead to more. ƒ The focus of the Watergy activities has not been on project implementation and the Alliance has not emphasized private sector participation in implementation (this will change in Karnataka when the G.O. is passed by GoK). ƒ Stakeholders are pleased with the ASE activities, see a continuing ASE role, and have expressed some specific needs, some of which the Alliance may be able to address ƒ The opportunity for energy efficiency improvement is huge, and with some patience, perseverance and modifications/refinements in focus and approach, much more can be accomplished in the future. ƒ The stakeholders need assistance and support in financial structuring and project implementation, but it is not clear whether the Watergy program as currently structured can effectively and efficiently provide such support. ƒ Better measurement and documentation of the results of Watergy activities are needed. Recommendations 1. USAID should decide the best manner to provide technical assistance and support to the stakeholders in the areas of financial structuring and project implementation. 2. The Alliance should establish a formal monitoring and evaluation scheme to measure and document the achieved savings from the implementation activities. 3. The Alliance should document more formally the Watergy program successes for dissemination to other states and municipalities in India. 4. For future Watergy workshops and training programs, post-program evaluations should be conducted and results documented to allow USAID to gauge the effectiveness of these activities. 5. The focus of future Watergy activities in India should be on providing assistance to selected additional states and municipalities. 6. In designing future Watergy activities, the expressed needs of the stakeholders should be assessed and addressed to the extent feasible. Evaluation of the WATERGY Program in India Under USAID Contract EPP-I-02-03-00007-00 Page 7 of 38 Section 1 Introduction 1.1 BACKGROUND Municipal water utilities are struggling to meet their objective of improving the reliability and quality of water supply services to their constituents while maintaining financial viability. In India, with the increasing global trends towards urbanization this task becomes more complex and challenging. A key issue faced by water utilities is the high amount of energy needed for water supply and delivery. In many cases, water utilities spend over 50% of their operating budget for energy, and rationalization of energy tariffs is leading to increases in these costs. Studies and audits have pointed out the large inefficiencies in the management of energy and water resources by water utilities. The implementation of energy and water efficiency measures can contribute significantly to the reduction of operating costs, increasing financial viability, and enhancing water services. The Alliance to Save Energy has been working for more than six years with municipalities in various parts of the world to build local capacity and promote the efficient use of energy and water in municipal systems. The Alliance to Save Energy (“Alliance”), in cooperation with the U.S. Agency for International Development (USAID) has developed the concept of Watergy to describe the linkage that exists between water and energy in the context of municipal water utilities. This linkage of water and energy exists given the part that energy plays in conveying water to the end user as well as its role in potable water disinfection and wastewater treatment. When water is wasted in a municipal water system, energy is almost always squandered as well. Watergy has attempted to assist water utilities to cost-effectively providing the consumer with the desired services associated with water, while using the least amount of water and energy possible. “Watergy efficiency” encompasses the spectrum of water efficiency activities, energy efficiency activities, and resulting synergies from co-managing water and energy resources. The Watergy program in India was initiated in 2000, and has focused on capacity building, resource mobilization, policy reform, information management, and donor/partner coordination. USAID has recognized that Watergy has initiated major activities and achieved significant results in India, and has raised awareness of issues relating to energy and water efficiency in water utilities, government policy makers, private sector equipment/service providers and donor agencies. While results have been positive, USAID needs to assess the best way to scale-up Watergy efforts, increase the implementation of energy and water efficiency projects and significantly leverage more commercial sources of financing. Given that the efforts in India to date have primarily focused on audits and capacity building, USAID believes that there is a need to rethink the Watergy approach and augment its assistance efforts to help achieve a more meaningful impact on the water sector in India. USAID therefore initiated this program evaluation of Watergy India activities. 1.2 OVERVIEW OF WATERGY ACTIVITIES IN INDIA Both energy and water are very scarce resources in most of India. Indian water utilities, like those in many other developing countries, waste large amounts of energy and water, and do not have the capacity to better manage water and energy resources. As these utilities attempt to assure Evaluation of the WATERGY Program in India Page 8 of 39 Under USAID Contract EPP-I-02-03-00007-00 financial viability and sustainability of water supply and delivery services, energy efficiency has become extremely important. There is a large potential for improvement of energy efficiency and reduction of energy costs in Indian water utilities. The most important energy efficiency measures include upgrading or replacement of major energy equipment and improved operational procedures. However, to date, the implementation of such measures has been low due to the limited technical knowledge, understanding and capability of water utility staff. Watergy activities in India have focused on: ƒ Capacity Building: Assisting municipal water utilities either directly or through trade associations, government institutions, and other stakeholders to develop the capacity to address energy and water efficiency opportunities. ƒ Resource Mobilization: Assisting in mobilization of funds and technical resources from the international donor community, government institutions, NGOs, trade associations and from the private sector to assist in capacity building and project implementation. ƒ Policy Reforms: Recommending policies and regulations that promote the efficient use of water and energy resources by both utilities and consumers. ƒ Information Management: Serving as a clearinghouse of information on improving municipal water utility efficiency including case studies, videos of efficiency techniques, software programs, technical overviews, and other pertinent resources. ƒ Donor/Partner Coordination: Developing ties and coordinating municipal water efficiency activities among donors and stakeholders. The Watergy activities have been carried out in the States of Karnataka, Andhra Pradesh, Maharashtra and Delhi. 1.3 Objectives of This Program Evaluation The principal objectives of the program evaluation are to provide an independent evaluation of the Alliance’s Watergy Program in India and to develop a comprehensive set of recommendations to assist USAID and the Alliance improve, scale-up and commercialize Watergy activities in India. The project is being conducted in two Phases: Phase 1 - Evaluation of the Watergy activities in India Phase 2 – Development of recommendations for enhancement of Watergy activities. This report provides the results of the Phase 1 evaluation. Phase 1 addressed the following: ƒ Measurable program impacts ƒ Success relative to defined quantitative indicators ƒ Qualitative measures of success ƒ Indirect program impacts ƒ Organizational and operational issues ƒ Effectiveness of program management (“Are they doing the right things?”) Evaluation of the WATERGY Program in India Page 9 of 39 Under USAID Contract EPP-I-02-03-00007-00 ƒ Efficiency of program management (“Are they doing things right”?) 1.4 Methodology The following tasks and activities were carried out to complete this program evaluation: Task 1 – Review of Background Watergy Documents The Consultant team reviewed the relevant background documents for the Watergy Program, including the Watergy program statement, stated objectives, program strategies and approaches, annual program descriptions and quarterly progress reports for India. Also a set of program performance indicators identified by USAID and the Alliance were reviewed. Task 2 – Evaluation Framework The Consultant team then developed a framework for the program evaluation. This included definition of the program evaluation objectives, identification of broad quantitative indicators for the program including assessment of actual results and changes in market as a result of Watergy, number of successful project models developed and disseminated, and other program impacts, as well as more qualitative measures such as effectiveness of the awareness and capacity building activities, stakeholder satisfaction, effectiveness of communications strategy, and indirect effects of the program. The evaluation framework also included program organization, effectiveness and efficiency of program management, and program sustainability. In conjunction with the evaluation framework, discussion guides were prepared for the interviews with Alliance Washington D.C. representatives, Alliance India project team, Indian project partners, and stakeholders. Task 3 – U.S. Data Collection The Consultant team collected the necessary data required for the evaluation from U.S. sources, including Alliance reports, interviews with Alliance headquarters management and program staff, and discussions with USAID staff. Task 4 – Site Visits Site visits were conducted in India to meet with Alliance in-country representatives, key project partners (USAID Mission staff to include staff supporting USAID’s USAEP program in India, Indian government agencies, financial institutions, partners/subcontractors, and private sector organizations such as ESCOs) and Alliance client organizations (water utilities and government partners at the state, local and national levels). Copies of the discussion guides were provided in advance to the participants in each meeting. Task 5 – Watergy Evaluation Report Based upon the results of the data collection and the interviews, the Consultant team prepared this report summarizing its findings. Evaluation of the WATERGY Program in India Page 10 of 39 Under USAID Contract EPP-I-02-03-00007-00 Section 2 Original Watergy Program Objectives 2.1 DEFINITION OF WATERGY “Watergy” is the term used by the Alliance to Save Energy and USAID to describe the nexus between water and energy within municipal water systems. Watergy efficiency denotes the interconnections between water and energy savings, combining activities that conserve water, with energy efficiency measures that reduce energy consumption, and synergies resulting from co-managing energy and water resources. 2.2 ORIGINAL WATERGY PROGRAM OBJECTIVES The original Watergy program objectives focused on the following activities: ƒ Capacity Building: Assist municipal water utilities either directly or through trade associations, government institutions, and other stakeholder to develop the infrastructure to address energy and water efficiency opportunities through good governance. ƒ Resource Mobilization: Mobilize funds and other technical resources from the international donor community, government institutions, NGOs, trade associations and from the private sector to assist in capacity building and project implementation. ƒ Policy Reforms: Promote policies and regulations for the efficient use of water and energy resources by both utilities and consumers. ƒ Information Management: Act as a clearinghouse of information on promoting municipal water utility efficiency including case studies, video of efficiency techniques, software programs, technical overviews, and other pertinent resources. ƒ Donor/Partner Coordination: Develop ties and coordinate municipal water efficiency activities among donors and stakeholders. Watergy activities could easily be further integrated with existing activities of the regional development banks and additional World Bank projects. 2.3 DISCUSSION OF PROGRAM OBJECTIVES While these objectives are clearly important to the Watergy mission of promoting water and energy efficiency, they raise a number of issues from the perspectives of program evaluation: ƒ The objectives are broadly stated and not very specific. ƒ It is difficult to develop quantitative performance indicators for these objectives. ƒ Even if performance indicators are developed, measurement of performance against such indicators would be very difficult without a very elaborate monitoring system. ƒ The objectives do not focus on results in terms of project implementation, energy and water savings, and related benefits. Evaluation of the WATERGY Program in India Page 11 of 39 Under USAID Contract EPP-I-02-03-00007-00 ƒ Greater effort to facilitate commercial implementation of Watergy projects, a priority at USAID, is not emphasized in these objectives. Evaluation of the WATERGY Program in India Page 12 of 39 Under USAID Contract EPP-I-02-03-00007-00 Section 3 Program Results/Achievements 3.1 INTRODUCTION The Alliance Watergy activities in India have been conducted primarily in the States of Karnataka, Andhra Pradesh, Maharashtra and Delhi. In this section the highlights of the program activities and results have been presented. Table 3.1 presents a summary of the activities. Table 3.1 - Summary of Watergy Activities Watergy Activities States Energy Audits Karnataka, Andhra Pradesh, Maharashtra, Delhi Seminars and Workshops Karnataka, Andhra Pradesh, Maharashtra, Madhya Pradesh Training and Capacity Building Maharashtra (Pune Municipal Corporation), Sri Lanka (attended by Delhi Jal Board), Delhi Preparation and Dissemination of Tools and Resources including Watergy Toolkit Karnataka, Andhra Pradesh, Maharashtra Assistance in Policy Reforms Karnataka, Maharashtra Assistance in Development of Energy Efficiency Action Plans and Programs Karnataka, Andhra Pradesh, Maharashtra Further information on these activities is provided in the following sections. 3.2 KEY STAKEHOLDERS The Alliance has partnered with key stakeholder organizations in each of the four States where Watergy activities have been conducted. Table 3.2 lists these key stakeholders. An overview of the collaboration between the Alliance Watergy staff and these organizations is provided below. Evaluation of the WATERGY Program in India Page 13 of 39 Under USAID Contract EPP-I-02-03-00007-00 Table 3.2 – Key Stakeholders State Key Stakeholder Karnataka Karnataka Urban Infrastructure Development finance Corporation (KUIDFC) Andhra Pradesh Andhra Pradesh Urban Services for the Poor (APUSP) Maharashtra Pune Municipal Corporation (PMC), Maharashtra Urban Development Department (UDD) Delhi Delhi Jal Board (DJB) Karnataka The key institutional partner of the Alliance Watergy activities in Karnataka is the Karnataka Urban Infrastructure Development Finance Corporation (KUIDFC), the government agency responsible for financing infrastructure development projects in Karnataka. The Alliance helped establish an Energy Management Cell in KUIDFC and funded a full-time position in the Cell for the first year. KUIDFC has funded the Cell in subsequent years. KUIDFC has worked with the Alliance try to change the manner in which the urban local bodies (ULBs) address their electricity bills. In the current situation, all of the bills for ULBs are aggregated by the electricity distribution company (DISCOM) and sent to the Karnataka Power Transmission Corporation (KPTCL) which in turn aggregates the bills for all the DISCOMs and presents the total to the State Finance Corporation (SFC). The SFC then makes a “payment” (in terms of an accounting entry) to KPTCL which in turn makes accounting entries for the DISCOMs. Mr. Tiwari described this process as “very shoddy accounting”. This process is being changed and a new government order (G.O.) is being prepared by the Government of Karnataka (GoK) to address energy efficiency projects and implementation of EE plans. (Note - This G.O has been in process for over one year now.) Additional info on this G.O. provided later in this report. “With the assistance of the Alliance to save Energy, we are now the champions of EE in Karnataka” – Mr. A.K. Tiwari, Joint Managing Director, KUIDFC Andhra Pradesh The principal Alliance Watergy partner in Andhra Pradesh is the Andhra Pradesh Urban Services for the poor (APUSP). This organization was established as a cooperative venture between the Government of Andhra Pradesh (AP) and the U.K. Department of International Development Evaluation of the WATERGY Program in India Page 14 of 39 Under USAID Contract EPP-I-02-03-00007-00 (DfID). The focus of APUSP activities is to improve municipal services and implement municipal reforms to reduce poverty and improve services to the poor. As a part of this effort, APUSP initiated energy efficiency (EE) activities to understand power use, identify opportunities for cost reduction, and increase the awareness and capacity of municipalities in EE project implementation. Initially APUSP funded energy audits of 5 municipalities conducted by the Engineering Staff College of India (ESCI). However, the quality of the audits was poor and no implementation activities resulted. APUSP then approached the Alliance for assistance, and the Alliance assisted APUSP in leveraging DfID funding to initiate energy efficiency activities. The Alliance helped launch the Municipal Energy Efficiency Outreach Program and also assisted in the establishment of an Energy Management Cell (EMC). The Alliance funded audits of water systems in cooperation with APUSP in Vijayanagaram and Karimnagar. The Alliance also funded an energy audit in Visakhapatnam in cooperation with USAEP. The audits were conducted by TERI and pointed out significant opportunities for EE project implementation. All three of these municipalities have initiated some EE activities. The APUSP EMC has developed and published the Energy Efficiency Action Plan. The Government of is now launching a program to conduct energy audits of 118 municipalities. An Expression of Interest (EOI) has been issued in April 2005 to select a group of auditors for this program. “We expect to empanel a number of energy auditors and move forward with implementation of energy efficiency at all 118 municipalities,” Dr. Vijay Kumar, Commissioner and Director, AP Department of Municipal Administration. Maharashtra The Alliance partners in Maharashtra are the Pune Municipal Corporation (PMC) and the Maharashtra Urban Development Department (UDD). USAEP initiated discussions with PMC after they had heard about the Watergy work in Karnataka and Andhra. Based on PMC’s interest, The Alliance, with its technical partner TERI, conducted a training course on pumping energy efficiency for the PMC engineers and completed an energy audit pf the PMC system. Based on the results of the audit, PMC has embarked upon a major investment program to implement EE measures. ASE has also collaborated with the All India Institute of Local Self Government (AIILSG) and the Maharashtra Energy Development Agency (MEDA) to initiate an EE program in Maharashtra. An Energy Management Cell (EMC) has been established at AIILSG to develop and implement a municipal water and energy efficiency outreach program for Maharashtra. The Maharashtra UDD, in cooperation with the Alliance, SAEP/USAID, MEDA, AIILSG, the Maharashtra Jeevan Pradhikaran (MJP), and the Maharashtra Water Supply and Sanitation Department (WSSD) has now initiated the development of a state-wide Energy Efficiency Roadmap for the ULBs in Maharashtra. The objective of this “Roadmap for facilitating and implementing water and energy efficiency projects” in ULBs in Maharashtra is to direct, assist and monitor ULBs in the state to undertake water and energy efficiency programs/projects for Evaluation of the WATERGY Program in India Page 15 of 39 Under USAID Contract EPP-I-02-03-00007-00 their water supply system, wastewater collection system and street lighting, and also to develop a systematic approach for energy efficiency program. “The Road Map is designed to reduce the energy consumption of Urban Local Bodies by adopting Energy Efficiency as a tool, which would subsequently result in enhancement of water supply and reduction of energy deficit of the state” N. B. Patil, Principal Secretary, UDD. Delhi Based on the information on the Alliance Watergy activities in Karnataka and Andhra Pradesh, the Delhi Jal Board (DJB) asked USAEP for assistance in energy audits. DJB is embarking upon a comprehensive reform program, particularly in the management of its distribution systems using private sector participation through Performance-based Management Contracts using World Bank financing. DJB is committed to improve the service delivery level in a phased manner, starting with two of its 21 operational zones and then gradually launching a citywide program. The program also includes a plan for improving the financial sustainability of the Board, and control of its operations cost, a major portion which is energy costs. Recognizing the importance of managing and controlling the energy costs, DJB has established energy management and cost reduction as a high priority in its overall reform process. DJB recently initiated a program of energy cost reduction. Key elements of the program include the establishment of an Energy Management Cell, compilation of a preliminary database on energy consumption at the major HT installations, commissioning of two energy audits at HT installations, and initiation of implementation of energy efficiency improvement activities at these two installations. The Alliance, with co-funding from USAEP and DJB initiated an energy audit (conducted by TERI) of the Haiderpur I Water Treatment Plant and associated pumping stations. The audit pointed out significant energy cost reduction opportunities and DJB has made a commitment to implement the audit recommendations. Subsequently, DJB asked TERI to also conduct an audit at the Delhi Cantonment Booster Pumping Station and is now considering an ESCO project to implement the EE measures at this station. The Alliance was not involved in this effort. The DJB has also developed a Strategic Energy Efficiency Plan under a World Bank initiative. “The TERI audit of Haiderpur I was the first major step in our energy efficiency activities and we hope to implement this and many other EE projects.” Mr. Uttam Kumar, Head, Energy Management Cell, DJB 3.3 ENERGY AUDITS A major focus of the Watergy activities has been on demonstrating energy savings opportunities by conducting pilot energy audits. A comprehensive energy audit of a municipal water system identifies the various opportunities for energy efficiency improvement. The Alliance Watergy team in India has funded or cost-shared energy audits in all of the four States (Karnataka, Andhra Pradesh, Maharashtra and Delhi). The Alliance has worked with The Energy and Resources Institute (TERI) as the strategic partner for conducting energy audits. Evaluation of the WATERGY Program in India Page 16 of 39 Under USAID Contract EPP-I-02-03-00007-00 TERI is a research and consulting organization specializing in energy auditing. They have worked under funding form a wide range of domestic organizations as well as donor agencies (including USAID, GTZ, ADB, DfID, World Bank, EU, and others). The TERI staff has extensive experience in energy audits of municipal pumping systems (over 200 in total), and also performed a large number of pumping system audits for industry and power plants. Their first major project in this area was audits of 60 pumping stations for the Government of Kerala. In Karnataka, TERI had completed energy audits of municipal pumping for the Bangalore Water Supply and Sanitation Board (BWSSB). As a result of these audits, BWSSC undertook a major implementation program. The Alliance selected TERI as a partner based on the success of their audits for BWSSB and the fact that they had a dedicated energy audit team and the needed audit instruments. TERI has developed excellent knowledge, capabilities and experience with municipal pumping energy audits and has now established a core staff of specialists in this field. It should be noted, however, that while TERI provides assistance and support in the implementation process, they do not conduct project implementation. All of the energy audits under the Watergy program in India have been conducted by TERI. Table 3.3 lists these energy audits. Table 3.3 – Energy Audits Conducted under Watergy State Energy Audits Conducted Comments Karnataka Hubli-Dharwad Mysore Tiptur-Arasikere Bellary Belgaum Gulbarga Hubi-Dharwad and Bellary have implemented some audit recommendations. Others have implemented some low￾cost/no-cost measures and are awaiting the passage of the Karnataka Government Order addressing project implementation Andhra Pradesh Visakhapatnam, Vijayanagaram, and Karimnagar All three municipalities have initiated implementation activities. Maharashtra Pune Municipal Corporation (PMC) PMC has committed to a major implementation program. Delhi Haiderpur I Pumping Station of the Delhi Jal Board (DJB) DJB has started implementation of audit recommendations. DJB has also conducted an energy audit of the Delhi Cantonment Pumping Station with TERI but without Alliance support. Discussions with the stakeholders indicated that they were generally very satisfied with the quality of the energy audits conducted by TERI. A review by the evaluation team of selected energy audits conducted under the Watergy program confirmed that they are of a high technical quality. Evaluation of the WATERGY Program in India Page 17 of 39 Under USAID Contract EPP-I-02-03-00007-00 Implementation of the audit recommendations has been initiated in all four states. In Karnataka some of the low-cost and no-cost measures have been implemented in Hubli-Dharwad and Bellary. In Andhra Pradesh, Visakhapatnam, Vijayanagaram, and Karimnagar have all in initiated implementation activities, and PMC and DJB are in the process of initiating implementation. Unfortunately, implementation has not progressed far enough to provide data to validate the audit findings. 3.4 SEMINARS AND WORKSHOPS The Alliance Watergy program has conducted a number of seminars and workshops on water and energy efficiency. A complete list of these is provided in Appendix A. A summary is provided in Table 3.4. Table 3.4 – Summary of Watergy Workshops and Seminars Workshop/Seminar Date/Location Purpose Karnataka Water Outreach Workshop Dec. 2002 Hubli Seminar on Energy Efficiency opportunities in municipal water supply & street lighting – Hubli Karnataka Water Outreach Workshop Feb. 2003 Bangalore Energy Efficiency Opportunities in municipal water supply systems at Tiptur￾Arasikere First ESCO/FI/Government of Karnataka Meeting Mar. 2003 Bangalore Round table discussion regarding involvement of private sector in municipal EE projects Second ESCO/FI/Government of Karnataka Meeting May 2003 Bangalore Follow-up Meeting including the Bureau of Energy Efficiency Workshop on Challenges to and Approaches for Implementing EE May 2003 Bangalore Present Watergy approach to industries; in cooperation with EnergyWise India Management Development for Senior Urban Health Officials July 2003 Hyderabad Alliance presentation on Watergy experiences as part of training program for senior water engineers conducted by ASCI USAEP/IIDC/IL&FS Task Force Joint Meeting Aug. 2003 New Delhi Alliance presentation on Watergy program to encourage implementation in Vizag Cities Matter: Energy Efficiency in the Water Sector Nov. 2003 Bangalore Educate ULBs in India, Sri Lanka and the Philippines on energy efficiency concepts for water systems in cooperation with ICMA Evaluation of the WATERGY Program in India Page 18 of 39 Under USAID Contract EPP-I-02-03-00007-00 Maharashtra Watergy Kick-off Workshop Feb. 2004 Pune Initiation of Statewide Watergy efficiency program in Maharashtra organized by AIILSG World Water Day Mar. 2004 Bangalore Kick-off workshop for UFW Reduction Program, in cooperation with USAID/USAEP and BWSSB Seminar on Water and Energy Conservation Feb. 2005 Mumbai Information dissemination on Watergy efficiency concepts to municipal corporations in Maharashtra organized by AIILSG 3.5 TRAINING AND CAPACITY BUILDING The Alliance has conducted the following types of training and capacity building activities: ƒ Capacity building through workshops and seminars (described in Section 3.4 above) ƒ Training provided to ULB/municipality engineers as a part of the energy audits conducted by TERI ƒ Formal training courses including classroom an field activities ƒ Capacity building through dissemination of Watergy tools and resources In the audits conducted under Watergy (by TERI - see list in Section 3.3 above), the TERI auditors have attempted to engage the local ULB/municipality engineers in the audit process in order to provide some knowledge and achieve some skills transfer. In addition the Alliance has conducted two formal training courses, as shown in Table 3.5 below. A list of the training and capacity building activities provided by the Alliance India staff is presented in Appendix B. Evaluation of the WATERGY Program in India Page 19 of 39 Under USAID Contract EPP-I-02-03-00007-00 Table 3.5 – Formal Training Courses Event Date/Location Purpose PMC Watergy Audit and Training Program Dec. 2004 Pune Capacity building for Pune Municipal Corporation engineers, institutional partners (Maharashtra Energy Development Agency) engineers and other ULB (Nagpur) engineers Sri Lanka Watergy Audit and Training Program Apr. 2004 Colombo Capacity building for Sri Lanka Water Board, Energy Managers Association and Industrial Services Bureau; Included attendees form the Delhi Jal Board Philippines Watergy Audit and Training Program Apr. 2004 Manila Capacity building for Water Utility engineers, ENMAP (Energy Managers Association of the Philippines), UP (University of the Philippines) and NWSDB, Engineers; No attendees from India Delhi Jal Board Audit Training Program May 2005 Delhi Training of DJB energy management cell staff and senior engineers on energy auditing The Alliance capacity building activities have also included establishment and support of Energy Management Cells in Karnataka and Andhra Pradesh and assistance to the Energy management Cell in the Delhi Jal Board. Specifically: ƒ The Alliance Watergy program helped KUIDFC establish an Energy Management Cell, select an individual as the key staff member and fund the cost of the person for one year. ƒ The Alliance also assisted in establishing the Energy Management Cell (EMC) in APUSP. The EMC has developed the Energy Efficiency Action Plan ƒ In Maharashtra, the Alliance has assisted in the establishment of an Energy Management Cell in the All-India Institute for Local Self Government (AIILSG). ƒ DJB had already established an Energy Management Cell (EMC) under World Bank funding. The Alliance funded, planned and coordinated the training of DJB EMC staff and senior engineers in energy auditing. The tools and resources developed and disseminated by the Alliance under the Watergy program for capacity building are discussed in Section 3.6 below. Evaluation of the WATERGY Program in India Page 20 of 39 Under USAID Contract EPP-I-02-03-00007-00 3.6 PREPARATION AND DISSEMINATION OF TOOLS AND RESOURCES To promote increased awareness and understanding of the need and the opportunities for energy efficiency improvement in municipal water systems, the Alliance has developed the following tools and resources which are incorporated into the “Watergy Toolkit”. This Toolkit includes: ƒ Videos - Optimization of pumping systems - Leak detection - Audit training ƒ Watergy Report and Executive Summary ƒ Other resources - Reports and case studies on - Energy audits - Water distribution systems optimization - Pumps and motors - Demand reduction and conservation - Leak detection - Operations and maintenance practices - Energy efficiency in wastewater treatment The Watergy Toolkit has been widely disseminated at all of the workshops, seminars and training programs in which the Alliance Watergy staff members have participated. Appendix C provides a list of the information materials included in the Watergy Toolkit. In addition to the Watergy Toolkit, the Alliance has assisted APUSP in the development of the Energy Efficiency Action Plan, which was prepared as a template for municipalities in Andhra Pradesh to initiate EE activities. 3.7 POLICY REFORMS The Alliance Watergy program has made an important contribution towards a major policy reform initiative in Karnataka. Currently the ULBs in Karnataka have very little incentive to implement energy efficiency activities because of the way in which their electricity bills are processed. As indicated earlier, do not directly pay their energy bills. All of the bills for ULBs are aggregated by the electricity distribution company (DISCOM) and sent to the Karnataka Power Transmission Corporation (KPTCL) which in turn aggregates the bills for all the DISCOMs and presents the total to the State Finance Corporation (SFC). The SFC then makes a “payment” (in terms of an accounting entry) to KPTCL which in turn makes accounting entries for the DISCOMs. Mr. Tiwari described this process as “very shoddy accounting”. Evaluation of the WATERGY Program in India Page 21 of 39 Under USAID Contract EPP-I-02-03-00007-00 The Alliance has worked with KUIDFC and the Departments of Finance, Urban Development (UDD), Municipal Administration (DMA), and Energy (including KPTCL) to make a major change in the policy framework. A new Government of Karnataka Order (G.O.) has been drafted. The draft of the G.O. specifies that a special bank account will be opened, jointly managed by KUIDFC and DMA, for the energy expenditures of 13 ULBs (including the four initially audited by the Alliance, and the ULBs included in the KUWASSIP and KUDCEMP programs). KUIDFC will work with 13 ULBs to reconcile and clear up the past due accounts and develop a pool of funds to address the energy bills of these ULBs The water systems will be metered, energy bills will be prepared by the DISCOMs for each ULB and copies will be sent to the ULBs. An implementation program (partly with private sector participation) undertaken, and the savings will be monitored. The current plan is to initiate pilot projects (some involving ESCOs) as indicated in Table 3.6 below: Table 3.6 – Planned Pilot Projects in Karnataka ULBs Planned Pilot Bellary and Tiptur-Arasikere Implementation by an ESCO selected by KUIDFC, with funding from iDeCK Mysore and Mangalore Implementation by ESCOs selected through a competitive bidding process KUWASSIP* ULBs World Bank funding, with implementation by KUWSDB or ESCOs KUDCEMP** ULBs Implementation by KUWSDB or ULBs with funding from ADB * Karnataka Urban Water Supply and Sanitation Improvement Program, funded by the World Bank ** Karnataka Urban Development and Coastal Environmental Management Program, funded by ADB The implementation of these pilots is currently on hold pending the formal approval and issuance of the G.O. 3.8 ASSISTANCE IN DEVELOPMENT OF ENERGY EFFICIENCY ACTION PLANS The Alliance has assisted in the development of energy efficiency action plans in Karnataka, Andhra Pradesh and Maharashtra. In Karnataka, as described in the previous section, the Alliance has collaborated with KUIDFC, KUWSDB, iDeCK, and the Departments of Finance, Urban development and Municipal Administration to help design a set of pilot programs for EE project implementation in 13 ULBs. Evaluation of the WATERGY Program in India Page 22 of 39 Under USAID Contract EPP-I-02-03-00007-00 In Andhra Pradesh, the Alliance assisted APUSP in the development of the Energy Efficiency Action Plan that provides guidelines for AP municipalities on designing and implementing EE projects. In Maharashtra, the Alliance has been working with the Urban Development Department, Water Supply and Sanitation Department, All-India Institute for Local Self-Government, Maharashtra Energy Development Agency, and USAID/USAEP to develop the “Road Map for Energy Efficiency in Urban Local Bodies in Maharashtra” which was launched in February 2005 at a major conference in Mumbai. 3.9 KEY PERFORMANCE INDICATORS In the original Watergy program design, no quantitative program performance indicators were specified. USAID and the Alliance have recently developed a set of program performance indicators that are likely to be used in the future for program evaluation. Table 3.7 lists some of the program output indicators and presents the findings of this evaluation against these indicators. Evaluation of the WATERGY Program in India Page 23 of 39 Under USAID Contract EPP-I-02-03-00007-00 Table 3.7 – Performance Indicators -Program Outputs Output Indicator Results Comments No. of project implementation models developed 3 Alliance focus has not been on alternative implementation models. The planned Karnataka pilot projects will use 3 to 4 different implementation models. Number of projects implemented 5 Projects have been implemented in Visakhapatnam, Vijayanagaram, Karimnagar, Hubli-Dharwad, and Bellary. Planned projects include Pune and Delhi. Percent reduction in energy use/costs See comments Preliminary results of monitoring activities indicate 5.4% savings in Visakhapatnam and 17.3% in Vijayanagaram. No. of government policies adopted/improved See comments Alliance has assisted in a major policy reform initiative in Karnataka, but the draft Government Order (G.O.) has not yet been approved and issued. No. of actions taken by municipalities N/A Many low-cost/no-cost actions have been undertaken by the five municipalities where projects have been implemented. No documentation on no. of actions. Increase in no. of households with access to water See comments Alliance estimates for Karnataka indicate 295,000 households may be potentially served with water with the energy that will be saved from the projects in the 13 ULBs included in the G.O. No. of projects with private sector participation 0 Private sector implementation has not been a major focus of Watergy in India 3.10 ESTIMATED AND ACHIEVED ENERGY SAVINGS The Alliance Watergy audits have identifies substantial potential energy savings. Since the actual implementation of the audit recommendations has only been initiated recently, the achieved energy savings are much less than the potential estimated savings. Table 3.8 provides a summary of the estimated potential energy savings and the actual achieved savings based on the measured results of the implementation of the audit recommendations. (It should be noted that none of the projects have a formal monitoring and verification plan in place and the “achieved savings” are as reported by the municipalities. Independent verification of these achieved savings was not conducted by the evaluation team.) Evaluation of the WATERGY Program in India Page 24 of 39 Under USAID Contract EPP-I-02-03-00007-00 Table 3.8 – Estimated Potential and Achieved Energy Savings kVA MWH Rs. Lakhs kVA MWH Rs. Lakhs Tiptur/Arasikere 148 364 21 N/A 110 4 Hubli/Dharwad 604 4,102 156 N/A N/A N/A Mysore 879 2,449 106 N/A 384 14 Bellary 473 1,294 65 40 N/A 1 Belgaum 325 2,307 94 N/A N/A N/A Gulbarga 350 1,304 62 N/A N/A N/A 7 KUDCEMP ULBs Visakhapatnam* 440 2,357 73 150 1,436 26 Vijayanagaram* 822 594 44 930 101 28 Karimnagar 180 104 20 180 55 7 Pune Municipal Corp. Delhi Jal Board** 3,159 10,718 534 2,159 2,658 165 Total 7,380 25,593 1,174 3,459 4,744 245 Sources: Data provided by Alliance India staff, Watergy Fact Sheet and Watergy Audit Reports Notes - * Based on results through April 2005. ** Delhi results are estimates based on initial implementation of some meaasures. N/A - Not available Maharashtra Delhi N/A Estimated Potential Savings Achieved Savings from Implementation Karnataka Andhra Pradesh Municipality N/A Evaluation of the WATERGY Program in India Page 25 of 39 Under USAID Contract EPP-I-02-03-00007-00 Section 4 Qualitative Program Evaluation 4.1 INTRODUCTION The qualitative program evaluation addresses the following: ƒ Effectiveness of the program – How effective was the Watergy program in meeting the original objectives? ƒ Efficiency of the program – How efficiently were the Watergy program resources used? ƒ Program management and operations – How well did the Watergy program staff function and what were their strengths and limitations? ƒ Sustainability of the operation – To what extent will the activities initiated by Watergy continue and accrue future savings? ƒ Environmental and social impacts – To what extent did the Watergy program contribute to environmental and social benefits? Each of these topics is discussed below. 4.2 EFFECTIVENESS OF THE PROGRAM The effectiveness of the program can be evaluated using a set of program outcome indicators. While such outcome indicators were not included in the original program design, USAID and the Alliance are developing a set of program outcome indicators. The results of the evaluation relative to possible program outcome indicators consistent with the original Watergy program objectives are summarized in Table 4.1 below. Evaluation of the WATERGY Program in India Page 26 of 39 Under USAID Contract EPP-I-02-03-00007-00 Table 4.1 – Performance Indicators – Program Outcomes Outcome Indicator Outcome Achieved? Comments Increase awareness of State government and municipal officials regarding EE Yes Alliance Watergy efforts have led to substantial increase in awareness and motivation related to energy efficiency in the focus states Establish integrated energy and water management strategy Yes In Karnataka, Maharashtra and Andhra, the Alliance Watergy efforts have contributed to the development of energy efficiency action plans Increase capacity of municipalities for defining and implementing EE projects Yes Significant contributions to capacity building have occurred in PMC, DJB, and selected ULBs in Karnataka and Andhra Pradesh Promote policies and regulations for Watergy efficiency ?? Major initiative has been drafted in Karnataka but the suggested policy reforms have not been implemented yet. Mobilize funds and resources from donors, government agencies, NGOs, private sector Limited Some resources have been mobilized from government agencies but not much from donors, NGOs and private sector Act as a clearinghouse for information on Watergy efficiency ?? While the Alliance has widely disseminated the Watergy Toolkit, they have not been a clearinghouse of information from other sources and projects Coordinate Watergy efficiency activities among donors and stakeholders Limited Alliance has coordinated with a number of partners and stakeholders but not with all the donors. A summary of the key findings relative to program effectiveness is provided below: ƒ The Watergy program has been effective in increasing the awareness and interest on the part of state government and municipal (ULB) officials with respect to Watergy efficiency in all of the four states that have been the focus of Watergy activities. ƒ Watergy has contributed significantly to the establishment of energy efficiency action plans and strategies. Examples of effectiveness include: - In Andhra Pradesh, the APUSP and DMA have published an Energy Efficiency Action Plan and have initiated audits of 118 municipalities. Evaluation of the WATERGY Program in India Page 27 of 39 Under USAID Contract EPP-I-02-03-00007-00 - In Karnataka, a plan for implementation of energy efficiency in 13 municipalities (including pilot programs involving ESCOs) has been prepared, and is awaiting the passage of the G.O. to move forward with implementation. - In Maharashtra, a “Road Map” for municipal energy efficiency has been launched. - In Delhi, the DJB has made a commitment to develop a strategic energy efficiency action plan Thus, in all of the four states that have been Watergy target areas the Alliance has contributed to desired program outcomes. However, since all of the outcomes have resulted from collaborative activities involving a number of organizations, it is difficult to gauge the extent to which these outcomes can be “credited” to the Watergy program. The evaluation team attempted to determine during the stakeholder interviews whether these outcomes would have been achieved without the Alliance participation. In all four states, the stakeholders commented that while it is possible that these activities and outcomes might have resulted from their own activities, the role of the Alliance was very important and that without the contributions of the Alliance, the activities would have been substantially delayed and possibly may not have taken place at all. The evaluation team has concluded from these interviews that the Alliance role was definitely very important if not crucial to the results that have been achieved in these four states. In addition: ƒ Significant contributions have been made by Watergy to capacity building in municipalities in all of the four target states, as reported by the stakeholders in the discussions with the evaluation team. ƒ The Alliance has been effective in targeting some of its awareness and outreach activities at senior executives (officers of the Indian Administrative Service or IAS) who are responsible for many of the important decisions relative to program implementation. In Karnataka, Andhra Pradesh and Maharashtra, the alliance has effectively communicated with state government officials to increase their awareness and interest in energy efficiency. ƒ Watergy has also conducted workshops, seminars and training programs for municipal engineers and officials. However, formal follow-ups of these activities have not been conducted and the evaluation team cannot assess the effectiveness of these activities. ƒ The Watergy activities have made important contributions to the drafting of major policy reforms in Karnataka and the expected policy changes, if and when implemented, would represent a significant breakthrough relative to implementation EE projects. Unfortunately government approval and issuance of the reformed policy directives appears to be stalled. ƒ The selection of TERI as the audit partner has been useful and effective in that TERI is widely respected for their technical knowledge and expertise in municipal EE. Evaluation of the WATERGY Program in India Page 28 of 39 Under USAID Contract EPP-I-02-03-00007-00 ƒ The selection of the specific target states (after the initial targeting of Karnataka) has been reactive and opportunistic, in that the Alliance India staff were approached by municipal officials in Andhra Pradesh, Maharashtra and Delhi seeking assistance. The Alliance has made good use of these opportunities to initiate activities in these states. . In other areas of Watergy activities, the program effectiveness is rather limited. For example: ƒ With respect to mobilizing funds and resources, the Alliance efforts have not demonstrated much success, particularly as related to the private sector. ƒ While the Alliance has widely distributed the Watergy Toolkit, the Watergy program has not achieved the objective of being a clearinghouse for information on Watergy efficiency. ƒ There appears to be limited activity on the part of the Alliance in coordinating Watergy efficiency activities among donors and stakeholders, and efforts to involve ESCOs and private sector FIs have to date not been successful. In terms of the program outcomes, a key disappointment is the limited amount of implementation that has occurred with respect to program implementation. This may be due to a number of barriers to implementation of Watergy efficiency projects, such as: ƒ Large turnover of municipal staff limits the effectiveness of the training and capacity building activities (example – APUSP) ƒ Despite top management commitment, municipal engineers are not fully supportive of efficiency activities (example – DJB) ƒ Policy initiatives and reforms take an enormous amount of time (example – Karnataka G.O.) ƒ Financing is a major barrier. While efforts are being made to develop innovative financing mechanisms (example – Karnataka), the Alliance staff are not very strong in financial structuring and do not have a good partner with skills and experience in financing of municipal projects (like TERI is on auditing). ƒ Mobilizing the private sector (such as ESCOs and private FIs) for municipal project implementation is difficult. The Alliance staff has limited skills and experience in this area, and its principal energy audit partner TERI is skeptical of the viability and capabilities of the private sector. Some of these barriers such as the bureaucratic delays in policy reform and the staff turnover cannot be addressed by the Watergy staff. The others can be addressed but the Watergy India has lacked the know-how and skills (and the partners) to effectively address these. Another important point to be noted is that the Alliance has not put in place a monitoring and verification scheme to measure the results of the implementation activities, thereby limiting an appropriate evaluation of the effectiveness of the program. Evaluation of the WATERGY Program in India Page 29 of 39 Under USAID Contract EPP-I-02-03-00007-00 4.3 EFFICIENCY OF THE PROGRAM The Watergy India budgets have been limited and the staff have made good use of the budgets to promote Watergy in four states. The activities in support of the stakeholders in the four target states have generally been carried out quite efficiently. However, since there has been very limited implementation, it is difficult to assess the efficiency of the activities in terms of the dollars spent relative to programs implemented or savings achieved. There are three other issues related to efficiency: ƒ Some of the stakeholders mentioned that the actual direct interaction with the Alliance staff had been limited. Discussions with the Alliance staff indicated that the available travel budget limited their ability to spend more “face time” with the stakeholders. If the Alliance staff were able to spend more time with the stakeholders, it may be lead to greater progress towards implementation. ƒ The Alliance staff appear to have been stretched to their capacity in attempting to work in four states simultaneously. However, since innovation is very slow in India, this has not significantly influenced the results. However, the Alliance should be careful in any expansion of the Watergy activities to additional states as the efficiency of the activities may suffer if the staff are stretched too thin. ƒ The Watergy India staff have not been very efficient at documenting some of their activities and results, and in measuring the effectiveness of the outreach efforts. In response tom questions raised in this evaluation, the Alliance India staff have now developed better documentation of their major activities and some of the results. 4.4 PROGRAM MANAGEMENT AND OPERATIONS The Watergy India staff have coordinated their activities very well with the Alliance Washington D.C. staff. There appears to be regular (almost daily communication between D.C. and India staff), and frequent visits to India by D.C. staff that include meetings with key stakeholders. The India staff have worked very well together as a team and have also coordinated their activities very well with their principal partner TERI (to the extent that the stakeholders do not distinguish between TERI and the Alliance staff when referring to the audit activities). Based on the discussions with the major stakeholders and the Alliance Watergy staff (both in Washington D.C. and in Bangalore) using a formal discussion guide, and assessment of the available program information, the evaluation team has identified the following strengths and limitations of the Alliance Watergy staff. Strengths ƒ The Alliance staff in India have very good skills in management and coordination and in working with different stakeholders and communication and outreach to the stakeholders and the public. ƒ The Alliance staff skills and experience in India, combined with the knowledge and capabilities of he Alliance D.C. staff, have made them specialists in municipal energy efficiency. Evaluation of the WATERGY Program in India Page 30 of 39 Under USAID Contract EPP-I-02-03-00007-00 ƒ The India staff have demonstrated excellent capabilities to access and communicate with senior IAS officials who are decision-makers in municipalities. ƒ The staff are very dynamic and enthusiastic and have created a very favorable impression on the stakeholders ƒ The staff have a very good understanding of the major institutional issues and barriers in India. ƒ The Alliance India and D.C. staff have shown their willingness and capability to adapt the Watergy efficiency programs to the needs of local stakeholders and market conditions in the different states in which they have worked. ƒ The access to U.S. experience and capabilities has enhanced the credibility of Watergy from the perspectives of the stakeholders. ƒ The Alliance has demonstrated its skills and capabilities in leveraging and adapting international experience to India. Limitations ƒ The Alliance has not implemented formal monitoring and evaluation schemes to measure and demonstrate the energy savings from implementation of EE projects. Measurement schemes are now being implemented by the local officials in Vijayanagaram and Visakhapatnam. However, a more formal approach is needed. ƒ The Alliance staff in India have limited technical capabilities with respect to the energy efficiency technologies and equipment. While the staff includes one engineer with good capabilities in EE, the Alliance has relied to a very large extent on its audit partner TERI for technical expertise. ƒ The Alliance staff have limited knowledge and experience with financial structuring of EE projects and do not have a good financial partner analogous to its technical partner TERI. Since one of the critical barriers in EE project implementation in India is the lack of knowledge and understanding on the part of municipalities and energy service providers regarding the needs of financial institutions, this is an important weakness. ƒ Similarly, the Alliance has limited knowledge and experience with energy service companies (ESCOs) and equipment manufacturers. ƒ The Alliance has placed a very heavy reliance on TERI in audits and implementation support. TERI does not believe that private sector implementation is a viable approach in India. The reliance on TERI may have limited the Alliance efforts related to promotion of private sector implementation approaches. ƒ There has been limited documentation of the successes of the Watergy program in India. ƒ The alliance has not formally evaluated or documented the effectiveness of their workshops, training programs, and other outreach activities. ƒ The Alliance staff lack hands-on experience in project implementation. A summary of the strengths and limitations is provided in table 4.1 below. Evaluation of the WATERGY Program in India Page 31 of 39 Under USAID Contract EPP-I-02-03-00007-00 Table 4.1 – Strengths and Limitations STRENGTHS LIMITATIONS ƒ Good skills in management, coordination and communication ƒ Specialization in municipal EE ƒ Access to and communication with IAS officers ƒ Enthusiastic and dynamic staff ƒ Good understanding of institutional issues ƒ Willingness and ability to customize programs to local needs ƒ Access to U.S expertise ƒ Skills in leveraging and adapting international experience to local needs ƒ No formal M&V program to measure & demonstrate implementation results ƒ Limited technical capabilities ƒ Lack of financial structuring expertise ƒ Limited knowledge and experience with ESCOs and equipment manufacturers ƒ Heavy reliance on TERI ƒ Limited documentation of successes ƒ No documentation of the effectiveness of the workshops/training programs ƒ Lack of hands-on experience in project implementation It should be noted that the Alliance staff (both in India and in Washington D.C.) have recognized a number of the above limitations and have attempted to address some of these. For example, the alliance staff in India have encouraged some of the municipalities (for example, Visakhapatnam and Vijayanagaram) to establish monitoring activities to measure and report actual savings achieved. They have also recognized their limitations with respect to financial structuring and private sector implementation and are attempting to develop the needed skills and capabilities to address these issues (through internal capacity building and/or partnering). For example the Alliance staff in India have worked with iDeCK to develop financial structuring concepts in Karnataka, and have attempted to get local ESCOs involved in developing project implementation approaches. It was reported by the Alliance India staff that such activities were limited by the available funding. 4.5 SUSTAINABILITY A key objective of USAID is to establish a sustainable program. The Watergy activities to date have made important contributions to this objective as discussed below: ƒ The Alliance Watergy efforts in Andhra Pradesh have led to EE project implementation by the municipalities of Visakhapatnam and Vijayanagaram. The favorable results of these activities have led to a major new initiative by the Department of Municipal Administration (DMA) to initiate a large-scale effort to conduct energy audits of 118 municipalities in Andhra Pradesh. The recent Request for EOI issued by the DMA is likely to lead to the selection of auditors, completion of audits and implementation of projects in the near future. This effort is also likely to increase the interest and capability of energy service providers (ESCOs, Evaluation of the WATERGY Program in India Page 32 of 39 Under USAID Contract EPP-I-02-03-00007-00 consultants/auditors, and equipment manufacturers and suppliers) in municipal EE projects. ƒ The Alliance Watergy efforts in Karnataka have directly contributed to the drafting of the G.O. which, when approved and issued, may provide an excellent set of pilot programs for municipal EE projects. The pilots will include demonstration of private sector implementation models, and thereby contribute to the development of interest and capabilities of ESPs in municipal EE. ƒ In Maharashtra, the energy audit and training for PMC and the collaboration with a number of stakeholders to develop the “Road Map”, has created some momentum towards implementation of EE projects. ƒ The Delhi Jal Board has expressed its interest and commitment to embark upon a major systemwide EE initiative partly as a result of the energy audit conducted by the Alliance/TERI. ƒ The Alliance has assisted in the establishment of energy management cells and provided training to the cell staffs in Karnataka (KUIDFC), Andhra Pradesh (APUSP), and Maharashtra (AIILSG), and provided training to the DJB energy management cell. These activities represent significant initial steps towards the establishment of sustainable markets for municipal EE projects. The progress to date has been limited by the fact that many of these initiatives are still in their very early stages (for example, until the G.O. is approved in Karnataka, very little implementation activity is likely to occur). Thus, much more needs to be done with respect to these and other similar initiatives to achieve sustainability. Another issue related to sustainability is the lack of private sector involvement in the Watergy projects to date. For long-term sustainability, it is desirable to establish market mechanisms with ESCOs and/or other private sector participants as active players. 4.6 ENVIRONMENTAL AND SOCIAL IMPACTS Environmental Impacts The energy savings from implemented projects will provide direct environmental benefits. As indicated above, while large potential energy savings have been identified by the Watergy audits, actual implementation of EE projects identified by the Watergy audits has been limited. Some energy savings have been documented by the implementing municipalities, and these savings presumably have resulted in environmental benefits. However, no measurement of environmental impacts has been conducted by the Watergy staff. In Karnataka, the KUIDFC has assembled a proposal to sell the carbon emission reductions resulting from the energy savings to the Community Development Carbon Fund (CDCF) of the World Bank. This proposal, which has been accepted by the World Bank, estimates potential carbon savings of about 35,000 tons per year. . KUIDFC has prepared the Project Design Document (PDD) and the Baseline Methodology for Water Pumping Efficiency Improvement. This Baseline Methodology has been approved by the CDM Methodology Panel of the UNFCCC. Evaluation of the WATERGY Program in India Page 33 of 39 Under USAID Contract EPP-I-02-03-00007-00 Former staff members of the Alliance have played a major role in the development of the baseline Methodology and the PDD. Social Impacts Social benefits of energy efficiency improvement can result if the energy savings lead to improved water services or services to additional households. Also, the energy saved can be used to serve energy to additional households. The Alliance has estimated the potential social benefits based on the projected energy savings identified in the audits. The actual achieved energy savings are limited (as discussed above), and no formal assessments of social benefits has been completed. Evaluation of the WATERGY Program in India Page 34 of 39 Under USAID Contract EPP-I-02-03-00007-00 Section 5 Conclusions and Recommendations 5.1 MAJOR SUCCESSES The Watergy program in India has made very significant contributions to increasing the awareness and interest regarding energy efficiency on the part of state government and municipal officials. India presents a very challenging environment for introduction of new ideas and concepts, particularly when these might disrupt the existing status quo. Government officials are hard to reach and considerable efforts, skills and time are required to convince them to initiate policy reforms and changes in operational procedures. The results achieved by the Alliance in the four target states represent major accomplishments and compare very favorably with other donor￾funded activities in the area of municipal energy efficiency. The following are the major successes of the Watergy program in India: ƒ The Watergy program has identified large potential energy and cost savings by conducting energy audits in the four target states. Initial results of implementation in several municipalities in Karnataka and Andhra Pradesh have demonstrated that these savings can indeed be achieved. ƒ Watergy has made a major contribution to policy reforms in Karnataka by assisting bin the drafting of a G.O. that when issued will provide a major impetus to implementation of EE projects and involvement of ESCOs in implementation. ƒ Watergy activities have resulted in a major initiative in Andhra Pradesh where the Department of Municipal Administration has initiated a major program to fund energy audits of 118 municipalities. ƒ Watergy has contributed to the establishment of the “Road Map” for energy efficiency in municipalities in Maharashtra. ƒ Watergy audits and training in Pune Municipal Corporation and Delhi Jal Board have led to the initiation of corporation-wide energy efficiency activities. ƒ The establishment of energy management cells in KUIDFC, APUSP and AIILSG are contributing to sustainability of energy efficiency activities in Karnataka, Andhra Pradesh and Maharashtra. 5.2 MAJOR LIMITATIONS The major limitation of the Watergy activities in India has been the limited amount of project implementation. While some implementation has taken place in Karnataka, it has mostly been low-cost/no-cost measures and the results to date are far short of the audit projections due to the fact that the G.O. has not yet been approved and issued by the Government of Karnataka. In Maharashtra and Delhi implementation is about to start. Evaluation of the WATERGY Program in India Page 35 of 39 Under USAID Contract EPP-I-02-03-00007-00 A key issue is the lack of innovation with respect to financial structuring and private sector participation. Financing has been cited by the stakeholders as a key barrier to implementation and innovative financing mechanisms need to be developed to facilitate project implementation. Watergy activities in Karnataka have contributed to the design of some financing schemes, but implementation has been stalled pending the G.O. approval. While the Alliance has developed an excellent relationship with its technical partner TERI for conducting energy audits, similar partnerships have not been established with suitable partners with expertise in financial structuring of EE projects and with private sector mobilization to implement municipal EE projects. Perhaps this limitation arises from the lack of strong in-house expertise in these areas both in India and in Washington D.C. The lack of formal monitoring and verification of the results of the implementation is another limitation. The Alliance is now documenting the achieved savings from project implementation in several municipalities in Karnataka and Andhra Pradesh, but these are based on self-reports rather than independent assessments. Another potential limitation is the almost complete reliance on TERI for the energy audits. While TERI has conducted many more municipal energy audits than any other Indian organization, they do not have the hands-on implementation experience. Informal discussions with some of the ESCOs active in India indicated that they believe TERI underestimates the implementation costs. A formal M&V of the projects implemented by the municipalities should be able to assess this criticism. The reliance on TERI as a partner may also have led to the limited Watergy activities relative to ESCO involvement, since TERI has expressed its skepticism of the value that ESCOs may provide in municipal EE project implementation. Another limitation of the Watergy activities in India is the limited cross-fertilization of Watergy efficiency concepts and practices form other countries to India. The stakeholders mentioned the alliance’s international presence and experience as a key strength and expect that the Alliance can successfully transfer some of the “best practices” from other countries to India. However, there is not much evidence of such transfer (for example project financing experience from South Africa or Mexico) by the Alliance Watergy staff in India. 5.3 STAKEHOLDER NEEDS The stakeholders interviewed during the course of this evaluation project were generally pleased with and appreciative of the assistance and support they have received from the Watergy program. However, the key stakeholders also mentioned that they did not see a major role for the Alliance as they move forward towards implementation. While this appears to a favorable indicator of the sustainability of the Watergy activities, it also points out the perception of the stakeholders that the value added by Watergy is primarily in awareness and capacity building, obtaining management “buy-in” and conducting energy audits, and not in financial structuring and project implementation. The stakeholders expressed some future needs for technical assistance and support to meet their goals and objectives relative to improving energy efficiency. Some of the salient needs identified by stakeholders include: ƒ Capacity building for project implementation Evaluation of the WATERGY Program in India Page 36 of 39 Under USAID Contract EPP-I-02-03-00007-00 ƒ Creation of a special fund for project implementation ƒ Development and demonstration of financing mechanisms ƒ Development of a “Best Practices” guidebook ƒ Development of standardized “models” (and related documents) for project financing and implementation ƒ Development of software for identification and analysis of EE opportunities ƒ Mobilizing additional resources from domestic and international sources ƒ Assistance in accessing carbon financing 5.4 SUMMARY OF FINDINGS The major findings of this evaluation are that: ƒ The Alliance has done a very good job with limited resources to create awareness, increase the motivation and lay the groundwork for implementation of water sector EE projects. ƒ The audits conducted with Alliance support have led to some implementation activities and are likely to lead to more. ƒ The focus of the Watergy activities has not been on project implementation and the Alliance has not emphasized private sector participation in implementation (this will change in Karnataka when the G.O. is passed by GoK). ƒ Stakeholders are pleased with the ASE activities, see a continuing ASE role, and have expressed some specific needs, some of which the Alliance may be able to address ƒ The opportunity for energy efficiency improvement is huge, and with some patience, perseverance and modifications/refinements in focus and approach, much more can be accomplished in the future. ƒ The stakeholders need assistance and support in financial structuring and project implementation, but it is not clear whether the Watergy program as currently carried out in India can effectively and efficiently provide such support. ƒ Better measurement and documentation of the results of Watergy activities are needed. Evaluation of the WATERGY Program in India Page 37 of 39 Under USAID Contract EPP-I-02-03-00007-00 5.5 RECOMMENDATIONS The following are the general recommendations from this program evaluation. More specific recommendations will be forthcoming upon the completion of Phase 2 of this evaluation: 1. USAID should decide the best manner to provide technical assistance and support to the stakeholders in the areas of financial structuring and project implementation. 2. The Alliance should establish a formal monitoring and evaluation scheme to measure and document the achieved savings from the implementation activities. 3. The Alliance should document more formally the Watergy program successes for dissemination to other states and municipalities in India. 4. For future Watergy workshops and training programs, post-program evaluations should be conducted and results documented to allow USAID to gauge the effectiveness of these activities. 5. The focus of future Watergy activities in India should be on providing assistance to selected additional states and municipalities. 6. In designing future Watergy activities, the expressed needs of the stakeholders should be assessed, documented, and addressed to the extent feasible. Evaluation of the WATERGY Program in India Page 38 of 39 Under USAID Contract EPP-I-02-03-00007-00 List of Appendices Appendix A – List of Watergy Workshops and Seminars Appendix B – List of Dissemination Activities for Watergy Toolkit Appendix C – Information Products in the Watergy Toolkit Appendix D – Discussion Guides (The Appendices are bound separately.) Evaluation of the WATERGY Program in India Page 39 of 39 Under USAID Contract EPP-I-02-03-00007-00