MID-TERM EVALUATION OF THE SOUTH ASIA REGIONAL INITIATIVE/ ENERGY (SARI/Energy) PROGRAM Prepared for USAID/Bureau for Asia/Near East Prepared by Niels de Terra, Team Leader Shawkat Ali Ferdousi Joanta Green Mahendra P. Lama Checchi and Company Consulting/Louis Berger Joint Venture 1819 H Street, NW Suite 900 Washington, D.C. 20006 Contract No. AEP-I-004-00-00022-00 Final Report January 27, 2003 LIST OF ABBREVIATIONS AND ACRONYMS ADB Asian Development Bank AusAID Australian Development Assistance Agency BSES BSES Ltd. (Diversified publicly-held company headquartered in Mumbai, India) CCO [SARI/Energy] Country Coordinator CERC Central Electricity Regulatory Commission (India) CFL Compact Fluorescent Lamp CIDA Canadian International Development Agency CLASP Collaborative Labeling and Appliance Standards Program (an NGO) CTO Contract Technical Officer (USAID) DOE U.S. Department of Energy DSM Demand Side Management ECO Energy Conservation and Commercialization Project (USAID India) EIA Energy Information Administration (of the U.S. DOE) ESCO Energy Service Company (generic term) EC European Commission GEF Global Environmental Facility GOB Government of Bangladesh GOI ` Government of India FSN [USAID] Foreign Service National IPP Independent Power Project LECO Lanka Electricity Co. LOE Level of Effort MDB Multilateral Development Bank MOU Memorandum of Understanding MW Megawatt NEA Nepal Electricity Authority NGO Non-Governmental Organization NORAID Nordic Countries Development Assistance Agency NTB Non-Tariff Barrier Partners Refers to all entities contracted by USAID to implent the SARI/Energy Program PBS Palli Bidyut Samities (Bangladesh-rural electricity distribution entity) PMP [USAID] Performance Monitoring Plan PPIAF Public-Private Infrastructure Advisory Facility PPA Power Purchase Agreement REB Rural Electrification Board (Bangladesh) SAARC South Asian Association for Regional Cooperation (Member Governments of Bangladesh, Bhutan, India, the Maldives, Nepal, Pakistan, and Sri Lanka) SAFIR South Asia Forum for Infrastructure Regulation (all SAARC countries) SAPP Southern Africa Power Pool SAFTA South Asia Free Trade Agreement SAGQ South Asia Growth Quadrangle (Bangladesh, Bhutan, India (NE and E regions), Nepal) SAPTA South Asia Preferential Tariff Agreement SAREC South Asia Regional Energy Coalition SARI/Energy South Asia Regional Initiative/Energy SOW Scope of Work TA Technical Assistance TERI Tata Energy Research Institute US-AEP U.S. Asia Environmental Partnership USAID U.S. Agency for International Development USG U.S. Government TABLE OF CONTENTS EXECUTIVE SUMMARY......................................................................................................................................I INTRODUCTION................................................................................................................................................... 1 PART ONE ............................................................................................................................................................. 2 Chapter 1. Relevance: Validity of the Original SARI/Energy Concept ......................................................... 2 Chapter 2. Effectiveness of Management Structure and Implementing Tools ............................................... 4 Chapter 3. Efficiency of Use of USAID Resources........................................................................................ 5 Chapter 4. Impacts since Start of Program ..................................................................................................... 6 4. 1 Impact of Capacity Building (Training Courses, Workshops, Peer Exchanges, Seminars)............. 6 4.2 Impact of Initiatives to Transfer Experience/Better Practices.......................................................... 7 4.3 Impact of the SARI/Energy Website(s)............................................................................................8 4.4 Impacts of Technical Assistance (TA) ............................................................................................. 8 4.5 Impact on Regional Energy Cooperation ......................................................................................... 9 4.6 Impact on Mobilization of the Private Sector................................................................................. 10 4.7 Impact of the Partnerships.............................................................................................................. 10 4.9 Impact of SARI/Energy in Creating a Brand ................................................................................. 11 4.10 Interaction with Multilateral Donors.............................................................................................. 11 4.11 Systems and Procedures for Measuring Program Impact and Performance................................... 12 Chapter 5. Sustainability............................................................................................................................... 12 PART TWO .......................................................................................................................................................... 13 Chapter 6. Recommendations for Full Implementation................................................................................ 13 6.1 Observations on Experience to Date .............................................................................................. 13 6.2 Specific Recommendations for Phase 2 ......................................................................................... 18 ANNEXES Annex I: Mid-Term Evaluation Terms of Reference and Methodology Annex II: Overview of Regional Trade Pacts in South Asia by Dr. Shawkat Ali Ferdousi Annex III: The State of Energy Cooperation in South Asia by Prof. Mahendra P. Lama Annex IV: List of Persons Interviewed by Evaluators Annex V: Report on Performance Monitoring Plan (PMP) Indicators by René Bernier, CDIE i EXECUTIVE SUMMARY Summary of Evaluation Findings Relevance of the Concept. When the U.S. Agency for International Development (USAID) started SARI/Energy in 1999 the countries of South Asia traded almost entirely with countries outside the region and all energy and water exchanges were strictly bilateral. The Evaluators find that South Asia is in even greater need of the SARI/Energy Program today than it was in 1999 for the following reasons: • The slow pace of reform in the electricity sector (Bangladesh, India and Nepal) and the lack of regional stability are the most important obstacles to attracting private capital, domestic and foreign. The Program can accelerate the reform process, increase trust and contribute to improved regional stability. • The sustainable supply of affordable and reliable electricity to rural areas is inextricably tied to the alleviation of rural poverty. All the SARI/Energy countries are struggling with this problem and the Program has already become the vehicle for transferring experience and lessons. • Poor governance in the electricity sector is already a well known problem, and it has been a fertile ground for corruption around the world. Corruption discourages foreign investment and reduces economic growth. Through its current activities and by involving new stakeholder groups, SARI/Energy can help improve energy sector governance. • One obstacle to regional energy trading is public sentiment in Bangladesh and Nepal that opposes “giving away” natural resources to India. The resources of the Program can inform on the economic benefits of energy cooperation, and demonstrate that through regional energy trading all trading countries are better off – no country loses. However, in the largest market, India, there is no premium for ‘clean energy’ such as hydropower and natural gas, which must compete with indigenous coal-fired generation. • Energy issues today are central concerns of civil society in SARI/Energy countries, suggesting that regional cooperation initiatives should find receptive stakeholders. Effectiveness of Management Structure and Implementing Tools. The Evaluation examined the experience of the first 20 months and found that in the future USAID should consider a revised structure that would ensure greater integration and operational flexibility to redirect resources across the entire Program in response to external developments. Two options to be considered are (i) a unitary contract structure with various safeguards to ensure access to expertise in small firms and to provide incentives for achieving Program milestones; and (ii) contracting a small firm to be USAID’s “program management support unit” as in the US-Russia Joint Electric Power Alternatives Study. Other options may also arise as USAID’s management and procurement practices evolve. Impacts. The extensive interviews carried out in Bangladesh, India, Nepal and Sri Lanka showed that during its first 20 months the Program’s broadest and most important impact has come from the range of Partner activities that brought individuals together for training courses, workshops, seminars, peer exchanges and partnership study trips. Interviewees repeatedly cited the relevance and high quality of these activities and senior government officials in all the countries have captured these impressions. Generic impacts include the establishing of cross-border relationships between senior officials and executives; capacity building that is relevant to the management of power sector reform, rural energy supply, energy trading practices; and implementation of energy efficiency programs and instruments. Such networks also contribute to build trust in the region. Technical assistance (TA) has had a mixed impact during first 20 months, with some important effects noted in the countries surrounding India, and fewer impacts in India itself. A major focus of TA was on regional energy trading projects and the official absence of the Government of India (GOI) for a nine month period meant that one of the most important “clients” for these studies was out of the process. The TA did succeed in raising the profile of these projects, which is an important precursor to initiatives that will follow. The Four Borders Study led to an initiative by the Government of Bangladesh (GOB) to place the project on the agenda of the first ii meeting of the South Asian Association for Regional Cooperation (SAARC) Energy Technical Committee in November 2002. The transfer of lessons and experience between SARI/Energy countries draws on TA and capability building resources and the Evaluation has documented important achievements in this field that would not have happened without the Program. The ownership and operation of energy sector assets lies almost entirely within the public sector domain in SARI/Energy countries. The current role of the domestic private sectors in energy is important as energy users, as manufacturers of energy equipment and as lenders. A main objective of power sector reform is to bring the private sector in as operators, owners and developers. The main task of bringing the private sector into the Program lays ahead, as this activity only started in early 2002. Evaluation Recommendations 1. Program Duration, Structure and Resource Needs • Approve a Program extension with a five-year time span, and leave open the possibility for a further extension if conditions warrant. • Budget for lower annual funding levels. • Emphasize a SARI/Energy development concept of ‘building blocks,” with the option to add blocks as opportunities arise. • Create a contracting-management structure that is flexible and that provides incentives for achievement of Program milestones, while ensuring that Program level of effort (LOE) is always advancing the objectives. • Provide required expertise. The Program will, at minimum, need the same range of resources as is now offered by the Partners. Socioeconomic analysis skills will also be needed. • Strengthen USAID Program Management capability to monitor deliverables for quality and relevance. Consider refining definition and verifiability of performance monitoring indicators. 2. Stakeholders • Reexamine and expand the definition of stakeholders as the basis for structuring support activities. It may be appropriate to focus on particular stakeholders in one or two countries in order to advance SARI/Energy objectives. Important stakeholder groups include: o Parliamentarians. o Influential Academics and other ‘Eminent Persons’. o Consumer groups. o Trade Unions. o Media. 3. Sustainability • Increase emphasis on sustainability via centers of excellence, other academic institutions, and use of local consulting firms. 4. Strengthening of Private Sector Participation Mechanisms • Refocus the Program initiative to emphasize the role of existing regional, national and local bodies. • Expand and refine the use of TA studies to inform the private sector on benefits obtainable from Program objectives. • Remove the requirement that training participants from indigenous private sector companies must pay for travel expenses connected to training courses. iii 5. Specific Program Elements • Tailor future Technical Assistance to the interests of target groups based on studies and reports that are authoritative and of the highest professional standards. • Make more use of existing studies and reports. • Expand sources for the transfer of lessons and experience to include Latin American and other Asian countries, and examine whether experience transfer mechanisms can be standardized. • Confine SARI/Energy initiatives in energy efficiency to initiatives suitable to local conditions and to actions that will be self-sustaining before the end of the Program. • Focus of capability building: o Complete a Training Needs Assessment. If the assessment cannot be completed during 2003, it should be a first priority during the full implementation period. o Reduce intensity of training for all countries except perhaps India, which missed out on a substantial block of training. The Indian States should receive more attention than was possible in the first 20 months o Use regular SARI/Energy events, e.g., training courses, as opportunities for press coverage. Evaluation Methodology Interviews of stakeholders were arranged by USAID Washington and the four Country Coordinators (CCO)s in consultation with SARI/Energy USAID Program Management. All of the Partners were interviewed in Washington, and those with a presence in the field were also interviewed on site. The Team developed an interview questionnaire (see Annex I) to be used as a guide during interviews. For the field interviews the Team split into two subteams, with Joanta Green and Shawkat Ali Ferdousi responsible for Bangladesh and Nepal, and Niels de Terra and Mahendra Lama responsible for India and Sri Lanka. Subsequently, all four Team Members prepared individual sets of notes summarizing the key points they had gathered from each meeting. The drafting of this report was carried out by the Team Leader drawing on inputs from the other Team members, who were all asked to provide inputs on key crosscutting issues as well as to specialize on specific areas of the Evaluation Statement of Work. 1 INTRODUCTION The 1999 SARI/Energy Strategy Document stated that “The USAID-funded SARI/Energy Program was designed to achieve the objective of energy development and encourage regional cooperation and eventual trade in energy resources among the South Asia countries.” The initiative to create the Program formed part of a broader U.S. initiative to promote regional stability and economic growth. The U.S. Government (USG) recognized that all the countries in the region could benefit by trading those forms of energy in which they enjoyed a comparative advantage and by learning from each other’s experiences with sector reform. Thus the multi-faceted SARI/Energy Program was launched based on close and continuing consultation with Governments and other stakeholders. As the 1999 Strategy Document states: • “Achieving this objective will require firm commitments by the governments in the region, through national policies and international agreements…, • The SARI/Energy program will serve as a vehicle to bring together energy sector players from both public and private sectors across the region to discuss and resolve issues that impede cooperation and investment in energy development; • “… Program support is designed to catalyze and facilitate a long-term process of rationalizing energy supply and distribution across the region, including the development of the cross-border infrastructure and market mechanisms that will be required for eventual trade in electricity, natural gas and other energy resources. • … Policy reforms, supported by SARI/Energy, will be fundamental in helping to reduce the long-term threats to the world’s environment from global climate change. … economic restructuring and democratic decision-making will support the creation of a policy and regulatory environment that ensures fairness and equity…”1 The SARI/Energy program design called for an initial period during which the basic concepts and program management structure would be tested. The design also planned for a mid-term Evaluation that would assess whether the original concept remained valid, and if so, to recommend the shape and duration of a full implementation phase. This report summarizes the findings and conclusions of the Program Evaluation. The Methodology and Terms of Reference (Statement of Work) for this evaluation are contained in Annex I. Interviews were carried out in Washington, D.C. and in Bangladesh, India, Nepal and Sri Lanka during October and November 2002, and a list of persons interviewed is in Annex IV. A working draft of this report was discussed at the Semi-Annual SARI/Energy Program Meeting in Bangkok December 10-12, 2002, and comments were received from all participants. Every effort has been made to take account of these comments while retaining the overall integrity of the Report. The Evaluation Team wishes to thank the Partners for preparing evaluation-briefing books, which facilitated our work. Our thanks also go to the CCOs, and all other officials and entities for their constructive comments and for their time in meetings with Evaluation Team members. The organization and main chapter headings of this report conform to the six headings of the Scope of Work (SOW) as suggested by USAID. It should be noted that the term “Program” and “SARI/Energy” are used interchangeably. 1 Evaluation Statement of Work quoting 1999 USAID SARI/E Strategy Document. 2 PART ONE Chapter 1. Relevance: Validity of the Original SARI/Energy Concept In 1999 the U.S. Government conceived and set out to implement policies to promote stability and economic growth in South Asia by launching a number of regional initiatives, the first of these being in energy. By making stakeholders aware of the important economic and financial advantages obtainable through regional energy cooperation and trading, decision-makers would become proactive. Multidisciplinary Program activities would increase awareness of these benefits and generate momentum for implementation at a political level. With active and sustained energy cooperation, all participating countries would be better off. SARI/Energy would initially build support through joint activities at the level of senior public sector officials, and would also reach out to the local private sectors. These informal networks were also seen as building trust among the SARI/Energy countries. In evaluating any multiyear development assistance program it is important to start with a baseline. Annex II, prepared for this Evaluation by a regional expert in the field, provides a baseline for this Evaluation with respect to pre-SARI/Energy regional trade and cooperation, and states that “South Asia countries, that had very open economies in the immediate post-independence period in the 1949s, had become some of the most highly protectionist in the world by the 1970s. Tariff and, even more importantly, non-tariff barriers (NTBs) were extremely high, state interventions in economic activity had become pervasive, attitudes to foreign investments were negative, often hostile, and stringent exchange controls were in place.” Annex II also points out that even by 1999 only 4.4 % of South Asian trade was within the region, and India and Pakistan imported only 0.8 % and 1.9 %, respectively, of their total imports from other South Asian countries. SARI/Energy’s starting point was that there was very little sense of shared regional energy interests. Bilateral energy issues had been a feature of the subcontinent for many years – Nepal-India power trade, Bhutan-India power exports, Pakistan-India power exchanges from 1947 until the 1950s and the more recent Lahore discussions, and possible Bangladesh-India gas exports. Five SARI/Energy countries were all facing similar problems of rural energy poverty, social and political obstacles to implementing power sector reform, and problems in attracting private capital to the power sector. Each country was dealing with these issues on its own. Thus, SARI/Energy, in launching a process intended to encourage regional energy cooperation, including energy trade, started in an environment that was strictly bilateral in energy terms and not at all regional in terms of general trade. This meant that the culture and mindset of government officials was not regional at the outset, however by the end of 2002 it was clear that the attractions of regional energy solutions were at least being appreciated by India’s neighbors. The slow progress of SAARC towards creating a free trade area is testimony to the historical primacy of bilateral and political issues over regionalism. SARI/Energy offered the participating countries a platform for understanding the commonality of their energy sector problems and for creating a community of policy-level individuals with insights into their neighbors’ perspectives. It now gives them a structured means of exchanging views and acquiring new skills in areas of immediate concern to the SARI/Energy countries. The Evaluators believe that the challenge of encouraging a new, regional mindset may have been underestimated at the outset of the Program. This is particularly so because the achievement of some key SARI/Energy objectives is dependent on the active buy in of the GOI which is not yet manifest. The Evaluators found enthusiasm for SARI/Energy in the other five countries, firsthand and from reliable sources for Bhutan and the Maldives. The Evaluators compared the regional situation prevailing in 1999, with the conditions at the end of 2002, and concluded that in spite of significant challenges, the basic concept and need for the Program remain valid for the following reasons: 3 • The Program provides a neutral forum for dialogue among South Asian nations on important issues to accelerate the efficient supply of energy services needed by all countries for economic growth. Examples include a dialogue on the future participation of Pakistan and Afghanistan in the Program in spite of recent tensions between India and Pakistan; and continued discussions on the reluctance of the GOI to engage in multilateral energy trading. • The slow pace of reform in the electricity sector (Bangladesh, India and Nepal) and the lack of regional stability are the most important obstacles to attracting private capital, domestic and foreign. Private capital is now in retreat. Not counting the suspended 2,180 Megawatt (MW) Dabhol project, eight Indian independent power projects (IPP) totaling 11,278 MW have been canceled or postponed by developers and lenders from France, India, South Korea, United Kingdom and United States since the start of the SARI/Energy Program. Without private capital the crisis in the power sector will only deepen. Even though the pace of reform may continue to be uneven, SARI/Energy countries can learn from each other’s successes and failures just as Latin American countries learned from Chile in the 1980s and 1990s. SARI/Energy, together with the multilateral development bank (MDB) forums, can support and accelerate the reform process by flowing experience and lessons among SARI/Energy countries and by bringing in￾depth knowledge of the reform and restructuring process to the groups that will support and manage reform. Regional energy cooperation will increase regional stability. • The sustainable supply of affordable and reliable electricity to rural areas is inextricably tied to the alleviation of rural poverty. All SARI/Energy countries have been struggling with this problem for decades; some have made more progress than others. Non-governmental organizations (NGOs) and donors are supporting dozens of national efforts to address the problems. Attempts at reform have been complicated by perceptions of electricity supply as an entitlement. SARI/Energy has already become an effective vehicle to transfer experience in rural energy supply. Recent events in Indian Punjab show the political sensitivity of reform. The imposition of tariffs for farmers, and the resulting protests, received large press coverage. • Poor governance in the electricity sector is already a well known problem, and it has made a fertile ground for corruption around the world. Corruption discourages foreign investment and reduces economic growth. For those countries that may export hydrocarbons (Bangladesh) or find new sources (India and Sri Lanka) the lack of proper governance could undermine the benefits of these new energy sources. Through its training and partnership programs, and by involving new stakeholder groups, SARI/Energy can make important contributions to improving governance. • Public sentiment that opposes “giving away” natural resources to India is a barrier to regional energy trading. Bangladesh and Nepal both have high proportions of their populations living in poverty and could benefit from revenues from new or increased energy exports, yet their publics appear not to be fully aware of the potential economic benefits of energy exports. In Bangladesh and Nepal there is public opposition to energy (and water) exports, and in the largest market, India, there is no premium for ‘clean energy’ such as hydro power and natural gas. • Energy issues today are central concerns of civil society in SARI/Energy countries, from complaints over high industrial tariffs in India, Nepal and Sri Lanka, to unreliability or absence of supply, to social resistance to new hydro and thermal power projects. This became evident to the Evaluators from their meetings with journalists, reading of local press, and inputs from the Team members from India and Bangladesh. From the above points the Evaluators conclude that the promotion of regional energy cooperation through SARI/Energy remains a highly worthwhile use of USAID resources because the Program has achieved momentum, with the public and private sector stakeholders now expecting something “concrete” to happen. This is very evident in Bangladesh and Nepal where the Four Border Study kindled strong interest leading to these countries taking the initiative to place this Project on the agenda of the November 2002 meeting of the 4 SAARC Energy Committee. Still more support for energy cooperation needs to be built and SARI/Energy can play a valuable role in moving governments towards solving critical problems that stand in the way of private investment. One subtext of the original concept does merit review and discussion and this is the assumption that the target stakeholders consist of public sector officials. The definition of stakeholders determines the focus and content of all training, technical assistance and private sector outreach and the Evaluators conclude that an expanded definition may be appropriate. This is discussed below in Chapter 6. Chapter 2. Effectiveness of Management Structure and Implementing Tools USAID understood at the outset that achieving SARI/Energy objectives would require a close interaction between Partners (contractors, co-operative agreement recipients and U.S. Government (USG) agencies). SARI/Energy is trying to bring about change that requires political decisions. Until now the participation of the U.S. Department of State in obtaining access to South Asian political level officials has proven very important and will be even more so in the future. At the USG level, the Program used a twinned management structure and this has worked well. USAID program management in Washington interacts with senior USAID officials, with State and other USG agencies (Department of Energy, Department of Commerce and U.S. Trade and Development Agency), and with the USAID SARI/Energy program management unit in New Delhi. Additionally, the decision to make the four CCOs Foreign Service National (FSN) Personal Service Contractors has resulted in productive interaction between SARI/Energy and USAID bilateral program managers in Bangladesh, India and Nepal. In Sri Lanka, where there is no bilateral energy program, the CCO has effectively performed the additional function of ensuring that Sri Lanka benefits from SARI/Energy and the Indian bilateral programs. By having “official” USG status, the CCOs have access to local government officials and thereby give the Program a “one stop” channel. The Evaluators understand that the SARI/Energy Partner contracting structure, in which all partners are equal and responsible for implementing their own work plans, was partly an outcome of constraints in the procurement process prevailing in 1999-2000, and partly due to the need to start work quickly by mobilizing some Partners earlier than others using existing mechanisms. The arrangement has yielded acceptable, and in some cases, good results, but it is not optimum. There is a consultative process to develop annual work plans, which has included the key public sector stakeholders or “champions”. At the semiannual meetings there are opportunities to amend work plans if there is a clear case and a consensus. However, the natural dynamic of implementing contracts and work plans means that the main focus of each partner is on its own work plan, on producing the stated deliverables according to a timetable, and to scheduling human resources to fulfill the work plan. While there is evidence of contacts between partners throughout the year, the Evaluators conclude that these types of interaction cannot achieve the necessary degree of program￾wide integration. This is not due to lack of goodwill or motivation but simply to the natural preoccupation of each partner with its work plan in the knowledge that any assessment of the Partner’s performance will be based on the quality and timeliness of its deliverables. Under such a contracting structure it cannot be expected that one Partner will stop work on a deliverable because it may no longer be important to the advancement of SARI/Energy objectives, and offer to transfer resources to another Partner. The areas where the split structure did not yield optimum outcomes were between training and technical assistance, and between technical assistance and private sector support. The problem was not so much a lack of co-ordination between the concerned partners but the absence of a managing and integrating entity with the knowledge, responsibility and authority to redirect activities that were becoming less relevant to SARI/Energy objectives. With the wisdom of hindsight, more effective integration might have been achieved if the TA partner had been a subcontractor to the Training partner. Another possible approach that USAID has used in the 5 past with multiple large contracts, such as the US-Asia Environmental Partnership (US-AEP) has been to use a small firm, which does not compete or have a conflict of interest with any of the main contractors, as a specialist “management support unit”. However, this can be a costly remedy and does not necessarily ensure the kind of integration needed by the SARI/Energy program, because such units do not have authority to reprogram resources. Until now the Partner contracting structure worked without major problems because USAID personnel (direct hires and contract hires) performed coordinating functions. However, an effective integrating management function that can redirect resources within the program as external developments may require expertise and knowledge that reside in specialized firms and other entities. This question will be carried forward in Chapter 6. Chapter 3. Efficiency of Use of USAID Resources This criterion refers to the “acceptable cost” of achieving results. As far as this Evaluation is concerned, it is a given that USAID general procurement procedures always seek to obtain “best value”, meaning that contractors and grantees are selected on the basis of USAID’s determination that the agreed offer will result in an efficient use of Government resources. Therefore, this Evaluation has focused on the efficiency of the Program contracting structure in the use of resources. The demands on USG assistance resources continue to grow faster than the actual budgets, and the Evaluators must consider all practical ways of achieving program goals in the most cost effective way. The structure of multiple, parallel contracts and co-operative agreements used at the start was necessary for the reasons stated above in Chapter 2. A unitary approach would probably have resulted in severe delays that could have undermined the credibility of the Program. Having reviewed activities during the first 20 months, the Evaluators suggest that, in seeking to make the most efficient use of resources in a large, complex and long term program in which the basic assistance needs may change markedly from year to year, the parallel, multiple contract approach may not result in the most efficient use of resources. Each contractor has a reasonable expectation that the overall scope of work against which it submitted its proposal will be implemented. The Evaluation found that the Program could make more use of preexisting studies and reports in the preparation of materials provided under Technical Assistance. Many studies have been prepared for governments and regional bodies by local institutes, universities, consulting firms and regional organizations on issues addressed by SARI/Energy. Thanks to the initiatives and proactive cooperative spirit shown by the four CCOs and the USAID bilateral energy program managers in Bangladesh, India and Nepal, as well as USAID Colombo, a demonstrated synergy has been achieved between USAID bilateral energy programs and SARI/Energy program activities. This is a major efficiency gain in using USAID bilateral and regional funds because extra benefits are obtained at no additional cost. In the case of Sri Lanka, where there is no current bilateral energy activity, the Program has opened a window to a broad range of participants. The launch of the South Asia Energy magazine is another instance of leveraging USAID resources to achieve a sustainable contribution to regional energy cooperation. Teamwork. The Program was designed to achieve close collaboration among Partners. Work plans are developed each year in a cooperative process so that Partners may take account of each other’s deliverables, avoid duplication and plan handovers from one type of activity to another, i.e. from a training course to a peer exchange. The level of teamwork has been reasonably good, with only a few cases of missed opportunities for substantive interactions. The experience of the first 20 months suggests that in the future new ways may be needed to maintain close integration of Partner activities with overall Program objectives. 6 Chapter 4. Impacts since Start of Program The Evaluation Statement of Work asks a number of important questions regarding Program impacts (see Annex I). An evaluation of impacts must also take into account the target groups and whether the target groups were receptive to the messages and incentives being conveyed. Impact assessments must also judge the degree to which the Program contributed to results, whether marginal or significant. In this regard, it is essential to note that India was officially absent from the Program for a period of nine months – until USAID and the GOI signed the Memorandum of Understanding (MOU) on the Program. This absence affected the impacts of a number of Program initiatives, particularly Technical Assistance, because while some nonpolitical initiatives were able to proceed on a less formal basis, energy trading initiatives could make no progress without GOI support. The most evident overall positive impacts are in the building of national institutions with regional expertise and outlook, and expanding the community of influential people with regional perspectives. 4. 1 Impact of Capacity Building (Training Courses, Workshops, Peer Exchanges, Seminars) One of the defining characteristics of SARI/Energy is that its ability to influence developments in a positive direction is affected by national and regional political events. During the first 20 months of the Program there were four unforeseen negative ‘force majeure’ events2 and one recent positive development, which was the start of the Sri Lanka peace process. The mobilization and delivery of capacity-building activities were rapid and the Program showed ingenuity in overcoming problems arising from the force majeure events and unanticipated procedural bottlenecks. All of the Partners had roles in capacity building, and from the extensive interviews carried out, the Evaluators conclude that capacity-building activities are responsible for the most significant impacts since Program startup. The real sense of SARI/Energy momentum described by most interviewees comes from capacity building. Senior GOI officials, among others, said that ‘capacity building is SARI/Energy’s major achievement to date’. The Program has sponsored capacity-building events with a total of approximately 650 senior officials and executives who have typically spent five days with professional colleagues from other SARI/Energy countries receiving quality capacity building, with role-playing and other interactive techniques, and the generic impacts evident from our interviews can be summarized as follows: • Relationships were established between senior SARI/Energy energy officials by bringing them together in one place and facilitating discussion, exchange of information, ideas and approaches resulting in further informal exchanges and networking. • There has been relevant capacity building among officials who are already or who will make use of new skills later in their careers, particularly in sector reform and rural energy supply. • SARI/Energy has created a non-political co-operative platform for energy sector operators, and has made progress towards creating a culture of thinking regionally on energy cooperation and energy issues, • SARI/Energy is creating awareness among public sector officials that SARI/Energy countries have common energy challenges, and that they can learn from each other. • The program has enabled Bangladesh, Bhutan, Nepal and Sri Lanka, for the first time to identify and discuss common energy issues. SARI/Energy capacity building activities began without the benefit of a standard Training Needs Assessment. Instead, informal feedback from stakeholders was used to select the topics for the capacity-building activities. The Evaluators regard this as having been a sensible approach, given the long lead times needed to plan and implement capacity building, and the imperative to show stakeholders that SARI/Energy was active. 2 The Indo-Pakistan military confrontation, the assassination of the King of Nepal and relatives, the destruction of Sri Lanka’s national aviation fleet by Tamil Tigers, and the nine-month hiatus in India’s involvement in the Program. 7 Capacity-building knowledge is being disseminated beyond courses because most organizations interviewed for the Evaluation have in house practices requiring all trainees to disseminate and share the results of their capacity building with colleagues. Typically, this means a written report and an oral briefing. In some companies with intranet the capacity-building materials were placed on the corporate intranet. In Nepal and Bangladesh we found that implementation of these practices is sporadic, and the Program could encourage more implementation in the future. While the Evaluators could not measure the secondary impacts of capacity building, the fact that knowledge dissemination is a general practice means that USAID resources devoted to capacity building are spreading beyond the actual trainees. Some interviewees cited teething problems with the very first courses offered. This was probably a normal part of the ramp-up process in a complex multicountry activity, and there is evidence that these problems were resolved in a timely and satisfactory manner. The capacity-building partners faced, and still face, a challenge posed by convoluted national nomination processes in some countries, particularly India. It was not uncommon for Indian public sector participants to be advised 24 or 48 hours before the start of a course that they had been selected to travel. The Evaluators were informed that discussions are underway to resolve this problem. The SARI/Energy courses respond well to Program Objectives because they provide grounding in the key issues faced by all of the countries. Specific instances of positive impacts and use of knowledge include: • The Director of Corporate Planning for the Nepal Electricity Authority (NEA) used knowledge from the course on “Introduction to Cross Border Issues” in preparing the NEA’s new Corporate Development Plan. • The course on “Applied Communication: Regulatory Issues in South Asia” resulted in two long articles in the Nepalese press on corruption, tariffs and efficiency. • The Bangkok course on “Energy Efficiency Standards and Labeling” led to the ongoing multicountry initiative under SARI/Energy auspices aimed at reaching agreement on standards for a specific set of energy-using items and appliances. • The course on “Regulatory Commissions: Management and Staff Functions” is influencing or has already influenced the drafting of legislation in Bangladesh, Bhutan, India and Sri Lanka. • The Evaluators examined participants’ comments made six months after the event for eight courses during 2001 and 2002. From this review it is clear that a majority of participants had retained and were applying practices, insights and lessons learned during their five day courses. The most significant issues that were raised by participants for attention in future courses were: • Fewer expatriate trainers and more from the region should be used. Interviewees want to see course materials that are more specific to situations in SARI/Energy and other emerging market countries and less related to U.S. practice. In fact, the Evaluation notes that Partners are already moving in this direction. • Where appropriate, courses should include site or field visits. • A number of participants interviewed felt that there was too much work and pressure, necessitating late nights to complete assignments, presentations or role-playing responses. The Evaluators do not consider that these comments warrant any diluting of course activities, and suggest that they may be an inevitable consequence of combining participants with different career backgrounds. • In all activities, when the term “energy” is used it will be taken to mean the entire energy sector. It should not be used as a synonym for “power” or “electricity.” 4.2 Impact of Initiatives to Transfer Experience/Better Practices Cooperation among all Partners is involved in the transfer process. Only the SARI/Energy Program has the regional overview to identify these opportunities and the resources to transfer experience in a sustainable manner. 8 • A major impact of experience transfer may be seen in the Rural Energy Supply course and the special workshop and field trip to Bangladesh for Indian officials to give them an in-depth view of the Rural Electrification Board (REB)/Palli Bidyut Samities (PBS) developments. In the six month post-course evaluation all respondents gave this activity the highest ratings, and showed that the pros and cons of Bangladeshi experience were well understood. GOI decision makers will be using the information gathered to shape GOI policy. The Indians wanted a longer field visit/workshop to follow for a larger group from India. • The Maldives State Electricity Co. requested a peer exchange with Sri Lanka’s Lanka Electricity Co. (LECO) to study first hand its highly efficient billing and loss reduction programs. This was an outcome from initial contacts in the Regional Utility Partnership. • A transfer of wind power procurement practice from India to Sri Lanka was facilitated by a peer exchange. Elements of Indian practice are being incorporated in Sri Lanka’s 20 MW wind generation procurement. • Transfer of power trading protocols, parameters and experience from the Southern African Power Pool (SAPP) to officials in the four contiguous SARI/Energy countries will be useful as and when a decision is taken to start electricity exchanges and trading. 4.3 Impact of the SARI/Energy Website(s) The Evaluation found that the Program website is off to a good start and those users interviewed expressed appreciation of this resource. Some comments: • The sarie-training.org portion of the website has better features and more content than the main website. It has a ‘feedback’ facility; it lists the names, addresses and e-mail contacts for all Program course “alumni”, it suggests e-newsletters on energy topics and reproduces some news items. • The offering of links is fairly limited and does not include many institutions in South Asia with energy activities. A check of the websites of the South Asian Forum for Infrastructure Regulation (SAFIR), the Central Electricity Regulatory Commission (CERC) and the Tata Energy Research Institute (TERI) showed that they do not have links to the SARI/Energy site. • The “Best Practices” section has only two items: Rural Electrification in Chile and Sri Lanka’s Retail Sector demand side management (DSM) Program for compact fluorescent lamps (CFLs). • The first of four planned “e-learning” courses is well presented on the website. • The “Program Reports” section contains only four of the 14 TA reports. • From the evaluation interviews it appears that only a minority of trainees actually accesses the SARI/Energy website and uses their e-mail addresses. Our sense is that the proportion may be around 10 %. The Webmaster may be able to provide better data on this important question. More publicity for the website should be considered. 4.4 Impacts of Technical Assistance (TA) Based on the interviews conducted by the Evaluation Team, the impact of TA reports and studies has been mixed during the first 20 months, with some important effects noted in the countries surrounding India, and few impacts in India itself. A major focus of TA was on regional energy trading projects and the official absence of India for a nine-month period meant that the one of the most important “clients” for these studies was out of the process. The TA did succeed in raising the profile of two regional power-trading projects – the Four Borders Study and the prefeasibility study for an India-Sri Lanka Interconnector - in Bangladesh, Nepal and Sri Lanka, which is an important precursor to initiatives that will follow. The Four Borders Study led to an initiative by the GOB to place the project on the agenda of the first meeting of SAARC’s Energy Technical Committee in November 2002. We found that dissemination of TA studies has been limited to a specific list of stakeholders, and that the majority of interviewees had not seen a report. The Evaluators were advised that this dissemination policy is 9 deliberate in that the studies are not intended to be for general distribution within the energy community, but rather for a specific targeted audience. Nevertheless, the Evaluators met interviewees in all four countries whom they assumed would have been part of a target audience and who were unfamiliar with relevant studies.3 The TA studies were intended to stimulate stakeholders and decision-makers to take actions and support initiatives that advanced regional cooperation. In view of the fact that, with the exception of the cases noted above, the Evaluators, during their three weeks of interviews, found little evidence of the intended impacts. Therefore, it would be desirable to consider whether more specific guidance is needed from USAID as to the subjects studied and the target audiences. There may be a need to tailor certain TA activities to the interests and concerns of critical national stakeholders for specific projects. The need to review and expand the concept of stakeholders is discussed below. IPP Pricing Benchmarking Study. It is too early to judge the impact of this important study, which has only just been completed. From the preliminary information the Evaluators have seen on this study, it appears to have the potential to play a major role with respect to two of SARI/Energy’s key objectives: • Improving the conditions for private sector investment by stimulating the development of new IPP models and providing greater transparency in the IPP procurement process. • The pioneering analysis of levelized IPP costs should benefit the power sector reform process in SARI/Energy countries by providing insights for decision makers responsible for setting generation expansion policies. The Program’s initiative to improve national energy statistics and demand forecasting tools has generated considerable goodwill; however, this part of the SARI/Energy program has not yet realized a measurable impact because progress in these areas is often gradual. This area of activity provides important support to efforts to improve the climate for foreign investment, and should be continued. Private investment can be discouraged by poor national energy data, and in some SARI/Energy countries there are serious deficiencies. The Program started out with two objectives supported by a very modest level of effort. The first task involved providing advice and assistance to SARI/Energy countries on the collection and presentation of national energy data. The second task involved assistance with tools used for forecasting energy demand. After workshop meetings and discussions with SARI/Energy country energy data and forecasting experts, the Evaluators have drawn the conclusion that in the future a more targeted approach will be needed to reflect the large differences between, say, India on the one hand, and Nepal and Bangladesh on the other. This applies to both areas of activity – national energy statistics and demand forecasting. This part of the Program does not incur development costs and it leverages the existing Department of Energy (DOE)/Energy Information Administration (EIA) expertise. It also benefits from cost-sharing, and thus represents exceptional value for money for USAID. 4.5 Impact on Regional Energy Cooperation The two major electricity trading projects addressed under Technical Assistance (Four Borders Study and the Indian-Sri Lanka Interconnector) may in the future be advanced by Program activities because both had been discussed by some of the countries concerned in prior years. The concept of the South Asia Growth Quadrangle (SAGQ) was developed in 1997. In that same year India and Bangladesh met and agreed at the technical level a 3 E.g., The Sri Lankan chairman of the SARI/Energy Standards and Labeling Working Group had not seen any of the three TA concept papers on this subject; the Chairman of the Central Electricity Authority (India) was unaware of the Best Practices in Rural Energy Services Study; the Center for Energy Studies in Nepal received no TA studies; in Sri Lanka the Interim Energy Regulator and CEO of Power Supply Committee had not seen either of the important Sri Lankan power sector TA studies; the Deputy General Manager for DSM of the Ceylon Electricity Board had not seen the studies on Standards and Labeling; in Bangladesh, SARI/Energy’s new center of excellence had not seen any TA studies, and EnergyPac, a private energy engineering firm had not seen the TA micro turbine study. 10 set of draft parameters for electricity exchanges that could be the core of a prospective SAARC-grid4 . Under Asian Development Bank (ADB) sponsorship, SAGC became the South Asia Subregional Economic Cooperation (SASEC) in 2001. SASEC’s Energy and Power Working Group has met twice since the start of the Program, and SARI/Energy participated in the most recent meeting. The connection to Sri Lanka was discussed in the 1970s when there was surplus power to be traded. The civil war then stopped power exchanges from becoming a reality, and today Sri Lanka and Southern India are both deficit in power. Some interviewees, in comparing the Four Borders initiative with SAPP, stated that the decision to start multilateral power exchanges in Southern Africa was taken in spite of high political tensions because drought conditions made exchanges an absolute necessity to resolve a crisis. While a similar crisis does not exist today, India experiences brownouts in some regions, its grid cannot wheel power from surplus to deficit regions, and India’s Eastern grid region is surplus while Bangladesh’s Western region is deficit. The lessons from SAPP in balancing thermal and hydro systems across surplus and deficit regions, and in the power exchange protocols and practices, are now available in South Asia thanks to SARI/Energy. The Evaluators noted the progress being made to develop and conclude a SARI/Energy Agreement on Harmonization of Energy Efficiency Standards and Labeling. Cooperation between the Training and TA Partners set this process in motion. If this multicountry initiative reaches closure, it would be an important milestone in public and private sector cooperation. Very recently USAID has brought in the Collaborative Labeling and Appliance Standards Program (CLASP) to strengthen this initiative. CLASP brings global expertise to this effort which, unlike regional electricity trading, should not face any political obstacles. 4.6 Impact on Mobilization of the Private Sector Program efforts in this area started much later than other activities owing to administrative delays. Thus, as of late 2002 activity had only been underway for nine months, with much of this concentrated on setting up an institutional activity framework. The Evaluators interviewed officials of chambers of commerce and industry associations in Bangladesh, India, Nepal and Sri Lanka. From these interviews it was not possible for the Evaluators to find impacts from this Program activity as of October 2002. The Evaluators reviewed Program documentation in this area, and this review suggests that the private sector mobilization effort requires closer integration with other Program activities in order to avoid duplication and make maximum use of Program resources. It would be appropriate for USAID Program Management to advise the Partners on this matter. This is an important, complex and challenging area for SARI/Energy. Sustainability suggests a need for the Program to work with and through existing bodies, of which there are many. Some of these bodies follow different paths within the same country, however on the issues being advanced by the Program there are unlikely to be disagreements. Recommendations for the future follow below in Chapter 6. 4.7 Impact of the Partnerships The Evaluation interviews showed that this particular mechanism has high impacts and benefits both in SARI/Energy countries and in the U.S. It is highly leveraged because U.S. partner companies cover the cost of the time of their executives who gain important knowledge and insights into the energy sectors of the region. The four regional partnerships have been efficiently administered following a proven formula, and well coordinated with the other Partners. The in depth exposure to issues combined with the intensive field visits are cited by interviewees as contributing to impacts on national energy policy development and implementation. • The Regulatory Partnership has had direct, measurable impacts. Sri Lanka’s recently passed Electricity Act incorporated provisions that were learned in the U.S. and Canada during the partnership study tour, and 4 See Annex II p. 5 11 Bangladesh and Nepali officials have also cited the influences of this activity on their work. The peer exchanges picked up activity from this partnership and will enhance the impacts. • Other possible partnerships in the region have also been identified: one in hydropower between India and Nepal and one between the Ceylon Petroleum Corp. and the Indian Oil Co. U.S. partners for these two activities will be sought so that they would at least be trilateral partnerships. A minor problem was reported by interviewees in Bangladesh and Sri Lanka over the use of legalistic MOUs. Some participants believed that by signing the MOUs they were being asked to bind their governments to treaty￾like obligations, which they could not do. Less legalistic, non-binding agreements to develop the partnerships should be adopted that would convey a best efforts commitment to achieve stated objectives. 4.8 Impact of Collaboration between SARI/Energy and Bilateral Programs The evaluation found that in Bangladesh, India and Nepal the USAID direct-hire managers of the bilateral energy programs were all very supportive of SARI/Energy and were working with SARI/Energy to achieve genuine synergy and leverage of resources. There is no evidence at all of any duplication between bilateral and SARI/Energy activities. The Director of USAID Sri Lanka expressed appreciation for the efficient way in which SARI/Energy worked with the Mission to enhance the Mission’s profile in the Sri Lankan energy sector, given that there is no bilateral energy program. Particular synergy is evident between Sri Lanka and the Indian bilateral program, where Sri Lankans have, through the Program, participated in activities of the Energy Conservation and Commercialization Project (ECO) in India. USAID Dhaka’s bilateral program is already supporting key SARI/Energy areas such as energy sector reform and rural energy supply, and the USAID Dhaka Energy Program manager works very closely with the CCO to ensure that SARI/Energy training candidate selection complements the bilateral activities. 4.9 Impact of SARI/Energy in Creating a Brand From the Evaluation interviews we conclude that SARI/Energy is known throughout the energy public sector. In the private sector it is known to some energy companies and to at least some officials in all the business and industry associations. It is referred to in press articles. Until now SARI/Energy is known mainly for its capacity￾building activities, and less for its TA. Because the capacity-building activities have been of a high standard, this quality is also associated with SARI/Energy. There is a general anticipation of a high profile “SARI/Energy Project” and it must be obvious that the Program could be firmly implanted in the region by, say, a SARI/Energy Ministerial Meeting launching such a project. The Evaluators noticed that to date the Contract Technical Officers (CTOs) have not required the Partners to adopt common branding practices and each Partner has followed its own corporate practices, which vary from giving little prominence to the SARI/Energy logo and maximum to its own, to giving SARI/Energy a high profile and making clear that the work of the Partner is part of the Program and funded by USAID, prominently displaying the SARI/Energy logo on business cards, training seminars banners, and on all deliverables. If USAID believes that it would be appropriate to reinforce branding then there is an example within the Team of an approach that gives top priority to the visibility of the SARI/Energy logo. To enhance the Brand simple new devices might be considered such as lapel pins with the SARI/Energy logo and SARI/Energy affiliation cards. However, if the current profile of SARI/Energy is thought to be appropriate then no further “branding” steps are required beyond ensuring that all Program activities maintain a high standard of quality. 4.10 Interaction with Multilateral Donors Interaction with the ADB and the World Bank has been positive and mutual benefits have been seen. The Program and ADB have agreed to work together in areas of mutual interest starting with Rural Energy Supply and Transmission Interconnections. SARI/Energy and the ADB's SASEC initiative are the only regional 12 programs by any donors. It has been agreed that SARI/Energy will provide the technical assistance for capacity building and preinvestment grade studies, ADB will consider providing loan funds for projects that are approved. SARI/Energy has sponsored participation by regional energy officials in three consecutive SAFIR courses. SAFIR is supported by a grant from the Public-Private Infrastructure Advisory Facility (PPIAF), a facility supported by multilateral and bilateral donors. 4.11 Systems and Procedures for Measuring Program Impact and Performance5 The Evaluators have taken note of the fact that USAID developed performance indicators in collaboration with the Partners in early 2001, and that the CCOs developed indicator baselines and targets. These were based on key milestones in the four main countries and the contributions of SARI/Energy activities to achieving the milestones. Having seen the PMP Report prepared by Mr. Bernier, the Evaluators concur that the performance indicators at the Program level, are broad, and that by themselves they cannot quantify the Program’s role in achieving a particular milestone. There are many factors are at work, some of which preceded the Program and others which are external. Therefore attribution of credits to the Program for achieving milestones can require background analysis. Additionally, at the PMP level is has been necessary to aggregate the performance of the Partners. The mid-term Evaluation did not turn up any information that would lead the Evaluators to question or propose additions to the Summary and Conclusions in Mr. Bernier’s report. At the level of monitoring Partner performance, the Evaluators recommend that USAID strengthen its systems for monitoring the quality and relevance of Partners’ deliverables. This should be addressed in the future. Chapter 5. Sustainability If SARI/Energy is to be sustainable once USG financial support has ceased, then a critical mass of stakeholders in each social and economic sector must take ownership of the main precepts being advanced by the Program. This applies to Government officials, state corporations, industrialists, NGOs, consumer groups, farmers, bankers, academics and the media. All of these will benefit from the lasting economic benefits of regional energy trading, steady progress on sector reform, improved governance and a permanent, multitiered exchange of experience among SARI/Energy countries. The Evaluation has documented substantial early progress towards institutional sustainability: • The relevance and quality of capacity building is recognized by governments. • The proportion of local trainers is increasing and has already surpassed that of expatriates. • Identification of centers of excellences is complete and cooperative work has started. • Significant use is being made of local consultants (though intensive supervision and training will continue to be needed.) Through the interviews, the Evaluation Team became aware of competent NGOs, academic institutions and institutes, and consulting firms in SARI/Energy countries that already solicit and receive grants from national governments, the Global Environmental Facility (GEF), foundations in Western countries, multilateral development banks and bilateral donors (Canadian International Development Agency (CIDA), Australian Development Assistance Agency (AusAID), the Nordic Countries Development Assistance Program (NORAID), and the European Commission (EC)). SARI/Energy is already actively supporting some of these through the Partners, but more can be done in this area, both through future U.S. contractors and through “sustainability” grants. 5 See “Review of SARI/E Results Framework and Performance Monitoring Plan”, report by Rene Bernier, CDIE, November 2002 at Annex V. 13 The use of local consultants, NGOs and consulting firms can contribute to sustainability and a start has been made in this direction. However it is essential to use a sufficient amount of expatriate level of effort to ensure that the quality of work by local experts meets international standards. This aspect will need more attention in the future. Domestic private sector companies, as well as local subsidiaries of multinationals, other than commercial lending institutions, have no major role in the energy sector in SARI/Energy countries. This was the situation in 1999 and it is likely to remain this way for some time to come. However, ultimately the private sector has a major role to play in managing, owning and financing the energy sector. In all SARI/Energy countries the slow pace of reform, which results in higher than necessary electricity prices, negatively affects the private sector. In India the private sector, especially the Confederation of Indian Industry (CII), is already active in pushing the case for accelerated power sector reform. In Sri Lanka the private sector lobbies the Government on high electricity tariffs. SARI/Energy can enhance its sustainability and accelerate power sector reforms by offering national private sector groupings the right kinds of training and technical assistance. PART TWO Chapter 6. Recommendations for Full Implementation 6.1 Observations on Experience to Date The first years of the Program were intended to test the concept and certain approaches to delivering the support and assistance necessary to achieve Program objectives. The observations set out in the next section represent the conclusions of the Evaluators as a basis for their specific recommendations to USAID on the future of the Program. 6.1.1 Government Support Motivation and support for the Program from governments in the SARI/Energy countries varies from India, where Government attitudes are not consistent and are sometimes ambiguous, to the other five, where there is clear and articulated support for the Program. For five of the SARI/Energy countries the Program has created interest and enthusiasm for reasons of geography and history. SARI/Energy brings these five countries into a new energy working relationship with India, in which “India’s Rim”, consisting of Bangladesh and India’s smaller neighbors, has shared energy issues and interests. The SARI/Energy Five’s historical relations with India in energy and water have been bilateral and weighted towards India. Interviews with officials in Bangladesh, Nepal and Sri Lanka have shown that in SARI/Energy venues these “rim” countries discuss regional issues of common interest and we must presume that officials from Bhutan and Maldives also participate in such informal exchanges. Officials in Bangladesh, Nepal and Sri Lanka expressed the view that the GOI demonstrates an ambivalent attitude vis-à-vis SARI/Energy and a preference for handling energy trade questions in the context of bilateral relations. Their view was that the Program does not have a bright future unless the GOI is behind it. For Bhutan, the Maldives and Sri Lanka, SARI/Energy is additionally valued by Governments because these countries have no bilateral USAID energy programs, and their participation in SARI/Energy has brought major benefits in capacity building and direct impacts on national policies. In India, positive support for SARI/Energy was expressed by senior Ministry of Power officials, heads of energy sector state corporations and BSES Ltd. Officials in other ministries gave the Evaluators the impression of being indifferent. Activities in capacity building through training and partnerships are widely appreciated in India. SARI/Energy’s role in demonstrating to Indian officials Bangladesh’s successes in rural energy has been 14 acknowledged by GOI officials, which, thanks to the Program, intends to adapt Bangladesh’s practices to Indian rural energy supply. Issues involving energy trade have a major political dimension for the GOI and the Evaluators found no signs that the GOI is receptive to such questions being advanced in a multi-lateral forum, such as SARI/Energy. However, SARI/Energy may be able to play a role in changing this position. It may be that the GOI would decide to see SARI/Energy as a platform for establishing new energy relationships with its neighbors and to recognize the real benefits and increased goodwill that the GOI could gain by taking a proactive role in SARI/Energy. In a number of areas India is ahead of its neighbors (aspects of electricity regulation, unbundling, deregulation of petroleum product prices, testing and ranking of consumer energy using appliances) and could share its experience with SARI/Energy partners. The opportunity to create goodwill through SARI/Energy could be of interest to Indian policy makers. 6.1.2 The Role of ‘Stakeholders’ in the SARI/Energy Program Achieving SARI/Energy Program objectives will enhance regional stability and economic growth. From the outset, SARI/Energy was designed to be a Program driven by stakeholders. We were informed by USAID that at the start ‘stakeholders’ were defined as the senior officials in the public sector, which dominates the energy sectors in SARI/Energy countries. For the Evaluators, having met with nearly 200 individuals in four countries, it became necessary to examine the notion of ‘stakeholders’ because if the definition is incomplete then future program activities may be misdirected. The typical concept of stakeholders refers to all those parties that are linked to a major infrastructure project or socioeconomic initiative because they will be affected by the initiative, for better or for worse. Stakeholders may have the capability to accelerate, delay or deny the initiative. Seen in this light, senior civil servants as stakeholders are part of the process but they are not ‘agents of change’ in the sense of stakeholders who will make the decisions on major politically sensitive initiatives such as energy trading. As a group, they have no particular reason to support or advocate the high profile initiatives of the Program in the absence of instructions from their political leaders. The default condition of most bureaucracies is not to promote change but to maintain the status quo. The Program design process included intensive consultations with top officials in ministries and parastatal companies, and the great majority of them expressed their support for the concept being put forward by the U.S. Government. In the case of state owned corporations, the chief executives may favor regional trading initiatives but it is their ‘parent’ ministries that decide major policy changes. The Program sensitizes stakeholders to economic benefits achievable through regional cooperation. There has been an assumption that capability building and technical assistance studies would somehow lead to a “trickle up” effect within governments. Thus, by demonstrating that energy trading projects were technically and financially feasible, political-level decisions to implement regional projects would follow. Thanks to Program activities there are supporters of SARI/Energy objectives in energy ministries and in energy parastatal corporations, but decisions on major initiatives are taken in a number of ministries (external affairs, finance and energy) and politically sensitive issues are decided at the level of ministers or higher. Based on the interviews conducted the Evaluators suggest that it is now time to revisit the notion of “stakeholders” and consider whether the definition should be expanded to include those constituencies in society that stand to benefit from power sector reform, lower cost energy and effective delivery of affordable and reliable energy in rural areas. The Evaluation interviews suggest that Heads of Government and respective ministers of external affairs, finance and economic affairs will make the decisions on starting regional power trading or natural gas exports and such decisions will be as outcomes of complex balancing of various energy and commercial trade issues and domestic political considerations. Nevertheless, there are other segments of civil society that can be said to have potential as SARI/Energy ‘agents of change’ or stakeholders – energy consumers in industry, urban and rural 15 households and manufacturers of energy equipment. The Evaluation has shown that the absence of technical feasibility studies has not been the obstacle to political-level decisions on energy trade. Rather, it is the other way around, once the political level decisions are taken, implementation details will be worked out at the technical level. In the future, USAID should consider a broader group of stakeholders whose interests have not yet been examined and addressed. 6.1.3 Program Contacting Structure Typically, USAID places responsibility for the successful and cost effective implementation of its assistance projects on the shoulders of private firms, nonprofit NGOs or, in some cases, other public sector entities with specialized knowledge. The performance of these implementing entities is monitored by USAID officials who have the authority to correct any shortcomings in performance. Since Program startup this supervisory authority was shared between USAID Washington and the New Delhi USAID Program Management as a function of the contractual relationships between Partners and USAID. This shared oversight was also important because of political sensitivities and the need to keep USG officials close to the Program. As stated in Chapter 2, the multiple contract structure arose not by design but from the legitimate need to launch Program activity with a minimum of delay. While multiple contractors can be asked to use their best efforts to co-ordinate their work, integration is another matter. This Program could be more effective with an integrator who would also have the authority to reprogram resources. Even though the bulk of USAID resources were under the CTO based in New Delhi, if it had been deemed necessary to rapidly redirect resources from one Partner to another, or to bring in new expertise, the administrative delays would have been significant. The Evaluators have discussed the kind of structure that would be most suitable for the future Program, which may last five years or even longer. Suitability is considered in terms of three main criteria: (i) ensuring continuous Program integration with resources always being used to further the agreed objectives of the Program; (ii) responding in a timely manner to opportunities presented by major developments in SARI/Energy countries by redirecting resources within the Program; and (iii) ensuring that all resources are being used in the most cost effective way. The Evaluators do not have one specific proposal to achieve the above objectives, recognizing that USAID’s management and contracting practices are continually evolving. Options that might be considered are: • A single contract with the prime contractor being an entity specialized in training, and with specific requirements for the other skills to be procured from subcontractors and delivered in accordance with an overall Program work plan. To ensure that small firms with needed skills were included, specific set asides could be stipulated. The performance of the prime contractor, and bonus payments to be divided among all contract participants, could be linked to verifiable results and indicators negotiated annually with USAID. Disincentives to frontload the work could be considered. Cooperative agreements and grants could be framed to permit flexibility and to ensure integration. • If it is decided to pursue multiple contracts, a small Program “management support unit” might be set up using a small business or 8(a) firm that would not be considered a competitor or have a conflict of interest with the main contractors, and that would have the expertise to monitor the on-going and planned work throughout the Program. Precedents for this include the US-AEP. However, this path can be a costly solution; and its effectiveness can be limited because the support unit does not have authority to reprogram resources. Under both options, USAID and/or State Department officials would continue to be responsible for all policy￾level dialogues with governments. 16 6.1.4 Technical Assistance Technical assistance was conceived of by USAID primarily as a means of stimulating and supporting decision￾making and advocacy by stakeholders. Impacts to date are not commensurate with the resources used, and we suggest that a lesson from the first two years relates to the definition and use of stakeholders. Careful attention is necessary when ‘stakeholders’ are asked to specify the work carried out under Technical Assistance because although these individuals may be ‘potential SARI/Energy stakeholders’ they may have other motivations for requesting analytical studies. Their motivations may be quite valid in terms of national energy priorities but not in terms of SARI/Energy priorities. The Evaluators’ review of Technical Assistance to date suggests that, in the future, proposals for Technical Assistance should be subject to rigorous examination and approval at the stage of work plans and scopes of work to ensure that the following tests are met: • Does the study clearly advance SARI/Energy Program objectives? • Who are the ‘clients’ for this study? Are they influential SARI/Energy stakeholders or agents of change? • Have they been consulted about the SOW and the proposed methodology and have their views been taken into account? • Will they take ownership of the study’s methodology and conclusions upon completion? • How will the clients use the study? • Do the intended authors of the study fully understand the clients’ needs? 6.1.5 Mobilization of Private Sector Participation The private sector needs to be brought into SARI/Energy more effectively and more efficiently. Although the private sector is only involved to a small degree in the ownership and operation of energy sector assets, private sector groups in the larger SARI/Energy countries have the potential to become investors in new assets and, eventually, operators and/or owners of existing assets. At least in India, the Evaluators were told that the amounts of capital available to the large groups are sufficient to enable them to become partners with foreign capital. As was noted above, the indigenous private sector’s most important involvement with the energy sector in SARI/Energy countries is as a consumer of electricity, natural gas and petroleum products. Other areas of involvement are the manufacture of energy equipment and the participation of private commercial banks as lenders to the energy sector. In fact, the Evaluators learned that Indian financial institutions have ended up holding most of the debt from the Dabhol project. Industrial enterprises in Bangladesh, India and Sri Lanka have already invested heavily in standby power generation and in the case of large, continuous process industries, self-generation. Invariably, these solutions are less environmentally friendly and more costly than central station grid power would be if it were reliable and efficiently run. Thus, domestic industry would stand to benefit from sector reforms, which are being encouraged by SARI/Energy. This point is not lost on Indian industry, which has mounted its own program to accelerate reforms. There is no shortage of industry associations in South Asia, and the challenge for the Program has been how to work effectively with the various, and sometimes competing, groups to strengthen private sector support for SARI/Energy objectives. The Evaluators met with national chambers of commerce that are already active in SARI/Energy countries in lobbying and expressing views on power sector reform, electricity tariffs and expanding the role of the private sector. The Evaluators have concluded that, in the future, the Program should have a focused and cost-effective approach that supports existing bodies throughout the region. Such support need not place large demands on USAID resources because it would leverage existing efforts and use other Program resources, such as TA and Training, to provide industry groups and chambers of commerce with studies demonstrating the benefits obtainable from accelerated reform and regional energy cooperation. 17 Sustainability is a key Program goal and we suggest that local and/or existing regional private sector bodies should be taking ownership of the process of mobilizing private sector support in the future if it is to be effective, tailored to national practices and customs, and sustainable. In terms of the global private sector’s involvement in the energy industries of South Asia, the great need is for investment and lending in the electricity sector. As noted in Chapter 1, in this area foreign investors and developers have retreated from India, the largest market, and are only likely to return when reforms make the sector more financially viable. This adds to the importance of SARI/Energy in contributing to accelerated power sector reform. The Evaluators heard from a number of interviewees that all private sector training participants are required by USAID to pay their own travel expenses. The Evaluators disagree with this requirement and regard it as counterproductive to the goals of the Program. The Program should be reaching out to the private sector, not discriminating against it. One interviewee pointed out that SARI/Energy pays training expenses for large state￾owned corporations that could easily afford to pay travel costs. Instead of marginalizing an already marginal private sector SARI/Energy should be proactive and increase private sector involvement by dropping the private sector payment rule. 6.1.6. Energy Efficiency Promotion The effort now underway to achieve a multilateral agreement on the Standards and Labeling of Energy Appliances is nonpolitical and stands a good chance of becoming SARI/Energy’s first concrete project to reach closure. It is vital both for its direct impacts on improving end-use efficiencies, and also as a way of preventing substandard products from spoiling markets for new technologies, such as compact fluorescent lamps. The promotion of the energy service company (ESCO) concept should be pursued with caution and be attuned to specific national circumstances and requirements. The ESCO concept has been promoted as a universally effective delivery mechanism for energy efficiency, and it has failed in many countries because of combinations of high transaction costs, high interest rates, low energy prices, poor governance and problems of access to credit. “Hothouse ESCOs” which depend on grant funding may not be sustainable. ESCOs owned by equipment suppliers, such as the one in Nepal, face potential conflicts of interest in serving energy efficiency clients. Other models may be viable, such as electric utility-owned ESCOs, or the models that are being tried in India. Having reviewed the proposed SARI/Energy transposing of Indian energy efficiency legislation to Sri Lanka, the Evaluators would simply emphasize that where energy prices are high (as in Sri Lanka), emphasis should be placed on transferring market-based solutions for energy efficiency and not on State-directed bureaucratic mechanisms. 6.1.7 “Best Practice” and “Experience/Lessons Learned” Transfers Until the necessary political decisions are taken to implement regional energy trading, the highest impact that the Program can make is probably in the area of identifying and facilitating “trade in experience,” both positive and negative. SARI/Energy countries are all embarked in similar directions in the areas of power sector reform, delivery of rural energy supply, promoting energy efficiency, improving governance and improving the climate for private investment. After Chile unbundled and privatized its power sector in the 1980s, other Latin American countries studied and applied the positive lessons and sought to avoid the mistakes. SARI/Energy has already started to become the dynamic vehicle for transferring experience. The Evaluators examined the successes the Program has had to date in flowing “best practice” between the countries. As a by-product of this examination, it may be useful to briefly present our observations on the Program’s use of the concept of “best practice”. This conceptual tool was first developed in technical and engineering applications, where “best practices” in such subjects as Steam Boiler Operation, Building Energy 18 Management or Industrial Energy Auditing could be set out with precise and measurable parameters. In such areas, the concept was useful for transferring knowledge to practitioners with the objective of optimizing technical performance with a minimum of resources. However, in our view the concept must be used with caution when it is applied to non-technical activities such as power sector reform or rural energy supply or energy efficiency promotion. Successes in such fields usually depend on unique combinations of legal frameworks, culture, history, stage of economic development and other societal characteristics. If an analysis of national experiences with “soft” practices and policies in one country is to be most useful for other SARI/Energy countries, we suggest that SARI/Energy adopt labels that more accurately reflect the process of experience transfer: “good practice” or “elements of best practice” or “transferable lessons” or simply “experience”. 6.2 Specific Recommendations for Phase 2 6.2.1. Program Duration, Structure and Funding The Evaluators recommend that USAID should plan a Program extension with a five-year time span, and leave open the possibility for a further extension if conditions warrant. In the experience of the Evaluators, some USAID projects have in the past been ended some years short of being able to realize their maximum benefits. The SARI/Energy Program contains an unusually large number of elements that are beyond the influence of USAID or even the U.S. Government. We believe that it is important for the SARI/Energy concept to be given a reasonable amount of time to make lasting impacts in an environment where important changes cannot be anticipated. SARI/Energy has been impacted by the India-Pakistan tensions, by the civil war in Sri Lanka, by changes in Government in Bangladesh and by events in Nepal. But the new Sri Lankan peace process is an example of how positive, unanticipated developments can arise and from which SARI/Energy as whole can benefit. An Indian-Pakistan rapprochement could bring major new opportunities. Adding new countries, possibly Afghanistan, may also be possible in the medium-term. Program Funding. Having examined the funding levels for the first three years, we believe that for the following five years the Program can proceed with lower resource levels in all areas without reducing its impacts. Program Structure. Emphasis should be placed on a SARI/Energy Program concept of ‘building blocks”, with the option to add blocks as opportunities arise. SARI/Energy already has four building blocks under construction: Block 1: Energy Efficiency Standards and Labeling; Block 2: Rural Energy Supply; Block 3: Electricity Trading and Block 4: Power Sector Reform. Each “block” can be used to focus Training, Partnerships and Technical Assistance. Management Structure. USAID should create a contract management structure that is flexible and that incentivizes integration, the achievement of Program milestones and the continuous dedication of Program LOE to the advancement of SARI/Energy objectives. Expertise Required. The Program will, at minimum, need the same range of resources as is now offered by the Partners. Socioeconomic analysis skills will be needed. Performance Monitoring. USAID Program Management capability should be strengthened to monitor deliverables for quality and relevance. Consider refining the definition and verifiability of indicators. 6.2.2. Stakeholders The definition of stakeholders should be reexamined and expanded to include agents of change as the basis for structuring support activities. It may be appropriate to identify a set of stakeholders for each building block and to work with knowledgeable host country people to add stakeholders over time. 19 Parliamentarians. An Energy Parliamentarians Partnership could bring together key parliamentarians from energy committees from SARI/Energy countries, the U.S. House and Senate, and perhaps one other Western country with a parliamentary system (Australia?). The debates on the social and economic benefits of energy sector reform (electricity and hydrocarbon) and the challenges of implementation could be discussed in informal settings. TA resources could be used to develop concise information notes on how parliamentary bodies handled reform legislation and implementation in SARI/Energy and other emerging market countries, including countries with federal systems (Brazil). Influential Academics and other ‘Eminent Persons.’ In SARI/Energy countries one finds influential individuals who are advising ministers and prime ministers, through formal advisory instruments or informally. Typically, these are senior academics, heads of institutes, retired senior civil servants and corporate executives. Their names are well known, they write op-ed pieces for the quality newspapers, and they appear on television and speak at conferences. Among these persons are those who are already favorably disposed towards greater regional cooperation and who can be identified through SARI/Energy’s existing local contacts. Using the new SARI/Energy “sustainability” grant facility, short analytical pieces could be commissioned for publication in the media. Influential persons could be invited to open SARI/Energy events with media coverage and a reasonable honorarium for their time. Consumer Groups. Consumer groups, which do have organizations at various levels of sophistication in most SARI/Energy countries, have a strong interest in reliable and affordable energy. Electricity trading and power sector reform can help bring this about. Trade Unions. Trade unions have not been part of SARI/Energy, yet they are indispensable partners in power sector reform. This may be a sensitive question for SARI/Energy governments, but one could see a combination of technical assistance (socioeconomic impact studies demonstrating how all social groupings can benefit from reform) and partnership mechanisms with unions in other countries educating trade union leaders and mitigating opposition to reform based on a lack of knowledge. 6.2.3. Sustainability Sustainability may be increased via centers of excellence, other academic institutions, and use of competent local consulting firms. Sustainability may also be enhanced by more media training, involving trade unions and parliamentarians. Specific recommendations for enhancing Program sustainability include: • Use the new “grant” facility to bring in more NGOs and centers of excellence. NGOs with proven track records can implement SARI/Energy tasks. For example, an Indian NGO interviewed by the Evaluators has a ten year track record of testing and ranking consumer appliances for energy efficiency. Their protocols ensure high technical standards as well as high standards of governance. They could transfer their experience to other SARI/Energy countries and take part in the SARI/Energy effort to harmonize standards and labeling. Another example is a Nepalese NGO which offers off-grid villages a complete energy package in which the cost and installation of a photovoltaic system is paid for by the village women’s output of artisanal/craft products. • Consider setting up a regional energy data center, possibly in one of the centers of excellence, drawing on the existing EIA and ADB energy databases. Interviewees in all countries requested this. • Commission research papers and studies to serve TA objectives, incorporating peer review mechanisms that would ensure rigorous and high standards products. • Consider establishing graduate fellowships in regional energy studies for the duration of the Program, with the assistance of a U.S. firm or nonprofit experienced in this field. (In Sri Lanka, there is a local engineering consulting company (Environment and Management Lanka Ltd) which funds graduate fellowships in energy studies and then offers a job to the fellowship recipient.) • Encourage and support graduate university courses on energy reform at the centers of excellence. 20 • Make more use of local firms, as one Partner has done in Bangladesh and Sri Lanka. In both these countries, existing consulting firms have been engaged by the Partner to be their local operators. 6.2.4. Private Sector Participation Mechanisms Private sector participation mechanisms can be strengthened and reshaped by: • Refocusing the Program initiative to emphasize the role of existing regional, national and local bodies. • Expanding and refining the use of TA studies to inform the private sector on benefits obtainable from Program objectives. • Removing the requirement that training participants from indigenous private sector companies must pay for travel expenses connected to training courses. 6.2.5. Specific Program Elements • Technical Assistance should be tailored to the interests of target groups to advance SARI/Energy goals. Studies and reports need to be authoritative and of the highest professional standards. • Sources for transfer of lessons and experience should be extended to include Latin America and other Asian countries. The feasibility of standardizing Program experience transfer mechanisms should be examined. • SARI/Energy’s relationship with the media needs more attention in the future. o Contract with local press clipping services to provide vernacular and English language press coverage on SARI/Energy. o Continue educating the media about current energy issues with a 2-3 day course on the fundamentals of sector reform, IPPs and related governance issues, rural energy supply, etc. • SARI/Energy initiatives in energy efficiency should be confined to realistic and achievable actions. • If a Training Needs Assessment cannot be completed during 2003, it should be a first priority for the next period of the Program. Suggestions for new courses from interviewees, which are not a substitute for a needs assessment, include: o Energy Security for SARI/Energy Countries. This concept is talked about by India’s neighbors, but is not well understood. o Energy Primer for Media Practitioners. Power sector reform, IPPS and rural energy supply. Include vernacular press journalists from Bangladesh, Bhutan, Nepal and India. o Management of Unbundling. Strong interest from Sri Lanka and other SARI/Energy countries could benefit whatever stage they are at. o Governance and Advocacy in Power (energy) Sector Reform o Management Information Systems for the energy sector o Due Diligence Techniques for Lenders o Preparation of Requests for Proposals/ Bid Documents o Asset Valuation • Reduce intensity of capacity building in accordance with the absorptive capacity of each country. Some countries have little or no capacity left, while others have capacity. • Use regular SARI/Energy events, such as training courses, as opportunities for press coverage. Invite journalists and photographers to a short press briefing at the end of the week and issue a short press release on the nature of course and the people attending. • Publicize/advertise the SARI/Energy website in ‘South Asia Energy’ and other media to attract more users. Seek and arrange links to the website with other South Asia websites. • SARI/Energy News. On the assumption a Program extension will go ahead, launch a monthly SARI/Energy newsletter for all alumni, possibly contracted to an existing NGO for sustainability. Given that the website is not reaching a majority of the SARI/Energy community, a simple monochrome newsletter, which may be passed around, is likely to be a more effective method of communicating. The newsletter could have articles by SARI/Energy alumni, vernacular notes page, alumni news corner, and a quarterly page on retail energy prices in SARI/Energy countries (strong interest among interviewees). ANNEX I SARI/E Evaluation Methodology, Scope of Work and Interview Questionnaire 1. Evaluation Methodology The Evaluation used two sources of primary information as the basis for its findings. Interviews were arranged by USAID Washington and the four CCOs in consultation with SARI/E Program Management. All of the Partners were interviewed in Washington, and those with a presence in the field, were also interviewed on site. The Team developed an interview questionnaire (see below) to be used as a guide during interviews. This questionnaire was shared with USAID in draft form prior to the start of the interview process. For the field interviews the Team split, with Joanta Green and Shawkat Ali Ferdousi taking primary responsibility for Bangladesh and Nepal, while Niels de Terra and Mahendra Lama conducted the interviews for India and Sri Lanka. Additionally, Messrs. de Terra and Lama spent two days in Kathmandu carrying out supplementary interviews. Subsequently, each of the four Team Members prepared summary notes of the key points perceived by that Team Member for each meeting. Since two Team members were present for most interviews the individual preparation of summary notes served to provide a back-up means of ensuring that important points were not missed. All of the Partners provided the Team with ‘briefing books’ consisting of collected deliverables and annual work plans. The Technical Assistance Partner provided copies of all studies and reports produced during Phase 1, and the four Team Members divided the work of reviewing each one of these. Evaluations by training participants were also provided. USAID provided all other source documents related to the design and implementation of the Program. The PMP Evaluator, Mr. Bernier, kindly shared his notes from Evaluation interview meetings at which he was also present. An outline of the Final Report and an Evaluation Work Plan were drafted by the Team and given to USAID for review and comment. Suggested changes were incorporated into the outline and Work Plan. In order to ensure consistency and coherence, the actual drafting was carried out by the Team Leader using inputs from the other Team members, who were all asked to provide inputs on key crosscutting issues as well as specializing on specific areas. A Working Draft of the Final Report was reviewed and discussed at the December 2002 Fifth SARI/Energy Semi-annual Program Meeting. Subsequently, comments and suggestions were received from USAID and all Partners. 2. USAID’s Scope of Work for the Evaluation Team The Evaluation Team was given the following Scope of Work by USAID: “ARTICLE III---STATEMENT OF WORK This scope of work is for an evaluation of the overall program, the validity of the hypotheses underpinning the strategy, the appropriateness of the implementation strategy in reaching the program’s objectives, and the cost￾effectiveness and impact of activities implemented by the SARI/Energy partners (see Section I.C, above) in each of the SARI/Energy countries regarding strengthening institutional capacity, promoting information-based dialogue and coalitions related to energy sector reform, and the formation of regional networks for sharing of best practices in an effort to influence decision-makers engaged in deliberations over energy sector reform and cross-border trade. Not included in this scope of work are complementary activities funded by other USAID operating units (such as the bilateral Missions), nor those activities funded by other USG agencies. In particular, the evaluation will assess the SARI/Energy Program along the following criteria: RELEVANCE. Are the original hypotheses on which the program was designed still valid? Does the SARI/Energy program continue to respond to the needs in the region, and has the program positioned itself to take advantage of emerging opportunities to promote greater cooperation in energy and promote regional stability? In light of current needs and opportunities for regional cooperation in energy, is there a need for an extension of the SARI/Energy program to meet the program objectives, including a discussion of the Project Activity Completion Date (PACD), funding levels, areas of assistance/ intervention, and program management structure? EFFECTIVENESS. To date, have the program management structure and the adopted implementing tools (contracts, cooperative agreements and USG inter-agency vehicles) been effective in ensuring maximum coordination of activities under SARI/Energy so as to avoid duplication of effort? Are there gaps in coordination that potentially hinder the achievement of results or which fail to take advantage of synergies among individual activities? (e.g., are training and technical assistance activities coordinated and sequenced so that they benefit from each other?) EFFICIENCY. Are results achieved under SARI/Energy being produced at an acceptable cost compared with alternative approaches accomplishing the same objectives? What alternative approaches exist which could achieve results at greater efficiency and what mechanisms can be recommended for implementing the alternative approaches? IMPACT: What has been the impact of activities implemented under SARI/Energy under the first 18 months of the program? The evaluation team will conduct a performance evaluation of the partners in implementing their respective scopes of work. Are partners meeting their responsibilities under their contracts or grants? Are partners planning their individual activities with the broader objectives and sub-objectives in mind? How well have implementing partners worked as a team to coordinate work plans and activities in the interests of achieving the objectives of the overall SARI/Energy program? Have technical assistance, training, and partnerships been targeted at the appropriate beneficiaries to ensure the greatest impact in advancing the policy dialogue in support of regional energy cooperation? To what extent have the outputs from technical assistance, training and other SARI/Energy-funded activities been utilized by targeted beneficiaries? What evidence is there that best practices have been taken up by additional individuals who received information from targeted beneficiaries? To what extent have SARI/Energy activities supported or complemented activities sponsored by other donor partners, such as the World Bank and the ADB in South Asia? Measuring program impact requires the existence of good performance monitoring systems at the level of individual partners, as well as at the level of program management. The evaluation team will also investigate whether systems have been established internally for tracking, monitoring, and reporting on results attributable to SARI/Energy activities and whether these systems utilize independently verifiable information. SUSTAINABILITY. Are the results and impact of SARI/Energy activities sustainable in terms of creating institutional capacity and filling gaps on behalf of the programs key beneficiaries? What evidence has there been of host countries taking ownership of the SARI/Energy program, including promoting the networks and forums and advocating the best practices developed and disseminated under SARI/Energy? Based on results to date, are these activities likely to engender sustainable development impacts after USAID funding has stopped? PROJECT ASSISTANCE COMPLETION DATE (PACD) EXTENSION. The original program framework envisaged that SARI/Energy program would be implemented through a two-phased approach. The first phase objective was proof of concept with phase two following to implement structures and solutions to effect regional energy trade and cooperation. The SARI/Energy evaluation team should assess the need for extension of the SARI/Energy PACD to meet the overall program objectives. The team should also recommend the new timeframe and program focus/ direction in phase two.” 3. SARI/Energy EVALUATORS’ QUESTIONNAIRE GENERAL QUESTIONS 1 What has been your involvement in SARI/Energy? 2 Has your participation in one or more SARI activities increased your awareness of the benefits of linking energy activities in the SARI region? How? 3 Has your participation in one or more SARI activities improved your abilities to carry out your responsibilities? How? Is knowledge acquired from SARI activities being disseminated within your organization? How? 4 Has your participation in one or more SARI activities resulted in new on-going professional relationships between yourself and officials/executives in other SARI countries? Specifics? 5 Did you get enough out of the training or other SARI events to justify being away from your desk? 6 Do you discuss SARI and other regional energy issues with your colleagues? 7 As a consequence of SARI activities, what materials, ideas or procedures did you acquire or learn about that you plan to use or recommend for use in your organization? Cd-roms? 8 What would you change in future SARI activities? 9 Are there any potential regional energy activities that you believe should be considered by SARI, including expanded trade in manufactured goods related to energy? 10 With which SARI Studies are you familiar? How useful to you was this/these study? Did it make a difference? Can you point to any key issues or options identified in the Study? How useful is this report in helping you answer the questions that you need answered in order to advance this idea? Did the study leave out any areas that you would have liked to see analyzed? 11 As a result of SARI activities, please cite developments in other SARI countries that may have applications in your country? 12 Are you aware of any institutional linkages between SARI countries related to energy? Do you believe that there is a need for new linkages? 13 Why were you selected for participation in a training course? 14 Has your position changed since your first participation in a SARI event? Have your responsibilities changed? 15 Did language in any way diminish what you were able to get out of your participation in SARI training or other events? If YES, how could the trainers improve your access in the future? 16 Do you believe that more benefits can be obtained for your country from regional cooperation than can be obtained from bilateral relations? What should/could be done to advance regional cooperation? 17 What is your overall impression as to how well the SARI program is managed? 18 In your country are there norms or precedents that are missing in order to facilitate regional cooperation? PRIVATE SECTOR 20 How would your business be affected or enhanced by more regional cooperation? 21 Has your participation in SARI contributed to your own advocacy of regional cooperation or that of your company? 22 Which industrial or business associations are you/your company in? Is there an energy sub-group? Do they lobby for new laws or regulations or Govt actions? 23 How do you regard Foreign Direct Investment in your county? How do you regard IPPs? 24 What is your view of efforts to reform the energy sector? 25 Do you interact with business groups/associations in other SARI countries? 26 Are there any ways in which capacity building (to enhance regional energy co-op) in your industry could be enhanced? CCOs 27 Describe the process for selecting participants? Why are there so few participants from outside capitals? 28 Does the Host Government determine who may be nominated? 29 Have the on-going delays in approval had any negative impacts? 30 What do the stakeholders in your country think of SARI? Do they identify it more with USAID or more with regional co-op? 31 [For CCOs in Bangladesh, Nepal and Sri Lanka] Do you feel that you are fully integrated in the whole SARI program process? 32 What is your relationship with the host Government? With the USAID officers responsible for bilateral energy programs? USAID BILTERAL OFFICIALS 33 Do you see benefits to your program from SARI activities? Synergies? Is it complementary? Any duplication? 34 NGOs, THINK TANKS, ACADEMICS 35 In any of your advocacy activities have you used any SARI resources? Products? Are there any other tools that SARI could develop that would be of use? 36 Overall, is SARI a useful resource for you? 37 How could SARI facilitate or help promote civil society support for regional energy cooperation? 38 [for Academics] Have you or any colleagues done any research related to regional energy co-op? MEDIA 39 What are the on-going energy questions/issues that you cover? Reform? Consumer issues? Etc 40 Have SARI materials been useful in your reporting? Could SARI provide new materials that would be of use? Press releases? 41 Has “regional energy cooperation” been discussed in editorials in your paper? 42 Does your paper have a position regarding “regional energy cooperation”? 43 Did any of your reporting result in feedback or calls from officials. Businessmen, etc? ANNEX II Overview of Regional Trade Pacts in South Asia Dr. Shawkat Ali Ferdousi, Associate Professor Institute of Business Administration (IBA), University of Dhaka, Dhaka, Bangladesh Introduction South Asia was slow to take account of developments in the global economy. It did not start to think about a regional trade agenda until the early 1990s. The emergence of new Regional Trade Agreements (RTAs) in North America (NAFTA) and Latin America (MERCOSUR) and the expansion of existing ones in Europe (EFTA and EU) and East Asia generated implications for South Asia as a region in the global economy. For South Asian countries, in the absence of an entry point into one or more of these more dynamic regional bodies, the next option appeared to be to cooperate amongst themselves in the hope that it would create economic benefits in the medium to longer term. The seven South Asian countries in December 1985 formed a regional cooperation framework- South Asian Association for Regional Cooperation (SAARC). Three of the eight SAARC Charters pertaining to regional cooperation have been highlighted below: Convinced that regional cooperation among the countries of South Asia is mutually beneficial, desirable and necessary for promoting the welfare and improving the quality of life of the peoples of the region; Convinced further that economic, social and technical cooperation among the countries of South Asia would contribute significantly to national and collective self-reliance; Recognizing that increased cooperation contacts and exchanges among the countries of the region would contribute to the promotion of friendship and understanding among their people. In reality, SAARC had been able to do very little in respect of economic cooperation and achieve its overarching goal. The proposal to set up a South Asian Preferential Trade Agreement (SAPTA) was accepted by all seven member states of the South Asia Association for Regional Cooperation (SAARC) in 1993 and SAPTA came into formal operation in December 1995 with the ratification of the first round of tariff concessions. In 1996, SAARC member countries agreed in principle to go a step further and attempt to enact a South Asian Free Trade Agreement (SAFTA) by 2000, but not later than 2005. South Asia has made some efforts in expanding concessions under SAPTA. The second round of negotiations were completed in 1997 and a third round in 1998. In total, SAPTA concessions have covered nearly 4600 tariff lines out of a total of 6000 under the HS code system. Despite the apparent progress made in pushing ahead with SAPTA negotiations, the actual trade impact on the region’s economies remains nebulous; the trade coverage of SAPTA concessions have been limited while other Non-tariff barriers and bureaucratic hindrances have constrained countries from benefiting from the agreement. Political tensions in the region, particularly between that of its major partners – India and Pakistan – have only served to undermine the agenda in economic cooperation. The political impediments that South Asian Countries face are quite different to those faced by countries of other regional groups such as the EU and ASEAN. Although cooperation amongst the SAARC members are based on areas of mutual cooperation where ‘bilateral and contentious issues’ are specifically excluded from its deliberations, these very problems continue to plague and divide South Asia as a region. Trade Liberalization in South Asia But a process of liberalization albeit hesitant and uneven across countries was started. By the end of 1990s, though important policy barriers to trade and foreign investment remained, throughout the region substantial progress had been made in the direction of trade liberalization. But in a global comparison of import protection rates, South Asia remained a highly protected region (Blackhurst et al, 1996) as shown in Table-1. Table-1 Status of Trade Liberalization Measures in South Asia Criteria Bangladesh India Nepal Pakistan Sri Lanka Exchange Rate Unitary Unitary Unitary Unitary Unitary Exchange Rate Managed Managed Pegged Managed Free Float Determination Float Float Float Payment Restrictions Current Accounts No No No No No Capital Accounts Yes Yes Yes Yes Yes State Monopolies in External Trade Yes Yes Yes Yes Yes Import Licenses Yes Yes Yes Yes Yes Simplification of Tariff Yes Yes Yes Yes Yes Average tariff – 1995 (unweighted) % 42 48 17 51 24 Maximum tariff rate – 1997 45 40 110 45 35 Existence of High Level of NTBs No Yes No No No Source: Samaratunga (1999); Weerakoon (1998). Notwithstanding tariff reductions, most of the countries continue to maintain significant quantitative restrictions. Thus, despite the policy reform process, South Asian countries on still lags significantly behind in terms of its ‘openness’ to trade with each other as well as the rest of the world (Table-2). Table-2 Openness in South Asia and Southeast Asia Trade Share of GDP (%) 1980 1990 1999 South Asia Bangladesh 24 23.3 29.6 India 17 16.4 19.2 Nepal 30 24.4 43.8 Pakistan 37 36.5 32.7 Sri Lanka 87 64.5 76.9 ASEAN Indonesia 54 44.2 62.4 Malaysia 113 138.3 202.0 Philippines 52 49.6 99.2 Singapore 440 327.4 295.5 Thailand 54.5 70.0 107.4 Source: World Bank, World Development Report, various years. South Asian Economic Cooperation Historically, the South Asia member countries have not been trading among themselves to any significant extent. The low complementarities of products of the region’s economies as well as the trade policy regimes followed by South Asian countries have acted as inhibiting factors. As indicated by Table-3, intra-South Asia trade accounted for less than 4 percent of trade in South Asia for much of the 1980s but has gradually increased to 4.4 percent by 1999. Intra-South Asia exports accounted for 4.6 percent of South Asia’s exports to the world and the corresponding share of intra-South Asia imports was even lower at 4.1 percent in 1999. The share of imports from South Asia for the two largest economies in the region – India and Pakistan – remain disappointingly low at approximately 0.8 and 1.9 percent of their total imports respectively (Table-4), Bilateral trade imbalances in the region are, therefore, a contentious issue. With the exception of Pakistan, all other South Asian countries are running significant trade deficits with India. The low volume of intra-South Asia trade is a reflection of the fact that much of the trade of South Asian economies is with developed countries. The industrial economies of the EU, USA and Japan together accounted for the following percentage shares of trade in 1998: Bangladesh (85.5 percent); India (56.5 percent); Maldives (68.2 percent); Nepal (61.2 percent); Pakistan (60.1 percent) and; Sri Lanka (76 percent). Table-3 Intra-SAARC Trade Intra-South Asia Trade (US $ mn) World Trade of South Asia Countries (US $ mn) Share of Intra-South Asia Trade in World Trade of South Asia Countries (%) 1980 1210 37885 3.2 1985 1054 44041 2.4 1990 1584 65041 2.4 1995 4228 104159 4.1 1996 4914 111479 4.4 1997 4390 115961 3.8 1998 6073 121331 5.0 1999 5640 129738 4.4 Source: IMF, Direction of Trade Statistics, Yearbook 1999. The low intra-regional trade of South Asia as a percentage of its total with the rest of the world is probably one of the lowest among regional groupings in the world. In 1995, for example, intra-South Asia trade was 4.4 percent, which compared poorly with 63 percent for intra-European trade, 37 percent for North America and 38 percent for East Asia (IDCJ, 1996). Given that the share of intra-ASEAN trade with the rest of the world was 7 percent at the time of its establishment in the 1960, the low level of trade amongst the South Asian countries is a cause for concern in attempts to successfully push ahead an agenda for regional economic integration. Table-4 Percentage Share of Intra-Regional Trade in South Asia: 1999 Share of Intra-South Asia Exports in Total Exports Share of Intra-South Asia Imports in Total Imports Share of Intra-South Asia Trade in Total Trade Bangladesh 1.9 13.6 9.5 India 5.0 0.8 2.8 Maldives 5.2 2.5 3.3 Nepal 28.5 31.7 30.8 Pakistan 3.6 1.9 2.7 Sri Lanka 2.8 12.3 8.6 South Asia 4.6 4.1 4.4 Source: IMF, Direction of Trade Statistics, Yearbook 1999. Despite the low complementarities of trade in South Asia, the potential economic benefits of cooperation were considered attractive enough to prompt the establishment of SAPTA. The first round of concessions came into effect December 1995 where tariff concessions were exchanged on a total of 226 products under the HS Code system on a product-by-product basis. The second round also adopted the same negotiating method and was competed in November 1996 with the exchange of concessions on additional 1900 products. The third round of negotiations was completed in November 1998, combining both a product-by-product and chapter wise approach to include further concessions on nearly 2500 tariff lines at 6-digit level of aggregation. With the conclusion of the third round, over 4600 tariff lines out of a total of 6000 have been covered by preferential access (Table-3). It has to be noted that the total concessions offered to all member countries still remains very limited at only 1900 of a total 6000 tariff lines available for concessions. The major impediment to economic cooperation amongst the South Asia countries has long been recognized as being political rather than economic per se. In South Asia, political divisions, lack of confidence and conflicts have hampered the process of regional cooperation. Tensions between India and Pakistan, and the natural preponderance of India among the other South Asian countries, have hampered regional cooperation. The drift towards bilateral trade agreements in the context of efforts to promote economic cooperation at a regional level raises complex policy issues. In fact, there is little evidence of similar trends in other regional groups. The vast majority of regional blocs has started from an agreed base on the intensity or degree of cooperation and has progressed from there, taking collective decisions with regard to either the speed of integration or admission of new entrants to the bloc. In those situations, the members of the regional grouping set specific guidelines and time frames within which potential members have to meet targets in order to qualify for membership. This is clearly the path that has been followed by the more successful regional groupings such as the EU and ASEAN member countries. South Asia, on the other hand, is currently faced with the specter of countries within an existing regional trade arrangement moving outside of it to form bilateral agreements that are to some extent liberal than anything under the regional framework. First and foremost, bilateral arrangements can undermine broad support for the formation of regional cooperation. Secondly, it raises the issue of how bilateral agreements are to be treated vis-à-vis regional initiative. Will they be incorporated into the regional cooperation framework for example SAFTA process or will they stand as separate bilateral trade agreements? If they are to be incorporated, then they may have to be used as the starting base from which to begin negotiations. If not, they will exist as parallel trade agreements. The Indo-Nepal Free Trade Agreement (FTA), however, offers far more liberal terms to Nepal than does the Indo-Lanka FTA. If bilateral agreements are to continue alongside the regional cooperation initiative, this will mean that countries will have to contend with a multitude of bilateral and regional agreements – what some economists have referred to as a ‘spaghetti bowl’ of overlapping trade agreements. Whether South Asian interests will be best served by such arrangements is again questionable; it will mean a greater administrative burden, as well as less cohesion in marketing South Asia as a region to foreign investors. Regional Cooperation in Energy Although SAFTA (South Asia Free Trade Agreement) has made some progress, nothing as yet has happened on the regional power trading front. The Declaration of the Tenth SAARC Summit at Colombo in July 1998, which was to be followed by the Eleventh Summit at Kathmandu but got postponed due to the Kargil incident, does mention “development of specific projects relevant to the individual needs of three or more Member States under the provisions of Articles VII and X of the Charter” on Sub-regional Cooperation. But the mention of “three or more Member States” has a special significance to regional water resources development and this could open up the avenues of power trading between India, Bangladesh, Nepal and Bhutan. As of now power trading is purely on a bilateral basis and limited to India/ Nepal (150 MW) and India/Bhutan (300 MW) only. Existing Cross-Border Power Trade India/Pakistan: On and Off Despite the terrible carnage following the 1947 partition of India and Pakistan, about 20 MW of power trading continued between the two Punjabs of India and Pakistan at Jogindarpur. The 1948 Kashmir war did not affect this power trading and only in the early 1950s did the two countries themselves sever the interconnection at the border. Nearly fifty years down the road, the two countries (Atal Bihri Vajpayee/ Nawaj Sharif’s Lahore talks) again resurrected the possibilities of Pakistan exporting into India initially 300 MW of power with a further potential of 3,000 MW later on. The IPPs in Pakistan saw the huge power hungry market of northern India as their saviour. Of course, this was a win-win situation for all: for Pakistan, for the IPPs in Pakistan and for India itself. The wrestling match over the price of power, Pakistan’s demand for 7.2 US cents per versus India’s offer of 3 US cents per unit based on India’s refusal to pay the 4 cents capacity charge and 0.2 cents wheeling charge, terminated inconclusively when the “Kargil/Kashmir incident” erupted. Though the thawing of the two countries’ relationship was expected from the “Delhi talks” (Vajpayee/Musharraf), the “Bin Laden affaire” on Pakistan’s northern border has again pushed the power trading possibilities into “deep freeze” for some time to come. India/Bangladesh: Not on as Yet Despite India’s Eastern Regional Grid having a power glut of 3,000 MW and the western zone of Bangladesh suffering from a power deficit of 300 MW, there is as yet no power trading between the two countries. In February 1997, India and Bangladesh met at Delhi6 at the technical level and agreed on the following important points: • that the interconnections between the two countries would be synchronous and not radial, with the recipient country deciding on the point of interconnection • power export from a new 400/220 KV substation at Krishananagar/ West Bengal/ India to Ishurdi/ Kushtia/ Bangladesh through a 220 KV double circuit line and also from Shajubazar/ Comilla/ Bangladesh to Kumarghat/ Tripura/ India through a 230 KV double circuit line to be initially operated at 132 KV. • the quantum of power flow to be limited to 150 MW during normal conditions and during emergencies up to 300 MW with both import and export on a 24 hour basis with energy settlement on a quarterly basis at rates to be mutually agreed on • the ground wires of the two countries’ interconnection would incorporate optic fiber communication links • the period for energy exchange shall be for five years and • these interconnections could open up the possibility of forming the SAARC Grid. Though the above draft agreement is very substantive, actual trading of power between the two countries has not occurred so far. India/ Bhutan: Very much On India and Bhutan already have the largest regional power trade with about 320 MW of trading from the 336 MW Chukha and the just commissioned 45 MW Kurichhu7 hydro projects. On an average, Bhutan used to inject into the Indian grid about 1300 to 1400 million Units per year. This power trading with India constituted 42% of her gross national revenue8 in FY 1997/98. The power tariff has recently been hiked up to 1.50 Nu per unit (i.e. 3.3 US cents per unit) and with Kurichhu’s additional input, Bhutan could well be trading over 1700 million units annually and netting in a comfortable US$56 million into her coffers. With another 61 MW Basochu hydro project, this power trading will take a quantum leap in February 2005 when the massive 1020 MW Tala hydro project9 will come into operation. The 400 KV double circuit transmission lines linking Tala with the Easter Regional Grid will be continued through the Siliguri “chicken neck” and continue on to interconnect the Northern Regional Grid10 through the 400/765 KV HVDC transmission lines. The India/ Bhutan power trading is very much on. India/ Nepal: On but very Little 6 Prof. Dr Md Golam Mohiuddin/ Bangladesh – Globalization, Regional Cooperation in the Power Sector of SAARC Region presented at Regional Seminar on “Need for Formation of SAARC Grid for Inter-Country Power Transmission and Load Sharing” 7 Power Trading Scenario and Potential in South Asian Region – Indian Perspective, TN Thajur/ Chairman & Managing Director, Power Trading Corporation of India Ltd. 8 Bhutan 8th Five Year Plan 9 Power Trading Scenario and Potential in South Asian Region – Indian Perspective, TN Thakur/ Chairman & Managing Director, Power Trading Corporation of India Ltd. 10 Rakesh Nath/ Power Trading Corporation of India – Prevailing Situation in Indian Power Sector … Barriers and Benefits of Expanding Regional Power Trade. In the last thirty years of power trading, the quantum stagnated at 50 MW though recently both the countries agreed to increase this power exchange to 150 MW. The prevailing rate is about 4.5 US Cents per unit at the 132 KV level with an escalation of 8% per annum. Historically, during the last ten years, the balance of power trade11 has been in India’s favour with Nepal to import over 200 million units annually during the last four years. This chiefly emanated from the policy of pursuing the 400 MW Arun III for nearly a decade. With the commissioning of the 144 MW Kaligandaki hydroelectric projects at the beginning of 2002, Nepal is expected to turn this tide when it will have an annual surplus of about 1,000 GWH12. This will be possible only if Nepal can have access to the Indian market. But the existing power exchange is on radial modes and not synchronous. The existing two major 132 KV interconnections (Duhbi/Purnea and Gandak/Ramnagar) as well as the two more planned and agreed 132 KV interconnections (Dhalkebar/ Sitamarhi and Birgunj/Motihari) are to the power surplus Eastern Regional Grid. The only 132 KV interconnection to the power deficit Northern Regional Grid, Tanakpur/ Bareilly, is used to import the free 70 million units that accrue from the Mahakali Agreement. The planned and agreed Butwal/ Gorakhpur 132 KV interconnection with the Northern Regional Grid will help to trade Nepal’s surplus power. The much talked about 6,480 MW Pancheshwar Project, which could have set the ground breaking rules on regional water resources cooperation in the whole South Asia region, is almost in a limbo. So is the private sector led cross border power trading the 750 MW West Seti Hydroelectric Project. 11 Nepal Electricity Authority – A year in review FY 2000/01 12 Power System Master Plan for Nepal – 1998, ADB/ NEA ANNEX III The State of Energy Cooperation in South Asia Prof. Mahendra P. Lama Economic Adviser to the Chief Minister of Sikkim Professor of South Asian Economies Jawaharlal Nehru University South Asia region consisting of Bangladesh, Bhutan, India, Maldives, Nepal, Pakistan and Sri Lanka possesses a rich endowment of water resources, forest, coal, oil and gas. However, the traditional sources of energy viz., firewood, animal dung, crop residues etc. are still the only or major energy sources to the larger populace in an overwhelmingly rural South Asia. This region has rich hydro-power potentials. The per capita commercial energy consumption in the region continues to be quite low. The persistent shortage of energy has been a major factor in keeping the region at a low growth equilibrium and has hampered productive activities, social development and investment climate. The region's dependence on import of petroleum has been steadily increasing. The traditional pattern of energy consumption has put tremendous pressure on the forest resources of South Asia. Equally critical has been the skewed distribution of the available energy both within a country and across the region. A very significant portion of the society still do not have access to modern sources of energy which otherwise could have transformed the quality of life and work substantially. There has been a remarkable increase in the demand for power in South Asia leading to a huge deficit in the power supply. In South Asia, power generation and its supply for long remained a state monopoly. Respective governments owned, operated and regulated the power entities. The performance of the utilities therefore, remained far from satisfactory. Most of the power generating units remained highly dependent on the subsidies and other inputs provided by the state. The power distributing units lacked commercial independence. On top of the low tariff rate, high system loss and low collection from the consumers ultimately made these entities both defaulters and sick. The restructuring of power sector has therefore been, inevitable. These countries have now introduced massive changes in their policies on private sector participation in the power sector. They are seriously unbundling the utilities. The stakeholders are now seriously deliberating on regulatory framework, quality of power, standardization and trading in energy related products. Some of the countries have moved far ahead of others in this respect. What lessons could be learnt from the experiences of each other in the management of power sector in the post-liberalized scenario is a critical question in South Asia today. Similarly, how have the other regional blocks like South African Power Pool (SAPP) which extensively cooperate in energy sector dealt with such issues also need to be seriously looked into. On an average the regional demand for power has increased at an annual rate of 9 percent, whereas the supply side has recorded both smaller and erratic growth pattern. This has increasingly led to power cuts and rationing. A major chunk of demand could come from the rural areas. The seasonality factor in both generation particularly in hydel power and spread of demand is highly noticeable in the South Asian countries. This has in turn generated a lot of interest in the cross border power trading in the region. There are distinct and tangible benefits for South Asian countries to cooperate in the energy sector. Border region of Bangladesh, Bhutan, India and Nepal have significant scope for both power generation and marketing. The surplus generated by the hydro plants in Bhutan and Nepal coincides with seasonal peak demands in the supply-short countries – India and Bangladesh. There are very high possibilities of exchange of power both between and through India to Sri Lanka, Pakistan and even Maldives and Myanmar. Exports of gas both through pipeline and by wire from Bangladesh to other South Asian countries are also a high possibility. Cross border power trade already takes place widely between India and Bhutan and to a certain extent between India and Nepal. For Bhutan the power export has brought huge revenues, thereby making its economy robust and resilient. The West Seti power project in Western Nepal, involving a private agency for the first time, is a third type of power exchange which is likely to take place in the region. This indicates a changing paradigm of power exchange. However, currently the power trading is in its infancy in South Asia region. Whatever ‘trade’ takes place as of today is basically bilateral exchanges or apportioning of power from surplus areas to temporarily needy regions. Neither any Power Purchase Agreement (PPA), for the purchase of power by India from these projects, was signed between India and Bhutan/Nepal, nor any principle for fixing the rates for purchase of power has been evolved. In all the past cases, the tariff has been fixed mostly on the basis of negotiations and to a large extent determined by political consideration, diplomatic goodwill and convenience. In the emerging situation, from the Indian side Power Trading Corporation of India will be entrusted with the job of handling power trading with its neighboring countries. Projects having cross-border implications are more often treated on political lines rather than on commercial considerations. Though there are strong urge among the stakeholders (including the utilities and technical/professional wings of respective countries) to cooperate on the energy sector, the gas exports from Bangladesh, power exports from Nepal and Pakistan have all faced severe political blocks. There are quite a few reasons for this including politico-strategic-hegemonic impression and fear emanating from India’s sheer size and economic might. The institutional and human capacities and investment-technology gaps in the exporting countries along with the question of sustainability of such exports have often come up as stumbling blocks. Governments need to only provide enabling agreements covering the project and sector at large. Depoliticization of deals though hard to practice, will also do away with unnecessary national prejudices. This will facilitate focused deliberation on deals. The choice of a model to trade or exchange electric power between countries is a crucial issue. There are instances of international power trading mechanisms in some regions across the world. India has recently introduced the concept of Regional Power Trading System which will help various regions of India in reducing power deficit by transferring surplus power from another region. This concept of power pool within India can also be enlarged to cover the neighboring countries like Bangladesh, Bhutan and Nepal after the establishment of sub-regional power pool and necessary interconnections among these countries are put in place. Equally critical task is to build the capacities of the policy makers in power sector across the region by re￾skilling and reorienting them to the advantages of power trading. What the present day policy makers in the region lack are information, sensitization and the alternative options and ways to consider the projects and their implementations for a cross border cooperation. A number of organizations in the region and outside have been consistently working towards fostering the cooperation in energy sector in South Asia. This includes the technical and professional public sector organizations including Petrobangla, Power Grid and Power Trading Corporations of India, Electricity Authorities of Nepal, Sri Lanka and Pakistan. On the other hand international agencies like the World Bank. ESCAP, ADB, USAID, UNDP have also been fairly active in the last few years. The SAARC has set up a Technical Committee exclusively on energy sector cooperation under its Integrated Program of Action. In its very recent meeting held in Dhaka, it has made some significant headway. A number of studies have already been conducted on various aspects of energy cooperation in the region. These are conducted by research organizations Bangladesh Unnayan Parishad (Dhaka), Centre for Policy Dialogue ( Dhaka), Institute for Integrated Development Studies (Kathmandu), Centre for Policy Research (New Delhi) and TERI and premier universities like Jawaharlal Nehru University (New Delhi), Dhaka University (Dhaka), Quad-i-Azam University (Islamabad) and Tribhuvan University (Kathmandu). Some of these institutes and universities have played very active role in advocating the cooperation issues on both water and energy in the region. In fact, they have been in the forefront of providing policy suggestions for a number of successful agreements like that of Farakka Water Sharing between India and Bangladesh and Mahakli Treaty between India and Nepal. They have also adequately highlighted the issues of energy cooperation between India and Pakistan. The private sector role in the energy cooperation issues in the region is rather slowly emerging. This is both because their marginal role in the past in their respective national energy sector and overwhelming public sector domination in energy related activities. After the reforms are initiated in the energy sector in the last decade or so, the private sector could now play an active role both at the national and regional level. The SAARC Chambers of Commerce and Industries, an apex body of the federations of chambers of commerce and industries in all the South Asian countries is now emerging as major force of bringing the energy cooperation issues to the forefront. This body has been recognized by the SAARC Secretariat as an apex body which played a major role in the ongoing liberalization of intra-regional trade under SAPTA and SAFTA. ANNEX IV List of Persons Interviewed Persons Interviewed in Washington, DC Herbert Davis Chief Executive Officer South Asia Regional Energy Coalition US Chamber of Commerce Sharin Huda Program Manager US Chamber of Commerce Mark Simpson Project Director US Chamber of Commerce Cynthia A. Lowry Regional Program Manangement, SARI/E USAID Alexander Patico Senior Program Manager Institute of International Education Ahmad Ghamarian Executive Director, The Energy Group Institute of International Education Larry C. Lai Managing Director Institute of International Education Catherine Sawas Senior Program Coordinator Institute of International Education Ian Driscall Director Deloitte Touche Tohmatsu Emerging Markets Fraser Morrison Client Service Leader Deloitte Touche Tohmatsu Emerging Markets Allen Eisendrath Senior Manager Deloitte Touche Tohmatsu Emerging Markets Aram Zamgochian Associate Project Director US Chamber of Commerce Carol E. Mulholland, Project Manager Energy & Environment Training Program (EETP) Academy for Educational Development Wendy R. Aulakh, Program Officer Energy & Environment Training Program (EETP) Academy for Educational Development Mark Rodekohr, Director Energy Markets & Contingency Info. Div. US Dept. of Energy, Energy Information Admin Monisha Shah International Energy Analyst, US Dept. of Energy, Energy Information Admin Rene Bernier, Senior Research Analyst USAID Development Information Services Academy for Educational Development Vinod K. Shrivastava President CORE International Lois A. Varrick Corporate Vice President CORE International Sridhar B. Samudrala Deputy Program Manager, Engineer USEA Energy Partnership Program Tricia C. Williams Senior Program Coordinator USEA Energy Partnership Program Gordon H. West, Deputy Assistant Administrator Asia and Near East Bureau USAID James A. Bever Bureau for Asia and the Near East USAID Raymond Holton, Esq. Senior Consultant, International Energy Services NEXANT, Inc. Matthew W. Addison Senior Vice President & Director NEXANT, Inc. Russell Profozich Project Manager NEXANT, Inc. Bernadette M.G. Mitchell, Senior Desk Officer Office of East and South Asia Affairs Bureau for Asia and the Near East, USAID Persons Interviewed In Bangladesh Aftab Ul Islam President American Chamber of Commerce in Bangladesh A Gafur Executive Director American Chamber of Commerce in Bangladesh Munir Md. Zaman Manager and Training Coordinator AED/E Gen Consultants Ltd. Cleveland L. Charles, Chief Economic/Commercial Office, First Secretary U.S. Embassy Mohammad Shah Alam Joint Secretary, Power Division Ministry of Power, Energy and Mineral Resources Masud Rahman Managing Director Crane Limited Md. Quamrul Ahsan, Professor Dept. of Electrical & Electronic Engineering Bangladesh Univ. of Engineering & Technology Rezwanul Kabeer Director EnergyPac Power Generation Ltd. Md. Asaduzzaman Asad Staff Reporter The Financial Express Faruque Ahmed Special Correspondent Bangladesh Sangbad Sangstha (BSS) Charles Uphaus, Director Office of Econ. Growth, Food & Environment USAID/Bangladesh Engr. Syed Abdul Mayeed Chairman, WAPDA BHABAN Bangladesh Power Development Board A.N.M. Rizwan Managing Director Power Grid Company of Bangladesh Ltd. Humayun Rashid Managing Director EnergyPac Engineering Ltd. Md. Tauhidul Islam Member (PBS & Training) Rural Electrification Board Eng. A.I.M. Nurullah Manager (CNG Project) Rupantarita Prakritik Gas Company Ltd. Md. Kawsar Azizur Rahman General Manager, Regional Sales Division Titas Gas T&D Co. Ltd. Dipal C. Barua Managing Director Grameen Bank Bhaban Ahsan Ul Haye Country Coordinator, SARI/Energy USAID/Bangladesh Mohammad Rezwan Khan, Professor Dept. of Electrical & Electronic Engineering Bangladesh Univ. of Engineering & Technology Syed Sajedul Karim Chairman, Petrobangla Bangladesh Oil, Gas and Mineral Corporation Md. Abdul Halim Mollah Member (Engineering) Rural Electrification Board Khandaker Shahidul Islam, Secretary-in-Charge Energy & Mineral Resources Division Ministry of Power, Energy & Mineral Resources S.M. Shamsul Alam, Secretary-in-Charge Power Division Ministry of Power, Energy & Mineral Resources Bruce Mcmullen Senior Energy Advisor USAID/Bangladesh Md. Anisur Rahman Director (Finance) Power Grid Company of Bangladesh Ltd M. Amzad Hossain Assistant Director, Power Cell Ministry of Power, Energy, & Mineral Resources Engr. Abu Naim Mohammad Director, Inspection and Testing Rural Electrification Board, Dhaka Md. Showkat Hossain Director, Finance Dhaka Electric Supply Company Ltd. Mohafil Ara Ahmed Deputy General Manager In Charge (MIS Div) Bangladesh Oil, Gas & Mineral Corporation Engr. B.D. Rahmatullah Director, Power Cell Ministry of Power, Energy & Mineral Resources Engr. A.S.M. Alamgir Kabir Director (Planning & Tariff), Power Cell Ministry of Power, Energy & Mineral Resources Shah Alam DCCI Representative Crane Ltd A.S.M. Bashirul Huq Senior Energy Specialist Persons Interviewed in India Nims Sharma Office Mgr., Coordinator Logistics & Training Academy for Educational Development Mercy K. Thomas Regional Project Management Assistant USAID/SARI K.K. Gandhi Senior Advisor Society of Indian Automobile Manufacturers S. Padmanaban Senior Energy and Environment Advisor Office of Environment, Energy & Enterprise S.S. Dua Acting Chairman & Managing Director BSES Limited Dr. Ashok Sarkar, Project Manager Energy Conservation & Enterprise USAID/SARI Robert W. Beckman Regional Coordinator and Program Manager USAID/SARI N.V. Seshadri, Program Specialist (Energy) Office of Environment, Energy & Enterprise USAID/New Delhi Richard L. Edwards, Director Office of Environment, Energy & Enterprise USAID/New Delhi S.K. Vikram Deputy General Manager-Commercial Vehicles TATA Engineering S Venkataraman Vice President-Technical Hindustan Motors Limited Ashok Saxena DY. General Manager (Product Engg.) Bajaj Auto Ltd. R Devarajan Special Director, Product Development Ashok Leyland Tantra Narayan Thakur Chairman & Managing Director Power Trading Corporation of India Don Priestman Chief of Party Academy for Educational Development D.N. Raina Deputy Chief of Party Nexant Hugh McDermott Chief of Party Nexant Dinesh M. Wahi Director, Logistics Support Manager CORE International P.K. Srivastava Regional Training Manager CORE International Pushpinder Dhillon Economic Officer U.S. Embassy Uma Ravikumar, Project Mgmt. Assistant Office of Environment, Energy & Enterprise USAID/New Delhi William Martin Deputy Director USAID/New Delhi Kavita Sinha Deputy Reg. Program Manager, SARI/Energy USAID/New Delhi Krishan Kaira, Addl. Secretary General Federation of Indian Chamber of Commerce and Industry (FICCI) Vivek Pandit, Assistant Secretary SARI Energy Committee FICCI H.K. Awasthi, Manager (Legal) Voluntary Organisation in Interest of Consumer Education (VOICE) H. Wadhwa, Manager (Tech. & Admin.) Voluntary Organisation in Interest of Consumer Education (VOICE) D.P. Sinha Member Central Electricity Regulatory Commission K. Venugopal Chief (Finance) Central Electricity Regulatory Commission Sulochana Muralidharan Senior Manager (Finance) National Thermal Power Corporation, Ltd. Chinmoy Chattapadhyay Chief Manager (LNG), Corporate Office Indian Oil Corporation Limited N. Biswas General Manager (IA), Corporate Office Indian Oil Corporation Limited R.V. Shahi Secretary Ministry of Power Arvind Jadhav Joint Secretary Ministry of Power Ajay Shankar Joint Secretary Ministry of Power Sharmila Chavaly Director Ministry of Finance Anya Roy Deputy Director Ministry of Finance Mahendra Kumar, Executive Vice President Business Development Power Trading Corporation of India Ltd. Rakesh Kumar, Addl. Vice President Client Service – Cross Border Power Trading Corporation of India Ltd. A. Mohan Menon, Addl. Vice-President Corporate Development Power Trading Corporation of India Ltd. Sanjeev Mehra Addl. Vice President Power Trading Corporation of India Ltd. V.S. Ailawadi Former Chairman Haryana Electricity Regulatory Commission Rajeev Mathur Deputy General Manager (Marketing) Gas Authority of India Ltd. Rajan Ghosh Addl. General Manager Gas Authority of India Ltd. Pat V. Sonti Director Energy Resources & Generation (India) Pvt. Ltd. V. Raghuraman Senior Adviser – Energy & Technology Confederation of Indian Industry (CII) Anant V. Naik Deputy Director – Energy Confederation of Indian Industry (CII) H.L. Bajaj, Chairman & Ex-Officio Secretary Central Electricity Authority Ministry of Power S. Vijayaghavan Joint Secretary Ministry of Petroleum & Natural Gas Tarun Shridhar Director Ministry of Petroleum & Natural Gas D.N. Raina Deputy Chief of Party, SARI/Energy Nexant K. Ramanathan Senior Fellow TOH R.P. Singh Chairman & Managing Director Power Grid Corporation of India Ltd. Anupam Sircar Deputy Chief of Party, USAID/SARI Academy for Educational Development Bokool Khanderia Member – Governing Council Congeneration Association of India Pat V. Sonti, Hon. Secretary & Treasurer Chairman-Energy Committee American Chamber of Commerce in India Persons Interviewed in Nepal Ram Bhakta Karki Director, Operations Department Butwal Power Company Ltd. Lekh Man Singh Director General Dept of Elect Devt, Ministry of Water Resources Bishnu B. Thapa Joint Secretary, Policy, Planning & Environment Ministry of Water Resources Mahesh P. Acharya Country Coordinator USAID – SARI Khem N. Poudyal, Assistant Director Center of Energy Studies Institute of Engineering Anthony B. Carvalho Hydropower Specialist USAID/Nepal Kedar Khadka Director, Good Governance Project Pro Public Keshab Poudel Managing Editor Spotlight The National Newsmagazine Sanjaya Dhakal Senior Reporter Spotlight The National Newsmagazine Narayan Prasad Shrestha Deputy Director, Centre for Energy Studies Institute of Engineering, Tribhuvan University Dr. Janak Lall Karmacharya Managing Director Nepal Electricity Authority G. Krish Krishnan Chief of Party Private Sector Hydropower Development Project -2 Sandip Shah General Manager Bhote Koshi Power Company Kush Kumar Joshi Chairperson, Energy Committee Fed. Of Nepalese Chambers of Commerce & Industry Er. Kiran Malla General Manager Butwal Power Company, Ltd. Prachar M.S. Pradhan Deputy Managing Director Nepal Electricity Authority Navin Singh Khadka Executive Editor Space Time Today Kanak Mani Dixit Editor/Publisher Himalmedia Pvt. Ltd. (HIMAL) Kunda Dixit Editor Nepali Times Dwarika Nath Dhungel, Ph.D. Executive Director Institute for Integrated Studies Sapana Shakya President Himalayan Light Foundation Adam Friedesohn Founder/Chief Advisor Himalayan Light Foundation Ashok Pande Coordinator Himalayan Light Foundation Binod Bhattarai Neapalese Federation of Environmental Journalists Correspondent Financial Times Persons Interviewed in Sri Lanka W.A.L.W.A. Perera General Manager LECO Rahula A. Attalage, Senior Lecturer Department of Mechanical Engineering University of Moratuwa Priyantha D.C. Wijayatunga Professor in Electrical Engineering University of Moratuwa Carol Becker, Ph.D. Mission Director USAID/Sri Lanka Teresa L. Manlowe Economic Officer/Commercial Attaché U.S. Embassy Sunith Fernando Director Resource Management Associates (Pvt.) Ltd. Dr. Talak Siyambalapitiya Managing Director Resource Management Associates (Pvt) Ltd. Vesana Wickremasena Deputy News Editor Daily Mirror Dammika Nanayakkara Manager – Engineering Lanka Transformers Ltd. Rishdha Zarook, Legal Advisor Bureau of Infrastructure Investment Division of the Board of Investment of Sri Lanka Dammika Nanayakkara Manager – Engineering LTL Energy (Pvt) Limited Kandasamy Naresh Kumar SARI/Energy Project Nexant Upali Daranagama Project Management Specialist (Energy) USAID/Sri Lanka Maxwekk Tissera, Addl. General Manager Distribution Operations Ceylon Electricity Board Bandula S. Tilakasena Deputy General Manager (Demand Side Management) Ceylon Electricity Board Avanthi Jayatilake Chief Executive Officer Environment & Management Lanka (Pvt) Limited M.F.M. Fallil Country Manager – Sri Lanka & Maldives AED/EML (Pvt) Ltd. Asoka N. Abeygunawardana Programme Coordinator Energy Forum Lalith Gunaratne Managing Director LGA Consultants (Pvt) Ltd. Gamini Senanayake Managing Director Wayamba Export & Trading House (Pvt) Ltd. Nihal Abeysekera, Chairman, Japan Lanka Ind. Development Centre and Sr. Vice President, Fed. of Chambers of Commerce & Industry Kosala Wickramanayake Managing Director, Gemini Service Master (Pvt) Ltd. Ananda P. Mallawatantri, Ph.D., Director US-Asia Environmental Partnership Program USAID/Sri Lanka Ravi Dassanayake Senior Project Executive DFCC Bank Piyal D. Hennayake Assistant General Manager, Development Banking Hatton National Bank Deepal Peiris Deputy Manager, Environmental Unit National Development Bank (NDB) Shavindranath Fernando, Project Director Upper Kotmale Hydro-Electric Power Project Ceylon Electricity Board Michael A. Warnakulasoorlya Additional Secretary Ministry of Power & Energy Rohitha M. Thanthilage, Consultant Industrial Services Bureau (ISB) North Western Providence Gamini Senanayake Director Industrial Services Bureau (ISB) Chathiyangani Kulatunge Additional Director Department of National Planning J. Nandakumar Chief Engineer/Operations Audits Ceylon Electricity Board Mr. Y.L. Farook System Development Manager Lanka Electricity Company Limited Mr. H.D.S. Thimothies Chief Engineer/System Control Ceylon Electricity Board Jayampathie Dewasurendra Assistant Director (Electrical & Electronic Eng.) Sri Lanka Standards Institution (SLSI) Gayathri Gunaruwan Economist The Ceylon Chamber of Commerce ANNEX V Report on Performance Monitoring Plan (PMP) Indicators by René Bernier, CDIE Review of SARI/Energy Results Framework and Performance Monitoring Plan Rene Bernier Senior Research Analyst, Asia Near East Bureau Development Information Services Project October – November 2002 Objective and Methodology Measuring program impact requires the existence of good performance monitoring systems at the level of individual partners, as well as at the level of program management. This report investigates whether systems have been established internally for tracking, monitoring, and reporting on results attributable to SARI/Energy activities and whether these systems utilize independently verifiable information. The report assesses the current indicators utilized for performance reporting against the six sub-intermediate results, including issues of appropriateness of indicators in measuring results, data collection and reliability, and management and reporting usefulness. This PMP assessment focused on the following issues: • What has been the experience with the PMP developed in 2000/2001? • Have the indicators been useful in providing information for program managers. • Are they providing useful information for reporting purposes? • Are reliable and objectively verifiable data available? • Are indicators reporting beyond the input/output level and addressing significant results? If not, what is the prospect for reporting on results-level data that will be measurable and attributable to USAID effort?13 Conducted between 16 October and 6 November, 2002, this assessment drew principally on discussions with USAID/New Delhi-based program management staff, partner organizations (contractors and grantees CCOs in the USAID missions in Bangladesh, India, Nepal, and Sri Lanka. The assessment also benefited from attending meetings with SARI/E participants and beneficiaries under the auspices of the mid-term evaluation. These meetings with participants served to provide supporting evidence that the program is achieving the results envisaged in the results framework. Results Framework and Performance Indicators The strategic objective (now the intermediate result)14of the SARI/Energy program is “Improved policies and agreements for cross-border cooperation on sustainable energy.” Towards this end the SARI/Energy program builds upon processes already underway on reform and engages governments, the private sector, and civil society groups in South Asia to work on a regional level to advance sustainable energy development. USAID believes that a regional approach accelerates progress by fostering cross-border dialogue and the exchange of ideas, creating and strengthening champions for reform, and building trust and confidence. Given the steps necessary to create an environment in which South Asian countries can cooperate on energy matters and eventually trade energy, the SARI/Energy program promotes sustainable regional development through the achievement of the six (sub-)15 intermediate results: • Private sector energy plans and investments in infrastructure that facilitate regional energy exchange developed; 13 See Annex I for the scope of work for the assessment of the results framework and performance monitoring plan. 14 SARI/Energy was initially Strategic Objective 31 (SO 31) of the Bureau for Asia and the Near East (ANE). In April 2001 SARI/Energy was combined with two other sub-regional activities to form SO 23: Encourage Economic Growth, with SARI/Energy then reporting as Intermediate Result 3 (IR 3) of SO 23 (498-023-003). 15 The initial Intermediate Results became sub-Intermediate Results during the “collapsing” process described above. • Policies, laws and regulations developed or revised, with private sector and civil society concurrence, to facilitate regional energy trade and cooperation; • Increased capacity of regulatory bodies to support energy exchange; • Adaptation of best practices and standards in the region that lead to energy efficiency and improved commercial practices; • Adaptation of best practices in the region that lead to improved rural energy services; and • Increased private sector participation in and civil society support for sustainable energy development. Program activities contributing to the above-intended results include: • Strengthening of institutional capacity; • Promotion of information-based dialogue and coalitions related to energy sector reform; and • The formation of regional networks for the sharing of best practices, in an effort to influence decision-makers engaged in deliberations over energy sector reform and cross-border trade. Assessment A. Results Framework SARI/Energy program managers and implementing partners (contractors and grantees) regard the results framework as a useful tool for maintaining focus on the program’s objectives of promoting trade in best practices and, hopefully, eventual trade in energy within the South Asia region between two or more countries. USAID and its SARI/Energy implementing partners continue to believe that the results capture the to-date and planned impacts of SARI activities. The Mid-term Evaluation will shed further light on the extent to which these results are being achieved across the program. The results framework does not capture one of the most significant results achieved to datenamely, the facilitation of networking and dialogue among energy sector players in the region achieved through SARI/Energy’s convening power in the form of workshops, executive partnerships, and training events. Many participants in SARI/Energy events noted this as a key result of the program, and one that is unique in the region. While this accomplishment is observable, it is by no means easily measured. There is substantial anecdotal evidence, however, that this networking is leading to cross learning among participants. Based on feedback from participants, this networking is gradually resulting in the adoption/adaptation of regional and/or common best practices, such as standards and labels, sharing of grid codes, model power purchase agreements, and rural electrification models, to name a few examples. B. Performance Indicators and Performance Monitoring System 1. Overview As listed in Appendix B, the performance indicators monitor results though measures that count the numbers of policies, agreements, laws, regulations developed or adopted, models and best practices developed and/or adapted resulting from SARI/Energy-funded activities, and documents or agreements produced that facilitate energy exchange. USAID developed the performance indicators in collaboration with the implementing partners in early 2001. The indicators were designed to draw on activity-level indicators developed by the implementing partners.16 The CCOs developed performance indicator baselines and targets based on key energy sector milestones in each of the four SARI countries and the contributions of SARI activities to their attainment. CCOs track and document these contributions through feedback from SARI program beneficiaries as to how training, technical assistance, peer exchanges, and partnerships contributed to the achievement of results. Physical documents are also used to verify the adoption of improved policies, laws, and regulations, and adoption of best practices that are attributable to SARI/E activities. 2. Indicator Definitions and Reliability Issues Do the performance indicators capture the intended results of the SARI/Energy program? Performance indicator definitions are generally broad. Performance indicators were designed to count the number of instances where SARI/Energy can take some or all the credit for producing a higher level outcome that reflects progress towards regional trade either in best practices and standards, or actual energy trade. The broad definitions facilitate the tracking of a wide range of achievements that reflect the multiplicity of policies, laws, regulations, and best practices being discussed, promoted, and adopted. Are the indicators providing useful information for performance management and reporting? Because the indicators count different types of accomplishments under a given intermediate result, performance data (targets and actuals) do not tell a performance story by themselves, either terms of monitoring progress or annual reporting. The performance narratives that accompany these indicators provide the details behind the indicators. Given SARI/E’s scope and time horizon, there are no alternatives to these types of indicators. Are the performance indicators objective? Whether the indicators are objective depends on the extent to which there is ambiguity about what the indicator measures. SARI activities promote the development and adoption/adaptation of regional and international best practices, and the adoption of policies, laws and regulations. The indicators that track the development or adoption of policies, laws, or regulations are unambiguous, provided that there is a prior determination that these policies, laws, and regulations represent progress and reform. There is also ample evidence of adoption and adaptation of what are considered to be best practices. There may be some ambiguity as to the extent to which what are being called best practices merit their designation as such. USAID may need to adjust what are being counted as best practices if there are serious doubts. Alternatively, what are being called best practices may simply be improved practices. 17 USAID program management should decide whether the latter designation would suffice. The Mid-term Evaluation may also shed some light on these best practices. Are the performance indicators are verifiable? The feedback from program beneficiaries as well as documentation produced by implementing partners and CCOs are sufficient to verify the attribution being claimed with each indicator. 16 Annex B presents the intermediate results, USAID indicator for those IRs, and indicators developed by each of the implementing partners. 17 This distinction was made by one of the participants in SARI events. He made the point that many government officials do not have substantial prior knowledge of international models and practices and may not be in any position to judge whether they are the best practices. Instead, government officials adopt/adapt new practices because they appear to produce better results than those practices previously employed by their organizations. Performance Data Tables Appendix D contains updated performance data tables that will be submitted in the FY2003 Annual Report. Estimates of 2002 actuals were collated from the CCO milestones where SARI/Energy made a contribution and where participant feedback or program documentation provided validation. Targets for 2003 were developed based on discussions with CCOs and selected implementing partners and are detailed in Appendix D. Summary and Conclusions The assessment of the SARI/Energy performance monitoring plan arrived at the following conclusions: • The results framework reflects intended results of SARI/Energy. There are good indications that these results are being achieved. • The performance monitoring system that combines tracking of milestones by CCOs with activity-level reporting by implementing partners is working. • The performance indicators developed in early 2001 are adequate. They capture the results envisaged under the results framework. In addition, they are verifiable and objective. The only caveat concerns whether what are being reported as best practices are indeed best practices. • The performance indicators, because they are broad, do not tell an interesting performance story on their own. The performance narrative details tell the story. The assessment documented the results achieved during 2002 in relation to each performance indicator. This should facilitate performance reporting in the Annual Report. The assessment also identified the results anticipated for 2003 in relation to the 2003 targets in the performance data tables. Appendices A. Scope of Work B. Results Framework and Indicators C. Performance Data Tables D. List of 2003 Performance Indicator Targets E. List of Persons Contacted and Consulted Appendix A Scope of Work for Results Framework/PMP Expert USAID will provide separately for the services of a Results Framework/Performance Monitoring Plan (RF/PMP) expert to contribute to the Performance Monitoring Plan (PMP) Indicators Appendix to this Mid-Term Evaluation of the South Asia Regional Initiative for Energy Cooperation and Development (SARI/Energy), with one page of text summary in the main body of the report. The evaluation team supplied by the contractor is expected to coordinate closely with the USAID RF/PMP expert, and to share information obtained throughout the evaluation period on interim results for incorporation into the PMP Indicator appendix. Objectives: The RF/PMP specialist will assess the program’s existing results framework and performance monitoring plan and make recommendations for possible modifications. The PMP assessment will focus on the following issues: • What has been the experience with the PMP developed in 2000/2001? • Have the indicators been useful in providing information for program managers. • Are they providing useful information for reporting purposes? • Are reliable and objectively verifiable data available? • Are indicators reporting beyond the input/output level and addressing significant results? If not, what is the prospect for reporting on results-level data that will be measurable and attributable to USAID effort? Tasks: • Review approved results framework and indicators and SARI/Energy performance reporting under R4/Annual Report. • Meet with Washington, DC based USAID and partner staff to discuss their perspectives on results reporting. • Meet with USAID/New Delhi-based program management staff and New Delhi-based project staff to collect information on data collection and reporting, usefulness of performance indicators. • Meet with SARI/Energy coordinators in the four USAID missions to collect information on data collection and reporting, usefulness of performance indicators. Reports and Deliverables: PMP specialist will write a report of 10-15 pages, including a 2 page summary, which will be attached as an appendix to the SARI/Energy evaluation report. This report will summarize findings on the current indicators utilized for performance reporting against the six sub￾intermediate results, including issues of appropriateness of indicators in measuring results, data collection and reliability, and management and reporting usefulness. Appendix B SARI/Energy Results and Proposed Indicators Result IR 1 – Private sector energy plans and investments in infrastructure that facilitate regional energy exchange developed IR 2 – Policies, laws and regulations developed or modified, with private sector and civil society concurrence, to facilitate regional energy trade and cooperation. IR 3 – Increased capacity of regulatory bodies to support energy exchange IR 4 - Regional adaptation of best practices and standards that lead to energy efficiency and improved commercial practices IR 5 – Regional adaptation of best practices that lead to improved rural energy services IR 6 – Increased private sector participation in and civil society support for sustainable regional energy development USAID Indicators Indicator 1 – International best practices identified and adopted in private sector investment plans to create capacity in cross-border infrastructure that meets regional needs Indicator 1 – Policies, laws, and regulations or procedures developed or adopted that facilitate regional energy trade Indicator 2 – Revised policies, laws and regulations reflect private sector/ civil society concerns Indicator 1 – Regulatory best practices developed or adopted by one or more countries Indicator 2 – Regulatory decisions reflect private sector and civil society input Indicator 1 – Number of best practices in pricing, transmission, and distribution developed or adopted in one or more countries Indicator 1 – Number of best practices in rural energy delivery developed or adopted in one or more country Indicator 1 – Programs and policies in place by business and professional associations that advocate regional cooperation, reform and sustainable energy development Nexant Indicator 1 - Number of commercial documents drafted and regional gas assessments completed to Indicator 1 – Number of forums held and recommendations made to develop/modify Indicator 1 – Number of initiatives developed/adopted by regulatory bodies that support regional Indicator 1 - Number of processes/ initiatives identified and addressed to facilitate harmonization of Indicator 1 - Number of best practices identified that offer potential for transfer that promote increased Indicator 1 - Cost of poor quality of service to economy and end-use sectors. Indicator 2 - Number facilitate infrastructure investment Indicator 2 – Number of energy constraints identified and assessed Indicator 3 – Number of forums held to address issues laws, regulations and agreements to facilitate regional energy trade and private sector project development energy trade and development of projects with private sector participation regional energy efficiency standards and practices. Indicator 2 - Number of practices adopted availability of commercial energy services to rural and/or low-income consumers. Indicator 2 - Number of practices adopted. of local/regional entities that adopt results to promote energy sector reform. AED Indicator 1 - Number of policies, laws and/or regulations in development, adopted or modified that address issues or include information covered in training. Indicator 2 - Number of cases where participants have utilized knowledge or skills gained from training towards the development, adoption or modification of policies, laws and regulations that facilitate regional energy trade. Indicator 1 - Number of regulatory practices that may be or have been adopted that address issues covered or use skills gained in training Indicator 2 - Number of cases where participants have utilized knowledge or skills gained from training towards adopting best practices. Indicator 1 - Number of cases where participants have utilized knowledge or skills gained from training towards adopting best practices. Indicator 1 - Number of cases where participants have utilized knowledge or skills gained from training towards advocating the development or developing programs or policies supporting regional cooperation and sustainable energy development. CORE Indicator 1 – Number of best practices in rural energy delivery adapted to and implemented in more than one country Indicator 1 – Programs and policies in place by business and professional associations that advocate regional cooperation, reform and sustainable energy development USEA Indicator 1 – Number of policies, laws, and regulations or procedures developed, modified, or implemented that include information gained from participation in SAREPP partnership Indicator 1 – Number of regulatory best practices adopted that are modeled from information gained from participation in SAREPP partnership Indicator 1 – A policy, guideline or procedure modeled from information gained through participation in SAREPP partnership. Appendix C – Performance Data Tables Performance Data Table Intermediate Result 23.3: Improved policies and agreements for cross-border cooperation on sustainable energy Objective Name: Encourage Economic Growth Objective ID: 498-023 Approved: 01/27/2000 Country/Organization: ANE Regional Result Name: Intermediate Result 23.3 - Improved policies and agreements for cross-border cooperation on sustainable energy Indicator: Number of policies and agreements adopted that promote cross-border cooperation Unit of Measure: Number (Cumulative) Source: Partners Indicator/Description: this indicator tracks the signing of memoranda of understanding and other agreements that establish frameworks for energy cooperation. Comments: In 2002, counterparts from each of the countries covered under SARI signed a memorandum of understanding to formalize the cooperative group that will deliberate on national transmission system improvements and potential regional transmission interconnection. Year Planned Actual 2000 0 (Baseline) 2001 0 0 2002 1 1 2003 2 Performance Data Table Sub-IR 23.3.1 – Private Sector energy plans and investments in infrastructure that facilitate regional energy exchange developed Objective Name: Encourage Economic Growth Objective ID: 498-023 Approved: 01/27/2000 Country/Organization: ANE Regional Result Name: Sub-IR 23.3.1 – Private sector energy plans and investments in infrastructure that facilitate regional energy exchange developed Indicator: International best practices identified and adopted in private sector investment plans to create capacity in cross-border infrastructure that meets regional needs Unit of Measure: Number (Cumulative) Source: Nexant Indicator/Description: This indicator tracks the preparation of documents, including commercial agreements and country gas assessments that will be used by parties in structuring commercial transactions. Comments: In 2002, a prefeasibility study for energy trade between India and Sri Lanka was developed. With SARI support, the Power Trading Corporation of India and the Snowy Mountains Engineering Company signed a memorandum of understanding to export electricity from the West Seti Hydropower Project in Nepal. In addition, the India-Bangladesh Power Export Study identified transmission potential and connection points for cross-border exchange. Year Planned Actual 2000 0 (Baseline) 2001 1 1 2002 2 4 2003 5 Performance Data Table Sub-IR 23.3.2 – Policies, laws and regulations developed or modified, with private sector and civil society concurrence, to facilitate regional energy trade and cooperation. Objective Name: Encourage Economic Growth Objective ID: 498-023 Approved: 01/27/2000 Country/Organization: ANE Regional Result Name: Sub-IR 23.3.2 – Policies, laws and regulations developed or revised, with private sector and civil society concurrence, to facilitate regional energy trade and cooperation. Indicator: Indicator 1 – Policies, laws, and regulations or procedures developed or revised that facilitate regional energy trade Unit of Measure: Number (Cumulative) Source: Nexant, AED, USEA Indicator/Description: Number of policy changes resulting from recommendations and training supported by USAID-funded activities Comments: In 2002 SARI activities contributed to the approval of the Sri Lanka Power Sector Reform program, the Bangladesh Power Grid Codes and Reliability Criteria, and the adoption of a SARI￾supported recommendation to restructure the Nepal Electricity Authority into three independent profit centers. Year Planned Actual 2000 0 (Baseline) 2001 2 2 2002 4 5 2003 8 Performance Data Table Sub-IR 23.3.3 – Increased capacity of regulatory bodies to support energy exchange Objective Name: Encourage Economic Growth Objective ID: 498-023 Approved: 01/27/2000 Country/Organization: : ANE Regional Result Name: Sub-IR 23.3.3 – Increased capacity of regulatory bodies to support energy exchange Indicator: Indicator 1 – Regulatory best practices developed or adopted by one or more countries Unit of Measure: Number (Cumulative) Source: Nexant, AED, USEA Indicator/Description: Number of regulatory initiatives/practices supporting regional trade resulting from technical assistance and training activities Comments: In 2002, activities contributed to the establishment of the Power Sector Regulatory Commission in Sri Lanka, strengthening the understanding of Bangladeshi officials of the function and role of the proposed regulatory body, the establishment of a regulatory partnership among SARI countries. Year Planned Actual 2000 0 2001 2 2 2002 4 5 2003 8 Performance Data Table Sub-IR 23.3.4 – Regional adaptation of best practices and standards that lead to energy efficiency and improved commercial practices Objective Name: Encourage Economic Growth Objective ID: 498-023 Approved: 01/27/2000 Country/Organization: : ANE Regional Result Name: Sub-IR 23.3.4 – Adaptation of best practices and standards in the region that lead to energy efficiency and improved commercial practices Indicator: Number of best practices and standards in pricing, generation, transmission, distribution, and end use in the region developed or adopted in one or more countries Unit of Measure: Number (Cumulative) Source: Nexant, AED, USEA Indicator/Description: Number of initiatives/practices promoting energy efficiency and improved commercial practices resulting from technical assistance, training, and partnership activities. Comments: SARI contributed to the harmonization of energy efficiency standards in 2002 with the agreement by India and Sri Lanka to adopt a common standard for ballast. India has adopted the label for refrigerators. In Sri Lanka, SARI facilitated the development of (1) a performance contract and (2) a monitoring and evaluation protocol for an Energy Services Company (ESCO). SARI supported the preparation of draft Energy Efficiency legislation that will be reviewed by the Government of Sri Lanka. With SARI support, Nepal adopted the CFL standard from the Sri Lankan model. SARI also supported the establishment of an ESCO in Nepal, and the linkage of a Sri Lankan ESCO with a Bangladeshi engineering company contemplating entering the ESCO sector. Year Planned Actual 2000 0 (Baseline) 2001 2 3 2002 5 11 2003 14 Performance Data Table Sub-IR 23.3.5 – Regional adaptation of best practices that lead to improved rural energy services Objective Name: Encourage Economic Growth Objective ID: 498-023 Approved: 01/27/2000 Country/Organization: : ANE Regional Result Name: Sub-IR 23.3.5 – Adaptation of best practices in the region that lead to improved rural energy services Indicator: Number of best practices in rural energy delivery identified, developed or adopted in one or more country Unit of Measure: Number (Cumulative) Source: Nexant, CORE Indicator/Description: Comments: SARI facilitated development of a power purchase agreement for Sri Lankan wind power. The Nepal Electricity Authority and Ceylon Electricity Board exchanged rural IPP PPAs. The Bangladesh rural energy services models from the REB and Grameen Shakti were adapted for rural energy projects in India Year Planned Actual 2000 0 (Baseline) 2001 2 3 2002 5 6 2003 8 Performance Data Table Sub-IR 23.3.6 –Increased private sector participation in and civil society support for sustainable energy development Objective Name: Encourage Economic Growth Objective ID: 498-023 Approved: 01/27/2000 Country/Organization: : ANE Regional Result Name: Sub-IR 23.3.6 – Increased private sector participation in and civil society support for sustainable energy development Indicator: Position papers and legislative drafts developed and advanced by private sector South Asia Energy Coalition (SAREC) members Unit of Measure: Number (cumulative) Source: US Chamber of Commerce, Nexant, AED, USEA Indicator/Description: Comments: Accomplishment in 2002: Presentation of Position Paper entitled “A Suggested Framework Of Action For Regional Business Associations And Apex Bodies” at workshops in Dhaka and New Delhi. Year Planned Actual 2000 0 (Baseline) 2001 1 1 2002 2 2 2003 6 Appendix D SARI/Energy 2003 Targets by Indicator. Result Name: Intermediate Result 23.3 - Improved policies and agreements for cross-border cooperation on sustainable energy Indicator: Number of policies and agreements adopted that promote cross-border cooperation Planned Value = 2 (cumulative), net increase of 1 - Signed MOU for 2-Borders Project between India and Sri Lanka Result Name: Sub-IR 23.3.1 – Private sector energy plans and investments in infrastructure that facilitate regional energy exchange developed Indicator: International best practices identified and adopted in private sector investment plans to create capacity in cross-border infrastructure that meets regional needs Planned Value = 5 (cumulative), net increase of 1 – Detailed Feasibility Study for 2-Borders Project Result Name: Sub-IR 23.3.2 – Policies, laws and regulations developed or revised, with private sector and civil society concurrence, to facilitate regional energy trade and cooperation. Indicator: Indicator 1 – Policies, laws, and regulations or procedures developed or revised that facilitate regional energy trade Planned Value = 8 (cumulative), net increase of 3 - (1) Sri Lanka Unbundling, (2) Corporatization/Privatization of Generation Companies in Bangladesh, (3) Elimination of Subsidies in Bangladesh Result Name: Sub-IR 23.3.3 – Increased capacity of regulatory bodies to support energy exchange Indicator: Indicator 1 – Regulatory best practices developed or adopted by one or more countries Planned Value = 7 (cumulative), net increase of 3 – (1) Development of Regulatory Rules in Nepal (2) Establishment of Regulatory Body in Bangladesh (3) Cost Recovery in Place in Bangladesh Result Name: Sub-IR 23.3.4 – Adaptation of best practices and standards in the region that lead to energy efficiency and improved commercial practices Indicator: Number of best practices and standards in pricing, generation, transmission, distribution, and end use in the region developed or adopted in one or more countries Planned Value = 14 (cumulative), net increase of 3 – (1) Harmonization of EE standards in Bangladesh (2) Adoption of Standards and Labels Legislation in Sri Lanka (3) CNG Developed in Bangladesh Result Name: Sub-IR 23.3.5 – Adaptation of best practices in the region that lead to improved rural energy services Indicator: Number of best practices in rural energy delivery identified, developed or adopted in one or more country Planned Value = 8 (cumulative), net increase of 2 – (1) Sri Lanka Wind Poser Assessment (2) Adapt Bangladesh RE Framework for Nepal Electricity Authority Rural Energy Cooperative Result Name: Sub-IR 23.3.6 – Increased private sector participation in and civil society support for sustainable energy development Indicator: Position papers and legislative drafts developed and advanced by private sector South Asia Energy Coalition (SAREC) members Planned Value = 6 (cumulative), net increase of 4 – (1) Innovative Approaches to Resolving Energy Sector Tariff Issues in South Asia (2) Island Based Renewable Energy Systems: Innovative Approaches to Promoting, Financing, and Capacity-Building for Environmentally Friendly Energy Systems (3) Background paper identifying key impediments to regional energy integration that the Coalition's membership can address and contend with in a timely manner (4) Methodology Paper for the developing of an effective advocacy strategy Appendix E List of Persons Contacted and Consulted USAID Washington: 1. Del McCluskey, Chief, Regional Programs, Bureau for Asia and Near East (ANE), USAID 2. Cindy Lowry, SARI/Energy Program Manager (HQ), ANE Bureau, USAID 3. Mr. Jim Bever, Director, South Asia Programs, ANE Bureau, USAID South Asia USAID/STATE Dept.: 4. Robert W. Beckman, Regional Coordinator and Program Manager, SARI/Energy, New Delhi 5. Kavita Sinha, Dy. Program Manager, SARI/Energy, New Delhi 6. Mercy Thomas, Regional Project Management Asst., SARI/Energy, New Delhi 7. N.V. Seshadri, SARI/Energy Country Coordinator, USAID/India 8. Ahsan Ul Haye, SARI/Energy Country Coordinator, USAID/Bangladesh 9. Mahesh Acharya, SARI/Energy Country Coordinator, USAID/Nepal 10. Upali Daranagama, SARI/Energy Country Coordinator, USAID/Sri Lanka 11. Richard L. Edwards, Office Director, Energy, Environment & Enterprise, USAID/New Delhi 12. Ashok Sarkar, Project Manager, Energy Conservation and Commercialization Project, USAID/ New Delhi 13. Bruce McMullen, Senior Energy Advisor, USAID/Dhaka 14. Cleveland L. Charles, Econ. Officer, U.S. Embassy/Dhaka Implementing Partners/Contractors/Grantees: Academy for Educational Development (AED) 15. Don Priestman, Chief of Party, New Delhi 16. Anupam Sircar, Dy. Chief of Party, New Delhi NEXANT Inc. 17. Hugh McDermott, Chief of Party, New Delhi 18. D.N. Raina, Dy. Chief of Party, New Delhi 19. Russell Profozich, Senior Consultant, Washington 20. Matt Addison, Project Manager, Washington CORE International 21. Vinod Shrivastava, Chief of Party, Washington 22. Lois Varrick, Corporate Vice President, Washington 23. P.K. Srivastava, Regional Training Manager, New Delhi 24. Dinesh Wahi, Director/Logistic Support Manager, New Delhi U.S. Energy Association (USEA) 25. Sridhar Samudrala, Deputy Program Manager – Asia, Washington 26. Amy Hallagan, Senior Program Coordinator, Washington 27. Tricia Williams, Senior Program Coordinator, Washington U.S. Chamber of Commerce 28. Herb Davis, Managing Director for South Asia, Washington 29. Aram Zamgochian, Associate Project Director , Washington Host-Country Counterparts: Nepal 30. Bishnu. B. Thapa, Joint Secretary, Policy , Planning, and Environment, Ministry of Water Resources 31. Lekh Man Singh, Director General, Department of Electricity Development, Ministry of Water Resources 32. Tara Bahadur Pradhanang, Electricity Tariff Fixation Commission, Ministry of Water Resources 33. Kiran Malla, General Manager, Butwal Power Company Ltd. 34. Ram Bhakta Karki, Director, Operation Department, Butwal Power Company Ltd. 35. Kush Kumar Joshi, Chairperson, Industrial Promotion, Environment and Energy Committee, Federation of Nepalese Chambers of Commerce and Industry Bangladesh 36. B.D. Rahmatullah, Director General, Power Cell, Ministry of Energy and Mineral Resources 37. A.S. M Alamgir Kabir, Director (Planning and Tariff), Power Cell, Ministry of Energy and Mineral Resources 38. Shah Alam, Joint Secretary, Power Division, Ministry of Power, Energy and Mineral Resources 39. Rafiqul Islam Khan, Sr. Assistant Secretary, Power Division, Ministry of Power, Energy and Mineral Resources 40. Masud Rahman, Dhaka Chamber of Commerce and Industry 41. Sayed Abdul Mayeed, Chairman, Bangladesh Power Development Board 42. Shamsul Alam, Secretary in Charge, Power Division, Ministry of Power, Energy and Mineral Resources 43. Khandaker Shahidul Islam, Secretary, Energy Division, , Ministry of Power, Energy and Mineral Resources 44. Rezwanul Kabeer, Director, Energypac Power Generation Ltd. 45. ANM Rizwan, Managing Director, Power Grid Corporation of Bangladesh