PROGRAMMATIC IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO FINAL REPORT September 6, 2012 This publication was produced for review by the United States Agency for International Development. It was prepared by Mendez England and Associates PROGRAMMATIC IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO FINAL REPORT Prepared under Task Order N o: AID-167-TO-12-00002 under the Evaluation Indefinite Quantity Contract, No: RAN-I-00-09-00018 Submitted to: USAID/Kosovo Submitted by: Arvid Kruze (Team Leader) Richard Tomiak (Team Member) Eun Joo Yi (Team Member) Contractor: Mendez England & Associates 4300 Montgomery Avenue, Suite 103 Bethesda, MD 20814 Tel: 301- 652 -4334 www.mendezengland.com DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. ACKNOWLEDGEMENT The evaluation team would like to acknowledge the kind assistance granted to them by the staff of the USAID Mission in Kosovo. Further, we wish to tha nk all stakeholders in Pristina who so generously gave their time and shared with us their insights into USAID’S involvement in the energy sector of Kosovo. IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO ii CONTENTS EXECUTIVE SUMMARY…………………………………………………………………… I Findings ................................................................................................................... 1 General Analyses ..................................................................................................... 4 Lessons Learned ...................................................................................................... 5 Recommendations for Future Program Design ....................................................... 5 Environmental Considerations ................................................................................. 5 Specific Areas of Involvement for the Future ......................................................... 5 A. INTRODUCTION AND BACKGROUND .................................................................... 7 A.1 Overview of the Kosovo Power Industry ..................................................... 7 A.2 USAID Involvement in the Energy Sector of Kosovo ................................. 8 A.3 Current USAID Strategy in the Kosovo Energy Sector ............................... 9 B. EVALUATION METHODOLOGY .............................................................................. 10 B.1 Evaluation Objectives ................................................................................. 10 B.2 Document Review ....................................................................................... 10 B.3 Quantitative Analysis and Performance ...................................................... 10 B.4 Qualitative Analysis and Interviews ........................................................... 11 B.5 Specific Analyses ........................................................................................ 11 B.6 Recommendations ....................................................................................... 11 C. PROJECT 1 – KEK NETWORK AND SUPPLY ........................................................ 11 C.1 Project Inception ......................................................................................... 11 C.2 The Years 2007-09 ...................................................................................... 12 C.3 The Years 2010-11 ...................................................................................... 13 C.4 Current Status .............................................................................................. 13 C.5 Impact Analysis – Based on Original Objectives ....................................... 13 C.6 Summary ..................................................................................................... 17 D. PROJECT 2 – EMSG ...................................................................................................... 17 D.1 Background ................................................................................................. 17 D.2 Examination of Performance Indicators and Accomplishments ................. 18 D.3 Analysis of Impacts .................................................................................... 18 D.4 Summary ..................................................................................................... 19 E. PROJECT 3 – KEDS ....................................................................................................... 19 E.1 Timeline ...................................................................................................... 19 E.2 Performance Indicators and Accomplishments ........................................... 19 E.3 Overall Positive Impacts ............................................................................. 20 E.4 Negative Impacts ......................................................................................... 21 E.5 Analysis and Summary ................................................................................ 22 IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO iii F. PROJECT 4 - PREPARATION OF SCOPING STATEMENT FOR ENVIRONMENTAL ASSESSMENT FOR REHABILITATION OF THERMAL POWER PLANT KOSOVO B ...................................................................................... 23 F.1 Background .................................................................................................. 23 F.2 USAID Environmental Procedures ............................................................. 23 F.3 Analysis of Environmental Procedures at the USAID Kosovo Mission 2006 to Present ..................................................................................................... 23 F.4 Environmental Engagements of the USAID Energy Program in Kosovo .. 24 F.5 Examination of Performance Indicators and Accomplishments ................. 24 F.6 Positive and Negative Impacts .................................................................... 25 F.7 Analysis and Summary ............................................................................... 26 G. PROJECT 5 – ADDITIONAL ENERGY SECTOR SUPPORT .................... 26 G.1 Background ................................................................................................. 26 G.2 Performance Indicators ............................................................................... 27 G.3 Impacts and Summary ................................................................................. 28 H. GENERAL ANALYSES ..................................................................................... 28 H.1 Inflection Points .......................................................................................... 28 H.2 Political and Donor Environment ................................................................ 29 H.3 SWOT Analysis .......................................................................................... 32 I. LESSONS LEARNED ........................................................................................ 34 J. RECOMMENDATIONS FOR FUTURE PROGRAM DESIGN ................... 35 J.1 General ........................................................................................................ 35 J.2 Environmental Considerations .................................................................... 37 J.3 Specific Areas of Involvement for the Future ............................................. 37 IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO LIST OF ACRONYMS AUK American University in Kosovo CENR Center for Energy and Natural Resource Development COR Contracting Officer’s Representative COTR Contracting Officer’s Technical Representative EA Environmental Assessment EAR European Agency for Reconstruction EC Energy conservation EC European Commission ECT Energy Community Treaty ECSEE Energy Community of South East Europe EMSG Economic Management for Stability and Growth EOIs Expressions of interest ERO Energy Regulatory Office ESCOs Energy service companies ESS Environmental Scoping Statement EU European Union GoK Government of Kosovo HED Higher Education Development HMIK Hydro-Meteorological Institute of Kosovo IEE Initial Environmental Examination IFC International Finance Corporation IMF International Monetary Fund KEDS Electricity Distribution Company in Kosovo Project KEK Kosovo Energy Corporation J.S.C. KLA Kosovo Liberation Army KOSTT Kosovo Transmission System and Market Operator J.S.C. LPTAP Lignite Power Technical Assistance Project MD Managing Director MED Ministry of Economic Development MEF Ministry of Economy and Finance MEM Ministry of Energy and Mining NARUC National Association of Regulatory Utility Commissioners NKPP New Kosovo Power Plant RE Renewable energy RFPs Requests for Proposal RIT Rochester Institute of Technology SME Small and medium enterprise SOW Statement of Work SWOT Strengths-weaknesses-opportunities-threats IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO 1 EXECUTIVE SUMMARY This is a rep ort on the Impact Evaluation of USAID’ s assistance provided to the Kosovo energy sector since 2007. The evaluation was conducted b y an independent external evaluation team co mmissioned by Mendez England & Associates (ME&A) on behalf of USAID/ Kosovo, pursuant to the Task Order AID-167-TO-12- 00002. The evaluation covered the following projects:  Kosovo Energy Corporation (KEK) Network and Supply Project  Energy Component - Economic Management for Stability and Growth (EMSG) Project  Advisory Services to Assist in Pri vatization of the Electricity Distribution Company in Kosovo Project (KEDS)  Preparation of Scoping Statem ent for Environmental Assessment for Rehabilitation of Thermal Power Plant Kosovo B  Additional Energy Sector Support The evaluation was co nducted during the period o f June – August 2012, by a team of three international consultants that consisted of two Energy Specialists and one Energy and Nat ural Resource Specialist. The main objective of this impact evaluation was to provi de USAID/Kosovo with up-to-date information of the overall impact/results achieved by USAID/Kosovo energy programs. The evaluation used an i ntegrated mixed methods approach including document review, quantitative analy sis, and qualitative analy sis from infor mation drawn from inte rviews, as well as specific analy ses including: 1) inflection point analyses; 2) an analysis of the political and donor environm ent in Kosovo; 3) a SWOT analysis of the USAI D strategies in the energy sector as o bserved in the projects’ analyses; and 4) lessons le arned analysis. Findings Project 1 – Kosovo Energy Corporation (KEK) Network and Supply The project has resulted in a wide rang e of operational improvements and efficiencies. As a result, electricity supply to Kosovo is more reliable than it otherwise would have been. Also, an increase in the collection rate on billed energy has significantly improved cash flow, resulting in a lower subsid y burden for the Government of Kosovo (GoK). The IMF views these financial developments at KEK as very positive. On the other hand, some negative impacts surfaced during the interviews inclu ding unnecessary disputes with other stakeholders in the Kosovo energy sector and pe rsistent noncompliance with orders issued by the regulatory office. The project’s successes need to be judged against the expansion of its scope of work and increased budget from $4.7 million in 2007 to over $28 million by 2012, as well as the fact that many of the objectives of the project as stated in the original Statement of Work (SOW) wer e not met, making the overall i mpact rather modest when viewed from a “value-for-money” perspective. Finally, the changing objectives and expanding scope of work over time reveal a lack of focus by USAID, which quite likely contributed to not fully meeting the original objectives. Project 2 – Energy Component: Economic Management for Stability and Growth (EMSG) Project The list of accomplishments for this project includes: 1. Assistance to the GoK to understand the steps to privatization and taking actions to support it 2. Finding a transaction advisor for the privatization of KEK 3. Support in issuing the very first Request for Proposals for Kosovo C 4. Support to the Energy Regulatory Office (ERO) on tariff matters 5. Support in the drafting of new mining laws IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO 2 Overall, the program has had a positive impact on the energy sector by increasing awareness and supporting the GoK to develop the necessary capacities to prepare for ener gy transactions. Many interviewees gave testimonials of the large and helpful role of the USAID contractors. A few also mentioned t hat they depend on the contractors’ inputs, proposals and capacity to help them. Positive impacts of EMSG include: 1. Visible progress in laws drafted and decisions taken for privatization; and 2. Gained trust and credibility with the GoK and other stakeholders involved in the energy sector Possible negative impact: 1. GoK may have become dependent on the advisors in the absence of formal capacity building The expressed objective of the EMSG energy component to support the energy transactions by assisting the GoK ministries seemed to have been met. Although based on the lack of perfor mance metrics, the intended effect of supporting KEK and Kosovo Electricity Distribution (KED) privatization was n ot fully achieved as evidenced by noted delays (in the co nsultant reports) and protes ted tenders, which resulted in new tenders having to be issued. In addition to the political en vironment, which was not as conducive for consensus on energy directions, the reports show that contractors took an active and, in some cases, a leading role. This may have created an unintended impact of having GoK d ecision-makers much too reliant on the consulting experts. This unintended impact may have delayed actions, although numerous changes in g overnment and ministries over the life of the project and, consequently, major shifts in strategy, also contributed to delays. Overall, EMSG energy component targeted the key energy stakeholders with close engagement, which allowed USAID to be closely connected to GoK’s energy policy and the privatization process. The lack of metrics, however, makes the evaluation of the impact of the advisory service somewhat difficult. Project 3 – Privatization of the Electricity Distribution Company in Kosovo Project (KEDS) The project’s accomplishments in Year 1 may be s ummarized as making progress in the development of an electricity market model. Principal project acco mplishments in Year 2 were: 1) losses study ; 2) evaluation of the 110 kV asset s ownership issue b etween the t ransmission owner/operator (KOSTT) and KEK; and 3) introduction of the concept of multi-year tariffs to the Energy Regulatory Office (ERO). Principal project accomplishments in Year 3 were: 1) launching the Multi-Year Tariff; 2) preparation of detailed market design for ERO approval; and 3) restructuring the ERO Board and team. Positive impacts of KEDS include: 1. A successful distribution company privatization transaction process 2. A more sustainable ERO and KOSTT, with m arket design and market rules development to support distribution company privatization; and 3. Valued privatization advisory services to the GoK    Negative impacts include: 1. A lack of effective donor coordination 2. Disagreements with KEK/ Tetra Tech 3. Some conflict between EU and US consultants, aris ing from differences in market, legal and regulatory approaches; and 4. Lack of continuity in energy sector representation at USAID Despite the above, overall, the feedback from the pr incipal beneficiaries (KOSTT, ERO, MED) and ot her energy sector stakeholders on the advisory services pr ovided under the project was very positive. The near completion of the KEDS transaction and the establis hment of t he ERO and KOSTT as self-sufficient and sustainable bodies were cited as evidence of the beneficial impact of the USAID energy sector program. IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO 3 Project 4 - Preparation of Scoping Statement for Environmental Assessment (ESS) for Rehabilitation of Thermal Power Plant Kosovo B. As suggested by Mission staff, this evaluation was oriented towards: 1) the following of USAID environmental procedures and potential impacts; and 2) the engagem ent of Mission staff o n environmental matters in the Kosovo energy sector. Regarding the first item, two cases are worth noting. The first case concerns the ESS for the feasibility study of Kosovo B, for which an Initial Environmental Examinati on (IEE) was submitted. The judgment in the IEE was later overturned and an ESS and Envir onmental Assessment (EA) were requested and eventually undertaken, which shows an increasing awareness of environmental issues over the 5-year period. On the other hand, in the second case, the KEK Network and Suppl y project has alwa ys been accorded a categorical exclusion in the IEEs, even though the y probably should have triggered an ESS and EA because of incidental involvem ent by Tetra Tech in the generation and mining functions. USAID’s increasing environmental awareness has also resulted in its being able to step in a nd fill a void created when the World Bank was held up in its air quality monitoring program, which is necessar y for World Bank participation in the Kosovo C project. This allowed substantial capacity building to be undertaken at the Hydro￾Meteorological Institute of Kosovo (HMIK) in preparation for actual air monitoring. Positive impacts of Project 4 include: 1. More awareness and vigilance by the Mission on environmental matters; and 2. An ability by the Mission to identif y a problem in the environmental area and m obilize required supplemental funding when necessary , as in the c ase of the World Bank’s air quality monitoring program and HMIK. Negative impacts include: 1. Environmental risks and considerations were not communicated to a wider public au dience, as evidenced by the fact that those interviewed did not know of the USAID environmental procedures, and no public outreach efforts were made to explain environmental impact of USAID help; 2. An IEE was not trig gered when Tetra Tech, on beh alf of KEK, became invol ved in generation and mining in 2009. As a result, USAID standard pr ocedures for dealing with potential envir onmental impact diagnosis were not followed. Project 5 – Additional Energy Sector Support Ancillary to the main USAID energy activities described above, USAID utilized partnersh ips with three US￾based organizations to provide additional technical training and professional exchange programs. These organizations included: 1) National Association of Regulatory Utility Commissioners (NARUC); 2) the Center for Energy and Natural Resource Developm ent (CENR), which allowed the Am erican University in Kosovo (AUK) to partner with the Rochester Institute of Technology (RIT); and 3) the United States Energy Association (USEA). It may be noted that the CENR program was designed and driven b y a Co ntracting Officer’s Representative (COR) in Washington, while all other projects in this report are Mission (Kosovo) based. The purpose of working with NARUC was to provide technical exchange and training between relevant energy market players and energy professionals. The two main beneficiaries of this support have been the ERO and KEK. Between 2009 and 2012, NARUC conducted informational workshops and training for KEK and ERO. In the li mited information available, performance indicators are not availa ble. However, ERO found the training provided by NARUC very useful and found topics relevant for the new regulators. The recent training included 10 participants, including board members and managers. CENR provided training opportunities and activities to the public and private sectors of Kosovo with the goal of building self-sufficiency in energy and natural res ource areas. Specific topics included economics, policy studies, energy and p ower systems engineering. T he center co nducted high-level trainings and established a Master of Science degree on regional is sues in energy and natural resource management and policy. The main beneficiaries of this support were the AUK, the GoK and the Independent Commission for Mines and Minerals. IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO 4 No formal performance indicators or metrics appear to have been established. However, the program did have clear objectives that were designed to establish and institutionalize a self-sufficient resource center within AUK. In general, the feedback in interviews e xpressed the benefits of technical exchanges, especially as they provide abilities to benchmark and share experiences with others working in a functioning energy market. USEA has undertaken var ious mandates in Kosovo over the past few years. Its experts have served partly as technical evaluators, partly as advisors and partly as trainers. B ased on inter views with USEA experts who happened to be in Kosovo at the same time as the evaluation team and Mission sta ff, USEA engagement has provided useful technical advice as well as helpful support in a non-consultant role. General Analyses Political environment. The political environm ent in Kosovo has shap ed USAID assistance in a number of ways. First, multiple changes in gove rnment since the War and GoK anti-privatization factions have caused delays to the privatization process, which, in turn, have resulted in increases in the cost of energy sector support to USAID, most notably to the KEK Network and Supply project. This situation also led to USAID allocating more advisory services than were perh aps necessary to the Ministry of Energy and Mining ( MEM) and the Ministry of Economic Development (MED) durin g the EMSG project. W hile the m inistries concerned did confirm that they valued the advice they had received from USAID contractors, there were situations early in the period (2008-2010) wher e some of the consultant s disagreed with each other, creating conflict of interest situations for USAID. As a result of the contr adictory advice being pro vided, confidence in contractor judgments was undermined, at least for a while. Since EMSG has ended, the potential for contradictory advice provided by different consultants still exists and may still be a consequence of the political environment. However, this may be inevitable, with USAID consultants working for organizations having natural adversarial relationships. A second political factor that has had an effect on USAI D assistance is the EU integration agenda, which has created pressure and m otivation on USAID to support co al generation in the face of EU am bivalence towards coal. Despite this and strong opposition to coal from other sources, Kosovo really has no choice but coal-fired generation, which USAID realizes in its support of the Kosovo energy sector. A third political factor is the culture of invasive political corruption within Kosovo, which may have affected various USAID decisions. One particularly stark example is a decision to support the non-unbundling of KEK - apparently because of the potential for political interference by KEK’s Board of Directors - during a time that ERO was constantly ordering KEK to unbundle. This action has not only undermined the notion of promoting a strong ERO, but it is also not in line with the co ncept of unb undling KEK for the purpose of the KEDS transaction. Donor environment. The donor environment has greatly shaped USAID ener gy assistance in Kosovo. Over the latter half of the 2002- 2012 decade, USAID has taken a leading role i n the privatization of KEK and the development of new coal-fired generati ng facilities. Today, there is a general consensus between GoK and donors on the distribution privatization, while there is so me disagreement on coal fired generation, in line with the ambivalence of the EU to become involved in coal plants. Also, the Worl d Bank, while supporting the new Kosovo C project, has been subject to many external pressures on this issue since 2006, w hich has resulted in long delays in its intended (but necessary) participation in the transaction. Lack of donor coordination has resulted in the following: 1. The decision to advise putting Kosovo B refurbishment into the Kosovo C investment plan package at about the time it was decided to decommission Kosovo A 2. Assignment of US advisors to the ERO at the same time that EU advisors were already in place, resulting in USAID’s contractors not being fully utilized and trusted initially by the beneficiary; and The Evaluation Team also noted that there is a perception (not necessarily justified) among som e donors that USAID as a donor is not a “not a team player.” Also, other donors’ slow response increas es the likelihood for USAID to take unilateral action, furthe ring the opinions expressed. However, the impact of IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO 5 USAID energy programs on beneficiaries is very palpable, as many of them have openly expressed that they felt USAID helped the most than any other donor. Interviews of key donor persons in Kosovo indicate that currently there is much better donor co-operation than before. Furthermore, the work of USAID in providing technical assistance and also management of the various stakeholders involved in the Kosovo C investment is gr eatly appreciated, as n ow there is closer cooperation. However, donor coordination still requires work. Lessons Learned  Lesson 1: There has to be a well-defined strategy to rationalise USAID involvement in Kosovo.  Lesson 2: Donor coordination is key to effective support programs.  Lesson 3: On balance, open-ended SOWs have had a negative impact.  Lesson 4: There is a need within the energy sector for a ‘fast track’ advisory service.  Lesson 5: Good procedures are of no use if they are not implemented properly. Recommendations for Future Program Design  Each new project proposal – as well as significant expansions in the scopes of work of ongoing projects – should be initiated through a concept paper, based on a thorough analysis of all the significant issues.  Project terms of reference should be more detailed and results driven, with appropriate M&E criteria that will reflect the extent to which the proj ect is achieving its stated objectives. And, the stated objectives should not be subject to easy revision over the life of the project.  As a matter of routine, regular audits and project evaluations should then be carried out at project level, to ensure compliance with the agreed project para meters and to identify any “project drift” aw ay from the agreed budget and SOW.  More attention must be given to ensuring that every proposal is beneficiary driven.  Consensus building, particularly with civil soci ety, should be come a formally recognised USAID obligation. While this may consume significant resources, there will undoubtedly be a positive payback.  Project proposals should be communicated to other donors working in the ener gy sector to ensure more effective donor coordinati on, particularly with the E U. It is und erstood that such efforts are currently being made.  In light of some serious but unsubstant iated issues that were raised by interviewees during the course of this project, USAID should establish a formal procedure for the independent recording and dealing with complaints, along the lines of the World Bank’s “Inspec tion Panel.” There is an email address at USAID where people can raise issues and ask for information; however, the email apparently ends up in the hands of someone directly involved in the p roject, whereas the intention shou ld be to have the complaint addressed by a completely independent party, probably in Washington DC. Environmental Considerations Strategic recommendations for environmental due diligences are: 1. Help strengthen Kosovo environmental institutions; and 2. Improve environmental communication and messaging Specific Areas of Involvement for the Future Current commitments should be carried through to their end. The KEK work will end June 2013, while there is a perceived need to continue providi ng advice and expertise to the other three organizations. Any new work here should be subject to the general recommendations provided in the previous two sections. IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO 6 Beyond the short-term, greater emphasis should be placed on institutional capacity building in support of long￾term economic growth, which is line wit h the second main objective of USAID strategy in Kosovo. This could include: 1. Support and build the role of government to set energy sector policies; 2. Invest in schools and relevant programs; 3. Support the Rule of Law in the energy sector; and 4. Encourage energy entrepreneurship, such as SME/ ESCOs IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO 7 A. INTRODUCTION AND BACKGROUND This evaluation report has been co mmissioned by ME&A on behalf of USAID/Kosovo . It represents the findings of an external and independent assessment of the USAID assist ance provided to the Kosovo energy sector since 2007. The evaluation covers the following projects:  KEK Network and Supply Project  Energy Component - Economic Management for Stability and Growth (EMSG) Project  Advisory Services to Assist in Pri vatization of the Electricity Distribution Company in Kosovo Project (KEDS)  Preparation of Scoping Statem ent for Environmental Assessment for Rehabilitation of Thermal Power Plant Kosovo B  Additional Energy Sector Support The evaluation team spent three weeks in Pristina, Ko sovo, between June 17, 201 2 and July 7, 2012. The principal activity over thi s period was interviewing stak eholders; however, much time was also dev oted to discussing and analyzing. A preliminary presentation of findings was made to USAID Mission staff in Kosovo on July 6. This report is presented in the following broad sections:  Introduction and background including: 1) an overview of the Ko sovo energy (power) sector; 2) a brief history of USAID involvement and objectives; 3) the SOW given in light of this background  Evaluation methodology, including a description of th e methods used to anal yze the USAID energy program in Kosovo  Analyses of the USAID energy program since 2007, in cluding: 1) a project-by -project analysis; 2) inflection point analyses; 3) an analysis of the political and donor environment in Kosovo; 4) a SWOT analysis; and 5) lessons learned analysis  Recommendations for f uture program design, given the results of the above a nalyses, including the approach to developing projects and specific areas of involvement, both short and long-term. A.1 Overview of the Kosovo Power Industry The power system of Kosovo has total installed electricity generation capacity of about 1,500 MW, with about 900 MW net operating capacity. Most of the generation is from two coal-fired thermal power plants, Kosovo A and Kosovo B. Kosovo A is the largest and oldest power plant, having three units (A3 to A5) in operation with a total available capacity of about 450 MW, according to local staff. Two of its units, A1 and A2, are out of operation, although KEK is proposing to rehabilitate A2, which has an installed capacity of 125 MW. Constructed during the 1960’s and 1970’s, the plant is old, unreliable and inefficient. It is also said to be the most polluting coal plant in Europe. The Kosovo B plant comprises two identical units, each having a net operating capacity of about 600 MW according to local staff, for a total net generating cap acity of over 1,000 MW from both plants. This i s based on interviews with the Kosovo A and B plant managers during June 2012. A recent report released by the World Bank1 puts this capacity at between 850 MW and 890 MW. Peak demand, which generally occurs on the coldest day s of th e year in Januar y, is currently 1,150 MW. Therefore, imports are required to largely make up the balance of electricity demand, which has been procured over the last decade from European traders via regional interconnections. Net imports have ranged between 5% and 17% of t otal annual consumption since 2001, and have m ore or less doubled since 2006. The volume of imports is constrained by the availability of surplus generation in exporting countri es, interconnection capacity and cost. There is a 400 kV interconnection with Serbia. However, this link cannot be relied on. Trans mission links also exist with Macedonia, Montenegro and Albania. 1 Background Paper: Development and Evaluation of Power Supply Options for Kosovo, World Bank, December 2011. IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO 8 Electricity consumption and peak demand in Kosovo ha ve more than doubl ed since the year 2000. Load shedding is one of the few tools available to KEK to control unm et demand during winter peak hours as well as periods of maintenance and outages. New generating capacity is needed immediately. Plans for a new, coal-fired “Kosovo C” (originally 2,000 MW, currently 600-700 MW) have been on hold since 2006 due to a number of factors includi ng, among others, changes in government, pressures from outside interest gr oups against the expansion of coal-fired generation, and subsequent delays in the World Bank schedule to pr ovide a partial risk guarantee (PRG) for financin g the plant. It thus appears that electricity shortages will continue indefinitely. Most electricity demand in Kosovo is from residential sources (approxim ately 63% in 2010) foll owed by industry. There are over 400,000 consumers, of which approximately 340,000 are residential and 60,000 are commercial, including about 250 i ndustrial. Technical and non-technical losses in the network rem ain high, together representing approximately 35% of total energy input to the grid. The collection rate of billed energy is over 90%. Through the UN ad ministration, which assumed control after the 1999 war, Kosovo became a signatory to the Athens Treaty (ratified in 2005), whic h created the Energy Community of South East Europe (ECSSEE). Achieving compliance with the requirements of the EU and the ECSSEE has in effect set the agenda for energy sector reform in Kosovo. The Kosovo Energy Corporation J.S.C. (KEK) is an almost vertically integrated public utility with operations spanning coal mining, generation and electricity distribution, but not transmission. It is the sole public supplier of electricity to consumers in the Republic of Kosovo. Over the period 2005 to 2006, KEK underwent a process of corporatization and unbundli ng which resulted i n a spin-off of its transmi ssion and dispatch business to a separate public company, Kosovo Transmission System and Market Operator J.S.C. (KOSTT). In 2008, the GoK approved the un bundling of the electricity distribution and supply network from KEK into a separate business, KEK Electricity Distribution and Supply (KEDS) which was then to be privatized through a competitive international tender. In June 2012, the winning bid for the distribution com pany was announced. The new investor, a Turkish consortium co mprising Limak Yatirim Enerji and Calik Enerji Sanayi, is currently expected to sign the purchase agre ement on October 18 and take over KEK’s distribution s ystem at the beginning of May 2013. Regulation of the Kosovo sector is carried out by the independent Energy Regulatory Office (ERO) which issues licenses for e nergy companies, sets an d approves tariffs in regulated businesses and draw s up regulations to define relationships between stakeholders in the energy sector. A.2 USAID Involvement in the Energy Sector of Kosovo USAID involvement in Kosovo bega n shortly after the Kosovo War end ed in 1 999. The m ission was established in 199 9, at a time when the UN ad ministration was responsible for infrastructure plannin g and development and for managing Kosovo utilities, including KEK. Given the prevailing political situation, KEK was used si mply as a mechanism to maintain social stability and, therefore, was not run as a commercially viable entity. USAID involvement in the energy sector at this time was very limited. In 2003, the European Agency for Reconstruction (EAR) asked USAID to co-fund the development of a billing and collection m odule of a central accounting s ystem for KEK. The system highlighted problem s of low collection rates and high losses and so, rather than hand over these issue s to the UN-managed KEK, USAID promoted the concept of “turn around managem ent”. This resulted in the appointment of the Electricity Supply Board of Ireland (ESBI) in 2004 to ta ke over the management and operation of KEK. USAID subsequently assisted in the process to establish an energy regulator and to develop energy legislation. In 2005, the Ministr y of Energy and Mining (MEM) was created, taking on r esponsibility for policy while the UN was the trustee of energy sector assets. USAID had, over 2003-04, undertaken a legal and regul atory pre-feasibility review (implemented by Pier ce Atwood) for a “greenfield” coal generating plant. Around 2005, MEM asked USAID for assistance with polic y issues, specifically on the utilisation of lignite resources and development of generation capacity. Accordingly, BearingPoint (now Deloi tte Consulting) was selec ted to wor k on the US AID funded Kosovo Economic IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO 9 Development Project (KEDP), which had an energy component on providing advisory services to the MEM. In 2006, the 2,000 MW Kosovo C projec t was conceived by the World Bank based on studies funded by EAR. The EMSG project, whose scope inclu ded providing advice to a wide range of Government m inistries, was launched in 2007 as a successor to the KEDP. With r espect to energy , USAID advised the MEM on issues surrounding the new coal plant at the Inter-ministerial Steering Committee. With USAID funding, the first call for expressions of interest (EOIs) to build, own and operate a 2,000 MW coal plant, was written by Deloitte/BearingPoint and there were responses from a nu mber of major players in the international power market. However, it was evident early on that World Bank involvement in the process would be advantageous. Around this time, the World Bank appointed a transaction a dvisor, at which poi nt, USAID backed away from any further heavy involvement regarding electricity generation in Kosovo. The EMSG project also ended. In the m eantime, under its “turnaroun d management” approach, USAID ap pointed another contractor, PA Consulting (later Tetra Tech) to succeed ESBI in ma naging KEK, except that under th e new contract, PA Consulting would act as advisor to KEK and not as a management contractor as ESBI had been. An overarching goal of this consultancy was to prepare KEK for pri vatization. Thus, the contract for the KEK Network and Supply Project came into effect in January 2007. It was also evident to USAID that more institutional support to the sector was required in paving the way for KEK privatization, namely for the ERO, KOSTT and in ME D, which was charged with the task of privatizing the power industry. Consequently, AEAI was appointed contractor in 2009 to provide these organizations with support under the KEDS project. They subsequently subcontracted to Deloitte to provide the MED with advisory services concerning the privatization of KEK. With the above USAID activit y in the energy sector of Kosovo, several incidental projects resulted. One such study was the Environmental Scoping Study completed in March 2011, which arose fro m the need to examine environmental aspects trig gered by an earlier feasibilit y study conducted by Tetra Tech for rehabil itating Kosovo B. The Mission subsequently asked its co ntractor AEAI to revise the Scoping Statement in 2011 in order to comply with request made by the USAID Bureau Environmental Officer (BEO); otherwise, the Scoping Statement was commissioned and completed in parallel with the Tetra Tech feasibility study for rehabilitation of Kosovo B. Other such USAID involvem ent has i ncluded various capacity building exercises and technical advisory services classified under “Additional Energy Sector Support” for the purpose of this evaluation. A detailed description and timeline of USAID involvement in the energy sector, provided by Mission staff, can be seen in Appendix A. A.3 Current USAID Strategy in the Kosovo Energy Sector The USAID/Kosovo Strategic Plan 2010-20 14 identified the lack of reliable energy supply as the m ost important impediment to private sector development and economic growth in Kosovo and concluded that overall sector reform, including privatization, was essential. The Plan also noted that since Kosovo is a signatory to the 2005 Athens Memorandum (which established the Energy Community Treaty), the agenda for its energy sector reform program is well defined and re sts on achiev ing compliance with E U rules for market structures and environmental standards and m eeting the requirements and obligations of the Memorandum and its successor documents. Within these constraints, the USAID energy strategy was aimed at three fundam ental elements necessary to achieve security of energy supply: 1) diversification of supply; 2) development of domestic resources; and 3) improved energy efficiency. The Strategic Plan stated that future assistance would focus on:  Improving KEK’s commercial operations  Commercialization of KEDS and preparation of a deal structure for its successful privatization  Providing technical assistance to establish KEDS as an unbundled corporate entity  Reform of the legal, regulatory and market framework  Improving environmental standards  Strengthening the capacity of the ERO  Strengthening the capacity of the market operator IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO 10  Environmental monitoring As part of its energy sector strategy, USAID also said it would e ngage in a feasibility study on the future of Kosovo B and undertook to assess opportunities to improve energy efficiency in Kosovo. B. EVALUATION METHODOLOGY B.1 Evaluation Objectives The main purpose of this independent evaluation is to provide USAID/Kosovo with an objective impact analysis of USAID assistance to Kosovo’ s energy sector since 2007. Depending on who the Mission decides t o share this report with, this information might be of interest to a wide array of stakeholders including USAID/Kosovo, Department of State, US AID/Washington and the Congress, key stakeholders in the GoK, other donors, and Kosovo’s civil society groups. Thus, the evaluation analyzes issues that may not only result in improvements to the delivery of USAID assistance to the sector, but will also add to the knowledge base of all stakeholders so that they can move forward with appropriate strategies to meet Kosovo’s energy objectives. The SOW provides the fo llowing general questions to be answered, which h ave been adopted as the more specific evaluation objectives: 1. What is the overall i mpact of USAID/Kosovo’s efforts in Kosovo’s energy sector since Janu ary 2007, both in overall terms and on an individual project level? This should include both positive and negative impacts as well as intended and unintended results. 2. How has the political and donor environment shaped the USAID/Kosovo energy program? 3. What are the benefits and potent ial negative implications for USAID/Kosovo to be i nvolved in Kosovo’s energy sector after the current program ends in June 2013? 4. To what degree did each project attain its desired results? B.2 Document Review A number of documents were provided by USAID for revi ew prior to departure for Kosovo. These included a series of quarterly and annual reports issued by Tetra Tech and AEAI from the start of their projects to-date, as well as the final environmental scoping statement report. Some useful information was found in the Tetra Tech and AEAI reports, which helped in developing tim elines on how the respective projects ev olved. None of the information provided pertained to either the EMSG project or any additional energy sector support. Towards the end of the team ’s first week in Kosovo, a more comprehensive number of documents was released by USAID, including original contracts, work plans, original contracts and sc opes of work, and a few key project documents. While in Kosovo, the team also gathered information and reports from the two main contractors, Tetra Tech and AEAI, as well as from a few of the various stakeholders met. A complete list of documents and information received is contained in Appendix B. B.3 Quantitative Analysis and Performance With respect to projects for which the data wer e available, appropriate performance indicators were used where possible to judge the results of reaching specific objectiv es. However, as will be seen in the ensuing analy ses, such performance indicators are largel y absent. These indicators have been used in instances where obj ectives have been defined. Impact may also be judged in terms of steady improvement in selected performance measures, of which there is an abundance in the case of the KE K Network and Supply Project; however, most of these indicators cannot be easily linked to the attainment of particular objectives, so their usefulness in a ssessing impact is weakened. In all projects, other than the KEK Network and Sup ply Project (at lea st partly), the absen ce of perfor mance IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO 11 indicators linked to specif ic objectives has dictated that much of the analyses be qualitative in nature an d, as such, subjective in nature, with impact judged in terms of a “pass/ fail” indicator. B.4 Qualitative Analysis and Interviews For the qualitative side o f the evaluat ion, the team interviewed USAID staff in Washi ngton, DC and the USAID/Kosovo staff in P ristina. During the first meeting with USAID staff in Kosovo, the schedule of interviews with relevant stakeholders in Pristina was discussed. Over the ensuing three weeks, 45 interview s took place. A complete list of interviewees is provided in Appendix C. For these interviews, questionnaires were prepared in advance and varied to some extent depending on the stakeholder. These were generally used as a rough guide in approaching the interviews an d were never read verbatim to the interviewee. These illustrative questionnaires are provided in Appendix D. B.5 Specific Analyses In accordance with the SOW, the projects w ere first analyzed individually with respect to objectives, performance expectations and actual im pacts, both quantita tively (where relevant inform ation was available) and qualitatively. The above analyses were then followed by a more global approach, which included: i) inflection point analyses; ii) an anal ysis of the polit ical and donor environm ent in Kosovo; iii) a strengths/ weaknesses/ opportunities/ threats (SWOT) analysis of the USAID strategies in the energy sector as observed in the projects’ analyses; and iv) lessons learned. Inflection point anal ysis identifies points in tim e at wh ich events occurred and a particul ar action taken in response to that event, which resulted in an impact. Then, if the impact was negative, what would have been a better decision – one that would have negated or at least mitigated the outcome. During the course of the evaluations, a number of negative impacts were identified that could have been avoided with alternative actions. These impacts have been analyzed with respect to these inflection points. The political and donor environment in Kosovo do play a role in how and where USAID can fund assistan ce to the Kosovo energy sector. These factors and their i mpacts have also been an alyzed, in accordance wit h the SOW. A SWOT analysis was then conducted for USAID in volvement in the Kosovo energy sector, which has formed the basis for the recommendations of the final section of the rep ort, in which the opportunities and threats (or negative implications) for further involvement have been identified. Finally, all the above analyses have contributed to a summary of lessons learned in evaluating the projects. B.6 Recommendations Recommendations for future involvement, at the end this re port, may be summarized in three categories. First, there is the general approach that shoul d be taken to identify, define, scope out and implement any potential project. Second, t here is a short-term path regarding specific projects that should probably be followed given current commitments. Lastly, there are reco mmendations for further longer-ter m projects that have been identified, arising mainly from stakeholder interviews. C. PROJECT 1 – KEK NETWORK AND SUPPLY C.1 Project Inception The KEK Network and Supply project was awarded to PA Consulting (later Tetra Tech) in Decem ber 2006 and began in January 2007 with mobilization of four full-time residents to Pristina. The project is ongoing and is projected to end June 2013 under the most current contract extension. The original contract with PA Consulting envisaged a two-year effort having a total contract value of about $4.7 million. The original SOW for this project states: “The project will assist KEK’s Network and Supply Divisions IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO 12 to improve functions, especially to increas e collections, with a goal of attracting future private sector participation. In addition, the Project will support senior Kosovar management as they assume gr eater responsibility and independence in provi ding strategic guidance and leadership to KEK.” With this major objective in mind, the SOW then elaborates by providing three main objectives for the project, with a number of performance targets for each objective. These obj ectives were (along with a few selected targets, am ong others): 1. Support to the KEK Managing Director (MD) – capacity building  By end of Year 1, the MD has basic skills for managing KEK  By end of Year 2, MD is fully trained and capable to independently carry out his or her functions 2. Increase billing and collections  By Year 2, ratio of billed to delivered electricity is 70%  By Year 2, ratio of revenues collected versus billed energy is 88% 3. Improve the institutional, policy and legal environment – work closely, and possibly lead the Task Force in identifying obstacles to KEK collection efforts and propose solutions  Supportive laws relating connection and disconnection are enacted  Judges are trained in relevant laws and understand the need to enforce them  Public perception of KEK is significantly improved, particularly as to corruption C.2 The Years 2007-2009 The contractor’s first work plan, issued March 2007, outlined how the project was going to m eet the three main objectives. The work plan identifie d eleven tasks in accord ance with the objectives and set up a performance based monitoring system (PBMS) to track certain indicat ors with targets set for both 2007 and 2008, as well as milestone indicators for each of the eleven tasks. This was in line with the original SOW. In March 2007, the contractor wrote to USAID explaining that project resources were insufficient for successful implementation and that t he magnitude of the proble m had been grossly underestimated by USAID. This was against a background where the World Bank was not progressing in the area of new generation, elections were taking place, political interference was endemic, and the attitude of the international community to Kosovo was ambiguous. A detailed breakdown of costs to increase the scope of work was provided to USAID and “approval memos” – providing the rationale for the proposed budget increase – were seen and approved by all the relevant USAID officers. Consequ ently, in the following year, the project’s budget was allowed to i ncrease from $4.7 million to $12.4 m illion. In the interim , the contract or was all owed to increase the project “burn rat e” in anticipation of the increase in the scope of work. The contractor’s 2007 Annual Report provides a fourth objective: establish new control policies and improve the control functions. The number of tasks, however, remained the same, at eleven, but it is not apparent which new tasks may have been added to address the new objectiv e. The changes in project objectives and tasks ar e summarized in Appendix E, along with all other changes up to 2012. Although the additi on of the additional objective during 2007 is perplexing, thi s change alone would probably not be sufficient to trigger a different decision process. The 2008 Annual Report, corresponding to the year in which the project’s budget increased from $4.7 million to $12.4 million, provided a total of eight objectives, to be addressed again with eleven tasks: 1. Support to the Managing Director of KEK 2. Increase Collections through Support to Network and Supply Divisions to Accelerate Potential Future Private Sector Participation 3. Improve the Institutional Policy and Legal Environment 4. Preparation of KEK Distribution Functions for Privatization 5. Support to KEK Commercialization 6. Anti-Corruption Efforts - Reduction and Prosecution of Fraud, Waste, and Abuse 7. Legal and Regulatory Support 8. Normalization of Service to Enclave Communities. IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO 13 It can be s een that the first three objectives had not changed from the original ones; h owever, the fourth objective added in 2007 was not there. Also, the eleven tasks in 2008 (sh own in Appendix E) were alm ost completely different fro m the original eleven tasks. Thus, the increas e in the scope of w ork resulted in the addition of f ive objectives and a complete revamping of the tasks required to achieve al l objectives. The question arises whether this, together with a substantial increase in budget of 2.6 times, should really have been subject to a new decision process, perhaps si milar to that used to define the original project in 2006. After all , the project was now 2.6 times larger than originally envisaged. The 2009 Annual Report stayed the course in terms of objectives and tasks. The contractor’s PBMS showed continuing improvements occurring at KEK over the period. C.3 The Years 2010-11 In December 2009, the total budget for t he project was increased from $12 million to $24 million. One reason for this large increase could be that co ntractor personnel would now be deplo yed to a greater extent in the districts outside Pristina. In any case , the project obj ectives changed drastically in a move to ostensibly concentrate more on privatization. The brand new objectives were as follows: 1. Support for technical preparation of the Distribution Company for privatization 2. Assistance with post-privatization implementation for the Distribution Company 3. Privatization support for the Thermal Power Plant Kosovo B. As can be seen in Append ix E, the above objectives came with eight tasks, most of them new. Bas ed on the contractor’s PBMS, KEK’s performance generally showed improvement throughout 2010 and 2011. C.4 Current Status A final modification to the contract took place March 30, 2012. The contract’s budget was increased by $4.2 million to a ceiling of $28. 6 million. It is set to expi re on June 30, 2013. The overall objectives of the project are now apparently the same a s those defined in the previous modification. However, the eight task areas have been consolidated into three and seem to resemble objectives. They are: 1. Support Management and Operations to Maintain (Preserve) Asset Value 2. Provide Advisory Support in the KEDS Privatiz ation Process Including Li mited Assistance Post￾Privatization 3. Provide Support for Privatization of Remaining KEK’s Assets C.5 Impact Analysis – Based on Original Objectives A list of project accomplishments, furnished by the contractor, is provided in Appendix F. There is little doubt that improvements to KEK’s operational efficiency have been made as the res ult of the technical assista nce, by many measures. However, given the changing nature of objectives and task s, impacts in this ca se are best assessed when compared to original expectations. While some might say this is unfair, as assumptions on which the original expectations were based might have been overly optimistic, these do provide an appropriate point of reference, given that the only complete SOW and objectives for the project appear to be contained in the original USAID RFTOP and subsequent contract with Tetra Tech (then PA Consulting). All modifications after this initial, well thought-out SOW targeted specific areas for im provement, but left the definition of performance indicators to the contractor, and a large p art of the new objectives and tasks. Even if the contractor’s indicators were appropriate, the changing objectives make it difficult to assess performance over time, as indicators for one objective are generally different from those of another. Therefore, the analysis which follows is based on the or iginal SOW for the original $4.7 million project. One would hope that a $28 million project would then have e asily achieved these objectives. If not, then perhaps more thought should have been devoted to re-thinking the project and the best way of spending the additional $23 million – with new objectives and accompanying performance indicators conceived by a technically competent energy sector expert. This was the way the original SOW was apparently conceived. IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO 14 As seen above, the first original objecti ve was support to the KE K MD, which was largely a capacity building effort, as can be ascertained from two associated performance indicators, among others:  By end of Year 1, the MD has basic skills for managing KEK  By end of Year 2, MD is fully trained and capable to independently carry out his or her functions The assessment of whether the above targets have been met is subjective, as it is easy to say yes or no based on anecdotal evidence. However, from interviews of tho se outside KEK/ Tetra Tech, there is strong reason t o believe that the above objectives have not been achieved. I n fact, the overwhelming perception among stakeholders is that the co ntractor is actually running KEK, which not only reflects on the level of the actual MD’s involvement in managing KEK, but is somewhat of a departure from the “support” that the contractor is supposed to be providing. As previously seen, the second major objective was to in crease billings and collections, with the foll owing performance objectives, among others:  By Year 2, ratio of billed to delivered electricity is 70%  By Year 2, ratio of revenues collected versus billed energy is 88% KEK performance with respect to these indicators is summarized on Table 1 in bold. Table 1 - KEK Billing and Collections Performance Indicators   Source: KEK, except for the last two lines, which have been estimated for the purpose of comparison with the Delhi distribution example. It can be seen on this table that billed to delivered electricity has, after 5 y ears (i.e., fro m 2007), reached a maximum of 65.3%, still short of the original 2-year target of 70%. On the other hand, the targeted collections rate of 88% was reached after 3 years and still continues to climb. Thus, one target in this case was reached and one was not. It should be noted that such targets are quite typical of those that have been used in similar situations in other countries. They address quite fundamental parameters in attempting to improve performance. Although KEK’s performance did improve in other ways (e.g., improved supply, reduced load shedding), it is commercially unacceptable to have such im provements defeated by continuing leak ages in the s ystem. The original targets are very simple, basic and legitimate, and what is more, they we re properly defined in the original SOW for the project. Other performance indicators, defined by the contractor and approved by USAID, Indicators 2006 2007 2008 2009 2010 2011 May 2012 Energy Delivered (GWh) 4004 4333 4729 5074 5357 5468 2485 Energy Billed (GWh) 2080 2258 2780 3049 3374 3569 1541 Unaccountable for energy (GWh) 1924 2075 1949 2025 1983 1899 944 Billed as a % of delivery 51.9% 52.1% 58.8% 60.1% 63% 65.3% 62% Losses as a % of delivery 48.1% 47.9% 41.2% 39.9% 37% 34.7% 38% Total 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% Collection rate 74% 77% 76% 81% 88% 91% 96% Energy paid for (estimated GWh) 1539 1739 2113 2470 2969 3248 1479 AT&C losses 61.6% 59.9% 55.3% 51.3% 44.6% 40.6% 40.5% IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO 15 do show improvements; however, it is submitted that they are not as powerful as the two indicators shown in bold on Table 1. It may be worthwhile to examine the above improvements in light of what is possible. In 2002, the distribution facilities of the Delhi Vidyut Board (India) were privatized and sold to Tatapower 2 . After 9 years, that is, to 2011, aggregate technical and commercial (AT&C) losses3 of the new company under Tatapower had fallen from 53% in 2002 to 13%. In the case of KEK, it has been estimated that this same measure of losses has fallen from 59.9% in 2007 t o 40.5% in 2012 (as seen in Table 1). In the case of Delhi, the improvement came about after privatization and shows the possibilities for KEK once the new investor takes control of the company. Given the above comparison, the question that arises is whether USAID’s $28 million effort in KEK was really worthwhile. The improvement in performance over 5 years is relatively m odest, with the new investor taking over a publi c company that is onl y in “less bad shap e” than it was 6 years previously . Grante d, all improvements were made with very little financial investm ent in distribution and suppl y facilities. Other achievements – e.g., first unqualified auditor’s opinion, controls of meters, clean-up of custo mer data and information, procedures im plemented for the handling of bills, collection and disconnections – m ight be evaluated in this light. However, the question then arises of wh ether the assi stance might have been better focused on say, obtaining the necessary capital funding for the distribution and supply system to fix the fundamental problem of leakages. The third ori ginal major objective was to im prove the in stitutional, policy and legal environm ent, with the following performance objectives, among others:  Supportive laws relating connection and disconnection are enacted  Judges are trained in relevant laws and understand the need to enforce them  Public perception of KEK is significantly improved, particularly as to corruption Based on interviews, the first performance indicator has been attained, with amendments to legislation that have declared electricity abuse as a cri minal act. Howev er, enforcement is still perceived to be a problem . This is reinforced by the still high am ount of unaccounted for en ergy seen under origi nal Objective 2, although about 4% of this amount may be attributed to losses in S erb enclaves in the north of the country and which the Go K has decided not to pursue as a matter of policy (at least for the time being). It is obvious that illegiti mate activities are still taking place and ar e being tolerated by the sy stem, whether within KE K or outside KE K through a corrupt judicial system (or both). From th e interviews, it has b een gathered that a nu mber of successful prosecutions have taken place, co mpared to virtually none prior to Tetra Tech’ s involvement. However, because theft i s still high, enforcement has not, on the whole, been successful. This is evident fro m the “Billed as a % of delivered” indi cator shown on Table 1. With respect to public perception of KEK, the company is still largely perceived as being corrupt, based on a recent Public Pulse Poll published by UNDP/ USAID in May 20124 , as well as the perception of the evaluation team from conversations with various people. Thus, it can be seen that, while some of the original performance objectives have been reached, others have not. Other Impacts As may be seen in Appendix F, the contractor’ s presence at KEK has resulted in a wide range of operational improvements and efficiencies. These are substantiated by steady increases in performance indicators presented in the projec t’s annual re ports. As a result, electric ity supply to Kosovo is more reliable than it pro bably otherwise would have been. Selected performance indicators are shown in Table 2, next page. 2 See http://www.tatapower-ddl.com/Display-Content.aspx?RefTypes=3&RefIds=142&page=Profile 3 AT&C losses include not only energy lost on the power system, but also energy not paid for. The KEK estimate is approximate, as an accurate estimate would require a bill-by-bill analysis. However, for a number of reasons, the AT&C estimate in Table 1 very likely over-estimates the improvement; i.e., the indicated 20% reduction, from 61.6% to 40.5%, was very likely less than 20%. 4 See http://www.kosovo.undp.org/repository/docs/2011/PP_FF_4_Eng.pdf IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO 16 The increase in the collection rate on billed energy, up to 91% in 2011 (and 96% for the first 5 months of 2012) from 74% in 2006, has resulted in a much improved cash flow, resulting in a lower subsidy burden for the GoK. Indeed, the IMF views th ese financial developments at KEK as very positive and that “things are head ed in the right direction.” Table 2 – Other KEK Performance Indicators Source: KEK On the other hand, based on the interviews, there have been a num ber of disputes between KEK and the other licensee in the power industry, KOSTT. These include:  KEK’s nonpayment of the full amount of charges invoiced by KOSTT. It is understood that this matter is currently before the court.  A continuing dispute over ownership of certain 110 kV facilities. Despite a decision having been mad e on this issue, KEK continued to pursue its position. Eventually, USAID through stronger management was able to manage the situation.  KOSTT maintains that KEK is an undisciplined consumer, to the extent that the agreed maximum levels of power interchange with Serbia are often exceeded. KEK simply does not follow the instructions of the system operator, thus allowing the Serbian author ities to justify their position that KOSTT cannot maintain system discipline and therefore, should not be recognized regionally as a legitimate sy stem operator. This was confirmed in another recent study by USEA. Although it is normal for licensees to have commercial disputes, such disputes are not healthy and are disruptive to a sector th at is attempting to revitalize. While it is bey ond the scope of this evaluation to pick sides, the evaluation team has heard on a few occasions that KEK is not a “team player.” However, the ERO’s inability to act as an effective mediator in certain instances may have fueled the disputes to a certain extent. This is also evidenced by KEK’s relationship with ERO. Appendix G provides a decision b y ERO to further allow KEK to postpone submission of a number of items ERO has ordered from KEK. The deadlines for each item, which KEK has rep eatedly not met (one might even go so far as to say ignored), date fro m 2008. In a more normal regulatory environment, such noncompliance would be viewed as inexcusable. In Kosovo, where a strong and independent regulator is an essential ingredient to successful energy sector reform, it is vital that all licensees play by the rules, which does not appear to be hap pening. It should be recognized that ERO may be partly to blame for not being aggressive enough in pur suing compliance. Also, an inter view with KEK’ s regulatory office suggested that ERO and its consulta nts did n ot interact sufficiently with KEK personnel regarding the precise nature of ERO requirements. However, given the sheer number of derogations listed in the ERO table in Appendix G over a five year period between 2008 and 2012 (e.g., as much as eight derogations for some items), it is difficult to assign a l arge part of t he fault to ERO. Si mply put, ERO has apparently been asking for five years for actions/ information, which KEK has not provided. It is also worth noting the nature of KEK’s noncompliance. The items on the ERO’s list can be broadly categorized into two: 1) unbundling; and 2) regulatory accounts, which are heavily inter-related. Given that one of the contractor’s mandates since 2007 has been to prepare for privatization, the fact that complete unbundling Definition of Indicator and Unit of Measure 2006 2007 2008 2009 2010 2011 Ratio of energy billed vs. energy available for sale 69.1% 69.9% 79.8% 79.3% 78.5% 81.9% Ratio of revenue collected versus billed 74.2% 76.6% 75.6% 81.4% 87.8% 91.1% Revenue collected as a percentage of value of energy available for sale 51.3% 53.5% 60.3% 64.5% 68.9% 74.7% Collected revenue in millions of Euro €96 €111 €135 €160 €178 €201 IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO 17 at this late st age has not yet taken plac e is surprising – but perhaps not, as Mission staff has explained that the decision to not unbundle was made by USAID and the US Embassy for reasons having t o do with potential political interference by KEK’s Board of Directors. Whatever the reason, this decision could be interpreted by some as being contradictory to the spirit of providing support to a strong and independent regulator and actually undermining its existence. A final impact of Tetra Tech’s presence at KEK is a consequence of its longevit y. The result of being on-site for 6 years can be good or bad, depending on how well this presence is perceived by other stakeholders. In this case, given the rather negative feedback received from stakeholders – ERO, KOSTT, USAID KEDS contractor, certain GoK personnel (“whenever we get together, there is always a fight”) - this impact has been judged to be negative. C.6 Summary The project produced some positive results, including more reliable el ectricity supply and a marked improvement in KEK’s financial status. However, th e improvements within KEK over 5 y ears (and counting) have not met many of the original objectives set for the original two-year project. The project’s successes need to be judged against the expansion of its scope of work and increased budget from $4.7 million to over $28 million, as well as the fact that the desired outcomes have not met original expectations, making overall impact rather modest when viewed from a “value-for-money” perspective. Finally, the changing objectives and expanding scope of work over time reveal a lack of focus by USAID, which quite likely contributed to not fully achieving what was originally intended. D. PROJECT 2 – EMSG D.1 Background The USAID Kosovo Mission carried out technical, lega l and commercial support to the Go K under the EMSG project, which was carried out from June 2007 to September 2010. The project entailed general support across a few sectors i ncluding tax, property, economic, budget, tr easury and privatization - which included an energy component. The USAID contractors were initially the consulting firm BearingPoint, which later became Deloitte. USAID included an energy component into the work plan, to provide specific energy related support to KEK and privatization of KEDs, directly supporting the Kosovo Ministry of Finance and MEM, now MED. The energy component comprised three main objectives (as stated in the SOW) that ended in 2009:  Technical and workflow-management support to the Lignite Power Tech nical Assistance Project (LPTAP) and NKPP (New Kosovo Power Plant - also called Kosovo C”) Project Office and Project Steering Committee – assistance ended in 2008;  Assistance to the ER O to strengthen monitoring, operations and im plementation of regulations – assistance ended in 2008, and;  Legal/policy advice to the former MEM - ended in 2009. The EMSG energy component’s main beneficiaries included: i) the for mer Ministry of Economy and Finance (MEF); ii) the GoK Steering Committees; iii) Transaction Advisors for KEDS; and iv) the New Kosovo Power Plant (NKPP). The objectives were to guide the re levant ministries and working groups, and expedit e the energy transaction timelines adopted by GoK. T he assistance also included significant coordination functions. Documents reviewed showed that the contractor’s energy related goal was to successfully help the privatization and investment, as well as provide prescriptive energy resource development advice. In this role, the contractor was to act in the interests of the GoK to prom ote optimal conditions for decision-m aking in the energy sector issues, meet challenges and to pursue opportunities. IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO 18 D.2 Examination of Performance Indicators and Accomplishments Performance Indicators. Drawing on the docum ents available for re view and interviews with som e of the stakeholders, it appears that project outp uts did not report against a set of label ed performance indicators. For example, there is no sectio n in the annual or final re ports listing the various perfo rmance metrics and activities or deliverables as signed to performance indicat ors. The reports do reflect the activities, lists of accomplishments, and docu ments created. These ite ms can be used as indicators of se rvices delivered. However, without benchmarks or minimum metrics thresholds, performance is difficult to ascertain. A greater part of the performance reporting is reflected in qualitative descriptions of advice given, coordination undertaken, or communication facilitated. Appendix H provides selected reporting statements that can serve as reference examples. Accomplishments. Following is a summary of listed accomplishments achieved by BearingPoint/Deloitte as found in the materials review:  Supported the GoK to understand the steps to privatisation and prepared supporting documents  Supported GoK to take actions related to privatization, such as to unbundle KEK  Suggested IFC as transactional advisor to the GoK for the privatization of the distribution system  Supported Lignite Power Technical Assistance Project (LPTAP) with Requests for Proposal (RFPs) and reviewed them with the transaction advisor  USAID advisors worked with and supported the ERO with topical KEK and KOSTT tariff matters  USAID Advisors assisted mining and resources development by supporting the drafting of new laws and their submission in 2010, which were designed to enhance operation of the m ining sector to attract private investment D.3 Analysis of Impacts The following im pacts are drawn more from document re view than from interviews with stakeholders and beneficiaries. This is because people tend to remember the particulars of assistance rather than the project under which it was provided. I nterviews with various beneficiaries related to or still employed by GoK revealed that beneficiaries were not familiar with the nam e of the assistance project. Also, since Deloitt e has had a more or less continuous presence in Kosovo that pre-dates even the EMSG project, it is difficult for t he beneficiaries to recall what specific assistance was provided under which project. Overall, the program seems to have had a positive i mpact on the energy sector by increasing awareness and supporting the GoK to develop the necessary capacities to prepare for energy transactions. Moreover, based on the reports submitted, GoK also benefited from documents prepared, studies and meetings organized. What has emerged in terms of the im pact is the perception that USAID has greatly sup ported the two main Ministries dealing with energy. Many interviewees also gave testimonials of the large role of the USAID contractors. A few also mentioned that they depend on the contractors’ inputs, proposals and capacity to help them. The following are potential positive and negative impacts, which include intended and unintended results. The analysis is limited due to the lack of metrics, review of deliverables and interviewees’ memories of the project. Positive impacts were: 1. USAID contractors’ comprehensive and operational support had significant impact resulting in visible progress in laws drafted, technical support leading to decisions taken for priv atization and cooperative meetings held. This has resulted in t he progression of GoK’ s decision making with res pect to the privatization of KEK from uncertain to decisive, leading to actual execution. 2. The EMSG energy advisors/contractors through long term and daily engagement may have gained trust and credibility with the GoK and other stakeholders involved in the energy sector - an unintended impact. Negative impacts were: 1. While day to da y comprehensive interaction was effective in advancing energy sector refor m, an unintended impact is that the GoK may have become dependent on the advisors in the absence of formal capacity building. IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO 19 D.4 Summary The expressed objective of the EMSG energy component to support the energy transactions by assisting the GoK ministries was somewhat met. The approach of supporti ng a newly formed government with experienced consultants in an operational and prescriptive manner was useful in direc ting discussion and influencing decisions made. However, the intended effect of promoting optimal conditions for actio n across the ener gy sector was not fully achieved. In addition to the poli tical environment, which was not conducive for consensus on energy directions, the reports show that contractors took an active and, in some c ase, a leading role. This may have created an uni ntended impact of having GoK decision makers much too reliant on the consulting experts. This unintended impact may have delayed actions, as noted in the problem sections in the contractor’s annual reports. Overall, EMSG energy components targeted the key energy stakeholders with close engagement, which allowed USAID to b e closely connected to Go K energy policy and the privatization process. T he lack of metrics, however, makes results of the advisory service impact difficult to evaluate. E. PROJECT 3 – KEDS E.1 Timeline In September 2009, USAID awarded a $2.9 million two-year contract to AEAI for “Advisory Services to Assist in Privatization of the Electric Distribution Company in Kosovo.” The original SOW for this contract identified two Task Areas:  Regulatory support to ensure a successful distri bution company privatization transaction and post￾privatization implementation  Market rules and related procedures and Codes to support distribution company privatization For a number of reasons to be discussed later, during the first two years of the contract, AEAI did not carry out many of the activities that had been planned. However, following chan ges in the o wnership of the contracting parties working on the EMSG energy component, USAID decided to consolidate all its energy advisory activities. Consequently, in September 2010, the first modification to the SOW was approved in order to sub-contract Deloitte to AEAI so that they could continue to p rovide their services and advice. The project duration was exten ded by 15 months, the bu dget increased by $3.3 million, and two more Task Areas were added to the SOW:  Privatization Advisory Services to the MEF, to the Inter-Ministerial Privatization Committee for KEDS, and to other key GoK stakeholders. This work was sub-contracted to Deloitte.  An independent study of KEK 2009 commer cial and technical losses, including uncol lected revenue. This study was carried out (for 2010 losses) through another sub-contract to AEAI. To accommodate the MED’s evolving timeline for the KEDS transaction and to allow for budget realignment, a second modification to the SOW was approved i n March 2012. This modification extended the project b y a further two months, to 28 February 2013, raised the budget ceiling by $1.6 million to a total of $7.8 million and revised the Task Areas, as follows:  Advisory Support to ERO  Advisory Support to KOSTT/ Electricity Transmission and Market Operator on market design and rules  Privatization Advisory Services to MED, to be carried out by Deloitte  Independent KEK Losses Stud y, which entailed li mited follow-up related to potential bidders’ due diligence E.2 Performance Indicators and Accomplishments The performance indicators for the KEDS project are f ound in each of the Quarterly Reports that has been issued. During the first 18 m onths of the contract, from its commencement till around spri ng 2011, the AEAI IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO 20 work program did not de velop in line with expectations. Hence, the entries in the Quarterly Reports for that period very often show comments such as "not asked for" or "transaction delayed." It should be noted that this happened not because the tasks were wrongly defined or the performance indicators inappropriate, but because the project was designed to s upport a transaction that, for a wide range of reasons, kept being delayed. Principal project accomplishments in Year 1 were:  Reached a broad consensus on the market model to be used i n Kosovo, on the basis o f which the generation Transaction Advisers could proceed.  Secured a positive decision from Brussels on the specific package of legislative proposals for shaping the laws and market model for Kosovo - and thereby bringing to an end a long and unprod uctive debate among consultants.  Worked with legal counsel at the ECSEE Secretariat in Vienna to accept draft changes in the law in line with the second legislative package and reflected these in the market model in time for their inclusion in the EC monitoring report. It should be noted that none of the above activities was specifically itemised in the list of agreed performance indicators. Principal project accomplishments in Year 2 were:  Delivered ‘Losses Study Report’ - an independent study which confirmed the Tetra Tech methodology to be sound. Steps were taken to address ERO’ s concerns to ensure that they would not dismiss the report later.  Brought in s enior international experts to evaluate views and proposals put f orward by various local parties and provide a neutral technical or operational opinion on the merits of each case (e.g., the 110 kV asset issue).  Demonstrated to ERO that adopting a longer-term appr oach to revenue setting in the form of multi-year controls would result in a better qualit y regulatory outcome, as well as giving greater certaint y over future revenue streams for potential investors and other licensees.  Developed an effective procedure to ensure the rapi d deployment of well-briefed quality experts to work on specific deliverables in short, sharp burst of ac tivity (rather than having under-utilised experts sitting around waiting for work to materialise), thus providing flexibility and relevance to projects. Principal project accomplishments in Year 3 were:  Launched the Multi-Year Tariff. The first round of consultation has been completed and the process is on schedule, with engineering and regulatory consultants engaged to support ERO.  Assisted KOSTT in getting its detailed market design prepared for ERO approval.  USAID and AEAI contributed to the recent restructuring the ERO Board and team. E.3 Overall Positive Impacts  It was acknowledged by alm ost all the stakeholders interviewed that AEAI and Deloitte have contributed to a successful distribution company privatization transaction process.  AEAI has been instru mental in est ablishing a sustainable ERO and KOSTT and has provided both organisations with effective advice on m arket design and m arket rules d evelopment to supp ort distribution company privatization.  Deloitte has provided valued privatization advisory services to the MEF, to the Inter-Ministerial Privatization Committee for KEDS, and to other key GoK stakeholders. The above im pacts are all positive, but the question arises of whether they could have been stronger and/or better focussed and any negative impacts reduced with a tighter SOW (as elaborated in Section E.4 below). IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO 21 E.4 Negative Impacts 1. Vague SOW. While it was clear that the overall objective was to provide support f or a successf ul and sustainable privatization transaction, the SOW for the KEDS project was intentionall y left broad and flexible - an approach which has both advantages and disadvantages. Flexibility in the SOW can certainly bring benefits in terms of r esponsiveness to the beneficiaries’ evolving requirements. However, t he lack of any pre-agreed deliverables - and therefore of a tangible work progra m - between USAID and ERO, left AEAI isolated and on the periphery of the action, ready to provide assistance, but being tol d by the regulator that the incum bent consultant engaged by the EC was already attending to matters satisfactorily. That left AEAI without any real work to do for ERO for at least the first 18 months of the project. A review of the materials produced by AEAI during the first two y ears of its assignment shows that in order to “fill the void,” AEAI produced more deliverables for USAID itself (around 60%) than for the clients it had been contracted to advise. Apart from representing a sub-optimal use of resources, this had other implications for the efficiency of the USAID energy sector program, which are discussed in the remaining paragraphs of this section. However, with the appointment of the new ERO chairman and AEAI’s ability to respond to a changing agenda, the relationship is m uch improved, to the extent th at AEAI might now be c onsidered the ERO’ s “preferred consultant.” 2. Lack of Effective Donor Coordination. The difficult situation described above was further compounded by a lack of coordination bet ween the EC consultant and USAID. AEAI’s role, as defined in the SOW, was to support KOSTT in becoming a separate and sustainable tr ansmission entity and to support ERO in establishing itself as an effective and i ndependent regulatory agency. Both KOSTT and ERO, however, already had very good EC advisors with whom they had built up a strong relationship over the years – KOSTT was being advised by IPA (UK) and Terna (Italy ), ERO by the Athens based consultancy, LDK. So inevitably, it was diffic ult for AEAI to establish a unique position for itself and this situation persisted for almost 2 years, with the result that much of the time was spent delivering work to USAID and not to the beneficiaries. Over time and with a lot of effort on the part of AE AI, an acceptable modus operandi has emerged, such that in its relationship with KOSTT, AEAI works primarily on market development and with ERO on transmission and distribution use-of-system tariffs and, more recently, on the multi-year tariff review. 3. Disagreements with Tetra Tech. AEAI spends a lot of t ime responding to and dealing with clai ms and statements made by Tetra Tech and trying to resolve the obstacles to progress that these create. While it is to be expected that there should be a natu ral and inevitable degree of friction between the various participants in an energy market, the relationshi ps between KOSTT and KEK and between ERO and KEK are not good, worse, in fact, than might be expected on the basis of what goes on in other countries. One reason for this is that in the past, ERO did not appear to be suppor tive of the privatization strategy that the GoK wanted to implement. A further source of tension was that various individuals had personal histories which brought them into conflict with one another. The outcome of these iss ues is that K EK/Tetra Tech has refused to com ply with certain basic requests for information from ERO, for example, in preparing regulatory accounts (Appendix G provides other examples of KEK non-compliance to date – see al so Section C.5 a bove), while ERO has on occasion been indecisive and failed to declare its decisi ons in a transparent and open manner. Disputes between KEK and KOSTT ( on the payment of invoices for services provided and on defining the boundary between distribution and transmission) have resulted in both these organizations taking the ir complaints to the Supre me Court for resolution. To the extent that some of these issues might be resolved if both sides were to sit down together and develop acceptable solutions, USAID could play a more proactive role in gettin g its consultants to facilitate and participate in such meetings. The structure of the KEDS project has also brought AEAI into another situati on of potential conflict. AEAI’s role is to support t he ERO and KOSTT in implementing the requirements of the Energ y Community Treaty; Tetra Tech has a negative relationship with both of th em. However, as explained above, during the first two years of the contract, AEAI spent a lot of time - around 60% - advising USAID on various issues. On several occasions, AEAI has found itself co mmenting to USAI D on Tetra Tech proposals from a totally neutral IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO 22 standpoint, yet, on the majority of occasions, finding in favor of their two beneficiaries. Of course, Tetra Tech would then c all their neutrality into question and progress would stall. Th ere is already evidence that this situation will become increasingly difficult to manage as the parties go through the Multi-Year Tariff process. 4. Potential Conflict of Interest Between Deloitte and AEAI. As mentioned above, Deloitte was sub￾contracted to AEAI for p urely pragmatic rather than l ogical reasons, i.e. so they could continue to advise the MED without having to go through extensive contractual re-negotiations. T his was done as their contract of engagement had expired and there wa s no quick alternative way to retain Deloitte’s services. However, given their respective roles and responsibilities, the sub-co ntracting of Deloitte under AEAI did create some potential conflict of interest issues - i.e., AEAI works primarily for ERO, whose function is to act as an independent body to balance th e interests of all stak eholders in the p ower industry, while Deloitte works for the Government, which is only one of those stakeholde rs. This potential conflict of inter est was partly addressed by creating “Chinese walls” between the two co nsultancies. Bringing these two separ ate and supposedly independent strands of involvement together under one contract was not helpful given the sensitive nature of the privatization issue. Sub-contracting Deloitte to AEAI heig htened ERO’s in itial distrust of the latter, since no t only were both consultants funded by the same (pro-privatisation) agency, USAID, but Deloit te was seen as being particularly close to the Transaction Adviser, IFC. USAID asked Deloitte to support the transaction and Deloitte carried out Transaction Advisory tasks when IFC was not in Kosovo. As a result, Deloitte was not only seen to be advising the MED but was also actively supporting IFC in transaction advisory work for the privatization of KEDS. This conflict of interest was to some degree also per ceived by KOSTT, though there appears to have be en a greater level of trust with KOSTT, possibly because they were not opposed to the concept of privatization. 5. Conflict Between EU and US Consultants. Kosovo is located in Europe and, und er UNSCR 1 244, is officially considered a potential EU c andidate by the EU. Therefore, the agenda for the refor m of its energy sector agenda is firmly set and involves implementation of the acquis communautaire as well meeting the other requirements of the Athens Treaty . Hence, other than for educational purposes, ther e is little point in debating alternative models. However, while EU environmental standards have been included in USAID energy sector program su pport, market workings or philosophies and legal solutions have sometimes been based on the North American model. These have been pushed quite forcibly by some American advisors – both those hired by USAID and those hired by other agencies - even though they did not conform to the EU approach. EU requirements have been referred to as an “obstacle” – even though the USA officially supports Kosovo’s wish to join the EU at some future time and that all efforts have been made by the USAID Mission to ensure compliance with EU requirements. 6. Lack of Continuity in Energy Sector Representation at USAID. In the interviews carried out by the project team, it was often pointed out that there had been many USAID staff changes since the beginning of the KEDS project (as can be seen in Appendix A), and that this lack of expert continuity had had an adverse impact on the effectiveness of the USAID energy program. This lack of continuity - especially in the energy sector - meant that in the absence of an expert, the contractor had to provide direction to USAID, which was itself being represented by an administrator. From consultant’s point of view, this makes life easier but creates the risk of that consultant either hijacking the work program or taking it off in an inappropriate direction. On the other hand, the present situation of actually having an energy sector expert at USAID appears to have resulted in an intensive doubling up, with the USAID sector expert working alm ost as a contractor would. This, in the absence of a well-defined SOW, means there is a risk of the consultant tea m getting pulled into areas that are not really within its scope. E.5 Analysis and Summary Overall, the feedback fr om the principal benefic iaries (KOSTT, ERO, MED) and other energy secto r stakeholders on the advisory services provided under the project was very positive. The near completion of the KEDS transaction and the establishment of the ERO and KOSTT as self-sufficient and sustainable bodies were cited as evidence of the beneficial impact of the USAID energy sector program. IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO 23 However, several negative impacts have been noted:  Firstly, the USAID energy program was involved i n each area of market transformation, advising the regulator, the government and public utility, thereby potentially compromising its neutr al role and undermining the trust of its clients.  Kosovo has t o follow EU standards ba sed on t he Athens’ Treaty. However, some advisors have on occasion pushed for solutions relating to m arket and legal/ regulatory issues that were bias ed towards a North American model, sometimes resulting in a degree of unnecessary conflict and confusion.  Leaving the SOW flexible can have some potential benefit; however, in the case of the KEDS project the lack of a well-defined SOW and associ ated deliverables left AEAI in an i mpossible position, exacerbated by the fact that there were already other consultants working with USAID clients.  US based staff are probably insufficiently familiar with the situation on the ground in Kosovo and need to be better informed. The absence of a local energy sector and project finance expert for a significant period constrained the effectiveness of the USAID operation in Kosovo. F. PROJECT 4 - PREPARATION OF SCOPING STATEMENT FOR ENVIRONMENTAL ASSESSMENT FOR REHABILITATION OF THERMAL POWER PLANT KOSOVO B F.1 Background USAID internal procedures provide detailed st eps, as pa rt of a system of c hecks and ba lances, to id entify, minimize and/or mitigate any potential environmental impacts of its programs and projects. The Environmental Scoping Statement for Environmental Assessment preparation, which began in January 2010 and is currently anticipated to end by March 2013, is a part of that pr ocess. During the course of the field work and discussions with the Mission, the full s cope of this evaluation was made clear to include the review of how environmental issues were addressed in the overall energy program. This section provides, based on document revie w and interviews with key stakeholders and analy sis, the ov erall impact of the US AID procedures and actions, and their implications. The focus is on the institutional decisions made and outcomes that have resulted in impacts. F.2 USAID Environmental Procedures Projects in Kosovo foll ow a specifi ed USAID process of identification, review and co mpliance of potential environmental impacts. For each proje ct, the Mission environment officer reviews the potential environmenta l impacts. Based on this r eview, the officer prepares an Initial Environmental Examination (IEE). In the IEE document, each task of th e proposed project or program is reviewed. Officers refer to USAID environment impact guidelines and references to asse ss the lack of or need for further environmental analy sis. If there is no environmental impact foreseen, a “c ategorical exclusion” can be requested and no other environmental assessment is made. However, if a potential environm ental impact is diagnosed, then the need for an environmental scoping statem ent (ESS) is triggered. If an ESS is required, t hen it serves as guidance for a detailed environmental assessment (EA) by highlighting key areas of potential impact. An IEE is also submitted for a project if the SOW or tasks are amended. F.3 Analysis of Environmental Procedures at the USAID Kosovo Mission 2006 to Present To conduct this evaluation, the team reviewed IEEs, subsequent ESSs and recent EA Documentation, classified by the Mission under Advisory Services to Assist in the Privatization of the Electricity Distribution Company in Kosovo. This documentation is summarized in Appendix I. The appendix shows that in 2008, an IEE with a c ategorical exclusion was r equested and a pproved, noting no potential environmental impacts of activities undertaken. In 2009, two IEE s were filed by the Mission. One of these IEE s was for KEK su pport, carried out by Tetra Tech, requesting a categorical exclusion from further environment assessment but noting a condition of environmentally sound practices for procurement, transportation and use of any equipment. The second IEE was filed for the feasibility report of rehabilitation and potenti al for expansion of Kosovo B (conducted by Tetra Tech ). The docu ment noted the n eed for this feasibility study to support the GoK in IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO 24 assessing the refurbishment and possible life extension of Kosovo B. Interviews with Mission staff confirm that the feasibility study was commissioned. The 2009 IEE calls for an ESS for the Kosovo B refurbishment study. Document flow shows that the initial environm ental impact diagnosis was conducted by the Contr acting Officer’s Technical Representative (COTR) of the program, which then appears to have been submitted to 3 to 4 higher levels (within USAID) for appro val including the bureau environmental officer, mission environmental officer, energy officer and Mission Director. All inst ances of IEEs that requested cat egorical exclusion from further environmental assessment stated that the activ ities did not involve power generation, transportation of coal and/or mining. From 2010 to 2012, t his process continued, where categorical exclusion for all activities was requested except for the Kosovo B rehabilitation feasibility study, where an ESS and EA was requested. The ESS in 2010 noted t hat the Worl d Bank will require an Environm ental and Social I mpact Assessment (ESIA). Interviews with stakeholders have revealed that during 2008 through 2009, contractors under the KEK Network and Supply project became involved in the generation functions of KEK. Un der the circumstances of lack of lignite supply at the gen eration site, generation technical issues and technical mining issues, contractors supported KEK to find both financial and technical solutions. Accounts from various contractors, governmental as well as USAID Mission staff confirm these activities. However, the IEEs did not reflect these involvements in power generation and mining. Whi le amendments did note conditi ons of utilizing envi ronmentally sound procurement, transportation and equi pment use, th ese do not account for direct invol vement of USAID contractors in im proving mining and power generation. Unlike the retroactive IEE and then ESS for the feasibility study of the Kosovo B refurbishment, no fu rther ESS was car ried out in the case of Tetr a Tech involvement in KEK generation and mining activities. F.4 Environmental Engagements of the USAID Energy Program in Kosovo Interviews concerning en vironment undertakings with in the energ y program found that t he Mission took a proactive and flexible approach as invest ment and priv atization transactions were being prepared. From the beginning of the NKPP project, the World Bank had agreed to fund and support air monitoring capabilities and data. However, the World Bank faced internal delay s and was u nable to disburse the agreed funding due to strong environmental opposition. In 2009, the World Ba nk approached USAID for help. USAID was then able to mobilize funding - thr ough existing activities and bu dget from the Kosovo energy privatization support program - for air monitoring capacity building and procurement of portable air monitoring equipment. This was done in cons ultation with the World Bank, which require d one year of air monitoring data as part of the investment package preparation for the NKPP project. USAID also requested AEAI, with The Cadmus Group, to prepare a Scoping Statement for the rehabilitation and possible life extension of Units 1 and 2 at Kosovo B. The Scoping Statement was designed to inform the GoK, LPTAP, the World Bank, and other donor organizations of the requirements for a future ESIA. USAID contractors worked with Hy dro-Meteorological Institute of Kosovo (HMI K) scientists in phased trainings on air monitoring methods using specific equipment. USAID also purchased analytical software to aid in obtaining additional weather data for the development of an air dispersal model. These initial activities were being undertaken as a short ter m solution. Regardless, the World Bank was further delayed, which prompted USAID to extend the air data collection program and actually begin air monitoring, using the portable units, and was recently able to extend activities until March 31, 2013. At the time of this evaluation, USAID was informed by the World Bank that they would take over this task once the USAID project ends on March 31, 2013. Based on ins pection and interviews wit h the HMIK and key USAID Mis sion persons, a num ber of tr ainings were completed by 2012. The beneficiaries found th e trainings to be excellent and very helpful. They would welcome more training as they still have equip ment (purchased by the EU) that they are not fully comfortable with. However, from a data quality point of view, they note that the portable air monitoring equipment is not rigorous enough to develop reliable air monitoring data sets. F.5 Examination of Performance Indicators and Accomplishments For the USAID energy programs, the environmental performance indicator is the IEEs. The IEEs clearly define potential environmental impacts, and provide specific rationale for exclusion or taking additional environmental action. The positive or negative findi ngs and final ly, approval, are, in effect, perform ance indicators. The IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO 25 inclusion of conditions also serves as a perform ance indicator. In the period from 2004 to 2012, while conditions were noted for some of the IEEs, every IEE (except for the Kosovo B refurbish ment study) was approved for an exclusion because no environm ental impacts were foreseen from the vari ous support work undertaken in the energy privatization support program. These performance in dicators may also be applied to the feasibility study of the Kosovo B refurbishment report. In this case, the positive IEE findings triggered additional performance triggers, i.e., the ESS and then the EA. The ESS, EA and analysis regarding air quality are examples of understanding the importance of environmental risks and impacts starting in 2010. USAID projects th at included air quality institutional capacity building related to the environm ental assessment for the new Kosovo power plant, as well as the am bient air qualit y management plan that i ncludes new generation, refurbishment of Kosov o B and Ko sovo A, responded proactively to the changes of another donor’s timeline (i.e., World Bank). Moving beyond the IEE and ESS, the performance indicators are less clear but include ite ms such as the num ber of trainings and the initial ai r dispersion models that were produced. Accomplishments included the USAID Mission’s ability to quickly and effectively mobilize resources to step in as other donors were not able to take action. This acti on achieved partial air monitoring institutional ability as well as continued support for the privatization and investment in the new Kosovo power plant. F.6 Positive and Negative Impacts The following im pacts are drawn mostly from interviews as well as ons ite visits rel ated to the USAID environmental procedure process and the work undert aken to support air m onitoring institution capacity building. Positive impacts include:  As noted, the circu mstantial conditions and the fact that the initial support activities did not pose potential environmental impacts were well docum ented. The IEEs were also approved and demonstrated a robust s ystem for initial diagnosis within USAID. The impact of retroactive review of IEE judgements - resulting in retroacti ve actions such as the ESS and EA - i s that the Mi ssion is now more vigilant regarding these matters. The impact of this will be a more critical and sensitive approach to future environmental impact diagnosis. Such awareness greatly supports a changed culture.  USAID’s full participatory approach in the ener gy sector has allowed the Mi ssion to clos ely follow environmental preparatory needs for both the privatization and the investment in the new Kosovo power plant. A positive impact of this engagement allowed the Mission to identify and mobilize supplemental funds in close collaboration with t he World Bank. This approach facilitated the air-m onitoring institutional capability and data needs. The result is increased t echnical skills and awar eness by the HMIK of new methods and its role. Another im pact is the tim ely procurement of portable air monitoring equipment. Furthermore, this supported the on-going development of the fun ction of the Kosovo Environmental Protection Agency. Negative impacts include:  Within the Mission, there seems to be an inform al directive of “keeping the lig hts on.” In this context, potential environmental impacts can be viewed as secondary or tertiary and potentially affect the wa y the IEEs are completed. This sense of urgency resulted in an impression that the environment was not a high priority, especially during the earlier period of the energy projects.  As a result, the opport unity to expand the air monitoring tasks into a full stationar y lab or to engage other donors (as World Bank was delay ed) was missed. At the tim e of the evaluation, an interview with the World Bank in Kosovo confirmed that investment in new coal-fired generation still needs one year of air monitoring data to proceed. However, as yet, that full capacity is not available in Kosovo, i.e., conducting the air-monitoring data-collection required in order for the World Bank investment to take place is still not possible. This is an added delay to the overall objective of securing energy for Kosovo.  Interviews revealed that co mmunication of the e nvironmental risks and c onsiderations were no t communicated to a wider public audience, although the Mission adhered to all USAID environmental impact procedures. Additionally, the restriction of access to energy program activities by making KEK IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO 26 Annual Reports confidential starting in 2009 resulted in a perception of less transparency . As the only power provider and main topic of environmental concerns, not publicly sharing information such as the Annual Reports or the work with HMIK unnecessar ily alienated civil society and NGOs active in the environment agenda.  As noted earlier, when Tetra Tech on behalf of KEK became involved in g eneration and mining in 2009, an IE E was not tr iggered. T he impact is that USAID standard proc edures for d ealing with potential environmental impact diagnosis were not followed. F.7 Analysis and Summary Analysis of the Preparation of Environmental Scoping Statement for the Rehabilitation of Kosovo Power Plant B shows a shift in objectives and appl ication of USAID potential environmental impact procedures. This is shown by the IEEs from 2006 to present, as well as the expansion of the scope of activities to acco mmodate changes in the energy sector and its stakeholder dynamics. Th e interviews and analyses show excep tional ability of the Mission to quickly mobilize and resp ond to circumstances. Furthermore, the active and close collaboration with other donors to find stop-gap envir onmental capacity needs has had a positive im pact on the USAID energy program. However, a holistic perspective on potential environmental impact was not present, which may have resulted in missed opportunities. For exam ple, the World Bank be gan to f ace environmental protest against coal-f ired generation early on in the investment preparation phase, which is really the underlying cause to the delay in their contribution to the air monitoring data needs5 . At this time, USAID had an opportunity to view environmental issues not only as an im pact or a formality but also as an innovative soluti on. While stop-gap instit utional capacity building was proposed and undertaken, the associated portable a ir monitoring equipment was insufficient for obtaining the quality measurements needed for the World Bank’ s ESIA. The pending ESIA requires one full year of air quality data, the collection of which has not yet begun. Although the World Bank led USAID to believe that there were no issues, an in-depth strategic assessment by USAID, starting from 2009, may have resulted in the Mission negotiating this task with the World Bank and buildin g an appropriate air monitoring station back in 2010. This would have not only optimized institutional capacity building, but could have also initiated the req uired data collection. Th e evaluation team does recognize, however, that the donor dynamics as well as the change in GoK progress on the energy privatization has had significant impact on the air monitoring topic. Furthermore, the absence of awarene ss or co mmunication of the Mission’ s careful adherence to USAI D environmental impact mitigation or minimization efforts were not communicated to key stakeholders as well as civil society. Interviews with GoK, NGOs and other stakeholders revealed that they either had no kno wledge that USAID activities had environmental safeguards procedures and or t hey had perceptions that USAID was implementing energy sector changes behind closed doors with no access to inform ation. This perception is especially poignant against the current backdrop of national and international protests by environmentalists. The result is that they are further bolstered by the l ack of knowledge and transparency of the environmental precautions and assessments that USAID has already addressed. G. PROJECT 5 – ADDITIONAL ENERGY SECTOR SUPPORT G.1 Background Ancillary to the main USAID energy activities that ha ve focused on the KEK network, KEDS privatization and support to t he new Koso vo power pl ant, the USAID Mi ssion utilized a partnership with three US-based 5 The World Bank has stated that Kosovo C is one of the most controversial projects in its portfolio because of worldwide opposition from environmental groups. To avoid similar criticism in the future, this is the last coal project the World Bank will support anywhere. Because these groups are closely monitoring the project, spending on any aspect of Kosovo C is very closely reviewed by the World Bank, to the extent that support for even the most environmentally benign activity related to Kosovo C tends to become delayed (e.g., air monitoring). IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO 27 organizations, mainly to provide additional technical training and professional exchange programs. This support has been utilized as early as 2006 until present. This section provides an overall i mpact assessment of t hese complimentary support programs and im plications in th e context of USAID’s energ y program. A thor ough evaluation of this project was limited by lack of records and documents, which co mprised mainly final contractor reports, with nothing or litt le in the wa y of original SOWs, project objectives and performance metrics. The evaluation team thus relied on the use of some documents, interviews and the recounting of the activities by Mission officers. The US-based organizations supporting the USAID Mission in these activities are:  National Association of Regulatory Utility Commissioners (NARUC). Founded in 1889, the Association is a resource for its m embers and the regulatory community, providing a venue to set and infl uence public policy, share best practices, and foster innovative solutions to improve regulation.  Rochester Institute of Technology (RIT). Founded in 1829, RIT is a privately endowed, coeducational university with nine colleges emphasizing career education and experiential learning.  The United States Energy Association (USEA). USEA is made up of public and private energy￾related organizations, corporations, and governm ent agencies. Together they represent interests of the U.S. energy sector by in creasing the understanding of energy issues, both dom estically and internationally. The purpose of working with NARUC was to provide technical exchange and training between relevant energy market players and energy professionals. The support was coordinated by USAID/ Washington. The two main beneficiaries of this support have been ERO and KEK. The Center for Energy and Natural Resource Developm ent (CENR) was established as p art of a cooperative agreement, administered by Higher Education Devel opment (HED), between the USAI D and the A merican Council on Education, with five other higher education associations. This agreement has mobilized the resources of American higher education in support of international development. CENR is part of t he American University of Kosovo (AUK) in partnership with RIT to focus on workforce development, consulting, re search and dissemination of information and data on energy a nd natural resources. Following are CENR’ s original objectives:  Collaborate with representatives from industry, government, and NGOs to define the com petency gaps and specific skill requirements of the energy and natural resource management workforce in Kosovo  Articulate required com petencies and skills into sp ecific training and educ ation programs those are accessible to both the incumbent and emergent workforce of Kosovo  Integrate energy and natural resource education and research programs into the AUK curriculum  Create the research and instructional infrastructure at AUK necessary to support ongoing development of energy and natural resource programs in Kosovo  Establish long-term industrial, governmental, and educational linkages between energy sector stakeholders in Kosovo and the US. CENR provided training opportunities and activities to the public and private sectors of Kosovo with the goal of building self-sufficiency in energy and natural res ource areas. Specific topics included economics, policy studies, energy and power sy stems engineering. Th e center conducted high-level workforce training, open forum debates, conferences, student research and field trips, and the establishment of a Master of Science degree on regional i ssues in energ y and natur al resource management and policy. The main beneficiaries of this support were the AUK, the GoK (specifically , MEM, Mi nistry of S pecial Planning and Environm ental Protection) and the Independent Commission for Mines and Minerals. USEA has provided varied support to the Kosovo energy sector, including analysis of certain sector issues, technical advice and capacity building. G.2 Performance Indicators NARUC. Between 2009 and 2012, NARUC conducted inform ational workshops and training for KEK, ERO and GoK. Members of NARUC traveled to Kosovo each year. Topics covered included tariff issues in Europe, customer tariff issues and market issue s. In the limited information available, performance indicators are not IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO 28 available. However, interviews with two of three be neficiaries gave positive f eedback (ERO and KEK). The evaluation team did not have the opportunity of interviewing GoK beneficiaries on this matter. ERO found the training provided by NARUC very useful and found topics relevant for the new regulators. The recent training of ERO members included 10 participants, including board members and managers. CENR. A review of the final partnership report submitted by the Partnership Director from RIT indicated that no formal performance indicators or metrics were established. However, the program did have clear objectives that were designed to establish and institutionalize a self-sufficient resource center within AUK. From March 2007 to December 31, 2010, RIT and AUK conducted activities that could pr ovide performance indicators in trainings, discussions, student exchange, research, consulting and conferences. The implied performance indicators show that the center was created and supported with sources of additional funding, and institutionalized into the AUK, thus meeting the objectives. Other reported results such as the lists of students and resear ch “capstones” also serve as performance indicators. However, results-oriented performance indicators such as student interviews (which would provide a means to measure effectiveness of the program or degree of l earning) are not available. One possible metric that could have been measured as an indicator of effectiveness is the num ber of student s that received internships, although t his metric might be difficult to track. USEA. USEA is a non-profit organization whose prim ary objective is to advise the World Energ y Council. In 1999, USEA was asked t o advise on regional transmission issues in south-east Europe. In 2006, the y were engaged by USAID to review the performance of ESBI in implementing the KEK ‘turnaround ’ contract. In a USEA study (updated in 2009), many suggestions were made on improving KEK mining operations. In 2012, a USEA mission was sent to Kosovo to review and explain a significant Kosovo power sector event in February 2012. In an interview, the USEA team was also asked what improvements, if any, had been made as the result of past energy sector support by the USAID energy team. They pointed to improvements in mining operations and operational improvements at KEK in general. USEA also organized an exchange visit of KEK m anagers to Southern California Edison (SCE) in November 2010, which involved distribution, transmission and cu stomer service operations. A second exchange group from Kosovo B later went to stud y generation issues in Louisiana and Texas. Both exchanges were said to be very successful. One US EA member noted capacity improvement of KEK e mployees but also noted their reliance on Tetra Tech experts. Over various mandates, USEA experts ha ve served partly as technical evaluato rs, partly as advisors and partly as trainers. For each role, perform ance indicators include on site write-ups and trip reports. Som e USEA recommendations were confirmed to ha ve been implemented while others were not. Based on interviews with USEA members and Mission staff, USEA engagem ent has provided useful technical advice as well a s helpful support in a non-consultant role. G.3 Impacts and Summary Because of limited documentation, assessing the ove rall impact of the three “additional energy sector support” activities is somewhat challenging. It can only be reported that the feedback in intervie ws expressed the significant benefits of tech nical exchanges, especially as they provide the abilit y to benchmark and to share experiences with others working in a functioning energy market. Meanwhile, USEA te chnical advice has generally been regarded as useful. H. GENERAL ANALYSES H.1 Inflection Points From the for going, four specific inflec tion point analy ses were developed as described be low. De cisions or processes that led to speci fic results or impacts, allowed the evaluation team to construct an alternative look at how different tasks and actions may have had different results at critical moments. The scenarios above were IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO 29 developed by identifying plausible inflection points - a moment and a decision which wa s a turning point. These analyses only serve as a tool to view events in a different light. Inflection Point 1 – KEK Network and Supply Project Expansion of Scope of Work Inflection Point 2 – KEK Network and Supply Project Confidential Annual Reports Inflection Point 3 – KEK Network and Supply Project Environmental Impacts Inflection Point 4 – KEK – KOSTT Invoices Dispute H.2 Political and Donor Environment Political Environment. The political environment in Kosovo h as shaped USAID assist ance in a num ber of ways. First, GoK anti-privatization factions have caused USAID to allocate more advisory services to the MEM and MED. There have b een four parliam entary elections in Kos ovo since the War. The most recent ones, in 200 7 and 2010, were monitored by the international community, which concluded that they were generally fair and free. IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO 30 However, Kosovo has a m ulti-party system, with a large number of diverse political parties which, when coupled with a system of proportional representation and guaranteed minority representation, means that no one party has been able to – or is likely to – command a parliamentary majority. Because of these arrangements, the Go K has always found it rather difficult to make progress on the two major energy privatizations, KEDS and coal generation. On the one hand, MED has said that "any decision taken on KEDS will be harmonised by a unanimous vote and government approved,” while on the other, there has been a significant level of conti nuing vocal and active oppositio n to privatization within parliam ent, making a unanimous vote difficult to achieve. This has resulted in delays to the privatization process, most notably KEDS (as other factors have had their effect on dela ying the coal-fired generation project) and is at least partly to blame for the large increase in cost associated with the KEK Network and Supply project. Also, tt would appear that in order to help the GoK resolve the on-going impasse regarding KEDS privatization, USAID strategy over the last few y ears has responded b y focusing o n giving the ke y ministries (Finance, MEM/MED) access to ever increasing num bers of supportive consultants and advisers who could provide the information and guidance needed to move the privatizations along to a successful conclusion. Over the last few years there have been many energy sector projects in Kosovo, funded by a number of different agencies and covering a range of issues. The GoK an d its ministries have therefore been receiving information and advice from a number of diverse sources and may not always have had the capacity to evaluate these inputs efficiently. However, the ministries concerned confirmed that they valued the advice they had received from USAID contractors. A second political factor that has had an effect on USAI D assistance is the EU integration agenda, which has created pressure and motivation on USAID for coal generation support. The Large Combustion Plant Directive was adopted by the European Parliament in 2 001. This Directive introduced measures to control emissions of oxides of nitrogen (NOx), sulphur dioxide (SOx) and particulates from large combustion plants (i.e., plant with a rated thermal input equal t o or greater than 50 MW t hermal). Its aim was to take st eps to reduce the emissions of these pollutants, which, it was argued, damage human health and contribute to acid rain. In Kosovo, however, a very specific situation has been developing over recent years. The existing lignite-fired generating units - which are very old and very dirty - are at t he end of (or bey ond) their econo mic lives. Meanwhile, the demand for electricity is growing rapidly (at just below 7% per annum between 2000 and 2010) and Kosovo, while not connected to any gas network, with no oil reserves and only limited renewable energy potential, has at its dispos al some the most extensive and inexpensive lignite deposits in the world. These factors have combined to make generation planning in Kosovo highly contentious, particularly because the GoK has requested a Partial Risk Guarantee from the Wo rld Bank to support t he development of a lignite fuelled facility. In its Kosovo Strategic Plan 2010-2014, USAID states that its fundamental energy sector objective is to improve the quality of energy supply and that, to achieve this , it intends to address three key strategic ele ments: diversification of supply, development of domestic resources, and energy efficiency. Hence, in lig ht of its strategy and despite the EU Directive and strong local opposition to the proposed new facility, USAID has moved towards a position of suppor ting the proposal for a new lignite-fired Kosovo C and the rehabilitation of Kosovo B. This strategy evolved over the same period as USAID’s emergence as a strong supporter of privatization and also at t he same time as USAID reacted to adverse publicity by restricting the amount of information made available to the public. Th e combination of these events has created a negative image of USAID in the eyes of some energy sector stakeholders. A third political factor is the culture of invasive political corruption within Kosovo. Although USAID is active in supporting the elimination of political corruption, the mere presence of corruption has resulted in at least one USAID action that has significantly undermined the notion of promoting a strong and independent ERO; i.e., the decision to support the non-unbundling of KEK - apparently because of the potential for political interference by KEK’s Board of Directors - during a time that ERO was constantly ordering KEK to unbundle. This action has not only weakened the ERO, but it is also not in line with the concept of unbundling KEK for the purpose of the KEDS transaction. Donor Environment. The donor environment has greatly shaped USAID energy assistance in Kosovo. IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO 31 The US Govern ment has played a ke y role in secu ring Kosovo’s independence, which is recognized by 86 countries (as of June 2 012). After the war in 1999, the UN provided temporary de facto governance following the disbandment of the Kosovo Liberation Army (KLA). The donors agreed that the US would bear the c ost of the war while the EU wo uld bear the costs of reco nstruction. Given th e heavy level of international dono r participation at the beginning, it is not surprising that the donor environment has significantly shaped the energy sector, especially given that secure and reliable electri city is crucial for economic recovery. Today, the World Bank, European Commission (EC), US Government/ USAID, KfW and GIZ are the main donors in the Kosovo energy sector. Between 1999 and 2003, donors discussed the merits of refurbishing existing power plants and new generation potential. Also, as the GoK took f orm in the early 2000s with successful elect ions, they signed onto EU led treaties and regional energy commit ments. These commitments were based on a long term goal of Kosovo becoming part of the EU. As related earlier, USAID’s involvement in energy grew from being very limited around this time to supporting the development of a billing and collection system at KEK (2003-04) at the request of the EU. Still, the general strategy was to exit from the energy sector after this work was completed. However, around 2005, this strategy changed as USAID became involved fu rther in GoK support programs and the hiring of co ntractors to support KEK. This was motivated by parallel EU accession preparation work that required clear rules and changes in the structure of Kosovo’s energy sector. Working groups and donors such as World Bank and EU, along with USAID, took a leading role in the privatization of KEK and the develop ment of new co al-fired generating facilities. Today, there is a general consensus between Go K and donors on the distribution privatization, while there is some disagreement on coal generation, in line with the ambivalence of the EU to become involved in coal plants. Also, the World B ank, while supporting the new Kosovo C project, has been subject to m any external pressures on this issue since 2006, which has resulted in l ong delays in its intended (but necessary) participation in the transaction. Current ly, the World Bank must wait for an ESIA to be car ried out which, in turn, requires one full year air quality data. With th ese constraints, the much needed generation will not be available until 2019. Donor effects have arisen from two sources: i) lack of donor action and or accountabilit y; and ii) lack of donor coordination. Lack of donor action and or accountabilit y should be interpreted in the cont ext of additional factors such as external markets, internal Kosovo political dynamics and the management /execution of donor assistance. The following examples are substantiated by interviews and/ or reviews of decisions and results: • The Mission needed to react to circumstantial changes, leading to increased scope and spending. An example of this result on t he USAID energy program is the EU/ GoK decision to decommission Kosovo A. Given that the country was already experiencing a shortage of electricity and that KEK/ GoK budget co uld not sustain paying for increased electricity imports, this agreement influenced USAID to take greater actions to address the urgency of the situation. Consequently, there was an increase in scopes of work and funding, such as the energy component being added to the EMSG project. • Decreased trust and/ or confidence in inactive donors and increased trust in active donors by the beneficiaries. Numerous interviews with GoK beneficiar ies found that not all donors w ork and deli ver support at the same pace. In t his environment, USAID’s assistance is “fam ously” quickly mobilized and responsive, while the support of other donors takes m uch longer or is more prescriptive. This impact may have influenced USAID to take growing leadership among the donors, as well possible decreased trust by other donors of USAID’s agenda. • Created the potential for short term solutions. An example of this donor influence on USAID is found in the way the World Bank failed to take action on the environmental impact assessment preparation. Environmentalists’ protests against the World Bank’s willingnes s to support Kosovo C forced the World Bank to take no action and request help from USAID. In reaction this this donor inaction, U SAID used its flexibility to provide initial support for building air monitoring capabilities. However, both capabilities and the portable air measuring units are not sufficient to fill the need gaps in air q uality data for the Kosovo C investment package. IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO 32 Lack of donor coordination, which is c ontinuous occurrence, may have led t o the following effects on USAID assistance: • Leaves the Mission in a reactionary role, leading to increased scope and spending. A particularly poignant example is the decision to add Kosovo B refurbishment to the Kosovo C investment plan. • Creates potential for duplication and/ or inefficiency. In this case, the as signment of both EU and US advisors to ERO resulted in USAID’s contractors not being full y utilized and trusted initially by the beneficiary. ERO was approached b y USAID in 2008 re garding potential support. In 2009, USAID sent a letter to the chairm an of the board with a proposal to support ERO. USAID wanted to provide supp ort in secondary legislation, tariffs, codes an d connection charges and tariffs. Ho wever, ERO already had EU support on these matters since 2006. ERO found the structure of EU support to be well organized and hence, working with EU was effective. However, the y found USAID support to be broad and not well defined; hence, it took some time to figure out how to utilize USAID support. ERO, with input from the consultants, decided to se parate the tasks so that E U and USAID consultants helped to f ulfill different needs. From interviews with USAID contractors and ERO, this took 2 years, during which ti me the US contractors were not fully utilized. • Decreases cooperative approach and impact on beneficiaries. From the interviews, a nu mber of opinions of USAID as a donor in the Kosovo energy sector have emerged. While overall, the majority is pleased with USAID assistance, others have expressed concerns. A few concerns relate to the lack of a cooperative donor approach and perhaps a com petitive approach among donors. From some of the intervi ews emerges a negative impact on donor coordination as USAID tends to react t o the needs al most too quickly. In othe r words, other donors’ slow response increases the lik elihood for USAID to take unilateral action, furtheri ng the opinions expressed. The im pact on beneficiaries is very palpable, however, as many have openly expressed that they felt USAID helped the most and other donors helped less. Today, there is a general consensus between GoK and donors on the distribution transaction, while Kosovo B has recently been separated from the Kosovo C in vestment package. Interviews of key donor perso ns in Kosovo indicate that there is m uch better donor co-operation than before. F or example, over the latter pa rt of the 5 year period covered by this eva luation, the EC, USAID (both Washington and Mis sion offices), State Department and World Bank have hel d a conferen ce call every 2-4 weeks to discuss the two m ain energy transactions. Furthermore, the work of USAID in provi ding technical assistance and also management of the various stakeholders inv olved in t he Kosovo C i nvestment is appreciated, as n ow there is closer cooperation (USAID had largely absconded from this area in 2006 when it was understood that the World Bank was “taking over” the transaction). Another example of im provements is the upco ming EU/World Bank led donor conference in Kosovo. On the other hand, a recent example of the continuing lack of donor coord ination occurred only recently (June 2012), when the USAID Mission and some other stakeholders were “surprised” by a KfW study on integrating Kosovo B into a district heating project. H.3 SWOT Analysis As part of the’ integrated mixed methods’ approach called for by the project SOW, the evaluation team carried out a SWOT analysis of the USAID approach to program implementation, based upon the insights gained fro m the interviews held and the documents reviewed. Strengths of the USAID Approach. USAID is a major donor and has inje cted a significant level of fundi ng and resources into t he Kosovo energy sector. In com parison with the other la rge donor agencies, USAID has shown itself to be ver y responsive to cli ents’ immediate needs in a changing environm ent, able to identi fy and assess the importance of new issues as they emerge. In combination with the ability to mobilize funds relatively quickly and to get high quality experts briefed and on the ground without delay, this means that USAID is filling a vital donor niche that would otherwise remain empty. Moreover, during the course of the evaluation, a discernibl e stakeholder view emerged that USAID experts tend to be more highly regarded than those of other donors. Weaknesses of the USAID Approach. A fundamental weakness of the USAID approach lies in the absence of any clear statement defining the rationale for and the scope of its involvement in the Kosovo energy sector. At a high level, the two main objectives of USAID strategy in Kosovo are: 1) to promote democratic governance; and IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO 33 2) to support economic growth. In respect of the second objective, a desired intermed iate result is to ensure reliable and secure energ y supply through the pri vate sector and also to facilitate compliance with EU requirements so as to achieve, in the longer-term , eventual membership of the EU. These two fundamental objectives are “supported” by the “five pillars” of Kosovo’s energy strategy: 1. Decommissioning of Kosovo A 2. Rehabilitation of Kosovo B 3. 600 MW of new generation at Kosovo C 4. Promotion of energy efficiency 5. Promotion of, and increase in renewable generation capacity. However, this level of an alysis does not explain wh y USAID chose to get involved i n the way it di d. The problems caused by this unstructured strategy have undoubtedly diluted the impact of the USAID program , as discussed in the following paragraphs. For example, the policy of issuing loosely defined SOWs, as de scribed in the section on the KEDS project, while a potential strength, has also contributed to a number of weaknesses in the execution of the USAID energy sector program. These specifically occurred when there w ere other consultants already working with a client and there was no discernible work program to pursue. Working to very general terms of reference also made it more difficult for US AID to manage changes in, and expansion of, project SOWs - and budgets - with sufficient rigour. The most significant example of this was the large and rapid increase in the nature of Tetra Tech’s involvement at KEK. Although it was established during the evaluation process th at various s upervisory procedures do exist, the t eam concluded that there was insufficient application of internal checks and of m onitoring and evaluation p rocedures given the significant scale of proposed revisions. In certain areas of activity, the performance monitoring indicators were found to be wanting, for example, in the case of the KEK project, where the P erformance Based Management System (PBMS) of the contract or was based on his own notions of appropriate performance indicators and associated targets, all to meet objectives defined by the contractor himself. It is subm itted that the three original USAID-defined objectives in this case and the associated performance indicators were entire ly appropriate. However, the indicators changed with the changing objectives of the project, which not only made performance difficult to evaluate, but the objectives on which they were based were so mewhat questionable, as they did not originate from USAID but were based on the contractor’s perception of what th ey should be. Appropriate PBMS indicator s cannot be defined without firm and lasting objectives. In other areas, such as during the first t wo years of the AEAI contract, whilst the performance indicators were appropriate, they were inapplicable because of unavoidable delays in the KEDS transaction. This reflects failure of management to adapt to a changing situation rather than a problem with the performance indicators. Measuring results in the ar ea of “providing advice” (AEAI/Deloitte) is not si mple, because the advice given is often of a political/strategi c nature and its perceived success or otherwise is aff ected by many exogenous and unpredictable factors. However, it should be possible to agree to a step-by -step road map of where the client wants to progress and timescales are involved. The lack of continuity in respect of specialist energy staff has at times had an adverse effect on the impact of the USAID energy sector program. Inevitably, each individua l will bring his/her own distinctive approach to the job, but ther e have been periods when there was no energy specialist in post, putting pre ssure on the non￾specialist staff who had to tem porarily fill the gap. When such a hiatus occurs, it potentially gives contracted consultants more freedom - to influence USAID decision making and to interpret their own roles - t han they should reasonably be expected to have. Although capacity building and sustainability are frequently mentioned objectives in USAID documentation, in reality these are neglected activities. On-the-job-training is the most common form of training encountered, but that only addresses the ne eds of a rel atively small number of s enior managers who co me into contact with USAID contractors. There is little evidence of new techniques and methodologies and new way s of thinking permeating down into t he host organisations. This i s less of an i ssue with the smaller bodies, such as KOSTT and ERO, but more problematic in the larger organizations, such as KEK and GoK ministries, where culture change as well technical change is a high priority. IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO 34 Opportunities for USAID. USAID could develop and formalize a separate “fast response” capability. This requires a form ally structured schem e that will ba lance the benefits of a flexible SOW with a disciplined delivery mechanism, for exa mple, along the lines of DFID’s successful “Nigeria Infrastructure Advisory Facility” (NIAF). 6 What is innovative about the NIAF approach is that it delivers rapid technical assist ance in response to client demand that is in line with an annual business plan and critically targets technical assistance to those areas that can achieve the highest impact and deliver the maximum benefit for Nigeria. In general, there is a positive attitude to US involvement from the local community in Kosovo and this goodwill should be harnessed. For example, it is possible that USAID PR materials relating to the energy sector would be more readily accepted by the public than sim ilar messages coming from other sources. While USAID doe s produce short project briefs which are updated regularly , the public needs to see the bigger picture – what happens if people don’t pay bills, why prices have to be cost reflective, the consequences of wasting energy, etc. Given that energy sector reform needs to follow the EU blueprint, USAID could focus its attention on creating a fertile environment in which the restructured Kosovo ener gy sector can flourish. As an alternative to directly supporting the energy industry and its r elated institutions, USAID could suppor t the sector by ensuring that all the necessary institutional infrastructure needed to ensure sustainability is in place. This can be done by funding educational training schemes at all level s, capacity building programmes and exchanges for all those who come into contact with the sector (lawyers, judges, municipality officers, teachers, bankers, etc.), and by helping local SMEs to acquire the entrepreneurial, financial and technical skills that will be needed if the qualit y of energy supply to the public at large is to improve. Threats for USAID. Beyond high-level statements such as “economic growth requires reliable energy supply”, which, in turn, requires private se ctor investment, there is no clear strategic statement that justifies the detail of USAID’s involvement in the Kosovo energy sector. This is a shortcoming that needs to be addressed as a matter of urgency if decisions on future invo lvement are to be made in a coherent way. At present, the answer t o the question “Why is USAID involved in t he Kosovo energy sector?” appears to be “Because it is there.” The lack of a formal strategy - in te rms of a detailed statement explaining what USAID is intending to achieve, in what areas it wants to be involved, and what form its involvement should take - has resulted in vague project terms of reference, duplication of effort, unpr oductive conflict between consultants, mistrust and potential conflict of interest situations. Due to certai n events in the past, it is perceived by some stakeholders that US AID has ad opted, perhaps by default, a negative, defensive, un-transparent - even secretive stance - on energy issues. This ser ves to undermine the goodwill that people in general feel to wards the USA and creates a feeling of mistrust in respect of other USAID activities. The lack of any meaningful coordination in Kos ovo means that the overall donor pr ogram is not being optimized; at best, resources are being wasted; and that, at worst, disputes, conflicts and mistrust characterize the relationships between the main agencies and their consultants. Given its role as a major player, the question of how donor coordination can be i mproved and formalized should be addressed by USAID - and perhaps the recent appointment of an i ndividual with responsibility for donor coordination to the Prime Minister’s office will help to smooth the road ahead. I. LESSONS LEARNED Lesson 1: There has to be a well-defined strategy to rationalize USAID involvement in Kosovo. A high level strategic statement, such as “to im prove the s ecurity of energy supply,” is too gener al and justifies just about every kind of intervention, but in a very unfocussed way. There have to be m ore specific objectives to justify the USAID presence, related to specific areas where USAID has some kind of “competitive advantage” or special niche role to pla y (for example, in providing a “fast response” facility to quickly address unexpected or urgent issues). It is u nderstood that the State De partment is developing a five-year Country Development Cooperation Strategy (CDCS), which could address this issue. 6 http://www.dfid.gov.uk/news/latest-news/2010/nigeria-project-wins-award/ IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO 35 Lesson 2: Donor coordination is a key to effective support programs. Everyone is in favor of donor coordination, but no one really knows how it can b e achieved. Now that the PM’s Office has appointe d an individual (brought in from the Ministry of European Integration) with a specific responsibility to manage donor coordination, USAID could offer to provide support to that individual in bringing the various parties together (which is apparently being done according to USAID staff). USAID could take a proactive position and offer to instigate some kind of regular coordination forum – for the GoK’s benefit – e. g., a monthly/quarterly cocktail evening at the Swiss Diamond Hotel, where each major donor informally explains what they have been up to and what they are planning to do. Lesson 3: On balance, open-ended SOWs have had a negative impact. Open-ended SOWs do allow for a flexible approach, but on balance, in the complicated environment of Kosovo, they have had a negative impact. Either contractors have been left with a vacuum to fill, because there was no common understanding of what was required of them (e.g., the first two years of AEAI pr oject), or they have taken advantage of the lack of direction to set their terms of reference. Lesson 4: There is a need within the energy sector for a ‘fast track’ advisory service. USAID is seen as providing a fast and effective response when new issues emerge. However, this service has not been achieved as a deliberate strategy, but rather by default, because other donors generally take longer to deliver than USAID. There is a need for a ‘fast track’ advisory service to address important issues which arise unexpectedly but this should be set up deliberately and be structured so as to ensure that all the risks asso ciated with a fast response facility are properl y managed. So, for example, a special ‘fast response facility’ fund could be created and managed by a single manager with a small admin team. The facility should operate to a very well defined set of rules about t he size, scop e and durati on of t he assistance packages on offer and about what information prospective clients have to provide in order to qualify for support (consistent with overall strategy, impact on the energy sector and on stakeholders, detailed objectives, detailed SOW, detailed work plan, detailed budget). The fund gives a guarantee that applications will be pro cessed, within, say, a week and that resources will be mobilized within two weeks. So me kind of oversight committee will h ave to be established to process applications, but it is the manager’s responsibility to perform an initial filter of applications. Depending on the demand and success of the facility, the fund can be increased or reduced in subsequent years. It should be noted that the “short-term” nature of the fund should be complementary to longer term development objectives such as the reform and restructuring of the energy sector. As such, any application for funding from this source should demonstrate that the proposed usage is consistent with the overall strategy to meet these objectives. Lesson 5: Good procedures are of no use if they are not implemented properly. A large and powerful organisation such as USAID, must, of necessity, have well-established policies and procedures with wh ich to manage the risks it faces in the course of doing business. The particular risk that caused th e evaluation team some concern, particularly in light of very loosely defined project terms of reference - was that of “project scope creep” and the associated escalation of budgets. Although there are agreed procedures in pla ce to appraise new project proposals, to evaluate requests to extend the scope of existing project SOWs and to increase budgets, it is not clear that these proce dures are bei ng rigorously a pplied. The relevant docu mentation from the projects under review was made available to t he evaluation te am and, while the required sequence of analy ses and decisions was followed quite strictly, it was not so clear that the rationale for the proposals in question ha d been scrutinised in sufficient detail, or that adequate atten tion had been given to alternative optio ns (for example, in deciding to sub-contract Deloitte to AEAI, or in e xpanding the Tetra Tech involvement at KEK). The conclusion to be drawn is that havi ng sound procedures in place is not sufficient. The important thi ng is to make sure that those pr ocedures are properl y implemented, by challenging and q uestioning every single proposal. J. RECOMMENDATIONS FOR FUTURE PROGRAM DESIGN J.1 General In light of the issues that were identified during the c ourse of this evaluation exercise, the evaluation team has some outline recommendations to make concerning future program design. IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO 36 Firstly, it is proposed that each new pr oject proposal - as well as m ajor increases in scope of work for existing projects - should be initia ted through a concept paper, based on a thorough analysis of all the significant issues. These should include, amongst others, quality of design and relevance (that is, the fit of the project into the USAID strategy f or the energy sector), its impact on stakeholders, the expected level of cost and long-term sustainability. Secondly, project terms of reference should be more detailed and results driven, with appropriate M&E criteria that will reflect the extent to which the project is achieving its stated objectives. In the case of “advisory services,” this may require some thought. However, following are suggested performance metrics for consideration, should a similar program such as EMSG be carried out:  Require a desired outcome/goal for each stakeholder the contractors engage with  Require and track number of persons in meetings, workshops and those trained  Track and report on the teaching materials and protocols developed  Require surveys and results of responses to track the number of satisfied participants In the case of environmental due diligence, while USAID has a clear process for initial potential environmental impact assessment, the use of only the IEE as a performance indicator encourages viewing environmental issues solely as risks and liabilities. In the case of Kosovo, aw areness or process that would have encouraged strategic thinking on how to incorporate environmental benefits may have been more effective. Foreseeable results may have been: i) a fully functional air monitoring station that is ISO accredited and a potential source of additional jobs; ii) contribution to the learning institutions of new technologies and methods; and iii) appropriate responses to some of the environm ental criticism. More importantly, Kosovo could have developed by 2012 one year of quality air monitoring data if a stationary air monitoring station was already built, which is critical for the World Bank and GoK to move forward for the new generation. The following are suggested perfor mance metrics for consideration:  Require IEE to be originated by the resident environmental officer  Add a section to the IEE for “opportunities for environmental leverage or benefits”  Introduce a beneficiary survey to those trained, to be reported to the Mission by the contractor  Introduce a few perfor mance indicators that will be ab le to access public perception, such as: nu mber times environmental issues are mentioned per week, number of environment related health issues, number of people trained Following are suggested performance metrics for training programs:  Track number of trainings, number of participants  Use survey to collect the number of satisfied participants with trainings provided  Track the number students who received internships  In the CENR program, a potential metric would be surveys of the AUK department regarding perceptions of the new center Thirdly, as a matter of routine, regular audits and project evaluations should be carried out at project level, to ensure compliance with the agreed project para meters, and to identify any “project drift” away from the agreed budget and SOW. Fourth, more attention must be given to ensuring that ever y proposal is beneficiary driven. Even if on occasion this means that a project is only approved by, rather than instigated by the client, it is essential to ensure that the client is full y aware of why the proposal is bein g made and formally signs on to the SOW, committing to participate in and support the planned initiative. This would avoid situations arising where consultants arrive to support organizations such as ERO, on ly to have to wait a couple of years before undertaking any meaningful work for them. Following on from this, consensus building, particularly with civil society, should become a formally recognised USAID obligation. Whil st this m ay consume significant resources, there will undoubt edly be a positive payback. The evaluation team recommends that t he PR resource working for Deloitte within the KEDS privatisation PIU - and th e material produced b y that resource - should be consid ered as a model of good practice. IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO 37 Project proposals should be communicated to other donors working in the energy sector to ensure more effective donor coordination, particularly with t he EU. A standa rdized process to achieve this m ight be pr oposed by USAID, encouraging regular communication but keepi ng the adm inistrative burden to a minimum. It is understood from Mission staff that regular meetings have been taking place since the fall of 2011. Finally, in light of som e serious but unsubstantiated issues that were raised by certain inte rviewees during the course of this project, the evaluation team recommends that USAID should establish a fo rmal procedure for recording and dealing with com plaints. Whether or not these com plaints are upheld, it is important that all participants in what is still a very problematic energy sector feel that they will receive a fair hearing, and those who may be abusing their positions should feel that there is an oversight process in place that will recognize and put a stop to unreasonable behavior. This process might take the form of a ‘USAID ombudsman’ or perhaps be based on the World Bank’ s ‘Inspection Panel’ (see www.inspectionpanel.org). As the complaints heard are unsubstantiated, the evalu ation team f eels it is ina ppropriate and beyond the scope of t his assignment to elaborate on them in this report. The is sue is raised only because the interviews revealed the apparent absence of a process for dealing with such complaints. There is an e mail address at USAID whe re people can raise issues and ask for information; however, the issue apparently ends up in the hands of someone directly involved in the project, wherea s the intention should be to ha ve the complaint addressed by a completely independent party, probably in Washington DC. J.2 Environmental Considerations Following are strategic recommendations for environmental due diligence:  Help strengthen Kosovo environmental institutions, in close cooperation with the EU environmental directorate and the US EPA, with the goal of establishing an ISO accredited environmental monitoring services laboratory. This will result in additional benefits such as job creation. The institutional strengthening may include – Development of technical exchanges between US, EU and Kosovo institutes and universities for technology transfer and capacity development in a cost effective manner – Use of EU and US government experts for quality control and helping to prioritize proposals from contractors  Improve environmental communication and messaging. Given the World Bank’s energy public awareness campaign that is planned, the USAID Mission might consider coordinating or developing a supplemental sound-bite to increase transparency of its environmental achievements in the energy sector. J.3 Specific Areas of Involvement for the Future USAID is currently involved in four different organizati ons in the energy sector: 1) KEK; 2) GoK (specifi cally, the MED); 3) ERO; and 4) KOSTT. This invol vement represents a co mmitment to these organizations that should be carried through to its conclusion. The KEK work will end June 2013, while there is a perceived need to continue providing advice and expertise to the ot her three org anizations. Any new work here shoul d be subject to the general recommendations provided in t he previous section. Seeing that USAI D is the incumbent in terms of providing services to these parties, potential c onflicts of interest can be managed (as they have been over the past few years) – as opposed to withdrawing completely from one organization to eliminate a possible conflict of interest. However, completely separate contracts are desirable. Beyond the short-term, greater emphasis should be placed on institutional capacity building in support of long￾term economic growth, which is in lin e with the second m ain objective of USAID strategy in Kosovo. A few areas of possible involvement are provided below: 1. Support and build the role of government to set energy sector policies. As mentioned earlier, the presence of USAID contractors within the GoK to provide advice on strate gic directions has bee n beneficial in pushing al ong the privatization agenda. As gathered from the interviews, this advice has been gratefully received and there is a strong desire within the GoK to retain these services. However, the requirement to have advisors present on an on going basis within the G oK suggests that some capacity building is required so that the GoK may eventually make such decisions independently. IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO 38 In most countries (as well as in Kosovo), governm ent policies are dictated at least partiall y by the political agenda that gets them elected. However, politicians and ministers rely on a civil service to provide advice on str ategy, and, in many instances, policy. Some thought should be given to how a strong and well educated civil service (or other GoK advisory group) can be developed and the specific support that could be provided to accomplish this. 2. Invest in schools and relevant programs. Partnerships such as those with NARUC and RIT should be encouraged and expanded. The NARUC partnership is fairly specific and is aligned with the current needs of the ERO. However, the RIT partnership is more far-reaching, as it c overs a wider area of the energy sector and is aimed at education requirements to fulfil modern energy sector needs. The RIT-AUK partnership is a step in t he right direction, but it is quite limited in that it serves onl y the AUK. A few interviewees mentioned that Kosovo’s educational system is producing graduates who are not properly equipped for current needs. A more extensive partnership undertaken in collaboration with say, the University of Pristina, along with more intensive interaction with formal energy related courses and programs would have a greater impact. 3. Support the Rule of Law in the energy sector. Tetra T ech’s failure to r each original objectives related to nontechnical losses, as explained in detail in section C.5, m ight be largely due to lack of enforcement - an area beyond KEK’ s control and more associated with pr oblems within the Kosovo judicial system. A Task Force to address commercial losses had been formed during the earlier part o f Tetra Tech’s mandate but was disband ed with the job of reduci ng these losses not quite done. It is understood that there had also been some work undertaken in this area in collaboration with Checci and Company Consulting, who are currently engaged on a judicial/ legal reform project in Kosovo fun ded by USAID. The results of this collaboration are not yet obvious. In any case, if the judicial/ legal area is where the problem of nontechnical losses fundam entally lies, then greater efforts should be made to concentrate specifically on electricity theft. If Tetra Tech is hindered by the legal system despite doing everything it can to eliminate theft, then the new investo r will face the same obstacles. The scope of the current legal/ judicial refor m project is not known. Perhaps the issue of addressing electricity theft from this angle is even a work-in-progress. If not, the reasons for Tetra Tech’s failure in this area should be investigated and more intense efforts should be made to address them. 4. Energy entrepreneurship. T he electricity suppl y industry is l arge. Indeed , KEK is currently the country’s largest employer. Consequently, many local enterprises benefit from KEK’ s business. With energy sector restructurin g and em phasis on the five pillars of Kosovo’s energy strategy, new opportunities will arise in areas of energy conservation (EC) and renewable energy (RE). USAID can help encourage the growt h of local enterprises to support these endeavours. One area that comes t o mind is energy service companies (ESCOs) that specialize in energy conservation. As there will likely be funding availa ble from other sources to help the GoK develop programs to encourage EC/ RE, there is no need for USAID to advise the GoK in these areas. Such programs are currently very popular with don ors. Nevertheless, demand side management (DSM) is likely a new concept in Kosovo that h as great potential, across all categories of consu mer, including i ndustrial, commercial, government and residential. Potential savings from energy conservation at the consumer level are probably quite significant. As GoK EC/RE programs are conceived and im plemented, a need for specialized services, fr om conducting industrial audits to insulating hom es, will grow, along with services to advise how various consumers can take advantage of these programs. USAID can help in setting up programs that will facilitate local small and medium enterprise (SME) growth in supporting t he energy sector, not only in EC/ RE, but also perhaps in other service areas. Kosovo is emerging from an economy that was not only socialist, but also dominated by Serbians, who have left. While the iron curtain across Europe fell over 20 years ago and former Eastern Bloc countries have integrated with Western Europe t o varying degrees, Kosovo has, due to various circumstances, been largely unable to develop capacity for entrepreneurship. Thus, although SME support for ESCOs is an obvious starting point, other similar energy sector o pportunities may exist. IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO ANNEXES APPENDIX A: TIMELINE OF USAID INVOLVEMENT IN THE KOSOVO ENERGY SECTOR ENERGY PORTFOLIO TIMELINE: MANAGEMENT COVERAGE AND ENERGY PROJECTS USAID/Kosovo Mission Directors Ken Yamashita – 2004-2006 Michael Farbman – 2006-2008 Patricia Rader – 2009-2011 Acting Mission Director: Azza El Abd – 2011 Senior Mission Adviser: Craig Buck – 2011 Directors of USAID/Kosovo Economic Growth Office: Sharon Hester – 2003-2008 (first as Senior Economic Policy Advisor and then as EGO Director, and then also Senior Economic Policy Advisor overlapping with Bill Lawrence – see further below) Naren Chanmugam – August 2007 through June 2009 Marry Hobs – July 2009 through July 2011 Acting Directors of USAID/Kosovo Economic Growth Office: Tim Hammann – September 2011 through March 2012 Elizabeth Santucci – from April 2012 CORs for Deloitte (formerly BearingPoint)/EMSG Project: Sharon Hester – March 2004 through June 2007 (including COR-ship for preceding Kosovo Economic Development Project, which also included an Energy component) Luan Gashi – June 2007 through September 2010 Senior Economic Policy Advisor (closely involved in the oversight of the EMSG) Bill Lawrence – from May 2008 through April 2012 COR for Tetra Tech/KEK Network and Supply Project, and AEAI (Advanced Engineering Associates International)/Advisory Services to Assist in Privatization of KEK Network and Supply Arben Nagavci – from award of original TOs (both mentioned above) COR for AEAI /Preparation of the Scoping Statement for Environmental Assessment for Rehabilitation and Possible Expansion of Kosovo B Power Plant, (including Air Quality Monitoring data collection) Michael Boyd – from the award of original TO through August 2010 Edi Shyti – from August 2010 Senior Energy Advisors Michael Trainor – from March 2007 through November 2008 Michael Boyd – February 2009 through August 2010 Roxanne Suratgar – from September 2011 US Ambassadors: Tina Kaidanow – 2007-2009 Christopher Dell – 2009-2012 (cont. on next page) Deloitte Consulting LLP (formerly BearingPoint)/Energy Component under the Economic Management for Stability and Growth (EMSG) Project Note: EMSG was a large omnibus program that provided technical assistance to the former Ministry of Economy and Finance (MEF) on fiscal, tax and other economic development issues. The description covers only the EMSG energy component. The preceding Kosovo Economic Development Project (also implemented by Deloitte/formerly BearingPoint) also included an Energy component i.e. Advisors to former Ministry of Energy and Mines (MEM). The EMSG Energy component described below is the succeeding program of KED Project regarding USAID Energy portfolio that is subject to the ongoing evaluation. Duration: June 2007 – September 2010 Ceiling Price: ca. $5.5 Million Under the EMSG, the focus of Energy component of the program included:  Technical and workflow management support to the World Bank funded Lignite Power Technical Assistance Project (LPTAP) on New Kosovo Power Plant NKPP project a.k.a. “Kosovo C” Project Office (i.e. Project Implementation Unit) and Project Steering Committee – assistance ended in 2008.  Assistance to Energy Regulatory Office to strengthen monitoring, operations and implementation of regulations – assistance ended in 2008, and  Legal/policy advice to the former Ministry of Energy and Mines MEM - ended in 2009.  From October 2009 through September 2010, the EMSG Energy component provided targeted support to the former Ministry of Economy and Finance (MEF) and the Government of Kosovo Steering Committees and Transaction Advisors for KEDS and NKPP, in order to guide and focus their work to expedite the ambitious transaction timelines adopted by the Government. This assistance sought to ensure that ongoing Kosovo energy sector transactions move in a parallel and coordinated fashion, working to ensure the interests of Kosovo are served while promoting optimal conditions for sovereign decision making and action across the spectrum of energy sector issues, options, challenges, and opportunities. (cont. on next page) Tetra Tech/KEK Network and Supply Project Original task Order (TO) Award Date: 12/26/2006 Ceiling: $4,749,382 Period of performance: Two yours, through 12/17/2008 SOW (Three Task Areas): 1. Support to the Managing Director of KEK 2. Increase collections through support to the Network and Supply Divisions to accelerate potential future private sector participation 3. Improve the institutional, policy and legal environment First Mod Award date: 7/10/2008 Ceiling increase/Total ceiling: $8,000,000 / $12,749,382 Time extension: 18 months, through 6/30/2010 Addendum to SOW Additional five Task Areas 4. Preparation of KEK Distribution Functions for Privatization 5. Support to KEK Commercialization 6. Anti-Corruption Efforts – The Reduction and Prosecution of Fraud, Waste, and Abuse 7. Legal and Regulatory Support 8. Normalization of Service to Enclave Communities Second Mod Award date: 12/17/2009 Ceiling increase/Total ceiling: $11,600,000 / $24,349,382 Time extension: 18 months, through 12/16/2011 Addendum to SOW Realignment (consolidation) of Task Areas 1. Distribution Company Privatization Support a) Support for technical preparation of the Distribution Company for privatization b) Assistance with post-privatization implementation for Distribution Company 2. Privatization Support for TPP Kosovo B Third Mod (no cost extension) Award Date: 12/19/2011 Time extension: Three months, through 3/31/2012 Addendum to SOW: No changes Fourth Mod Award Date: 3/30/2012 Ceiling increase: $4,250,291 / $28,599,673 Time extension: 15 months through 6/30/2013 Addendum to SOW: Realignment (consolidation) of Task Areas 4. Support Management and Operations to Maintain (Preserve) Asset Value: 5. Provide Advisory Support in the KEDS Privatization Process Including Limited Assistance Post￾Privatization 6. Provide Support for Privatization of Remaining KEK’s Assets SOW also includes requirements to ensure compliance with conditions set forth in the respective IEE Amendment. (cont. on next page) AEAI (Advanced Engineering Associates International)/Advisory Services to Assist in Privatization of KEK Network and Supply Original TO Award Date: 9/30/2009 Ceiling: $2,899,725 Period of performance: Two years, through 9/30/2011 SOW (Two Task Areas): 1. Regulatory support to ensure a successful distribution company privatization transaction and post￾privatization implementation 2. Market rules and related procedures and Codes to support distribution company privatization First Modification Award date: 9/29/2010 Ceiling increase/Total ceiling: $3,300,000 / $6,199,575 Time extension: 15 Months, through 12/20/2012 Addendum to SOW Additional Two Task Areas 3. Privatization Advisory Services to the Ministry of Economy and Finance, the Inter-Ministerial Privatization Committee for DistCo, and other key Government of Kosovo stakeholders (subcontracted to Deloitte) 4. Independent study of KEK 2009 commercial and technical losses including uncollected revenue (actual study carried out for 2010 loses, by another sub to AEAI) Second Modification Award date needed before: 3/30/2012 Ceiling increase/Total ceiling: $1,625,142 / $7,824,717 Time extension: Two months, through 2/28/2013 Addendum to SOW Revised Task Areas to reflect MED (Ministry of Economic Development) transaction timeline, and Budget Realignment 1. Advisory Support to ERO (Energy Regulatory Office) 2. Advisory Support to KOSTT/Electricity Transmission and Market Operator (on market design and rules) 3. Privatization Advisory Services to MED 4. Independent KEK Losses Study (limited follow up related to potential bidders’ due diligence) (cont. on next page) AEAI /Preparation of the Scoping Statement for Environmental Assessment for Rehabilitation and Possible Expansion of Kosovo B Power Plant, (including Air Quality Monitoring data collection) Original TO Award Date: 01/22/2010 Ceiling: $600,000 Period of performance: Three months, through 04/30/2010 1. Preparation of the “Scoping Statement” for Environmental Assessment for the rehabilitation and potential for expansion of Thermal Power Plan (TPP) Kosovo B and ancillary facilities” First Mod (no cost extension) Change in the Estimated Completion Date and Scope of Work Award date: 6/4/2010 Ceiling increase/Total ceiling: $0 Time extension: 4 months, through 8/31/2010 Addendum to SOW Additional tasks 1. Development of Air Monitoring Plan and Training Plan 2. Technical Specifications to procure a stationary and mobile air quality monitoring program (for World Bank) 3. Institutional Capacity Building Plan Second Mod (no cost extension) Award date: 09/3/2010 Ceiling increase/Total ceiling: $0 Time extension: 5 months, through 1/31/2011 (followed by 60 day COR no-cost extension, through 4/1/2011) Addendum to SOW Additional tasks 1. Purchase of Air Pollutant Dispersion Model and Training on Air Dispersion Modeling Third Mod (no cost extension) Award Date: 05/2/2011 Time extension: One year, through 1/31/2012 Addendum to SOW: Additional Tasks 1. Provide technical assistance to build institutional capacity in ambient air quality data collection that will be needed as part of an environmental assessment (EA) developed for NKPP. 2. Gather information that can be used by USAID/Kosovo to evaluate whether the environmental activities proposed for NKPP are aligned with the recommendations outlined in the TPP Unit B Scoping Statement. Fourth Mod Award Date: 3/21/2012, which is the end date of the Contract Ceiling increase: $1,452,365 / $2,052,364.64 Time extension: 14 months through 3/31/2013 Addendum to SOW: Realignment/additional tasks and budget realignment 1. Air Quality Monitoring Data Collection as baseline for an EA for (GenCo) NKPP 2. Environmental Assessment of the Feasibility Study to Rehabilitate and Extend the Life of the Kosovo B units prepared under Tetra Tech/KEK Network and Supply Project 3. Environmental Audit of Previous Tetra Tech/KEK Network and Supply Project activities. END APPENDIX B: LIST OF DOCUMENTS AND INFORMATION RECEIVED Documents and Information Received by the Evaluation Team Provided by USAID before departure to Kosovo: 1. AEAI 3rd Quarterly Report 2010 2. AEAI 2010 Quarterly Report Q1 FINAL 3. AEAI QUARTERLY REPORT April June 2011 FINAL 4. AEAI QUARTERLY REPORT January March 2011 5. AEAI QUARTERLY REPORT January to March 2012 6. AEAI QUARTERLY REPORT July September 2011 7. AEAI QUARTERLY REPORT Oct-Dec 2010 FINAL 8. AEAI QUARTERLY REPORT October - December 2011 FINAL 9. AEAI Quarterly Report Q2 April - June 2010 10. AEAI Quarterly Report Q4 2009 11. Kosovo B Final Scoping Statement (Revised 2011) 12. Tetra Tech - KEK 2010 Annual Report March 2011 13. Tetra Tech - KEK 2011 Annual Report Final Draft 14. Tetra Tech 2007 Annual Report Final 15. Tetra Tech 2008 Annual Report Final Draft April 2009 16. Tetra Tech 2009 Annual Report Final 2 Additional Information Provided by USAID in Kosovo: 17. Independent Review of KEK Distribution Losses for 2010 Final Report (3 volumes) 18. Modification 4 to AEAI contract, March 2012 19. AEAI contract, September 2009 20. Modification 1 to AEAI contract, March 2010 21. Modification 2 to AEAI contract, June 2010 22. Modification 3 to AEAI contract, September 2010 23. KEDS PBMS Report Final, February 2010 24. KEDS PBMS Report Draft, January 2010 25. KEDS Second Annual Work Plan, December 2010 26. Additional Task on Air Quality Monitoring (undated) 27. Amended scope of work for Task 11 (undated) 28. April 2012 AEAI Cadmus Work Plan 29. Contract for Preparation of Scoping Statement, January 2010 30. Final Draft Work Plan for Preparation of Scoping Statement (undated) 31. Final Ambient Air Monitoring Plan for Kosovo, January 2011 32. Final Capacity Building Plan for Kosovo Air Monitoring, January 2011 33. Modification 1 to air monitoring contract with AEAI, March 2010 34. Modification 2 to air monitoring contract with AEAI, June 2010 35. Modification 3 to air monitoring contract with AEAI, September 2010 36. Modification 4 to air monitoring contract with AEAI, April 2011 37. Modification 5 to air monitoring contract with AEAI, March 2012 38. NKPP Capacity Building AEAI Cadmus Final Work Plan, June 2011 39. Revised NKPP Capacity Building AEAI Cadmus Work Plan, December 2011 40. USAID Kosovo Additional Task Air Quality Monitoring, May-June 20 10 41. USAID Kosovo Environmental Task Order - Air Dispersion Modeling SOW, August 2010 42. KEK Financial Statements and Independent Auditor’s Report 2006 43. KEK Financial Statements and Independent Auditor’s Report 2007 44. KEK Financial Statements and Independent Auditor’s Report 2008 45. KEK Financial Statements and Independent Auditor’s Report 2009 46. KEK Financial Statements and Independent Auditor’s Report 2010 47. KEK Financial Statements and Independent Auditor’s Report – unbundled divisions -2010 48. KEK Financial Statements and Independent Auditor’s Report 2011 49. IFC Information Memorandum - Overview of the Kosovo Energy Sector, Legal Framework, and Economic and Political Environment, May 2011 50. Ministry of Economic Development - Publicly Owned Enterprises Performance Report 2010 51. Ministry of Economic Development - Assessment of business performance of Central POE Boards of Directors for 2010 52. World Bank letter to Arben Gjukaj, April 2012 53. EMSG 1st Annual Report, July 2008 final 54. EMSG 2nd Annual Report, July 2009 final 55. EMSG 3rd Annual Report, July 2010 final 56. EMSG Final Project Report, August 2010 57. Generation Investment Study Update Volume 1, January 2007 58. Generation Investment Study Update Volume 2, January 2007 59. Generation Investment Study, Volume 1, Executive Summary, December 2004 60. Generation Investment Study, Volume 2, Electricity Demand Forecast, December 2004 61. Generation Investment Study, Volume 3, Generation and Transmission Main Report, December 2004 62. Generation Investment Study, Volume 4, Demand - Appendices, December 2004 63. Generation Investment Study, Volume 5, Generation and Transmission Appendices, December 2004 64. Generation Investment Study, Volume 6, PSS/E Analyses and Results (Appendix 13), December 2004 65. Initial Environmental Examinations filed for various energy projects, 2005 to 2012 (10 in all) 66. Positive Determination Approval Sheet, 2009 (blank) 67. Technical Direction to PA re metering program, May 2009 68. Portfolio Review Sheet, KEK Network and Supply Project, February 2009 69. Portfolio Review Sheet, KEK Network and Supply Project, March 2008 70. Portfolio Review (all ongoing projects), March 2010 71. KEK Network and Supply Project, Final Work Plan 2008-2009, October 2008 72. KEK Network and Supply Project, Revised Work Plan 2008-2009, July 2009 73. KEK Network and Supply Project, 2011 Annual Work Plan 74. KEK Network and Supply Project, 2010 Annual Work Plan 75. KEK Network and Supply Project, First Annual Work Plan 2007 76. KEK Network and Supply Project, RFTOP 77. KEK Network and Supply Project, Amendment 1 to RFTOP 78. KEK Network and Supply Project, Contract with PA Consulting 79. Kosovo B Investment Requirements and Rehabilitation Feasibility Study, August 2010 80. KEK Network and Supply Project, Addendum to SOW under Modification 7 81. KEK Network and Supply Project, PA Consulting proposal in response to RFTOP 82. NARUC, Final list of invitees for High Level Public Forum (undated) 83. NARUC, List of invitees to NARUC training on Energy Regulation and Tariff Development (undated) 84. USEA, Final Trip Report - KEK Management Trainings, 2007 85. USEA , Final Trip Report - Kosovo Privatization Workshop, December 2008 86. USEA, Progress Towards Privatization: An Update of the 2006 Qualitative Assessment, April 2009 87. PP presentation: Assessment of Energy Situation in Kosovo, by Dr. Robert Ichord and Mr. Andres Doernberg, January 2006 88. KEK project major achievements revised 14 June 2012 89. PP presentation: Kosovo Energy Matrix, by Dr. Robert Ichord and Mr. Andres Doernberg, January 2006 90. World Bank Background Paper: Development and Evaluation of Power Supply Options for Kosovo, December 2011 91. Kosovo Power Project Expert Panel Report, January 2012 92. USEA, Qualitative Assessment of Preparations for Transition to Local Management Within KEK, June 2006 93. USEA, Second Field Visit (August 2006) Report on Qualitative Assessment at KEK, August 2006 94. PP presentation: USAID Role in Kosovo Energy Sector, by Dr. Robert Ichord, June 2006 95. Various internal USAID memorandums justifying increases in scopes of work for the KEK Network Supply project and the KEDS project, 2006-2011 96. USAID/ Kosovo Strategic Plan 2010-2014, May 2010 97. Energy Portfolio Timeline: Management Coverage and Energy Projects Information received from stakeholders: 98. List of AEAI deliverables (actual deliverables contained on a separate CD) 99. KEK organization charts 100. Sample KEK Energy Accounting Report 101. World Bank pamphlet – Inspection Panel 102. World Bank pamphlet – The World Bank in Kosovo 103. KOSTT information pamphlet on 400 kV interconnection line with Albania 104. KEK Information Packet for Stakeholders and Media, Update: April 2012 105. Total KEK energy Accounting: Five Months Ended May 2008-2012 106. PP presentation: KEK Overview, Challenges and Opportunities, by Masoud Keyan, October 2007 107. PP presentation: KEK Generation Capacity Issues, by Arben Gjukaj, March 2012 108. PP presentation: Review of KEK’s Present Status and Future Plans, by Masoud Keyan, March 2007 109. KEK Employment Manual APPENDIX C: LIST OF INTERVIEWS List of Meetings 1 2 3 14.06.12 14.06.12 14.06.02 USAID (telcon), Michael Farbman, former USAID/Kosovo Director USAID, Washington, DC, Robert Archer, Steven Burns USAID, Federal South Plaza, William Gibson, Regional Environment Officer 4 5 15.06.12 15.06.12 Tetra Tech, Washington DC, David Keith, Tatyana AEAI (telcon), Celia Whitaker, Dr. Olga Mandrugina, Dr. Gopal Kadagathur 6 7 8 18.06.12 18.06.12 18.06.12 USAID, Prishtina, Ardian Spahiu, Roxanne Suratgar, Arben Nagavci, Edmond Shyti Parliament Building, Deputy Myzejene Selmani, AKR Party Deloitte offices, Project Implementation Unit, Fllanza Hoxha, KEK PIU 9 10 11 19.06.12 19.06.12 19.06.12 Tetra Tech, KEK office, Masoud Keyan, Llyr Rowlands, Ardian Spahiu, Roxanne Suratgar KEK Executive Directors, Arben Gjukaj, Managing Director, Salih Bytyqi, CFO 12 19.06.12 KOSTT office, Skender Gjonbalaj, Market Operating Director 13 14 15 20.06.12 20.06.12 20.06.12 Offices of Coal Division, Adil Januzi, Executive Director Kosovo A office, Obilic, Hamdi Gashi, Manager Kosovo A Kosovo B office, Obilic, Luigj Imeri, Director Kosovo B 16 17 18 19 20 21.06.12 21.06.12 21.06.12 21.06.12 21.06.12 Tetra Tech, KEK office, Krassimir Kanev, Finance/Accounting Advisor, TT Ministry for Economic Development, Besim Beqaj, Minister, Liridon Mavriqi, Advisor KEK Board of Directors, Fadil Citaku, Muje Rugova, Arben Gjukaj IMF, Central Bank Building, Selim Thaci, Economist, IMF KEK office, Givi Gjarkava, Audit Manager 21 22 22.06.12 22.06.12 KEK office, Fadil Sejdiu, HR Director KEK Swiss Diamond Hotel, USAID, Ardian Spahiu and Roxanne Suratgar 23 24 25.06.12 25.06.12 Swiss Diamond Hotel, Embassy of USA, Andrea J. Tomaszewicz, Senior Economic Officer Swiss Diamond Hotel, Felanze Pula, Former KEK Manager 25 26 25.06.12 25.06.12 Hotel Prishtina, EU adviser, KOSTT, Les Clarke, IPA KEK offices, Llyr Rowlands, Tetra Tech, Lawyer 27 28 29 26.06.12 26.06.12 26.06.12 Deloitte offices, Andrew Smith, Deloitte Consulting Transaction Adviser INDEP office, Krenar Gashi, Executive Director, Rinora Gojani, Researcher KIPRED office, Ardian Arifaj, Programme Coordinator 30 31 32 27.06.12 27.06.12 27.06.12 KOSTT office, Naim Bejtullahu, Deputy CEO for Development Air Monitoring Institute Prof. Syle Tahirsylaj (Director of HMK), Letafete Latifi (HMI) AEAI office, Celia Whitaker, AEAI 33 34 28.06.12 28.06.12 Sirius Hotel, USEA Project team Sirius Hotel, Arben Nagavci, Edi Shyti 35 36 37 29.06.12 29.06.12 29.06.12 KEK local employees – Ismet Latifi, Petrit Pepaj, Nermine Arapi EU Office in Kosovo, Mr. Asin, Deputy Head of Operations USAID Office, Arben Nagavci, Edi Shyti 38 39 40 02.07.12 02.07.12 02.07.12 University of Education, Ethem Ceku Air Monitoring Institute/Station, Director and Lab Technician ERO Office, Prishtina 41 03.07.12 LPTAP, Lorik Haxhiu, Project Adviser 42 04.07.12 Office of the Prime Minister, Mrika Kotorri, Adviser 43 44 45 05.07.12 05.07.12 05.07.12 World Bank, Mr. Jan-Peter Olters, Manager World Bank Office in Kosovo KfW, Bahrie Dibra, Project Coordinator for the Financial and Energy Sectors USAID, Ardian Spahiu APPENDIX D: ILLUSTRATIVE INTERVIEW QUESTIONS EVALUATION QUESTIONS BENEFICIARIES AND RECIPIENTS OF ASSISTANCE (KEK, KEDS, NKPP/LPTAP, GOK, KOSTT, ERO) PROGRAMMATIC IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO For beneficiaries and recipients of assistance: 1. What is the nature of your organization? What functions does your organization undertake? By what mandate? 2. What is your relationship with USAID? Through which project? 3. What is the nature of the assistance being provided? 4. What have been the results to date? 5. How satisfied are you with the experience with the USAID assistance? How would you rate your level of satisfaction on a scale of 1 – 10 (10 highest)? 6. Had you not been able to obtain this assistance, what other alternatives would have been available to you? 7. How effective do you view the USAID contractor’s work of implementing the USAID assistance? Can you provide comments about the experience, qualifications, and effectiveness of the contractor team? 8. Can you provide the evaluation team with information to help us understand the impact of the assistance on your activities (e.g., for KEK, it might be decreased losses or increased collection efficiencies)? 9. How sustainable are the impacts of the assistance? Do you anticipate that your organization will continue with the same practices after the USAID assistance has finished? 10. Do you see the assistance as being relevant in light of the current regulatory and legal environment in Kosovo? 11. What, in your opinion, is the main constraint acting on USAID assistance in Kosovo? 12. What, if any, kind of training have you received? What kind of training? Should this training be augmented? How? 13. In your view, how effective is the USAID assistance? Do you believe that it has achieved what it set out to do? 14. In your opinion, how could the assistance be improved? 15. What other types of assistance could be offered by USAID to the energy sector in Kosovo? 16. Do you have a specific recommendation that the evaluation team could provide to USAID to help it improve future assistance? In light of your experience, what advice would you give USAID? EVALUATION QUESTIONS INTERNATIONAL PARTNERS PROGRAMMATIC IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO For international partners: 1) Can you please summarize your partnership/ relationship with USAID/Kosovo for assistance to the Kosovo energy sector? 2) Are you involved in energy sector assistance projects in Kosovo other than those with USAID? What are they? 3) What are the respective roles of you, USAID/Kosovo and USAID/Washington with regard to your partnership/ collaboration? 4) How satisfied are you with the USAID assistance? What are the strongest points of USAID energy sector assistance? Are any improvements needed? 5) In your opinion, how successful is USAID energy sector assistance? How well has it met GOK objectives? 6) How effective do you consider the work to be of implementing the assistance provided? Do you have any observations about the experience, qualifications, and effectiveness of the implementing agency/ contractor? 7) How sustainable are the improvements brought about by the assistance? Will the changes continue once the assistance has ended? 8) Are there any areas of Kosovo energy sector assistance in which provision of assistance might be more effective? 9) What, in your opinion, is the main constraint acting on USAID assistance in Kosovo? 10) Do you see the particular USAID assistance highly appropriate and relevant, in light of the current regulatory and legal environment in Kosovo? Do you believe it fits well within the desired development strategy for Kosovo? 11) Are the objectives of the assistance being accomplished? Please elaborate. 12) Do you have any information on effects of the project(s) gender-wise? 13) Do you have a performance monitoring plan for the projects with indicators that are regularly monitored? If so, what are they? Who keeps track of them? Are they available? 14) Would you be interested in collaborating on any extension to the present assistance? Should any new, follow-on assistance be designed any differently than the present one? Please explain in detail. 15) Does your component of the project(s) have a human capital development component or an institutional strengthening component? If not, in your opinion, should there be? 16) Can you recommend any individuals, groups, or organizations in Kosovo the evaluation team should be sure to contact? 17) On a scale of 1 – 10 (10 highest), how would you rate USAID assistance to the Kosovo energy sector? 18) Do you have a specific recommendation that the evaluation team could provide to USAID to help develop similar assistance in the future? In light of your experience, what advice would you give USAID? EVALUATION QUESTIONS USAID CONTRACTORS PROGRAMMATIC IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO For USAID contractors, Tetra Tech and AECOM: 1) Can you please summarize your mandate for assistance to the Kosovo energy sector? Which particular project(s)? 2) Are you involved in energy sector assistance projects in Kosovo other than those with USAID? What are they? 3) What are the respective roles of you, USAID/Kosovo and USAID/Washington with regard to your involvement in the Kosovo energy sector? 4) In your opinion, how appropriate is the USAID assistance? What are the strongest points of USAID energy sector assistance? Are any improvements needed? 5) In your opinion, how successful is USAID energy sector assistance? How well has it met GOK objectives? 6) How effective do you consider the work to be of implementing the assistance provided? 7) How sustainable are the improvements brought about by the assistance? Will the changes continue once the assistance has ended? 8) Are there any areas of Kosovo energy sector assistance in which provision of assistance might be more effective? 9) What, in your opinion, is the main constraint acting on USAID assistance in Kosovo? 10) Do you see the particular USAID assistance highly appropriate and relevant, in light of the current regulatory and legal environment in Kosovo? Do you believe it fits well within the desired development strategy for Kosovo? 11) Are the objectives of the assistance being accomplished? Please elaborate. 12) Do you have any information on effects of the project(s) gender-wise? 13) Other than the performance indicators provided in the quarterly/ annual reports, do you have any other performance monitoring indicators that are regularly monitored? If so, what are they? Who keeps track of them? Are they available? 14) Should any new, follow-on assistance be designed any differently than the present one? Please explain in detail. 15) Does your component of the project(s) have a human capital development component or an institutional strengthening component? If not, in your opinion, should there be? 16) Can you recommend any individuals, groups, or organizations in Kosovo the evaluation team should be sure to contact? 17) On a scale of 1 – 10 (10 highest), how would you rate USAID assistance to the Kosovo energy sector? 18) Do you have a specific recommendation that the evaluation team could provide to USAID to help develop similar assistance in the future? In light of your experience, what advice would you give USAID? EVALUATION QUESTIONS KNOWLEDGEABLE THIRD PARTIES PROGRAMMATIC IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO For third parties such as donors, NGOs, and government officials: 1. What is the nature of your activities in the Kosovo energy sector? 2. What is your relationship with USAID assistance to the Kosovo energy sector? Which particular project(s)? 3. How would you rate your knowledge of USAID assistance on a scale of 1 – 10 (10 highest)? 4. In your opinion, how effective is the USAID assistance? Do you have any observations about the experience, qualifications, and effectiveness of the implementing contractors? 5. How sustainable is the assistance? Do you anticipate that the impacts will continue after the assistance ends? What do you think will happen when the assistance ends? 6. Do you see the assistance as highly appropriate and relevant, in light of the current regulatory and legal environment in Kosovo? Do you believe it fits well within the overall development strategy for the Kosovo energy sector? 7. In your view, how effective is the assistance? Do you believe that it has achieved what it set out to do? 8. In your opinion, how successful is USAID energy sector assistance? How well has it met GOK objectives? 9. Are there any areas of Kosovo energy sector assistance in which provision of assistance might be more effective? 10. Can you recommend any individuals, groups, or organizations in Kosovo the evaluation team should be sure to contact? 11. What, in your opinion, are the main constraints in effectively carrying out the assistance? 12. Should the projects have an increased human capital development component or an institutional strengthening component for any of the targeted organizations? 13. How would you rate overall, on a scale of 1 – 10 (10 highest), USAID assistance to the Kosovo energy sector? 14. How, in your opinion, should future USAID assistance differ from the current assistance? 15. What other options could be used by USAID to encourage growth and better efficiency in the Kosovo energy sector? 16. Do you have a specific recommendation that the evaluation team could provide to USAID to help develop similar assistance in the future? In light of your experience, what advice would you give USAID? EVALUATION QUESTIONS USAID/KOSOVO PROGRAMMATIC IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO For USAID/Kosovo: 1. How does USAID/Kosovo anticipate using this assessment? What are the special areas of concern? What should the assessment be sure to cover? 2. Can you briefly summarize USAID’s past involvement in Kosovo’s economic development and its specific plans for the future? 3. What is a rough estimate of the amount of annual funding that might possibly be available for USAID’s future energy sector development programs in Kosovo? 4. Are there policy or other issues that presently cause disagreement between USAID and GOK? 5. What are the respective roles and responsibilities of USAID/Kosovo, USAID/DCA/ Washington, and the Kosovo implementing agencies in terms of project implementation? What were the respective roles in the design of each of the projects? 6. What is the cost of each of the projects? Can the evaluation team obtain a cost summary of the project-to-date? 7. How do the energy projects fit with USAID’s development strategy for Kosovo? 8. Can you please provide the evaluation team with the performance monitoring plan for each of the projects with the targets for these indicators, and the latest reports on how well the targets have been achieved? 9. Are any other reports available, notably for the EMSG project, the Environmental Scoping project and other energy sector projects that would fall under “additional energy support”? 10. Can you please provide the evaluation team with annual progress reports for the projects? 11. In your view, how sustainable are the changes brought about by the projects? 12. How effective are the projects’ contractors? How effective are the targeted recipients of the assistance? 13. How important is energy sector assistance to USAID’s overall portfolio in Kosovo? How important is the project to USAID support to the electricity sub-sector? 14. What, in your opinion, are the main constraints in effectively carrying out the assistance? 15. Has USAID considered other assistance alternatives to the energy sector? If so, what alternatives were considered? 16. Did the design of any of the projects include remedial measures to overcome gender-based issues? Are any indicators regularly monitored that highlight the gender issue? 17. Other than “additional energy sector support”, do the projects have a human capital development component or an institutional strengthening component for any of the targeted organizations? If not, in your opinion, should these support activities be augmented? 18. Who are the key organizations or people in Kosovo that USAID feels that the evaluation team should meet while conducting the evaluation? EVALUATION QUESTIONS USAID/WASHINGTON, DC PROGRAMMATIC IMPACT EVALUATION IN THE ENERGY SECTOR IN KOSOVO For USAID/DC: 1. To the extent that USAID/Washington is engaged and aware of the detailed activities carried out in the energy sector since 2007 in Kosovo, can you please give us your perceptions of the results of this project? 2. Can you please provide the evaluation team with the performance monitoring plan for each of the projects with the targets for these indicators, and the latest report on how well the targets have been achieved? 3. Can you please provide the evaluation team with annual progress reports for the EMSG, Environmental Scoping and “additional support” projects? 4. Can you please provide the evaluation team with information on the performance of the energy sector assistance compared to the performance of USAID energy programs worldwide - or with other countries in the region? 5. Have any previous evaluations been performed of energy sector projects in Kosovo? Are these available? How would you rate these evaluations? 6. What are the respective roles of USAID/Kosovo and USAID/Washington with regard to each of the projects to be evaluated – KEK network, KEDS distribution privatization, EMSG, Environmental Scoping, additional support? 7. Does USAID/Washington have any particular interest of concern in this assessment? How does USAID/Washington anticipate using the assessment? What are the special areas of concern to USAID/Washington should the assessment be sure to cover? 8. Can USAID/Washington recommend any individuals, groups, or organizations in Kosovo the evaluation team should be sure to contact (in addition to USAID/Kosovo and implementing agencies)? 9. Are there any future plans for assistance in the Kosovo energy sector? Have any specific plans been made? 10. Other than “additional energy sector support”, do the projects have a human capital development component or an institutional strengthening component for any of the targeted organizations? If not, in your opinion, should these support activities be augmented? 11. How would you rate overall, on a scale of 1 – 10 (10 highest), the usefulness of USAID assistance to the Kosovo energy sector? 12. Has USAID considered other assistance alternatives to the energy sector? If so, what alternatives were considered? APPENDIX E: KEK NETWORK AND SUPPLY PROJECT – OBJECTIVES AND TASKS First Annual Work Plan, 2007 (March) Objectives with respective supporting tasks: Objective 1: Support to the Managing Director of KEK Task 1: Provide technical assistance to the Managing Director of KEK Objective 2: Increase Collections through support to the Network and Supply Divisions to accelerate potential future private sector participation Task 2: Evaluate the Network and Supply Divisions and make a recommendation on whether they should be combined into a single organizational unit Task 3: Advise on the interface between the Network Division and the Transmission/Market Operator Task 4: Advise on the operation of the Network Division with particular emphasis on reducing losses Task 5: Provide assistance and support the new anti-corruption department Task 6: Create a comprehensive metering program Task 7: Advise on the operation of the Supply Division with particular emphasis on increasing collections Task 8: Create a program for Revenue Cycle Management Task 9: Institute an annual external audit of Network and Supply Task 10: Prepare a priority investment plan for Network and Supply Objective 3: Improve the Institutional Policy and Legal Environment Task 11: Advise KEK on revising laws and regulations to enable KEK to be able to operate with international best practices Observations: Objectives precisely match those of the original SOW 2007 Annual Report Objectives: Objective 1: Provide support to the managing director of KEK, with an emphasis on enhancing the company’s revenues. Objective 2: Increase collections through support to the Network and Supply Divisions of KEK to accelerate potential future private sector participation (making recommendations on whether the divisions should be combined into a single organizational unit, advising on the interface between the Network Division and the transmission/market operator, providing support to the new Anti-Corruption Department, creating a comprehensive metering program, advising on the operation of the Supply Division, with emphasis on increasing collections, creating a program for revenue cycle management, instituting an annual external audit of the divisions, and preparing a priority investment plan for the divisions). Objective 3: Improve the institutional, policy and legal environments in which KEK operates, and advise KEK on the revision of laws and regulations to enable it to operate using international best practices Objective 4: Establish new control policies and improve the control function to reduce illegal and improper activities within KEK Tasks: Task 1: Provide technical assistance to the Managing Director of KEK Task 2: Evaluate the Network and Supply Divisions and make a recommendation on whether they should be combined into a single organizational unit Task 3: Advise on the interface between the Network Division and the Transmission/Market Operator Task 4: Advise on the operation of the Network Division with particular emphasis on reducing losses Task 5: Provide assistance and support the New Anti-Corruption Department Task 6: Create a comprehensive metering program Task 7: Advise on the operation of the Supply Division with particular emphasis on increasing collections Task 8: Create a program for Revenue Cycle Management Task 9: Institute an annual external audit of the Network and Supply Divisions Task 10: Prepare a priority investment plan for Network and Supply Task 11: Advise KEK on revising laws and regulations to enable KEK to be able to operate with international best practices Observations:  Compared to the original work plan, one more objective has been added. Otherwise, objectives are substantially the same, except they have been elaborated  Tasks are the same, but not specifically associated in the report with any of the objectives. If they are associated to objectives as in the original work plan, Objective 4 seems to have no associated tasks 2008 Annual Report Objectives: Objective 1: Provide support to the managing director of KEK, with an emphasis on enhancing the company’s revenues. Objective 2: Increase collections through support to the Network and Supply Divisions of KEK to accelerate potential future private sector participation (making recommendations on whether the divisions should be combined into a single organizational unit, advising on the interface between the Network Division and the transmission/market operator, providing support to the new Anti-Corruption Department, creating a comprehensive metering program, advising on the operation of the Supply Division, with an emphasis on increasing collections, creating a program for revenue cycle management, instituting an annual external audit of the divisions, and preparing a priority investment plan for the divisions). Objective 3: Improve the institutional, policy and legal environments in which KEK operates and advise KEK on the revision of laws and regulations to enable it to operate using international best practices. Objective 4: Preparation of KEK Distribution Functions for Privatization Objective 5: Support to KEK Commercialization Objective 6: Anti-Corruption Efforts – The Reduction and Prosecution of Fraud, Waste, and Abuse Objective 7: Legal and Regulatory Support Objective 8: Normalization of Service to Enclave Communities. Tasks: Task 1: Support Management and Operation to Preserve Assets Task 2: Amend Distribution Structure and Organization Task 3: Improve Energy Accounting Task 4: Increase Collections Task 5: Assist KEK to Take Over and Clean Up CCP (Customer Care Package) Task 6: Assist KEK to Takeover Computerized Accounting System and Unbundle Accounting Task 7: Support Tariff Applications and Improve Regulatory Compliance Task 8: Improve Internal Controls and Strengthen Internal Audit Task 9: Provide Legal Support for Unbundling and Privatization Task 10: Assist Privatization Transaction Advisor Task 11: Support Normalization of Service to Enclaves Observations:  The three original objectives remain; the fourth added in 2007 has disappeared; five new objectives have been added.  Eleven completely new tasks have replaced those in 2007. 2009 Annual Report Objectives and tasks are the same as in 2008 2010 Annual Report Objectives: Objective 1: Support for technical preparation of the Distribution Company for privatization Objective 2: Assistance with post-privatization implementation for the Distribution Company Objective 3: Privatization support for the Thermal Power Plant Kosovo B. Tasks: Subtask 1: Support Management and Operation to Maintain Asset Value Subtask 2: Prepare Technical and Contractual Documentation for Investor Due Diligence Subtask 3: Provide Advisory Support in Privatization Process Subtask 4: Strengthen Skills and Technical Capacity of Counterparts Subtask 5: Support Management Post-Privatization Subtask 6: Prepare a Thermal Power Plant Kosovo B Investment Requirement and Rehabilitation Feasibility Study Subtask 7: Prepare Technical and Contractual Documentation for Investor Due Diligence [for privatization of Kosovo B] Subtask 8: Strengthen Skills and Technical Capacity of Counterparts [for privatization of Kosovo B] Observations:  Completely new objectives  Completely new tasks  The tasks were divided into two main “task areas”, one dealing with KEK distribution (comprising Subtasks 1 to 5) and the second with the privatization of Kosovo B (Subtasks 6 to 8) 2011 Annual Report Objectives and tasks are the same as in 2010 2012 Modification (from Action Memorandum Amendment to Mission Director, February 23, 2012) Objectives: The “overall objectives” of the project are not changed (although these are not specified in the Memorandum) Tasks: 1 Support Management and Operations to Maintain (Preserve) Asset Value 2 Provide Advisory Support in the KEDS Privatization Process Including Limited Assistance Post-Privatization 3 Provide Support for Privatization of Remaining KEK’s Assets Observations:  Eight subtasks from 2010 have been modified/ “consolidated” into three tasks. APPENDIX F: LIST OF TETRA TECH ACCOMPLISHMENTS KEK Network and Supply Project Major Accomplishments and Successes January 2007 to June 2012 Support to KEK Managing Director and the Board of Directors Support to KEK’s Management and Board of Directors to implement internal reforms and improve energy security resulting in: 1. Secured lignite suppl y, by opening of interim supply from Sitnica mine and thereafter the new Sibovc Southwest mine. 2. Rehabilitation and/or purchase new equipment required for opening the new Sibovc Southwest mine with a capital budget of nearly 200 million Euros. The new mine produced 1.5 million tons of lignite in 2010, 3.4 million tons in 2011, and will produce 7 million tons in 2012. 3. Elimination of the cause of the low-pressure rotor failures by upgrading the water treatment system at Power Plant B, and purchase of new LP rotors. 4. Reactivation of unit A5 in 2008. 5. Accelerated replacement of several failed generator step-up transformers in Power Plants A&B. 6. Increased KEK generation from power plants by 30%, from 3,970 GWH in 2006 to 5,136 GWH in 2011. 7. Implementation of hydraulic ash transfer system for Power Plant A to depleted coal pit. 8. Replacement of the old e lectro-static precipitators (ESP) with new ESPs, leading t o a reduction of the plant’s dust and particulate emission levels. Commercial Operations (Supply) 1. In 2006, KE K only billed 69% of the available ener gy (31% was unaccounted for). I n 2011 82% of available energy was billed. 2. In 2006, KEK only collected 74% of the amount it billed. In 2011, 91% was collected. 3. Result, collections increased from €96 million in 2006 to €201 in 2011. T he restructuring of the Netwo rk and Supply functions and focus on District Operations facilitated the improvements. 4. Historically, very few customers paid their electric ity bill each month. During 2006, approximately 93,000 payments were made each month. By 2011, the number of payments per month increased to 230,000 5. In 2008, KEK initiated a new payment option for its customers allowing them to pay their electricity bill at any commercial bank in Kosovo. I n 2009 all Western Union offices were added to this pa yment mechanism. By early 2012, approximately 18,000 payments per month are received from this (KOS Giro) mechanism 6. In 2010, KEK initiated a new payment option for it s customers allowing them to pay their electricity bill automatically each month using the ne w Direct Debit mechanism made available by the Central Bank of Kosovo. More than 1,400 customers are currently enrolled in the program. 7. Due to the additional pa yment options made available to customers in the past 5 years, more payments are being processed through the banking s ystem as opposed to in cash. The percent of collecti ons received in cash has decreased from 76% in 2006 to 54% in the first quarter of 2012. 8. For a period of 10 years following the war, KEK was required to provide electricity to minority consumers in Kosovo without payment. Beginning in mid 2009, with the support of the International Community KEK regularized service to all minority consumers in Kos ovo south of the Iber River. Over 2 1,000 minority consumers are now paying regularly and over €14 million has been collected from them. 9. KEK now treats all consum ers south of the Iber River in a commercial manner – those customers who do not pay are disconnected. This includes facilities of all religious faiths, Internally Displaced Person Collective Centers, municipalities, and water companies. 10. KEK experienced significant problems with Trepca (a 110KV customer) and the debt of the custo mer for facilities in South Kosovo am ounted to more than €4 million as of August 2010. At that time, a debt settlement agreement was reached whereby Trepca agreed to pa y for its current consum ption in full each month plus €40,000 each month for past debt. The customer continues to honor that agreement each month. 11. An excessively high Public Broadcast t ax was required to be added to electricity bills since 2003. KEK worked hard to overco me the political pressure within Kosovo and the pressure fro m international organizations and in 2009 was able to discontinue billing this tax (amounting to €15 million per year). All collections from electricity consumers (other than VAT) now go to KEK. 12. Developed and implemented district regulations and procedures to im prove operations in KEK’s 7 districts to detect and prevent theft, bill customers corr ectly, enforce pay ment discipline and disconnect the customers that don’t pay. Information Systems and Finance and Accounting 1. Substantial improvements in budgeting and cost control including: a. Development and implementation of detailed corporate Business Plans and Division Action Plans in support of the Business Plans; b. Introduction of zero based budgeting by divisions and departments; c. Development and implementation of an Integrated Budgeting, Procurement and Cost Control System 2. The improvements in budgeting and cost control resulted in significant improvement in the company financial position and performance (losses of more than 30 million were turned into profits of more than 15 million, liquidity increased 2.5 times, KEK was able to finance large O&M and capital investment programs and improve efficiency and effectiveness of operations). 3. Development and implementation of new accounting policies and procedures and information systems and internal controls including: a. Development and implementation of a sophisticated Asset Register; b. Development and implementation of new Centralized Document Database and Accounts Payable, Accounts Receivable and Treasury modules of the Computerized Accounting System; c. Development and implementation of new sophisticated HR and Payroll System. 4. The improvements in accounting and finance resulted in obtaining an unqualified auditor’s opinion in 2011 for the first time in KEK’s history. 5. Building a state of the art Data Center which provides all necessary information services. Legal 1. Drafted, negotiated and concluded contracts for major capital investments in KEK mines, power plants and distribution assets with a co mbined total of nearly 300 million Euros; and providing oversight for the successful completion of these projects. 2. Drafted and negotiated several credit facility agreements with the Government of Kosovo, thereby securing critical funds for KEK to finance its capital investment plants in mines and power plants. 3. Drafted and negotiated overdraft facilities with a local bank, thereby ensuring that KEK had working capital during critical periods bet ween 2007 and 2010; given the improvements in its financial performance, KEK no longer utilizes the overdraft facility. 4. Trained and guided t he KEK Board of Directors to ensure that it is cognizant of, and o perates largely in accordance with good corporate governance principles and their fiduciary duties and responsibilities. 5. Undertook all necessary preparatory steps for the priva tization of KEK’ s electric distributi on and suppl y businesses; specifically drafting all incorporation documentation for the new distribution and suppl y company (KEDS) and se curing its business r egistration in 2009; and creati ng a phy sical data roo m in anticipation of investor due diligence. 6. Produced key contractual documents, to include the Tr ansfer Agreement, Shared Services Agreement and Collection Agreement, which are required for the spin-off of KEK’s electric distribution and supply businesses in support of KEDS privatization. 7. Drafted in ex cess of 100 executive orders in order to implement improved internal rules and procedures within the company. 8. Prepared a comprehensive KEK Em ployment Manual, which consolidates all internal policies and procedures that concern employee conditions into a 200 plus page document. 9. Supported KEK to successfully place on concession – on a rehabilitate and operate basis - its remaining four small hydro plants; the process was completed in 2010. 10. Secured key modifications to the 2010 Law on Energy, which afford KEK a statutory ninety-nine (99) year right of use over all energ y facilities that it possesses or uses – thereby giving legal certainty and clarity to KEK’s property rights in preparation for privatization. 11. Secured key modification to the 2010 L aw on Electricity, which explicitly make electricity theft a criminal offense. 12. Developed an entirely new electric supply agreement for KEK’s single largest custo mer, with a more favorable tariff structure for KEK and one that is appropriate to a large (90 MW) customer with a high load factor. 13. Developed new electric supply agreements for KEK’s second and third largest industrial customers. 14. Developed a new power purchase agreement for the Uj mani Hydropower Plant with a new pricing structure focused on having the facility available as much as possible for KEK dispatching. 15. Drafted and advised KEK on over 300 contracts and other legal docu ments in order to intr oduce international best practices and commercial standards. 16. Drafted a legal co mplaint on behalf of KEK that was sub mitted to the Energy Community Treaty Secretariat, citing the continued illegal actions of the Serbian State Electric Utility (EPS) and its proxy Elektrokosmet in northern Kosovo. 17. Drafted and negotiated contracts with a value of over 25 million Euros for the insta llation of new electrostatic precipitators in three units of Kosova A TPP; this represents KEK’s largest investment aimed at reducing the environmental impact of its power plants - specifica lly the reduction of ash and particulate emissions. 18. In preparation of the declaration of independence in 2008, worked with the Kosovo Trust Agency and the UN Mission in Kosovo’s (UNMIK) legal office in connection with drafting a determination for UNMIK to designate those parcels of land required for the construction of a new 7 km transmission line in North Western Kosovo as being in the “public interest”. 19. Introduced new template contracts for KEK energy import/exports based on European Federation of Energy Traders (EFET) standards in conjunction with new suite of import options, to include emergency, day ahead and base load contracts. Network Division 1. Supported KEK with the technical specifications, tendering and procurement of approximately 133,300 energy accounting meters and customer meters in amount of approximately € 9.2 million to reduce commercial losses, improve energy accounting and to assist with regularizing customers. 2. Supported KEK with the tendering for upgrade and concession of small HPPs that will increase the hydro power output at HPP Istog, HPP Radavc and HPP Dikance. 3. Supported KEK with the technical specifications, tendering, procurement and implementation of several projects to enhance reliability, supply quality and mitigate congestion including a. Re-construction of SS 110/35kV Palaj (€ 7 mil) to ensure N-2 reliability criteria of electricity supply to KEK Mines & Generation auxiliary b. Construction of new SS 110/10kV Prishtina 7 (€ 4.9 mil) c. Upgrades of 110kV Substations (Prizreni 1, Prishtina 3, Ferizaj 1 and Peja 1) (€ 2 mil) d. Protection of MV network at all 110/xxkV and 35/xxkV substations (€ 2.7 mil) e. Medium Voltage network reinforcement projects including i. 75 km of new 10(20) kV feeders ii. 181 km of new 0.4 kV lines iii. 24 new 10(20)/0.4 kV TS. 4. Supported KEK in development of several strategy papers, plans and procedures for implementation: a. Development of Metering Strategy b. Distribution Network Maintenance Procedure c. Development of Network Development Plan 2010-2014 for the first time. d. Development of Distribution Connection Charging Methodology e. Development of new Technical Loss Calculation Methodology 5. Supported KEK with engineering and implementation of the KEK HQ & IT Data Center redundant electricity supply project, which ensures N-3 reliability of electricity supply and virtually uninterruptable electricity supply of KEK’s main data servers. 6. Supported KEK in development of the Long Term Capacity Management Study 2011 (Least Cost Plan) 7. Supported KEK with a practical assessment of Renewable Energy Sources and Demand Side Management potentials of Kosovo. Internal Audit and Field Enforcement 1. Established internal audit and field enforcement functions at KEK to investigate alleged irregular conduct by employees, to monitor compliance of company regulations and to inspect customers to detect unauthorized electricity consumption. 2. Established an Internal Audit Committee within the Board of Directors. 3. Managed the work of KEK’s internal audit and field enforcement functions function resulting in: a. The conduct of 350 audits in all aspects of KEK’s operations. b. The implementation of 350 changes in existing business and administrative processes and the execution of newly developed processes. c. The cancellation or re-evaluation of 23 tenders resulting more than €3 million in cost savings. d. Different types of disciplinary measures against 768 employees. e. The submission of 2034 cases of theft, unauthorized consumption, corruption and other criminal offences to law enforcement agencies for further investigations. f. Disconnections of more than 3,211 problematic customers resulting in the collection of €3.6 million euros of additional revenue. g. €27 million in additional revenue and cost savings by March 2012. Human Resources 1. Coordinated the signing of an agreement between Mercy Corps and KEK for internship program at KEK. 2. Designed a redundancy process and coordinated th e resolution of unsystem ized employees in KEK. 3. Optimized existing organizational structures for Generation, Coal Production, Network, Suppl y and Customer Services, Corporate Services, F inance and Energy Balance and Trading divisions and prepared new structures for post privatization. 4. Developed the “KEK Payroll Ma nual,” which details the new pay system for the com pany employees at the managerial level. 5. Prepared a job description tem plate and initiated a process to update all ex isting job descriptions for all divisions at KEK. 6. Prepared a document which maps progression level for Generation and Mine employees of various disciplines. 7. Provided supervision of and assi stance to KEK on drafting job a nnouncements, job descriptions, interviewing, recruitment, implementing disciplinary measures, and preparing of payroll and salary deductions. 8. Advised KEK on the developm ent and im plementation of an early retirem ent scheme for employees aged 55 and over and disabled employees under the age of 55. 9. Designed a for m and a process for KEK em ployees to declare their electricity customer information in order for KEK to process em ployees’ electricity consumption through payroll and ensure that its employees consistently pay for their electricity use. 10. Assisted KEK with its applications to the National Qualification Authority for accreditation of certain vocational courses run by the KEK Training Center. Security and Asset Protection 1. Managed the mapping and installation of an extensive surveillance camera network throughout KEK in order to reduce absenteeism, to reduce theft and sabotage and to monitor the movement of assets. 2. Managed the implementation of security booms at all entrances at KEK premises in order to allow for better inspections and monitoring of the movement of KEK assets. 3. Managed the deployment of turnstiles at KEK’s headquarters in order to improve employee attendance therefore reducing absenteeism. 4. Developed a new structured database and associated forms to manage vehicle deployment, to map all vehicle related expenditure, to standardize request. 5. Oversaw an auction of some 136 old vehicles owned by KEK, which netted €60,000 – the first time in its history. 6. Designed a rotation system for security guards assigned to KEK’s facilities to minimize familiarization between guards and KEK employees in order to addressing the rising theft attempts at KEK. Developed an Emergency Evacuation Plan document for implementation throughout all KEK work sites/premises. APPENDIX G: ERO NOTICE Prishtinë, 10 February 2012 ERO Code: V_396_2012 Board of the Energy Regulatory Office, - Taking into account: - Energy Regulatory Office (ERO) competences entrusted to it under Article 14 paragraph 2 and Article 25 of the Law on Energy Regulator (No. 03/L-185); - Provisions of Article 49 paragraph 2 on the Rule of Licensing of Energy Activities in Kosovo; - The request of the Kosovo Energ y Corporation (KEK j.s.c.) to derogate some Articles of the License on Generation TC Kosova A, TC Kosova B, Distribution, Public Supply and Supply/Trade with electricity dated 16.01.2012, In its meeting held on 10 February 2012 adopted this: DECISION I. TO ALLOW KEK j.s.c to derogate (according to attached Annex) the application of some articles on:  License on Generation of Electricity TC Kosova A  License on Generation of Electricity TC Kosova B  License on Distribution System Operator  License on Public Supply with Electricity, and  License on Supply / Trade with Electricity. Justification ‐ On 16.01.2012, ERO received (in electronic format) the request of the enterprise KEK j.s.c which asked for a timeline (derogation) of some articles of the License on Generation TC Kosova A, TC Kosova B, Distribution, Public Supply and Supply / Trade with electricity to be postponed. ‐ Kosovo Energy Corporation, being unable to meet the conditions of the Licenses, has asked for a deadline of the application of some Articles of License to be postponed, as to the Annex to this decision. ‐ ERO has assessed all documents submitted by the enterprise KEK j.s.c. and following the presentation of the required evidences, pursuant to legal provisions mentioned at the introduction of this decision, has decided as in the disposition to this decision. II. All articles and other provisions of the Licenses of KEK j.s.c. remain into force. III. This Decision is issued in Albanian and will be translated into Serbian and English. In the event of any discrepancies between translations, the Albanian version shall prevail. IV. This Decision becomes effective on the date of its approval by the ERO Board and shall be published in the official web site of ERO. Legal remedy: Against this Decision, the unsatisfied party may file an administrative conflict before the competent Court within t hirty (30) days following its receipt or the date of its publication in the web site of ERO, whichever occurs last. * Attached to the decision you may find the annex with the allowed derogations (postponement) dated 10.02.2012 for KEK j.s.c. ERO Board _______________________ Dr. Ali Hamiti, chairman _______________________ Përparim Kabashi, member _______________________ Blerim Koci, member _______________________ Enver Halimi, member TC KOSOVA A GENERATION LICENSE No. Article Derogation dated 15.08.08 until: Derogation dated 17.02.09 until: Derogation dated 09.02.10 until: Derogation dated 03.04.10 until: Derogation dated 10.09.10 until: Derogation dated 28.01.11 until: Derogation dated 30.08.11 until: Derogation dated 10.02.12 until: Remarks from ERO 1 Article 2 – separated accounts on Generation businesses 31 December 2008 31 March 2009 31 March 2010 30 June 2010 31 December 2010 30 June 2011 31 December 2011 31 March 2012 - 2 Article 14 – Environment 31 December 2009 - 30 June 2010 - 31 December 2010 31 December 2011 - 31 December 2012 - 3 Article 15 – insurance obligation 31 December 2009 - 31 December 2010 - - - 31 December 2012 31 December 2012 - TC KOSOVA B GENERATION LICENSE No. Article Derogation dated 15.08.08 until: Derogation dated 17.02.09 until: Derogation dated 09.02.10 until: Derogation dated 03.04.10 until: Derogation dated 10.09.10 until: Derogation dated 28.01.11 until: Derogation dated 30.08.11 until: Derogation dated 10.02.12 until: Remarks from ERO 1 Article 2 – separated accounts on Generation businesses 31 December 2008 31 March 2009 31 March 2010 30 June 2010 31 December 2010 30 June 2011 31 December 2011 31 March 2012 2 Article 14 – Environment 31 December 2009 - 30 June 2010 - 31 December 2010 31 December 2011 - 31 December 2012 3 Article 15 – insurance 31 - 31 - - - 31 31 obligation December 2009 December 2010 December 2011 December 2012 Distribution System Operator’s License No. Article Derogation dated 15.08.08 until: Derogation dated 17.02.09 until: Derogation dated 09.02.10 until: Derogation dated 03.04.10 until: Derogation dated 10.09.10 until: Derogation dated 28.01.11 until: Derogation dated 30.08.11 until: Derogation dated 10.02.12 until: Remarks from ERO 1 Article 2 – Separated accounts for Distribution Operator Businesses 31 December 2008 31 March 2009 31 March 2010 30 June 2010 31 December 2010 30 June 2011 31 December 2011 31 March 2012 2 Article 11 – Safety Standards on Distribution System and Planning 31 December 2008 30 June 2009 30 June 2010 - 01 November 2010 31 March 2011 31 December 2011 31 June 2012 - 3 Article 12 – Safety Operation Standards 31 December 2008 30 June 2009 30 June 2010 - 01 November 2010 31 March 2011 31 December 2011 31 June 2012 - 5 Article 22 – Insurance obligation 31 December 2009 - 31 December 2010 - - - 31 December 2011 31 December 2012 - APPENDIX H: EMSG SCOPE OF WORK Annex: United States Agency for International Development Economic Management for Stability and Growth, Focus on Energy: Objective 4: Assist Kosovo’s Energy Sector to Become More Efficient, Transparent, and Modern By BEARINGPOINT Deloitte , Matthew Smith, Chief of Party Contract Number GEG-I-00-04-00004-00, Based on Annual report 2008-2010: the following sections paraphrase the statements from the reports as reported by the contractors. Brief Summary Statement of each Annual Report 2008 On 17 February, 2008, Kosovo issued a declaration of independence. Following this, EMSG supported Kosovo election of the PDK majority (LDK) coalition government, formed in January 2008 and KEK Privatization. 2009 Second year, Government decided to privatize KEK Distribution an d Supply and appoint a Transaction Advisor. GoK instructed the New Kosovo Transaction A dvisor to finally accelerate. Regard to mining, a new Mining Law was presented to the Assembly along with focus on pre-feasibility of a green-fields lignite mine. 2010 Third year assumed a key role in supporting broader national energy initi atives and has been effectively engaged with the multiple stakeholders involved in both the KED S and LPTAP transactions. The new mining law was adopted and the Government formed an inter-ministerial team to consider revitalizing the Trepca assets. KEK's distribution functions (Network and Supply) are privatized. 2008  Government proceeded with the privatization.  EMSG assisted on MOU (Sept 2008) between t he US and Kosovo govern ments which include a commitment to proceed with the privatization.  Supported MEM and KEK to unbundle KEK distribution and supply to prepare for privatization.  Activities included buil ding capacity within the gover nment, within KEK, facilitating MEM, OPM, the privatization committee, and the transaction advisors. 2009  Advisors worked with KEK, and the POE Unit to assist in the financial and legal unbundling of DistCo.  Supported the Privatization Committee (PC) plan a nd prepare for the unbundling and privatization of the Distribution and Supply Functions of KEK (DistCo)  Advisors continued to support in the establishment of the PC, hiring of a Transaction Advisor (TA)), prepare draft tenders for TA procurement, to implement the privatization tender.  Supported the tendering process which was complicated with procurement irregularities, re-tendering and protesting of the awarded.  With the MFE, supported the staff of the Project Implementation Unit (PIU) for the DistCo privatization.  Compiled comprehensive portfolio of documents, analyses, decisions, for Kosovo Assembly endorsement of the DistCo privatization.  Advisors supported and participated in the USAID/USEA Privatization Workshop.  Advised on energy legislation, administrative instructions, and secondary legislation,  Understood capacity of MEM relating to EC Direc tives on energy and energy efficiency, meeting obligations under the Energy Community,  Coordinated US and EU advisors to the ERO, KO STT and MEM, providi ng input to the MTEF and 2009 Budget Book, Energy Paper for the IMF, participating in the USG energy briefing to the ICO,  Participated in the USAID energy briefings to the US Department of State, the US Departm ent of the Treasury, and the IMF. 2010  Continued to support the KEDS transaction advisor, throughout the development of the IFC Key Issues Report,  Supported IFC due diligence review process for the KEDS transaction:  Coordinated the development of requi red amendments to the three draft ener gy laws a mong the various USAID consultants.  Energy Team provided informal mentoring and coaching, as well as more formal skills training and professional development to various members of the PIU on an ongoi ng basis. Some examples of formal training include legal, management, market, financial, technology and communication areas.  Developed a 12-month strategic co mmunications plan to support the privatization of KEDS pre-, during, and post-privatization. LPTAP has proceeded transparently and in a timely manner so that foreign investors have confidence in the process. 2008  Supported LPTAP by reviewing and analyzing World Bank reports  Advisors work with MEM and staff to understand and assimilate the l egal, regulatory, business, environmental and social provisions for the Kosovo C project and provided day-to-day policy advice and implementation assistance to bridge gaps both in MEM and in the Project Office of LPTAP  In close cooperation with the Legal Office of MEM, the advisors led efforts to revise primary and secondary legislation to ensure that LPTAP meets appropriate regulatory, environmental and social goals, but also with EC Directives and ECT guidelines.  Advisors assisted MEM t o analyze Kosovo C developm ent, and provi ded technical, leg al, financial and regulatory support to assess Kosovo electricity load growth through 2030.  Advisors developed within MEM an understanding for the divergent roles of policy-maker and  Supported MEM, the KEK advisors, and the Govern ment of Kosovo including the Office of the Prime Minister, to help ensure that short-ter m and tr ansitional lignite prod uction requirements and that development of the new mines is consistent with the future requirements of Kosovo C.  Advisors assisted MEM to understand a nd implement its obligations under the Athens Energy Treaty and any other relevant international commitments such as the Energy Charter Treaty. 2009  Advisors continued to support MEM, t he LPTAP Project Office (PO), the P roject Steering Comm ittee (PSC) and USAID/USG on the New Kosovo transaction.  Assistance included comprehensive review and analysis of the various iterations of the Working Framework Document (WFD) and TA/LA presentations, development of a new comprehensive WFD that met all of the Kosovo Government and USG requirem ents for interim power, removing KEK from the Kosovo Consolidated Budget, de veloping the lignite m ines, building new generation, m eeting EC Directives including on competition, market opening and envir onment, preparing briefing papers for USAID and th e USG, analyzing the impacts on tariffs of increasing the lignite royalty/fee.  New Kosovo project has experienced significant delays, and has not progressed sufficiently since the original EOI and investor shortlisting. As of June 2009, key decisions required for t he preparation of t he RFP have not been taken. ERO is a strong and independent regulator, efficiently and accountably implementing its rules and monitoring responsibilities, giving confidence to international investors in the energy sector. 2008  With assistance of the USAID resident regulatory advisor, the ERO has successfully completed its review of KEK retail tariffs and KOSTT transmission tariffs.  Developed and adopt tariffs for renewable energy (for hydro, and for wind power). 2009  Along with the USAID resident regulator y advisor supported the ERO and reviewed KEK retail tariffs and KOSTT transmission tariffs, using transparent processes.  Advised the MEM on potential changes to the regulatory and energy laws of Kosovo; In support of the Trepca Revitalization to the Ministry of Energy and Mines (MEM) 2009  Regarding Trepca revitalization, the GOK not ready to resolve issues.  Advised key new play ers in the intricacie s of the current situation and options for resolution in a n internationally legitimate manner.  Replacement laws have been developed for m ining, reorganization and the special chamber that will enable a debtor led restructuring program.  Key progress is adoption of the mining law to international best  Developed a reorganization plan for Trepca and mining strategy that would have a positive impact on GDP impact once adopted. Work with the ICMM to promote private sector investment  Advisers prepared briefing material s, developed the presentation with counterparts and contributed t o the Ministers introductory speech. LPTAP has proceeded transparently and in a timely manner so that foreign investors have confidence in the process. 2010  Energy Team assu med a key role in broader national energy initiatives and engaged wit h the multiple stakeholders, donors and officials involved in both the KEDS and LPTAP transactions  Took a leading role in the Market Model Working Group,  Review all draft project agreements for the New Kosovo project as well as the draft RFP package  EMSG worked with KOS TT to convince LPTAP st aff and the New Kosovo transaction advisor that the planned size and config uration of New Kosovo sh ould be 2 x300 MW of CFB boilers rather than o ne 600 MW supercritical boiler.  Energy Team was instrumental in recruiting both PWC and LPTAP along with IFC into the Market Model Working Group (initially comprised of senior policy officials and their ECLO-funded advisors from MEM, ERO and KOSTT) Support for Ministry of Energy and Mines (MEM) Support of the Trepca Revitalization  EMSG supported the ministry’s strategic and policy team on the Trepca reorganization and ad vised (March 2010) an inter-ministerial team develop revitalizing the Trepca assets. EMSG discouraged t urning Trepca into a Publicly -Owned Enterprise which would sa ddle the KCB with huge potential liabilities (circa 3 50 million Euros). Work with the Industry to promote private sector investment  EMSG experts worked closely with i ndustry professionals to assist them in understanding the legal framework and pol icies as well as en courage foreign investors to visit and i nvestigate opportunities in Kosovo. Development of the concept for a feasibility study on mining the Dukajini lignite field  Developed a pre-feasibility study on the Dukagjini coal field APPENDIX I: USAID ENVIRONMENTAL PROCEDURES FOR KOSOVO Annex of US AID (Internal and external) Environmental Procedures for KOSOVO Energy sector 2006-2012 Year Document Summary Originated Approved (highest) 2006 ENVIRONMENTAL COMPLIANCE FACESHEET & REQUEST FOR CATEGORICAL EXCLUSION (2006- KOS-005) Request and approval of a categorical exclusion from further environmental review, since activities are limited to training and education , NB this IEE does not cover activities related to power generation and mining Sharon Hester, Director of EGO Heather Goldman Mission Director 2008 AMENDMENT NO. 2 TO INITIAL ENVIRONMENTAL EXAMINATION (2008- KOS-006) (to original IEE 2004-KOS 008, 2006-KOS-005) Request of conditional negative IEE, categorical exclusion from further environment review. In line with amendment to the SOW of KEK and KEDS support. The condition is details as any procurement, transportation and use of any equipment will be done in a. NB this IEE does not cover activities related to power generation and mining environmentally sound manner Arben Nagavci, Program Specialist/En gineer, EGO Michael Farbman 2009 AMENDMENT NO. 3 TO INITIAL ENVIRONMENTAL EXAMINATION 2009- KOS-041 (to original IEE 2004-KOS 008, 2006-KOS-005, 2008- KOS-006) Request of IEE, positive determination with requirement due to potential for significant adverse effect of activities covered Under Task Area 2 sub-part (i). Appropriate environmental review required Statement of Work for the “Scoping Statement for Environmental Assessment for rehabilitation and potential for expansion of Thermal Power Plan (TPP) Kosovo B”. Arben Nagavci, Economic Growth Officer Patricia Rader Mission Director 2009 ENVIRONMENTAL COMPLIANCE FACESHEET, (DCN: 2009-KOS-013) Request of conditional negative IEE, categorical exclusion from further environment review. Regarding program: Advisory Services to Assist in Privatization of the Electricity Distribution Company in Kosovo. The condition is details as any procurement, transportation and use of any equipment will be done in a. NB this IEE does not cover activities related to power generation and mining environmentally sound manner Arben Nagavci Program Specialist/En gineer, EGO Patricia Rader Mission Director 2010 AMENDMENT NO. 1 TO INITIAL ENVIRONMENTAL EXAMINATION DCN: 2010-KOS-038 (to original IEE: 2009- KOS-013) Request of conditional negative IEE, categorical exclusion from further environment review. Advisory Services to Assist in Privatization of the Electricity Distribution Company in Kosovo. Recommends education, training and workshops Arben Nagavci, Program Management Specialist, EGO Patricia Rader Mission Director 2010 Positive Determination Approval Sheet DCN 2010-KOS-027 (to original DCN 2009- KOS-041) Positive Determination Approval Sheet for the Scoping Statement for Environmental Assessment for rehabilitation and potential for expansion of Thermal Power Plan (TPP) Kosovo B” Document details what environmental impacts should be included into an environmental impact analysis. Michael Boyd Senior Energy Advisor Patricia Rader Mission Director 2012 Environmental Review and Monitoring Support for Rehabilitation and REQUEST FOR CATEGORICAL EXCLUSION, amendment for change of time, scope and cost extension. To extend the ambient air monitoring Economic Growth Office Maureen A Shauket Mission Potential for Expansion of Thermal Power Plant (TPP) Kosovo B, DCN￾2012-KOS-013, program initiated in 2011 to build capacity within GoK/Kosovo Environmental Protection Agency(KEPA)/Hydro Meteorological Institute (HMI)) to manage air quality relative to lignite-fired thermal power plant (TPP) units within Kosovo in order to provide a critical input for a bankable environmental assessment. And also recommend an EA Director 2012 INITIAL ENVIRONMENTAL EXAMINATION AMENDMENT NO. 2 DCN-2012-KOS-035 (to original DCN of Original IEE: 2009- KOS-013 DCN of Amendment 1: 2010-KOS-038) NB document date 2011 but approved 2012. Document request for an environmental exclusions as it outlines each potential environmental impacts of the projects under the project for KEK and KEDs support. This doc includes MoTT (contracted for env analysis of KEDs privatization prep) and IFC presentation recommending how the environmental issues should be divided between KEK and new potential investor. Economic Growth Office Maureen A Shauket Mission Director 2012 INITIAL ENVIRONMENTAL EXAMINATION Amendment No. 4 KEK Network and Supply Project DCN: 2012-KOS-037 (to originals DCN of Original IEE: 2004- KOS-008 DCN of Amendment 1: 2006-KOS-005 DCN of Amendment 2: 2008-KOS-006 DCN of Amendment 3: 2009-KOS-041 DCN of Scoping Statement: 2010-KOS￾027 Task Order Revision, including a Time and Cost Extension. This document has retroactive language as it reviews and amends and extends the tasks, this time making it clear the environmental risk and impact will be used and also be mandatory.. Conditions from the examination reflect proactive environmental safeguard approach. Economic Growth Office 2012 Annex 1 Statement of Differences to the Programmatic Impact Evaluation in the Energy Sector in Kosovo prepared by Mendez England and Associates, made by Deloitte Consulting LLP, subcontractor to Advanced Engineering Associates International, Inc. implementing Advisory Services to Assist in the Privatization of the Electricity Distribution Company in Kosovo Project, under contract EPP-I-00-03-00004-00, TO 9 Programmatic Impact Evaluation in the Energy Sector in Kosovo – Mendez England Associates Deloitte Comments (under Advisory Services to Assist in the Privatization of the Electricity Distribution Company in Kosovo Project implemented by Advanced Engineering Associates International, Inc. (AEAI) Contract No. EPP-I-00-03-00004-00, TO 9) Report Location Specific Issue: Deloitte Comment 1. Executive Summary, Findings – Page 7; ‘The project’s successes need to be judged against the expansion of its scope of work and increased budget from $4.7 million in 2007 to over $28 million by 2012, as well as the fact that many of the objectives of the project as stated in the original Statement of Work (SOW) were not met, making the overall impact rather modest when viewed from a “value-for-money” perspective.’ i). Significant delays (beyond the control of USAID and contractors) are a primary factor in the increase in cost (e.g., had the privatization occurred within the planning timeline, such costs would have been avoided). While this is fact acknowledged later in the report, the significance of project delay is a primary factor in increased contractor budgets and should be acknowledged in the Executive Summary. ii). The value-for-money argument should consider that KEK was able to significantly increase cash flow from increased collections (from €96 million to €201 million) in 5 years, which his substantially reduced GOK required support to the sector. While this fact is acknowledged later in the report, the significance of the increase in cash collection on reducing GOK subsidies (and enhancing GOK fiscal stability) should not be minimized when considering the value-for-money perspective. 2. Executive Summary, Findings – Page 7 ‘Finally, the changing objectives and expanding scope of work over time reveal a lack of focus by USAID, which quite likely contributed to not fully meeting the original objectives.’ i). The complex nature of the electricity business and the issues faced by the Kosovo electricity sector make the initial identification of objectives difficult; objectives must be able to be adapted to reflect requirements while recognizing that oversight and control of the process by USAID is necessary. ii). Original project objectives may be eclipsed by new requirements. That USAID is able to adjust programs to reflect changing circumstances is vital for ultimate project success. 3. Executive Summary, Findings – Page 7 Project 2 (EMSG) project – list of accomplishments i). In the list of accomplishments of the EMSG project, the list did not include the updating of Kosovo’s coaly royalty program to reflect world-wide industry standards, the net result being an increase in revenues to the GOK of €16 million/year at current coal consumption levels. While the increase in coal royalties does impact final tariffs, the increase in coal royalties is in line with industry norms and provides a mechanism for the GOK to receive revenues from its reserves. Receiving revenues via coal royalties provides a more secure revenue stream than does, for instance, securing an equity position in generation plants that utilize coal. 4. Executive Summary, Findings – Page 8 Project 2 (EMSG) project – possible negative impact: ‘GoK may have become dependent on the advisors in the absence of formal capacity building’ ‘contractors took an active and, in some cases, a leading role. This may have created an unintended impact of having GOK decision-makers much too reliant on the consulting experts. This unintended impact may have delayed actions …’ i). Consultants within the EMSG project worked closely with beneficiary counterparts on all issues, providing informal capacity building on an on-going basis. Such informal ‘learning by doing’ capacity building provides beneficiaries the opportunity to build capacity within the context of their daily jobs, while also enhancing decision-making. Deloitte has not witnessed that ‘GOK decision-makers’ are too reliant on consulting experts. In fact, it is the consulting experts who are often assisting the GOK in carrying out the decisions of the appropriate GOK authorities during the course of the KEDS transaction. 5. Executive Summary, Findings – Page 8 Project 3 – Privatization of the Electricity Distribution Company (KEDS) i). The report does not mention as an accomplishment the significant transaction management activities that were undertaken throughout the project which have culminated in the selection of a preferred bidder via a transparent, competitive bid process for the KEDS business. ii). Significant informal (e.g., on the job training) capacity building across institutions during numerous meetings with key stakeholders provided the framework for support within Government institutions (and other bodies) for the privatization process. 6. Executive Summary, General Analysis – Page 10 One particularly stark example is a decision to support the non-unbundling of KEK - apparently because of the potential for political interference by KEK’s Board of Directors - during a time that ERO was constantly ordering KEK to unbundle. This action has not only undermined the notion of promoting a strong ERO, but it is also not in line with the concept of unbundling KEK for the purpose of the KEDS transaction. i). While there is a duty to unbundle under law (such duty came into effect in Nov, 2010), the ERO did not require KEK to unbundle (i.e., it did not enforce the law). ii). The ERO accepted the argument that the investor in KEDS is best positioned to unbundle the company after privatization. The delay in the privatization was a significant reason why the unbundling has been delayed. iii). The investor is obligated to unbundle within 1 year of taking control of KEDS; the ERO has agreed with this position. 7. Executive Summary, General Analysis – Page 11 ‘Lesson 3: On balance, open-ended SOWs have had a negative impact.’ i). SOWs must reflect the dynamic nature of the issues as they evolve in the energy sector in Kosovo. While it is recognized that contract oversight is a fundamental requirement, sufficient flexibility must be provided within contracts in order that USAID may efficiently assist the beneficiary. 8. Executive Summary, Recommendations for Future Program Design – Page 11 ‘Consensus building, particularly with civil society, should become a formally recognized USAID obligation. While this may consume significant resources, there will undoubtedly be a positive payback.’ i). It should be noted that the KEDS advisors (supported by USAID) alongside their GOK beneficiaries have worked in a coordinated manner to reach out to NGOs and civil society to ensure transparency and information access throughout the KEDS transaction. KEDS has an active public information room that has been made available to NGOs but has never been visited by such NGOs/civil society; the GOK ran a formal public forum at the Hotel Pristina and NGOs/civil society were invited – only 1 NGO attended; the KEDS advisors (in association with the GOK’s PIU) offered to meet NGOs/civil society on multiple occasions (and met labor unions on 5 separate occasions), but the NGOs declined to attend meetings that were set up on their behalf. 9. C.5, Page 21 ‘the question then arises of whether the assistance might have been better focused on say, obtaining the necessary capital funding for the distribution and supply system to fix the fundamental problem of leakages’ 1). The reduction of technical losses (referred to as ‘leakages’ in the report) requires substantial investment -- significantly beyond $24million. It is likely that such limited investment would not have had a significant impact on the level of technical losses. ii). Deloitte believes that the presence of a technically and managerially competent advisor to KEK has been fundamentally important in obtaining investor interest in KEDS. iii). Ongoing support by TT in the provision of data, in the drafting of contracts required for the privatization, the provision of an opening Balance Sheet, and other requests, are instrumental in achieving a successful KEDS transaction within the timeline envisioned for completion. 10. C.6, Page23 ‘The changing objectives and the expanding scope of work over time reveal a lack of focus by USAID, which quite likely contributed to not fully achieving what was originally intended.’ i). See previous comments. ii).The changing objectives and expanding scope indicate the ability to adapt to dynamic conditions rather than a ‘lack of focus.’ The need to increase cash flow at KEK (via increases in collections) was fundamentally important to reduce the reliance by KEK on GOK subsidies. Reduction of subsidies increases fiscal stability for Kosovo (as indicated by the IMF) and also provided comfort to investors that cash flow issues at KEK could be solved. iii). Kosovo’s reliance on aging power generation facilities coupled with the requirement to open a new coal mine (as the previous mine was exhausted) are/were significant risk factors for economic stability in Kosovo. Given the cost of imported power (which would have to be subsidized by the GOK given issues of tariff affordability), operational failure at the facilities upon which Kosovo relies would have caused significant fiscal deterioration at the federal level. USAID’s decision to provide advisors to KEK should be seen as an insurance policy against catastrophic failure. iv). The presence of international advisors at KEK that are familiar with the requirements of international investors has significantly reduced privatization transaction risk. 11 Project 2 – EMSG D.3 – Analysis of Impacts ‘The EMSG energy advisors/contractors through long term and daily engagement may have gained trust and credibility with the GoK and other stakeholders involved in the energy sector - an unintended impact.’ i). Gaining the trust of the beneficiaries via credible advisory and capacity building presence allows the GOK and other stakeholders confidence to make decisions and to have confidence in the outcome of the decisions that they make. While this may be viewed as an ‘unintended impact,’ the Negative impacts were: 1. While day to day comprehensive interaction was effective in advancing energy sector reform, an unintended impact is that the GoK may have become dependent on the advisors in the absence of formal capacity building. reason USAID advisors are seen as effective by the counterparty is entirely due to their credibility and the confidence that the beneficiaries have in the advice they are receiving. Further, in most instances, institutional capacity is best learned ‘on the job’ working through issues rather than formalized ‘training’ that may have little relevance. ii). The EMSG project involved beneficiaries at every point, including meetings, presentations, emails, etc. This involvement provided significant opportunities for beneficiary institutions to build capacity and make decisions as evidenced by the strong decision making capacity of the MOF (and subsequently the MED). 12 Project 3 – KEDS E.2 Performance Indicators and Accomplishments Principal project accomplishments in Year 1 were: Reached a broad consensus on the market model to be used in Kosovo, on the basis of which the generation Transaction Advisers could proceed. Secured a positive decision from Brussels on the specific package of legislative proposals for shaping the laws and market model for Kosovo - and thereby bringing to an end a long and unproductive debate among consultants. Worked with legal counsel at the ECSEE Secretariat in Vienna to accept draft changes in the law in line with the second legislative package and reflected these in the market model in time for their inclusion in the EC monitoring report. It should be noted that none of the above activities was specifically itemized in the list of agreed performance indicators. i). The development of a market design for the functioning of the Kosovo electricity market post privatization was fundamentally important both for a). ensuring Kosovo’s compliance with the EU Energy Treaty (to which Kosovo is a signatory) and b). the operation of the electricity market post unbundling of KEK. Ensuring compliance of Kosovo law with the requirements of the second legislative package a). more fully integrates Kosovo into the regional energy market and b). provides the legal basis for the functioning electricity market post market opening. Each of these items is critical for a functioning, liberalized electricity market. That advisors under the KEDS project were involved in the market design was fundamentally important to ensuring the market design was compatible with GOK’s legal obligations and energy strategy, including privatization. While the report finds that ‘none of the above actions was specifically itemized in the list of agreed performance indicators’, that these critical actions were undertaken by USAID advisors (with the support of USAID) indicates the ability of USAID to adapt to the dynamic issues facing Kosovo in the energy sector over the past three years. USAID advisors support in these issues is likely seen as important by the beneficiaries. 13 Project 3 – KEDS E.2 Performance Indicators and Accomplishments Principal project accomplishments in Year 2 were: Delivered ‘Losses Study Report’ - an independent study which confirmed the Tetra Tech methodology to be sound. i). It is noted that the USAID commissioned study did not confirm that the ‘Tetra Tech methodology’ is sound. The methodology that was reviewed and that was confirmed by the study is KEK’s loss calculation methodology, not that of Tetra Tech. This comment indicates a bias on the part of the report authors that KEK and Tetra Tech are synonymous. 14 E.4 Negative Impacts - 4. 1. Vague SOW. While it was clear that the overall objective was to provide support for a successful and sustainable privatization transaction, the SOW for the KEDS project was intentionally left broad and flexible - an approach which has both advantages and disadvantages. i). The report concludes that a ‘vague SOW’ was associated with the AEAI inability to work effectively within the ERO. This association is not correct. ii). As stated above, flexible SOWs are critical at a time of dynamic change within the energy sector. That USAID provided flexibility in SOW was critical to ensure that USAID administrative policies did not impact the timelines of critical actions (i.e., conforming Kosovo’s laws to comply with GOK’s legal obligations under the EU Acquis). 15 E.4 Negative Impacts - 4. As mentioned above, Deloitte was subcontracted to AEAI for purely pragmatic rather than logical reasons, i.e. so they could continue to advise the MED without having to go through extensive contractual re-negotiations. This was done as their contract of engagement had expired and there was no quick alternative way to retain Deloitte’s services. However, given their respective roles and responsibilities, the sub￾contracting of Deloitte under AEAI did create some potential conflict of interest issues - i.e., AEAI works primarily for ERO, whose function is to act as an independent body to balance the interests of all stakeholders in the power industry, while Deloitte works for the Government, which is only one of those stakeholders. This potential conflict of interest was partly addressed by creating“ Chinese walls” between the two consultancies. Bringing these two separate and supposedly independent strands of involvement together under one contract was not helpful given the sensitive nature of the privatization issue. i). This comment is not consistent with comments in the report that beneficiaries do not focus on the contractual vehicle under which advisory and capacity building services are provided. ii). While Deloitte’s services are focused at the PIU (the Project Implementation Group that is tasked by the GOK with the day to day running of the privatization) and the MED, Deloitte (with leadership from the GOK and MED) took significant steps to ensure all stakeholders (including the ERO) were consulted on all issues that pertain to the transaction (e.g., the perception from the report that Deloitte worked only for the MED to further MED policies is not correct). As a case in point, an Energy Projects Technical Working Group was created from all Kosovo government institutions that have a role in the energy sector and the PIU (with the assistance of Deloitte) ensured that the TWG was involved in all key issues of the privatization. iii). It is noted that a ‘Chinese wall’ was established between Deloitte and AEAI on matters of a regulatory nature. Deloitte used the formal ‘public consultation periods’ provided by the ERO to comment on regulatory matters rather than influencing them directly with the ERO. 16 E.4 Negative Impacts – 5. 5. Conflict Between EU and US Consultants. ‘ … the agenda for the reform of its energy sector agenda is firmly set and involves implementation of the acquis communitaire as well meeting the other requirements of the Athens Treaty. Hence, other than for educational purposes, there is little point in debating alternative models. However, while EU environmental standards have been included in USAID energy sector program support, market workings or philosophies and legal solutions have sometimes been based on the North American model. These have been pushed quite forcibly by some American advisors – both those hired by USAID and those hired by other agencies - even though they did not conform to the EU approach. EU requirements have been referred to as an “obstacle” – even though the USA i). The report states that ‘market workings or philosophies and legal solutions have sometimes been based on the North American model.’ However, as there are many different market models and philosophies in North America (much as there are many different market designs within member countries of the EU), the comment that an ‘American Model’ was forcibly pushed is not based in fact; there is no such thing as an ‘American model’. ii). The statement that ‘market workings or philosophies and legal solutions have sometimes been based on the North American Model’ (whatever that means) is not consistent with the fact that advisors under the KEDS transaction actively assisted in ensuring Kosovo legislation was in compliance with Kosovo’s obligations under the EU acquis (see ‘Performance Indicators and Accomplishments). officially supports Kosovo’s wish to join the EU at some future time and that all efforts have been made by the USAID Mission to ensure compliance with EU requirements. 17 E.4 Negative Impacts – 6. This lack of continuity - especially in the energy sector - meant that in the absence of an expert, the contractor had to provide direction to USAID, which was itself being represented by an administrator. From consultant’s point of view, this makes life easier but creates the risk of that consultant either hijacking the work program or taking it off in an inappropriate direction. On the other hand, the present situation of actually having an energy sector expert at USAID appears to have resulted in an intensive doubling up, with the USAID sector expert working almost as a contractor would. This, in the absence of a well-defined SOW, means there is a risk of the consultant team getting pulled into areas that are not really within its scope. i). The absence of an expert at USAID does not ‘make life easier;’ rather, the presence of an expert at USAID is useful in that the expert can understand more fully the complex issues that must be addressed in order to provide maximum benefits to beneficiaries consistent with USAID policy. Further (among other benefits) USAID expertise is important for developing a USAID strategy consistent with beneficiary strategies for the energy sector and ensuring ‘buy in’ by senior USAID officials and the Department of State of the defined strategy and its implementation. 18 E.5 – Analysis and Summary However, several negative impacts have been noted: Firstly, the USAID energy program was involved in each area of market transformation, advising the regulator, the government and public utility, thereby potentially compromising its neutral role and undermining the trust of its clients. Kosovo has to follow EU standards based on the Athens’ Treaty. However, some advisors have on occasion pushed for solutions relating to market and legal/ regulatory issues that were biased towards a North American model, sometimes resulting in a degree of unnecessary conflict and confusion. i). That USAID advisors were involved with working with different beneficiaries was important in providing consistency of approach to the transformation of the energy sector consistent with Kosovo’s Energy Strategy as approved by the Kosovo Assembly. In areas of common interest – such as the development of laws consistent with EU legislation, the development of a market design consistent with EU legislation – USAID advisors worked effectively with all stakeholders and their advisors (including those from the EU) and built capacity across all institutional bodies in order that the market transformation could be adopted by each of these government institutions in a consistent manner. In areas where there was the potential for areas of professional difference – the development of tariffs by the ERO that would determine licensee revenues – USAID consultants worked separately (i.e., ‘Chinese Walls were in place) and the normal processes (such as the use of Public Consultations by the ERO) were utilized to express differences. ii). As previously stated, there is no such thing as ‘a North American model.’ It is noted that there were professional differences between advisors (including between USAID advisors) on market design, but this questioning provides a basis for improvement in the final solution. 19 F.7 Analysis & Summary ‘Interviews with GoK, NGOs and other stakeholders revealed that they either had no knowledge that USAID activities had environmental safeguards procedures and or they had perceptions that USAID was implementing energy sector changes behind closed doors with no access to information. This perception is especially poignant against the current backdrop of national and international protests by environmentalists. The result is that they are further bolstered by the lack of knowledge and transparency of the environmental precautions and assessments that USAID has already addressed. i). As previously noted, the KEDS project has taken a proactive approach to provide for the transparent implementation (via the Government authorized Project Implementation Unit) of GOK energy strategy as approved by the Kosovo Assembly. The PIU a). has a full time public relations specialist who actively seeks to involve all external stakeholders (media, NGOs, civil society) to ensure these groups have accurate information; b). maintains an active data room where external stakeholders (NGOs, civil society, civilians) are provided data regarding the KEDS transaction, c).held numerous meetings (or meetings were attempted to be held) with external stakeholders – 5 meetings were held with union leaders, while NGOs and civil society declined to attend meetings with the PIU that had been set up in advance, d). in association with the MED, held a conference at the Pristina Hotel (and members of NGOs/civil society were invited will in advance), but only 1 member of an NGO attended. ii). It is noted that NGOs and civil society made minimal (if any) comments during the EROs Public Consultation periods. 20 General Analysis H.2 – Political and Donor Environment Because of these arrangements, the GoK has always found it rather difficult to make progress on the two major energy privatizations, KEDS and coal generation. On the one hand, MED has said that "any decision taken on KEDS will be harmonized by a unanimous vote and government approved,” while on the other, there has been a significant level of continuing vocal and active opposition to privatization within parliament, making a unanimous vote difficult to achieve. This has resulted in delays to the privatization process, most notably KEDS (as other factors have had their effect on delaying the coal￾i). The Assembly of Kosovo provided ‘in principal’ approval for the KEDS privatization in 2008. Further, a recent vote in the Assembly in Q2, 2012 to stop the process of privatization of KEDS was defeated in the Assembly. Therefore, opposition to the KEDS transaction at the Assembly has not ‘resulted in delays to the privatization process.’ ii). Delays in the process were primarily political – driven by national elections being called during the privatization as well as the restructuring of GOK Ministries responsible for the energy sector. Further, gaining stakeholder consensus in the fired generation project) and is at least partly to blame for the large increase in cost associated with the KEK Network and Supply project. KEDS transaction has taken time to achieve, but this process has provided a wider basis for acceptance of the outcome of privatization. iii). Until recently, the KEDS privatization was expected to be concluded following the generation project. 21 General Analysis H.2 – Political and Donor Environment ‘This strategy evolved over the same period as USAID’s emergence as a strong supporter of privatization and also at the same time as USAID reacted to adverse publicity by restricting the amount of information made available to the public.’ i). The basis for this conclusion is not clear. By example, and as detailed in comment No. 19, USAID contractors took significant steps to ensure the availability of data and information to the public, NGOs and civil society as well as other interested parties (labor unions) with a vested interest in the outcome of USAID’s efforts to support GOK energy strategies and policies. 22 General Analysis H.2 – Political and Donor Environment • The Mission needed to react to circumstantial changes, leading to increased scope and spending. An example of this result on the USAID energy program is the EU/ GoK decision to decommission Kosovo A. Given that the country was already experiencing a shortage of electricity and that KEK/ GoK budget could not sustain paying for increased electricity imports, this agreement influenced USAID to take greater actions to address the urgency of the situation. Consequently, there was an increase in scopes of work and funding, such as the energy component being added to the EMSG project. i). The comment is critical to understand the expansion of USAID’s support for Kosovo’s energy strategy and should be highlighted in the Executive Summary. 23 General Analysis H.3 SWOT Analysis ‘A fundamental weakness of the USAID approach lies in the absence of any clear statement defining the rationale for and the scope of its involvement in the Kosovo energy sector. At a high level, the two main objectives of USAID strategy in Kosovo are: 1) to promote democratic governance; and 2) to support economic growth …’ i). USAID’s approach is consistent with the GOK’s energy strategy as adopted by the Kosovo Assembly which has the ultimate goal of supporting economic growth in Kosovo. 24 General Analysis H.3 SWOT Analysis Measuring results in the area of “providing advice” (AEAI/Deloitte) is not simple, because the advice given is often of a political/strategic nature and its perceived success or otherwise is affected by many exogenous and unpredictable factors. However, it should be possible to i). The USAID advisors provided several project schedules to the PIU and MED and used appropriate project management principles in implementing the KEDS project. Given the complex nature of the KEDS transaction, the many stakeholders involved, and a degree of political opposition to agree to a step-by-step road map of where the client wants to progress and timescales are involved. privatization (especially, for a time, within the ERO), the implementation of the transaction could not be expected to be an efficient process. Further, the process of ensuring cross￾sector stakeholder buy in took significant resources and time. 25 General Analysis H.3 SWOT Analysis Although capacity building and sustainability are frequently mentioned objectives in USAID documentation, in reality these are neglected activities. On-the-job￾training is the most common form of training encountered, but that only addresses the needs of a relatively small number of senior managers who come into contact with USAID contractors. i). This statement is too broad; each project should be reviewed on an individual basis to determine if capacity building – both formal and informal (‘on-the-job training’) -- has been effective. Within the parameters of the KEDS transaction, the opportunity for capacity training is primarily ‘on-the-job’, as it is the issues that arise that drive the opportunity to build capacity via the active involvement of members of the PIU as well as the stakeholders (e.g., the Energy Projects Technical Working Group) involved in the transaction. Further, this informal training has been augmented by formal training – for example, formal training has been provided in assessing differences between different market design options, different ownership structures (privatization vs. PPP concession), tariff methodology development, corporate valuation, etc. Beneficiaries of both formal and informal training have been both ‘senior managers’ as well as their support staff. 26 Section I, Lessons Learned J. Recommendations for Future Program Design i). Issues raised in Section I. Lessons Learned and J. Recommendations for Future Program Design have been addressed elsewhere in this ‘Statement of Differences’ and will, therefore, not be repeated. Annex 2 Statement of Difference to the Programmatic Impact Evaluation in the Energy Sector in Kosovo prepared by Mendez England and Associates, made by Advanced Engineering Associates International, Inc. implementing Environmental Review and Monitoring Support for Rehabilitation and Potential for Expansion of Thermal Power Plant Kosovo B in Kosovo Project under contract EPP-I-00-03- 00004-00, TO 11 Task Order Number and Title Page num ber Text of the Final Report Suggested Comments/Changes Environmental Review and Monitoring Support for Rehabilitation and Potential for Expansion of Thermal Power Plant (TPP) Kosovo B in Kosovo Project, Contract No. EPP-I-00-03- 00004-00, TO 11 Page 26 Secti on F.7. Furthermore, the absence o f awareness or communication of the Mission’s careful adherence to USAID environmental impact mitigation or m inimization efforts were not communicated to key stakeholders as well as civil society . Interviews with GoK, NGOs and other stakeholders revealed that they either had no knowledge that USAID activities had environmental safeguards procedures and or they had perceptions that USAID was implementing energy sector changes behind closed doors with no access to information. This perception is especially poignant against the current backdrop of national and international protests by environmentalists. The result is tha t they are further bols tered by the lack of knowledge and transparency of the environmental precautions and assessments that USAID has already addressed. The 2010 TPP B Scoping Statement preparation involved stakeholder interviews/meetings. Stakeholder consultation during the preparation of the Scoping Statement included meetings with governmental agencies and scoping meetings or workshops with a variety of organizations, and provision of written statements or information by some of the stakeholders. A listing of stakeholders interviewed individually or during group workshops is provided in Appendix B of the Scoping Statement. Meetings with Governmental Agencies: · Ministry of Economy and Finance (MEF) · Ministry of Energy and Mining (MEM) · Ministry of Environment and Spatial Planning (MESP) · Institute of Spatial Planning (ISP) · National Institute of Public Health(NIPH) · Kosovo Environmental Protection Agency(KEPA) · U.S. Agency for International Development (USAID) Workshops with representatives from: · European Commission Liaison Office to Kosovo · INKOS Institute · KEK J.S.C. · KfW · Lignite Power Technical Assistance Project (LPTAP) · LPTAP Transaction Advisors (Price Waterhouse Coopers) and Legal Advisors (Hunton & Williams) · Nongovernmental organizations (NGOs) (Aureola, Kosovo Foundation for Open Society, The Regional Environmental Center) · USAID partners (PA Consulting and Deloitte Consulting) · World Bank The most frequently cited comments made by governmental representatives and workshop participants included: · The need to develop Kosovo’s energy sector in a rational manner to reduce costly imported electricity. · Load shedding during the day. · Undertaking resettlement in a transparent and equitable manner. · Creating job opportunities as a component of the NKPP transaction. · Challenges operating TPP B in the absence of a comprehensive maintenance plan. · The relative merit of constructing different sized generating facilities to meet the energy demands of Kosovo. · Loss of electricity and water through non-technical means. Notes of Meetings with NGOs (copies of business cards provided at end of Appendix B). Regional Environmental Center, Field Office Kosovo/a Date/Time: 11 Feb 2010, 08:30 Name: Dr. Zeqir Veselaj, Director Tel/Fax: (+381 38) 225 123 Email: zveselaj@kos.rec.org Website: http://kos.rec.org Topics Discussed: · Pros and cons of alternative privatization schemes for the mining and power complex · EIA review and monitoring responsibilities of the various stakeholders · Environmental permitting requirements for various facilities of the complex · Possible longer-term roles REC can play in post-EIA follow￾up, including overseeing implementation and effectiveness of the EMP for the mining and power complex Kosova Foundation for Open Society Date, Time: 10 Feb 2010, 14:30 Name, Title: Luan Shllaku, Executive Director Tel/Fax: (+381 38) 542 157 (-160) Email: luans@kfos.org Website: www.kfos.org Topics Discussed: · Pros and cons of alternative privatization schemes for the mining and power complex · Pollution impacts of the mining and power complex relative to villages in the complex and whether or not specific villages are likely to be resettled depending on cost · Air, water and land impacts of mining and power complex and issues relating to complying with EU standards Aureola Date, Time: 10 Feb 2010, 14:30 Name, Title: Sanije Grajqevci, Executive Director Tel/Fax: 044502573 Email: sanijeg@gmail.com, aureola_p95@hotmail.com Topics Discussed: · Need for increased fairness in compensation for land taking by the mine · Need for participation in the economic benefits of the power plants, e.g. well-paying jobs Page 24, Secti on F.4 USAID also purchased analytical software to aid in obtaining additional weather data for the development of an air dispersal model. As part of the capacity building efforts, USAID purchase air dispersion modeling software for HMI and LPTAP staff members to become familiar with tools for estimating concentrations of pollutants in the ambient air. Page 27. F.7 However, a holistic perspective on potential environmental impact was not present, which may have resulted in missed opportunities. For example, the World Bank began to face environmental protest against coal-fired generation early on in the investment preparation phase, which is really the underlying cause to the delay in their contribution to the air monitoring data needs5. At this time, USAID had an opportunity to view environmental issues not only as an impact or a formality but also as an innovative solution. While stop-gap institutional capacity building was proposed and undertaken, the associated portable air monitoring equipment was insufficient for obtaining the quality measurements needed for the World Bank’s ESIA. The pending ESIA requires one full year of air quality data, the collection of which has not yet begun. Although the World Bank led USAID to believe that there were no issues, an in-depth strategic assessment by USAID, starting from 2009, may have resulted in the Mission negotiating this task with the World Bank and building an appropriate air monitoring station back in 2010. This would have not only optimized institutional capacity building, Although the environmental impacts may not have been fully articulated, since 2010 USAID engaged in a number of activities to strengthen the environmental context in Kosovo related to the expansion and rehabilitation of the power plants. These activities included: Preparation of an ambient air management plan outlining a series of steps to manage and enhance ambient air quality in Kosovo. Preparation of capacity building plan to enhance the governance of ambient air monitoring activities related to expansion of the energy sector. Conduct of meetings with staff from the Ministry of the Environment to obtain input on the technical specifications for fixed ambient air quality monitoring equipment Preparation of technical standards for fixed ambient air quality monitoring equipment. Conduct of capacity building activities to transfer information about the management of air emissions and monitoring activities. Gathering recent operating, terrain, and weather data to run air dispersion models to obtain up-to-date information about potential impacts on ambient air quality relative to TPP operations. Meeting with local residents where the mobile monitoring equipment but could have also initiated the required data collection. The evaluation team does recognize, however, that the donor dynamics as well as the change in GoK progress on the energy privatization has had significant impact on the air monitoring topic. is being used to explain the purpose and scope of the program. Preparation of an environmental briefing sheet outlining the overall scope of the ambient air monitoring program. Implementation of an ambient air monitoring program using mobile equipment to generate the information that can be used to inform the decision making process as to the types of steps necessary to align current TPP operating practices and development plans with EU ambient air quality provisions. Scheduling a study tour for HMI staff to transfer additional information about ambient air quality management. Page 9, Secti on A.2 With the above USAID activity in the energy sector of Kosovo, several incidental projects resulted. One such study was the Environmental Scoping Study completed in March 2011, which arose from the need to examine environmental aspects triggered by an earlier feasibility study conducted by Tetra Tech for rehabilitating Kosovo B. The Mission subsequently asked its contractor AEAI to revise the Scoping Statement in 2011 in order to comply with request made by the USAID Bureau Environmental Officer (BEO); otherwise, the Scoping Statement was commissioned and completed in parallel with the Tetra Tech feasibility study for rehabilitation of Kosovo B. With the above USAID activity in the energy sector of Kosovo, several projects resulted. One such study was the Environmental Scoping Study completed in March 2010, which arose from the need to examine environmental aspects related to Tetra Tech conducting a feasibility study for rehabilitating Kosovo B. This report was based on field visits, interviews with GoK representatives and civil society, and review of available reports. The Scoping Statement was approved by the Mission in April 2010 and a Positive Determination Approval Sheet was signed on April 4, 2012 by the Mission Director, COTR, and the BEO highlighting a number of environmental and social issues associated with rehabilitation activities. The approved report was subsequently discussed in 2011 with USAID, the acting BEO, and USAID energy staff. During these discussions, the potential impacts on air quality and climate change related to the power plants were reviewed and whether there was a need to revise the report. Other such USAID involvement has included various capacity building exercises and technical advisory services classified under “Additional Energy Sector Support” for the purpose of this evaluation. Page 53 Addendum to SOW: Realignment/additional tasks and Addendum to SOW: Realignment/additional tasks and budget realignment budget realignment 1. Air Quality Monitoring Data Collection as baseline for an EA for (GenCo) NKPP 1. Gather ambient air monitoring data to inform the scope of the ESIA. Annex 3 Statement of Difference to the Programmatic Impact Evaluation in the Energy Sector in Kosovo prepared by Mendez England and Associates, made by Tetra Tech ES, Inc. implementing Korporata Energjetike e Kosovës (KEK) Network and Supply Project under contract EPP-1-00-03- 00008-00, TO 04 Tetra Tech ES, Inc. 4601 N. Fairfax Drive, Suite 601, Arlington, VA, 22209 USA Tel 1-703-387-2100 Fax 1-703-387-2160 www.tetratech.com September 19, 2012 Mr. Arben Nagavci Program Management Specialist USAID/Kosovo VIA EMAIL SUBJECT: Tetra Tech Comments on the Final Report “Programmatic Impact Evaluation in The Energy Sector in Kosovo” dated September 6, 2012 Dear Mr. Nagavci, Thank you for the opportunity to review the above referenced report. It is unfortunate that Tetra Tech (Tt) was unable to comment on the report before it was finalized as there are errors in fact which, when combined with incomplete analysis, have led to conclusions that are not justified based on the evidence and sound analysis. This report, in its present form, will provide reviewers with an incorrect understanding of the USAID program and its results; more worryingly, it provides an incorrect understanding of lessons learned and recommendations for future program design activities. It seems evident from the content of the report that the context of the USAID program over these past six years was also not adequately understood by the review team. As you are well aware, USAID assistance to KEK was initiated during a time when the United Nations was planning to exit Kosovo as it prepared to declare independence and over 20,000 NATO troops in Kosovo were trying to provide a safe and secure environment. The environment was such that many customers had their own generators, due to poor and unreliable service with load shedding about 14% of the time, demand increasing by 6% per year, a coal mine being depleted, generating units increasingly prone to breakdown and failure, and combined commercial losses and bad debt of over 49%. The situation was highly politicized: for instance, as you will recall, in April 2007 the Prime Minister demanded that the 5% KEK tariff increase approved by ERO be nullified or he would ask the public not to pay for electricity. The sector was in desperate conditions and it was an important step that USAID sought to have its advisers take actions in support of KEK management to help turnaround the system. This included improving collection so that KEK’s expenses could be funded, ensuring that the new mine would be opened, and providing proper maintenance for generating units that had not been maintained properly for almost two decades. All of the aforementioned had to be managed during very uncertain times; hostility and social unrest were a constant risk. Throughout, the report ignores the environment in which this program was undertaken and the specific circumstances found within Kosovo. This lack of understanding of the context becomes even more apparent when, for instance, we see in the report comparisons of the state-owned KEK distribution with a private Indian utility. Fundamentally, it should be stressed that USAID invested about €20 million ($28 million) in Kosovo’s energy sector through the KEK program. In return, the assistance has supported KEK to collect €414 million more in revenue over a six year period. This is but one fact that demonstrates the value for money of the USAID 2 program. This accomplishment is more notable when compared with the EU’s estimated €1 billion assistance to Kosovo (prior to the USAID KEK program) which funded some physical reconstruction but achieved minimal sustainable operational improvements. Since 2007, USAID’s assistance to KEK has resulted in the following:  By the end of 2012, KEK will collect €414 million Euros more, as compared to its 2006 annual baseline of €96 million;  The new mine is fully operational;  More generating capacity was brought on line and output and reliability of all operating units has improved;  KEK is able to import €35 million per year from its own resources, while grants from the Government of Kosovo for import in 2012 amount to only €13 million;  KEK’s 2012 investments reached over €100 million, including €25 million in environmental related areas; and importantly,  USAID’s objective of privatization of KEK’s Network and Supply was achieved. All of the above was achieved with minimal tariff increases (around 5%) prior to 2012. The above successes should also be viewed against the USAID Kosovo mission objectives for the energy sector, namely:  Reliable, 24 hours per day, seven days a week, electricity for all paying households and businesses, leading to improved quality of life, accelerated economic development, employment and investment;  Political and economic security and independence through sufficient domestic energy supply from Kosovo’s own abundant lignite coal;  Financial viability and self-sustainability of Kosovo’s energy utilities to stop the enormous drain on the Kosovo Consolidated Budget (KCB); and,  Private sector participation in order for Kosovo to realize significant benefits from its large lignite coal resource. This program has contributed to all four objectives. It should also be noted that the report’s assessment of the program grossly over emphasizes certain aspects that are peripheral in the views of stakeholders (e.g. ERO derogations which are not restricted to KEK), yet it pays inadequate attention to the overarching changes in the energy sector in Kosovo for the better (e.g. regularization of electric supply to minority communities in southern Kosovo being just one of many such examples). For instance, no mention was made in the report of the extensive support being provided by the USAID program to each of the seven KEK districts, which remains the heart of the KEK Network and Supply Project and where the bulk of the resources have been devoted. Overall, we view this report as inaccurate and biased, reflecting the views of a subset of stakeholders without thorough fact checking and analysis to support the conclusions reached. In our view, it is in the best interest of USAID to reopen this report (i.e., to no longer view it as final) and to allow independent critique to prevent misperceptions or worse still, contribute to improper program planning in the future. 3 Attached please find our detailed comments. Thank you once again for the opportunity to comment. If you have any questions or concerns, please do not hesitate to contact me at Dean.White@TetraTech.com. Sincerely, Dean S. White President Attachment CC: E. Santucci, USAID/Kosovo R. Suratgar, USAID/Kosovo T. Smith, Tetra Tech L. Rowland, Pierce Atwood Tetra Tech Comments Page 1 of 18 18 September 2012 Comments of Tetra Tech on the Final Report “Programmatic Impact Evaluation in The Energy Sector in Kosovo” dated September 6, 2012 The above referenced report was provided to Tetra Tech only in its final form. The following are the detailed comments provided in table format, as suggested by USAID. Reference Issue Tetra Tech Comments Executive Summary Page 1 Unnecessary disputes with other stakeholders Tetra Tech (Tt) relations with stakeholders have been very positive. Information concerning KEK, its operations, and results was freely shared with stakeholders. All issues raised by stakeholders were promptly addressed by Tt and periodic meetings organized by Tt facilitated two way communication. This issue will be addressed in more detail in comments to the main body of the report. Executive Summary Page 1 Persistent Noncompliance with orders issued by the regulatory office Such a comment in the Executive Summary of a report is misleading and is not supported by facts The report references: 1. Derogations requested by KEK from its license obligations and granted by ERO. This is entirely lawful and foreseen by the license conditions themselves to reflect the actual situation in Kosovo. Derogations were also requested and obtained by KOSTT, yet the report omits this fact. Indeed, the report grossly overemphasizes this aspect and the actual implications. 2. Unbundling of KEK. As explained below, the decision to delay legal unbundling was primarily motivated by the desire to improve KEK performance prior to privatization of its distribution business. This objective has been achieved – an investor has been selected and will assume control in 2013, prior to which KEK will unbundle its distribution and supply businesses. 3. Regulatory accounts. KEK has always provided the information it has to the ERO. The accusation that KEK did not prepare “Regulatory Accounts” is true – for the simple reason that ERO and its advisors never specified any Regulatory Accounts. As early as 2008 when KEK was revising its Chart of Accounts, a meeting was held with ERO to request a regulatory chart of accounts that could be incorporated into the accounting system. The ERO did not respond. In 2012 when ERO consultants imposed the OFGEM tariff process (almost verbatim) on KEK, they complained that KEK was not keeping track of costs consistent with the OFGEM method. The reason is, quite simply, that neither the regulator in Kosovo or its consultants specified it wanted costs accounted for in that manner. Mentioning such a minute detail as this while making no mention of the significant progress in areas such as reducing corruption within KEK, focusing management on operating as a commercial business, and providing extensive legal counsel to protect KEK and its assets gives the reader a false impression. Tetra Tech Comments Page 2 of 18 18 September 2012 Reference Issue Tetra Tech Comments Executive Summary Page 1 Many of the objectives of the project as stated in the original Statement of Work (SOW) were not met This blanket conclusion is very misleading and not supported by the facts. Comments on the specifics in the main body of the report provide evidence that objectives were met, in the context of the realities of the actual situation in Kosovo, which evolved in real time. Executive Summary Page 4 The report states: A third political factor is the culture of invasive political corruption within Kosovo, which may have affected various USAID decisions. One particularly stark example is a decision to support the non-unbundling of KEK - apparently because of the potential for political interference by KEK’s Board of Directors - during a time that ERO was constantly ordering KEK to unbundle. This action has not only undermined the notion of promoting a strong ERO, but it is also not in line with the concept of unbundling KEK for the purpose of the KEDS transaction. The pervasiveness of corruption at all levels of government, within KEK, and among business entities is indeed significant in relation to the KEK Network and Supply Project. Significant effort was made by Tt in monitoring procurements, establishing and supporting strong Internal Audit and Anti-Corruption units in KEK, continual monitoring of district operations, and establishing documented procedures. These efforts were undertaken despite considerable opposition both within KEK and from external forces. However, we persisted in our efforts and these have resulted in significant benefit to KEK and the people of Kosovo. The “stark example” cited is neither stark nor an example of the significant efforts to reduce the impact of corruption on KEK. The issue that was thoroughly discussed between Tt and USAID is that corporate governance is an entirely new concept in Kosovo. It was a major effort to build capacity within the current KEK Board of Directors. “Investing” time and effort in bringing a new group of Board members would not have been cost effective. However, this was only one of many reasons why it was decided to defer the legal unbundling of KEK, which were discussed at some length in 2007 and 2008 with international and local stakeholders including the Government and responsible Ministries. Ultimately, it was recognized that the priority was to improve KEK’s performance and operations – and this would be better achieved through maintaining one legal entity. The decision not to legally unbundle at an early stage of the project was taken by the Government of Kosovo and not by KEK. In any event, the above analysis was correct at that time and remains valid today – in the period 2007 to 2012 KEK has progressed from being an insolvent and dysfunctional company to one that is financially solvent and functioning. Contrary to what is reported, this decision has not prejudiced the KEDS transaction. Indeed, one could argue that making KEK solvent facilitated the KEDS transaction – otherwise bidders would have been even more reluctant to purchase the distribution business. Any delay in completing the KEDS transaction cannot be attributed to KEK or the decision not to unbundle. In 2008, KEK – with Tt assistance – took steps to create and incorporate KEDS even though this was not their responsibility. KEK has - with significant assistance from Tt – taken all relevant steps to prepare for legal unbundling and will do so as a condition precedent to completion of KEDS transaction. Indeed, KEK and Tt have undertaken work in the context of the KEDS transaction that should have been undertaken by the transaction advisory team (e.g. drafting of a Collection Agreement) but, in the interests of facilitating Tetra Tech Comments Page 3 of 18 18 September 2012 Reference Issue Tetra Tech Comments the completion of the transaction, KEK and Tt stepped forward to complete these tasks. As far as the ERO is concerned, all accounting information was unbundled and tariffs were developed for each licensed activity. That was sufficient for the state of the Kosovo energy sector then and now. The KEDS transaction has not been negatively impacted by unbundling. As mentioned above, KEK was, and is, prepared to unbundle at any time required for the privatization transaction. Background Pages 7 & 8 Statistics on the industry Gross errors are made in this section of the report. USAID Kosovo has all the correct data for the last six years and that was available to the review team and this data should have been used prior to this report being finalized. There are numerous factual errors in this section related to information that is readily available. It may be an indication of the lack of effort devoted to this report and the minimal amount of time spent to become familiar with the situation on the ground. Factual errors include:  The available capacity of the Kosovo A plant is not 450 MW. The value is closer to 350 MW.  The Kosovo B units do not each have a net capacity of 600 MW. The total net capacity of the plant is about 600 MW.  The volume of imports is NOT constrained by interconnection capacity.  There is no validity to the statement that the 400 kV interconnection with Serbia cannot be relied upon.  The problem with the lack of import is not the power system but is due to several factors including the lack of a proper cost reflective tariff increase to pay for import, insufficient government subsidies and inadequate rule of law. B.4 Qualitative Analysis and Interviews Page 11 List of interviewees The list of interviewees contains some important omissions and other inclusions that are questionable. Important omissions include: (i) previous USAID senior energy advisors in Kosovo, (ii) the last USAID mission director; (iii) large KEK customers – e.g. Ferronikeli and Sharrcem; and (iv) other important stakeholders (particularly for minority communities) – KFOR, UNMIK, OSCE and ICO. It is unclear to us why Felanze Pula was interviewed – she is a former employee of KEK and also the former Managing Director of Termokos. She was dismissed from the first and resigned from the second post. As a former KEK employee, it is questionable what weight was and should be placed on her view point, which is likely influenced by her removal from the company. C.5 Impact Analysis The report states: “There is little doubt that The deficiencies identified by Tt in KEK were not known or made aware to anyone when the USAID RFP for KEK was developed. Based on preliminary assessment, Tt communicated to USAID in its first quarterly Tetra Tech Comments Page 4 of 18 18 September 2012 Reference Issue Tetra Tech Comments Page 13 improvements to KEK’s operational efficiency have been made as the result of the technical assistance, by many measures. However, given the changing nature of objectives and tasks, impacts in this case are best assessed when compared to original expectations”. report the major areas of deficiency. Tt also embarked on further evaluation of deficiencies to determine the depth of the problems and resources needed - internal and external - for their remedy. At project inception, based on the information provided by USAID as part of the RFP, it was anticipated that KEK's situation would allow for improvement and that the management had the know-how and ability to perform their function with USAID advisors serving as coaches. However, upon deployment it was found that owing to both systemic and management deficiencies, the original assumptions were flawed. Impact Analysis Page 14 The report states – with reference to capacity building of the KEK MD: “..However, from interviews of those outside KEK/Tetra Tech, there is strong reason to believe that the above objectives have not been achieved. In fact, the overwhelming perception among stakeholders is that the contractor is actually running KEK…” The report simply relies on the ‘perception’ and hearsay of stakeholders. It is unclear which stakeholders are being relied upon, nor is it clear if the consultants attempted to verify and corroborate such comments. As anybody who has worked in Kosovo for a reasonable time will know, media reports are typically sensationalist and widely inaccurate – equally, opinions and perceptions are open to politicization and personal agendas. Therefore, Tt does not accept the consultants’ conclusion, which is wholly reliant on personal views and perception. The consultants own assessment based on their interaction with the MD is missing. Table 1 Page 14 Billing and Collection Performance Indicators Although billing and collection performance has been consistently measured and reported to USAID in each quarterly report in “Appendix B. Performance-Based Management System Results” (see Attachment I) , the authors chose instead to create Table 1, apparently to make a comparison to a utility in India. That table is misleading as far as “Unaccountable for Energy” is concerned. The authors included energy delivered to Minority Areas (which is accounted for), energy consumed by the mines (which is accounted for), and technical losses (which are accounted for based on engineering estimates) in a measure that should only include unaccounted for energy. That of course distorts the “billed as a % of delivery” statistic which is then compared to a distribution company in India which does not have the issue of minority energy, does not operate a mine, and does not have a network configuration similar to Kosovo to compare technical losses to. Additionally, the statistics for the first five months of 2012 are not comparable to full year statistics due to the seasonality of the Kosovo situation (unlike in Delhi). In this context, it is unclear why the authors attempted to compare KEK with a distribution company in Tetra Tech Comments Page 5 of 18 18 September 2012 Reference Issue Tetra Tech Comments India. What is the commonality between a privatized distribution company in India and a publicly owned Kosovo utility?! Perhaps the authors would have been better advised to look at regional examples in Albania and Macedonia. Impact Analysis Page 15 The report states: “the question then arises of whether the assistance might have been better focused on say, obtaining the necessary capital funding for the distribution and supply system to fix the fundamental problem of leakages.” Continuous improvement was made in :(1) reduction of commercial losses, (2) billing and collecting from all 22,000 minority consumers in Southern Kosovo (who had not paid for electricity in over 10 years), and (3) increasing the collection rate, as fully documented in reports to USAID. It is noteworthy that the report makes little reference to the successful regularization of minority consumers in southern Kosovo – a step that would not have been taken without Tt’s leadership. The statement made by the authors that “leakages” can be fixed by capital funding, shows a lack of understanding that most of the problems experienced in Kosovo relate to people issues, especially corruption on the part of employees, lack of dedication to improvement, and the pervasive corruption in the judicial and other sectors. What facts have been provided to substantiate the above assertion and resulting conclusion? No mention was made of the extensive efforts to restructure district operations, where the theft and corruption occurs. Tt focused on the root causes of losses and worked in the field to achieve reduction in electricity theft and increase payment percentages each year. Capital requirements to reduce technical losses by just one percentage point in Kosovo have been estimated at €75 million, significantly more than the $28 million devoted to the KEK Network and Supply Project. The authors also lose sight (whether intentionally or by oversight) of some very simple arithmetic: With USAID support, KEK’s annual revenue has increased each year of the project over and above the 96m Euros, collected in 2006 (i.e. the base year). If you combine the increased revenue, above the base year figure, a total of €414 Million additional revenue has been collected during a time when there were minimal tariff increases. Such a return on USAID’s investment is difficult to characterize as ‘relatively modest’. Other Impacts Page 15/16 KEK’s nonpayment of the full amount of charges invoiced by KOSTT. It is understood that this matter is currently before the court. The report fails to state that KEK paid KOSTT the full amount of transmission fees that the regulator allowed in KEK’s costs and included in retail tariffs. Payment of greater amounts would have resulted in KEK providing more money to KOSTT than the regulator allowed. This is a very legitimate commercial dispute which is being handled in accordance with Kosovo law. Are the authors of the report suggesting it should be handled in a different manner? Tetra Tech Comments Page 6 of 18 18 September 2012 Reference Issue Tetra Tech Comments It should be noted that two ERO members left ERO and became KOSTT executives; the report ignores any possible conflict of interest and why the ERO’s prior year rulings granted large percentage increases to KOSTT while not reflecting these in retail tariffs. Other Impacts Page 16 A continuing dispute over ownership of certain 110 kV facilities. Despite a decision having been made on this issue, KEK continued to pursue its position. Eventually, USAID through stronger management was able to manage the situation. KEK did not initially agree with the decision that 110KV substations should be transferred from KEK to KOSTT (for many legitimate technical and operational reasons). Indeed, removing such assets from KEK while the KEDS transaction was in progress could legitimately be seen as an example of ‘asset stripping’ – therefore, KEK was right to voice concerns. Ultimately, once the Government made the decision to transfer the assets, KEK complied with the decision and even developed all the information that the regulator needed to reflect the transfer in tariffs. This issue was resolved long before the authors of the report came to Kosovo. Other Impacts Page 16 KOSTT maintains that KEK is an undisciplined consumer, to the extent that the agreed maximum levels of power interchange with Serbia are often exceeded. KEK simply does not follow the instructions of the system operator, thus allowing the Serbian authorities to justify their position that KOSTT cannot maintain system discipline and therefore, should not be recognized regionally as a legitimate system operator. This was confirmed in another recent study by USEA. It is not surprising for KOSTT to state this. They are not serving any customers and get paid 100% of their approved tariff, regardless of any and all situations. Kosovo does not have enough capacity, and it has to manage the load to the best of its ability. KEK has paid back any and all its compensations to Serbia/the Operator without any problem so the consequences of this seem overstated. Tt has not seen the USEA study and would appreciate an opportunity to review. If these studies were conducted simultaneously with this review, why has the USEA review not been shared with KEK or Tt? KOSTT’s failure to secure recognition as a TSO from Serbia or elsewhere is primarily a political problem stemming from Serbia non recognition of Kosovo as an independent state and has little to do with KEK. Other Impacts Page 16 KEK’s request for derogation of certain license requirements The report fails to note that the licenses were written as if they were for a fully developed country – albeit with the provision that ERO could issue derogations to licensees in order to reflect the actual progress and status of the energy sector in Kosovo. Appropriately, ERO recognizes that some of the conditions cannot be met within the time frame; therefore, they have provided a mechanism to allow for derogations. The derogations noted pertain to separate accounts for licensed activities (which KEK provides each year once the unbundled financial statements are audited, which is later than 31 March) and an obligation for insurance, which may be appropriate in developed countries but insurance is not available in Kosovo and ERO recognizes this reality. Tetra Tech Comments Page 7 of 18 18 September 2012 Reference Issue Tetra Tech Comments It should be noted that in the same period, KOSTT has obtained and secured a greater number of derogations from ERO on its license obligations (see Attachment II). The issue of unbundling is fully understood by ERO and it will be accomplished at the appropriate time, just prior to privatization of KEDS. Summary of KEK Network and Supply Project Page 17 “Value for Money” Raising the issue of value for money without taking the time to properly analyze the results leads to incorrect conclusions. As explained above, significant value has been provided to the people of Kosovo, the energy sector of Kosovo, and KEK (i.e. €20 million investment versus a payback of about €414). In addition, the value provided is far in excess of the budgeted cost of the KEK Network and Supply Project ($28 million) as illustrated by the following:  As indicated above, no mention was made of the fact that all minority consumers south of the Iber River were regularized in 2009, including Orthodox religious facilities. Those consumers had not paid for electricity for 10 years. Since they were regularized as KEK customers, they have paid €15 million to KEK (approximately $20 million). This initiative was a prime example of the teamwork between the local Embassy, USAID Kosovo, and Tetra Tech. It is also an example of the close work with stakeholders (UNMIK, ICO, several embassies, OSCE, EU LEX, KFOR, etc).  No mention was made of the fact that Tt legal and internal audit advisors spend significant time monitoring the procurement process to minimize corruption and the cost of materials and services, benefiting Kosovo, KEK and electricity consumers on the order of millions of dollars  No mention was made of Tt efforts to develop a debt settlement agreement with Trepca, resulting in a payment mechanism for €4 million of unpaid debts. Trepca continues to honor its commitment to pay the current bill in full every month and €40,000 of old debt.  The increase in collections from €96 in 2006 to over €200 million in 2011 is due in large part to the extensive efforts of the USAID program in the seven districts. The increase in collections is not due to price increases from ERO (which were minimal), but rather to the day to day efforts of advisors in the districts who are monitoring performance in the areas of meter reading, bill delivery, detection of theft, and the disconnections needed to enforce payment discipline. Negative Impacts Page 21 Compliance with ERO Requests for Information KEK has always provided the information it has to ERO. The accusation that KEK did not prepare “Regulatory Accounts” is true, for the simple reason that ERO and its advisors never specified any Regulatory Accounts. Tetra Tech Comments Page 8 of 18 18 September 2012 Reference Issue Tetra Tech Comments As early as 2008 when KEK was revising its Chart of Accounts, a meeting was held with ERO to request a regulatory chart of accounts that could be incorporated into the accounting system. ERO did not provide any information. In 2012, when ERO consultants imposed the OFGEM tariff process (almost verbatim) on KEK, they complained that KEK was not keeping track of costs the way specific by OFGEM. The reason is that the regulator here in Kosovo never specified it wanted costs accounted for in that manner. Analysis and Summary Page 23 The statement: “US based staff are probably insufficiently familiar with the situation on the ground in Kosovo and need to be better informed.” The US based staff advising KEK are extremely familiar with the actual situation on the ground here in Kosovo. They are the ones working every day on technical, operational, and commercial issues, including in field locations. Tetra Tech Comments Page 9 of 18 18 September 2012 ATTACHMENT I: Key Indicators as Reported in the Tetra Tech 2011 Annual Report to USAID Appendix B. Performance-Based Management System Results 1. Key Indicators (KI) (Reported Quarterly) No. Objectives Supported by These Results Task Reference Supported by These KI Definition of Indicator and Unit of Measure 2006 Actual/ Calculat ion 2007 Actual 2008 Target 2008 Actual 2009 Target 2009 Actual 2010 Target 2010 Actual 2011 Target 2011 Actual 1 1, 2, 3 1 Reduce commercial losses as compared with previous year (ratio of commercial losses vs. energy available for sale) 31% 30% 25% 20% 10% 21% 15% 22% 12% 18% 2 1, 2, 3 1 Reduce technical losses (ratio of technical losses vs. energy delivered to distribution) 18.2% 17.4% 17% 16.6% 16.5% 17.7% 16.4% 17.1% 16.3% 16.8% 3 1, 2, 3 1 Ratio of energy billed vs. energy available for sale 69.1% 69.9% 75% 79.8% 90.0% 79.3% 85% 78.5% 88% 81.9% 4 1, 2, 3 1 Ratio of revenue collected versus billed 74.2% 76.6% 80.0% 75.6% 89.0% 81.4% 86% 87.8% 90% 91.1% 5 1, 2, 3 1 Revenue collected as a percentage of value of energy available for sale [ratio of revenue collected vs. billed] x [ratio of energy billed vs. energy available for sale] 51.3% 53.5% 60% 60.3% 80.0% 64.5% 73% 68.9% 79% 74.7% 6 1, 2, 3 1 Collected revenue in Euros €96 M €110.8 M €116 M €135 M €140 M €160.3 M €155 M €178 M €160M €201 M Tetra Tech Comments Page 10 of 18 18 September 2012 Attachment II: KOSTT Derogations from ERO DEROGATION REPORT FO-ÇLRR-014 ver. 1.0 page 1 to 9 Derogation Report for Grid Code, Metering Code and Transmission System Operator License July 2012 Tetra Tech Comments Page 11 of 18 18 September 2012 DEROGATION REPORT FO-ÇLRR-014 ver. 1.0 page 2 to 9 No. Part of the Grid Code Article Paragraph Required Derogation Period (month/year) Permitted Derogation Period (month/year) Due Derogation Period Reason for Derogation Request 1. Planning Code 4. Transmission Planning Standards and Criteria 4.1.1.2 1. 2 years 2. 2 years 3. 2 years 4. 2 years 1. 2 years 2. 1 year 3. 1 year 4. 1 year 01.11.2009 31.12.2010 31.12.2011 31.12.2012 (N-1) criteria cannot be fulfilled further for elements of the system as follows: - ATR in SS Peja 3 (for 400 yearly hours not filled) - Line 110 kV 1809 SS Prizren 2 – SS Rahovec 2. Operational Planning Code 3.8 Reserve Requirements 3.8.1.1 1. 3 years 2. 1 year 3. 1 year 4. 1 year 5. 1 year 1. 18 months 2. 1 year 3. 6 months 4. 6 months 5. 6 months 27.08.2010 30.06.2011 31.12.2011 30.06.2012 31.12.2012 In the absence of the regulatory units this provision will not be implemented until is realized KOSTT agreement with companies from neighboring countries for spin reserve. 3.8.1.2 The secondary regulation currently is provided according Tetra Tech Comments Page 12 of 18 18 September 2012 DEROGATION REPORT FO-ÇLRR-014 ver. 1.0 page 3 to 9 to the technical agreement from EMS. KOSTT in January 2010 signed an Ancillary Services Agreement with TPP Kosova B, implementation will start by creating the conditions, whereas ASA KOSTT-KESH was signed in March 2010, which will be implemented after insuring the necessary technical infrastructure and final definition how to operate with these ancillary services. 4.4.6 Bids and Offers 4.4.6.1 1. 3 years 2. 3 years 3. 1 year 4. 1 year 1. 3 years 2. 1 year 3. 6 months 4. 6 months 01.11.2010 31.12.2011 30.06.2012 31.12.2012 During the transitional period of implementation Bids and Offers will not be applied. Since do not have yet balancing market, we do not have offers 4.4.6.3 nor requirements. 3. Balancing Code 2.4.4 Balancing Arrangements 2.4.4.1 1. 3 years 2. 2 years 3. 2 years 4. 1 year 1. 3 years 2. 1 year 3. 6 months 4. 6 months 01.11.2010 31.12.2011 30.06.2012 31.12.2012 Until the balancing market is not included in Market Rules, KOSTT will try to balance system using available Tetra Tech Comments Page 13 of 18 18 September 2012 DEROGATION REPORT FO-ÇLRR-014 ver. 1.0 page 4 to 9 2.4.4.2 productions, imports, and power cuts according to the reduction plan ABC. 3.9 Voltage Schedule 3.9.1.3 1. 6 months 2. 1 year 3. 1 year 4. 1 year 5. 1 year 6. 1 year 1. 6 months 2. 6 months 3. 1 year 4. 1 year 5. 1 year 6. 1 year 01.05.2008 01.11.2008 01.11.2009 31.12.2010 31.12.2011 31.12.2012 Characteristics of current generation units do not enable implementation of this requirement. 4.4.5 Time Control 4.4.5.1 1. 1 year 2. 1 year 3. 1 year 1. 1 year 2. 1 year 3. 1 year 31.12.2010 31.12.2011 31.12.2012 Until not become full members of ENTSO-E, we can not fulfill this obligation. 4.4.5.2 Tetra Tech Comments Page 14 of 18 18 September 2012 DEROGATION REPORT FO-ÇLRR-014 ver. 1.0 page 5 to 9 4. Operation Code 5.5.3 System Stability and Overloading 5.5.3.2 1. 1 year 2. 1 year 3. 3 years 4. 2 years 5. 2 years 1. 1 year 2. 1 year 3. 1 year 4. 1 year 5. 1 year 01.11.2008 01.11.2009 31.12.2010 31.12.2011 31.12.2012 Bullet 3 of this article Criteria N-1 is not met for the system elements referred to as article 4.1.1.2 of the Planning Code. In normal conditions (N criteria) are no overload. 5.5.5 Underfrequency Load Shedding 5.5.5.1 1. 6 months 2. 6 months 3. 1 year 1. 6 months 2. 6 months 3. 6 months 31.12.2011 30.06.2012 31.12.2012 Is verified a physical presence by a joint commission KOSTT – KEK and is realized a study (by sector for long term planning) about strategy load shedding if needed. This study was submitted to KEK to see the possibility of deploying this protection in case of need on the low side voltage (10KV and 35KV), but still we do not have any answers. 5.8.11 Black Start Test 5.8.11.1 1. 2 years 2. 1 year 3. 2 years 4. 6 months 5. 1 year 1. 2 years 2. 1 year 3. 6 months 4. 6 months 5. 1 year 01.11.2009 31.12.2010 30.06.2011 31.12.2011 31.12.2012 KOSTT will plan to carry out black start test during 2012 when the conditions are most suitable, since we signed the agreement between TSO-s with Albania. Tetra Tech Comments Page 15 of 18 18 September 2012 DEROGATION REPORT FO-ÇLRR-014 ver. 1.0 page 6 to 9 8 System Test 8.4.1.2 1. 1 year 2. 1 year 3. 16 months 4. 2 years 5. 1 years 6. 1 year 1. 6 months 2. 6 months 3. 6 months 4. 6 months 5. 6 months 6. 1 year 27.08.2009 27.02.2010 31.12.2010 30.06.2011 31.12.2011 31.12.2012 KOSTT consider that in present circumstances Kosovo’ power system, there is no opportunity for testing required by this provision, including generating units. Tetra Tech Comments Page 16 of 18 18 September 2012 DEROGATION REPORT FO-ÇLRR-014 ver. 1.0 page 7 to 9 No. Code Article Paragraph Required Derogation Period (month/year) Permitted Derogation Period (month/year) Due Derogation Period Reason for Derogation Request 1. Metering Code 8.2 Capacity and Energy Measurements 8.2.1.6 1. 1 year 2. 6 months 3. 2 years 4. 14 months 5. 2 years 6. 2 years 1. 1 year 2. 6 months 3. 1 year 4. 1 year 5. 1 year 6. 1 year 01.11.2008 01.05.2009 01.05.2010 30.06.2011 30.06.2012 30.06.2013 Have to be installed metering transformers with separate secondary windings. Currently in all interconnection lines 400 kV, 220 kV and 110 kV as in the HPP Ujmani and SS Valaq are not with separate secondary windings. Tetra Tech Comments Page 17 of 18 18 September 2012 DEROGATION REPORT FO-ÇLRR-014 ver. 1.0 page 8 to 9 No License Article Paragraph Required Derogation Period (month/year) Permitted Derogation Period (month/year) Due Derogation Period Reason for Derogation Request 1. TSO 9. Availability and Maintenance of Data 9.4 1. 1 year 2. 6 months 3. 1 year 4. 1 year 5. 1 year 6. 1 year 7. 1 year 8. 1 year 9. 1 year 1. 6 months 2. 6 months 3. 6 months 4. 6 months 5. 6 months 6. 6 months 7. 6 months 8. 6 months 9. 6 months 24.11.2008 24.06.2009 24.11.2009 30.06.2010 31.12.2010 30.06.2011 31.12.2011 30.06.2012 31.12.2012 Maintaining of data information about the border capacities is made but no publications due to different circumstances. We also do not have the proper hardware and software for calculation of load flow, congestion management, power plan dispatch and ancillary and balancing market. All of this are being conducted through PSSE software, but yet they won’t be valid without the combination of neighboring TSO￾s models and this is as a consequence of KOSTT not being part of regional mechanisms like ITC/CBT and membership in ENTSO-E, illegal and unauthorized actions of EMS in allocating of KOSTT interconnections capacities. 2. 11. System Operation 11.1 1. 1 year 2. 1 year 3. 2 years 4. 13 months 5. 2 years 1. 6 months 2. 1 year 3. 7 months 4. 6 months 5. 1 year 24.11.2008 24.11.2009 30.06.2010 31.12.2010 31.12.2011 Because that balancing mechanism is not included in transitional Market Rules, the requirements of this article cannot be fully implemented (h). Under the current circumstances, 11.2 (d, h dhe i) Tetra Tech Comments Page 18 of 18 18 September 2012 DEROGATION REPORT FO-ÇLRR-014 ver. 1.0 page 9 to 9 6. 1 year 7. 1 year 6. 6 months 7. 6 months 30.06.2012 31.12.2012 KOSTT do not manages interconnection capacities and can not be anticipated imports and exports in interconnection lines (d and i bullets). Similar obligation as in (i) bullet is in Operational Planning Code, article 3.9 Interconnector Capacity. 3. 12. Economic purchasing of assets, services and ancillary services 12.1 1. 3 years 2. 2 years 3. 2 years 4. 2 years 5. 2 years 6. 1 year 7. 1 year 1. 1 year 2. 6 months 3. 6 months 4. 6 months 5. 6 months 6. 6 months 7. 6 months 01.11.2009 30.06.2010 31.12.2010 30.06.2011 31.12.2011 30.06.2012 31.12.2012 Due to actual circumstances in Kosovo Power System, Tertiary Regulation cannot be ensured according to the conditions foreseen to the Grid Code as well as in tariffs cannot be covered. 12.2 End of the Document