DRAFTREPORT POWERING PROGRESS PROJECT END OF PROJECT EVALUATION REPORT FINAL REPORT February, 2013 PPP End of Project Performance Evaluation Report, February 2013 ii POWERING PROGRESS PROJECT END OF PROJECT PERFORMANCE EVALUATION REPORT FINAL REPORT Submitted to: USAID/East Africa Submitted by: Dr. Anthony Getambu (Team Leader) Contractor: Sewa Business Services Ltd Amirali Building Ground Floor P O Box 90237 -80100 Mombasa, KENYA Tel +254-733-752-038 Email: agetambu@sewa.co.ke DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Acknowledgement The evaluation team from Sewa Business Services Limited sincerely thanks USAID/EA staff and all stakeholders for their support in this work. The team is grateful to all the interviewees from the various power sector organizations, governments, missions, the EAC and the private sector, for sparing time off their busy schedules for discussions in this evaluation and giving their insights on the project. Finally, we sincerely thank EAPP and Nexant Inc. staff for their generosity in providing the relevant reports and feedback on the Powering Progress Project to facilitate completion of this work. PPP End of Project Performance Evaluation Report, February 2013 iii Table of Contents Acknowledgement ..................................................................................................................................... ii Table of Contents ..................................................................................................................................... iii List of Tables ............................................................................................................................................ iv List of Acronyms ....................................................................................................................................... v AfDB ......................................................................................................................................................... v African Development Bank ....................................................................................................................... v EXECUTIVE SUMMARY ....................................................................................................................... 1 1. INTRODUCTION ............................................................................................................................. 7 1.1. Background........................................................................................................................................ 7 1.2. USAID Strategy in Eastern Africa .................................................................................................... 8 1.3. The Powering Progress Project in Context ........................................................................................ 9 1.4. PPP Tasks and Contract Modification ............................................................................................. 10 2. EVALUATION PURPOSE AND METHODOLOGY ................................................................... 12 2.1. Purpose ............................................................................................................................................ 12 2.2. Methodology.................................................................................................................................... 12 2.3. Limitations ....................................................................................................................................... 14 3. FINDINGS ...................................................................................................................................... 15 3.1. Relevance ........................................................................................................................................ 15 3.2. Achievements Including Outputs and Outcomes ............................................................................ 15 3.3. Effectiveness.................................................................................................................................... 23 3.4. Efficiency ........................................................................................................................................ 26 3.5. Client Satisfaction ........................................................................................................................... 31 3.6. Impact .............................................................................................................................................. 32 3.7. Sustainability ................................................................................................................................... 33 3.8. Best Practices................................................................................................................................... 36 3.9. Lessons Learned .............................................................................................................................. 37 4. CONCLUSIONS AND RECOMMENDATIONS .......................................................................... 39 Annex A: Tasks in the Initial Project Design .......................................................................................... 43 Annex B: Statement of Work .................................................................................................................. 48 Annex C: List of Documents Reviewed .................................................................................................... 1 Annex D: Work plan ................................................................................................................................. 2 Annex E: Eastern Africa Regional Power Projects ................................................................................... 7 Annex F: Evaluation Methodology ........................................................................................................... 9 Annex G: INTERVIEW GUIDES ........................................................................................................... 11 Annex H: SWOT Analysis of PPP .......................................................................................................... 19 Annex I: List of Key Informants Interviewed ......................................................................................... 20 Annex J: Lists of PPP participants .......................................................................................................... 23 PPP End of Project Performance Evaluation Report, February 2013 iv List of Tables Table 1: PPP Achievements ...................................................................................................................... 19 Table 2: Activities funded by EAPP Development Partners ..................................................................... 29 PPP End of Project Performance Evaluation Report, February 2013 v List of Acronyms AfDB African Development Bank COMESA Common Market for Eastern and Southern Africa COR Contracting Officer’s Representative EAC East African Community EAPP Eastern Africa Power Pool EEPCo Ethiopian Electric Power Corporation EOI Expression of interest ERC Energy Regulatory Commission ETAs Electricity Trading Agreements EU European Union EWSA Energy, Water & Sanitation Authority (Rwanda) FERC Federal Energy Regulatory Commission FMA Norwegian Government Ministry of Foreign Affairs GECOL General Electric Company of Libya GoK Government of Kenya IDS EAPP Institutional Development Strategy IGMOU Inter-Governmental Memorandum of Understanding IMF International Monetary Fund IPP Independent Power Producer IUMOU Inter-Utilities Memorandum of Understanding KenGen Kenya Electricity Generating Company KETRACO Kenya Electricity Transmission Company KPLC Kenya Power and Lighting Company MOU Memorandum of Understanding PPP End of Project Performance Evaluation Report, February 2013 vi NARUC National Association of Regulatory Utility Commissioners PJM Pennsylvania-New Jersey- Maryland power pool PPA Power Purchase Agreement PPP Powering Progress Project PTS Power Transmission Standards RDB Rwanda Development Board REGIDESO Regie de Production et de Distribution d’eau et de d’electricite (Burundi) RFP Requests for Proposal RFTOP Request for Task Order Proposal SADC Southern Africa Development Community SAPP Southern African Power Pool SETCO Sudanese Electricity Transmission Company Limited SIDA Swedish International Development Agency SINELAC Societe International d’Electricite des Pays des Grands-Lacs(Burundi, DRC and Rwanda) SOW Statement of Work TANESCO Tanzania Electric Supply Company UETCL Uganda Electricity Transmission Company Limited US United States of America USG Government of the United States of America USAID United States Agency for International Development USEA United States Energy Association WAPP West African Power Pool Was Wheeling Agreements WB The World Bank PPP End of Project Performance Evaluation Report, February 2013 1 EXECUTIVE SUMMARY Introduction This report is the end of project performance evaluation for the Powering Progress Project (PPP) funded by USAID/EA at a cost of US$ 2,104,953 and implemented by Nexant Inc. between April 2010 and July 2012. The evaluation was conducted by Sewa Business Services Limited in the period between November 30, 2012 and February 5, 2013. The purpose of PPP was to provide technical assistance and capacity building support to key entities in Eastern Africa (Burundi, Democratic Republic of Congo (DRC), Egypt, Ethiopia, Libya, Kenya, Rwanda, Sudan, Tanzania and Uganda) and to establish a regional electricity market. The primary focus of PPP was to:  develop model bilateral Electricity Trade Agreements (ETAs) and Wheeling Agreements (WAs);  develop Regional Power Transmission Standards for EAPP member countries;  and build capacity to exploit clean and renewable energy resources, harmonize regional policies and regulations for improved cross-border energy trade, and to improve the technical and financial performance of Eastern Africa Power Pool (EAPP) member utilities. Countries within the East African Community (EAC) and Eastern Africa Power Pool (EAPP) region have generally been planning and implementing their power system in an isolated manner focusing on satisfying their respective national demand growth. Although, bilateral power exchange agreements exist between some countries in the region, maximum benefits have not been realized as the volumes of power exchange are insignificant and exporting parties often fail to meet their contractual obligations, due to power deficits in their systems caused by absence of a robust and conducive institutional, regulatory and commercial framework. Power trading through PPP End of Project Performance Evaluation Report, February 2013 2 common planning and implementation of regional generation and interconnection projects has been identified as one important strategy for tackling the problems associated with power supply shortages, low access, high cost and poor supply reliability. Governments within the Eastern Africa region are now working towards integrating their power systems and creating a regional power market through EAPP, a regional organization established in 2005. EAPP membership comprises ten countries, thirteen utilities, and seven national electricity regulators from the member countries. The countries are Burundi, Democratic Republic of Congo (DRC), Egypt, Ethiopia, Libya, Kenya, Rwanda, Sudan, Tanzania and Uganda. Evaluation Purpose and Methodology The objective of this assignment was to conduct an end-of-project performance evaluation for the Powering Progress Project in order to: a) enable USAID/EA determine the extent to which PPP’s objectives were achieved; and to b) examine the efficiency and effectiveness of the PPP implementation strategy. The evaluation applied a mixed-method approach which involved two qualitative data collection methods; in￾depth key informant interviews and document review. The use of mixed methods was to enable triangulation of data from different sources hence validating the authenticity of the data. The methodology consists of the following steps: Literature review of project-related documents; identification of key informants and sites to visit done through directed or purposive sampling method; primary data collection through documents review and in– depth key informant interviews and discussions with staff from USAID/EA, EAPP, Nexant and other stakeholders including participants from Eastern Africa regional utilities, governments and the private sector; and data and information analysis. Summary of Findings PPP End of Project Performance Evaluation Report, February 2013 3 Nexant was largely effective and efficient in implementing PPP as it produced most of the deliverables in time and at no additional cost; more staff were trained than planned; their support in bilateral power trade and cross￾border electrification (CBE) agreements was useful. However, a few challenges hindered effective implementation of the PPP. These included limited presence of the PPP Nexant staff in the region, which posed communication challenges and affected engagement levels between the contractor, EAPP and the beneficiaries. 1. It was evident that different EAPP member utilities had different priorities and had varying levels of technical capacity, and therefore each country’s needs should have been assessed and considered at the project design stage. 2. There were inadequate donor consultations right from the project design to implementation, which led to overlaps. A coordinated approach would have helped avoid overlaps and duplication in donor-supported projects and programs, and help increase effectiveness and efficiency. 3. PPP helped build and strengthen the institutional capacity of EAPP and other institutions, as affirmed through training of at least 103 energy sector staff; development of IDS for EAPP and PTS for the region, and in their role in negotiations of Ethiopia-Kenya power sales agreement and the Uganda-Tanzania cross-border trade agreement; the IDS providing a detailed roadmap for institutional development of EAPP and leading to the EAPP 2012-2014 Corporate Plan. It was noted that lack of consistency in regional staff involved in EAPP training and other activities impede effective capacity development in the region. 4. The face-to-face approach applied in the regional training workshops organized under PPP had limited effectiveness and coverage due to limited resources required for travel and upkeep, especially in the absence of cost-sharing arrangements.. Alternatives such as in-country trainings, e-learning and video-conferencing have potential for wider coverage and more effectiveness. 5. PPP contributed to improving the sustainability of the EAPP’s programs through providing a roadmap for capacity development through the IDS and the subsequent three- year EAPP Corporate Plan, which when PPP End of Project Performance Evaluation Report, February 2013 4 implemented will enable the region to have skilled staff and managers to manage and operate a modern power pool. PPP presented several training sessions and facilitated knowledge transfer and skills development across the member states in the region. Inclusion of a study tour to the US provided exposure to staff of EAPP and its member organizations to operations and regulation of a modern power pool, although this involved a small number of staff in the region due to the resource limitations and unavailability of some of the nominated staff. 6. PPP contributed to EAPP’s overall institutional goal and strategic objectives by creating a regional power market and optimal use of renewable energy resources through power trade. This is evident in the conclusion of the Ethiopia-Kenya electricity trade agreement and the Uganda-Tanzania cross-border electrification projects during the implementation of PPP. The conclusion of these agreements clearly demonstrates the relevance, effectiveness, impact and sustainability associated with PPP. 7. PPP registered immediate positive impact that was in line with project objectives and the power pool goals. Examples of these include the Power Transmission Standards (PTS) and a Gap Analysis tool, which had immediate impacts in EAPP and COMESA; the IDS which found immediate application by EAPP; support in the successfully concluded Ethiopia-Kenya Bilateral Power Supply Agreement (PSA) and the Uganda￾Tanzania CBE agreements and in the review of Cross-Border Policy by EAC. 8. The evaluation found evidence that PPP beneficiaries started taking over ownership of the outputs from PPP, including promoting regional cooperation and advocating the best practices developed and disseminated under PPP. Most results and impacts of PPP are sustainable in terms of creating institutional capacity and filling gaps in EAPP and members states. However, further assistance from donors and more effort by EAPP are needed to make other outcomes ultimately sustainable, such as training of trainers so as to have a pool of regional experts to help develop local capacity further. PPP End of Project Performance Evaluation Report, February 2013 5 9. PPP was relevant and beneficial to the development of capacity for EAPP and its members. Its implementation was largely successful despite the initial implementation challenges that led to modification of the project design. Recommendations 1. In future, similar USAID-supported projects need to have more presence or representation of the implementing contractor in the region so as to ensure more engagement with the client and beneficiaries at country level, in order to maximize information flow, interaction with other donor projects with similar objectives and to ease communication, project coordination and alignment. 2. In future, task-specific regional training needs assessments should be conducted so as to ensure that the course contents are suitable for varying levels of participants from the various countries. 3. There is need to improve donor coordination in the EAPP and EAC region so as to avoid overlaps and duplication of effort. EAPP and EAC need to work in close consultation during design and implementation of power projects since EAC membership in now a subset of EAPP. This should be done as the regional power regulatory structures and bodies take shape. 4. All key stakeholders should be involved or kept informed during project design to avoid implementation challenges and later modifications that result in delays during implementation. This may be achieved through a combination of enhanced donor engagement and EAPP technical sub-committee members. 5. EAPP member countries should ensure that there is consistency of participants involved in the regional power pool project activities, to maximize effectiveness, impact and sustainability of the gains made in capacity building and regional power market development. To achieve the intended objectives of the power grid standards and grid code harmonization, the Power Transmission Standards and Gap Analysis component of PPP should be rolled out to all member countries. USAID and other donors should collaborate with EAPP, EAC and COMESA in this important project for realization of the intended benefits and to sustain the gains made through PPP. PPP End of Project Performance Evaluation Report, February 2013 6 6. In future capacity building, there will be need for USAID and other donors to consider cost-sharing and inclusion of more in-country trainings of high priority, conducted by a combination of international resource persons and regional professionals with relevant hands-on experience in the region. To increase coverage and efficiency, alternative approaches in training such as webinars, e-learning, virtual training, video conferences and in-country training workshops should be considered, as they can save on cost of travel and upkeep. There should be follow-up activities in such training and projects. Twinning and attachments to pools for hands-on training would increase effectiveness. The Norwegian Government and SIDA have included in their ongoing technical assistance to EAPP a twinning arrangement with a developed power pool in Europe. 7. Taking into account inadequate resources and capacity noted during PPP implementation, USAID and other donors should consider providing additional institutional development support for implementation of the IDS and the EAPP Corporate Plan for 2012-2014. Implementing the strategy and plan is imperative in steering the region towards the power market and trade envisaged, since EAPP is the focal point. Furthermore, it is necessary for operationalization of the power pool and facilitation of the envisaged power trade. 8. USAID and other donors should consider supporting conclusion of future bilateral agreements, wheeling agreements and cross-border electrification projects in the EAPP region, based on the models developed by Nexant under PPP. This would help expedite creation of the desired regional network and power market. . 9. Donors should consider providing strategic support for training of trainers in the region, covering such fields as financial modeling, power pool planning and operations, and power transmission standards. This would help make PPP outcomes ultimately sustainable in the power pool. PPP End of Project Performance Evaluation Report, February 2013 7 1. INTRODUCTION 1.1. Background This report is the end of project performance evaluation for the Powering Progress Project (PPP) sponsored by USAID/EA and implemented by Nexant Inc. of the United States, between April 2010 and July 2012. The evaluation was conducted by Sewa Business Services Limited in the period between November 30, 2012 and February 5, 2013, under Contract No. AID 623-C-13-00001. The purpose of PPP was to provide technical assistance and capacity building support to key entities in eastern Africa and to establish a regional electricity market. The primary focus of PPP was to: develop model bilateral Electricity Trade Agreements (ETAs) and Wheeling Agreements (WAs); develop Regional Power Transmission Standards for Eastern Africa Power Pool (EAPP) member countries; and to build capacity to exploit clean and renewable energy resources, harmonize regional policies and regulations for improved cross-border trade, and improvement of the technical and financial performance of EAPP member utilities. EAPP membership comprises ten countries, thirteen utilities, and seven national energy/electricity regulators from the member countries. 1 EAPP aims to reduce power generation and transmission costs, improve the reliability of the power supply, and encourage investments in power trading within the region by implementing large-scale projects that benefit from economies of scale. The essential element in achieving the above will be the establishment of a conducive environment that will attract necessary public and private-sector investment to increase access to sustainable, commercially viable supplies of electricity from increased cross-border energy. In 2006, the Common Market for 1 The countries are Burundi, Democratic Republic of Congo (DRC), Egypt, Ethiopia, Libya, Kenya, Rwanda, Sudan, Tanzania and Uganda. The EAPP member utilities are: Regie de Production et de Distribution d’eau et de d’electricite(REGIDESO)of Burundi, Societe Nationale d’Electricite(SNEL)of DRC, Egyptian Electric Holding Company, Ethiopian Electric Power Corporation(EEPCo), Kenya Electricity Generating Company (KenGen), Kenya Electricity Transmission Company (KETRACO) and Kenya Power, General Electric Company of Libya(GECOL), Energy, Water & Sanitation Authority (EWSA) (formerly ELECTROGAZ) of Rwanda, Sudanese Electricity Transmission Company Limited(SETCO) (formerly NEC), Societe International d’Electricite des Pays des Grands-Lacs(SINELAC)of DRC, Rwanda and Burundi, Tanzania Electric Supply Company(TANESCO) and Uganda Electricity Transmission Company Limited(UETCL). Uganda and its transmission utility are the latest EAPP members having joined the power pool in 2012. PPP End of Project Performance Evaluation Report, February 2013 8 Eastern and Southern Africa (COMESA) adopted EAPP as a Specialized Institution for Electric Power in the Eastern Africa Region. 1.2. USAID Strategy in Eastern Africa USAID has been involved in providing capacity building and technical assistance to two other power pools in Africa - the West African Power Pool (WAPP) in western Africa and the Southern African Power Pool (SAPP) in southern Africa. USAID/EA has worked closely with the Eastern Africa Power Pool and other development partners to complement and build on ongoing work in the sector. This falls within the five-year African Global Competitiveness Initiative (AGCI) and is aimed at promoting the export competitiveness of enterprises in sub￾Saharan Africa in order to expand African trade with the United States, other international trading partners, and regionally within Africa. AGCI builds on the African Growth and Opportunity Act (AGOA) and works with other donor initiatives. To achieve improved export competitiveness of African businesses, AGCI has two main strategies, to: 1) provide technical assistance that advances export competitiveness; and 2) support complementary activities that contribute to the objectives of AGCI. AGCI seeks to leverage its resources with those of other USAID-funded programs and those financed by other international financial institutions as well as with investment resources from the private sector; AGCI strives to achieve a greater development impact through a multiplier effect. Over the past three years, USAID/EA has provided Technical Assistance through the African Growth and Competitiveness Initiative’s East Africa (AGCI-ECA) Trade Hub4 to EAPP members in their initial discussions on electricity exchange and trade. These discussions, supported under the then East African Trade Hub in Rwanda and Uganda in 2008, formed the basis of developing electricity trading regimes among the EAPP member countries. Activities to further develop electricity trading and wheeling agreements implemented under PPP were aimed at building on the earlier negotiations under the Trade Hub that would contribute to reduced barriers to trade, improved market access, and regional integration. PPP End of Project Performance Evaluation Report, February 2013 9 PPP was also in line the USAID Global Climate Change Initiative (GCCI), 2010, USAID’s Climate Change and Development Strategy (2012-2016) whose objectives are to: invest in clean energy technology and reduce deforestation to decrease greenhouse gas emissions; help countries and communities prepare for and respond to changes in climate; and make climate change considerations part of all aspects of USAID’s programs and operations. This USAID initiative aims to achieve the following results by 2016: help 20 partner countries develop and implement strategies for increasing economic growth with lower emissions; integrate climate change into food security, global health, democracy, and other development priorities; and learn the most effective ways to support low emission, and climate resilient growth. 1.3. The Powering Progress Project in Context Eastern Africa covers an area of about 7 million square kilometers with a population of more than 360 million people. The region has an average GDP per capita of approximately $ 408 and total annual energy consumption per capita of 320 kWh. The region suffers from a low industrialization rate, a high dependence on low-energy biomass fuels, and limited access to electric energy. The eastern Africa countries are working towards integration of their power systems and creation of a regional power market through the EAPP, a regional organization established in 2005. The Powering Progress Project (PPP) was initially to be implemented from April 24, 2010 to April 23, 2012 but was extended to July 24, 2012. The initial ceiling cost for PPP was set at $2,481,037 million for the core and optional tasks but the final reviewed allocation, following project modification, was $2,104,953.PPP was funded as a Task Order implemented by Nexant Inc. under Energy II IQC [AID-EPP-I-OO￾03-00007 (AID-623-TO-I0-00002)]. PPP objectives were: to increase capacity of exploiting clean and renewable energy; improve EAPP’s capacity to operate and manage a modern power pool; harmonize policies and regulations for improved cross-border trade ; and increase private-sector investment in electricity by addressing specific policy and regulatory barriers. PPP End of Project Performance Evaluation Report, February 2013 10 1.4. PPP Tasks and Contract Modification The original PPP design comprised of the following tasks: Core Tasks A. Electricity trading skills enhancement to increase capacity of government and utility representatives to pursue and negotiate electricity trading agreements. B. Establishing Mechanism for Electricity Trading with a target of not less than three cross-border trading agreements facilitated. C. Development of Power Transmission Standards. D. Best practices on operation of modern power pool- for improved skills among EAPP membership, board members and staff for modern power pool operations through knowledge and skills transfer. Optional Tasks Option 1: Additional Electricity Trading Agreements (ETAs) Option 2: Electricity wheeling agreements Option 3: Improving electricity utility financing to attract private sector. Option 4: Best practices for operation of a power pool Option 5: Technical Assistance to East African Community (EAC) to develop a cross-border electrification protocol, with a target of no less than three cross-border rural electrification protocols and implementation plans developed. In Optional Tasks 1 and 2, the PPP contractor was to engage the EAPP members in a participatory process to develop new draft ETAs using existing power sale agreements, and finally have signed ETAs. PPP established that only the Ethiopia-Kenya electricity trade agreement was ready as it was under negotiation and the interconnector project was at an advanced stage. Optional Tasks 1 and 2 were therefore merged with Core Task B on Mechanisms for Electricity Trading. Optional Task 3 was excluded because it was underfunded, less relevant, PPP End of Project Performance Evaluation Report, February 2013 11 and the activity was being undertaken by Mercados Energy Markets International (EMI) under EU funding. Optional Task 4 was also dropped, as it was deemed similar to Core Task D. Nexant, EAC and USAID modified Optional Task 5 to support the further development of the Cross-Border Electrification Policy (CBE) policy, building on the work performed by Mercados EMI, since EAC had, upon obtaining other unanticipated funds, contracted Mercados to execute this program by the time Nexant began implementation of PPP. Countries in the EAC/EAPP region have generally been planning and implementing their power systems in an isolated manner, focusing on satisfying their respective national demand growth. A few bilateral power exchange agreements exist between some countries in the region but the power trade is too limited. Power trading through common planning and implementation of regional generation and interconnection projects is identified as an important strategy for tackling the problems associated with power supply shortages, low access, high cost and poor supply reliability. The benefits include complementary hydropower reservoir operation, optimal capital investment in generation and transmission, inter-country support in restoring the power system after a national blackout, improved system load factor and other operational efficiencies. PPP was designed to address some of these constraints and to meet the objectives of EAPP. Existing and ongoing regional power generation and transmission projects are shown in Annex E. PPP End of Project Performance Evaluation Report, February 2013 12 2. EVALUATION PURPOSE AND METHODOLOGY 2.1. Purpose The primary purpose of this evaluation as described in the SOW is to enable USAID/EA determine the extent to which PPP’s objectives and results have been achieved and to examine the efficiency and effectiveness of the PPP implementation strategy. The evaluation seeks to highlight the best practices and record lessons learned from the project for future USAID/EA programming. The fundamental questions to be resolved by this evaluation are:  To what extent has PPP been able to build capacity and strengthen EAPP and other institutions within its member states?  Was Nexant effective and efficient in implementing PPP activities?  Was PPP effective in achieving EAPP’s overall institutional goal and strategic objectives?  Are the PPP achievements/outcomes sustainable? 2.2. Methodology The evaluation team used a mixed-methods approach where two qualitative data collection methods were used: in-depth key informant interviews and document review. The use of mixed methods was to enable triangulation of data from different sources hence validating the authenticity of the data. This approach offers a high level of rigor and empirical validity to support evaluation analysis and conclusions. The evaluation required the following steps: Inception meeting with USAID/EA: The objective of the meeting was briefing, acquaintance with key project staff, obtaining project related documents and agreeing on work plan. Literature Review: Relevant documents from various sources including EAPP, USAID/EA, Nexant and the internet were reviewed on commencement of this evaluation. The documents included but were not limited to PPP PPP End of Project Performance Evaluation Report, February 2013 13 scope of work, Nexant’s contract/contract modifications, detailed work plan, performance management plan (PMP), progress reports from the start of the project and final report. Other reports reviewed included project activity reports, such as study tour reports, workshop reports, training materials, EAPP’s institutional development strategy and Norwegian Embassy project scope of work. A list of documents and reports reviewed is contained in Annex C of this report. Data collection tools: Interview guides were developed for each stakeholder group to be interviewed. They were tailored to the research questions and key informant. Each of these groups provided different types of information. The interview guides were pre-tested with the EAPP staff. Duplicate notes were taken for most meetings. Illustrative interview guides are provided in Annex G. Key Informant Interviews Identification of key informant, sites and participants: Directed or purposive sampling method was applied in identifying sites to be visited and key informants. The countries selected were Ethiopia, Kenya and Rwanda. Ethiopia was selected as it had the most number of beneficiaries besides hosting EAPP Secretariat and was involved in the PTS pilot project and the ETA for the interconnector with Kenya, which was part of the projects supported through PPP. Kenya was selected as it hosted a significant number of beneficiaries, has the largest representation of power utility organizations that are members of EAPP, and is an active member of both EAPP and EAC. Kenya also participated in the pilot Power Transmission Standards (PTS) project, and was involved in the interconnector with Ethiopia. Rwanda is a potential key beneficiary of the interventions. Key informants from project beneficiaries identified are those who participated in most of the tasks. Key Informant Interviews: The team interviewed staff from USAID/EA, EAPP staff and its sub-committee members, Nexant Inc., and PPP participants/beneficiaries from Kenya, Ethiopia, Rwanda. In addition, the team interviewed staff at East African Community (EAC) and Common Market for Eastern and Southern Africa (COMESA), involved with energy and EAPP issues, Rwanda Development Board (RDB), other donors/projects supporting EAPP and two private sector organizations in Kenya – the Kenya Private Sector Alliance (KEPSA) and the Kenya Association of Manufacturers (KAM). A complete list of key informants is presented in Annex I. PPP End of Project Performance Evaluation Report, February 2013 14 Primary Data Collection: Interview guides developed were used to conduct the semi-structured in-depth interviews and lead in the discussions with the key informants. A total of 31 respondents were interviewed, with face-to-face interviews conducted with informants from Kenya, Ethiopia, Rwanda, COMESA and EAC. Nexant Inc. staffs were interviewed through telephone calls and emails. At the end of each of the visits to Ethiopia and Rwanda, a briefing of USAID mission in the country was done. Data and Information Analysis: All data and information collected was analyzed and harmonized to generate the debriefing presentation, first draft report and the final performance evaluation report. This being a qualitative study, data gathered was analyzed through identifying common ideas and patterns observed, then interpreting by attaching significance to the themes and patterns. 2.3. Limitations The key constraint faced by the evaluation team was the timing of the evaluation itself. The evaluation was conducted in month of December, which is a vacation period in the region with the Christmas holidays, and most potential respondents were either away or preparing for vacation. It was therefore difficult to secure interviews with some of the beneficiaries. Another challenge was that some of the project outputs were similar to those from other ongoing projects funded by different donors, and the evaluation team had difficulties isolating gains that were solely attributable to the PPP interventions. Other limitations included accuracy level of all data/information obtained on specific tasks based on feedback received, especially where participants were involved in more than one task or workshop conducted by different donor-supported projects but with common objectives. In some cases, participants could not accurately recall details of the activities they were involved in, possibly due to the time lapse since the occurrence of the activity, and also their involvement in various similar activities implemented by other donors under EAPP. PPP End of Project Performance Evaluation Report, February 2013 15 3. FINDINGS 3.1. Relevance How relevant is this project to the short, medium and long term development needs of EAPP and its partner states? PPP aimed to improve capacity of EAPP, EAC and their member states in implementing regional electricity trade in clean energy. The key informants interviewed were concurrent on the need and relevance of PPP. The EAPP Permanent Secretariat, for example, lauded the Institutional Development Strategy (IDS), as it greatly enhanced EAPP’s ability to manage and operate a power pool. PPP developed model Electricity Trading Agreements (ETAs), Wheeling Agreements (WAs) and cross-border electricity trading agreements (CBEs) that would, in the medium to long term, permit transactions to occur between buyers and sellers including countries that do not share a border. Key informants repeatedly reported that the associated trainings were very useful since they enabled enhancement of the capacities of regional power sector staff in power trade agreements, financial modeling and tariff development and in advancing towards harmonization of their power transmission standards even though some countries were not ready for regional power trade. EAPP staff who participated in the study tour activity observed that this was timely as they are in the process of setting up a regional power pool and they now know best practices on the operation of such a pool. However, staff of regulatory agencies in the EAPP region were not included in the tour, yet it focused more on regulation. Key informants also reported that the PTS is very important for harmonization of the regional grids and that COMESA and EAPP are pushing for adoption of codes and sub-codes across the region. 3.2. Achievements Including Outputs and Outcomes What are the major outputs, outcomes/impacts of the project on EAPP and partner states? PPP major outputs and the associated outcomes are discussed below: PPP End of Project Performance Evaluation Report, February 2013 16 Institutional Development Strategy: All EAPP staff interviewed were in agreement that IDS and the associated 3- year EAPP Corporate Plan are among the most important PPP outputs as they would enable EAPP to increase capacity to perform and meet its objectives. The Corporate Plan, which is a product of the IDS, has deliverables including roles, responsibilities and timelines. The strategy has an associated budget of $27 million for the various elements. Development Partners [the Norwegian Agency for Development Cooperation (NORAD), the World Bank and the African Development Bank (AfDB) among others] are consequently aligning their support to the Corporate Plan and strategy. This provides an environment for improved donor coordination to avoid overlaps and duplication, and a systematic approach in capacity development. Electricity Trading Skills Enhancement: A two-day workshop was held in Nairobi, Kenya, attended by 27 participants aimed at increased capacity of government and utility representatives to pursue and negotiate electricity trading agreements. The attendees reported gains in understanding electricity markets and pricing, power pool operations, planning and experiences from others based on case studies and discussions held. They got a better understanding of how the electricity markets are developed and implemented, the requirements of the market as it evolves and other power trade functions. Mechanisms for Electricity Trading: The aim was to develop and facilitate signing of bilateral trade agreements in the region, based on the assumption that several bilateral Electricity Trade Agreements were being negotiated. Participants in the electricity trading skills enhancement workshop reported that they advanced their knowledge on electricity trading/wheeling agreements and associated financial modeling which will be useful when actual electricity trading starts. Under the merged task, the draft Ethiopia-Kenya Power Sale Agreement (PSA) 2012 was reviewed and a financial model developed to help rationalize pricing negotiations for the interconnection project. The Ethiopian utility, EEPCO, received technical assistance from Nexant in the PSA negotiation. The Ethiopia-Kenya PSA was concluded with support from the project. In addition, 25 participants (2 women, 23 men) from EAPP stakeholders were trained under this task in a two-day workshop held in Arusha, Tanzania, in June 2012 covering electricity PPP End of Project Performance Evaluation Report, February 2013 17 trading, wheeling, and application of the EAPP Financial Model. The financial models and the template ETAs and WAs developed provide a foundation for future agreements and for training of more stakeholders on power trade and wheeling agreements. From the introductory modeling done during the training, one pilot utility, KETRACO, indicated that they are now developing basic models for power wheeling agreements for some transmission lines. They applied knowledge gained from the two-day power modeling training workshop held in Tanzania to prepare applications for national licenses for power wheeling. Participants found the model useful but reported that the training period was not adequate for them to understand the model fully. Some participants who were at a more advanced level in modeling tariffs and power trade agreements rated the training workshop lowly as they expected more challenging, real and relevant wheeling tariff modeling based on a deeper understanding of the region and its requirements, taking into account the diversity of the participants. Thus, training needs assessments should have been conducted so as to determine the varying training needs of different participants. Power Transmission Standards: Nexant developed draft regional technical standards for the exchange of electricity within the interconnected transmission systems. Utilities from Kenya and Ethiopia were used to develop and validate PTS. A Workshop on the Interconnection Code and Power Transmission Standards of the COMESA Sub-Committee of Technical Regulations was held in May 23 - 25, 2012 in Nairobi, Kenya, attended by 28 participants. Key informants that were trained and involved in the pilot project indicated that they are now able to use the Gap Analysis tools for harmonization of regional power transmission standards. The PTS and the gap assessment tool were presented to EAPP member utilities and a follow-up workshop was conducted for all COMESA members to initiate adoption by COMESA so that future regional interconnection projects would require compliance with the Interconnection Code (IC) and PTS as approved by the Standards and Quality Assurance Committee (SQA) of COMESA. The COMESA Council of Ministers is in the process of adopting the IC and PTS for application throughout Eastern and Southern Africa. Ethiopia has already adopted the standards and this is expected to happen in other EAPP member countries. The evaluation team learned that the Namibia power regulator is also considering adopting PTS. TANESCO is already using PTS to review the national grid code and they plan to PPP End of Project Performance Evaluation Report, February 2013 18 eventually adopt it as the national standard. The outcome is therefore evident in the tools developed and the start of the harmonization of the power transmission standards in the EAPP region and beyond. Best Practices on Operation of Modern Power Pool: The objective of this task was to introduce EAPP participants to Best Practices on the operation of a regional power pool through a study tour of a modern power pool planned to target 11 members of staff from the PPP beneficiary institutions. A study tour was conducted to Washington D.C, in the United States, on March 26 - 30, 2012, attended by eight staff from the EAPP region, to observe the operation of power pools and utilities; identify components that contribute to a successful, functioning power pool; and to gain external knowledge of power pool operations. The study tour took the eight participants to Pennsylvania - New Jersey - Maryland Interconnection LLC (PJM), a regional transmission operator; the Federal Energy Regulatory Commission (FERC); the United States Energy Association (USEA), and the National Association for Regulatory Utility Commissioners (NARUC). Participants interviewed said they got exposure to power pool operations, market design and plans and understanding operations of the market and monitoring center. They appreciated how developed power markets evolved over time and described exposure to the US power trade as, “very insightful”. The sessions were said to have been very interactive but had no adequate time for the many questions from the participants. Participants appreciated status of developed power markets, how far they have come, and their issues and trends. This helped build conviction among participants that a regional power market is achievable in the EAPP region too. A participant from Rwanda indicated that he noted PJM had a lot of data for their system, and the Rwandese utility, EWSA, is also trying to advance in the same direction as it is installing a modern data acquisition system. The participants learned that regulators in PJM had full access to system data, and have common committees and very close communication and cooperation with utilities. However, the study tour and associated discussions related more to regulators but the participants would have preferred to also get perspectives of the utilities. Moreover, the study tour was short and could not include any hands-on experience and visits to utilities despite most of the participants having come from power utilities. PPP End of Project Performance Evaluation Report, February 2013 19 Cross-Border Electrification Protocol: The aim of this optional task was to provide technical assistance to EAC through developing a draft regional regulatory framework for cross-border rural electrification for EAC and its stakeholders and at least 3 cross-border rural electrification protocols and implementation plans. The EAC considered support in the Tanzania-Uganda Cross-Border Electricity (CBE) trade agreements valuable as it promoted the desired integration process and provides visibility of integration at grass-root level. Nexant conducted a legal and technical review of the EAC’s policy framework in accordance with the modified contract. A total of 15 participants (inclusive of 3 female) from the EAC countries, were trained on the financial models developed at the CBE workshop held in Arusha, Tanzania in July 2012. EAC said the two financial models developed by Nexant for the cross-border bilateral projects are the most attractive outputs of PPP. EAC also indicated that Nexant pointed out areas that could be improved in the Mercados cross-border draft policy and this made the policy review easier. EAC plans to present the draft cross-border policy to the EAC Sectoral Council on Energy for adoption. Participants interviewed indicated that they required more time to understand and apply the models to gain proficiency. No other follow-on activities were reported. The evaluation team summarized the outputs achieved by PPP as presented in Table 1 below. Table 1: PPP Achievements Task Expected Outputs Achievements Comments Institutional Development Strategy(IDS)  IDS for EAPP  An Institutional Development Strategy for the Eastern Africa Power Pool (January 2012), including a memorandum regarding comments were submitted.  IDS was prepared and it turned out to be one of the most valued outputs of PPP. PPP End of Project Performance Evaluation Report, February 2013 20 Task Expected Outputs Achievements Comments Core Task A Electricity Trading Skills Enhancement.  2-day workshop conducted for 30 participants from target countries and others within the region involved in electricity negotiations to take place in Nairobi, Kenya.  Workshop manual with COTR￾approved curricula.  Workshop Report documenting workshop proceedings, participation and outcomes in English and French languages available for distribution to participants.  2-day workshop in Kenya attended by a total of 27 participants.  Manual developed for the workshop.  Electricity Trading Skills Workshop - Completion Report.  Presentations and post-workshop evaluations.  Good workshop attendance with a variance of 3 in participants. Core Task B Establishing Mechanism for Electricity Trading (Includes Optional Task 1 on Additional Electricity Trading Agreements and Optional Task 2 on Electricity  A model ETA written and shared with all stakeholders.  Facilitate signing of at least 6 ETAs and 3 WAs  A report containing the proceedings and outcomes of the bilateral negotiations and recommendations on the best methods for implementation of the ETAs.  A training workshop to enable EAPP members to understand and work with the financial models used in negotiating specific regional power projects.  ETA and PSA for EEPCO, Kenya Power and KETRACO.  ETA for TANESCO, Kenya Power and KETRACO.  Model wheeling agreement.  Model Wheeling Agreement Among Ethiopian Electric Power Corporation, Kenya Power and TANESCO  Wheeling Agreement for Kenya Power and UETCL, and EWSA  Wheeling Agreement between Société Nationale d’Electricité and Energy, EWSA and Régie de  The EEPCO/KPLC ETA was concluded and signed.  ETA templates were prepared and shared with all stakeholders for use by countries when agreements are in progress.  3 WAs PPP End of Project Performance Evaluation Report, February 2013 21 Task Expected Outputs Achievements Comments wheeling agreements) Production et de Distribution d'Eau et d’Electricité.  Transmission Network Service Agreement between KETRACO and Kenya Power  A training workshop to enable EAPP members understand and work with the financial models used in regional power projects held in Arusha, June 2012, attended by 25 participants inclusive of two women. developed but not signed. Core Task C Development of Power Transmission Standards.  A draft document of regional electricity transmission standards to be written with input from and shared with stakeholders.  A report documenting the development process, input and outcomes on the drafting of regional electricity standards which also outlines recommendations and what additional steps are necessary, if any, to adopt the transmission standards submitted to EAPP and its members. An electronic copy must be submitted to USAID/EAST AFRICA.  PTS, various Codes, Gap Analysis, and training deliverables.  Power Transmission Standards Guide for Use of Power Transmission Standards and Associated Tools.  Power Transmission Standards￾Gap Analysis Workbooks  Workshop on the Interconnection Code and Power Transmission Standards of the COMESA Sub￾Committee of Technical Regulations May 23 - 25, 2012,  No set target number of participants for the PTS workshop. PPP End of Project Performance Evaluation Report, February 2013 22 Task Expected Outputs Achievements Comments  Workshop on the Interconnection Code and Power Transmission Standards of the COMESA.  A training workshop to enable EAPP regulators to apply the gap analysis tool.  A workshop for utilities for application of the regional PTS Gap Analysis Tool. Nairobi, Kenya  COMESA Combined Standards Core Task D Best practices on operation of modern power pool.  Facilitate Study tour for no less than 11 Technical Committee members to Pennsylvania-New Jersey-Maryland (PJM) power pool and other organizations in the US.  Study tour Report with recommendations for follow on activities produced and copies made available for distribution. Electronic copy of final version delivered to USAID/East Africa.  Study Tour, March 26 - 30, 2012, Washington, D.C. attended by 8 participants.  Study Tour Report -Best Practices of a Modern Power Pool  Variance-3 in participants led to achievement of 62.5%. Optional Task 5 Technical Assistance to East African  Application of Cross-Border protocols for 3 rural electrification projects within the EAC region.  A draft regional regulatory framework for cross-border rural electrification for the EAC written and shared with  Final Report for Option 5;  A legal brief on small-scale cross￾border electrification projects; a pro-forma financial model for Mutukula and Murongo A and B project cases; project descriptions  No set target number of participants for the CBE workshop. PPP End of Project Performance Evaluation Report, February 2013 23 Task Expected Outputs Achievements Comments Community (EAC). stakeholders.  Report on process to engage EAC members to adopt regional protocol for cross-border rural electrification prepared. for the three project cases;  An assumptions book for the two financial models; and workshop materials presented at the CBE workshop, held in July, 2012. 3.3. Effectiveness Was PPP effective in achieving EAPP’s overall institutional goal and strategic objectives? The management of the activities in PPP enabled completion of the tasks under the modified contract. Participants interviewed were happy with the piloting approach used in development of PTS, as it enabled the realization of the suitable tools for immediate adoption and application. The key informants, however, cited the short duration of workshops and trainings as a cause for inhibiting full effectiveness and realization of desired results under PPP. This is mainly because the classroom approach is less intensive in skills’ development compared to hands-on training where participants deal with real cases. The study tour was effective in providing insight and exposure, as reported by key informants, necessary for them to comprehend operations of a modern power pool. One participant described the content and resource persons in the study tour as “excellent”. Key informants indicated that the effectiveness would have been even more had the tour been longer or had included hands-on experience and/or attachments/visit to the power utilities participating in the power pool. An example cited is the twinning arrangement for EAPP with a European power pool in the technical assistance by Norway and SIDA. The participants in the tour also needed to get perspectives from the modern utilities in order to understand the operating environment and how the utilities benefited from the market. Nexant developed two financial models which were effectively applied in the CBE project between Uganda and Tanzania. The models are considered by EAC the most attractive outputs in this task. Success of the Kikagati￾Murongo helped prevent disputes and enhanced regional cooperation. EAC would have improved the PPP End of Project Performance Evaluation Report, February 2013 24 effectiveness of PPP had it been involved more, but it could not due to human resource capacity limitations. The forum where the models were presented was useful as discussions were held, awareness created, and good participation by stakeholders registered. Nexant also highlighted areas that could be improved in the Mercados cross-border draft policy and this made review easier and clearer. The workshops provided effective forums for discussions on private sector participation in energy generation, distribution and supply –possible models, experience in the region, restructuring and extension of electricity supply services to another country. According to EAC, the Nexant team was suitable for the work as they were knowledgeable and experienced. However, they did not provide draft reports for circulation in good time before the workshops as required for increased efficiency, yet EAC could not raise the matter due to lack of a legal framework between the two organizations. Nexant was flexible in the CBE task as they would go out of their way to meet EAC at different forums and/or venues where possible, recognizing that the EAC energy sector activities had to go on. They took up the work on bilateral agreement for Kikagati-Murongo though it was not in the original project design. Nexant helped in bringing out some issues on the policy for input in the later modified project design developed and in the aspects of negotiations between Uganda and Tanzania, the two countries involved in the CBE project. As mentioned earlier, EAC is poised to present the revised Cross-Border Policy to the EAC Sectoral Council. The PPP trainings carried out through presentations made in workshops were found to be inadequate since the participants needed more time to interact and understand the financial models and agreements from the basics so as to expand their knowledge and proficiency in applying it. The program management structure and the adopted implementing PPP were largely effective but there was room for improvement. Overlaps were noted in ongoing CBE work activities by EAC under different donor funding. Improved coordination would have helped in avoiding a situation where EAC went ahead with CBE citing delays in procurement of the PPP contractor. There were overlaps between capacity building activities undertaken under PPP and those funded by the EU and Norwegian Government. Donor coordination and harmonization in handling capacity building activities for EAPP and EAC activities would help in minimizing or avoiding such overlaps. PPP End of Project Performance Evaluation Report, February 2013 25 Training programs and other activities could be sequenced to complement each other, enhance synergy and optimum use of resources. EAPP PS expressed concerns over lack of consistency/continuity in participants from member utilities, regulators and governments involved. This has negative impact on effectiveness of trainings provided to EAPP members and also on input to EAPP-related work performed through regional staff. EAPP PS recognizes that although member organizations value and are committed to the regional power pool, they often have other equally important and sometimes more urgent tasks competing for attention from the same staff involved in regional power project activities. On the other hand, some training participants repeatedly benefit from capacity building. Although this is appropriate for continuity, there may be need for diversification of the staff trained, to spread capacity more widely within institutions. The EAPP PS indicated that for increased effectiveness, national procedures for debriefing, to report back proceedings, after attending regional meetings should be established to ensure follow-up actions and continuity. Moreover, the EAPP PS suggested inclusion of EAPP work in individual performance contracts of staff of member organizations involved in key EAPP work, to help ensure that each staff fulfills his/her obligations. They recommended that nominees to EAPP sub-committees should have permanent alternates for consistency, continuity and contingency/succession planning. Three inflection points were identified in this project. Firstly, the review of the project design achieved through rearrangements of tasks and review of deliverables to make the project implementable. The second was the extension of the project beyond the initially agreed completion dates, following a request by Nexant. The third was the change in the project team following a notice issued by USAID to terminate the contract with Nexant. The Project Manager and Project Coordinator were replaced in August 2011. The PPP team leader pulled out for personal reasons but was not replaced for 6 months. This had negative impact on the pace of project implementation. The deficiency in management teams led to inability to meet the reporting requirements in four occasions. PPP End of Project Performance Evaluation Report, February 2013 26 The extension enabled Nexant to develop a financial model for the Ethiopia-Kenya Interconnection to help rationalize pricing negotiations, conduct a workshop in May 2012 on regional Power Transmission Standards (PTS) for adoption throughout Eastern and Southern Africa by COMESA, conduct the Study Tour and hold the Cross-Border Electrification workshop in July 2012. The decision by USAID to grant the no-cost extension was appropriate given the situation. Had this not happened, it would have most likely impacted negatively on effectiveness of PPP and possibly resulted in premature closure. Contract terminations are known to result in undesirable protracted disagreements. Was cooperation and coordination among donors effective? As mentioned above, there were overlaps among implementing partners in PPP assistance to both EAPP and EAC. Donors and EAPP did not work closely in coordinating work plans and activities so as achieve EAPP’s overall objective more efficiently. There is, therefore, need for better donor coordination for more effective use of resources. Table 2 highlights the various development partners supporting EAPP and the activities they are funding. 3.4. Efficiency Was the project implemented and managed in a cost-effective manner? The project was undertaken within the budget and the outputs expected in the modified contract were all delivered. The initial ceiling cost of PPP was set at $2,481,037 million for the core and optional tasks. A total of $2,104,953 was spent in the modified project which excluded Optional Tasks 3 and 4. This expenditure was equivalent to the set ceiling for the project following modification. PPP had performance monitoring indicators for measuring its outputs during implementation. The initial project monitoring plan had a target for 5 organizations to undertake governance improvement activities and Nexant achieved 7 organizations. PPP End of Project Performance Evaluation Report, February 2013 27 The project trained 103 staff of organizations in the region following project modification compared to the initial target of 41 persons. The beneficiaries included technical staff, economists, financial experts, managers and policy makers from EAPP Permanent Secretariat, power utilities, government ministries and regulatory authorities. The evaluation survey further established that most of the members of the three technical sub￾committees of EAPP were among the beneficiaries of the capacity building activities. EAPP is in the process of making these sub-committees operational and has created web-based networking forums to facilitate communication and information sharing among members. Involving policy makers and other senior management staff from the region provides an opportunity for efficient uptake of best practices and implementation of the recommendations from the project. PPP activities largely complemented other activities by other EAPP Development Partners. Model ETAs, WAs and cross-border power trade agreements as well as the PTS were built on previous work by Mercados under EU funding. In preparing the IDS, technical assistance in PPP was complementary to previous and ongoing donor support to EAPP and the region. The strategy would complement on-going capacity building activities being undertaken by the EAPP Permanent Secretariat; the Mercados EMI/SOFRECO completed Deliverable 21 Detailed Training Programme of July 2010; Deliverable 7- Development of Regional Strategic Road Map to 2025; Deliverable 11 -EAPP Regional Market Design; the NORAD funded brief assessment of future training needs as a part of their review of Norwegian support to EAPP; and an assessment of capacity building needs in the region was being conducted for the Nile Basin Initiative under the Regional Power Trade Project in 2011. Further capacity building is being carried out under a Norwegian government program. Following preparation of the institutional development strategy, EAPP adopted the report and developed their three year corporate plan. This utilization of the technical assistance through application of best practices happened efficiently within the PPP implementation period. There were cascaded impacts registered after PPP developed Power Transmission Standards (PTS) and a Gap Analysis tool based on a review of the EAPP Interconnection Code (IC) to facilitate compliance with the IC. Follow-up workshops were held to initiate adoption by EAPP and COMESA so that future regional PPP End of Project Performance Evaluation Report, February 2013 28 interconnection projects would require compliance with the IC and PTS as approved by the Standards and Quality Assurance Committee (SQA) of COMESA. The methods of training applied were mainly face-to-face. In order to increase coverage and efficiency, alternative approaches in training such as webinars, e-learning, virtual training, online teaching, video conferences and in-country training workshops should be considered. These approaches can save on cost of travel and upkeep required in face-face training workshops. However, distant training does not provide some of the benefits of face￾to-face training such as direct networking of the professionals. What could have been done to improve the efficiency of the project? Training could have been carried out less expensively, if more were done virtually, through online teaching or video teleconferencing. Organizing training of trainers’ sessions would have significantly developed regional expertise. Virtual training, if used in future, is likely to result in a multiplier effect in the event other donors need to enhance the support provided. It was evident that different member utilities had different priorities and therefore future assessment of regional training needs should also consider each country’s needs. The regional power pool is not as functional as expected, since most of the member states still prefer to operate on bilateral basis and have not seen the opportunity for promoting the regional agenda. The next phase of PPP should consider targeting a higher percentage of policy makers and political leaders, perhaps focusing on harmonization of high level, cross-border, power-related regulatory and commercial issues that could be important to policy makers. Most of the stakeholders interviewed indicated that they had attended one or more of Nexant’s trainings and that they had benefited. While the individuals and institutions may have benefited, EAPP needs to explore mechanisms to build upon and consolidate training results in order to promote the regional agenda. EAPP needs to encourage countries to form permanent working groups to deal with important issues. The power pool may also need to decentralize its staffing in order to develop a better and more nuanced understanding of individual institutions as well as help identify individuals who could benefit from more training. PPP End of Project Performance Evaluation Report, February 2013 29 Nexant did not establish adequate presence or representation within the project region. This resulted in communication challenges due to time difference between the region and the USA, and limited engagements between PPP contractor and project implementing partners/beneficiaries. Local presence can be achieved through establishing a regional project office or consortium contracts with local firms. Contractor presence would help improve effectiveness and efficiency in project implementation. PPP contractor had challenges at the start of the project which made it replace some key management staff in the project team. However, this action took long, leading to unnecessary delays in the project implementation. Similarly, delays and changes may have affected timely submission of reports on CBE to EAC which required them for reference in policy review and reporting. The EAC key informant expressed a desire for a contractual framework between EAC and contractors in future engagements such as PPP. The various PPP tasks could have been sequenced in more optimal way so as to provide adequate time for completion of CBE, ETAs and WAs which, despite time extension, were not completed as planned. The challenges mainly resulted because the tasks were scheduled to start towards the end of the project. Table 2: Activities funded by EAPP Development Partners Project Title and Budget Development Partner Activities Technical Assistance & Capacity Building Project (€ 2.7m or $3.5m) European Union (EU)  Preparation of the Eastern Africa Power Market Development.  Preparation of a strategic plan and a 5 year business plan.  Preparation of power market rules and agreements for cross-border trade. Regional Master Plan & Grid Code Study (AfDB/NEPAD AfDB/NEPAD IPPF  Objective was to develop EAPP/EAC Power Master Plan & Common Grid Code and train staff to enable them update the master plan in future.  The project is completed and the Power Master Plan and Grid Code final report submitted in May 2011. PPP End of Project Performance Evaluation Report, February 2013 30 IPPF) ( $1.7m) Ministry of Foreign Affairs Norway Project ($2.2M ) Government of Norway  Recruitment of staff for the Coordination Centre (CC) and Independent Regulatory Board (IRB) - completed  Installation and operationalization of the data base in progress.  Installation and operationalization of the Communications Infrastructure  Facilitation of Institutional set up of the CC and the IRB.  Recruitment of Data Base Consultant - completed  Recruitment of Training Consultant - completed  Recruitment of Management Consultant – completed  Training of over 60 experts (Secretariat Staff and utility representatives) on issues of regional power trade. Powering Progress Project ($2.1m) USAID  Electricity trading skills enhancement for government and utility representatives to pursue and negotiate ETAs.  Establishing Mechanism for Electricity Trading no less than 3 CBE trading agreements facilitated.  Development of the PTS and Gap Analysis tool  Best practices on operation of modern power pool- overseas study tour for improved skills among 11EAPP technical staff for modern power pool  Option 1: at least 3 Additional ETAs  Option 2: at least 6 Electricity Wheeling Agreements  Option 5: Technical Assistance to East African Community (EAC) - cross￾border electrification protocol no less than 3 cross-border rural electrification protocols and implementation plans.  Implementation of Regulatory Regime in line with MFA, AfDB and EC Projects.  Development of IDS PPP End of Project Performance Evaluation Report, February 2013 31 3.5. Client Satisfaction Determine if the primary beneficiary EAPP and its utility members’ needs were met. If the EAPP members’ needs were not met –provide detailed explanation. If not met, what was not met and why? Key informants interviewed expressed their satisfaction with the IDS developed by PPP. The strategy fits in PPP’s specific objective to improve EAPP’s capacity to operate and manage a modern power pool. It is also seen in the vision for EAPP for medium term and long term development needs through the strategy prepared for EAPP under PPP. EAPP and other key informants were satisfied with the PTS and the Gap Analysis outputs and the methodology applied in carrying out this Task. The PTS is seen as being of high quality and suitable for immediate application in the EAPP member countries. All interviewees gave consistently positive views and feedback on their satisfaction with PTS and Gap Analysis, and recommended that the Gap Analysis pilot be rolled out to all EAPP members. According to EAC, the Nexant team was suitable for the work as they were knowledgeable and experienced. Their approach was good as they also included discussions with key stakeholders, Ministries, and utilities. However, they needed to provide CBE related draft reports early for circulation to invitees in good time before workshops. Nexant took up the work on bilateral agreement for Kikagati-Murongo though it was not in the original project design, and helped bring out some issues on the policy for input in the later scope developed. EAC is to present the revised Cross-Border Policy to the EAC Sectoral Council but Nexant was yet to give their final report and recommendations -thus their effectiveness in this role was not without challenges. Nexant helped in the aspects of negotiations between Uganda and Tanzania. The regional staff that attended the workshops on financial model, power trade and wheeling agreements expressed satisfaction with the trainers and the workshop training content. The key informants found the study tour and PPP workshops suitable and beneficial. However, they expressed need for longer workshops and training sessions to improve effectiveness and sustainable impacts. Hands-on approach would have boosted their level of satisfaction especially in financial modeling. PPP End of Project Performance Evaluation Report, February 2013 32 3.6. Impact What were the impacts from PPP? The Power Transmission Standards (PTS) and a Gap Analysis tool had immediate impacts as EEPCO adopted them so as to work towards integration of power networks in readiness for regional power trade. EAPP and COMESA are in the process of adopting the standards to facilitate compliance with the EAPP Interconnection Code (IC). The IDS for EAPP found immediate application: it gave rise to the EAPP Corporate Plan, which is already being used by donors to align/coordinate support to EAPP. PPP contributed to successful conclusion of the Ethiopia-Kenya bilateral for 400MW PSA. The Ethiopia-Kenya line will have a capacity of 2000MW, thus providing surplus capacity for regional trade. The model ETAs and WAs developed by PPP would then be applied in agreements for trading surplus power beyond Kenya. The Uganda-Tanzania CBE agreements and the review of the EAC draft Cross-Border Policy benefited from PPP. The financial models developed for the PSA and CBE had immediate application and would provide a foundation for future negotiations of contracts in the regional power market. PPP enhanced human resource capacity for the 103 participants trained through the workshops held and the study tour of the US. Which constraints impeded the full realization of the intended impacts of the project and how did the Contractor respond to them? For about a year during project implementation, Nexant had no local representative, which resulted in poor coordination of the projects considering the difference in time zones and also difficulty in securing project data and organizing project events. Nexant responded to this by recruiting a local project officer to facilitate follow-up on various project issues and liaison with EAPP to organize trainings. Nexant had a challenge obtaining feedback from the beneficiaries. In their assessment, this could have been due to poor communication flow between EAPP and its members. Another challenge was scheduling training, due to participants’ unavailability and Nexant had therefore to arrange for shorter trainings. Participants, however, found the workshops too short for the comprehensive contents in PPP trainings. Participants in the trainings were not always consistent. EAPP had a PPP End of Project Performance Evaluation Report, February 2013 33 challenge communicating to its membership and consequently, reviews and comments on Nexant’s outputs came late. To the extent possible, measure the increase in the delivery of EAPP services to partner states? The development of the IDS and corporate strategy has helped EAPP strategize and focus better on their own needs and those of the member countries resulting to more and targeted funding from developments partners. Not being yet an operational pool, EAPP’s services to member states are still limited but the foundation laid by PPP provides a basis for the pool to provide better services and support to its members in future. What were the unintended impacts from PPP? Namibia, in the SAPP region, is reported to be considering adopting the Power Transmission Standards developed under PPP. This has led to interest among other countries in SAPP region following suit and is expected to ultimately facilitate intra-regional and inter-regional power trade and promote sustainability of the outcomes of PPP. 3.7. Sustainability What are the prospects for the sustainability of the end results produced by PPP? The evaluation team found evidence that PPP beneficiaries are taking ownership of the outputs from PPP, including promoting regional cooperation and advocating the best practices developed and disseminated under PPP. The IDS prepared by PPP enabled preparation of the EAPP Corporate Plan for 2012-2014. Implementation of these two strategic plans would bridge capacity gaps in the region and thus enhance capacity for power pool operation to promote regional power trade. Key informants recommended that, for PPP activities to be sustainable through implementation of IDS, EAPP should take the lead in mobilizing funding for some projects on behalf of member countries, as pointed out in the IDS. Nexant provided additional recommendations to be PPP End of Project Performance Evaluation Report, February 2013 34 considered for implementing the IDS, hopefully with additional support from USAID, to make the outcome more sustainable. The EAPP Permanent Secretariat considers PTS in addition to IDS as the greatest achievements of PPP. A key informant suggested that EAPP consider recruiting staff from outside the member states, such as from SAPP and WAPP, so long as such staff bring with them real experience and skills that are much needed for effectiveness and sustainability in the future. An additional justification for this was that the staff from outside EAPP would be more objective and non-partisan for EAPP sustainability. Power Transmission Standards and the associated outputs developed are seen as a valuable resource for harmonization of the grid code. PPP had immediate impact from this Task as the Interconnection Code and PTS developed are in the process of being adopted by EAPP and COMESA. Ethiopia also adopted the standards and TANESCO is using PTS to review the national grid code and it plans to eventually adopt it as the national standard. Moreover, Namibia, in the SAPP region, is reported to be considering adopting the standards. The entire region is expected to follow suit and this would ultimately facilitate regional power trade and promote sustainability of the impacts and outcomes of PPP. Under the tripartite agreement between EAC, COMESA and SADC, COMESA is pushing for all grid codes and sub-codes and standards to be used among all member states. Further roll out and implementation across EAPP and COMESA states will no doubt enhance sustainability of the outcome. The Gap Analysis tool associated with PTS has been tested and found suitable for application in harmonization of power transmission standards and grid code in the region. The Gap Analysis spreadsheet, developed under this activity, is a tool for use by member utilities to address the gaps between their current national standards and the proposed regional standards. The tool enables review of the current standards and analysis to determine gaps, impacts and what needs to be done to bridge the gaps. The approach used in the gap analysis work aimed to make the participants own the work of grid code review, gap analysis and preparation of a roadmap for bridging the gap(s). PPP End of Project Performance Evaluation Report, February 2013 35 The Gap Analysis tool was tested through a pilot project involving Ethiopian utility, EEPCO, and its Kenyan counterparts, Kenya Power and KETRACO. The tool enables prioritizing of the list of gap items so as to develop a roadmap to compliance with the EAPP Grid Code standards. Nexant demonstrated the use of the gap assessment and roadmap development tools by working with the three utilities that practically applied the tools, although Kenya has not gone further in using the tool after the training was completed. It is recommended that national teams comprising utilities and regulators be set up to ensure implementation of transmission standards and grid code. Informants were satisfied with the tools developed and the training methodologies applied, but they needed more than the two-day workshops to fully understand and apply the tools. The two-day workshops were not enough to make the regional staff fully conversant with the tools developed. It was envisaged that all EAPP members were to later be trained but this is yet to happen. The national teams involved in the pilot project determined what they needed to do upon completion of PPP and forwarded their reports to Nexant. A larger training was to take place to present the tool to all members of the pool, but this has not taken place yet. Another concern raised is that the Kenyan national regulator, Energy Regulatory Commission (ERC), was not involved in the pilot project yet it is a key stakeholder in the national grid code and is its custodian. It may be more challenging to review the grid code fast without involving regional power regulators. The study tour to the US was important but by nature costly as it involved air travel and upkeep for participants overseas for the entire training duration. The number of people supported in such a program is therefore subject to the available resources, which are often limited as was the case in PPP. Two facets of sustainability need to be looked into – firstly, the program itself as part of capacity development and secondly, the knowledge gained and its application for the target purpose. More donor support or internal mobilization of resources would be necessary to sustain the program for a more expansive coverage and increased impacts. Moreover, target nominees and groups should be carefully selected to ensure relevance and focus for optimum results. PPP End of Project Performance Evaluation Report, February 2013 36 Most results and impacts of PPP are sustainable in terms of creating institutional capacity and filling gaps in EAPP and members states but further assistance from donors and more effort by EAPP are needed to make others ultimately sustainable. Which of the achieved results appear to be unsustainable, or less sustainable, and why? The PPP objective was to facilitate regional electricity trading. This is not likely to happen in the short run, as the interconnecting power lines are not yet in place and also because the countries seem to prefer to sign bilateral agreements. There is need to institutionalize dialogue among the countries of the region at the techno political level. Multilateral forums, where countries routinely speak to one another at the technical, policy and political levels, need to be fully institutionalized. Such forums, as opposed to the current bilateral conversations, can promote the regional agenda and facilitating them may become an important niche for EAPP. Was the scale of USAID support appropriate to ensure sustainability? The scale of USAID support was moderately appropriate to initiate sustainability of outcomes and therefore there is need for continued support to further institutionalize the regional agenda to make EAPP and the regional electricity trading markets functional. EAPP as an organization has weak structures and has not been able to realize its objectives and, therefore, requires more support to strengthen the structures. Additional resources should have been set aside for furtherance of activities initiated by PPP, such as PTS, which required setting up country committees and involvement of national electricity regulatory bodies in the work for continuity of the work started by the project. Besides resources, strategies should have been put in place for furtherance of activities and mobilization of resources to complete the initiatives. 3.8. Best Practices Flexibility of parties in PPP contract: Projects that involve negotiations between various parties and multiple countries come with their own challenges and hence flexibility among the various players is crucial. USAID, Nexant and other stakeholders in PPP project exercised flexibility. This was a good attribute as modifying the project through merging tasks made the PPP contract more easily implementable. PPP End of Project Performance Evaluation Report, February 2013 37 Piloting approach in undertaking project tasks: In developing the PTS, Nexant devised an apt pilot project involving three utilities from Kenya and Ethiopia. The practical application of the grid code gap analysis tool developed for harmonization of standards was a means of validating the model and an effective approach for imparting skills and knowledge. Taking ownership of project outcomes: EAPP took ownership of IDS , setting a good example to the beneficiaries and stakeholders in PPP. Donors also started making use of the IDS and the EAPP Corporate Plan, thus enhancing sustainability of the PPP outcomes. 3.9. Lessons Learned 1. Adequate consultation during project design and implementation: There were design related implementation challenges due to initial assumptions in PPP which led to modification of the project design to make the PPP implementable. These could have been avoided if more consultations were held among USAID, EAPP, EAC and other partners of EAC, right from project inception. 2. Assessment of the Capacity of Implementing Partners: The capacity and availability of implementing partners should be confirmed at the project design stage. EAPP and EAC were expected to play bigger roles but this later presented challenges during PPP implementation. The two organizations were also handling other parallel donor-supported activities that required attention from their lean staff in addition to the organizations’ core responsibilities. They also had no additional budget to operationalize PPP activities rather than rely on funding from USAID through Nexant, especially with additional member countries joining the power pool. 3. Importance of Local Presence of the Implementing Contractor: Lack of permanent presence or representation of the contractor within the region during project implementation resulted in some challenges related to communication, information flow, project coordination and alignment for optimal performance. 4. Consistency in Participation in Regional Power Pool Work: Consistency of participants involved in EAPP and other regional power projects is of concern to EAPP and its members. It would help to have dedicated staff from the various member states and organizations and consistent alternates assigned to EAPP-related tasks. PPP End of Project Performance Evaluation Report, February 2013 38 5. Importance of Donor Coordination: There were overlaps between activities undertaken under PPP and funded by other donors to EAC and EAPP. Donor coordination in project activities would have helped in avoiding overlaps and resulted in more effective use of resources. 6. Need for Technical Capacity Enhancement in both EAPP and EAC: In EAC, technical input and comments on draft documents still have room for improvement, as experienced in the draft CBE Policy that benefitted from PPP input. EAPP has similar capacity enhancement needs. Sufficient technical skills and staff are required in these regional bodies for more efficiency. PPP End of Project Performance Evaluation Report, February 2013 39 4. CONCLUSIONS AND RECOMMENDATIONS Conclusions 1. Nexant was largely effective and efficient in implementing PPP: they produced most of the deliverables in time and at no additional cost; trained more staff than planned; provided useful support in bilateral power trade and CBE agreements. A few challenges, however, hindered effective implementation of PPP. These included limited presence of the PPP Nexant staff, which posed communication challenges and affected engagement levels between the contractor, EAPP and beneficiaries. 2. It was noted that different member utilities had different priorities. Consequently, the course content in some of the 2012-2014 capacity building workshops did not match their requirements for the varying levels of participants. 3. There were inadequate donor consultations right from the project design to implementation, which led to overlaps. A coordinated approach would have helped avoid overlaps and duplication in donor-supported projects and programs, and increased effectiveness and efficiency.PPP helped build and strengthen the institutional capacity of EAPP and other institutions as affirmed through: training of at least 103 energy sector staff; development of IDS for EAPP and PTS for the region, and in Nexant’s role in negotiations of Ethiopia￾Kenya power sales agreement and the Uganda-Tanzania cross-border trade agreement; the IDS providing a detailed roadmap for institutional development of EAPP and leading to the EAPP Corporate Plan. It was noted that lack of consistency in regional staff involved in EAPP activities and training impede effective capacity development in the region. 4. The face-to-face approach applied in the regional training workshops organized under PPP had limited effectiveness and coverage due to limited resources required for travel and upkeep, more especially in the absence of cost-sharing arrangements. Alternatives such as in-country trainings, e-learning and video￾conferencing have the potential for wider coverage and more effectiveness. PPP End of Project Performance Evaluation Report, February 2013 40 5. PPP contributed to improving the sustainability of the EAPP’s programs through providing a roadmap for capacity development through the IDS and the subsequent three-year EAPP corporate plan, which if implemented would enable the region to have skilled staff and managers to manage and operate a modern power pool. PPP presented several training sessions and facilitated knowledge transfer and skills development across the member states in the region. Inclusion of a study tour contributed in providing exposure of participants to operations and regulation of a modern power pool, although this was limited to a small number of staff in the region due to the resource limitations and unavailability of some nominated staff. 6. PPP registered immediate positive impacts that were in line with project objectives and the power pool goals. Examples of these include the Power Transmission Standards (PTS) and a Gap Analysis tool, which had immediate impacts in EAPP and COMESA; the IDS, which found immediate application by EAPP; and support in the successfully concluded Ethiopia-Kenya bilateral PSA and the Uganda-Tanzania CBE agreements and in the review of Cross-Border Policy by EAC. 7. The evaluation found evidence that PPP beneficiaries started taking ownership of the outputs from PPP, including promoting regional cooperation and advocating the best practices developed and disseminated under PPP. Most results and impacts of PPP are sustainable in terms of creating institutional capacity and filling gaps in EAPP and members states. However, further assistance by donors and more effort by EAPP are needed to make other outcomes ultimately sustainable, such as training of trainers so as to have a pool of regional experts to help develop local capacity further. 8. PPP was relevant and beneficial to development of capacity for EAPP and its members. Its implementation was largely successful despite the initial implementation challenges that led to modification of the project design. Recommendations 1. In future USAID-supported projects similar to PPP, there will be need to have more presence or representation of the implementing contractor in the region so as to ensure increased engagement with the client and PPP End of Project Performance Evaluation Report, February 2013 41 beneficiaries at country level, maximize information flow and interactions with other donors with similar objectives, and ease communication, project coordination and alignment. 2. In future task-specific regional training, needs assessments should be carried out so as to ensure the course contents are suitable for varying levels of participants from the various countries. 3. There is need to improve donor coordination in the EAPP and EAC region, to avoid overlaps and duplication of effort. EAPP and EAC need to work in close consultation during design and implementation of power projects since EAC membership in now a subset of EAPP. This should be done as the regional power regulatory structures and bodies take shape. In future, there should be more engagement between USAID and these partners right from project inception through design and contracting to implementation. 4. All key stakeholders should be involved or kept informed during project design to avoid implementation challenges and later modifications that result in delays during implementation. This may be achieved by using a combination of enhanced donor engagement and EAPP technical sub-committee members. 5. EAPP member countries should ensure that there is consistency of participants involved in the regional power pool project activities, to maximize effectiveness, impact and sustainability of the gains made in capacity building and regional power market development. To achieve the intended objectives of the power grid standards and grid code harmonization, the Power Transmission Standards and Gap Analysis component of PPP should be rolled out to all member countries. USAID and other donors should collaborate with EAPP, EAC and COMESA in this important project for the realization of the intended benefits and to sustain the gains made through PPP. 6. In future capacity building, there will be need for USAID and other donors to consider cost-sharing and including more in-country trainings of high priority, conducted by a combination of international consultants and regional professionals with relevant hands-on experience in the region. To increase coverage and efficiency, alternative approaches in training such as webinars, e-learning, virtual training, video conferences and in-country training workshops should be considered as they can save on cost of travel and upkeep. There should be follow-up activities in such trainings and projects. Twinning and attachments to pools for hands-on PPP End of Project Performance Evaluation Report, February 2013 42 training would increase effectiveness. The Norwegian Government and SIDA have included in their ongoing technical assistance to EAPP twinning attachments with a European power pool. 7. Taking into account inadequacy of resources and capacity noted during PPP implementation, USAID and other donors should consider providing additional institutional development support for implementation of the IDS and the EAPP Corporate Plan for 2012-2014. This implementation is imperative in steering the region towards the power market and trade envisaged, as EAPP is the focal point. It is also necessary for operationalization of the power pool and facilitation of the envisaged power trade. 8. USAID and other donors should consider supporting concluding future bilateral agreements, wheeling agreements and cross-border electrification projects in the EAPP region, based on the models developed by Nexant under PPP, so as to expedite creation of the desired regional network and a power market. 9. Donors should consider providing strategic support for training of trainers in the region covering such fields as financial modeling, power pool planning and operations, and power transmission standards. This would help make PPP outcomes ultimately sustainable in the power pool. PPP End of Project Performance Evaluation Report, February 2013 43 Annex A: Tasks in the Initial Project Design Task No. Result Task Deliverables A Electricity trading skills enhancement. Increased capacity of government and utility representatives to pursue and negotiate electricity trading agreements.  Organize and facilitate in English and French, a two-day workshop for electric utility and government representatives from each of the nine member countries in close coordination with the EAPP.  Develop a workshop manual on power pools and cross border electricity exchange and trade.  Document workshop proceedings, participation and outcomes in English and French languages and share with Workshop participants and key stakeholders.  Two – day workshop conducted for 30 participants from target countries and others within the region involved in electricity negotiations to take place in Nairobi, Kenya.  Workshop manual with COTR approved curricula.  Workshop Report documenting workshop proceedings, participation and outcomes in English and French languages available for distribution to participants. B Establishing Mechanism for Electricity Trading. No less than three cross border electricity trading agreements facilitated.  Write a draft model of an electricity trade agreement (ETA) for the EAPP and share with the stakeholders.  Using existing power sale agreements as a basis to write at least three bilateral ETAs and share with relevant stakeholders.  Engage the relevant stakeholders to discuss the bilateral ETAs, excluding price schedule, with the aim of having them implemented by the end of the project. Each bilateral discussion will have at least three meetings involving at least three representatives from electricity ministries and utility management responsible for electricity trading. The venue of the meetings will be agreed upon by the negotiating countries/utilities.  Prepare a report on the proceedings and outcomes of the electricity trade agreement negotiations and recommend the best way for implementing the agreements.  A model ETA written and shared with all stakeholders.  At least three bilateral ETAs written and vetted with the respective stakeholders.  A report containing the proceedings and outcomes of the bilateral negotiations and also recommendations on the best methods for implementation of the ETAs. C Development of Power Transmission Regional power transmission standards developed  Review various models of regional transmission standards and develop a written draft regional electricity transmission standard for the regional  A draft document of regional electricity transmission standards to be written with input from and shared with stakeholders. PPP End of Project Performance Evaluation Report, February 2013 44 Task No. Result Task Deliverables Standards. grid code and share with EAPP and its membership.  Engage the EAPP and its membership to discuss the draft transmission standards with the aim of implementation.  A report documenting the development process, input and outcomes on the drafting of regional electricity standards which also outlines recommendations and what additional steps are necessary, if any, to adopt the transmission standards submitted to EAPP and its members. An electronic copy must be submitted to USAID/EAST AFRICA. D Best practices on operation of modern power pool. . Improved skills among EAPP membership, board members and staff for modern power pool operations through knowledge and skills transfer.  Develop a study tour in collaboration with EAPP based on assessed needs to promote skills transfer opportunities and follow‐ up for EAPP members.  In collaboration with EAPP and prospective power pool study sites, design and conduct one study tour for the purpose of skills transfer for no less than 3 staff of EAPP and for an additional 8 people from member countries to visit power pools in Africa and elsewhere i.e. U.S. , Latin America, etc. The contractor will provide all logistical support to participants during study tour and costs must be included in the proposed budget.  Study tour plan developed and list of potential participants developed in collaboration with COTR identified.  Study tour for no less than 11 participants for five days each overseas and for two days in at least one regional African power pool completed. The contractor will provide logistical support to participants during overseas study tour and cost must be included in the proposed budget.  Study tour Report with recommendations for follow‐ on activities produced and copies made available for distribution. Electronic copy of final version delivered to USAID/East Africa. Option 1 Additional Electricity Trading Agreements￾East African Power Pool. Three additional cross border electricity trading agreements facilitated.  Using existing power sale agreements as a basis, where they exist, draft at  Least three electricity trade agreements and share with the stakeholders.  Engage the stakeholders to discuss the draft electricity agreements,  Excluding price schedule, with the aim of implementing them. Each  Bilateral effort will involve at least three meetings with no less than three representatives from ministries and utility management with the venue agreed on by the participating countries.  At least three electricity trading agreements developed and shared with the respective stakeholders.  A written Report on the proceedings of the negotiations between the participants submitted to EAPP and the countries involved with electronic copy submitted to USAID/EAST AFRICA. PPP End of Project Performance Evaluation Report, February 2013 45 Task No. Result Task Deliverables Option 2 Electricity wheeling agreements￾Eastern Africa Power Pool No less than 3 electricity wheeling agreements developed.  A model draft wheeling agreement to be used as reference for bilateral wheeling agreements and share with the key stakeholders.  Choosing from the wheeling agreement schedule agreed by the EAPP, at least three wheeling agreements for specific stakeholders and engages them in discussing them with the aim of implementation.  A written draft model wheeling agreement developed and shared with EAPP and other stakeholders.  At least three bilateral wheeling agreements and engage the negotiating country pairs to discuss and work towards finalizing them. If not finalized, outline status and additional steps that need to be taken to actualize. Option 3 Improving electric utility finances to attract private investors to EAPP Improved customer billing and financial management information systems that increase transparency and improved operational performance.  Prepare a draft regional cost-reflective tariff framework that is supportive of cross‐ border investment in consultation with EAPP, regulators and utilities in the region.  Prepare a model electricity network balance in consultation with EAPP and affiliate utilities in the region.  Develop a guide for regional use on standards/specifications, costing elements and requirements for utility information systems assessment, development, procurement and management. The document should serve as a roadmap for establishing automated, upgradable utility Information management systems that integrate relevant data sources. The data sources could include but are not limited to: customer information systems (CIS) and billing data, interactive voice recognition (IVR) call management data, supervisory control and data acquisition (SCADA), mobile crew management (MCM) data, automatic meter reading (AMR) data, automated vehicle location (AVL) data, engineering analysis data supported by 3rd party software packages (such as load monitoring and balancing) and systems that allow on‐ site engineers to retrieve data (ex. field maps, work orders, codes) or enter, via a portable device, inspection data (ex. such as poles or underground facilities, code violations).  A draft regional cost‐ reflective tariff framework that is supportive of cross‐ border investment written and shared with stakeholders.  A model electricity network balance written and shared with EAPP and respective stakeholders.  At least three utilities identified in collaboration with EAPP, assisted in developing their respective network balance from the draft model.  A model regional roadmap that outlines standards/specifications, costing elements and requirements for phased in utility information systems assessment, development, procurement and management developed.  One utility, identified in collaboration with EAPP, information systems assessment, development, procurement and management plan developed.  SOW: TA for Power Sector Institutions in East Africa 15  A model tender document for purchase of customer information and billing systems or first phase of roadmap.  A resource guide with a list of utility information systems management vendors, product lists, pricing, etc. with PPP End of Project Performance Evaluation Report, February 2013 46 Task No. Result Task Deliverables The guide should contemplate integrating geographic information systems (GIS) data with these data sources. The roadmap should present logical phasing‐ in for implementation based on additions to the system in SOW: TA for Power Sector Institutions in East Africa 14 relation to meeting pre‐ requisite requirements, timing and funds/resource availability.  In collaboration with EAPP, identify one pilot utility that will be provided with technical assistance to develop a tender for the initial phase of an automated information management system to implement a computerized integrated billing information system to improve electricity billing and revenue collection systems.  Identify suite of vendors of computerized integrated billing information systems designed to assist utilities undertake a wide range of activities in the effective management of administrative tasks. Some of these will include, but are not limited to, utility customer accounts registration, and record maintenance. Other features include facilitating management of specific cumulative billing information that ensures that bills are up to date; receipting and auditing facilities that ensures transparency of cash collection; auditing facilities to ensure transparency of financial adjustments and which facilitates the management of basic ledger accounts and transactions; electronic reporting as well Drill Down  Enquiries; facilities exchange of data to and from third party products; and establishment of administrative facilities to ensure integrity of the system. demonstrated success working in Africa and/or other developing regions/countries developed. PPP End of Project Performance Evaluation Report, February 2013 47 Task No. Result Task Deliverables Option 4 Best practices on operation of modern power pool. Improved skills among EAPP membership, board members and staff for modern power pool operations through knowledge and skills transfer.  Develop a study tour in collaboration with EAPP based on assessed needs to promote skills transfer opportunities and follow‐ up for EAPP members.  In collaboration with EAPP and prospective power pool study sites, design and conduct one study tour for the purposes of skills transfer for no less than 6 people from member countries to visit power pools in Africa and elsewhere i.e. U.S. , Latin America, etc. Participant logistical support is to be covered under this contract.  Study tour plan developed and list of potential participants developed in collaboration with COTR identified.  Study tour for no less than 11 participants for five days each overseas and for two days in at least one regional African power pool completed.  SOW: TA for Power Sector Institutions in East Africa 16  Study tour Report with recommendations for follow on activities produced and 5 copies made available for distribution. Electronic copy of final version delivered to USAID/East Africa. Option 5 Technical Assistance to East African Community (EAC): No less than three cross￾border rural electrification protocols and implementation plans developed.  Conduct a participatory process with stakeholders to develop a draft regional policy and regulatory framework for cross border rural electrification.  Engage EAC member states in a process with the aim of implementing the regional cross‐ border rural electrification policy.  A draft regional regulatory framework for cross border rural electrification for the EAC written and shared with stake holders.  Report documenting process to engage EAC members to adopt regional protocol for cross￾border rural electrification prepared. PPP End of Project Performance Evaluation Report, February 2013 48 Annex B: Statement of Work 1.0 GENERAL DESCRIPTION The contract shall be for a Firm Fixed Price (FFP), delivered to the location specified in 4.0 below 2.0 PURPOSE The purpose of this Firm Fixed Price Contract is for procurement of Consultancy Services as stated in the SOW in attachment 1 3.0 PERIOD OF PERFORMANCE The Period of Performance for this FFP Contract is October 1, 2012 through November 31, 2012 4.0 PLACE OF DELIVERY The place of delivery is: USAID/EA P.0.Box 629 Village Market 00621 Nairobi Kenya 5.0 DELIVERABLES & PAYMENT See attachment 1. Submit your cost proposal in MS Excel spreadsheet. 5.1 PAYMENT Payment will be made upon acceptance by the USG at the place identified in 4.0 as follows: a. Initial Payment: 30% Mobilization fee upon delivery and acceptance of Work plan b. Second Payment: 30% of the total Cost Upon Delivery of and USAID Acceptance of the Draft Evaluation Report c. Final Payment: Upon Delivery and USAID acceptance of the Final Evaluation Report: 5.2 NOTICE REGARDING LATE DELIVERY PPP End of Project Performance Evaluation Report, February 2013 49 Late deliveries will result in a price reduction of 1% per day. 6.0 CONTRACT CLAUSES Clauses as required by Federal Acquisition Regulations will be incorporated into the FFP Contract. 7.0 BEST VALUE DETERMINATION The award will be based on a best value determination. PPP End of Project Performance Evaluation Report, February 2013 50 Attachment 1. STATEMENT OF WORK (SOW) Evaluation of USAID/East Africa Technical Assistance Support to the Eastern African Power Pool (EAPP) USAID/East Africa (USAID/EA) is seeking a team of external consultants (hereafter referred to as Consultants) to conduct an end-of-project performance evaluation for the Powering Progress Project (PPP). PPP is funded as a Task Order implemented by Nexant, Inc. under Energy II IQC [AID-EPP-I-OO-03-00007 (AID-623-TO-I0- 00002)]. PPP is composed of core and optional activities with the maximum total funding of $2,481,037. The period of performance was two years with a starting date of April 24, 2010 and ending on April 23, 2012. However, due to the complexity in implementation and the desire to achieve the project objectives, PPP has been extended by an additional three months and will end on July 24, 2012. Background Information on the PPP and it regional implementing partners The purpose of PPP is to improve the capacity of the East Africa Power Pool (EAPP), East African Community (EAC) and their partner states to implement regional electricity trade in clean energy in Eastern Africa. EAPP partner states include Burundi, Djibouti, Democratic Republic of Congo (DRC), Egypt, Ethiopia, Kenya, Libya, Rwanda, Sudan, Tanzania and Uganda (Fig.1) with a combined population of 360 million people. Fig.1: EAPP Countries highlighted in yellow (also in yellow are Somalia and Eritrea, which do not have official EAPP membership) The technical assistance provided under PPP is geared toward supporting the EAPP to fulfill its mission. EAPP’s mission is to make available to the Eastern African region -- by the pooling in a coordinated and optimized EAPP PPP End of Project Performance Evaluation Report, February 2013 51 manner -- affordable, sustainable, and reliable electricity in order to increase the rate of access to electricity in the region while promoting regional integration. PPP is designed to contribute to EAPP’s efforts to improve the policy and regulatory environment for regional trade in electricity so as to encourage increased private sector investment, with a focus on clean energy resources. PPP’s specific objectives are to: 1) develop a legal mechanism for electricity trading among electric utilities in the region through a consultative process; 2) increase the capacity of the EAPP to operate and manage a modern power pool; and 3) increase private sector investment in electricity by addressing specific policy and regulatory barriers. The results to be achieved under PPP are:  Increased capacity of selected government and utility representatives to pursue and negotiate electricity trading agreements  Improved skills among EAPP staff and its member countries to manage and operate a modern power pool  Regional power transmission standards developed and shared with EAPP partner states  At least three bilateral electricity trading and three wheeling agreements facilitated  Development of cross-border protocols for rural electrification within the EAC region The direct beneficiaries of the project are the Common Market for Eastern and Southern Africa (COMESA), EAPP, the East African Community (EAC), the Nile Basin Initiative and electric power utilities, regulators and governments and the people of Burundi, Djibouti, DRC, Egypt, Kenya, Rwanda, Sudan and Tanzania. EAPP was established in 2005 through an intergovernmental memorandum of understanding (MOU) by energy ministers from seven Eastern Africa countries. This MOU was followed by an inter‐utility MOU by the chief executives of the countries’ nine power utilities. Ethiopia, Djibouti, Sudan and Uganda have developed plans to interconnect and strengthen their power systems for electricity sharing and have also joined the EAPP. The EAPP is a special institution of COMESA working on electric power projects within the Eastern Africa Region and operates within the framework of COMESA Energy Policy but has its own Steering Committee and Council of Ministers. EAPP is attracting interest and support from various donors to improve electricity access in the region. Its activities include establishment of a regional electricity market and power pool road map supported by the European Commission (EC), development of Regional Power Master Plan and Grid Code through African Development Bank (AfDB), capacity building for the Power Pool Coordination Centre through a partnership between the Norwegian Agency for Development (NORAD) and the Swedish International Development Agency (SIDA). Purpose and Objective of the Evaluation The purpose of this end-of-project evaluation is to: (1) enable USAID/EA determine the extent to which PPP’s objectives and results have been achieved; and (2) to examine the efficiency and effectiveness of the PPP implementation strategy. PPP End of Project Performance Evaluation Report, February 2013 52 The specific objectives of the evaluation are to:2  Assess the extent to which PPP has been able to build capacity and strengthen the EAPP and other institutions within its member states  Determine Nexant’s effectiveness and efficiency in implementing PPP activities  Assess the PPP’s effectiveness in achieving EAPP’s overall institutional goal and strategic objectives  Evaluate the overall outcome/impacts of the PPP  Assess PPP’s contribution to improving the sustainability of the EAPP’s programs through skilled staff and managers to manage and operate a modern power pool  Document lessons learned for future USAID/EA energy programming The evaluation should include, but not be limited to, the questions summarized in the Evaluation Matrix shown in Table 1. Table 1: Evaluation Matrix Category Issue to be Addressed Impact  What were the major outcomes/impacts of the project on the EAPP and partner states?  What are the unintended impacts?  Which constraints have impeded the full realization of the intended impacts of the project and how did Nexant respond to them?  To the extent possible, measure the increase in the delivery of EAPP services to the partner states. Sustainability  What are the prospects for the sustainability of the end results produced by PPP?  Which of the achieved results appear to be unsustainable, or less sustainable, and why?  Was the scale of the USAID support appropriate to ensure sustainability? Client Satisfaction  Determine if the primary beneficiary (EAPP) and its’ utility members needs were met, if EAPP members’ needs were not met – provide detailed explanation. If not met, what was not met and why? Effectiveness  Was the project implemented and managed in a cost-effective manner?  Was cooperative and coordination among donors effective?  What could have been done to improve the effectiveness of the project? Relevance  How relevant is this project to the short, medium, and long-term development needs of EAPP and its partner states? The Assessment report will document: 1) the outcomes and impact resulting from investments made by USAID, including discussions of PPPs contributions to the larger regional power development and integration program implemented by EAPP (which includes funding from multiple donors); 2) the effectiveness or PPP’s approach to 2 When possible and appropriate, e.g., when data on capacity building beneficiaries, data should be broken down by sex. PPP End of Project Performance Evaluation Report, February 2013 53 strengthening the capacity of EAPP and EAC in achieving their respective objectives in the energy sector; and 3) lessons learned and best practices for future programming. The audience of the evaluation report will be the USAID/EA Mission, specifically the Regional Economic Growth and Integration (REGI) Office, USAID/Washington Africa Bureau, and the implementing partner, Nexant, Inc. The consultant will also provide a shorter, public version of the final evaluation report to EAPP and the EAC. Evaluation Design and Methodology The team contracted by USAID to complete the PPP end-of-project evaluation will undertake an independent analysis, based on the key issues included in this SOW. The approach and methodology of the evaluation must be consistent with USAID’s current evaluation policy; please consult http://www.usaid.gov/evaluation/USAIDEvaluationPolicy.pdf for more detailed information. The methodology for the evaluation shall include a combination of the following: 1. Desk review of documents including initial project scope of work, past evaluations, impact studies, annual reports, field reports, any other relevant documents 2. Interviews and discussions with staff members of the EAPP Secretariat, Nexant Inc. and other key stakeholders including personnel from partnering utilities, USAID staff, private firms, government representatives, NGOs, as well as other development partners While this SOW outlines a number of areas to be covered by the evaluation team, the consultants may propose additional areas of focus that could enhance the quality and utility of the end-of-project evaluation. The proposal should include a plan with details on how the team will evaluate different PPP activities and results -- including proposed site visits to at least three (3) PPP-assisted countries -- as part of the evaluation process. Deliverables The period of performance is expected to be 7-10 weeks; between August and October 2012. Work Plan: The evaluation team will prepare a detailed work plan based on the consultant’s proposal. The work plan should detail the evaluation design/operational work plan which will include details about the methodology proposed for the evaluation. The Work Plan will be submitted to the COR for approval within three days after the contract is signed. Debriefing: The team will present the major findings of the evaluation to USAID/EA and other invited guests using a PowerPoint presentation. The debriefing should be scheduled to occur before the team’s departure from PPP End of Project Performance Evaluation Report, February 2013 54 the region, as per approved work plan. The debriefing will include a discussion of achievements and issues, as well as recommendations. Draft Evaluation Report: A draft report of the findings and recommendations should be submitted to the COR within five business days after the debriefing. The written report should clearly describe findings, conclusions, and recommendations. USAID/EA and partners will provide comments on the draft report within ten working days of submission. The team will consider both USAID/EA and partners comments when drafting the Final Evaluation Report. Final Report: Based on the provisions of the USAID evaluation policy, a formal and final evaluation report shall be presented to the COR. The final report will incorporate the consultant’s responses to Mission comments and suggestions no later than ten days after USAID/EA provides written comments on the team’s draft evaluation report (see above). The report format will include an executive summary, table of contents, list of acronyms, methodology, findings, recommendations and lessons learned. The report will be submitted in English, electronically, and must not be more than 25 pages, excluding annexes. The report will be disseminated within USAID. A brief summary of this report (the public version), not exceeding ten (10) pages and excluding any potentially sensitive information, will be submitted (also electronically, in English) for dissemination among implementing partners and stakeholders. Team Composition The evaluation team will be composed of a Team Leader and two specialists with the option of adding an Administrative Assistant. All team members must have relevant prior experiences in Africa, familiarity with USAID’s objectives, approaches, and operations and prior evaluation/assessment experience. In addition, individual team members should have the technical qualifications identified for their respective positions (see Table 2). The Team leader is responsible for the overall management of the evaluation team and ultimately the final products. The Team Leader should be an established expert in evaluation, with a documented track record. He or she should be familiar with energy issues in the region and have substantial experience working with institutional development in developing countries, particularly in Africa. The specific duties and qualifications are described in the Table 2 below. In addition, the team leader is responsible for:  Coordinating evaluation activities and ensuring the production and completion of an evaluation report in conformance with this scope of work and timelines.  Ensuring high quality analysis, professional quality writing in the English language, technical integration, and contextual readability in the findings and recommendations. The evaluation team shall also include two specialists, and may include some hours of technical or logistical support by a Technical Assistant. The two specialists should be professionals from the African continent, who have direct experience on energy issues in Eastern and/or Southern Africa. They should be familiar with technical and policy issues related to PPP End of Project Performance Evaluation Report, February 2013 55 regional power trade such as interconnections, transmission, tariffs, and policy & regulatory issues among others. The specific duties and qualifications are described in Table 2 below. An optional fourth member can be included as an Administrative Assistant, to assist with research, desk review, interviews, and logistics/planning to assist in the preparation of the draft and final report. The specific duties and qualifications are described in Table 2. Table 2: Evaluation Team Level of Effort Requirements Position Expertise Experience Minimum qualification Responsibilities Team Leader Should have substantial and demonstrated expertise in evaluation techniques involving projects providing technical assistance, capacity development, training, financial management, and partnership components  At least 7-10 years of relevant prior experience with Monitoring and Evaluation activities Significant experience in Africa would be highly advantageous Minimum of MSc., or related degree in one of the following fields: economics, energy, engineering, environment, development, with excellent mastery of written English Lead person for the evaluation ensures coordination, quality control and timely completion of all deliverables Energy Specialist Should possess demonstrated energy related experience on; technical issues, including to power generation, transmission, and system planning  At least 7 years of experience in power development, system planning with some prior experience in the field of Monitoring and Evaluation in Eastern Africa Minimum of MSc or similar degree in one of the following fields: engineering, technology, or planning Support the Team Leader on analyses related to power generation, transmission and system planning Policy and Regulatory Specialist Possess demonstrated experience on energy￾related policy, regulatory and institutional development issues including energy markets, tariffs, regulations, and regional energy trade  At least 5 years of experience in energy market development, policy and regulatory analysis with at least 2 years’ experience in field of Monitoring and Evaluation in Eastern Africa MSc or similar degree in one of the following fields: law, international development, economics, sociology, or planning Support the Team Leader on the analyses related to policy, regulatory and institutional issues Administrative Possess demonstrated  At least 4 years of At least a BSc or BA in Support PPP End of Project Performance Evaluation Report, February 2013 56 Evaluation Management: USAID/EA, through the COR, will provide overall direction to the evaluation team in the course of implementation. The evaluation team will be responsible for arranging meetings with key stakeholders identified by USAID, prior to the initiation of field work as per the approved work plan. Any international travel proposed by the evaluation team, in addition to the travel included in the approved work plan, must be approved in advance by the COR. The evaluation team is also responsible for arranging all travel and up-front payments. USAID/EA personnel will be made available to the team for consultations regarding information sources and technical issues, before and during the evaluation process. The Contractor must respond to the following timelines in Table 3 below: Table 3: Deliverable Schedule Task Delivery schedule 1 Work Plan Within 5 days after contract signature 2 Debriefing After completion of field work, but prior to departure from the region, as per the approved work plan 3 Draft Evaluation Report Within 5 days after the debriefing as per the approved work plan 4 Final Evaluation Report (both Public and internal) Within 10 days after receipt of USAID’s comments on the Draft Evaluation Report EVALUATION CRITERIA Proposals will be evaluated based upon the degree to which the overall proposal successfully addresses the Statement of Work according to the following criteria: A. Technical Approach (40 points): Assistant skills in providing technical and logistical support to a team of experts experience analyzing development issues, writing technical briefs, and supporting a project team one of the following fields: international development, economics, environment, engineering, or planning, or another relevant field information collection, report writing, and logistics for the Evaluation PPP End of Project Performance Evaluation Report, February 2013 57 i. The proposal shows a solid understanding of the regional context – including both the development challenges and the practical working environment. ii. The proposal contains a clear and reasonable description of how the evaluation will be implemented. iii. The proposal reflects a suitable methodology for the collection and analysis of data on which the evaluation will be based. iv. The proposal provides an explanation of how the implementer will coordinate with the team implementing the end of Project review of the Powering Progress Project. v. The clarity, readability and ease of following the proposal along the evaluation criteria in the RFQ. B. Proposed Consultants and Past Performance (60 points) i. The Team Leader must have substantial and demonstrated expertise in evaluation techniques involving projects providing technical assistance, capacity development, training, financial management, and partnership components. ii. The Team Leader should also have at least 7-10 years of demonstrated relevant prior experience in Monitoring and Evaluation activities in the productive sectors (energy, infrastructure, agriculture). Significant experience in Africa would be highly advantageous. iii. Other technical staff must meet the minimum criteria as stated in Table 2: Evaluation Team Level of Effort Requirements of the SOW. iv. Demonstrated leadership, written and verbal communications skills in English. v. The bidder has successfully managed evaluations of similar nature, size, and complexity. vi. Demonstrated capacity and relevant experience in the design and implementation of evaluation methodologies appropriate to this task. C. Cost Proposal PPP End of Project Performance Evaluation Report, February 2013 58 The Contracting Officer will conduct a cost analysis and evaluate costs based on the proposal’s realism, reasonableness, and completeness and whether the cost elements are in line with the technical proposal provided. Source selection for the award of the Purchase Order resulting from this solicitation will be made to the lowest priced, technically acceptable proposal. Sewa Powering Progress Project Work Plan November, 2012 1 Annex C: List of Documents Reviewed 1. The PPP Scope of Work; 2. Nexant’s contract/contract modifications; 3. Nexant’s detailed work plan; 4. Nexant’s performance management plan; 5. Nexant’s progress reports, study tour reports, workshop reports and the final progress report; 6. Project outputs including wheeling agreements, electricity trading agreements, gap analysis tool, grid codes developed, training materials and EAPP’s institutional development strategy; 7. Rapid review of other development partners project documents including the Norwegian Project; European Commission (EC) project documents, French Development Corporation (AfD) and African Development Bank (ADB); 8. Review of other power pools’ websites including the South African Power Pool and West African Power Pool. PPP End of Project Performance Evaluation Report, February 2013 2 Annex D: Work plan 1 Background This work plan has been prepared based on a directive from the USAID Contracting Officer’s Representative (COR). The work plan reflects the contractual scope of work for the end of project performance evaluation for the Powering Progress Project (PPP). The PPP project ran from April 24, 2010 to April 23, 2012 with an extension to July 24, 2012. PPP was funded as a Task Order implemented by Nexant, Inc. under Energy II IQC [AID-EPP-I-OO-03-00007 (AID-623-TO-I0-00002)]. The PPP was intended to assist key entities involved in developing the enabling environment for the East African electricity market by providing technical assistance in critical areas to:  Develop model bilateral Electricity Trade Agreements (ETAs) and regional Power Transmission Standards for the Eastern Africa Power Pool (EAPP) member countries.  Build capacity to exploit clean and renewable energy resources, harmonize regional policies and regulations for improved cross-border trade, and improve the technical and financial performance of EAPP member utilities. 2 Evaluation Objectives The objective of this assignment is to conduct an end-of-project performance evaluation. We will be seeking to:-  Assess the extent to which PPP has been able to build capacity and strengthen the EAPP and other institutions within member states;  Determine Nexant’s effectiveness and efficiency in implementing PPP activities;  Assess the PPP’s effectiveness in achieving EAPP’s overall institutional goal and strategic objectives;  Evaluate the overall outcome/impact of the PPP;  Assess PPP’s contribution to improving the sustainability of the EAPP’s programs through skilled staff and managers to manage and operate a modern power pool; and  Document lessons learned for future USAID/EA energy programming. 3 Work Plan 3.1 Preparation Phase 3.1.1 Contracting and Mobilization 1 This will involve contracting and financial mobilization and introduction to the USAID East Africa team, discussions on guidelines for USAID evaluation reporting (style and format) and gather project background information including but not limited to the design and implementation documentation from USAID, work plan and periodic reports, Memorandum of understanding between EAPP and USAID, workshop reports and press releases submitted to USAID by Nexant. Other documents to be reviewed will include bilateral electricity trading agreements and wheeling agreements, EAPP’s strategic plan, the power transmission standards developed, regulatory regulations developed in line PPP End of Project Performance Evaluation Report, February 2013 3 with each donors requirements, staff training materials, cross border protocols for rural electrification within the East African Community (EAC) region, study tour reports, past evaluations, impact studies, training reports and any other relevant documents. Schedule This activity will be completed by December 4, 2012. Deliverables Work plan. 3.1.2 Literature review We will conduct an in-depth literature review of the following documents: 9. The PPP Scope of Work; 10. Nexant’s detailed work plan; 11. Nexant’s performance management plan; 12. Nexant’s progress reports, study tour reports, workshop reports and the final progress report; 13. Project outputs including wheeling agreements, electricity trading agreements, gap analysis tool, grid codes developed, training materials and EAPP’s institutional development strategy; 14. Rapid review of other development partners project documents including the Norwegian Project; European Commission (EC) project documents, French Development Corporation (AfD) and African Development Bank (ADB); 15. Review of other power pools websites including the South African Power Pool and West African Power Pool. Schedule This activity will be completed by December 4, 2012. 3.1.3 Identification of key informants and development of interview guides Interview guides will be developed, key informants identified and interview schedules prepared for those to be visited or contacted through telephone and email for those who will not be met including Nexant Inc, East African Community, Common Market for Eastern and Southern Africa (COMESA) and other participating countries that will not be visited . The interview guides will be designed to determine the relevance, sustainability, client satisfaction, effectiveness and impact of the project as per the objective of the evaluation. Schedule This activity will be completed by December 4, 2012. 3.1.4 Primary data collection PPP End of Project Performance Evaluation Report, February 2013 4 We will visit Ethiopia, Rwanda and Kenya and conduct telephone interviews for those who will not be visited. a. Ethiopia field visit We will contact and interview the following key informants: 1. EAPP Secretariat; 2. Ethiopia Electric Power Corporation (EEPCo); 3. EAPP member states representatives; 4. Ethiopia Ministry of Water and Energy 5. Key development partners in the country including the Norwegian Agency for Development (NORAD), European Commission (EC), French Development Corporation and African Development Bank. b. Kenya field visits We will contact and interview the following key informants: 1. Kenya Power (KP); 2. Kenya Electricity Generating Company (KenGen); 3. Electricity Regulatory Commission (ERC); 4. Kenya Electricity Transmission Company (KETRACO); 5. Rural Electrification Authority (REA); 6. Electricity Regulatory Commission; and 7. Ministry of Energy. c. Rwanda field visits We will contact and interview the following key informants: 1. Energy, Water and Sanitation Authority (EWSA); 2. Rwanda Utilities Regulatory Agency (RURA); 3. Nile Equatorial Lakes Subsidiary Action Programme (NELSAP); 4. Ministry of Energy and Water. d. Interview with the implementing partner Nexant Inc. e. Interviews with Regional Economic Communities and other Regional Entities including: 1. East African Community (EAC) 2. Common Market for Eastern and Southern Africa (COMESA) f. Interviews with USAID/EA Program Manager and other persons associated with the project. Schedule PPP End of Project Performance Evaluation Report, February 2013 5 This activity will be completed by December 31, 2012. 3.1.5 Data and information analysis All data and information collected will be analyzed and harmonized to generate the debriefing presentation and first draft report. This being a qualitative study, the data gathered will be analyzed through identifying common ideas and patterns that we will observe then interpreting the data by attaching significance to the themes and patterns observed. Schedule This activity will be completed by January 14, 2013. 3.1.6 Debriefing on the Draft Report A debriefing session will be held for USAID/EA and other invited guests. This session will seek to present the findings as well as discuss any issues arising and recommendations for similar initiatives going forward. Following this session, the draft report incorporating any revisions arising from the debriefing session will be sent out to USAID/EA and partners for comments for the finalization of the report. Schedule Debriefing of the USAID/EA team is planned for January 15, 2013 and the draft report is expected to be completed by January 20, 2013. Deliverable Draft report 3.1.7 Final Reporting Comments and suggestions from USAID/EA and partners will be used to develop the final report, not exceeding 25 pages excluding annexes. In drafting the final report we will identify action oriented practical and specific recommendations. A brief summary of this report, not exceeding 10 pages and excluding any potentially sensitive information, will also be submitted for dissemination among implementing partners and stakeholders. Schedule This activity will be completed by February 5, 2013. Deliverable Final PPP End of Project Evaluation Report PPP End of Project Performance Evaluation Report, February 2013 6 4 Overall Schedule The overall schedule is described in Figures 4-1 Figure 4-1: Overall Schedule Activity Week One Week Two Week Three Week Four Week Five Week Six Week Seven Week Eight Week Nine Week Ten Preparation Phase Contracting Work plan preparation Work plan approval Introduction of teams (Kick off meeting) at USAID offices Provision of pre requisite project documentation and contacts Literature review, research design, development and testing data collection tools Interviews with USAID/EA Interviews with Ethiopia based beneficiary organizations Interviews with Kenyan based beneficiary organizations Interviews with Rwanda based beneficiary organizations Data Analysis and Draft Reporting Debriefing Meeting Draft report preparation Comments from USAID Final Report Preparation PPP End of Project Performance Evaluation Report, February 2013 7 Annex E: Eastern Africa Regional Power Projects Existing Interconnectors The existing power interconnections in the region include: 1) DRC, Burundi, and Rwanda interconnected from a jointly developed hydro power station Ruzizi II, (capacity 45 MW) operated by a joint utility, SINELAC; 2) Kenya – Uganda interconnection; 3) Ethiopia and Djibouti 4) Cross-border electrification between Uganda and Rwanda, Tanzania and Uganda, Kenya and Tanzania, and Ethiopia and Kenya; 5) Ethiopia and Sudan interconnection 6) Egyptian power system interconnection through Libya to other Maghreb countries and Southern Europe; and through Jordan to Eastern Mediterranean. Ongoing regional power interconnection The projects include: 1) Tanzania- Kenya (400 kV, 2 circuits, 260 km) 2) Rusumo- Rwanda (220 kV, 1 circuit, 115 km) 3) Rusumo - Burundi with the grids of Tanzania, Rwanda and Burundi (220 kV, 1 circuit, 158 Km) 4) Rusumo - Tanzania (220 kV, 1 circuit, 98 km) 5) Ethiopia - Kenya (500 kV HVDC, bipole, 1120 km) Several potential generation projects have been identified for development as regional projects for power trade in the envisaged regional power market. They include hydropower projects in DRC, Ethiopia, Tanzania and Uganda, and two methane power projects at Lake Kivu in Rwanda. Kenya’s geothermal projects also have potential to provide surplus renewable energy for regional power trade. The projects identified in the Eastern Africa Power Pool Power Master Plan (EAPP PMP), to be commissioned in the medium term are listed below. Generation Projects Country Plant Name Type Installed Capacity (MW) Eastern DRC Ruzizi III Hydro 145 Ruzizi IV Hydro 287 Ethiopia Mandaya Hydro 2000 Gibe III Hydro 1870 Border Hydro 1200 Gibe IV Hydro 1468 Karadobi Hydro 1600 PPP End of Project Performance Evaluation Report, February 2013 8 Rwanda Kivu I Methane 100 Kivu II Methane 200 Tanzania Stiegler’s Gorge (I, II, III) Hydro 1200 Uganda Karuma Hydro 700 Ayago Hydro 550 Murchison Falls Hydro 750 PPP End of Project Performance Evaluation Report, February 2013 9 Annex F: Evaluation Methodology Evaluation Design Approach The evaluation team consisted of Anthony Getambu, Phd (Team Leader/Monitoring and Evaluation Specialist), Boniface Kinyanjui (Energy Specialist) and David Mwangi (Policy Specialist) supported by Josephine Ngethe. The evaluation team designed the evaluation methodology consistent with the current USAID evaluation policy. The evaluation used a mixed methods approach. This approach offers a high level of rigor and empirical validity to support evaluation analysis and conclusions. The rationale behind the methodology was that in order to assess the fundamental viability of PPP scale up, it was necessary to determine if PPP activities had been relevant to support the regional power pool objectives, determine how effectively those activities had been implemented, assess client satisfaction, assess the extent to which the activities have made an impact, document challenges and lessons learnt and make recommendations for future programs. This evaluation required the following steps: Literature Review Literature Review: A number of documents relevant to PPP were provided by USAID/EA for review on commencement of this evaluation. These include PPP final report, progress reports from the start of the project, the project contract document, workshop reports, the PPP Performance Management Plan, EAPP Corporate Plan, the Institutional Development Strategy (IDS) and Power Transmission Standards (PTS) and other deliverables from specific PPP tasks. A complete list of documents and reports received is contained in Appendix B of this report. Other EAPP documents were also reviewed in addition to project documents from other activities funded by other donors who included AfDB, European Commission and the Norwegian MFA. The reports provided useful information for developing timelines, the work plan, and understanding the perspective and evolution of the projects over the implementation duration. During the data gathering field work covering three countries, Ethiopia, Kenya and Rwanda, the team obtained useful information and other relevant project documents from EAPP. The PPP Final Report by Nexant was received in mid December 2012. The final report provided more comprehensive and sequenced information on the project. The team also conducted an informal review of functional power pools to understand dynamics of power pools development. A complete list of documents and reports received is contained in Appendix B of this report. Key Informant Interviews Identification of key informants, sites and participants: The countries to be visited were selected purposefully. Ethiopia hosts the EAPP head office and was involved in the PTS pilot project and the ETA for the interconnector with Kenya, which was part of the projects envisaged for support through PPP. Kenya has the largest representation of power utility organizations, is a member of EAC, participated in the pilot PTS project, and was involved in the interconnector with Ethiopia; and Rwanda is a potential key beneficiary of the interventions. Within each country, the team aimed to interview key informants from across the participating organizations in order to obtain a representative view of each. Primary Data Collection PPP End of Project Performance Evaluation Report, February 2013 10 The team visited Kenya, Ethiopia and Rwanda, and interviewed staff from USAID/EA, EAPP, Nexant Inc, and PPP participants/beneficiaries from Kenya, Ethiopia, Rwanda, Tanzania and Uganda. In addition, the team interviewed staff dealing with energy and EAPP issues from East African Community (EAC), Common Market for Eastern and Southern Africa (COMESA), Rwanda Development Board (RDB), other donors/projects supporting EAPP and two private sector organizations in Kenya – the Kenya Private Sector Alliance (KEPSA) and the Kenya Association of Manufacturers (KAM). In this regard, the team interviewed an Energy Specialist of RDB, an Energy Economist of COMESA, the Chairman of KEPSA and the CEO of KAM. At the end of each of the visits to Ethiopia and Rwanda, a briefing of USAID mission in the country was done. The interview guides developed were used to conduct the semi-structured interviews and lead in the discussions with the key informants. These illustrative interview guides are provided in Appendix D. A complete list of the key informants interviewed is provided in Appendix E. A total of 28 respondents were interviewed over the three weeks with face-to-face interviews with informants from Kenya, Ethiopia, Rwanda and EAC. Other respondents from EAPP member countries, COMESA and Nexant Inc. staff members who were part of the implementation team were interviewed through telephone calls and emails. PPP End of Project Performance Evaluation Report, February 2013 11 Annex G: INTERVIEW GUIDES EVALUATION QUESTIONS FOR EAPP Relevance  As EAPP, how were you involved in the design of this project?  What were the main needs of EAPP at the time of the project design?  Was the design of this project relevant to the short, medium and long term development needs of EAPP and its partner states?  To what extent did this project respond to the identified needs of EAPP?  To what extent did this project contribute to the goal for regional power in the: o Short term? o Near term? o Long term?  What were the main shortcomings of the design of this project?  What do you think should be done to ensure relevance of future programs to your needs at the design stage? Performance assessment  What were the major achievements of the project?  Did the project achieve all its planned activities in time? Probe further, if yes what conditions helped in this achievement and if no, what problems hindered the project from full achievement of its objectives?  In your opinion as EAPP, which activity of the project was most successful and why?  In your opinion as EAPP, which activity of the project was least successful and why?  As EAPP how were you involved in the implementation of the project?  As EAPP what do you think should have been done to ensure your full participation in the project implementation process?  How were the member countries involved in program implementation?  What criterion was used to select participants to attend various trainings?  Was gender considered in selection of participants for skills development and capacity building under the PPP?  In your opinion, what should have been done to ensure effectiveness and efficiency in achievement of the project objectives?  Client Satisfaction  Were the needs of EAPP and its utility members met as envisaged in the PPP? If their needs were not met, what was not met and why?  How would you rate the Powering Progress Project overall?  How was the training? a. Content PPP End of Project Performance Evaluation Report, February 2013 12 b. Organization c. Appropriateness of training methodology d. Training materials  Overall, to what extent was this project useful to your organization? (not useful 1, very useful 5)  To what extent do you think you can apply the information/knowledge acquired from the project to your work and the envisaged regional power trade? (least extent 1, great extent 5)  What was most useful about the project?  What was least useful about this project?  What were the weaknesses of this project?  What are your general comments and suggestions for improving such projects? Effectiveness  In your opinion do you think the project activities were implemented according to the original plan? Probe if no why not? And if what contributed to this?  Was the management of the project responsive to the concerns of the stakeholders involved?  In your view, was cooperation and coordination among players in PPP effective?  Do you have any observations about the experience, qualifications, and effectiveness of the PPP contractor?  In your own assessment was the USAID assistance appropriate and relevant in the Eastern Africa context?  What do you think were the main constraint to effective implementation of project activities?  What could have been done to improve the effectiveness of the project? Impact  What are the major outcomes/impacts of the project on EAPP and its members?  Which new/improved services has EAPP been able to provide to partner states as a result of the PPP project?  Do you think the capacity developed through PPP will help your organization carry out its functions/work more effectively?  Which constraints impeded the full realization of the intended outcome of the project and how did the contractor respond to them?  What did you gain from this project that was unexpected?  To what extent has the service delivery capacity of EAPP to member states increased as a result of PPP? (least extent 1, great extent 5) Sustainability  What are the prospects for the sustainability of the end results of PPP?  Which of the achieved results appear to be unsustainable, or less sustainable, and why?  Was the scale of USAID support appropriate to ensure sustainability? PPP End of Project Performance Evaluation Report, February 2013 13  Are the results and impact of PPP activities sustainable in terms of creating institutional capacity and filling gaps on behalf of the program’s key beneficiaries?  What evidence has there been of member countries and utilities taking ownership of the PPP including promoting the networks and forums and advocating the best practices developed and disseminated under PPP?  Based on results to date, are these activities likely to engender sustainable development impacts after USAID funding has stopped?  Do you have any lessons that were learnt from the PPP project with regard to the following:  Project implementation team?  USAID and implementing partners?  Future project? EVALUATION QUESTIONS FOR UTILITIES/MINISTRIES/REGULATORS Relevance  As an EAPP member organization how were you involved in the design of this project?  What were the main needs of your organization in the regional power context at the time of the project design?  Was the design of this project relevant to the short, medium and long term development needs of EAPP and its partner states?  To what extent did this project respond to the identified needs of your organization?  To what extent did this project contribute to the goal for regional power in the: o Short term? o Near term? o Long term?  What were the main shortcomings of the design of this project?  What do you think should be done to ensure relevance of future programs to your needs at the design stage? Performance assessment  What were the major achievements of the project?  In your opinion as a beneficiary, which activity of the project was most successful and why?  In your opinion as a beneficiary, which activity of the project was least successful and why?  As a beneficiary how were you involved in the implementation of the project?  As a beneficiary what do you think should have been done to ensure your full participation in the project implementation process?  How were you involved in program implementation?  What criterion was used to select participants to attend various trainings? PPP End of Project Performance Evaluation Report, February 2013 14  Was gender considered in selection of participants for skills development and capacity building under the PPP?  In your opinion what should have been done to ensure effectiveness and efficiency in achievement of the project objectives?  Client Satisfaction  Were your needs met as envisaged in the PPP? If they were not met, what was not met and why?  How would you rate the Powering Progress Project overall  How was the training? a. Content b. Organization c. Appropriateness of training methodology d. Training materials  Overall, to what extent was this project useful to your organization? (not useful 1, very useful 5)  To what extent do you think you can apply the information/knowledge acquired from the project to your work and the envisaged regional power trade? (least extent 1, great extent 5)  What was most useful about the project?  What was least useful about this project?  What were the weaknesses of this project?  What are your general comments and suggestions for improving such projects? Effectiveness  Do you think the project was implemented and managed in a cost-effective manner?  In your opinion do you think the project activities were implemented according to the original plan? Probe if no why not? And if yes what contributed to this?  Was the management of the project responsive to the concerns of the stakeholders involved?  In your view, was cooperation and coordination among players in PPP effective?  Do you have any observations about the experience, qualifications, and effectiveness of the PPP contractor?  In your own assessment was the USAID assistance appropriate and relevant in the Eastern Africa context?  What do you think were the main constraint to effective implementation of project activities?  What could have been done to improve the effectiveness of the project? Impact  What are the major outcomes/impacts of the project on EAPP and its members?  Which new/improved services has EAPP been able to provide to you and your organization as a result of the PPP project?  Do you think the capacity developed through PPP will help your organization carry out its functions/work more effectively? PPP End of Project Performance Evaluation Report, February 2013 15  Which constraints impeded the full realization of the intended outcome of the project and how did the contractor respond to them?  What did you gain from this project that was unexpected?  To what extent has the service delivery capacity of EAPP to member states increased as a result of PPP? Sustainability  What are the prospects for the sustainability of the end results of PPP?  Which of the achieved results appear to be unsustainable, or less sustainable, and why?  Was the scale of USAID support appropriate to ensure sustainability?  Are the results and impact of PPP activities sustainable in terms of creating institutional capacity and filling gaps on behalf of the program’s key beneficiaries?  What evidence has there been of member countries and utilities taking ownership of the PPP including promoting the networks and forums and advocating the best practices developed and disseminated under PPP?  Based on results to date, are these activities likely to engender sustainable development impacts after USAID funding has stopped?  Do you have any lessons that were learnt from the PPP project with regard to the following:  Project implementation team?  USAID and implementing partners?  Future project? EVALUATION QUESTIONS FOR NEXANT INC  How relevant is this project for the short, medium and long term development needs of EAPP and its member states?  Are there any design gaps realized during the project implementation?  What necessitated the project extension?  How did you collaborate with other donors working with EAPP?  In your assessment how well have implementing partners worked as a team to coordinate activities in the interest of achieving the project objectives?  What could have been done to improve the effectiveness of the project?  What are the prospects for the sustainability of the end results of PPP?  In your view are the results and impact of PPP activities sustainable in terms of creating institutional capacity and filling gaps on behalf of the program’s key beneficiaries?  Is there evidence that best practices have been taken up by additional individuals who received information from targeted beneficiaries?  To what extent did this project meet the goal for regional power trade achievable in the: o Short term? o Medium term? PPP End of Project Performance Evaluation Report, February 2013 16 o Long term?  To what extent have the outputs from technical assistance and training been utilized by target beneficiaries.  To what extent have the PPP activities supported or complemented activities sponsored by other donor partners?  Do you believe the project achieved its objectives? Performance assessment  To what extent were the project activities identified in the performance management plan completed? Specifically,  Did the project support in the development of a strategy?  Has the project resulted in facilitation and signing of cross border electricity trading agreements? How many have been signed?  Has the project resulted in development of regional power transmission standards?  Have the skills of the EAPP membership, board members and staff to manage a modern power pool improved?  Did the project support in the development of a regional regulatory tariff methodology?  How many cross-border rural electrification protocols and implementation plans have been developed?  Do you have any lessons that were learnt from the PPP project with regard to the following:  Project implementation team?  USAID and implementing partners?  Future project? EVALUATION QUESTIONS FOR COMESA  One of the outputs of the powering progress project is the IC and the PTS. Has the COMESA Council of Ministers finally adopted the IC and PTS for application throughout the Eastern and Southern Africa?  How have COMESA member countries embraced the standards? Are they already adopted by the member countries?  In your assessment, does EAPP secretariat have the capacity to drive the regional power agenda?  Are there opportunities that EAPP can maximize on?  There are very many development partners wishing to support the energy sector and are already supporting EAPP. In your view, has the support to EAPP made a difference?  Did you observe any gaps or overlaps among the various development partners supporting your organization during the project implementation?  There are very many development partners wishing to support the energy sector and are already supporting EAPP. In your view, has the support to EAPP made a difference?  In which trainings were you involved in under the powering progress project? PPP End of Project Performance Evaluation Report, February 2013 17  What skills, knowledge and capacity have you acquired/enhanced under the PPP?  Which constraints may have impeded the full realization of the objectives PPP?  Are there any other key areas that you think should have been covered?  Which constraints may have impeded the full realization of the objectives PPP?  What are the prospects for the sustainability skills, knowledge and capacity acquired under the project?  How would you assess the training materials in terms of: a. Content b. Organization c. Appropriateness of delivery approach used d. Training materials  How effective do you view the USAID contractor’s work of implementing the USAID assistance? Can you provide comments about the experience, qualifications, and effectiveness of the contractor team?  What could have been done to improve the effectiveness of the project?  In your assessment is there a challenge promoting a regional energy agenda?  What interventions can be undertaken to promote the regional agenda?  Provide any lessons that were learnt from the PPP project with regard to the following - project implementation team - future project - USAID and implementing partners. EVALUATION QUESTIONS FOR EAC  One of the outputs of the powering progress project is the Rural electrification protocol.  Have EAC countries adopted it?  In your assessment, does EAPP secretariat have the capacity to drive the regional power agenda?  Are there opportunities that EAPP can maximize on?  There are very many development partners wishing to support the energy sector and are already supporting EAPP. In your view, has the support to EAPP made a difference?  In which trainings were you involved in under the powering progress project?  What skills, knowledge and capacity have you acquired/enhanced under the PPP?  Which constraints may have impeded the full realization of the objectives PPP?  Are there any other key areas that you think should have been covered?  Which constraints may have impeded the full realization of the objectives PPP?  What are the prospects for the sustainability skills, knowledge and capacity acquired under the project?  How would you assess the training materials in terms of: a. Content PPP End of Project Performance Evaluation Report, February 2013 18 b. Organization c. Appropriateness of delivery approach used d. Training materials  How effective do you view the USAID contractor’s work of implementing the USAID assistance? Can you provide comments about the experience, qualifications, and effectiveness of the contractor team?  What could have been done to improve the effectiveness of the project?  In your assessment is there a challenge promoting a regional energy agenda?  What interventions can be undertaken to promote the regional agenda?  Provide any lessons that were learnt from the PPP project with regard to the following - project implementation team - future project - USAID and implementing partners. PPP End of Project Performance Evaluation Report, February 2013 19 Annex H: SWOT Analysis of PPP The SWOT analysis below was carried out on PPP and its implementation environment based on the overall findings, analysis, insights from literature reviews and feedback obtained from stakeholders during interviews. The results are summarized below. Findings STRENGTHS  Strong relevance of PPP in that power interconnectors and generation projects as well as the regional market are being developed.  Good political support and climate in the EAPP member countries.  Strong willingness by USAID and other donors to support development of EAPP and exploitation of renewable energy resources in the region.  Existence of a good number of professional staff in the region involved in EAPP activities, with capacity to drive regional power market forward given the necessary support and continued capacity building.  The donor resources are being directed to a region with abundant renewable energy in the Eastern Africa region which can be harnessed in an optimal manner capable of growing the regional power market.  COMESA linkage with EAPP enabling wider adoption of PTS and grid code standards. WEAKNESSES  Assumptions made during project design which later necessitated variation of project design, deliverables and implementation period.  Limited consultations at project inception and design between beneficiaries and donors.  Inadequate presence of implementing contractor within the region resulting in poor communication due to time difference between the region and USA, and little engagement between PPP contractor and project implementing partners/beneficiaries.  Lack of donor coordination resulting in overlaps and possibly duplication of effort, in feasibility studies and capacity building/development initiatives.  Lack of consistency and continuity in the staff of EAPP member utilities that attend to EAPP work. OPPORTUNITIES  Project can be scaled up for full realization of intended benefits.  Donors’ continued interest and goodwill in support of EAPP and the power sector in the region.  Available global support for development of renewable energy projects in developing countries  PPP support able to work in harmony with and complement other similar ongoing relevant capacity enhancement projects in the region supported by different development partners.  Capacity available at COMESA to assist. THREATS  Lengthy negotiations for power trade and wheeling agreements as well as on shared resources for cross-border projects. PPP End of Project Performance Evaluation Report, February 2013 20 Annex I: List of Key Informants Interviewed Name Organization/Country Title Telephone Email 1. Eng. Jasper Oduor EAPP/Ethiopia Executive Secretary, +251 913968145 joduor@eappool.org 2. Ephraim Tesfaye EAPP /Ethiopia Economist +251 922750274 etesfaye@eappool.org 3. Patrice Manirakiza EAPP/Ethiopia Power Engineer +251 921424319 pmanirakiza@eappool.org 4. Ms. Hoda Ismael EAPP/Ethiopia IRB Legal Expert +251 922106048 hismael@eappool.org 5. Zelalem Gabrehowot EAPP/Ethiopia Technical Assistant +251 921943117 zgebrehiwot@eappool.org 6. Sheriff Ewiss EAPP/Ethiopia Finance & Admin Assistant +251 911023005 sewiss@eappool.org 7. Joseph Magochi EAPP/Ethiopia IT Expert +251 912608276 jmagochi@eappool.org 8. Mekuria-Lema EEPCO /Ethiopia Corporate Planning Chief Officer +251 911833289 esplanning469@gmail.com 9. Katrine Vestbostad Royal Norwegian Embassy/Ethiopia Counselor / Climate Change, Environment and Clean Energy +251 113710799 katrine.vestbostad@mfa.no 10. Lebby Changulla TANESCO/Tanzania Planning Manager lebbi.changullah@tanesco.co.tz 11. Hancox Jaoko, Ph.D USAID/EA/Kenya Energy Specialist and Regional Programs Manager Economic Growth Integration +254 20 8622500 hjaoko@usaid.gov PPP End of Project Performance Evaluation Report, February 2013 21 Name Organization/Country Title Telephone Email 12. Mr. Godfrey Kariuki KETRACO/Kenya Head of Planning, +254 20 4956000 gkariuki@ketraco.co.ke 13. Patrick Mawala KPLC/Kenya Energy Purchase and System Control Manager +254 20 3201000 pmawala@kplc.co.ke 14. Erastus Kiruja Kenya Power/Kenya Chief Engineer, System Studies +254 20 3201485 ekiruja@kplc.co.ke 15. Buge Wasioya ERC/Kenya Senior Manager, Power Systems +254 20 2847000 buge.wasioya@erc.go.ke 16. Eng. Joe Nganga ERC/Kenya Director for Electricity +254 20 2847000 joe.nganga@erc.go.ke 17. Timothy Mwenda KETRACO/Kenya Electrical Engineer, +254 20 4956000 tmurithi@ketraco.co.ke 18. Anthony Karembu KenGen/Kenya Chief Energy Economist +254 20 3666329 akarembu@kengen.co.ke 19. Mr. Patrick Obath KEPSA/Kenya Chairman 254 20 2730371/2; 2727883/936 info@kepsa.or.ke 20. Ms. Betty Maina KAM/Kenya Chief Executive Officer 254 020 2324817/8; 020 8155531/2 info@kam.co.ke 21. Rutembesa Jean Paul EWSA/Rwanda Head of Electricity Planning +250 788407148 jprutembesa@ewsa.rw 22. Claver Gakwavu EWSA/Rwanda Head of Electricity Planning Statistics and Research +250 788605760 cgakwavu@ewsa.rw PPP End of Project Performance Evaluation Report, February 2013 22 Name Organization/Country Title Telephone Email 23. Jean Baptiste Bwanakeye RURA/Rwanda Former Director for Electricity, RURA jbbwanakeye@yahoo.fr 24. Oliver Ngororabanga RDB/Rwanda Energy Specialist 25. Peter Kinuthia EAC Senior Energy Officer +255 272504253/8 PKinuthia@eachq.org 26. Francis Rugambwa . SINELAC/ (Burundi/Rwanda/DRC ) Manager, Regional Control Centre, +250 788306895 ruga_francis@yahoo.co.uk 27. Mohamedain Eltigani Seif-Elnasr COMESA Energy Economist + 260 977 456204 seifeinasr@comesa.int 28. Dick Edwards Nexant Inc./ USA PPP Contractor – Team Leader +1 (503) 929-2946 dedwards@nexant.com 29. Ray Holton Nexant Inc./ USA Legal Specialist +1 (503) 929-2946 rholton@nexant.com 30. Lauren Wygonski Nexant Inc./ USA PPP Programme Coordinator+1 (503) 929-2946 lwygonski@nexant.com 31. Edward Hoyt Nexant Inc./ USA Rural Electrification, Clean & Renewable Energy Advisor +1 (503) 929-2946 EAHoyt@nexant.com PPP End of Project Performance Evaluation Report, February 2013 23 Annex J: Lists of PPP participants Name Gender Organization Title Raphael Mwaura M KPLC System Development Manager Patrick Mawala M KPLC Energy Purchase and System Control Manager Agnes Ongadi F KETRACO Head of HR, ADM, and Corporate Affairs Godfrey Kariuki M KETRACO Economist Duncan Macharia M KETRACO Company Secretary John Mativo M KETRACO Head of Technical Division Justus G. Kageenu M KETRACO Chairman Mahmoud A. A. Sayed M EAPP Project Coordination Consultant Eritrea Mehari F EAPP EAPP/Project Sec. Jasper Oduor M EAPP Executive Secretary Joseph K. Magochi M EAPP ICT Expert Patrice Manirakiza M EAPP Power Engineer Alemu Tenibo M EAPP Power Economist Sherif A. Ewiss M EAPP Assistant Secretary Ziporah Mujawingoma F RECO Head of Procurement Acquisition and Management of Electrical Materials Pascal Mutesa M RECO Manager of National Control Centre Jean Baptiste Bwanakeye M RURA Director of Electricity Regulation Paul Ndoriyija M REGIDESO Chief of Electricity Equipment Service Aloys Sahiri M Ministry of Energy and Mines Advisor in Energy Sector Gosaye Mengistie Abayneh M Dept. Head Ministry of Mines and Energy Development Tulimbumi Bwambi Abel F Ministry of Energy and Minerals Legal Officer II Leonard Robert Masanja M Ministry of Energy and Minerals Senior Engineer Iziba Mabiala M Ministry of Energy Mohamedain Eltigani Seif￾Elnasr M COMESA Energy Economist Fredrick John Sajjabi M Ministry of Energy and Mineral Development Senior Energy Officer Peter Kinuthia M East African Community Senior Energy Officer Francis Rugambwa M Rwanda/SINELAC SINELAC Manager – Regional Control Center Adjua Adjei-Danso F Energy Markets Group Mohammad Ahmad M Energy Markets Group Musara Beta M Energy Markets Group Mohamed Sebugenyi M UETCL Principal Control Engineer Martin Erone M UETCL Legal Officer RUTEMBESA Jean Paul M EWSA Head of Electricity Planning Lydia Wanja F KETRACO Chief Legal Expert Godfrey Macharia Kariuki M KETRACO Economist/Head of Planning Daniel Mulatu M EEPCO Planning Esubalew Kebede M EEPCO Ethiopia Kenya Power System Interconnection Project Manager Endale Mamo M EEPCO Economist Mekuria Lemma M EEPCO Corporate Planning Director PPP End of Project Performance Evaluation Report, February 2013 24 Name Gender Organization Title Zelalem Gebrehiwot M EAPP Technical Ass/ ES Patrice Manirakiza M EAPP Power Engineer Eritrea Mehari F EAPP EAPP/Proj Sec. Eng. Yousif Hamza Yousif M SETCO Engineer Yasir Mohamed Elhassan M SETCO Economist Rwabangi Luteganya M TANESCO Abdul Masunga M TANESCO Huruge Deogratias M REGIDESO Electricity National Dispatching Bah. Thierno M AfDB Ephrem Tesfaye M EAPP Power Economist Anthony Karembu Njeru M KenGen Chief Energy Economist Kagaba Paul Mukiibi M ERA Assistant Technical Compliance Manager Vincent Olie Ochwo M UNBS Head Engineering Standards Division Gerald Bwowe Muganga M UETCL Manager‐Planning & Investments Alex Mudasingwa M RWRA Electricity Regulation Officer Olivier Mukeshimana M RBS Standards Officer Danny Rwagasan M EWSA Belayneh Gizaw Feleke M Abdu Hussen Ahmed M EEA Electrical Engineer Nagwan Shaaban Mahmoud Elfass M EOS Standard Specialist Hatem Amer M Egypt Era Manager of Central Administration for Licensing and Tariff Atef Elsayed M Izaiah Mulenga M ZBS Standards Officer Yoane Mukabe M ERB Claver Gakwavu M EWSA Head of Electricity Planning, Statistics, and Research Zelalem Gebrehiwot M EAPP Technical Assistant to the Executive Secretary Justin Muna Mwangi M KETRACO Projects Coordinator/Civil Engineer Patrick Idawo Mawala M KPLC Energy Purchase and System Control Manager Etienne Tshibangu Munyenz M SNEL DRC Chief Project Director Ephrem Tesfaye Belayneh M EAPP Power Economist Simon Ngure M KENGEN Regulatory Affairs Director Amare Hadgu Seyoum M EAPP EC TA Regional Engineer Mr Nolasque NDAYIHAYE M Ministry of Energy and Mines‐ Burundi Director of planning and electric Project studies Mr Pascal NDAYISHIMIYE M REGIDESSO Director General of REGIDESO Dr. Frederick Nyang M ERC‐Kenya Emelda Odhiambo F KPLC Chief Engineer, Commercial Services Gakwavu Claver M EWSA Head of Electricity Planning Statistics and Research Alexis Mutware F RURA Mr. Mathew Mbwambo M EWURA Mr James Philip K Sembeguya M Electricity Regulatory Authority Statistician/IT Officer Muganga Gerald M UETCL Manager Planning and Investments Mr Philip F P Ggayi M Rural Electrification Agency Senior Planning Engineer Esther Ruto F Kenya Rural Electrification Manager PPP End of Project Performance Evaluation Report, February 2013 25 Name Gender Organization Title Authority PPP End of Project Performance Evaluation Report, February 2013 26