Power Africa Transaction and Reforms Program (PATRP) Performance Evaluation Report February 2020 This report was produced at the request of the United States Agency for International Development (USAID). It was prepared independently by Integra LLC under the Learning, Evaluation, and Analysis Project (LEAP III). The authors’ views expressed in this publication do not necessarily reflect the views of USAID or the United States Government. 1 POWER AFRICA TRANSACTION AND REFORMS PROGRAM (PATRP) PERFORMANCE EVALUATION FINAL REPORT Contract Title: LEAP III: Learning, Evaluation, and Analysis Project Contract Number: GS-10F-083CA / 7200AA18M0004 Activity Number: LEAP III 2018-11 – 1009.1029 Submitted: February 4, 2020 Contractor: Integra Government Services International LLC 1100 Vermont Avenue NW, Suite 750 Washington, D.C. 20005 Limestone Analytics LLC (Subcontractor) USAID Office: Power Africa COR: Dr. Yoon Lee, ylee@usaid.gov DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. 2 TABLE OF CONTENTS TABLE OF CONTENTS 2 ACRONYMS 3 EXECUTIVE SUMMARY 5 1. EVALUATION PURPOSE & QUESTIONS 9 2. PROGRAM BACKGROUND 13 3. EVALUATION METHODOLOGY 15 4. FINDINGS FROM KENYA 18 5. FINDINGS FROM ETHIOPIA 24 6. FINDINGS FROM NIGERIA 31 7. FINDINGS FROM GHANA 38 8. OVERARCHING FINDINGS 46 9. LESSONS AND RECOMMENDATIONS 61 ANNEX I: STATEMENT OF WORK 64 ANNEX 2: EVALUATION METHODS AND LIMITATIONS 75 ANNEX 3: DATA COLLECTION INSTRUMENTS 82 ANNEX 4: SOURCES OF INFORMATION 89 ANNEX 5: CONFLICTS OF INTEREST 94 3 ACRONYMS AIP USAID’s African Infrastructure Program ATC&C Aggregate Technical, Commercial, and Collections BTG Beyond the Grid CES Community Engagement Strategy COR Contracting Officer's Representative DISCO Distribution Company DT Distribution Transformer ECG Electricity Company of Ghana EEA Ethiopian Energy Authority EEP Ethiopian Electric Power EEU Ethiopian Electric Utility EQ Evaluation Question ESRP Energy Sector Recovery Program FC Financial Closure Forex Foreign Currency and Exchange GLOS Government Letter of Support GMP Gas Master Plan GMSP Grid Management Support Program GW Gigawatt GWh Gigawatt-Hour IA Implementation Agreement IPP Independent Power Producer KenGen Kenya Electricity Generating Company KETRACO Kenya Electricity Transmission Company KII Key Informant Interview kv Kilovolt kWh Kilowatt-Hour KWPP Kipeto Wind Power Plant LEAP III Learning, Evaluation, and Analysis project LTWP Lake Turkana Wind Power LTWPP Lake Turkana Wind Power Plant 4 M&E Monitoring and Evaluation M2C Meter 2 Cash MCC Millennium Challenge Corporation MMscfd Million Standard Cubic Feet Per Day MoE Ministry of Energy MoWIE Ministry of Water, Irrigation, and Electricity MW Megawatt NBET Nigerian Bulk Electricity Trading Plc. NERC Nigerian Electricity Regulatory Commission PATRP Power Africa Transactions and Reforms Program PATT Power Africa Transition Tracker PCOA Put-Call Option Agreement PDS Power Distribution Services PMP Performance Monitoring Plan PPA Power Purchase Agreement PPP Public-Private Partnership RFP Request for Proposals SAAR South Addis Ababa Region SHS Solar Home System SSA Sub-Saharan Africa TA Transaction Advisor TTIP Tema Takoradi Interconnection Project USG United States Government USAID United States Agency for International Development WAGP West African Gas Pipeline WAPCO West African Pipeline Company 5 EXECUTIVE SUMMARY This report summarizes the findings of a 2019 evaluation of the Power Africa Transactions and Reforms Program (PATRP) contract, implemented by Tetra Tech between 2014 and 2019. This evaluation focuses specifically on the work of PATRP in Kenya, Ethiopia, Ghana, and Nigeria. The goal of this evaluation is to answer nine evaluation questions (EQs). The EQs and associated findings are briefly summarized below. The evaluation is based on numerous primary and secondary sources of evidence. The primary source of data was key informant interviews (KIIs) with more than 60 individuals affiliated with PATRP and its partners between September 30 and November 5, 2019. Secondary sources include numerous project￾and transaction-related documents and datasets. OVERVIEW OF PATRP PATRP provided technical assistance, training, capacity building, and transaction support services under Power Africa, a U.S. government-led partnership coordinated by USAID that aims to increase access to electricity. PATRP activities fell under four main categories: 1. Institutional support to the Power Africa Coordinator’s office 2. Late-stage transactional support 3. Small-scale projects, rural electrification, and mini-grids support 4. Regulatory and institutional strengthening and policy reform PATRP tracked “a pipeline of power generation projects from the concept phase to bring late-stage power generation, transmission, and distribution projects to financial closure (FC).” This transaction-focused approach is meant to spur “systemic changes and fundamental reforms in the power sector that will facilitate even greater private sector involvement and investment over the long term.” The map above illustrates the span of PATRP’s work through Transaction Advisors (TAs), as well as utility, Beyond the Grid (BTG), and gas advisors.1 1 Copied from the 2018 PATRP Annual Report. 6 SUMMARY OF EVALUATION RESULTS BY EQ Evaluation Question Key Findings 1. To what extent was the transaction￾focused approach to advancing megawatt (MW) deals successful? ● Eight major transactions, with a combined capacity of approximately1,700 MW, closed in the four countries covered in this report. Five were in Kenya, and none were in Ethiopia. ● In Kenya, some transactions closed primarily due to the efforts of the TA. ● In Ghana, overcapacity makes a transaction-focused approach an ineffective strategy for improving the country’s energy situation, at least in the short term. ● In Nigeria, high aggregate technical, commercial, and collections (ATC&C) losses, primarily tied to distribution companies (DISCOs), which are not reflected in tariffs has led to payment issues that add risk to any new generation transactions. The enabling environment in Nigeria makes a transaction-focused approach significantly less impactful than in other countries. ● No generation capacity reached financial close in Ethiopia. ● Partners in the public and private sectors placed high value on TAs, although their impact may not always have been best captured through the transactions that closed. ● The focus on “closure” of transactions is not typically representative of the complex environment within which each project exists and simplifies the nature of the problems Power Africa addresses. However, the involvement of PATRP in some cases (such as Kipeto), led to FC. 2. How effective was PATRP’s approach to strengthening utility viability, as evident in their work in Nigeria and Ethiopia? ● Clear quantitative improvements to key performance indicators (ATC&C losses, revenue) were seen across multiple DISCOs in Nigeria during the period of PATRP involvement. However, the overall situation for DISCOs in Nigeria is still dire, and the sustainability of marginal improvements from technical assistance is jeopardized by an unstable financial environment in the power sector that is overrun by insolvent entities. ● Improvements in revenue collections in Ethiopia occurred as a result of PATRP’s diagnostic assessment of the utility’s operations and the implementation of the Meter 2 Cash pilot project in the South Addis Ababa Region (SAAR). However, the underlying data assumes are not verified by accurate meter reading, and attributes 100 percent of increased revenue to PATRP support. 7 Evaluation Question Key Findings ● The PATRP work demonstrated the possible benefits of working with utilities, but also showed the limitations of this approach if the enabling environment is not addressed. 3. How effective was PATRP’s approach to expanding off-grid connections? ● Off-grid connections established in the four countries examined in our study (Kenya, Ghana, Ethiopia, Nigeria) comprise over 96 percent of all PATRP off-grid connections. ● PATRP’s counting methodology leaves room for doubt with regard to its role in the connections measured. ● BTG advisors were generally responsible for the kinds of major successes observed for off-grid work. 4. How effective was PATRP’s approach to policy reform? ● Some notable efforts shaped the policy direction. ● Enabling environment issues are difficult to overcome with the limited resources available in each country. ● Stakeholders observed that PATRP’s efforts were more effective when the advisors were local residents with international experience, or when advisors spent a longer time in the country. 5. What were the positive and negative consequences of achieving targets? Were those intended? ● Not all transactions offer good value for money, and adding MWs of generation capacity to the grid can have both positive and negative impacts on the overall health of each country’s power sector. ● Adding more capacity in Ghana may have further burdened an already overwhelmed electricity market. ● Closing the Azura gas deal in a non-competitive and non-transparent way in Nigeria may have created an unrealistic standard for future power purchase agreement pricing, setting is a bad precedent for the sector. ● In Kenya, Cummins Baringo Biomass Plant technology was not properly vetted, and feedstock was incompatible with the technology. As a result, the plant is not operational. ● However, multiple stakeholders in Kenya reported that they have seen increased interest from US firms in investing in the country's power sector. 6. Is there evidence that PATRP’s achieved results had a wider effect than anticipated? If so, in what way? ● TAs adapted to the unique context of each country and pursued various policy or regulatory goals in consultation with local partners. ● In Ghana, for example, the TAs contributed significantly to the development of the natural gas sector, the Energy Sector Recovery Program (ESRP), reforms to the Gas Pricing Policy, and the relocation of the Karpowership, which is expected to save the country more than $100 million. 8 Evaluation Question Key Findings ● In Ethiopia, TAs provided valuable input on geothermal and public￾private partnership (PPP) proclamations, paving the way for improved transactions in all sectors. 7. What is the possibility of positive effects being likely to continue after the program has ended? ● Generation deals typically involved PPAs and other guarantees that ensure they will operate in the future. ● The sustainability of utility work is threatened by an ineffective payments system in Nigeria’s energy system. ● In Ethiopia, the sustainability of utility improvements is seen as dependent on the implementation of activities under the East Africa Energy Program. ● Many policy efforts will continue to influence the power sector in each country going forward. ● The experience accumulated through PATRP is expected to shape many future interventions in the power sectors of these countries. 8. To what extent were program activities adapted for the local context? ● In Ghana, TAs addressed the significant financial issues in the sector by participating in the ESRP efforts, proposing a cost-saving relocation of an existing generation facility, modeling sector finances, and participating in numerous policy efforts that key stakeholders valued highly. ● In Nigeria, TAs were able to help with gas flaring auctions. However, the difficult enabling environment in Nigeria meant transactions were difficult to advance, although the subsequent Nigeria Power Section Program (NPSP) shifted its focus to the enabling environment. 9. How well did the program develop and make use of monitoring and evaluation (M&E) systems? ● Generation capacity is a poor metric for capturing program success because it equates all generation efforts, regardless of value, price, political context, and overall impact to the health of each country’s electricity sector. ● Connections as a measure is also somewhat problematic, or at least unclear. For much BTG work, estimates of new connections relied on questionable assumptions that can lead to significant overestimation of attributable connections. ● Some actors believed that the centralized nature of PATRP meant that data was not shared as well as was expected. Distribution of data may have improved with the transition to a new program that adopted a more decentralized approach. 9 1. EVALUATION PURPOSE AND QUESTIONS Power Africa, a United States Government (USG) entity, solicited the services of Integra to conduct a performance evaluation of a USAID-funded project, PATRP. The findings of this evaluation will be used to judge the effectiveness of PATRP’s transaction-focused approach to improving the accessibility of electricity in sub-Saharan Africa (SSA). The results of this evaluation will be shared with Power Africa’s leadership to improve internal capacity to make programmatic adjustments, inform future project design and strategies, and assist in evidence￾based decision-making. The transaction-focused approach is a relatively new concept to USAID and divergent from other foreign assistance models. This evaluation will be a valuable addition to the research literature on the subject and can be used to address the value of replicating the model within USAID, and across other development programs. EVALUATION QUESTIONS In the Statement of Work, USAID requested that the evaluation team address nine EQs (some with component sub-questions) in its study. These nine questions are presented with the evaluation team’s interpretation of their intent and meaning. EQ 1: To what extent was the transaction-focused approach to advancing MW deals successful? To answer the first EQ, the report will need to define the “transaction-focused approach” is and how it differed from traditional approaches to a program of this nature. The report describes examples of transactions, and the support they received, to help readers better understand the nature of this approach. Given that this question leaves some room for interpretation, as part of its response the evaluation team will explain its definition of “successful” before judging the overall extent of PATRP’s success. The first EQ is accompanied by two sub-questions that further clarify the type of knowledge USAID is seeking: Do parties to financially closed transactions that received PATRP support feel that PATRP’s involvement expedited or otherwise assisted the transaction? Why or why not? This sub-question seeks input from those partners that were part of transactions that reached FC and asks them to reflect on the value of PATRP support. To address this sub-question, the report will comment on the extent to which PATRP can be causally linked to the FC of transactions. The extension of this sub-question is asking why PATRP involvement was valued (or why it wasn’t). In doing so, the report can define the conditions and actions that led to PATRP’s successes, as well as its failures. 10 What have been the most effective factors in expeditiously closing transactions? What challenges did PATRP encounter? How did it overcome them? The second sub-question asks for a list of the most important factors to PATRP’s closing transactions “expeditiously” (which refers to both speed and efficiency). Correctly defining success factors can improve the ability of USAID to forecast successful programming in future efforts, as a function of definable pre￾conditions. This sub-question also asks for an explanation of the challenges that PATRP faced, and the ways that they were overcome (assuming they were). This line of inquiry will define, and consider methods for overcoming, the obstacles that prevent successful achievement of PATRP’s goals. Where solutions were effectively implemented, they can be highlighted and analyzed for reuse. Where obstacles were not overcome, we will discuss whether appropriate solutions may have existed and whether the design of the program was conducive to overcoming these obstacles. EQ 2: How effective was PATRP’s approach to strengthening utility viability, as evident in its work in Nigeria and Ethiopia? The second EQ is concerned with the effects of PATRP’s work with utilities in Nigeria and Ethiopia. The effectiveness of the work may not be the same in both countries given the notably different contexts. To address this question, it will, therefore, be important to provide an in-depth descriptive account of the interventions in Nigeria and Ethiopia, and a list of quantifiable outcomes for the utility and the electricity sectors. We will also include anecdotal information from the KIIs to clarify how partners’ subjective feelings about the impact of the work. To what extent did PATRP’s work to strengthen the utility lead to and/or is expected to lead to new grid connections and the ability of the Government to add MWs to the grid? This sub-question is asking for a link between PATRP’s work and key output metrics for the program, such as connections or generation capacity. This sub-question will force the evaluation team to quantify the impacts of utility work. By articulating the impacts of PATRP’s work in this way, the evaluation team will have the opportunity to discuss the PATRP theory of change and explore the causal links among inputs, outputs, and outcomes. The evaluation team will also be able to comment on the appropriateness of defining success by metrics such as the number of MWs or connections. To what extent are PATRP’s achievements (e.g., loss reduction, system improvements) expected to be sustainable? Sustainability is a key consideration for programming of this nature, especially given USAID’s promotion of self-sufficiency. This sub-question seeks judgment on the nature of PATRP’s achievements. Are some benefits expected to continue in the future? If so, which ones? Once the program ends, which will change without the presence of the embedded advisors? By answering these sub-questions, the evaluation can discuss which activities engendered benefits that were not measured during the program's lifespan. The evaluation team will also address how to promote the sustainability of outcomes to maximize overall impact. 11 What key challenges did PATRP encounter in implementing this work, and how did it seek to overcome them? To what extent were those efforts successful? As with the sub-question from EQ 1, the evaluation team will consider obstacles to success and how to overcome them, with specific focus on the challenges relevant to utilities. EQ 3: How effective was PATRP’s approach to expanding off-grid connections? EQ 3 seeks clarification on PATRP’s impact with respect to off-grid customers. This will involve looking at the ways PATRP supported off-grid companies and seeing how this work is tied to outcomes in the sector. The evaluation team will make a judgment about the outcomes attributable to PATRP relative to their efforts in the off-grid sector. How effective were PATRP’s efforts to bring lending institutions into the off-grid sector to finance off-grid companies—solar home system (SHS) and micro-grid? The first sub-question for EQ 3 asks about PATRP’s work to connect off-grid companies with financing, with an emphasis on companies that produce SHS and microgrids. The evaluation team will consider anecdotal evidence from interviews with relevant stakeholders, contextualized by data about the sector overall. To what extent do off-grid companies that received PATRP support feel they benefited, and how? To address this sub-question, the evaluation team will present anecdotal evidence from representatives of off-grid companies about the benefits of PATRP activities. This can be contextualized by anecdotal evidence provided by PATRP representatives, and data that includes metrics like number of new connections, grids, customers, etc. Relevant work with policy or enabling environments can also be considered. What key challenges did PATRP encounter in implementing this work, and how did it seek to overcome them? To what extent were those efforts successful? Again, the evaluation team is asked to consider obstacles in the off-grid sector and solutions to them. EQ 4: How effective was PATRP’s approach to policy reform? Which PATRP interventions had the most significant impact on policy reform, and why? The fourth EQ considers the work done through PATRP to advance policy reforms in the four evaluation countries. In order to judge whether PATRP had an effective approach to policy reform, we will need to define what mechanisms were used to influence policy, what resources were available to policymakers, and what legal or institutional frameworks contextualized policy work in each country. Then, the evaluation team will need to judge whether, given the resources available and context of each country, policy achievements were satisfactory. The second part of the question asks which interventions had the most impact on policy reform. The evaluation will address this question by considering the different interventions that PATRP implemented, and evaluating which ones generated tangible impact and any common characteristics of those interventions. 12 EQ 5: What were the positive and negative consequences of achieving targets? Were those intentional? EQ 5 asks for feedback related to the targets set by the program, and the results of pursuing those targets. In some cases, the metrics targeted by PATRP may not have produced the results expected at the conception of the project. Some discussion will occur related to the “correct” approach to targeting, and the perverse incentives that can emerge if targets are improperly defined. EQ 6: Is there evidence that PATRP’s achieved results had a wider effect than anticipated? If so, in what way? EQ 6 captures what is not addressed in the previous question, that is, what accomplishments of PATRP are not measured by the chosen metrics or do not align with the program targets. EQ 7: What is the possibility of positive effects being likely to continue after the program has ended? EQ 7 addresses sustainability. Will the work that PATRP completed lead to results that last beyond the life of the program? The evaluation team will address which PATRP efforts are expected to generate long￾term impacts, and what factors will influence sustainability. EQ 8: To what extent were program activities adapted for the local context? EQ 8 asks for examples of ways that the program adapted to the specific needs of the countries it operated in. The evaluation team will address this question by providing examples of ways PATRP implementation differed in each country and discussing the impacts of these differences. This question can also be addressed by discussing contexts to which PATRP did not adapt well, with a view to future improvements. EQ 9: How well did the program develop and make use of monitoring and evaluation systems? M&E systems can improve transparency and adaptability throughout the lifespan of a project. In addressing EQ 9, the evaluation team will consider what data was collected to aid M&E efforts and discuss whether this data was appropriate for the needs of key stakeholders. 13 2. PROGRAM BACKGROUND POWER AFRICA Power Africa is a collaborative initiative that aims to support the addition of 30,000 MW and 60 million connections across SSA. The expressed purpose of Power Africa is to promote sustainable development, increase trade linkages, and end extreme poverty. To attain this goal, Power Africa pools resources from 12 USG agencies; multilateral, bilateral, and international institutions; African governments; and over 150 private sector partners. PATRP “In 2014 the U.S. Agency for International Development (USAID) selected Tetra Tech to serve as the prime contractor for the Power Africa Transactions and Reform Program (PATRP), which provides technical assistance, capacity building, and transaction support services under Power Africa.” -—Tetra Tech Website PATRP’s activities are organized under four main objectives2: Objective 1: Institutional Support to the Power Africa Coordinator’s Office Objective 2: Late-Stage Transaction Support Objective 3: Small-Scale Projects and Rural Electrification / Mini-Grids Support Objective 4: Regulatory and Institutional Strengthening and Policy Reform Under Objective 1, PATRP leads and directs partner country activities, including transaction-centered support to the energy sector and BTG initiatives. Additionally, PATRP is responsible for M&E, technical support, and reporting. Objective 2 deploys TAs who connect investors and lenders to energy projects and help remove roadblocks to the FC of energy investments. Although TAs primarily support transactions led by the private sector, they also develop power generation projects at all stages, and may be embedded within financial institutions, national utilities, or government ministries. Activities associated with Objective 3 increase sources of generation and improve access to energy for rural and peri-urban communities with no connectivity to the national grid. Objective 4 enhances the enabling environment for energy sector development. For example, activities under this objective include regulatory reform (e.g., tariff structure), policy and legal reviews, and improvement programs at electric utilities. RESULTS FRAMEWORK PATRP’s performance management plan and M&E activities are based on the Power Africa Initiative results framework. Specifically, PATRP’s performance is tracked by its activities supporting Power Africa’s Development Objective and Intermediate Results. In total, PATRP has eight intermediate and sub￾intermediate results, each with associated performance indicators, as shown in Figure 2.1. 2 USAID PATRP Power Africa Transactions and Reforms Program (2018). 2018 Annual Report: October 1, 2017–September 30, 2018. Prepared by Tetra Tech ES, Inc., Contract AID 623-C-14-00003. 14 Figure 2.1: PATRP Results Framework The results framework for PATRP is a hypothesis that links key intermediate results to an overarching development objective. PATRP’s development objective is to “increase the quantity and quality of transmission, distribution, and cleaner generation projects to expand access to electricity and spur economic growth.” The two main results that the Theory of Change assumes are prerequisites for this objective are: 1. Increase investment in the power sector in SSA 2. Improve capacity to grow and manage the power sector in SSA This evaluation includes empirical evidence that can be used to either support or reject the results framework, which can then be refined for future Power Africa initiatives. 15 3. EVALUATION METHODOLOGY The goal of this evaluation is to study the progress of PATRP in improving the supply of, and access to, electricity in SSA, and to better understand the ways PATRP has been both effective and ineffective in achieving its goals. The evaluation team developed an evaluation methodology that seeks to address nine EQs, selected by USAID. This approach can be characterized as a mixed-method approach that combines a review of existing secondary data sources as well as a number of semi-structured KIIs. It reflects USAID’s parallel combinations approach, which uses two approaches to collect and analyze information that is then synthesized to answer individual EQs.3 This study has compiled information about how PATRP activities were implemented with respect to each program objective. The evaluation team will assess whether PATRP activities align with the stated goal and targets of improving the supply of, and access to, electricity through transaction-based support, policy reforms and institutional strengthening, and the provision of on-grid and BTG energy solutions. DATA SOURCES Program Documents/Data and Information (Secondary Sources) The performance evaluation team analyzed numerous sources of information, including program and administrative documentation, reporting data from the Power Africa Transaction Tracker (PATT) database, and qualitative research. Key secondary data sources include: ● Program materials that document the transactions, projects, and policy reforms supported by PATRP. Examples include country investment plans, quarterly and annual reports, M&E and performance monitoring plans (PMPs). ● Administrative documents from national electric utilities, national regulatory agencies, and off-grid energy providers provide descriptions of partner performance and progress toward common objectives. For example, annual reports from utilities and off-grid energy companies document, among other important details, customer growth and financial viability. ● The PATT database provides source material for all qualified Power Africa transactions, including partner involvement for each PATRP-supported transaction or project. PATT data can be used to identify the generation capacity, cost, technology, and other details for each transaction under Power Africa. KIIs (Primary Qualitative Sources) This performance evaluation includes information obtained by interviewing key stakeholders in order to understand how PATRP was implemented and how program partners and participants believe the program has met its objectives. Key stakeholders interviewed represent diverse interests and perspectives. They include: ● PATRP TAs from each country ● Representatives of corporations that were part of PATRP-linked transactions 3 See USAID Technical Note: Conducting Mixed-Method Evaluations, Version I, June 2013. 16 ● Representatives of electric utilities in Nigeria, Ghana, Kenya, and Ethiopia that received PATRP support ● Off-grid energy company staff who interacted with PATRP ● Representatives of governments, ministries and regulatory commissions that received PATRP support ● USG staff and implementing partners This performance evaluation report identifies implementation successes, findings, and observations, and a number of challenges in the program. It also outlines lessons learned and key takeaways that provide further insight. Additionally, the performance evaluation report recommends actions and next steps for consideration by technical teams, activity managers, and senior management in the design of future Power Africa programs. LIMITATIONS OF THIS PERFORMANCE EVALUATION In a number of ways, the data and information collected through the mixed-methods performance evaluation approach may be unsuitable to providing accurate, concrete, and conclusive answers to the EQs. Some of these limitations are summarized below: The Content of KIIs Includes Subjective Accounts of PATRP KIIs allow stakeholders, with significant knowledge and experience, the opportunity to share their thoughts and experiences directly with an evaluation team, typically in a partially open format that is not restricted by the conventions associated with quantitative data collection. A feature of the resulting data that is both a strength and a weakness is that responses will reflect the key informants’ individual beliefs, values, feelings, biases, and interests. If these responses are equated with objective facts, the resulting analysis can be distorted. To avoid confusion that may affect the analysis in the report, this document strives to provide relevant context to information gathered through KIIs with secondary sources where possible. The report also indicates when a statement refers to a comment that the evaluation team heard in an interview as opposed to facts that are verifiable through secondary sources. To make this distinction, the report uses quotation marks or describes the source of information (for example, “The evaluation team heard...”, “key stakeholders expressed...”, etc.) The large number of stakeholders interviewed allows for improved accuracy in many cases. One interviewee may provide an inaccurate account of an event or may share a recommendation that would poorly serve the program. However, when the same event is described numerous times from different perspectives, the overall result provides a much clearer picture that should better reflect objective reality. When such comments are combined with secondary sources and quantitative data, the risk of subjective accounts distorting the analysis of the evaluation decreases significantly. KIIs Drew from a Limited Sample Time and resource constraints meant that the evaluation team was able to interview approximately 70 people as part of our evaluation. This number is sufficient to learn some lessons and gather information about some results of the program but does not guarantee that every activity or output associated with the PATRP program has been closely examined or discussed with all relevant stakeholders. 17 The qualitative data in this report reflects a fraction of the knowledge that exists about the PATRP program. However, when combined with secondary sources and relevant quantitative data, the evaluation team believes the number of KIIs was sufficient for the purpose of answering EQs. Selection Bias(es) The list of interviewees contacted was assembled from a contact list provided to the evaluation team by Tetra Tech and another from the Power Africa mission offices. Interviewee selection was therefore influenced by these organizations, both of which have interests that may have potentially affected the contacts they chose to share. However, given that the interests of both groups are likely to be significantly different, and because the evaluation team had access to numerous secondary sources to validate what was said in KIIs, the evaluation team believes that the list of stakeholders invited for interviews was adequately constructed. Interviewees provided diverse answers and perspectives that do not seem likely to significantly bias the analysis in this evaluation. The KIIs were conducted on a voluntary basis, and not all key stakeholders who were invited to interviews responded or made themselves available. This means that interviews that were conducted may represent a subgroup whose characteristics or properties are not representative of the entire pool of stakeholders. The stakeholders in our interviews may have elected to respond or participate because they have strong feelings about the program, or because they have special interests that they feel may be best represented by influencing the program’s evaluation. The group of interviewees may have also been more involved with the program and therefore easier to locate. The evaluation team acknowledges this limitation. Timing KIIs were conducted in October of 2019. In many cases, key PATRP stakeholders had already left their positions with PATRP, in some cases for work in other countries. Where possible, interviews were conducted with key staff, such as former TAs, by telephone or over the internet, but the evaluation team acknowledges that the timing of the evaluation means there is a higher likelihood that the voices of some key stakeholders may not have been included. 18 4. FINDINGS FROM KENYA Kenya’s electricity comes predominantly from renewable sources. With an installed capacity of roughly 2.8 Gigawatt (GW)4 and a peak demand of around 1.8 GW5, the country generates two-thirds of its power from geothermal and hydro, with the rest coming from thermal and renewable resources, such as wind and solar. The Kenyan Electricity Generating Company (KenGen) is majority government-owned and is the primary power producer in the country, providing about 60 percent of Kenya’s power.6 Kenya has been open to independent power producers (IPPs) since the 1990s, and major IPPs in the country include Lake Turkana Wind Power (LTWP) and Kipeto Energy Limited. However, lack of transparency in procurement processes and gaps in feasibility assessments remain problematic, perhaps most clearly evidenced by the controversial Lamu Coal Power Station, which was slated for development but is currently on hold due to public backlash over environmental concerns.7 At the national level, the energy sector is governed by the Ministry of Energy (MoE). The Kenya Electricity Transmission Company (KETRACO) is responsible for high-voltage transmission, and the Kenya Power and Lighting Company (Kenya Power) is the sole off-taker in the country. Aging transmission infrastructure, public budget constraints, and increased intermittency of the power supply all pose significant challenges to the stability of the national grid in the coming years. In an effort to adapt to these barriers, KETRACO is undertaking procurement for five privatized transmission lines under a PPP pilot program. Under the new Energy Act 2019, the former Energy Regulatory Commission has become the Energy and Petroleum Regulatory Authority. The new Energy Act also introduced the Rural Electrification and Renewable Energy Corporation as the successor to the Renewable Energy Authority, as Kenya strives to achieve its goal of universal access to electricity by 2025.8 Whether Kenya will reach this ambitious goal depends to a great degree on the success of the off-grid energy sector in reaching remote rural citizens. A newly drafted grid code has filled a gap in the regulation of off-grid energy supply, but providers of SHS and mini-grid technology may face challenges in navigating the regulatory environment in this relatively young market. 4 “2018 Integrated Annual Report and Financial Statements.” Kenya Electricity Generating Company Plc, June 30, 2018. 5 “Maximizing Financing for Development in Action: The Kenya Energy Sector Experience.” World Bank Group, April 2019. 6 “Women in Energy: $1 billion dollars in negotiations.” Mining Review Africa, August 2019. 7 “Kenya halts Lamu coal power project at World Heritage Site.” BBC News, June 2019. 8 “Highlights of Kenya’s Energy Act 2019.” Rodl & Partner, 2019. 19 Kenya Overview Population ~50 million GDP per Capita $1,507.81 Map from: The World Factbook (CIA) Generation Capacity ~2,800 MW Peak Demand ~1,800 MW Capacity per Capita 56 watts Annual Demand per Capita 164 kWh Key Issues ● Low transparency in procurement ● Gaps in feasibility assessments ● Aging transmission and distribution infrastructure ● Budgetary constraints ● Increased intermittency of the power supply ● Evolving off-grid energy regulatory environment 4.1 PILLAR 1: GENERATION Five large-scale transactions that PATRP supported in Kenya reached FC. Of those, only the Lake Turkana Wind Power Plant (LTWPP, 310 MW) is currently in commercial operation. The Cummins Baringo Biomass Plant (8.4 MW) is not in operation because the mathenge weed feedstock contained too much moisture to be compatible with the plant’s technology. The remaining transactions that reached FC are progressing toward commercial operation. LTWPP (310 MW) The LTWPP, the largest grid-connected wind farm in SSA, reached FC in December 2014. PATRP provided support to the Grid Management Support Program (GMSP), which was a precondition to the project’s financing agreement to prove that Kenya’s grid could support the load from LTWPP. PATRP also provided support in establishing operational procedures for the plant, including system emergency procedures and real-time data exchanges between the plant and Kenya Power’s substation. However, despite reaching FC in 2014, the LTWPP did not come online until 2019 due to significant delays in the construction of the 434-km transmission line connecting the plant to Kenya Power’s Suswa substation. The contractor that KETRACO initially hired to construct the line went bankrupt in 2016, leaving the plant disconnected from the grid until a new contractor could be brought in to finish the job in 2018, and tests could be run and completed in early 2019. The delays also left taxpayers responsible for compensation payments from Kenya Power to LTWP under the terms of the power purchase agreement. The LTWPP has also faced criticism for failing to properly engage with local communities 20 impacted by the project.9 The issues associated with bringing LTWPP to commercial operation might raise concerns about the way in which the project feasibility and procurement process was undertaken. Olkaria V Geothermal Plant (158 MW) KenGen’s Olkaria V Geothermal Plant was brought to FC in December 2017. PATRP helped craft the strategy for the project, identifying three clear strategic dimensions that align with the Government of Kenya’s objectives. PATRP also offered critical assistance to KenGen in interpreting the technical specifications provided by the project financiers, which helped foster understanding between KenGen and the financiers. Perhaps most significantly, PATRP helped KenGen draft its Community Engagement Strategy (CES), which enabled KenGen to effectively engage with the Olkaria communities that would be affected by the geothermal project. The CES helped KenGen demonstrate to community members how communities in other jurisdictions have worked well with utilities to ensure projects result in win-win circumstances. The CES is greatly valued by KenGen, and other stakeholders in the sector have reportedly recognized a need for their own CESs. Kipeto Wind Power Plant (KWPP, 100 MW) Key informants widely acknowledged the KWPP as PATRP’s biggest success story in Kenya. PATRP’s support for this transaction demonstrates how effective the transaction-focused approach can be when the efforts of embedded TA are collaboratively defined and effectively targeted. However, the KWPP project ran into significant, simultaneous problems that threatened to derail progress toward FC. The first dilemma resulted from the spillover effect of backlash from the local community at the site of another wind power project being financed by Kipeto’s primary shareholder. For political reasons, the need arose for the initial primary shareholder to be replaced if the project was to reach FC. During this turbulent period, PATRP advocated on behalf of the project developer to maintain support for the project from government, regulatory and financial stakeholders. Key informants saw this advocacy as invaluable and a significant reason why the PPA was not re-opened for negotiation, and why US lenders did not withdraw their funding for the project. Meanwhile, in 2015 the status of two endangered species of raptors that are native to the proposed Kipeto site area was downgraded from “endangered” to “critically endangered.” At the request of the new majority shareholder, Actis, PATRP supported the drafting of a Biodiversity Action Plan to identify and mitigate the risks that the wind farm posed to those raptor populations. PATRP also helped produce the National Strategic Environmental Assessment for Windpower and Biodiversity in Kenya, which helped guide the Kipeto project’s environmental risk management and will continue to serve as a resource to the sector. Early and frequent community consultation throughout this process was crucial to ensure that the findings of the Biodiversity Action Plan and environmental assessment were accepted by nongovernmental organizations and conservation advocates. As was the case at Lake Turkana, KWPP’s success was jeopardized by delays in building the transmission line. Lenders did not want to commit funds for the project until it was clear that the transmission line would be built and that the grid could support the load from KWPP. Again, PATRP’s work on the GMSP and the System Integration Study helped address the lenders’ concerns and secure the project funding. 9 https://qz.com/africa/1700925/kenyas-huge-wind-power-project-in-turkana-hurts-local-people/. 21 Finally, the project needed a government letter of support (GLOS) in order to be implemented. PATRP TAs were instrumental in wording the GLOS in a way that satisfied lenders. Once the letter was drafted, there was a lengthy delay in obtaining an opinion from the attorney general on the GLOS. A PATRPs TA leveraged local connections to expedite receipt of the opinion, which cleared the way for the project to move ahead. Cummins Baringo Biomass Plant (8.4 MW) The Cummins Baringo Biomass Plant was an innovative project that promised to create multiple benefits for Baringo County. The plant was designed to convert a local invasive weed, methenge, into electricity, heat, and residual char.10 It was anticipated that harvesting the weed would employ 2,500 local workers and open land for agricultural purposes. PATRP provided technical and financial support to the transaction, including the project feasibility study and negotiation of the PPA with government officials. However, the biomass plant is not yet in commercial operation, because the methenge feedstock is incompatible with the plant’s technology.11 The evaluation team was not able to meet with anyone who was directly involved with the project, but the current status suggests there must have been flaws in the project screening process. Olkaria I Unit VI (83.3 MW) This expansion of the Olkaria I geothermal power complex reached FC in December 2018; as of early 2019, PATRP was still providing TA services to this transaction. PATRP’s main contribution was providing technical support to KenGen staff for a detailed capital expenditure optimization during the project design phase. This support not only strengthened the financial viability of the Olkaria I expansion but also strengthened KenGen’s capacity and desire to undertake similar exercises on all future geothermal projects. 4.2 PILLAR 2: CONNECTIONS PATRP’s BTG partners reported adding over 559,000 new off-grid connections in Kenya in 2017 and 2018. These connections are attributed to PATRP’s support of 14 projects during this time. Figure 4.1 shows the breakdown of these connections by technology. The numbers are based on the total connections recorded in the PATT database as of the end of October 2019. 10https://ke.usembassy.gov/cummins-cogeneration-kenya-limited-cckl-12mw-biomass-fueled-on-grid-electricity-generation￾project-in-baringo-county-kenya/ 11https://www.nation.co.ke/counties/baringo/Baringo-residents-hopes-for-income-fade-Mathenge-factory-stalls/3444812- 4752950-15njw8d/index.html 22 Figure 4.1: Off-Grid Connections Added in Kenya, by Technology The evaluation team attempted to meet with representatives from several off-grid energy technology providers that received support from PATRP, but ultimately met with only one. However, the evaluation team received additional input on the off-grid sector from the Energy and Petroleum Regulatory Authority and financiers. From these perspectives, PATRP’s presence was regarded as useful for proactively facilitating connections and information sharing among stakeholders in the off-grid sector; several described PATRP as a “connector-of-dots.” Key informants felt that this type of intangible support helped de-mystify the off-grid market for investors, while simultaneously helping off-grid providers attract investment and adapt to changes in the evolving regulatory framework. Although not a direct form of support to off-grid providers, PATRP’s efforts developing the mini-grid regulation under the new Grid Code was acknowledged and appreciated. The Power Africa Enabling Environment Data Tracker12 shows that Kenya now has the highest ranking among Power Africa countries for the strength of its mini-grid regulatory framework. The new regulation was not yet gazetted at the time of this evaluation, and key informants commented that off-grid providers, particularly smaller companies, will face challenges navigating the new regulation. 4.3 PILLAR 3: POLICY REFORM PATRP supported a wide variety of activities that fall under the category of “enabling environment” or “policy.” Highlights of these activities that were discussed with key informants are described in Table 4.1. 12 “Power Africa Enabling Environment Data Tracker.” USAID Data Services, 2019. 23 Table 4.1: Policy Reform Work in Kenya Institution Area of PATRP Support Perceived Outcomes and Sustainability Kenya Power ● 5-year Corporate Strategic Plan ● Helping Kenya Power look at opportunities for cooperation with other utilities at the international level, new focus areas for business ● Contract extension was needed to sort out strategies and initiatives in key areas ● Use restrictions and lack of training on executive management software is a current barrier to effective implementation of the Strategic Plan KenGen ● Good to Great Transformation Strategy ● CES ● Olkaria VI PPP ● Identified seven new priority initiatives that are being followed through on to increase competitiveness ● Built capacity to engage with communities to ensure projects are win-win ● KenGen now has the capacity to analyze and understand PPP documents, providing a path for increased private investment; seeing more interest from US firms KETRACO ● Wheeling fee calculation methodology, model, and agreement document drafts ● Transmission line PPP strategy paper and workshop ● Improved comprehension of wheeling arrangements, which will assist in formulating wheeling tariffs for other regional transactions. ● Enabled KETRACO to identify the best structure for PPPs in transmission lines, identify screening criteria and understand the implications of risk allocation. Energy and Petroleum Regulatory Authority ● GMSP ● Tariff Reduction Plan ● Anticipated gazetting of the new mini-grid regulation will encourage greater participation in the off-grid market ● Revision committee has been formed to look at emerging issues and incorporate them into the new Grid Code MoE ● Kenya National Electrification Strategy ● PPP Framework ● KOSAP ● High-level Sector Finance Steering Committee ● IPP task force ● Moe’s understanding of private sector alternatives to traditional debt facilities improved ● Drafting of technology-specific PPAs is anticipated to cut negotiation times and improve IPPs’ risk allocation ● Convening power of financial TA was integral to getting buy-in from top-level stakeholders on the Sector Finance Steering Committee 24 5. FINDINGS FROM ETHIOPIA Approximately 86 percent of Ethiopia’s 4.3-GW installed capacity comes from hydropower.13 The remaining 14 percent of capacity is provided by renewable (8 percent) and thermal (6 percent) plants. Although the country maintained a relatively low retail tariff rate (3 cents per kilowatt-hour) for many years, electrical connectivity was at 23 percent (8 percent for rural citizens) when Power Africa began providing support to the sector. The country’s hydro-dominated energy mix has been severely affected by drought, leading to power cuts as it struggles to serve a population of over 100 million and meet growing electricity demand, which is forecasted to grow about 30 percent per year.14 The sector is overseen by the Ministry of Water, Irrigation and Electricity (MoWIE), which is in charge of the development, planning, and management of the country’s energy resources, and oversees the four main public electricity sector institutions. The Ethiopian Energy Authority (EEA) is Ethiopia’s energy sector regulatory body. EEA is responsible for issuing licenses; reviewing and approving PPAs; proposing tariff methodologies and levels for government approval; and overseeing the generation, transmission, and distribution of electricity to domestic and foreign consumers. Ethiopian Electric Power (EEP) is the fully state-owned enterprise responsible for constructing and operating the country’s generation, transmission, and sub-transmission infrastructure. EEP’s distribution counterpart is the Ethiopian Electric Utility (EEU), which is also fully state-owned and handles day-to-day operation and maintenance of Ethiopia’s distribution network, billing and collection, client relations, and associated functions. The Government of Ethiopia has set ambitious generation capacity expansion targets, with a goal of over 10,000 MW of installed capacity by 2022 and expanded regional power exports.15 The government recognizes the need to encourage private investment in the sector to achieve these goals, but has limited experience negotiating deals with IPPs and is wary of signing bad deals. From the perspective of IPPs and financiers, high commercial losses, low collections, and outdated tariff rates have prevented EEU from achieving creditworthiness. In the off-grid sector, the foreign exchange shortage and complex regulatory environment make it challenging for small off-grid technology providers to become profitable. 13 “Power Africa in Ethiopia.” Power Africa, March 2016. 14 “Ethiopia—Energy.” The International Trade Administration, October 2019. 15 Tsagas, Ilias. “Ethiopia’s first 100 MW auction to conclude in June; scaling solar tender to follow in the summer.” PV Magazine, June 2017. 25 Ethiopia Overview Population ~ 110 million GDP per Capita $1,902.50 Map from: The World Factbook (CIA) Generation Capacity ~4,300 MW Peak Demand ~5,000 MW Capacity per Capita 38 watts Annual Consumption per Capita 69 kWh Key Issues 1. Grid power supply volatility 2. Limited understanding of project finance in the public sector 3. Public utility has not been creditworthy 4. Forex and regulatory obstacles hinder the off-grid sector 5.1 PILLAR 1: GENERATION PATRP provided technical advisory support to several large transactions in Ethiopia. However, none reached FC. Corbetti and Tulu Moye Geothermal (2x150 MW) PATRP provided a technical and financial advisory team to EEP to support negotiations with Tulu Moye and Corbetti Geothermal on two four-phase, 500+-MW geothermal power plants. PATRP TAs worked with the African Development Bank to secure funding to retain external legal counsel for the transactions and helped structure and negotiate the PPAs and implementation agreements (IAs). It was during the process of developing a management framework for the Corbetti project that the Government of Ethiopia recognized a need to create a broader framework for geothermal projects rather than negotiating frameworks on a project-by-project basis. This gap in the enabling environment was addressed through the drafting of the Geothermal Proclamation, which separates geothermal law from mining law and clears the way for more efficient negotiation of geothermal projects in the future. The negotiations for Corbetti and Tulu Moye were delayed during and after the drafting of the Geothermal Proclamation. The IAs developed under the mining law contained elements that were inconsistent with the new geothermal law, so the Government of Ethiopia worked with the IPPs to align the IAs with the new legislation.16 During this time, the projects were scaled back to two phases totaling 150 MW each. PATRP trained representatives from government and publicly owned enterprises in project finance, but high administrative turnover has left gaps in understanding of project finance in these institutions. 16 Richter, Alexander. “Corbetti geothermal project resolving issues with Government of Ethiopia.” Think Geoenergy, August 2017. 26 Despite these setbacks, Corbetti and Tulu Moye reached commercial close in 2017. However, nearly two years later FC has not been achieved and both projects face pressure from the government to re-negotiate the PPAs. (Corbetti initially signed a conditional PPA in 2015, one month before President Obama was to visit Ethiopia.) Metehara Solar (100 MW) The Metehara Solar PV Power Plant project is one of five Ethiopian projects under the International Finance Corporation-funded Scaling Solar initiative. Metehara Solar was the first large-scale solar IPP tendered by the Ethiopian government, but it has not closed due to inefficiencies in the tendering and procurement process and a significant flaw in land procurement process. EEP approached PATRP for assistance to competitively tender Metehara Solar, since EEP had no prior experience negotiating an IPP contract. EEP was rushed to issue a request for proposals (RFP) in two to three months, so the tender documents were issued before they were complete, with the expectation that EEP would finalize them while IPPs prepared their bids. During this time, EEP negotiated with the Ministry of Finance and Economic Cooperation and the National Bank of Ethiopia, which led to adjustments in the RFP documents and delays of eight to nine months before IPPs could submit their bids. The project was eventually awarded to ENEL Green Power, but it was soon discovered that the site that selected for the project could no longer be used. The land authority for Oromia Region was not properly consulted during the RFP process, and local communities did not want the project implemented on their land. PATRP provided guidance to EEP on best practices in environmental compliance and helped prepare an Environmental and Social Impact Assessment, which informed the selection of the new site. It is not clear why PATRP’s technical advisory team did not identify the land procurement issue before awarding the contract. The need to identify an alternative site led to significant and costly delays. The project continues to face substantial challenges. The International Finance Corporation withdrew from financing the project, and the Government of Ethiopia expressed interest in renegotiating the terms of its agreement with ENEL. Multiple stakeholders stated that such a move by the government would send a strongly negative signal to the market, with potentially lasting negative consequences for private sector involvement in Ethiopia’s power sector. 5.2 PILLAR 2: CONNECTIONS PATRP’s main contribution to increasing on-grid connections in Ethiopia was through its support to EEU with the pilot Meter 2 Cash (M2C) program. PATRP’s utility reform team undertook a diagnostic assessment of the utility’s commercial operations and identified several significant problems limiting EEU’s ability to accurately bill customers and generate revenue: Lack of meter reading oversight—Key informants reported that, although meter inspections were taking place, there was no supervisor to hold meter readers accountable and validate the meter readings. Missing, damaged, and corrupt meters—EEU had no processes in place to identify and repair or replace non-functional or missing meters. Poor customer service and collection policies and procedures—Key informants stated that the main barrier to connecting new customers is not the connection fee, but rather wait time; there is a backlog of 20,000 to 30,000 customers waiting to be connected to the grid. Most customers are required 27 to pay their monthly electricity bills at a payment center by a specific date. However, EEU was not following up with unpaid accounts for collection or disconnection. Low capacity for energy accounting—EEU staff lacked the technical knowledge to perform energy accounting calculations, and meters were not installed at the distribution transformer (DT) level to enable such calculations. PATRP worked with EEU management and staff to address these problems through the M2C pilot program in the SAAR. Key informants were aware that, by targeting these problems, PATRP’s support to the utility produced significant improvements in collection in the region. Indeed, based on PATT database records, EEU saw over 13,000 new connections in SAAR and an increase in revenue of over $5 million during the two-year pilot program. However, it is unclear how much of the increased revenue can be attributed to the M2C program, as other factors, including an increase in EEU’s residential tariff rates in December 2018, may have increased EEU’s revenue. It is not possible to quantify loss reductions since meters were not installed at DTs during implementation of the program. The increase in EEU’s residential tariff rates in December 2018 was the first of four annual increases through 2021. PATRP advocated for and provided capacity building to EEU, EEP, EEA, and MoWIE on the tariff review exercise that led to MoWIE’s authorization of the tariff increases. Table 5.1 shows the increases in the EEU’s tariff rates over the four-year period. Informants saw PATRP’s support to the tariff review as another important contribution to enhancing EEU’s commercial viability, and that of Ethiopia’s electricity sector as a whole. 28 Table 5.1: EEU’s Tariff Amendment Schedule17 December 2018 Onward December 2019 Onward December 2020 Onward December 2021 Onward Tariff Category kWh/Month Birr/kWh 1. Residential Tariff Block 1.1 1st block Up to 50 kWh 0.273 0.273 0.273 0.273 1.2 2nd block Up to 100 kWh 0.459 0.562 0.664 0.767 1.3 3rd block Up to 200 kWh 0.781 1.062 1.344 1.625 1.4 4th block Up to 300 kWh 0.913 1.275 1.638 2 1.5 5th block Up to 400 kWh 0.975 1.383 1.792 2.2 1.6 6th block Up to 500 kWh 1.042 1.497 1.951 2.405 1.7 7th block Above 500 kWh 1.141 1.588 2.034 2.481 2. General Tariff 2.1 Flat rate 1.035 1.398 1.761 2.124 3. Low-Voltage Industry Tariff 3.1 Flat rate 0.816 1.054 1.293 1.531 3.2 Demand charge rate 50 100 150 200 4. Medium-Voltage Industry Tariff 15 kilovolts (kv) and 33 kv 4.1 Flat rate 0.605 0.801 0.997 1.193 4.2 Demand charge rate 36.885 73.77 110.655 147.54 17 “Current Tariffs.” Ethiopian Electric Utility, 2019. 29 December 2018 Onward December 2019 Onward December 2020 Onward December 2021 Onward Tariff Category kWh/Month Birr/kWh 5. High-Voltage Industry Tariff Above 66 kv 5.1 Flat rate 0.517 0.654 0.791 0.928 5.2 Demand charge rate 21.91 43.82 65.73 87.64 6. Streetlight Tariff 6.1 Flat rate 1.035 1.398 1.761 2.124 7. Bulk Supply Tariff 7.1 Demand charge rate per kWh 39.291 78.582 117.872 157.16 7.2 Generation tariff per kWh 0.222 0.444 0.665 0.887 PATRP’s support to the off-grid sector in Ethiopia was hindered in the early stages by a primary focus on mini-grids rather than SHS. One PATRP staff member expressed frustration at the initial focus on mini￾grids and suggested that Ethiopia would have achieved more off-grid connections if SHS had been considered earlier. In fact, PATRP began providing support to SHS suppliers in 2017. The PATT database lists 1,662 off-grid connections—all of them SHS—attributed to PATRP support. Despite recording a relatively small number of off-grid connections, the key informants spoke highly of the assistance provided by PATRP TAs. One small solar company reported receiving input from a PATRP TA that supported on company’s successful application to the Off-Grid Energy Challenge Grant program. Key informants from this company also reported that PATRP provided technical support on its flagship mini-grid project, including for tariff-setting. The successful mini-grid project connected 198 households, and the company was selected as the IPP for four of 12 mini-grids tendered by the government under a pilot program. Key informants from both small solar companies with which the evaluation team met suggested that there is a need for advocacy to improve the off-grid regulatory environment. The foreign currency and exchange (Forex) shortage in Ethiopia was cited as one of—if not the most—significant barriers to growth as a small solar company. PATRP started work on a program that seeks to leverage remittance funds to pay for SHS; this and other creative solutions to the Forex shortage are necessary if Ethiopia’s off-grid sector is to grow. 30 5.3 PILLAR 3: POLICY REFORM PATRP supported a wide variety of activities that fall under the category of “enabling environment” or “policy.” Highlights of these activities that were discussed with key informants are described in Table 5.2. Table 5.2: Policy Reform Work in Ethiopia Institution Area of PATRP Support Perceived Outcomes and Sustainability EEP Hydropower plant diagnostic assessment Clearer understanding of operational issues at hydropower plants, and the potential improvements that can be achieved EEU Institutional gender assessment EEU worked with the World Bank to develop a Gender Mainstreaming Policy and Procedure18 as a result of PATRP’s recommendations EEA GMSP Tariff review Tariff rates have been increasing gradually over four years, beginning December 2018 The outputs of GMSP have the potential to improve operation of the national grid and capacity to forecast demand, but outcomes are yet to be seen MoWIE Geothermal Proclamation PPP Proclamation Energy law amendments New legislation and amendments to existing energy law provide a framework for improved negotiation of geothermal projects specifically and IPP projects more generally 18 “Strengthening Gender Equality and Increasing Women’s Participation in the Ethiopian Power Sector.” Power Africa, December 2017. 31 6. FINDINGS FROM NIGERIA Nigeria is the most populous country in Africa with a population of roughly 200 million. Despite its abundant oil and gas resources, Nigeria lags significantly behind most of the world in energy access and consumption. Many of the numerous reasons for gap directly contextualize the work of PATRP. Nigeria’s electric power sector is characterized by significant challenges and inefficiencies related to distribution and transmission, as well as the overall macroeconomic challenges and liquidity shortages that plague the country. While there is significant unfilled demand for electric power, additional generation is unlikely to improve the difficulties of the power sector in a meaningful way until the overall health and performance of the transmission and distribution sub-sectors improve. ATC&C losses in Nigeria are high. In the first quarter of 2019, DISCO collection efficiency stood at 64 percent, while remittance to Nigerian Bulk Electricity Trading Plc. (NBET) and market operators was 27.8 percent. This inefficiency strains the power sector, since the revenue shortfall significantly limits NBET’s ability to commit to new power purchasing agreements. PATRP embedded technical advisors in Nigerian DISCOs and other key institutions in an attempt to address these and other related concerns. In April 2018, PATRP transferred TA and transmission support to a new implementing partner, NPSP, which is managed by Deloitte. The new program places increased emphasis on reforms to the enabling environment and targeted policy work, which are necessary to address serious bottlenecks throughout the Nigerian energy sector. Supporting transactions to add generation capacity has reduced urgency until the overall enabling environment improves. Nigeria Overview Population ~200 million GDP per Capita $2,028.18 Map from: The World Factbook (CIA) Generation Capacity 5 GW Peak Demand >5 GW Capacity per Capita 25 watts Annual Consumption per Capita 145 KwH Key Issues 1. Pervasive financial issues throughout the system due to ATC&C losses and non-cost-reflective tariffs. 2. Infrastructure issues in transmission and distribution subsectors, tied to lack of capital 3. Low energy penetration, especially in rural areas 4. Underdeveloped gas sector that is unable to capitalize on Nigeria’s resource wealth 32 6.1 PILLAR 1: GENERATION In interviews, most relevant stakeholders considered PATRP’s overall impact on generation generally underwhelming. Only one major deal (Azura-Edo) reached FC. Stakeholders who were interviewed believed some of the factors that enabled the deal to close were not likely to be replicated. It appeared that certain allowances were provided to Azura-Edo, given the relative novelty of the IPP in Nigeria. Azura-Edo Gas Plant (459 MW) Azura-Edo Gas-Fired Power Plant Phase 1 (459 MW) is an open-cycle gas-turbine power station. According to the World Bank, “Azura-Edo is the first wholly project-financed IPP in Nigeria. As such, it is regarded as a ground-breaking project set to pave the way and set important benchmarks for future private sector driven, project financed IPPs in Nigeria.”19 The Azura-Edo transaction is the only generation deal that reached FC in Nigeria during PATRPs tenure. As such, it has received significant attention, and hopes were high for it to serve as a model for future IPP projects. However, some actors in the Nigerian power sector suggested that the Azura Edo project benefitted from certain allowances and conditions that were unlikely to be afforded to future IPP projects. For example, Azura Edo received tax and fiscal considerations that, according to a former TA, “cannot and should not be repeated for future projects.” Political issues surrounding the project were also addressed directly through the White House; this is unlikely to be a sustainable strategy moving forward. The Azura-Edo power plant had a 72 percent dispatch rate for Q1 2019, approximately two percentage points above the industry average.20 “Azura got a lot of ‘cut-off’ because it was the first of its kind. Those exemptions could not and should not be replicated for other projects. There were transmission risks that were taken by the government just to give Azura the first go that would not be replicated in other projects.” —Former Transaction Advisor 19 Molle, Anthony, “Financial Solutions Brief for Azura-Edo IPP”, World Bank Group, 2018. 20 “Quarterly Report (2019 Q1)”, Nigerian Electricity Regulatory Commission. 33 Figure 6.1 maps the complex financing structure of the Azura Edo Transaction, which involved numerous actors, contracts, and guarantees. A TA can play a valuable role in such a transaction, liaising among entities and ensuring the proper documents are signed and circulated. Figure 6.1: Complexity in Financing Arrangements: Financial Structure of Azura Edo21 14 Stalled Solar IPPs (1,125 MW) In 2016, Nigeria signed PPAs worth $2.5 billion with 14 IPPs to add 1,125 MW of installed solar capacity to the national grid. However, the majority of these projects have yet to reach FC due to an ongoing dispute with NBET about the tariffs that were originally agreed upon through a presumably long-term fill￾in tariff structure. The 2016 PPAs had priced solar power at 11.5 cents per kWh; however, the federal government set the price at 7.5 cents per kWh.22 This is an example of how closing deals can be exercises in futility when the legal and regulatory enabling environment is seriously flawed. The PPAs were ultimately meaningless, and significant time, effort, and other resources were wasted for the majority of these IPP projects. 21 Molle, Anthony, “Financial Solutions Brief for Azura-Edo IPP”, World Bank Group, 2018. 22 Babalwa Bungane, “Nigerian Gov Continues to Battle Tariff Structure with Solar IPPs,” ESI-Africa.com (blog), October 5, 2018. 34 Table 6.1: Stalled Nigerian Solar IPPs with Obsolete PPAs Company Capacity State 1 Afrinergia Power Limited 50 MW Nasarawa 2 CT Cosmos Limited 70 MW Plateau 3 Pan Africa Solar 75 MW Katsina 4 Nigeria Solar Capital Partners 100 MW Bauchi 5 Motir Desable Limited 100 MW Nasarawa 6 Nova Scotia Power Devevelopment Ltd 80 MW Jigawa 7 Anjeed Innova Group 100 MW Kaduna 8 Nova Solar 5 Farm Limited 100 MW Katsina 9 KvK Power Limited 100 MW Sokoto 10 Middle Band Solar One Limited 100 MW Kogi 11 LR Aaron Power Limited 100 MW Abuja 12 En Africa 50 MW Kaduna 13 Quaint Abiba Power Limited 50 MW Kaduna 14 Oriental Renewable Solutions 50 MW Jigawa 6.2 PILLAR 2: CONNECTIONS The majority of the on-grid connections that PATRP claims to have established were located in Nigeria. (279,645). However, on further examination, it may not be clear whether these “new” connections are necessarily increasing access to electric power. Customers previously connected to the grid but not paying for power have been added to DISCOs customer records as a result of revenue protection exercises, which may account for many of these connections. The metering gap for end-user customers continues to be a key challenge for the electric power industry. Nigerian Energy Regulatory Commission (NERC) records indicate that, of 8,840,801 registered electricity customers, only 3,793,895 (42.9 percent) had been metered by the end of the first quarter of 2019. The remaining 57 percent still received estimated billing, which “has contributed to customer apathy towards payment for electricity.”23 DISCO Work Nigeria’s 11 DISCOs are a serious bottleneck in meeting the goal of improving access to affordable and reliable grid electricity. As part of PATRP’s work in Nigeria, advisors were embedded in four of the 23 “Quarterly Report (2019 Q1)”, Nigerian Electricity Regulatory Commission. 35 country’s 11 DISCOs. The aim of these assignments was to reduce average technical and commercial losses at the DISCO level, which can be as high as 60 percent. Of course, failures in cost recovery impose significant financial strain on the DISCOs and, as a result, impact their ability to make payments to the rest of the upstream electric power sector. Table 6.2: DISCO Collection Efficiency in Nigeria, 2019/Q124 DISCO Total Energy Received (GWh) Total Energy Billed (GWh) Billing Efficiency (%) 2019/Q1 2018/Q4 2019/Q1 2018/Q4 2019/Q1 2018/Q4 Change Abuja 1,002 955 782 739 78.0% 77.4% 0.7% Benin 680 631 575 517 84.6% 81.9% 2.6% Eko 873 856 762 707 87.3% 82.6% 4.7% Enugu 609 587 413 370 67.8% 63.0% 4.8% Ibadan 879 860 738 660 84.0% 76.7% 7.2% Ikeja 930 942 893 850 96.0% 90.2% 5.8% Jos 329 324 223 194 67.8% 59.9% 7.9% Kaduna 604 595 418 417 69.2% 70.1% -0.9% Kano 474 526 396 415 83.5% 78.9% 4.6% Port Harcourt 507 504 378 357 74.6% 70.8% 3.7% Yola 306 280 185 164 60.5% 58.6% 1.9% DISCOs 7,193 7,062 5,762 5,391 80.10% 76.30% 3.80% PATRP’s advisors addressed ATC&C losses by implementing revenue protection strategies. These included working with technicians to inspect power lines and residential connections, audit meters, and use other methods to check for electricity theft. “That’s been one of the major reasons for success: they were not office guys, you know, [Laughs] they were field guys. If you were in a meeting and a marketer or manager says I have not been able to reach this environment, they would say “let’s go together.” They were really hands-on.” —DISCO Representative 24 “Quarterly Report (2019 Q1)”, Nigerian Electricity Regulatory Commission. 36 DISCO representatives expressed the opinion that the in-the-field hands-on approach was crucial to the success of the program. Advisors would frequently go into the field to ensure that the training they had provided to technicians was being implemented correctly. Many embedded advisors had worked as police officers or lawyers, and this experience with enforcement was considered an asset for revenue protection efforts and preventing illegal hook-ups. The DISCOs’ ATC&C loss reduction is a major success story for Nigeria that could be emulated elsewhere if appropriate. While the overall performance for DISCOs in Nigeria is still poor, the PATRP intervention shows how utility viability can gradually improve with the right assistance. Slow and steady improvements can, over time, bring Nigeria’s distribution subsector in line with international standards. Risks posed by an unstable system make such a scenario less viable, underscoring the need for a multitude of improvements including systemic reform and sustainable training. “They broke down the whole commercial value chain, from when you contract a customer to when you are getting a bill, and we developed a strategy for all the pain points. So, for example, for customers that were not on our grid, they recommended interventions, for example from the revenue protection side. How do you do inspections to ensure that you bring all those free riders that are not connected to your grid? So, one of the key activities they focused on was revenue protection.” —DISCO Representative 37 Figure 6.2: Nigerian ATC&C Loss Reductions with PATRP25 25 “PATRP Annual Report 2018.” 38 7. FINDINGS FROM GHANA Ghana is an interesting case study among the diverse challenges that afflict energy sectors in Africa. The country has seen feast and famine in terms of electricity supply within the last decade. PATRP embedded two TAs in Ghana’s MoE from November 2014 to October 2019. The purpose of this assignment was to support the MoE and other sector entities in the context of the Government of Ghana’s stated desire to increase private sector investment in the power and natural gas sectors. PATRP’s effort was well-received by the MoE. Ghana Overview26 Population ~30 million GDP per Capita $2,202.3 Map from: The World Factbook (CIA) Generation Capacity ~5 GW Peak Demand ~2.7 GW Capacity per Capita ~167 watts Annual Consumption per Capita ~351 kWh Key Issues: 1. High cost of excess capacity threatens financial stability 2. Distribution companies’ inefficiencies 3. Many PPAs signed during dumsor are not competitive 4. Nascent gas market requires infrastructure and regulatory attention 26 https://data.worldbank.org/indicator/EG.ELC.ACCS.ZS?locations=GH 39 Dumsor [doom-sō], noun. 1. A period of time in which darkness is more prevalent than light. As of the writing of this report, Ghana’s installed generation capacity was nearly double the peak demand. However, this has not always been the case. In the past decade, Ghana has experienced numerous periods of generation capacity deficit, which had detrimental impacts on commercial and industrial activity and, more generally, on quality of life. These periods were due in part to an electricity generation mix that depended heavily on hydropower, which can fluctuate heavily, especially in periods of drought. During a period of shortages (2013–2015), known as dumsor the government signed numerous emergency PPAs to compensate for the lack of generation. However, as one stakeholder explained, these purchases were akin “to buying an umbrella during a thunderstorm,” meaning that the buyer was at the mercy of the supplier and could easily end up being gouged. The result has been excess capacity, which comes at a high price in the form of capacity charges (a price paid to a producer whether or not electricity is generated), resulting in significant impact on the government’s finances. Figure 7.1: Ghana Electricity Generation and Consumption, 2006–2016 40 “2,300 MW of the installed capacity has been contracted on a take-or-pay basis. On average, less than 40 percent of the contracted take-or-pay capacity is actually used, meaning that we are basically throwing away money by paying for the remaining 60 percent of excess capacity, which we do not actually consume. In monetary terms, what this means is that we are paying over half a billion US dollars or over GH¢2.5 billion annually for power generation capacity that we do not need.” —Ken Ofori-Atta, Minister of Finance, 2019 Mid-year Budget Review 7.1 PILLAR 1: GENERATION Kpone (350 MW) The Kpone IPP is a $900 million, 350 MW combined cycle gas turbine project located in the Tema power enclave to the east of Accra. The project reached FC in 2014, before PATRP’s TAs were embedded in the MoE. Key stakeholders explained to the evaluation team that negotiations had been ongoing for the project since it received a generation license in 2003. Thus, negotiations went on for more than a decade before PATRP began. PATRP helped the government develop a generic PPA template that may have helped close the project. However, as with all transactions, attributing the financial closure of the project to a specific activity or actor is difficult. Other stakeholders interviewed by the evaluation team pointed to the work of USAID’s African Infrastructure Program (AIP),27 which preceded PATRP, as a contributing factor in the closure of Kpone. AIP’s legal and financial support to the Electricity Company of Ghana (ECG) was of particular value in helping the offtaker better understand the needs of the private sector partners. According to the AFC website,28 Kpone is, among other things, ● The first private sector greenfield IPP to be project-financed in Ghana ● The first IPP in Ghana to obtain a generation license ● The first IPP to be developed on a build-operate-transfer basis, meaning that the plant will be returned to the Ghanaian government after 20 years of operation ● The largest privately financed IPP in SSA in the last 10 years Bridge Power (~400 MW over four stages) Bridge Power is a multi-stage, ~400 MW thermal generation deal with an expected overall cost of $953 million. The facility will use a mix of liquified petroleum gas and natural gas and operate in both standard and combined cycle modes in different stages of the project. Like the Kpone IPP, Bridge is sited in the Tema power enclave, where the majority of Ghana’s thermal generation is located. 27 AIP provided capacity building and late-stage transactional services on clean and conventional energy projects to countries in sub-Saharan Africa. AIP started in September 2008. 28 https://www.africafc.org/What-We-Do/Projects/Cenpower-Kpone-IPP.aspx 41 The first phase of the project reached FC under PATRP. However, based on the current conditions in Ghana, including a recent government moratorium against new PPAs or other contracts, many stakeholders we spoke with were not optimistic about prospects for the project in the near future. Some stakeholders mentioned the significant costs created by the perception of high political risk in response to the government's recent statements. As a result of the government’s threats to renegotiate contracts, financial stakeholders may require higher investment guarantees, which further increase the costs of new transactions and create a vicious cycle. Relocation of Karpowership (470 MW) In June 2014, during the period of emergency generation contracting, Ghana signed a PPA with Karpowership for two 225-MW dual-fuel barges to supply to ECG over ten years. These generation facilities were initially fired by heavy fuel oil, which costs significantly more than Ghana’s abundant supply of natural gas. In an attempt to save costs, the barges were relocated west to take advantage of the availability of natural gas in Takoradi. PATRP Tas embedded in the MoE wrote the original relocation proposal, which the cabinet approved. The project aimed to fully offtake 180 million standard cubic feet per day (MMscfd) of OCTP Sankofa gas, but was constrained due to infrastructure bottlenecks. PATRP assisted the MoE and contracting parties in contract negotiation and supervised implementation to ensure timely completion of the project. Karpowership relocated to Takoradi in August 2019. Connection to the grid was completed in September, with conversion to gas expected by December 2019. According to one estimate, as a result of reductions in imported heavy fuel oil consumption, this initiative will save nearly $100 million per year in fuel costs. 7.2 PILLAR 2: CONNECTIONS The PATT database lists 8,452 new connections in Ghana attributable to PATRP, although this number may not reflect Power Africa’s overall achievement. The number is relatively small compared to the targets set for PATRP and achievements in other countries. More than 90 percent of the new connections were in the form of SHS, with the remaining through microgrids and solar lanterns. PATRPs efforts in this area were primarily associated with its BTG effort. Two embedded advisors began work there in late 2017, but the program was active in Ghana before its on-the ground presence. Off-Grid Solar The evaluation team spoke with some representatives of off-grid solar companies that had worked with PATRP. Stakeholders generally appreciated the support they had received and the networking opportunities that PATRP created, although they did not always explicitly link PATRP support to the creation of new connections. Gender work with one company was particularly well received, and the evaluation team heard from interviewees that other firms are introducing gender-specific interventions. As a result of interest in gender work, new financing opportunities arose. More than $10 million was raised, due in part to the events that Power Africa helped set in motion. Some important feedback concerned challenges present in Ghana’s mini-grid regulatory environment. Current regulation of mini-grids is not adequately developed, and mini-grids are subject to many of the same rules as on-grid activities. This makes the investment viability of mini-grid electrification especially challenging, given the heavy subsidy for lifeline energy consumers. While some mini-grid commercial 42 activities have been permitted to continue, the regulatory framework makes it impossible for the nascent industry to expand further, a significant roadblock that is an ongoing area for Power Africa in Ghana. As in Nigeria, company representatives with whom we spoke mentioned that access to affordable capital is the main constraint to scaling up. Although the BTG program attempted to connect some companies with financing, none of these efforts had yet resulted in new financial partnerships. 7.3 PILLAR 3: POLICY WORK Reverse Flow of West African Gas Pipeline (WAGP) PATRP TAs embedded at the MoE were instrumental in effecting the WAGP flow reversal, also known as the Takoradi–Tema Interconnection Project. WAGP allows natural gas to flow from Takoradi, in the west, where the majority of Ghana’s natural gas is processed, to Tema, in the east, where most thermal energy generation projects are being implemented. This work will significantly improve the reliable supply of affordable natural gas to Ghana’s power producers, thereby improveing the overall health of the energy sector by reducing the cost of fuel for generation. PATRP TAs assisted the MoE to evaluate, approve, and contract the construction of the Tema Takoradi Interconnection Project, which will allows the transport of up to 120 MMscfd of gas from Takoradi to the Tema power enclave over the existing delivery capacity, which can fuel roughly 600 MW of generation. PATRP TAs produced or provided: ● A concept paper ● A cabinet memorandum ● Negotiation support for the construction management agreement between the West African Pipeline Company (WAPCO) and Eni ● Negotiation support for the gas transportation agreement between Ghana National Petroleum Company and WAPCO ● Negotiation support for gas supply agreements between the Ghana National Petroleum Company and gas power plants Two of the three stages of the project were completed in April 2019, enabling 60 MMscfd of gas to flow from to Tema. The third stage is underway with completion expected by December 2019. At that time, the full flow of up to 120 MMscfd is expected to result in annual fuel cost savings of approximately $250 million. Stakeholders considered PATRP TAs valuable assets for the reverse flow work because of their technical knowledge and project management ability. The evaluation team heard that undertakings of this nature, which involve the coordination of multiple private and public sector actors, benefit from a third party that can keep the project moving forward. 43 Figure 7.2: The Tema Takoradi Interconnection Project (TTIP, or Reverse Flow) Power Distribution Services (PDS)-ECG Privatizatio ECG's losses are exacerbated by high technical losses; poor revenue collection, from government entities and consumers; and rising dollar-denominated payment obligations. Tariff policies that provide subsidies to consumers have damaged the financial health of ECG and NEDCo. 29 The Millennium Challenge Corporation (MCC) agreed in 2014 to provide $498 million in funding to Ghana’s power sector to help stimulate further private investment. One reform under the agreement handed over operations at state-run ECG in March to Ghana PDS, a consortium led by the Philippines￾based electricity company Meralco. On March 1, 2019, PDS took over electricity distribution and collection operations from ECG under a 20-year concession agreement. Ghana’s finance minister, however, informed US officials that the government was cancelling the 20-year concession, saying the payment guarantees provided were not satisfactory. In a statement, the US Embassy said the decision to terminate the contract was unjustified 29 Aaron Yaw Ahali. Improving Electricity Access in Ghana Challenges and the Way Forward. International Journal of Energy and Power Engineering. Special Issue: Electricity Market. Vol. 5, No. 2-2, 2016, pp. 9–17. doi: 10.11648/j.ijepe.s.2016050202.12 44 and that the MCC was therefore cancelling a second tranche of $190 million.30 The Privatization effort was not a PATRP initiative, although the embedded TAs were in communication with key stakeholders given the privatization effort’s strong relevance to the entire sector. Gas Master Plan (GMP) PATRP TAs supported the then-Ministry of Petroleum to obtain, and later implement, the GMP—a 25- year road map of Ghana’s strategic development and natural gas resources management. TAs initially reviewed a document submitted by a World Bank-funded consultant. The TA’s submitted a report indicating that the plan was generally sound and suggested only minor improvements. The plan was then workshopped through a MoE committee and submitted to the Cabinet. A PATRP TA who was part of the implementation committee was selected as the primary resource, with responsibilities including drafting a new pricing policy, as suggested in the GMP. The new policy is expected to save up to $430 million over the next four years. The evaluation team heard that the GMP stalled after the election of a new government in Ghana in 2017. A revised plan was submitted, but it remains unapproved. Energy Sector Recovery Program (ESRP) The Ghanaian energy sector is afflicted with serious financial difficulties, in part as a result of expensive PPAs that were typically signed during periods of energy insecurity that have affected the country over the last two decades. Another contributing factor is been foreign exchange losses, as generation costs are billed in US funds but the distributors charge consumers in Ghanaian Cedis, which have depreciated against the dollar. Other issues include the high cost of fuel, non-cost-reflective tariffs, transmission and distribution losses, and low recoverability of consumed power cost by distribution companies, especially ECG. To address ongoing financial challenges in the power sector, Ghana is implementing an ESRP—a time￾bound, milestone-based set of over 20 reform measures intended to help reduce the cycle of debt in the power sector, from generation to distribution, over the next five years. The ESRP relies heavily on financial analysis conducted by PATRP embedded advisors, who made key part contributions to ESRP arrangements and implementation. Time will tell how effective this program will be, but MoE stakeholders clearly considered the embedded PATRP TAs valuable assets. 30 “Ghana Loses $190 Million U.S. Grant over Canceled Power Contract.” Reuters, October 23, 2019. https://www.reuters.com/article/us-usa-ghana-power-idUSKBN1X225D 45 SPOTLIGHT: POWER SECTOR CASH FLOW MODEL IN GHANA One of the major issues faced by Ghana power sector is financial instability. As a result of poor forecasting, the country signed an excessive number of PPAs during the period of load shedding in the first half of the decade. Expensive contracts led to annual shortfalls in excess of $1 billion, causing an existential threat to the financial viability of the sector and placing a significant burden on the country's finances. One of PATRP’s embedded advisors at the MoE in Accra led the development of a series of models that can be used to calculate key financial indicators and map connections among actors in the sector, including: ● IPPS ● Generation companies ● DISCOs By defining the detailed financial structure of the industry, these models improve understanding of the sector’s financial woes. The models can be used to conduct sensitivity analysis (a way of analyzing the impact of different policy approaches) and to understand the consequences of possible scenarios that could contextualize the sector in the future. Stakeholders told the evaluation team that the power sector models were instrumental in the creation of the ESRP reform package. Several commented that modelling is one of P most impactful outputs that PATRP delivered in terms of potential long-term impact on the Ghanaian power sector. This kind of technical support, if not otherwise provided, can add significant value to a government or struggling energy sector. In future programming in Ghana and elsewhere, Power Africa could consider ways to integrate this kind of cash flow modelling into its programming from early stages to inform the goals set in each country and the work plan. 46 8. OVERARCHING FINDINGS EQ 1. TO WHAT EXTENT WAS THE TRANSACTION-FOCUSED APPROACH TO ADVANCING MW DEALS SUCCESSFUL? “In sub-Saharan Africa, a power project can stall for a variety of reasons. By addressing the critical impediments that hinder progress of a particular deal, we create a ripple effect for other deals facing similar issues in the country. Our approach to reforming Africa’s energy sector is to tackle one power project at a time.” — “About Us: Power Africa.” U.S. Agency for International Development, https://www.usaid.gov/powerafrica/aboutus. Defining Success Answering EQ 1 will require a functional definition for “success” as it is used in the question. Among the ways one could judge the “success” of the program are whether: 1. PATRP achieved contractual targets for generation capacity, connections, and other key performance indicators 2. PATRP’s transaction-focused approach generated positive results for each country of operation that are considered good value relative to the resources the program was provided and in the context of each country’s unique energy challenges 3. The program improved the overall health of the energy sector, the general level of prosperity, or the quality of life of citizens in its operational countries The answer to each of these questions can differ, underscoring the need to carefully explain what is meant by “success.” Success can be measured on a number of axes, including achievement of performance metrics, resource efficiency, the closing of transactions, quality and reliability, and improvement in the overall health of the power sectors in each country. 47 Looking at the Quantitative Data Table 8.1: All Financially Closed Transactions in Evaluation Countries FY Country Project Name Transaction Technology MW 2015 Ghana Kpone Independent Power Plant Natural Gas 350.0 2019 Ghana Early Power Phase 1 Natural Gas 202.0 2015 Kenya Lake Turkana Wind 310.0 2017 Kenya Olkaria V Geothermal 158.0 2019 Kenya Kipeto Wind 100.0 2019 Kenya Malindi Solar 40.0 2020 Kenya Olkaria 1 Unit 6 Geothermal 83.3 2016 Nigeria Azura-Edo Natural Gas 459.0 Figure 8.1: Generation Capacity Reached FC (MW), by Country and Technology PATRP saw eight significant generation transactions reach FC during its operation in the four countries considered in this evaluation, with a combined generation capacity of just over 1,700 MW. More than half of this capacity came through three large gas projects in Ghana and Nigeria, and the rest from a more diverse mix of renewable transactions in Kenya. No transactions reached FC in Ethiopia during the program period. Given that approximately 1,700 MW reached FC, we might consider the outcome of the transaction￾centered approach a qualified success. If the overall goal of the PATRP program was to get 6,421.2 MW of generation capacity to FC between 2014 and 2018, and if we consider that the countries in our evaluation are only four of more than 20 countries where PATRP operated, this achievement might seem 48 proportionate. However, the four countries also received the highest level of PATRP engagement and had dedicated TAs. In this light, perhaps 1,700 MW is not a very good outcome. It is also important consider the nature of the added MW. In Ghana and Nigeria, all generation capacity was added through three large natural gas thermal generation projects. Natural gas is somewhat cleaner than oil or other heavy fuels but is not truly “clean” and not renewable. The accomplishments in Kenya are more impressive. Kenya added 691 MW to its generation capacity with a mix of geothermal, solar, and wind—all renewable sources of electricity. The fact that multiple generation deals were closed across various technologies, seems to indicate that PATRP’s approach can be successful in the right circumstances. In Ethiopia, no generation deals were closed with PATRP’s assistance. This might imply that PATRP’s approach was a failure; however, numerous stakeholders expressed gratitude for the support provided and commented that, despite deals not reaching FC, the focus on transactions helped stimulate and guide policy reform work in numerous ways. Thus, although the transaction-focused approach may not have been a success as measured in MW, it may have made tangible improvements to the sector overall. “The approach that they take is extremely important. They take a broad transaction approach. They don’t just give you money. They know that to address the difficulties in the power sector, you need to look at it broadly across the chain. They do transmission, they do distribution, they do MW, which is all important to address the problems of the African electricity sector.” —Private Partner Attribution of Closed Deals to PATRP In answering EQ 1, the evaluation team had difficulty attributing the main outcomes measured (MW) to the actions of PATRP TAs. In many cases, the stakeholders told us that, while TAs were often helpful and easy to work with, it was not always possible to identify whether their involvement pushed a specific project toward FC. Many of the projects PATRP counts toward its overall achievements were valued at hundreds of millions of dollars and involved dozens of stakeholders involved from institutions around the world. While TAs may have helped with some part of the process, it is possible in many cases that a deal might have reached FC even without the PATRP program. In some cases, especially for projects that reached FC in PATRP’s early years, key stakeholders often cited previous USAID initiatives as the primary sources of assistance with which they were familiar. For example, in Nigeria, we heard that USAID programming helped close the Azura Edo deal. However, multiple stakeholders referred to AIP program activities, not PATRP. It is difficult to attribute success to one program rather than the other, especially given that many of AIP personnel transitioned to PATRP. Ultimately, this evaluation is not designed to answer the attribution question conclusively. However, there are clear examples of transactions in which PATRP was instrumental, if not necessarily essential in all cases. 49 “Without that support, I think I can say we probably wouldn’t have reached financial close, or it would have at least been more difficult. It would be much more difficult for the offtaker to understand what is needed by a private entity.” —IPP Developer Table 8.2: Challenges and Success Factors for TAs Key Factors for TA Success Challenges for TA Success ● TAs’ affiliation to a third party with nothing to gain established confidence among project stakeholders, particularly those with little prior exposure to project finance or the local regulatory environment. ● TAs added capacity in resource-constrained environments ● A mix of foreign and local technical experience enabled TAs to effectively diagnose problems and propose best-practice solutions ● TAs with strong political acumen and longstanding local influence were capable of strategically overcoming barriers to FC ● Enabling environment for private sector participation in the sector was not sufficiently mature for TAs’ influence to be fully accepted and acted on ● Misalignment of expectations, ambiguity of objectives or lack of buy-in from receivers of PATRP support led to underutilization of TA services Key Factors for Expeditiously Closing Transactions 1. Enabling environment ○ Openness to private sector participation ○ Existing supply of and demand for electricity in the relevant country 2. Good TAs ○ Knowledge of project finance ○ Ability to build trusting relationships 3. Government Support ○ High-level support for the Azura Edo deal (White House) Major Challenges in Closing Transactions 1. Lack of alignment with government o Azura Edo had government holdout o In Ethiopia, the government's priority is hydropower o Early/bridge power is currently under government review 50 o NBET signed PPAs with solar IPPs in Nigeria, but these have not been able to close due to government opinion that the price is not right 2. High costs associated with risk mitigation o Put-call option agreements (PCOA) needed in many cases o Insurance costs add significantly to project financing 3. Changing conditions o Election of new governments o Renegotiated PPAs o Constant decreases in solar prices “If you want more MW, you need open, transparent, and streamlined procurement, where everyone knows the process, rather than bilateral projects where everyone has a deal with a minister and every PPA is unique.” —-Former Transaction Advisor EQ 2. HOW EFFECTIVE WAS PATRP’S APPROACH TO STRENGTHENING UTILITY VIABILITY, AS EVIDENT IN THEIRWORK IN NIGERIA AND ETHIOPIA? PATRP’s work with distribution utilities in Nigeria and Ethiopia demonstrated the potential benefits of embedding advisors to support utilities. However, serious challenges remain, especially for DISCOs in Nigeria. Nigerian DISCOs Decreased ATC&C Losses and Increased Revenues PATRP’s utility support in Nigeria succeeded in reducing ATC&C losses at four of the 11 state DISCOs that distribute Nigeria’s electricity. A team of advisors devised business plans for the DISCOs and then worked with management and staff to implement commercial loss reductions. The program was considered a success. However, DISCOs are still a key bottleneck for the Nigerian energy sector; even while the evaluation team was on the ground in Nigeria, NERC was threatening to revoke their licenses. The DISCOs have not paid NBET in full for the electricity they obtain from the Transmission Company of Nigeria, increasing cash flow issues in the sector. The piloted approach to DISCOs’ loss reduction may be one way to improve their commercial performance, but the sector may be past the point of recovery. Improvements that are too gradual may result in collapse—or reform. A top-down regulatory reform project may be necessary before significant resources can be invested in technical capacity. “If you just add more generation, but the wires are still damaged, you are pouring water into a bucket with holes.” —DISCO Representative The Ethiopian Pilot was Successful, Although Some Data is Lacking EEU collections rate was reported at about 60 percent when PATRP began providing support. The project worked with EEU management to conduct a diagnostic analysis of operations and maintenance gaps at the 51 utility. The assessment found that, while many necessary policies and procedural structures were in place, not all were being followed. For example, although EEU employed meter inspectors, there was no supervisor to hold inspectors accountable. As a result, meter readings were often erroneous or unvalidated, and inoperable meters were not repaired. EEU also identified gaps in customer service policies and procedures, which limited the utility’s ability to collect revenue from customers. In light of these issues, PATRP worked with EEU under the M2C pilot program in the SAAR region to build the capacity of EEU staff. The training sessions improved meter reading practices and lines of accountability within the utility. PATRP also identified DTs where meters should be installed to improve EEU’s capacity to perform energy accounting activities. EEU followed up on this recommendation by installing meters at substations throughout Addis Ababa. While EEU stakeholders attributed increased revenue to PATRP support, it is certain that this was entirely the case. PATRP has not been involved with other initiatives to increase EEU revenue, such as prepaid metering. Furthermore, is impossible to quantify PATRP's impact since meters were not installed at DTs in SAAR throughout PATRP implementation. Challenges in Utility Work 1. Perceived aggressiveness—Teams of advisors were embedded with Nigerian DISCOs with the goal of improving commercial performance. We heard that advisors’ “aggressive” performance management approach contrasted with the previously “relaxed” culture at DISCOs. Some perceived a “punishing” approach as advisors worked to introduce a culture of accountability, and their departure was celebrated because “now, nobody is watching us.” However, this attitude apparently subsided as the value of the advisors’ approach became apparent. 2. Cultural and communication issues—Key informants described communication issues with some embedded advisors. Communication sometimes suffered when English was not their first language. Additionally, advisors took an informal approach to titles, which could offend counterparts and strain communication. However, in time these issues were resolved through increased familiarity and understanding, underscoring the benefits of sustained interventions. 3. Disagreement with project pace—Embedded advisors had targets and goals to achieve within a relatively short tenure (two years). However, the DISCOs they worked with were not limited by this timeframe and felt the advisors’ pace was too aggressive. However, stakeholders believed that the timeframe ultimately improved performance management. “In Africa we like titles, so you’d want to be addressed as “engineer” or whatever your title is. When someone calls you by your first name, you get offended, so you don’t even listen to what they are saying because you’re offended.[...] But as we got used to each other that barrier was overcome.” —DISCO Representative Did PATRPs’ Utility Work Affect Connections or Generation Capacity? Fonitoring and evaluation data corresponding to the PMP found that PATRP added more than 500,000 new on-grid actual direct connections in Nigeria through work with DISCOs. However, PATRP embedded advisors’ work was primarily in revenue protection, which in many cases addressed energy theft by 52 eliminating the connections of non-paying customers. It was not clear that the work with DISCOs was actually been designed to increase connections. Discussions with stakeholders and some secondary sources revealed that many of the connections that DISCOs added were accounts that had been cut off for stealing electricity and subsequently added as paying customers. Therefore, “new connections” may more accurately refer to new customers. This metric then also ignores the reduction in customers that resulted from revenue protection efforts. AEDC alone saw more than 600,000 disconnections during the period when it received provided assistance.31 This is not to suggest that energy theft should not be addressed, or that adding customers is not a valuable benefit; ATC&C losses are extremely high, and commercial losses comprise the majority of their value. Rather, it is useful to understand the nuances of this potentially confusing metric. EQ 3. HOW EFFECTIVE WAS PATRP’S APPROACH TO EXPANDING OFF-GRID CONNECTIONS? As in other areas of the evaluation, the evaluation team heard success stories tied to BTG work under PATRP, as well as challenges that prevented the program from being more effective. The evaluation team sees room for improvement, particularly with regard to promotion of off-grid connections. Looking at the Data According to the PATT database, 1.1 million actual connections were recorded. Figure 8.2 shows that more than half of these connections were recorded in Kenya—more than the next 14 countries listed. Figure 8.2: Actual Connections, by Country and Technology32 31AEDC Board Presentation from Tetra Tech, April 26, 2017. 32 PATT Database 53 PATRPs BTG Program Created Networks and Helped “Connect the Dots” Working with individual companies and trade associations, PATRP TAs provided frequent, reliable and helpful information to sector stakeholders at no cost on topics such as emerging regulations, risks, and opportunities, and provided this information. Some of the smaller companies appreciated the opportunity to build their organizational capacity by collaborating with PATRP TAs. Multiple stakeholders used the phrase “connected the dots” to describe the impact of PATRP’s presence in the off-grid sector. Although PATRP support did not always do so, its work connecting stakeholders should not be underappreciated in a market populated with small companies working to overcome evolving challenges with limited resources. Representatives of solar companies and Stanbic in Kenya noted that PATRP helped facilitate investment in that type of business. Respondents from Stanbic said PATRP helped de-mystify the sector and understand risks and opportunities in the emerging market. Small solar providers in Ethiopia said that PATRP helped them connect with investors, develop creative business models to overcome the Forex shortage, and obtain input on funding applications and investment pitch slide decks. Methodological Issues in Measuring New Off-Grid Connections Measurement of off-grid connections is plagued by attribution issues. PATRP defines “anticipated connections” in a given year as the number of sales of a certain type of off-grid system made by the company in the base year before PATRP started supporting it. PATRP then defines “actual connections” as the actual number of sales in a given year when the company received PATRP support. The number of connections attributed to PATRP support is calculated as the difference between actual and anticipated connections, or the incremental number of sales above or below the number of sales reported in the base year. Attribution of connections to PATRP does not depend on whether the support was major or minor and does not account for other factors that might influence changes in a company’s sales over time. Figure 8.3 explains Power Africa's connections measurement methodology, as described in an internal memo shared with the evaluation team. Figure 8.3 Connections Methodology OPERATIONS SUPPORT TO SHS/LANTERN COMPANIES PATRP calculates connections achieved from operational support to companies through a baseline methodology. PATRP’s attributable connections are calculated each quarter over the reporting period (between the start and end dates of our support), using the following formula: 𝑃𝐴𝑇𝑅𝑃 𝐴𝑐𝑡𝑢𝑎𝑙 𝐶𝑜𝑛𝑛𝑒𝑐𝑡𝑖𝑜𝑛𝑠 = 𝑆𝑎𝑙𝑒𝑠34563 − 𝑆𝑎𝑙𝑒𝑠89 Where,𝑆𝑎𝑙𝑒𝑠34563 is the total quarterly sales from the reporting period and𝑆𝑎𝑙𝑒𝑠89 is the total quarterly from the baseline period for the corresponding quarter. In the case of a newly established company (i.e., a zero baseline), this calculation yields 100 percent attribution of sales to PATRP. FINANCING SUPPORT TO SHS/LANTERN COMPANIES When PATRP supports a specific financial transaction (e.g., a fundraising round), we can claim 100 percent of the sales attributable to that particular transaction. 54 MICRO-GRID SITE DEVELOPMENT Micro-grid sites are treated as traditional power projects, with PATRP claiming 100 percent of new connections at the site. Increasing Off-Grid Electricity Connections Stakeholders commented on progress in increasing BTG connections in Kenya, particularly though PATRP support for BTG connections for off-grid pilot projects and solar home system installations. Similarly, in Ethiopia PATRP successfully supported several small renewable energy start-ups. However, many challenges remain for solar start-up companies in Ethiopia, including Forex constraints, lack of adequate working capital, and difficulties associated with obtaining approval to import solar systems and associated components. A common theme for the future is the need for coaching and advisory support to the trade association of small solar companies, which expected to grow from 20 to 70 members by early next year. There is also a need to provide a detailed road map to small renewable energy start-up companies on how to navigate the governmental import and approval process. EQ 4. HOW EFFECTIVE WAS PATRP’S APPROACH TO POLICY REFORM? WHICH PATRP INTERVENTIONS HAD THE MOST SIGNIFICANT IMPACT ON POLICY REFORM, AND WHY? Based on what the evaluation team heard from key stakeholders, some of PATRP's greatest successes may have been related to policy work. Embedded Advisors Embedded advisors were the key to policy success. Stakeholders described the TA’s role as that of an independent party who can interact with the government and private sector. Advisors embedded with government or state-owned enterprises were highly valued, key informants told us. The most successful TA were typically embedded for significant periods; indeed, longer engagements resulted in more informed recommendations. “It took me at least nine months to figure out how to do the job.” —Embedded Transaction Advisor Some stakeholders expressed frustration about advisors who flew in for temporary engagements. Stakeholders recommended the use of local advisors with international experience, when possible, as a way to retain knowledge within the country. Challenges A common impediment to policy work was lack of governmental support or alignment. In Nigeria, Ghana, and Ethiopia, changes in government during PATRP implementation stalled a number of policy initiatives. For example, in Ghana, significant work went into the review and revision of the GMP, which was resubmitted after the change of government in 2017. However, the plan has not yet been approved. In 55 Ethiopia, major efforts were made to sensitize government officials on project finance but, while they understood the need to encourage private investment in the power sector, they were wary of entering into bad deals. The change of administration in 2017 had created the need to reinforce training and awareness among government staff. “If you don’t have the full commitment of the government, you have a big hill to climb. In Nigeria, that was the biggest challenge. It always seemed like you were rowing upstream.” —Former Transaction Advisor Government Involvement and Alignment ● A common theme in East Africa was that PATRP interventions at the technical working committee and mid-management levels were very effective, but that the project should have pursued greater interventions at the higher political level to “open their eyes.” Many mid- to upper-level managers with whom the team spoke at state-owned utilities commented that PATRP needed to involve politicians more—not just technocrats who understand the problems and issues in the sector. ● Many government stakeholders also saw a greater need for longer-term interventions and study tours to observe similar emerging markets in countries that have successfully implemented policy reforms. Such activities would promote new thinking among key government decision-makers. In this regard, some stakeholders felt that PATRP did not spend enough time bringing the whole management onboard with respect to several policy reform initiatives. ● Stakeholders in Kenya felt abandoned after PATRP delivered the PPP framework, which was initially well received, but lacked continuing support for implementation. EQ 5. WHAT WERE THE POSITIVE AND NEGATIVE CONSEQUENCES OF ACHIEVING TARGETS? WERE THOSE INTENTIONAL? The PATRP program had numerous targets, of which the most commonly mentioned were the number of MW reaching FC and the number of new connections. Other targets included risk measures used, transmission lines added, employees trained, and policy reforms written and adopted. Positive Consequences of Targets PATRP had three general goals: helping generation transactions reach FC, increasing access to electricity by expanding the number of households with connections, and improving the overall enabling environment of the sector through policy reform work. Program targets aligned with these areas imperfectly, but in a general sense directed mostly positive efforts. The PATRP program worked with late-stage transactions in an effort to meet their targets for generation capacity reaching FC and, in some cases, the program was considered instrumental to achievement of FC. Negative Consequences of Generation Targets and Focus Not all transactions give good value for money, and adding MWs of generation capacity to the grid can have both positive and negative impacts on the overall health of a country’s power sector. For example: 56 ● Adding capacity in Ghana may have further burdened an already overwhelmed electricity market. ● The Azura gas deal in Nigeria may have benefited from certain allowances that created an unrealistic standard for future PPA pricing in the sector. More transparent, competitive procurement processes could help address this potential issue. ● In Kenya, Cummins Baringo Biomass Plant technology in was not properly vetted, and feedstock was incompatible with technology. As a result the plant is not operational. ● Emphasis was on MW of generation capacity without consideration of affordability. If an electricity supply is not affordable, additional generation capacity is irrelevant. Consequences of Policy Target The consequences of policy targets are unclear to the evaluation team. It is difficult to find evidence that the policy work done by PATRP was in any way contextualized by a need to meet quantitative targets for policy reforms. Setting a number for policies proposed and implemented appears to be a generally ineffective approach that could be replaced by a target with a clearer relationship to impact, or to better measures of input (level of effort, et.), which could at least direct resource allocation. “If you get too stuck on metrics, then you get into very specific activities.” —Former Transaction Advisor “MWs is a good target in ‘Africa,’ where there is a deficit everywhere, so you want to put in as many MW as possible, but you need to go beyond the numbers to see what success is. Here [in Ghana] the economy is reeling under the costs of all these MW we’ve got. In my mind, in Ghana, success is minimizing the cost, or optimizing the use, or building capacity, or building the capability, so that in the future we don’t get into this mess again. You can’t have a cookie-cutter approach where you say ‘Let’s build more power plants’ where no power plants are needed.” —IPP CEO EQ 6. IS THERE EVIDENCE THAT PATRP’S ACHIEVED RESULTS HAD A WIDER EFFECT THAN ANTICIPATED? IF SO, IN WHAT WAY? Given the broad scope of the PATRP program and its ambitious approach to everything from generation capacity to policy reform, it is setting a high bar to ask if the project “had a wider effect than anticipated.” Some stakeholders believed the program produced success beyond their expectations, while others disagreed. The PATRP Results Framework quotes the Development Objective: “to increase the quality and quantity of transmission, distribution, and cleaner generation projects, to expand access to electricity and spur economic growth.” Intermediate results were expected to add actors to the sector; attract new investment opportunities; reduce risk and transaction costs; and improve technical and planning capacity, the viability of financial utilities, and regulatory frameworks. Once we account for these results, not many others remain. However, several results are standouts: 57 The Karpower Relocation in Ghana Reduced Costs Significantly In Ghana, TAs proposed the relocation of the Karpowership generation facility, which is expected to save the country at least $100 million in coming years. While such policy proposals are not exactly outside the scope of the program, the success of this proposal represents the kind of impact that might not have been predictable or anticipated at the start of the PATRP program. Input on Geothermal and PPP Proclamations is Paving New Paths in Ethiopia In Ethiopia, TAs provided valuable input on the geothermal and PPP proclamations, paving the way for improved transactions in all sectors. PATRP helped draft the Geothermal Proclamation, which stemmed Corbetti geothermal transaction. Geothermal law was previously included under the Mining law, which required a unique management framework to be developed for each geothermal project. The Geothermal Proclamation eliminates the need for separate management frameworks, which reducing the time needed to negotiate future geothermal projects. PATRP also provided support to but was not directly involved in drafting the PPP Proclamation, which provides previously missing legislation on structuring PPPs and led to the creation of a PPP unit within Ethiopia’s Ministry of Finance and Economic Cooperation. Both the PPP Proclamation and PPP Unit are significant achievements that institutionalize some of the knowledge that PATRP transferred to the Government of Ethiopia. Lesser Effect than Anticipated in Nigeria Overall, Despite Some Successes with DISCOs In Nigeria, the program was generally viewed as having failed to meet expectations, given that no generation capacity was added after the closing of Azura-Edo, in addition to many remaining challenges in the sector. However, the success of the DISCOs’ embedded advisors was welcome—and perhaps surprising in this context. “I think it's been tough for me, as someone who’s worked in Nigeria for many years, knowing that it has the 7th largest population in the world, and yet only has 4000 MW of reliable capacity. From that standpoint, the program did not exceed expectations. We didn’t get them to 8000 MW.” —Former Transaction Advisor EQ 7. WHAT IS THE POSSIBILITY OF POSITIVE EFFECTS BEING LIKELY TO CONTINUE AFTER THE PROGRAM HAS ENDED? Sustainability is difficult to forecast. The PATRP program made numerous impacts on the power sectors in supported countries, some of which will continue into the future. For example, generation deals that were closed during PATRP should have a longer impact window than the program itself due to the contractual mechanisms underlying PPAs and other guarantees. Many of the policy reforms should also have an extended impact, assuming some level of stability around their implementation. However, in some cases the sustainability of impacts is uncertain. For example, the sustainability of utility loss improvements in Nigeria, sustainability is threatened by significant instability in the country’s energy system, which could pose existential threats to future utility operation. If the health of the Nigerian energy sector does not improve, the benefits (and sustained impact) of PATRP support could be undone by future changes to the system. Similarly, the sustainability of utility improvements in Ethiopia is seen as dependent on implementation of activities under the East Africa Energy Program. 58 Embedded Advisors and Sustainability Embedded advisors can be a valuable resource for ministries, utilities, regulators, and other entities at the heart of a developing country’s energy sector. Those advisors often have technical and administrative capacities that significantly improve an organization’s ability to achieve its goals. However, embedded assignments are temporary, and if an organization integrates an advisor into its core operation, the end of the assignment can be difficult. Ideally, advisors should be able to operate independently of the entity in which they are embedded so core functions are not impacted by their presence and they can pursue projects outside the mandate of the partner organization. However, embedded advisors typically form close working relationships in the entity that may blur the distinction between advisory work and providing an extra pair of hands. In many cases, the quickest way to achieve objectives is for skilled TAs to do that others may not be doing, or may do poorly. If this becomes a pattern, the entity may come to depend on the advisor’s skills or output; which could be an issue if major reforms or negotiations are incomplete when TAs finish their assignments. Ensuring that well-structured transitional plans are agreed to well in advance of advisors finishing their engagements can limit this issue. “I think that concept of embedded advisors really, really helped. They gave you a theory and then they practicalized it. They tell you disconnect like this. You go with them to the field with you and they say, ‘When I said disconnect like this, this is what I meant.’ So it sticks” —DISCO Staff EQ 8. TO WHAT EXTENT WERE PROGRAM ACTIVITIES ADAPTED FOR THE LOCAL CONTEXT? The problems addressed by PATRP varied significantly across countries given the varied challenges that contextualize different energy sectors. Overall, the evaluation team saw significant evidence that PATRP staff were allowed the flexibility to address the challenges in front of them, and that the program was able to tailor support by adapting program activities to the local context. In Ghana TAs worked to address significant financial issues in the sector by supporting ESRP development, proposing a cost-saving relocation of an existing generation facility, creating a set of well-used financial models, and other policy efforts that key stakeholders valued highly. There was less imminent pressure to focus on closing transactions, given Ghana’s generation overcapacity. While the embedded TAs were involved with the transaction pipeline, they rightly prioritized policy work. Stakeholders in Ghana observed that the country’s needs are mostly related to optimizing existing generation capacity, which advisors accomplished through policy work related to the fuel supply as well as In Nigeria TAs supported attempts to launch gas flaring auctions, although progress has been limited. Generally, the enabling environment in Nigeria made it difficult to advance transactions. As a result, the subsequent NPSP program shifted its focus to the enabling environment. 59 EQ 9. HOW WELL DID THE PROGRAM DEVELOP AND MAKE USE OF M&E SYSTEMS? The evaluation team had concerns about some of the metrics and key performance indicators found in the PMP. These are discussed below. Generation Capacity Metrics Do Not Sufficiently Reflect PATRP Impact Generation capacity to reach FC is an insufficient metric to capture program success because it equates all generation efforts—regardless of value, price, political context, and overall impact on the health of each country’s electricity sector. While it has the benefit of simplicity, this metric does not sufficiently align with the key metrics that sector planners and policymakers look at (resource adequacy, affordability, and reliability). In most cases, the reason for such low consumption of electricity in SSA is not simply that generation capacity, measured in MW, is insufficient. In Ghana, generation capacity in MW is nearly double peak demand. In Nigeria, the contracted generation capacity in MW is approximately 13 GW, yet only 4 GW are delivered to customers as a result of a number of issues related to transmission and distribution, regulation, and political interference. This criticism is addressed to some degree in the more recent Power Africa 2.0 strategy, which broadens the focus of the program to a larger group of issues in the power sector, including transmission. It Is Difficult to Accurately Measure and Attribute Connections that Resulted from PATRP Connections as a measure is also somewhat problematic, or at least unclear. In some cases, DISCOs that helped disconnect non-paying electricity users were subsequently used to report new on-grid connections despite a net-negative impact on electrical connections when customers were “regularized.” Similarly, for much of BTG off-grid work, estimates of new connections made dubious assumptions about the attributable impact of PATRP support, often claiming all new connections created by a partner business that provided relatively limited support. This most likely led to an overestimate of the impact of Power Africa’s effect on new connections. The evaluation team acknowledges that accurately measuring attributable connections would be difficult and, if done in a methodologically valid way, quite costly. However, given that connections are one of the main targets for Power Africa, decision-makers should consider whether they are comfortable with the current methodology for measuring program impact. Alternative ways to quantify success include: 1) setting targets for the percentage of people who can access electricity in each country, which would capture policy work and other changes to the sector, or 2) measuring reductions in the cost of accessing electricity for strategically selected markets, which would better reflect improvements to the supply side of the market and demonstrate that electricity is becoming more accessible for the poor. Monitoring the Enabling Environment and Policy is Important, and there is Room for Improvement PATRP counted the number of policy proposals drafted and adopted but did not their evaluate quality, which is problematic. One well-targeted, high-quality policy will have a more significant impact than a numerous bad ones. The PATRP program saw some important successes in its work with governments, which are not well represented in current data. In future programming, one option to improve 60 measurement of policy and enabling environment achievements efforts would be to introduce a road map for each country, with energy sector issues identified, and reforms proposed. Then, rather than measuring the general number of policies proposed or moved into legislation, the monitoring and evaluation framework could measure progress toward overcoming major challenges. This would enable decision￾makers to better understand progress on the enabling environment and the performance of Power Africa related to policy reform. Decentralization Improved Access to Relevant Data for Key Decision-Makers In cases where new programming replaced PATRP, such as in Ghana or Nigeria, we heard multiple times that decentralization had improved the accessibility of useful resources or relevant data. CORs and other staff commented that the PATRP model of placing a centralized management team in Pretoria led to data sharing issues. 61 9. LESSONS AND RECOMMENDATIONS VARIED CONTEXTS ACROSS COUNTRIES REQUIRE TAILORED APPROACHES The PATRP program was implemented across multiple unique countries in SSA. It set overall goals for generation capacity, connections, and policy work, yet such aggregate measures did not capture many of the successes and failures that occurred in the unique context of each country. In Ghana, for example, over-contracting in recent years led to an excess supply and burdened government finances through capacity payments. In such a context, adding MWs of generation capacity might be considered an actively harmful outcome, despite the fact that it would be seen as a step toward achieving PATRP’s overall targets. Therefore, it would be more appropriate to endorse country-level objectives and targets, which could vary significantly for each country's power sector. Key objectives in Ghana, for example, could include 1. Oversee the privatization of ECG as it transitions to PDS 2. Oversee reforms to the natural gas regulatory framework to align with international best practices 3. Support the development and implementation of a mini-grid legal framework 4. Support the government in designing and operationalizing a full set of reforms for least-cost procurement of new energy resources 5. Support efforts to achieve a reduction in the electricity sector’s annual operating deficit In Nigeria, objectives could relate to financial insolvency among DISCOs or a much-needed review of the tariffs system to reflect costs. In each country, the highest priority needs will differ, as will the resources and time required to address them. Recommendation 1: Improve Country-Level Planning In future contracts that are similar in nature to PATRP, we recommend that Power Africa create a country-level plan that would contain: ● Country-level targets that address the unique context and obstacles in each country that impede the overall success of the power industry ● A road map to success for each country that identifies key milestones toward the development of its power sector ● An assessment of stakeholder support or buy-in for the country to ensure alignment ● An assessment of the resources necessary to accomplish the objectives in each country GOVERNMENT ALIGNMENT IS CRITICAL The PATRP program included a set of targets for success applied to its programming across SSA. However, each country where PATRP operates is a sovereign entity, and PATRP ultimately had no authority to 62 change its regulatory or legal frameworks. PATRP can offer support, advocate for specific policy reforms, or connect key stakeholders, but at the end of the day, the program delivered will fail if it does not align with the wishes of each country’s government. Government positions often change when new lawmakers are elected. In some of the countries we studied, PATRP faced challenges related to changes in government in the midst of the contract. Newly elected lawmakers were often reluctant to implement policies or regulatory reforms that were developed in consultation with previous regimes. In order to improve the overall impact of future efforts, USAID could consider aligning future programming with electoral cycles. Consultations could occur with new governments to develop objectives that satisfy the interests of both Power Africa and the governing body, assuming such overlap exists. This would reduce political risks that affect the overall success of policy work and ensure that Power Africa does not commit resources to work that does not align with the will of governments. If an elected government does not agree with Power Africa’s overall objectives or approach, it would be more effective to allocate aid resources where they are better able to achieve an unambiguously positive result. Some governmental stakeholders recommended that Power Africa endeavor to involve more political appointees in its transaction and policy interventions, possibly through the creation of, and support for, a senior-level coordination committee that meets regularly. Future transaction and policy intervention efforts should attempt to be more inclusive of host government stakeholders and be made at higher levels, particularly among political appointees. Recommendation 2: Define Objectives in Consultation with Governments We recommend that, for future contracts similar in nature to PATRP, Power Africa align its activities with the objectives of elected governments before committing resources to maximize the overall efficiency of its activities. Governments ultimately have jurisdiction over the regulatory and legal frameworks that control outcomes in the power sector; without their involvement, Power Africa is unable to generate significant results. Government consultations could include: ● Defining the key goals that governments have for the power sector during their tenure ● Assessing political abilities required to generate movement on key issues ● Signing memoranda of understanding to establish partnerships between Power Africa and government bodies ● Establishing which ministries, agencies, or other institutions are instrumental in overcoming key road blocks and possibly establishing embedded advisors in strategic locations NOT ALL MW AND CONNECTIONS ARE THE SAME, AND NOT ALL ARE BENEFICIAL The primary goal of Power Africa is defined as MW of generation capacity, as well as the number of new connections. While additional generation capacity is certainly needed in many parts of SSA, there are many locations where it is not a key bottleneck for economic prosperity. In some countries, there are significant issues with distribution or transmission subsectors where adding more generation will not solve underlying issues. In other countries, overcapacity has created significant financial challenges for public 63 sectors, adding to the financial pressures of the government. Similarly, connections to electricity are valuable in many contexts, but it is difficult to measure the number of connections attributable to Power Africa work, compared to those that were established by the many private partners in the energy sector. As a result, the estimates of new connections are highly disputable. On their own, neither metric is capable of capturing the programs overall impact without further contextualization. Recommendation 3: Improve the Alignment of Metrics with Impact We recommend that, in the future, Power Africa avoid giving contractors targets for generation capacity or connections. Power Africa could instead choose qualitative goals or milestones as targets for each country and then create an M&E system for each country that aligns with these goals. These could include: ● Country-level targets that address the unique context and obstacles in each country that restrict the overall success of the power industry ● More price-related targets. For example, what is the levelized cost of new generation added? What is the average cost of electricity in urban areas? What is the average price of electricity in rural areas? 64 ANNEX I: STATEMENT OF WORK 65 66 67 68 69 70 71 72 73 74 75 ANNEX 2: EVALUATION METHODS AND LIMITATIONS EVALUATION METHODS The goal of this study is to assess the progress of PATRP in improving the supply of and access to electricity in sub-Saharan Africa. To accomplish this task, the evaluation team has developed an evaluation approach that seeks to address a set of nine pre-subscribed evaluation questions provided by USAID (Annex 1).33 The evaluation team is proposing a mixed-methods approach to include a review of existing secondary data and semi-structured key informant interviews. This approach is reflective of USAID’s parallel combinations approach where two different approaches are used to collect and analyze information, which is then synthesized to answer individual evaluation questions.34 The performance evaluation is designed to answer research questions directly related to PATRP objectives two through four. To properly identify sample populations and data sources, the evaluation team has developed an evaluation design matrix linking each evaluation question to a PATRP objective (Annex 2). Of the nine pre-subscribed evaluation questions, numbers one through four are directly related to PATRP objectives. The remaining questions apply to participants or stakeholder under each objective and have been subsumed under a general question – Question 5: did PATRP produce impactful and accountable results? This approach has been taken to effectively communicate the evaluation approach and to identify potential stakeholders for interviewing. Table 1 presents a summary of the evaluation design matrix, including the main evaluation question, the associated program objective, and the key approaches to answering the question. Table 1: Summary of Evaluation Design Matrix Evaluation Question: Objective: Performance Evaluation Approach: Did PATRP's "transaction-focused" approach help generation deals reach financial closure? Objective 2: Late-Stage Transaction Support Conduct in-depth interviews with transaction partners, USG staff, and implementing partners Quantitative and qualitative review of the PATT database (Pillar I) for partner countries Review USAID administrative documents Did PATRP strengthen utility viability in Nigeria and Ethiopia? Objective 4: Regulatory and Institutional Strengthening and Policy Reform Conduct in-depth interviews with electric utilities, USG staff, and implementing partners 33 See Performance Evaluation Statement ofWork (Annex 2) 34 See USAID Technical Note: Conducting Mixed-Method Evaluations, Version I, June 2013. 76 Quantitative and qualitative review of the PATT database (Pillar 3) for partner countries Review of Nigeria Electricity Regulatory Commission (NERC) and EEA documents Review of Administrative Documents for Utilities in Nigeria and Ethiopia (e.g. load growth; losses) Review USAID administrative documents Did PATRP's work increase off-grid connections? Objective 3: Small Scale Projects and Rural Electrification / Mini￾Grids Support Conduct in-depth interviews with electric utilities, USG staff, and implementing partners Quantitative and qualitative review of the PATT database (Pillar 2) for partner countries Review administrative documents for off-grid energy providers (e.g. connections; financial viability) Review USAID administrative documents Did PATRP impact policy in a meaningful way? Objective 4: Regulatory and Institutional Strengthening and Policy Reform Conduct in-depth interviews with host-country officials, electric utilities, USG staff, and implementing partners Qualitative review of the PATT database (Pillar 3) for partner countries Review USAID administrative documents Did PATRP produce impactful and accountable results? Questions address the following for objectives 2 through 4: • Impact • Sustainability Conduct in-depth interviews with USG staff, partners, electric and off-grid energy providers, and implementing partners Review USAID administrative documents 77 • Local Adaptation • M&E The team has also designed a general questionnaire addressing evaluation questions that are pertinent to each stakeholder group (Annex 3). Evaluation questions five through nine, which are presented as one over-arching question in the Evaluation Design Matrix, are included in this general questionnaire with the understanding that some questions (e.g. M&E) will not be asked of all stakeholders. The performance evaluation will collect information about how PATRP activities were implemented with respect to each program objective. The evaluation team will assess if PATRP activities align with the stated goal and targets of improving the supply of and access to electricity through transaction-based support, policy reforms and institutional strengthening, and the provision of on-grid and BYG energy solutions. The performance evaluation will analyze various sources of information, including program and administrative documentation, reporting data from the PATT database, and qualitative research. A summary of key data sources includes: • Program materials will document the transactions, projects, and policy reforms supported by PATRP. Additionally, these sources will document key partner involvement with each activity. Examples include country investment plans (CIPs), quarterly and annual reports, and monitoring and evaluation (M&E) and performance monitoring plans (PMP). • Administrative documents from national electric utilities, national regulatory agencies, and off-grid energy providers will describe partner performance and progress towards common objectives. For example, annual reports from utilities and off-grid energy companies document customer growth and financial viability. Similarly, regulatory agencies often provide source materials regarding energy transactions and regulatory reforms. • The PATT database will provide source material for all Qualified Power Africa Transactions, including partner involvement for each PATRP-supported transaction or project. For instance, PATT data can be used to identify the size (MW), cost (US$), and technology (e.g. solar, gas, etc.) supported for each transaction. • The evaluation team will use qualitative data to understand how PATRP was implemented and how program partners and participants believe the program has met its objectives. o Key informant interviews with participants to PATRP-supported financial transactions will help the evaluation team understand the type of support provided to transactions and the perceived effectiveness of this approach. These interviews will focus on partners to transactions that are active or have achieved financial closure in Ethiopia, Ghana, Kenya, and Nigeria. o Key informant interviews with electric utilities and off-grid energy providers receiving PATRP support will help the evaluation team understand the type of support provided energy providers and perceived effectiveness towards increasing the supply of and access to energy. While interviews for off-grid energy providers will occur in each of the four 78 focus countries of this evaluation, interviews with the electric utilities will only occur in Ethiopia and Nigeria. o Key informant interviews with host-country governments receiving PATRP support will help the evaluation team understand the types of actions supporting policy reform and the perceived effectiveness of this support. These interviews will focus on host-country government officials in each of the four partner countries where PATRP-supported policy reforms occurred. o Key informant interview with USG staff and implementing partners will help the evaluation team understand the challenges to program implementation and M&E processes for tracking progress towards each objective. These interviews will focus on personnel based in South Africa, Washington, D.C., and each of the four focus countries of this evaluation. The performance evaluation will identify implementation successes and challenges, demonstrating lessons learned from the program to inform technical teams, activity managers, and senior management on the design of future programs of Power Africa. Furthermore, the performance evaluation will provide policy recommendations for the direction and structure of any future programming. A special emphasis will be placed on an assessment of the overall effectiveness of the activity’s transaction focused approach in improving the supply of and access to electricity in sub-Saharan Africa, including documentation of the value of replicating the transaction-focused model within the Agency. The evaluation team will also provide recommendations for monitoring, evaluation, and learning for future programming. STUDY POPULATION AND SAMPLE The performance evaluation will focus on the assessment of outcomes of major activities under PATRP objectives two through four. The performance evaluation will also address a series of questions assessing common outcomes across each of these objectives. Under the current design, the performance evaluation will focus on program participants, beneficiaries, and partners under each respective objective. Moreover, USAID has instructed the team to concentrate its assessment of evaluation question two in only Ethiopia and Nigeria, while the remaining evaluation questions will apply to PATRP activities in the following countries: Ethiopia, Ghana, Kenya, and Nigeria. Therefore, study populations will be based upon the identification of key PATRP activity stakeholders under each objective in the relevant partner countries. In Annexes 4 through 7, the evaluation team has identified a list of potential bilateral, multilateral, and private sector stakeholders to interview by country and PATRP objectives. For evaluation question one, which evaluates PATRP’s transactions-based approach, the initial study population will be identified through the assessment of the PATT database. Specifically, the evaluation team will estimate the total number of PATRP-supported projects and partners for Pillar I who reached financial close. After this population is identified, we will work with USAID to identify stakeholders to interview under each transaction According to the PATT database for Pillar I (Objective 2)35, there are currently 59 PATRP-supported transactions that are financially closed or active.36 Nigeria and Kenya had the highest number of PATRP￾assisted transactions (20 each), followed by Ghana (11) and Ethiopia (8). Of these 59 transactions, there are an estimated total of 137 partners, including actors from the private sector, bilateral and multilateral 35 Objective 2 represents the population for evaluation question 1. 36 Based on PATT estimates as of August 9th 2019. 79 organizations, and other organizations.37 Table 2 provides a summary of the total number of partners by technology and country. Solar and natural gas account for the highest share of PATRP-supported transactions at 38 percent and 30 percent, respectively. Kenya, Ghana and Nigeria all had over 40 PATRP￾assisted transactions, while Ethiopia had only 8. Table 2: Number of Partners for PATRP Transactions by Technology and Partner Country The evaluation team proposes the random selection of 50 percent of financially closed transactions in Ethiopia and Ghana, representing almost 10 transactions. In Kenya and Nigeria, the evaluation team proposes a random selection of 33 percent of financially closed transactions, which represents a total of 12 transactions. For each of these 22 transactions, the evaluation team will attempt to interview one government partner and one private sector partner. In total, this would produce 44 interviews related to PATRP’s transaction activities.38 The evaluation team will work with USAID for final approval of this list and introductions. The team also acknowledges the potential for non￾responses from counterparts and therefore the total number of interviews will likely be less than 44. For evaluation question 2, PATRP activities to strengthen utility viability in Nigeria and Ethiopia, the evaluation team will initially estimate the total study population by identifying utilities supported by PATRP in the PATT database, Pillar 3. The evaluation team will work with USAID and the implementing partner to identify at least 5 utilities in Nigeria and Ethiopia for interviews. For each utility, the team will attempt to interview at least 2 stakeholders with knowledge of the PATRP initiative. For evaluation question 3, PATRP's work to increase off-grid connections, initial estimates will be based upon the total number of PATRP-supported projects and partners in the PATT database, Pillar 2. According to the PATT database for Pillar 2 (Objective 3)39, there are currently 53 PATRP-supported projects. Of this, there are an estimated total of 121 partners, including off-grid energy providers, investors, and other donors.40 Kenya represents the partner country with the most PATRP support projects under Objective 3 (20), followed by Nigeria (14), Ethiopia (10), and Ghana (9). Table 3 provides a summary of the total number of partners by technology and country. Solar home systems account for largest share of PATRP-supported transactions at 76 percent, while the micro-grid and lanterns account for 17 percent and 7 percent, respectively. 37 This estimate does not exclude organizations that may be involved in multiple transactions in any one country. 38 137 partners / 59 PATRP transactions = 2.32 partners per transaction. Twenty-two transactions * 2 = 44 partner interviews (rounded). 39 Objective 3 represents the population for evaluation question 3. 40 This estimate does not exclude organizations that may be involved in multiple transactions in any one country. Technology Ethiopia Ghana Kenya Nigeria Total Wind 1 0 11 0 12 Solar 7 11 19 15 52 Natural Gas 0 21 0 20 41 Biomass 0 2 4 5 11 LPG 0 7 0 0 7 Hybrid RE/RE 0 0 6 3 9 Geothermal 0 0 2 0 2 Hydro 0 0 1 2 3 Total 8 41 43 45 137 80 Table 3: Number of PATRP-supported Projects for Objective 3 by Technology and Partner Country Country Ethiopia Ghana Kenya Nigeria Total # of Projects by Technology Lanterns 0 1 0 5 6 Solar Home Systems (SHS) 10 6 15 7 38 Microgrid 0 2 5 2 9 Total 10 9 20 14 53 # of Partners by Technology 53 Lanterns 0 4 0 5 9 Solar Home Systems (SHS) 32 17 35 8 92 Microgrid 0 4 14 2 20 Total 32 25 49 15 121 The evaluation team proposes the random selection of 33 percent of off-grid projects, representing around 16 BTG projects. For each of these 16 transactions, the evaluation team will attempt to interview one off-grid energy provider and another private sector partner (e.g. investors or bilateral donor). In total, this would produce around 32 interviews related to PATRP’s BTG activities. The evaluation team will verify the list of off-grid contacts with USAID. The team also acknowledges the potential for non-responses from counterparts and therefore the total number of interviews will likely be less than 32. For evaluation question 4, the evaluation will need direction from USAID regarding the host-country government agencies, regulatory institutions and policies receiving PATRP support. The team will ask USAID to work with PATRP in compiling a list of all relevant stakeholders that had an extensive engagement with the PATRP program. From this list, we will randomly select at least 10 stakeholders per country to reach out to for interviews. We anticipate that we will not be able to reach all stakeholders. LIMITATIONS Key informant interviews and site visits will be a major data source for the PATRP evaluation. The evaluation team will likely depend on USAID advisors and implementing partners to identify and communicate with some key stakeholders. There is thus some risk of selection bias due to the potential of selecting a large proportion of interviewees with only positive opinions of the program. Although the evaluation team will invite a variety of stakeholders to participate in the interview process, individual stakeholders are the ultimate decision-makers as to whether they would like to be interviewed. This creates the risk of interviewing a significant proportion of stakeholders who are motivated by their strong opinions about the program. PATRP is also near the end of its program lifecycle. This presents a risk of recall bias in that stakeholders may not remember how PATRP supported their projects or the extent to which this support was effective. There are also data collection challenges related to complexity of the program and the limited amount of time the evaluation team will spend in each country. The PATRP performance evaluation assesses three 81 objectives with distinct outcomes and stakeholder groups. Identifying and coordinating interviews with individuals representative of each PATRP objective will be challenging. Realizing these limitations, the evaluation team will work with USAID and implementing partners to conduct interviews with stakeholder groups representative of the greater population. The evaluation team will be cognizant of situations where biases may still exist and note this information in the PATRP performance evaluation. The team will mitigate the potential risk of recall bias by selecting key informant interviews representative of activities across the entire program lifecycle. Finally, the evaluation team will work diligently to identify and analyze secondary information that can be triangulated with data from key informant interviews. This will include the desk study performed prior to country visits. 82 ANNEX 3: DATA COLLECTION INSTRUMENTS KEY INFORMANT INTERVIEW SCRIPT - PATRP EVALUATION Introductions and Participant Consent “Thank you for taking the time to speak with me today about your experiences related to the PATRP program. My name is _________ and I am working with Integra Government Services, which has been contracted by the US Agency for International Development to collect data on the PATPR initiative to date. The purpose of our interview is to learn more about the effect of capacity building, technical assistance, and transaction support provided by the PATRP initiative. I encourage you to be open and honest in your reflections on both successes and challenges your organization has experienced while working with the PATRP initiative. Unless you specify otherwise, feedback will remain confidential and reported in aggregate. Before we begin with my questions - do you have any questions for me about this data collection effort?” Consent Do you understand the purpose of this interview and wish to proceed? ❏ Yes ❏ No Are you willing to speak on the record during the interview? ❏ Yes ❏ No Identification Country ❏ Ghana ❏ Nigeria ❏ Kenya ❏ Ethiopia ❏ Other ___________ Name of Key Informant (s): 83 Institution(s) Represented: Power Africa Transactions Affiliated with: EQ1 1.1 Which types of transaction support were provided by PATRP? No Yes, significant impact Yes, limited impact Yes, no impact N/A Drafting project documents Credit analysis for letters of comfort Feasibility studies Risk mitigation Assistance developing power procurement tenders 84 Other (manually specify) 1.2 What does it take to close a transaction in your country? How does this compare to PATRP’s past efforts? 1.3 Reflecting on your experience, do you feel that PATRP's involvement had a significant impact, had a limited impact, or had no impact on the closing of transactions? ❏ Significant impact ❏ Limited impact ❏ No impact 1.4 Why do you feel that PATRP's involvement had the impact (or lack of impact) noted in the previous question? 1.5 What key challenges did you and your organization encounter when closing PATRP transactions? 1.6 What kinds of solutions did you and your organization use to overcome challenges related to closing PATRP transactions? 1.7 How did PATRP support the solutions used to overcome these challenges? EQ2 2.1 From your observations, did PATRP's work lead to new grid connections? 85 ❏ Yes ❏ No ❏ N/A 2.2 From your observations, did PATRP's work lead to new electricity generation? ❏ Yes ❏ No ❏ N/A 2.3 To what extent did PATRP's work strengthen "the utility"? ❏ Helped overcome a financial barrier ❏ Helped build capacity ❏ Improved customer service ❏ Reduced losses ❏ Other (manually specify) ❏ Did not strengthen the utility 2.4 Are PATRP's achievements (e.g. loss reduction, system improvements) expected to be sustainable? ❏ Yes ❏ No ❏ Don't Know 2.5 Why do you feel that that PATRP's achievements are expected (or not expected) to be sustainable? 2.6 What key challenges did PATRP encounter in implementing this work? 2.7 How did it seek to overcome the previously mentioned challenges? 2.8 Was PATRP able to overcome these challenges? Please describe. EQ3 3.1 How effective were PATRP's efforts to bring lending institutions into the off-grid sector to finance off-grid companies (solar home system and micro-grid)? 86 ❏ Very Effective ❏ Somewhat Effective ❏ Ineffective 3.2 Why do you feel that that PATRP's efforts were effective (or not effective) in bringing lending institutions to the off-grid sector? 3.3 To what extent do off-grid companies feel they benefited from PATRP support in the following areas? No impact Yes, significant impact Yes, limited impact Yes, no impact N/A Capacity building Financial support Technical assistance Developing new partnerships or markets 3.4 What key challenges did PATRP encounter in implementing this work? 3.5 How did it seek to overcome the previously mentioned challenges? 3.6 Was PATRP able to overcome these challenges? Please describe. 87 EQ 4 4.1 What type of policy reform did PATRP support? ❏ Energy sector law ❏ Energy policy ❏ Strategies or plans ❏ Regulations ❏ Other (manually specify) 4.2 What were the outcomes of these policy reforms? 4.3 Reflecting on your experience, do you feel that PATRP's policy support had a significant and positive impact, a significant and negative impact, or an insignificant impact? ❏ A significant and positive impact ❏ A Significant and negative impact ❏ An Insignificant impact 4.4 Why do you feel that PATRP's policy support had the impact (positive or negative) noted in the previous question? 4.5 In your opinion, what is the most effective ways to support policy reforms? How does this compare to PATRP’s past efforts? EQ 5 5.1 What were the positive consequences of achieving targets? 5.2 What were the negative consequences of achieving targets? 5.3 Were these consequences intentional? Explain. 88 EQ 6 6.1 Did PATRP's achieved results exceed your expectations? ❏ Yes ❏ No 6.2 How did PATRP exceed (or not exceed) expectations as noted in the previous question? EQ 7 7.1 Do you believe the positive effects generated by the program are likely to continue beyond it's program completion? ❏ Yes ❏ No ❏ Unsure ❏ N/A EQ 8 8.1 Do you believe PATRP activities were well-adapted for the local context? ❏ Yes ❏ No ❏ Unsure ❏ N/A 8.2 Please elaborate. EQ 9 9.1 How well did the program make use of monitoring and evaluation systems? ❏ Used well ❏ Used poorly ❏ Did not use ❏ Don't know ❏ N/A 9.2 Please elaborate. 89 ANNEX 4: SOURCES OF INFORMATION BIBLIOGRAPHY OF DOCUMENTS REVIEWED “2018 Integrated Annual Report & Financial Statements.” Kenya Electricity Generating Company Plc, June 30, 2018. Https://Www.Kengen.Co.Ke/Images/Annual_reports/2018-full-year-kengen-integrated￾report.Pdf. “2020-2024 Performance Improvement Plan.” Abuja Electricity Distribution Plc, 2019. A. K. Ofosu Ahenkorah. “Integrated Power System Master Plan For Ghana.” Energy Commission, Usaid, December 6, 2018. Http://Www.Energycom.Gov.Gh/Files/Ghana%20integrated%20power%20system%20master%20plan%20 _volume%202.Pdf. Ahali, Aaron. “Improving Electricity Access In Ghana Challenges And The Way Forward.” International Journal Of Energy And Power Engineering 5 (January 2, 2016): 9–17. Https://Doi.Org/10.11648/J.Ijepe.S.2016050202.12. Ajayi, Adewale, Chibuzor Anyanechi, Segun Sowande, And Nike Oyewolu. “A Guide To The Nigerian Power Sector.” Kpmg Nigeria, September 2016. Https://Assets.Kpmg/Content/Dam/Kpmg/Ng/Pdf/Tax/A-guide-to-nigerian-power-sector.Pdf. Bungane, Babalwa. “Nigerian Gov Continues To Battle Tariff Structure With Solar Ipps.” Esi-africa.Com (Blog), October 5, 2018. Https://Www.Esi-africa.Com/Industry-sectors/Renewable-energy/Nigerian-gov￾continues-to-battle-tariff-structure-with-14-solar-ipps/. Chijoke Mama. “Solarplaza On-grid Solar In Nigeria: Two Years After The Ppas.” Accessed November 15, 2019. Https://Www.Solarplaza.Com/Channels/Markets/11772/-grid-solar-nigeria-two-years-after￾ppas/. Chinedu Nebo, And James F. Entwistle. Memorandum Of Understanding Between The Federal Government Of Nigeria And The Government Of The United States Of America Regarding Power Africa (2014). Https://Www.Usaid.Gov/Sites/Default/Files/Documents/1860/Nigeria-mou.Pdf. “Current Tariffs.” Ethiopian Electric Utility, 2019. http://www.eeu.gov.et/index.php/current-tariff Dioha, Michael O., And Nnaemeka Vincent Emodi. “Investigating the Impacts Of Energy Access Scenarios In The Nigerian Household Sector By 2030.” Resources 8, No. 3 (July 18, 2019): 127. Https://Doi.Org/10.3390/Resources8030127. Eberhard, Anton, Katharine Gratwick, Elvira Morella, And Pedro Antmann. “Independent Power Projects In Sub-saharan Africa,” N.D., 385. Eberhard, Anton, And Marek Raciboski. “Emergence and Sustainability Of Independent Power Projects In Nigeria,” N.D., 39. 90 “Ethiopia - Energy.” The International Trade Administration, October 2019. https://www.export.gov/article?id=Ethiopia-Energy Ezekiel Adesina, Ben Asante, Natalia Camba, And Fisoye Delano. Understanding Natural Gas and Lng Options. Power Africa Understanding Series 1, 2017. Https://Www.Energy.Gov/Sites/Prod/Files/2017/11/F46/Understanding%20natural%20gas%20and%20lng% 20options%20october%2011%202017_1.Pdf. Frederick A. Okang, Frank Otchere, And Kassim Abubakar. “2019 Electricity Supply Plan for The Ghana Power System.” Energy Commission, 2019. Http://Www.Energycom.Gov.Gh/Files/2019%20electricity%20supply%20plan.Pdf. “Frequently Asked Questions (Cenpower).” Faq. Cenpower, 2015. Http://Www.Cenpowergen.Com/Press/Cenpower_faq.Pdf. “Funding Proposal For Nigeria Solar Ipp Support Program.” Accessed November 7, 2019. Https://Www.Greenclimate.Fund/Documents/20182/574760/Funding_proposal_-_fp104_-_afc_- _nigeria.Pdf/9245df67-82fd-282f-4103-ab4122a3e662. “Ghana Loses $190 Million U.S. Grant Over Canceled Power Contract.” Reuters, October 23, 2019. Https://Www.Reuters.Com/Article/Us-usa-ghana-power-iduskbn1x225d. “Highlights of Kenya’s Energy Act 2019.” Rodl & Partner, 2019. https://www.roedl.com/insights/erneuerbare-energien/2019-05/highlights-of-kenyas-energy-act-2019 Jim Jewell, And Natenna Dobson. “Lng Handbook Slides.” December 2016. Https://Www.Energy.Gov/Sites/Prod/Files/2016/12/F34/Lng%20handbook%20slides%20december%20201 6awl.Pdf. Kapika, Joseph, And Anton Eberhard. Power Sector Reform And Regulation In Africa. Capetown: Hsrc, 2013. “Karpowership Prepares To Relocate, Shuts Down For 17 Days.” Accessed November 11, 2019. Https://Www.Ghanaweb.Com/Ghanahomepage/Newsarchive/Karpowership-prepares-to-relocate-shuts￾down-for-17-days-771401#. Ken Ofori-atta, (Minister For Finance). “Mid-year Fiscal Policy Review Of The 2019 Budget Statement And Economic Policy & Supplementary Estimates Of The Government Of Ghana For The 2019 Financial Year.” Ministry Of Finance, Ghana, July 29, 2019. Https://Drive.Google.Com/File/D/10bsdwq1wohs￾ijk2haz-uyks7sldjtym/View?Usp=Embed_facebook. “Kenya halts Lamu coal power project at World Heritage Site.” BBC News, June 2019. https://www.bbc.com/news/world-africa-48771519 Kingsley Asare. “Ghana Bridge Power Project Environmental and Social Impact Assessment.” Jacobs Consultancy Ltd., August 7, 2017. Https://Www3.Opic.Gov/Environment/Eia/Bridgepower/Esia_vol_2.Pdf. Kingsley Osei. “Will the Put/Call Option Agreement (‘pcoa’) Adopted In The Early Power Consortium Ipp In Ghana Set A New Trend For Mitigating Project Risks In Africa?” Accessed October 31, 2019. Https://Www.Linkedin.Com/Pulse/Putcall-option-agreement-pcoa-adopted-early-power-consortium￾osei. 91 Lovemore Seveni, Miguel Franco, And Carolina Barreto. “Power Africa Africa Off-grid Project Fact Sheet,” August 2019. Https://Www.Usaid.Gov/Sites/Default/Files/Paop_fact_sheet.Pdf. Majzoub, Yousef, Colin Mccollester, Grayson Heffner, And Peter Graham. “Nigeria Power Sector Program (Npsp) Distribution Subsector Assessment,” N.D. Map Design Unit Of The World Bank. “Ethiopia Geothermal Sector Development Project.” Ethiopia: World Bank Group, January 2014. Https://Development-data-hub-s3- public.S3.Amazonaws.Com/Ddhfiles/145078/Eth40624.Pdf. “Maximizing Financing for Development in Action: The Kenya Energy Sector Experience.” World Bank Group, April 2019. https://www.worldbank.org/en/results/2019/04/18/maximizing-financing-for￾development-in-action-the-kenya-energy-sector-experience Mohamed Badissy, Lucy Chege, Patrick M. Dougherty, Alex Evans, And Nnaemeka Ewelukwa. Understanding Power Project Financing. Power Africa Understanding Series 2. Accessed November 5, 2019. Https://Cldp.Doc.Gov/Sites/Default/Files/Understandingpowerprojectfinancing.Pdf. “Nigeria Power Sector Program Gas Sector Assessment.” Npsp End to End Sector Assessment. Deloitte Consulting Llp, April 2019. “Nigeria Power Sector Program Legal and Enabling Environment Assessment.” Npsp End to End Sector Assessment. Deloitte Consulting Llp, April 2019. “Nigeria Power Sector Program Long-term Planning Assessment.” Npsp End to End Sector Assessment. Deloitte Consulting Llp, April 2019. “Nigeria Power Sector Program Off-grid Sector Assessment.” Npsp End to End Sector Assessment. Deloitte Consulting Llp, March 2019. “Nigeria Power Sector Program Sector Financial Assessment.” Npsp End to End Sector Assessment, April 2019. “Nigeria Power Sector Program Technical System Operations Assessment.” Npsp End to End Sector Assessment, April 2019. “Nigerian Energy Sector’s Crippling Debts Delay Next Power Plant.” Voice ff America. Accessed November 4, 2019. Https://Www.Voanews.Com/Africa/Nigerian-energy-sectors-crippling-debts-delay￾next-power-plant. Oliver Irwin. “Nigerian Solar: The Path to Bankability.” Proximo (Blog). Accessed November 15, 2019. Https://Www.Proximoinfra.Com/Articles/6766/Nigerian-solar-the-path-to-bankability. “Power Africa Annual Report 2017.” Power Africa, August 2017. “Power Africa Enabling Environment Data Tracker.” USAID Data Services, 2019. https://public.tableau.com/profile/eads#!/vizhome/PowerAfrica_0/EnablingEnvironmentDataTracker “Power Africa in Ethiopia.” Power Africa, March 2016. https://www.usaid.gov/sites/default/files/documents/1860/Ethiopia%20Country%20Fact%20Sheet_0.pdf Tetra Tech. “Power Africa Transactions and Reforms Program.” Accessed August 7, 2019. Https://Www.Tetratech.Com/En/Projects/Power-africa-transaction-and-reforms-program. 92 “Power Africa Transmission Roadmap.” Power Africa, November 2018. “Quarterly Report First Quarter 2019.” Nigerian Electricity Regulatory Commission, Q1 2019. Richter, Alexander. “Corbetti geothermal project resolving issues with Government of Ethiopia.” Think Geoenergy, August 2017. http://www.thinkgeoenergy.com/corbetti-geothermal-project-resolving-issues￾with-government-of-ethiopia/ Sarkodie, Samuel Asumadu. “Lessons to Be Learnt From Ghana’s Excess Electricity Shambles.” The Conversation. Accessed November 6, 2019. Http://Theconversation.Com/Lessons-to-be-learnt-from￾ghanas-excess-electricity-shambles-121257. Shinya Nishimura, And Paivi Koljonen. “Combined Project Information Documents / Integrated Safeguards Datasheet (Pid/Isds).” The World Bank, December 2017. Http://Documents.Worldbank.Org/Curated/En/953141527091798128/Pdf/Project-information￾document-integrated-safeguards-data-sheet-ghana-energy-sector-transformation-initiative-project￾p163984.Pdf. “Strengthening Gender Equality and Increasing Women’s Participation in the Ethiopian Power Sector.” Power Africa, December 2017. https://medium.com/power-africa/strengthening-gender-equality-and￾increasing-womens-participation-in-the-ethiopian-power-sector-fdc4f4049e55 Tsagas, Ilias. “Ethiopia’s first 100 MW auction to conclude in June; scaling solar tender to follow in the summer.” PV Magazine, June 2017. https://www.pv-magazine.com/2017/06/09/ethiopias-first-100-mw￾auction-to-conclude-in-june-scaling-solar-tender-to-follow-in-the-summer/ “Women in Energy: $1 billion dollars in negotiations.” Mining Review Africa, August 2019. https://www.miningreview.com/energy/women-energy-1-billion-dollars/ 93 DATABASES REVIEWED All data was extracted between September and October 2019. 1. Power Africa Tracking Tool (PATT) - The PATT contains records of all officially recognized Power Africa transactions. PATT is a source of quantitative data that underlies analysis of generation and connections results. 2. PATRP Life of Program M&E Dashboard Summary (PATRP_MEC_190926.xlsx) Shared by Tetra Tech with the evaluation team. Summarizes PATRPs accomplishments relative to the PMP. This database contained unique data not found in the PATT regarding the success of utilities and the policy work of TAs. 94 ANNEX 5: CONFLICTS OF INTEREST 95 96 97