SOUTHERN AFRICA ENERGY PROGRAM 
EVALUATION REPORT
October 19, 2022
DISCLAIMER: This publication was produced at the request of the United States Agency for International Development. It was
prepared independently by Social Impact, Inc.. The authors’ views expressed in this publication do not necessarily reflect the views of
the United States Agency for International Development or the United States Government.

i | USAID SAEP EVALUATION REPORT USAID.GOV 
CONTENTS
Acronyms ....................................................................................................................................................... iii
Executive Summary ...................................................................................................................................... iv
Introduction.................................................................................................................................................... 1
Design and Methodology ............................................................................................................................... 2
Evaluation Questions............................................................................................................................................................................................................ 2
Methodology............................................................................................................................................................................................................................. 2
Data Analysis............................................................................................................................................................................................................................. 5
Ethical Considerations.......................................................................................................................................................................................................... 6
Gender and Social Inclusion Plan.................................................................................................................................................................................... 6
Limitations and Biases........................................................................................................................................................................................................... 6
Findings and Conclusions .............................................................................................................................. 9
EQ1: To what extent was SAEP able to meet its purpose and related outputs and outcomes
(MWs, Connections, Enabling Environment)?......................................................................................................................................................... 9
EQ2: What enabled and/or hindered achievements, and how were any hindrances addressed?.............................................11
EQ3: How did the program design, target setting, and operational decision-making shape and contribute
to the achievements of the Program?........................................................................................................................................................................16
EQ4: What were the benefits and drawbacks of Power Africa setting the connections and MW targets 
for the program?...................................................................................................................................................................................................................22
EQ5: In which countries did SAEP have the most success versus the least success, and why? ...................................................28
EQ6: How is the SAEP perceived and valued by the stakeholders of the Program?.......................................................................30
Recommendations ....................................................................................................................................... 32
Annex 1: Statement of Work ..................................................................................................................... 35
Annex 2: Bibliography .................................................................................................................................. 38
Annex 3: Informed Consent Statements .................................................................................................. 39
Annex 4: Data Collection Instruments...................................................................................................... 41
Annex 5: Conflict of Interest Disclosures.................................................................................................. 59
Annex 6: Evaluation Design Report ........................................................................................................... 66
USAID.GOV USAID SAEP EVALUATION DESIGN REPORT | ii
TABLES
Table 1: Evaluation Design Matrix...................................................................................................................................................................... 3
Table 2: Survey Sample Size by Respondent Type .................................................................................................................................. 4
Table 3: Relevant EQs for Each Respondent Type ................................................................................................................................. 4
Table 4: SAEP Contributions to the Sector..............................................................................................................................................31
FIGURES
Figure 1: SAEP Actuals vs. Targets for Key Indicators............................................................................................................................ v
Figure 2: Countries Where SAEP Found the Most and Least Success......................................................................................vii
Figure 3: Survey Respondent Perceptions of SAEP’s Impact...........................................................................................................vii
Figure 4: Original SAEP Design Summary ..................................................................................................................................................... 1
Figure 5: Total Respondents by Respondent Type.................................................................................................................................. 5
Figure 6: Total Respondents by Gender........................................................................................................................................................ 5
Figure 7: Total Respondents by Country....................................................................................................................................................... 5
Figure 8: Potential Partner Respondents and Organizations by Country................................................................................... 7
Figure 9: Actuals vs. Targets for Key SAEP Indicators............................................................................................................................ 9
Figure 10: Seamstress using SAEP-supported off-grid solar lights. ..............................................................................................19
Figure 11: Wind turbines from an SAEP partner..................................................................................................................................24
Figure 12: SAEP-supported Solar Home System Kick-Starter Program in Malawi............................................................28
Figure 13: Satisfaction with SAEP by Respondent Type ....................................................................................................................32
iii | USAID SAEP EVALUATION REPORT USAID.GOV 
ACRONYMS
COVID-19 Novel Coronavirus of 2019
ET Evaluation Team
EQ Evaluation Questions
GESI Gender Equality and Social Inclusion
IP Implementing Partner
KII Key Informant Interview
MEL Monitoring, Evaluation, and Learning
MW Megawatt
PAIS Power Africa Information System
PATT Power Africa Tracing Tool
PMEP Performance Management & Evaluation Plan
POC Point of Contact
RFP Request for Proposals
SADC Southern Africa Development Community 
SAEP Southern Africa Energy Program
SAPP Southern Africa Power Pool
SOO Statement of Objectives
SI Social Impact
USAID United States Agency for International Development
USG United States Government
USAID.GOV SAEP EVALUATION REPORT | iv
EXECUTIVE SUMMARY
CONTEXT
Power Africa is a United States government (USG)-led partnership that launched in 2013. By 2030, Power
Africa aims to achieve 30,000 megawatts (MW) of new generation capacity, 60 million new power
connections for homes and businesses, and the unlocking of energy sector potential by improving the
enabling environment.
1 The Southern African Energy Program (SAEP), a flagship Power Africa initiative
funded by the United States Agency for International Development (USAID), aims to increase investment in
electricity supply and access in Southern Africa by strengthening regulation; improving planning, procurement,
and management of electricity trade; demonstrating and scaling energy efficiency technologies and practices; 
and providing capacity building to institutions and human resources. 
PURPOSE
This evaluation provides USAID/Southern Africa and the USAID Power Africa team with feedback regarding
SAEP successes, challenges, and areas for improvement, which could inform future programming in the
energy sector and support programmatic adjustments. Other potential evaluation users include implementing
partners (IPs), regional organizations, national ministries, private sector partners, and regulators, who can
learn from SAEP’s work.
EVALUATION METHODOLOGY
The SAEP evaluation addressed six core evaluation questions (EQs): 
1. To what extent was SAEP able to meet its purpose, related outputs, and outcomes?
2. What enabled and/or hindered achievements, and how were any hindrances addressed (including
but not limited to COVID-19 [novel coronavirus of 2019])?
3. How did the program design, target setting, and operational decision-making shape and contribute to
the achievements of the program?
4. What were the benefits and drawbacks of Power Africa setting the connections and MW targets for
the program?
5. In which countries did SAEP have the most success versus the least success, and why?
6. How is the SAEP perceived and valued by the stakeholders of the program?
To answer these EQs, the evaluation team (ET) employed a mixed-method design across SAEP’s ten
countries. On the qualitative side, the ET conducted a desk review of relevant documents and key informant
interviews (KIIs). On the quantitative side, the ET administered a short, online survey and reviewed the
Power Africa Information System (PAIS), Power Africa Tracking Tool (PATT), and program monitoring data.
The Social Impact (SI) team conducted remote data collection to reduce potential risks related to COVID-19
and mitigate logistical constraints. As a result of small sample sizes, the ET reports findings for all SAEP
partner respondents together rather than disaggregating the findings by type of partner (government, utility,
etc.). Across the EQs, all respondent groups tended to provide similar feedback. Thus, unless otherwise
noted, each finding is consistent across respondent groups (IPs, USAID, and partner organizations). 
1 USAID. 2022. Power Africa Website. https://www.usaid.gov/powerafrica
v | USAID SAEP EVALUATION REPORT USAID.GOV 
FINDINGS AND CONCLUSIONS
EQ1: TO WHAT EXTENT WAS SAEP ABLE TO MEET ITS PURPOSE, RELATED OUTPUTS, AND OUTCOMES?
SAEP met most of its performance targets, including for new generation and transmission. However, SAEP
has not yet met its target of 3 million new electricity connections. Given the extension of the period of
performance to June 2023, achieving this target remains possible. In interviews, respondents shared largely
positive feedback regarding the results, feedback that aligned with the quantitative results tracked in the
performance indicators. Despite SAEP’s focus on new generation, transmission, and connections, however,
more respondents noted SAEP’s success in building capacity (mentioned in 27 interviews) than they did the
generation, transmission, or connection outcomes (17, 11, and 13 interviews, respectively), suggesting
stakeholders viewed SAEP’s capacity building work as particularly beneficial. 
Figure 1: SAEP Actuals vs. Targets for Key Indicators (as of June 2023)
EQ2: WHAT ENABLED AND/OR HINDERED ACHIEVEMENTS, AND HOW WERE ANY HINDRANCES
ADDRESSED? 
Respondents shared numerous factors enabling and/or hindering SAEP achievements. In addition to the
overarching impact of COVID-19, these factors fell into three main categories: SAEP-level factors, partner￾level factors, and context-level factors (in decreasing level of importance).
COVID-19 posed a challenge to SAEP implementation through delays and slower operations (21 interviews)
and negative effects from travel restrictions (19 interviews). SAEP adapted using virtual approaches (19 
interviews), though many stakeholders (22 interviews) felt that in-person activities were more effective. 
SAEP-level factors centered on SAEP’s technical capabilities across outcome areas (25 interviews) and the
strength of their relationships with partner organizations (23 interviews), both of which most respondents
viewed as SAEP strengths that enabled program achievements. 
Partner-level factors, primarily noted by IP and USAID staff, centered on several interrelated factors: partner
organizations’ willingness to engage with SAEP (19 interviews), their buy-in to the program’s activities (21 
interviews), and the extent to which SAEP’s priorities aligned with those of its partners (14 interviews). 
Where SAEP and stakeholder priorities align, the ET found partners more likely to engage and more likely to
have strong buy-in, all of which support strong success. Insufficient alignment and buy-in is a barrier to
success, demonstrating the importance of meeting local needs to improve success. Given the large number of
USAID.GOV SAEP EVALUATION REPORT | vi
SAEP countries and partners, these partner-level factors acted equally as often as drivers of success as
hindrances, based on respondent feedback. 
Context-level factors centered on political influences (12 interviews) and cultural and linguistic differences
(ten and nine interviews, respectively). The former included large political events such as elections, political
unrest, and other political changes, which fell outside of SAEP’s control. Respondents noted cases where
SAEP’s response to cultural and linguistic differences bolstered success and where it remained a barrier, 
suggesting room for improvement.
EQ3: HOW DID THE PROGRAM DESIGN, TARGET SETTING, AND OPERATIONAL DECISION-MAKING SHAPE
AND CONTRIBUTE TO THE ACHIEVEMENTS OF THE PROGRAM? 
The ET examined several components of program design and implementation to answer this EQ, including
SAEP’s strengths and weaknesses, the effectiveness of decision making, SAEP’s approach to gender equality
and social inclusion (GESI), and respondents’ recommendations for future programs. Overall, the SAEP
design and implementation approach contributed positively to program achievements. SAEP strengths
included strong staff (24 interviews), effective communication (23 interviews), and a flexible and adaptable
approach to implementation (11 interviews). Weaknesses included staffing concerns (14 interviews), such as
issues with specific staff, staff turnover, and staff continuity with partners, and insufficient localization and local
input into design (12 interviews). 
Overall, respondents applauded the SAEP team for its effective communication and decision-making
processes (13 interviews). On the other hand, respondents noted that USAID’s decision-making processes
often acted as a hindrance to effective implementation (11 interviews); in particular, respondents highlighted
negative impacts from frequent requests from USAID Offices and Missions to add activities or partners into
SAEP’s scope, which pulled SAEP staff in multiple, sometimes competing, directions; in some cases, these
requests did not align well with SAEP’s core objectives. 
SAEP’s GESI approach focused primarily on gender and not on other social groups, according to respondents
(20 interviews). IP respondents noted GESI-related benefits related to the involvement of women in the
sector (eight interviews) and stakeholder engagement (six interviews). Respondents felt that the effects were
largely superficial (seven interviews), in part due to pervasive challenges within the male-dominated sector
(12 interviews). 
When asked for future programming recommendations, respondents shared a variety of technical and design
suggestions. On the technical side, respondents often suggested that future programs continue to focus on
the same core areas as SAEP such as new connections (29 interviews), capacity building (21 interviews),
generation (19 interviews), transmission (16 interviews), and institutional strengthening (11 interviews). In
terms of design, respondents recommended that future programming include more, early input on program
design and planned activities from local stakeholders (24 interviews), increase the utilization of local experts
(12 interviews), and address the funding/financing gap in the power sector (19 interviews). 
EQ4: WHAT WERE THE BENEFITS AND DRAWBACKS OF POWER AFRICA SETTING THE CONNECTIONS
AND MW TARGETS FOR THE PROGRAM? 
Only four IP or USAID staff recalled how SAEP’s original performance indicators, in particular the fee
indicators, were selected or how the targets were set. Thus, the ET relied heavily on document review to
answer this question, including the original SAEP request for proposals (RFP) and technical portions of the
SAEP contract. The RFP shows that USAID set minimum requirements for bidders of 3 million new
connections and 1,000MW of new, renewable energy generation capacity. In its proposal, Deloitte both
3.0 3.0 
The Power Sector Generation Capacity Transmission Capacity Power Access 
USAID/USG (N=6) IP (N=13) Partners (N=23) Total (N=42) 
3.0 
2.5 2.6 
1.9 
2.3 2.5 
1.9 
1.4 1.3 
2.3 2.3
1.7 1.8 
2.4 
1.0 
2.0 
3.0 
vii | USAID SAEP EVALUATION REPORT 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
increased and modified the latter to reflect 4,000MW of generation and transmission capacity. Per the RFP 
requirements, Deloitte proposed the list of fee indicators. The fee indicators ultimately drove SAEP’s focus 
given the financial incentive to reach the targets. 
Overall, both USAID and IP respondents regarded USAID’s use of fee indicators positively. The fee 
indicators helped the SAEP team remain focused (five interviews) and pushed them toward ambitious goals 
(five interviews). On the downside, however, they voiced several concerns with the indicators and targets, 
including the targets’ unrealistic nature (particularly the new connections target) (11 interviews). Concerns 
included how attributable the MWs and new connections were to the SAEP-provided technical assistance 
(seven interviews), the timelines required to move from the direct benefits of SAEP’s technical assistance to 
longer-term outcomes like generation, transmission, and connections (five interviews), insufficient local input 
and/or calibration of indicator selection and target setting to local needs (four interviews), and how the 
regional targets led SAEP to focus on the largest projects (and therefore the largest markets) even when 
smaller, beneficial projects existed (four interviews). 
EQ5: IN WHICH COUNTRIES DID SAEP HAVE THE MOST SUCCESS VERSUS THE LEAST SUCCESS, AND WHY? 
According to respondents, SAEP achieved the most success, as defined by the respondents, in Malawi and 
Mozambique (15 and 12 interviews, respectively) and the least success in Madagascar (six interviews), 
Lesotho (four interviews), Namibia, Botswana, and Eswatini (three interviews each). In addition, Zambia 
posed an interesting case as respondents included it on both the most (nine interviews) and least (three 
interviews) successful lists, due to SAEP’s early, successful work in the country followed by significant political 
changes that hindered SAEP’s later work in Figure 2: Countries Where SAEP Found the Most and Least Success the country. The factors driving success 
mirrored those outlined in EQ2 such as 
alignment of priorities between SAEP and its 
partners, stakeholder buy-in, and the strength 
of relationships. For lack of successes in 
Lesotho, Botswana, Eswatini, and Namibia, 
respondents noted that the small size of the 
market in these countries acted as a barrier, 
in line with the EQ4 finding that larger 
projects received more attention. 
EQ6: HOW IS SAEP PERCEIVED AND VALUED BY THE STAKEHOLDERS OF THE PROGRAM? 
Overall, stakeholders provided positive feedback regarding SAEP. They were satisfied with SAEP’s services 
and believed that SAEP positively affected the power sector. Across stakeholder groups, USAID respondents 
provided the most positive feedback whereas SAEP partners provided the least positive feedback. 
Figure 3: Survey Respondent Perceptions of SAEP’s Impact 
USAID.GOV 
USAID.GOV SAEP EVALUATION REPORT | viii
Interestingly, even though SAEP has yet to achieve its performance targets for new connections, stakeholders
valued SAEP’s impact on power access more highly than any other area. Meanwhile, feedback regarding
SAEP’s impact on generation and transmission received the lowest scores, even though SAEP met those
targets. This dichotomy likely relates to stakeholder feedback under EQ4 that the new connection target was
unrealistic and challenges linking SAEP’s technical assistance with generation and transmission targets. 
RECOMMENDATIONS
Based on the findings and conclusions of this evaluation, the ET recommends the following for USAID as it
proceeds with designing the SAEP follow-on program:
1. USAID should ensure the localization of future programs. Localization involves active
engagement with local stakeholders throughout the project life cycle so that the activities meet the
needs of those it is intended to help. Though both USAID and Deloitte played parts in how
localization occurred under SAEP, for future programming, USAID should begin the local
engagement process early to ensure that local priorities and needs are taken into account even at
the proposal stage. If engagement could not begin before the solicitation phase, then USAID should
maintain sufficient flexibility to allow for anticipated partners to influence the design and
implementation approach after award. Additionally, USAID should ensure that future programs are
staffed in a way that addresses the different cultural and linguistic needs across the region, such as
improved engagement of local staff and experts. (The ET developed this recommendation based on
localization findings from EQ2, EQ3, and EQ4.)
2. USAID should ensure that future indicator selection and target setting are based
on clear criteria. Given that the fee indicators drive program focus, USAID should ensure that
the indicators represent the intended focus of future programs and that they link clearly with
anticipated interventions. Additionally, the targets should have clear justifications based on realistic
criteria that incorporate local input, needs, and priorities. This would ensure ambitious, yet realistic,
targets given regional realities. USAID should consider ways to balance regional flexibility with
ensuring greater distribution of benefits across countries, regardless of the size of the power market.
(The ET developed this recommendation based on indicator and target-related findings from EQ4.)
3. USAID should ensure that future programming improve on the GESI approach. To
ensure a cohesive GESI approach, USAID should first define its GESI goals for a power activity such
as SAEP. They can then hold IPs accountable to that vision. (The ET developed this recommendation
based on GESI findings from EQ3.)
4. USAID should ensure that future programming addresses the financing gap in the
power sector. Though USAID could consider directly funding power investments, future
programs could also address the financing gap via other means. Such programming should be based
on a thorough study of the primary constraints of power sector financing in Southern Africa that
identifies specialized programming requirements, such as credit guarantees or other lending de￾risking mechanisms, to meet the needs of both credit seekers and financiers. (The ET developed this
recommendation based on design findings from EQ3.)
1 | SAEP EVALUATION REPORT USAID.GOV
INTRODUCTION
CONTEXT
Power Africa is a United States government (USG)-led partnership that launched in 2013. By 2030, Power
Africa aims to add 30,000 megawatts (MW) of new generation capacity, 60 million new power connections
for homes and businesses, and the unlocking of energy sector potential by improving the enabling
environment.2
 
A lack of adequate and reliable power in the Southern Africa region limits competitiveness and affects the
livelihoods and health of the population, particularly those living outside urban areas. The five-year Southern
African Energy Program (SAEP), a flagship program funded through a partnership with Power Africa and
USAID/Southern Africa, aims to increase investment in electricity supply and access in Southern Africa by 
strengthening the regional enabling environment and facilitating power transactions. SAEP, implemented by
Deloitte, launched in 2017 with activities in ten countries: Angola, Botswana, Eswatini, Lesotho, Madagascar,
Malawi, Mozambique, Namibia, South Africa, and Zambia.3 Working across all of these countries enabled
SAEP to take a regional approach to addressing sector needs. In 2022, SAEP received a one-year extension,
making it a six-year program expected to end in June 2023.
Figure 4: Original SAEP Design Summary (SAEP Fact Sheet)
Though SAEP aims to increase generation and access to electricity, SAEP does not directly fund major
infrastructure investments. Rather, SAEP works to incentivize, facilitate, and enable investments through
technical assistance, policy reform, capacity development, and transaction facilitation. Across the ten partner
countries, SAEP’s work ebbed and flowed, with some countries receiving more support in some years than in
others.
To identify interventions during the annual work planning process, the SAEP team meets with key
stakeholders in each partner country to discuss potential interventions and ways in which SAEP could
2 USAID. 2022. Power Africa Website. https://www.usaid.gov/powerafrica. 3 SAEP originally targeted Zimbabwe as well, but ultimately did not implement activities in the country.
USAID.GOV SAEP EVALUATION REPORT | 2
support their electrification, generation, and other sectoral improvement goals. SAEP’s flexible and adaptable
approach to implementation involves iterative changes and adaptations to the work plan. 
SAEP’s theory of change states that by strengthening regulation; improving planning, procurement, and
management of electricity trade; demonstrating and scaling energy efficiency technologies and practices; and
providing capacity building to institutions and human resources for energy sector management, SAEP aims to
increase investment in electricity supply and access in Southern Africa. SAEP’s support works to address
reliance on electricity/energy imports. Long-term program targets include 3,000MW of new power
generation, 1,000MW of new transmission capacity, and 3 million new connections (on- and off-grid). 
EVALUATION PURPOSE AND AUDIENCE
The purpose of this evaluation is to provide USAID/Southern Africa and the USAID Power Africa team with
feedback and recommendations for improving the SAEP activity and informing future programming in the
energy sector. This evaluation equips USAID teams with evidence to support programmatic adjustments,
inform future design efforts and strategies, and make informed decisions. Other potential users of the
evaluation include implementers, regional organizations, national ministries, private sector partners, and
regulators, who could adapt lessons learned from SAEP.
Commissioned by Power Africa and contracted via USAID/Southern Africa, Social Impact (SI) was selected
to conduct a utilization-focused, mixed methods evaluation with an emphasis on broad stakeholder
engagement to promote its use and evidence-based decision making. SI’s technical design kept quality,
utilization, and feasibility firmly in focus while ensuring broad coverage of all ten SAEP countries. 
DESIGN AND METHODOLOGY
EVALUATION QUESTIONS
The Evaluation Team (ET) worked closely with USAID to refine and finalize the Evaluation Questions (EQs)
in order to ensure mutual understanding and maximize utility and learning:
1. To what extent was SAEP able to meet its purpose, related outputs, and outcomes (MWs,
Connections, Enabling Environment)? 
2. What enabled and/or hindered achievements, and how were any hindrances addressed (including
but not limited to COVID-19)? Provide specific examples. 
3. How did the program design, target setting, and operational decision-making shape and contribute to
the achievements of the program? 
4. What were the benefits and drawbacks of Power Africa setting the connections and MW targets for
the program?
5. In which countries did SAEP have the most success versus the least success, and why? (focus on off￾grid interventions; unlocking financing for private sector companies; improving the enabling
environment) 
6. How is the SAEP perceived and valued by the stakeholders of the program? 
METHODOLOGY
The SAEP evaluation relied on multiple methods to obtain data across the ten active SAEP countries. On the
qualitative side, the ET conducted both a desk review of relevant documents and literature and key
informant interviews (KIIs). On the quantitative side, the ET carried out a systematic review of the Power
3 | SAEP EVALUATION REPORT USAID.GOV
Africa Information System (PAIS) and Power Africa Tracking Tool (PATT) as well as program monitoring
data and administered a short, online survey that allowed key stakeholders across the region to have their
perspectives heard. To reduce potential risks related to COVID-19, the ET only conducted remote data
collection. The following sections describe SI’s mixed methods approach to the evaluation, as outlined in
Table 1. Annex 6: Evaluation Design Report provides the ET’s full design report and methodology.
Table 1: Evaluation Design Matrix
EVALUATION QUESTIONS DOCUMENT/ 
DESK REVIEW
SECONDARY 
DATA
QUANTITATIVE
SURVEY KIIS
Question 1: To what extent was SAEP
able to meet its purpose and related
outputs and outcomes (MWs,
Connections, Enabling Environment)?
X X X X 
Question 2: What enabled and/or
hindered achievements, and how were
any hindrances addressed?
X X X 
Question 3: How did the program design,
target setting, and operational decision￾making shape and contribute to the 
achievements of the Program?
X X X X 
Question 4: What were the benefits and
drawbacks of Power Africa setting the
connections and MW targets for the 
program?
X X X 
Question 5: In which countries did SAEP
have the most success versus the least
success, and why?
X X X X 
Question 6: How is the SAEP perceived
and valued by the stakeholders of the
Program?
X X 
Document/Desk Review. The ET conducted a systematic document/desk review to understand the
availability and quality of secondary data in each of the ten SAEP countries and the relationship of that data to
the evaluation questions, as well as to identify key issues and constraints for further examination in primary
data collection. The desk review included SAEP background documents, including quarterly and annual
reports, programmatic documents, and relevant stakeholder documentation (see Annex 2: Bibliography for a 
full list of reviewed documents). 
Secondary Data. The ET reviewed PAIS, PATT, and SAEP monitoring data. The ET used data from the
PAIS to understand SAEP partners, commitments (and progress toward them), private sector engagement,
and a broad range of other work completed through Power Africa. Data from the PATT provided insights
on power transactions, the energy mix, and generation capacity. As in the document/desk review, the ET
examined data for trends over time and to better understand both progress and challenges. The ET found
these databases particularly useful in answering EQ1 on SAEP’s ability to meet its purpose and objectives. 
USAID.GOV SAEP EVALUATION REPORT | 4
Quantitative Survey. The SI team implemented a short, online quantitative survey to assess
achievements, benefits, successes, and challenges during implementation. This survey enabled rapid collection
of key perspectives from a broad set of stakeholders (across the same stakeholder groups noted below for
the KIIs) in the ten active SAEP countries that may not have been otherwise reached through interviews. The
survey tool contained questions to better understand perceptions of SAEP implementation, achievements,
and challenges, with a particular focus on EQs 1, 2, 5, and 6. The data complemented the secondary data and
document/desk review, and the ET triangulated it with the more in-depth interview data. Table 2 outlines the
final sample size.
Table 2: Survey Sample Size by Respondent Type
RESPONDENT TYPE NUMBER OF RESPONDENTS
USAID/United States Government Staff 6 
Implementing Partner Staff 13
Utility Representatives 4 
Partner Government Representatives 3 
Regional Entities 1 
Private Sector Actors 8 
Other Donors 7 
Total 42
Key Informant Interviews. The SI team held KIIs with a sample of stakeholders to provide context and
document stakeholder experiences related to SAEP’s interventions. Employing a semi-structured interview
approach, KIIs included questions regarding stakeholder experiences with program implementation and
perceived program effects, including respective contributions of different activities to USAID’s results. The
ET tailored interview protocols to the different stakeholder types to fit their roles. KIIs lasted approximately
60 minutes for all partner organizations and up to 120 minutes with USAID and IP respondents. Targeted
stakeholder groups for KIIs included USAID and embassy staff, IP staff, utilities, regulators, government
representatives, representatives of other donors, and private sector actors. The ET selected respondents
purposively across countries and stakeholder types to include as broad a spectrum of experiences as possible.
Table 3 summarizes how the ET used feedback from each type of respondent to answer each of the EQs, 
based on each groups’ relevant knowledge and familiarity with SAEP programming:
Table 3: Relevant EQs for Each Respondent Type
RESPONDENT TYPE EQ1 EQ2 EQ3 EQ4 EQ5 EQ6
USAID/USG Staff X X X X X X
IP Staff X X X X X X
Utility Representatives X X X
Regulator Representatives X X X
Partner Government Representatives X X X
Regional Entities X X X X
Private Sector Actors X X X
Other Donors X X X
Regional 24 
Malawi 17 
Zambia 9 
South Africa 7 
Madagascar 6 
Eswatini 5 
Angola 5 
Namibia 4 
Mozambique 2 
Lesotho 0 
Botswana 0 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
The figures below outline the final respondent sample sizes, distributed across the ten active SAEP countries 
and stakeholder types (please refer to the Limitations and Biases section for additional information regarding 
sampling challenges). 
Figure 5: Total Respondents by Respondent Type (n=79) Figure 7: Total Respondents by Country (n=79) 
21 
14 
12 
9 
7 
7 
5 
4 
IP Staff 
Private Sector Actors 
Government… 
Utilities Providers 
USAID and Mission… 
Regulators 
Other Donors 
Regional Entities 
Figure 6: Total Respondents by Gender (n=79) 
Female, 23, 
29% Male, 56, 71% 
DATA ANALYSIS 
The ET analyzed data through an iterative process during and after the data collection. Triangulation of the 
desk review documents, quantitative survey, and the KII data allowed the ET to assess trends, consistency, 
affinity, and contradiction between quantitative and qualitative data, in order to better answer the EQs, 
generate more reliable findings, and reach well-founded conclusions. The ET applied a gender equality and 
social inclusion (GESI) lens to understand how findings differed by gender. 
Quantitative Data Analysis. The ET performed rigorous data quality checks and reviewed the data 
using SI’s high frequency check protocols. These checks allowed the ET to identify survey logic, data range, 
missing values, implausible values, digit preference (for numeric responses), and outlier errors in the data. 
After the survey closed, the ET executed a final cleaning process before analysis. The ET utilized Excel 
software to explore trends and disaggregate by stakeholder type, country, and gender. 
Qualitative Data Analysis. During data collection, the ET took detailed notes and recorded (with 
consent) all discussions. After data collection, the ET utilized a structured coding approach to conduct 
content and thematic analysis using Dedoose software for identification of emergent themes and contextual 
factors upon which to draw findings and conclusions and to develop evidence-based recommendations. The 
ET utilized a Findings, Conclusions, and Recommendations Matrix to triangulate data across data sources and 
to ensure an evidence-based, triangulated, and systematic analysis process. As a result of small sample sizes, 
the ET reports findings for all SAEP partner respondents together, rather than disaggregating the findings by 
type of partner (government, utility, etc.). Across the EQs, all respondent groups tended to provide similar 
feedback. Thus, unless otherwise noted, each finding is consistent across respondent groups (IPs, USAID, and 
partner organizations). 
5 | SAEP EVALUATION REPORT USAID.GOV 
USAID.GOV SAEP EVALUATION REPORT | 6
ETHICAL CONSIDERATIONS
SI’s in-house institutional review board reviewed the evaluation tools, data collection instruments, and
protocols prior to the start of data collection. All members of the ET and management team received ethical
research training to ensure honesty and integrity, protection of respondents, and data security. All
respondents provided verbal or written informed consent prior to their participation in the evaluation. The
evaluation did not attribute any information collected through KIIs to individual participants.
GENDER AND SOCIAL INCLUSION PLAN
As part of the evaluation, the ET assessed the extent to which SAEP implemented and/or adopted planned
activities, such as gender-specific training, guidelines, indicators, policies, and strategies. With a GESI-sensitive
approach, the ET reviewed secondary data and aimed to maintain gender balance in the KIIs to the extent
feasible. As is consistent with more males working across the power sector, the ET interviewed more male
than female respondents. The ET used a GESI scorecard to maintain USAID’s Automated Directives System
205 standards. This scorecard allowed the ET to assess gender sensitivity throughout the evaluation process,
including within the submitted data collection instruments. The ET disaggregated its analysis in the following
sections by gender and stakeholder type as necessary and possible to discern differences and similarities in
experiences, perceptions, needs, barriers, and other findings.
LIMITATIONS AND BIASES
The potential limitations as well as mitigation measures for the SAEP evaluation include:
Effects of Short Evaluation Timelines. To meet the timeline of the evaluation USAID established, the
ET combined methods and used a remote approach to mitigate this challenge and quickly collect data from a 
broad range of stakeholders.
Effects of Large Geographic Dispersion. SAEP’s regional focus introduced a unique challenge for
ensuring that the ET included the views and perspectives of all stakeholder groups in the evaluation. The
dispersion created both logistical challenges for reaching key stakeholders via traditional, in-person data
collection in each country and challenges for aggregating findings across widely varying contexts. The
evaluation design and remote approach helped mitigate the logistical challenges while ensuring efficient data
collection. The SAEP team assisted the ET in encouraging stakeholder participation in the survey and
interviews. Regarding aggregation of findings, the ET spoke to a variety of stakeholders and examined data
for any potential inter-country differences. To further enable participation, the ET conducted both the
survey and interviews in English, French, or Portuguese (based on respondent preference).
Response Bias. The ET recognized that SAEP stakeholders may carry unobserved biases specific to their
role, cultural considerations, and/or other factors. This bias could influence their responses, such as being
overly positive about achievements or saying what they think the ET or the Power Africa Program Office
wants to hear. The SI team designed the instruments carefully and triangulated evidence across methods to
minimize potential bias and ensure validity and reliability.
Selection Bias. Neither the interviews nor survey captured the views of all potential respondents, as some 
potential respondents did not reply to ET outreach and others chose not to participate in the evaluation.
Thus, differences between who participated and who did not could introduce bias into the evaluation. To
limit potential selection bias, the ET invited a wide range of stakeholders across all ten countries to participate
and sought the support of SAEP staff to encourage participation. The ET invited all stakeholders with email
addresses to participate in the survey and contacted more than 200 potential respondents for an interview
7 | SAEP EVALUATION REPORT USAID.GOV
(first via email, with follow-ups by phone if the ET did not receive an email response or if email addresses
were unavailable). Despite the ET’s efforts, however, final sample sizes fell short of expectations for several
reasons:
● Fewer Potential Partner Respondents Than Expected: Overall, the ET received names
and contact information (phone and/or email) for 352 stakeholders involved to some extent with
the program (including IP and USAID staff). Among SAEP partners, this included 296 individuals
across 140 organizations. However, the number of potential partner respondents and organizations
varied widely across countries, as seen in Figure 8.
Figure 8: Potential Partner Respondents and Organizations by Country
103
51
45
24
20
14
13
11
6
5
4
42
25
20
10
18
4
3
8
3
5
2
Zambia
Malawi
Mozambique
Namibia
South Africa
Eswatini
Angola
Madagascar
Botswana
Regional
Lesotho Respondents Organizations
While the ET anticipated that levels of SAEP activity differed across the ten countries, it had not 
anticipated the extent to which this was the case. Since the number of potential respondents
reflected the extent to which SAEP was active in each country, the varying distribution, itself, was not
a problem for the validity of the evaluation’s findings. However, it did limit the ET’s ability to achieve
the intended sample size because the original sampling plan assumed that the distribution between
countries was more even than it was in reality. For example, the ET had anticipated interviewing
respondents from more than five organizations in every country. However, in four countries, SAEP
did not even work with that many organizations; thus, the originally planned sample (which the ET
designed before receiving detailed contact information) was not possible. Though the ET reached
out to more potential respondents in countries with greater SAEP activity, the small potential pool in
many countries limited the ET’s opportunities to reach the anticipated sample without overly biasing
the sample toward perspectives from a few select countries. 
● High Turnover: Many of the potential partner respondents the IP shared either had incorrect
contact information or were no longer with their organizations, resulting in numerous undeliverable
emails, a challenge that was consistent across different types of partners (e.g., government staff,
utilities, etc.). For the survey, of the 333 emails the ET sent (not all potential respondents on the ET’s
list included email addresses), 73 (22 percent) bounced back as undeliverable. Similar issues occurred
with KII invitations. Often, the ET reached out to multiple people at the same organization. In many
USAID.GOV SAEP EVALUATION REPORT | 8
cases, the ET heard from other respondents at the organization that our intended recipients no
longer held the same positions or worked for the same organization as when they engaged with
SAEP (see EQ2 for related findings on partner staff turnover). This suggests that, after a particular
activity or intervention ended, SAEP may not have maintained contact with these organizations.
● Low Levels of SAEP Awareness: Even among potential respondents that the ET contacted
successfully, a surprising number expressed that they lacked sufficient familiarity with SAEP’s work to
respond to the ET’s questions, which was particularly common among other donor and government
stakeholder respondents.
● Fewer Group Interviews: When reaching out to partner organizations, the ET anticipated
conducting one KII with two to three individuals within each organization. However, in the end,
partners often preferred an individual KII, identifying a single individual to speak with the ET rather
than a small group. This was not an inherent issue for the evaluation, as the ET was still able to
achieve its goal of understanding perspectives from that organization. However, the ET’s sampling
plan assumed there would be more small group interviews than occurred in practice, resulting in a
smaller total sample size. Of the 56 interviews conducted, only 17 included more than one person.
Thus, though the ET collected views from that organization as intended, the total number of
individual respondents was lower than expected.
Due to the above issues, the ET received feedback from fewer respondents than anticipated. On the survey,
the ET received 42 responses, a response rate of 16 percent of successfully delivered emails. Participation
rates of 20-30 percent in online surveys are more common. This resulted in a slightly higher than desired
margin of error of 13.8 percent. For KIIs, the ET originally anticipated speaking with at least 136 respondents;
ultimately 79 respondents participated in interviews. The lower-than-expected response rate mostly arose
among partner staff, as there were no significant issues in contacting IP or USAID staff. There were no
notable differences in response rates across different types of partners (utilities, regulators, government
entities, etc.). Across countries, unfortunately, the ET did not find any respondents willing to participate in
Botswana or Lesotho (which only had six and four potential respondents to begin with, respectively, none of
whom agreed to an interview). Largely due to the unequal gender distribution within the sector (and
therefore within those with whom SAEP engaged), the ET received feedback from more male than female
respondents. 
The reduced sample sizes had two main implications for this report. First, since the number of respondents
in each country was quite low in some cases, the ET could not disaggregate findings by country. Regarding
Botswana and Lesotho in particular, though the ET did not speak with any local partner respondents in these
countries, the evaluation still touches on SAEP’s work there through respondents (such as IP and USAID
staff) who were knowledgeable about SAEP’s work across countries. Rather, the ET was only unable to
capture the perspectives of local partners in these countries. Overall, the final number of respondents for
some types of partner organization was small. Thus, the ET aggregated all partner respondents together, 
disaggregating findings by three types: USAID staff, IP staff, and partners.
9 | SAEP EVALUATION REPORT USAID.GOV
FINDINGS AND CONCLUSIONS
EQ1: TO WHAT EXTENT WAS SAEP ABLE TO MEET ITS PURPOSE AND RELATED OUTPUTS
AND OUTCOMES (MWS, CONNECTIONS, ENABLING ENVIRONMENT)?
FINDINGS
Multiple data collection sources informed this EQ, including document/desk review, secondary data, a 
quantitative survey, and KIIs. In particular, program monitoring data helped to provide insights on
performance against the targets while KIIs provided stakeholder perceptions of success. The following
sections begin with a review of the relevant performance indicators, followed by a discussion of stakeholder
perceptions of SAEP success.
PERFORMANCE INDICATORS
As of June 2022, SAEP achieved its targets for new generation and new transmission (5,771MWs achieved
out of 4,000MWs targeted) as well as new policies (33 out of 31) and people trained (2,033 out of 1,331)
but had not yet achieved its goal for new electricity connections (2,138,029 out of 3 million) according to the
most recent monitoring data.
Figure 9: Actuals vs. Targets for Key SAEP Indicators (as of June 2022)
STAKEHOLDER PERCEPTIONS
Definition of Success
• Stakeholders generally believed success
aligns with SAEP’s indicators
• Impacts on people’s lives were missing
from the indicators, however
Unintended Outcomes 
• Few respondents cited unintended
outcomes
• Those that did noted some
expanded/enhanced activities
Top Successes
• Capacity development
• New generation
• Regulatory support
• New connections
• Utility support
• New transmission
Unachieved Outcomes 
• Many respondents felt SAEP achieved its
goals and had no unachieved outcomes
• Those that did noted new connections
USAID.GOV SAEP EVALUATION REPORT | 10
Definition of Success. The ET asked all respondents about their perceptions of SAEP success, including
how they would define success for the program. In most KIIs (33 interviews), respondents believed that their
definition of SAEP success aligned with the program’s performance indicators, namely MWs of generation
and transmission, new power connections, and capacity building. However, a number of respondents went
further to note that their definition of success included improving people’s lives (11 interviews). The
quantitative indicators do not really capture impacts on people’s lives, though it is possible that such impacts
were captured via other means such as success stories. These respondents indicated that, while it is
important to measure performance in terms of the indicators, there was often more to the story. For
example, respondents noted how additional generation capacity could facilitate productive uses or how 
having a transmission or distribution line could bring power to people without it. These findings were
consistent across stakeholder groups. 4
 
One IP respondent spoke to their definition of success:
“From the program perspective, when you achieve your key performance indicators, you are successful, of
which I believe SAEP has achieved. But I base my version of success on that in any project I’ve got inputs,
outputs, outcomes and impact… But if you go to a rural village and you find the people, now they’re using
that power to charge their phones, they are using that power to light their homes at night, during the day
they are using electricity for various livelihoods, that would be the impact of SAEP that I will evaluate.” 
Top SAEP Successes. When asked about SAEP’s biggest successes, respondents most frequently noted
capacity development (27 interviews), new generation (17 interviews), regulatory support (14 interviews),
new connections (13 interviews), utility support (13 interviews), and new transmission (11 interviews). SAEP
provided extensive training and capacity development services covering varied topics to a wide variety of
stakeholder organizations, including utilities, regulators, government entities, and others. In discussing this
capacity development work, respondents spoke positively not only about the technical aspects of training, but 
also about SAEP’s efforts to ensure gender balance, to the extent feasible, among trainees.
Regarding new power generation and transmission capacity, the qualitative data aligned with performance
indicator data in showing SAEP’s achievement of the intended results. Interestingly, however, though the
program had not yet met the target for new connections, many respondents applauded SAEP’s success in
this area. This view that SAEP achieved positive new connection outcomes may be due to the common belief
that new connections had unrealistic targets (discussed in more depth under EQ4). Respondents provided a 
variety of perspectives regarding success in new connections. When asked in which areas SAEP achieved the
most success, some referred generally to new connections, such as one IP respondent who simply noted, “So
new connections—that has been a great success.” Meanwhile, several others called out SAEP’s off-grid work
in particular, such as a USAID respondent who noted, “I would say the off-grid sector is probably the place
where it’s been truly transformational in the sense that… there was no off-grid sector in a lot of these
countries.” Thus, even though SAEP may not have achieved its quantitative target, qualitatively, many
respondents felt that SAEP made a substantial impact nonetheless.
Unachieved Outcomes. When the ET asked about outcomes that SAEP was unable to achieve, a large
number believed that SAEP had achieved all its goals (16 interviews). The ET found this view particularly
common among partner organizations (13 of the 16 interviews), potentially because many partner
4 Most findings in the evaluation are consistent across stakeholder groups. Thus, unless otherwise noted, findings do not vary notably between
stakeholdertypes (IP staff, USAID staff, and partners). 
11 | SAEP EVALUATION REPORT USAID.GOV
respondents equated success with achieving the goals of the specific interventions in which they engaged with
SAEP (rather than on the broader SAEP results framework). 
Where respondents noted unachieved outcomes, respondents most frequently cited new connections (23 
interviews), a finding aligned with the performance indicator data noted above. In most of the other cases,
when respondents talked about what SAEP did not achieve, they spoke about failures in specific
interventions. For instance, several respondents in Madagascar discussed the delays and difficulties grant
winners experienced in utilizing their grant funds for the intended purposes. One IP respondent noted, “In
Madagascar, I think the work that we have been trying to do has been catastrophic. There’ve just been
enormous challenges related to things that are out of [our] control… like mini-grid companies not being able
to pass their equipment through ports of entry because of changes in taxation laws.” In this case, some grant
winners planned to use their grants to expand their mini-grid operations by purchasing new materials such as
solar panels. According to respondents, the firms had to purchase the goods themselves and then SAEP
would reimburse them once the grantee received the materials. However, changes in tax law meant that the
grantees would have to pay significant import duties they had not planned for. As a result, the grantees have
not yet received their materials for which they paid upfront and cannot get reimbursed by SAEP, resulting in
substantial financial hardship.
Unintended Outcomes. In addition to intended outcomes, the ET asked about SAEP’s unintended
outcomes, which can reveal unanticipated benefits and/or consequences. When asked, respondents noted
few unintended outcomes, with a notable number (11 interviews) responding directly that there were no
unintended outcomes to the program. Respondents who noted unintended outcomes most frequently
highlighted, positively, SAEP’s efforts to further aid partners with additional interventions after completion of
the originally planned interventions (18 interviews). For example, several partner respondents noted
appreciation for how, after they started working with SAEP, they realized additional weaknesses in their
operations, which SAEP helped them to address. One such partner respondent noted, “SAEP would
periodically give a lot of documents that would summarize general issues in the region from their experience.
We would realize things we could be doing better that were coming from those documents. One of them I 
remember with regards to how you treated customer time assets. Unintentionally, it then revealed an issue
we had. We then worked with SAEP toward [addressing this] issue.” 
CONCLUSIONS
SAEP met most of its outcome targets, with the exception of new connections, which was still in progress. 
The program had few unintended or unanticipated outcomes. Though SAEP had not yet met its target for
new connections, stakeholders nonetheless believed it belonged among SAEP’s top successes. 
EQ2: WHAT ENABLED AND/OR HINDERED ACHIEVEMENTS, AND HOW WERE ANY
HINDRANCES ADDRESSED?
FINDINGS
Factors Affecting
Success
• COVID-19
• SAEP-Level
• Partner-Level
• Context-Level
SAEP’s work across its ten partner countries and myriad stakeholders and partner organizations meant that
the program encountered widely varying circumstances and factors, each of which could potentially enable
and/or hinder achievements. To assess these factors, the ET asked all interviewees a series of questions
USAID.GOV SAEP EVALUATION REPORT | 12
regarding SAEP’s work and the factors influencing its success (or lack thereof). In particular, USAID
expressed interest in learning about the impacts of the COVID-19 pandemic, and thus, in addition to asking
about general factors influencing results, the ET also asked specifically about the effects of COVID-19. 
Along with COVID-19, respondents discussed numerous factors that affected SAEP’s achievements. These
factors fell into three main categories, in descending order of noted frequency: SAEP-level factors, partner￾level factors, and context-level factors. For all factors, respondents noted a variety of cases in which the
factor could act either as an enabler to SAEP success or as a hindrance. For example, partners’ willingness to
engage with the SAEP team could be an enabler in cases where the organization wanted to engage, but a 
barrier in cases where an organization resisted SAEP’s efforts. 
The following sections describe the implications of the COVID-19 pandemic followed by an exploration of
the three factor categories that most significantly affected SAEP success.
COVID-19
Effects of COVID-19 • Delays in Implementation
• Travel Restrictions
Though respondents in a few interviews (nine) thought that COVID-19 did not have a substantial impact on
implementation, the majority (37 interviews) talked about how COVID-19 resulted in challenges to SAEP
implementation, most notably delays in implementation (21 interviews) and the inability to travel (19 
interviews). 
The specific reasons for implementation delays differed. In some cases, respondents said that shutdowns
resulted in delays, both at the start of the pandemic, and then in early 2022 when Southern Africa was hit by 
a particularly strong wave of infections. In other cases, the general slowdown in operations across the world
caused delays, as people learned to transition their work online. 
The inability to travel also caused challenges for a regional program like SAEP. Prior to the pandemic, SAEP
staff spent significant amounts of time traveling to the ten partner countries, meeting with partners and
stakeholders, and working to advance SAEP objectives. This travel largely came to a standstill with the onset
of the pandemic, according to interviewees. The COVID-19-related delays and inability to travel often went
hand-in-hand. As one IP respondent noted, “I would have traveled to Angola without COVID, to be honest,
because that’s the most effective way… we need relationships, like you need to sit together. We need to eat
together, go to project sites. That was hard. Internet connections were actually good, but we couldn’t form
those bonds or relationships in the same way.” Thus, the inability to travel affected relationship building and
the team’s ability to move activities forward as swiftly as they might have been able to do otherwise.
After the pandemic started, SAEP quickly moved to a virtual approach—both for stakeholder engagement as
well as for training and capacity building. Interviewees across stakeholder groups felt that SAEP made a 
positive and timely transition (19 interviews). Nonetheless, a significant number of respondents (particularly
among IPs and partners) noted that in-person approaches to program implementation were significantly
better than virtual approaches (22 interviews). Here, respondents did not feel it was a shortcoming on
SAEP’s side that made virtual approaches less helpful than in person ones. Rather, they believed that technical
assistance could be provided more effectively face-to-face than virtually, a common observation globally since
the start of the pandemic. In one interview, a utility representative talked about a particular training they
received for how to use a new program. They appreciated the support and virtual training received from
SAEP. However, they lamented that SAEP staff did not travel to provide direct support, sitting at the same
13 | SAEP EVALUATION REPORT USAID.GOV
computer to work through questions and issues as they went through the process. Overall, respondents
appreciated the support and noted that work together in-person could have made it even better.
SAEP-LEVEL FACTORS
SAEP-Level Factors • SAEP Technical Capabilities
• Strength of Relationships with Partners
Aside from COVID-19, respondents most frequently highlighted SAEP-level factors as influencing success.
Among these factors, respondents most frequently noted the technical capabilities and expertise of the SAEP
team across outcome areas (25 interviews) and the strength of the relationships that SAEP had with its
partner organizations (23 interviews). In terms of technical capabilities, respondents felt that SAEP’s ability to
meet partners’ specific needs, even for niche technical expertise, was a key factor for success. In most cases,
respondents viewed SAEP’s technical expertise favorably and cited them more frequently as enablers of
program achievements. However, there were a few cases where stakeholders noted that SAEP staff did not
have the necessary expertise available to suitably address their particular need. For example, one partner
representative indicated that though SAEP staff expertise was great in most areas, they lacked expertise in
some of the other areas they could use support in. “…Perhaps SAEP themselves wouldn’t have a broad
enough set of skills… the breadth of the skill wasn’t always there to cover [all our needs].”
As for the strength of relationships, respondents, particularly USAID and IP respondents, saw this factor as a 
significant driver of successful partnerships. Where SAEP and/or USAID had built strong relationships with
potential partners, they laid the groundwork for successful activities. As one USAID staff member noted,
“One thing that was surprising to me upon joining was how much it seemed program success was depending
on relationships built or preexisting on usually government ministers or officials, between Power Africa and
USAID. Or because the country had worked with us in the past… Those relationships were key.” On the
other hand, one partner representative noted how a lack of continuity in those relationships as one program
ends and another begins could hinder success, reflecting on the handover from a prior activity to SAEP. 
“…Nothing is ever perfect, but I wish that the handover was more paid attention to. The handover in terms
of the relationships and the pipeline of support. The opportunity for the successor is extremely important.” 
PARTNER-LEVEL FACTORS
Partner-Level
Factors
• Stakeholder Buy-in
• Internal Stakeholder Dynamics
• Stakeholder Willingness to Engage with SAEP
• Alignment of SAEP Goals with Partner Priorities
The second-most frequently cited set of factors were related to the partners with whom SAEP engaged.
Respondents noted stakeholder buy-in (21 interviews), concerns with internal stakeholder organizational
dynamics (20 interviews), stakeholder willingness to engage with SAEP (19 interviews), and alignment of
stakeholder priorities with SAEP’s scope and capabilities (14 interviews). It is important to note that IP and
USAID staff highlighted these partner-level factors far more often than the partners themselves did.
Three of these factors (stakeholder buy-in, willingness to engage, and alignment of priorities), are highly
interrelated. Though respondents discussed them separately, they evoked very similar themes. A 
stakeholder’s willingness to engage can be directly linked to whether SAEP’s priorities are aligned with their
own. One USAID representative noted that a key factor in success was, “how receptive [partners were] and
the level of engagement, communication of the stakeholders we’re trying to engage. How receptive they are
to return our calls right off the bat. The reason they might not be returning our calls is the areas SAEP is
focusing on may not be focusing on what the ministry is. Power generation alone, transaction distribution,
USAID.GOV SAEP EVALUATION REPORT | 14
utility operations and sustainability might not be the same priorities. Maybe they’re looking at the country[‘s]
financial distribution and oil and gas extraction. We don’t do fuel.” Thus, whether SAEP’s scope was aligned
with stakeholder priorities could have a direct impact on the organization’s interest in engaging in the first
place. A similar linkage existed with stakeholder buy-in and alignment of SAEP and stakeholder priorities.
Insufficient alignment and buy-in was a barrier to success, though respondents noted these partner-level
factors acted equally as often as drivers of success, given the large number of SAEP countries and partners. 
The fourth partner-level factor respondents highlighted was concern with internal stakeholder dynamics,
including factors such as slow review times or bureaucratic approval processes, high staff turnover, etc. An IP
respondent highlighted concerns regarding staff turnover within partner organizations. “So, this has happened
quite a few times when we were going down the path. We think that we have something good going and
then the person… gets removed and we have to start afresh, or there’s a new role. They take up their new
role and get comfortable in it, and when they do eventually get comfortable, they don’t think that their
predecessors’ priorities were real priorities, and they want us to focus on something else.” Challenges with
partner staff turnover affected the evaluation, as noted in the limitations section. When the ET reached out
to potential respondents, a significant proportion had left their positions and organizations. Additionally, a 
stakeholder cited partner-related delays in new connection activities, such as Angola’s Energy Sector
Efficiency and Expansion Project, as part of the reason SAEP has not yet achieved its new connection target.
A few stakeholders noted cases in which internal partner operations were a positive factor for success, such
as where a stakeholder was able to respond quickly and decisively to move forward with a collaboration.
However, in most cases, respondents indicated that operational dynamics within partner organizations were
hindrances to success.
CONTEXT-LEVEL FACTORS
Context-Level
Factors
• Political Factors
• Language Factors
• Cultural Differences
• SAEP’s Ability to Navigate Cultural and
Linguistic Differences
• SAEP’s Use of Local Staff
Context-level factors were related to the different countries, cultures, and subregions that SAEP covered.
These factors centered on political factors, including large political events like elections, political unrest, and
other political changes that fell outside of SAEP’s control (12 interviews), language factors (10 interviews),
cultural differences (nine interviews), SAEP’s ability to navigate these linguistic and cultural differences (12 
interviews), and SAEP’s use of local staff (11 interviews). These context-level factors, excepting political
factors, were frequently overlapping and intertwined. While some respondents focused only on how the
linguistic and/or cultural differences between countries could affect outcomes, other respondents noted how
SAEP staff’s ability to navigate these differences was the more crucial component. 
As with previous factors, respondents felt that context-level factors could act as either enablers or barriers
to success. In discussing SAEP’s ability to navigate linguistic and cultural differences, and their use of local staff,
respondents highlighted several positive cases where SAEP’s approach enabled greater success. One
respondent noted benefits to SAEP’s use of local experts. “People who understand the dynamics of the
different countries. So, people who understand how to navigate the country and the politics and use their
networks to engage correctly. That’s been quite important because it’s given us an opportunity at least to get
to the table and to present our case.” On the other hand, other respondents felt that SAEP was not always
successful in navigating these cultural and linguistic differences and/or that there was room for improvement.
For example, one USAID respondent noted, “The struggle has been… to get [partners] around the table and
to actually listen to what we have to say,” which they attributed to challenges navigating cultural aspects of
15 | SAEP EVALUATION REPORT USAID.GOV
engagement. To this end, many respondents (12 interviews) recommended that a future activity increase
and/or improve the engagement of local staff as a means of better navigating these cultural and linguistic
differences (EQ3 discusses recommendations for future programming in more detail). 
On linguistics, respondents noted that operating in the SAEP non-Anglophone countries was more
complicated than operating in Anglophone countries. In Mozambique, Angola, and Madagascar, SAEP either
had to bring in interpreters for English-speaking staff, which was seen as being less efficient or effective, or
they encountered challenges finding staff with the right skill sets that spoke the appropriate language.
Regarding Angola, an IP staff member noted that, “One of the biggest constraints that probably the team
faced when they went into Angola was the language barrier… A part of the team couldn’t really understand
the language, and this really created a huge problem.” Despite the initial challenges, others believed that SAEP
eventually mitigated these challenges through staffing changes, and several stakeholders felt that SAEP found
success working in Angola and Mozambique (EQ5 discusses relative success between countries in more
detail). Madagascar, on the other hand, posed the biggest challenge since it was the only French-speaking
country under SAEP, according to respondents. 
The final contextual factor influencing success involved political factors, which respondents felt were largely
outside the control of SAEP (12 interviews). Political factors included changes due to elections, political
unrest, and shifting political priorities, as well as political priorities of the U.S. Government as they related to
SAEP’s operations. Several IP respondents noted the influence of political changeovers, in particular, on the
high rates of staff turnover within partner organizations. Regarding the influence of USG political priorities, a 
few respondents noted the inability of SAEP to operate in Zimbabwe as a constraint.
MITIGATION EFFORTS
Mitigation Efforts • SAEP Staffing Changes
• Flexibility and Adaptability • Adding Additional Technical Assistance
Many respondents, particularly from partner organizations, lacked familiarity with SAEP’s internal operations
or decision making. Thus, far fewer respondents spoke concretely about SAEP’s mitigation efforts (21 
interviews) than were able to speak to challenges writ large (53 interviews). Where respondents noted
mitigation measures, respondents mentioned SAEP staffing changes (10 interviews), SAEP’s efforts to adapt
to the changing environment (10 interviews) and adding additional technical assistance (four interviews).
Regarding staffing changes, respondents noted that SAEP was able to modify their staffing approach over
time to address emerging partner needs and performance issues on their teams. Respondents typically
reflected positively on these staffing changes.
Respondents also appreciated SAEP’s efforts to remain flexible and adaptable (EQ3 discusses this in depth), 
which allowed them to mitigate challenges as they arose. In the case of Zambia, for instance, respondents
reflected on the significant work SAEP initially conducted with stakeholders. But, after elections brought in
new political leaders, the environment changed and SAEP was no longer able to work there as extensively
and the Program shifted resources to elsewhere in the region. The ET heard from a stakeholder that yet
another significant political change in 2021 allowed SAEP to rekindle their work in Zambia. Interviewees also
noted another significant shift, wherein SAEP was unable to work in Angola for the first few years of the
program, but this changed in later years and SAEP quickly ramped up their work to achieve positive results
(EQ5 discusses both the Zambia and Angola cases in more detail).
Finally, SAEP adapted to evolving and emerging partner needs by offering additional technical assistance. In
the Malawi case, for instance, respondents talked about how SAEP coordinated with the Millennium
USAID.GOV SAEP EVALUATION REPORT | 16
Challenge Corporation Compact there to ensure that they addressed stakeholder needs, which involved
adding several new interventions for SAEP.
CONCLUSIONS
The COVID-19 pandemic was a notable obstacle to SAEP implementation, as it was for most businesses and
organizations around the world. The pandemic led to delays and an inability to travel, which, in turn, led to
potentially lower-impact interventions than if in-person activities were an option. 
Outside of COVID-19, factors affecting success included SAEP-specific factors like the strength of SAEP’s 
technical capabilities and the strength of relationships between SAEP and its partners, partner-level factors
such as stakeholder buy-in and alignment of priorities, and context-level factors such as political influences
and cultural and linguistic factors. Given the regional nature of the SAEP program and the wide number and
variety of stakeholder groups with whom SAEP interacted, each of these factors emerged in both positive
and negative forms. As a whole, however, the SAEP-level factors were largely seen as enabling success while
the partner- and context-level factors were mixed, depending on the given situation.
Given SAEP’s mostly positive achievements against its performance targets, and the largely positive feedback
from EQ1 about successful achievements, SAEP appears to have adequately negotiated the challenges they
faced to continue making progress toward key outcomes. Nonetheless, room for improvement remains. 
EQ3: HOW DID THE PROGRAM DESIGN, TARGET SETTING, AND OPERATIONAL DECISION￾MAKING SHAPE AND CONTRIBUTE TO THE ACHIEVEMENTS OF THE PROGRAM? 
FINDINGS
Based on discussions with USAID during the evaluation design process, this EQ reports on numerous aspects
of program design and implementation, including overall strengths and weaknesses, operational decision
making, SAEP’s iterative approach to work planning, SAEP’s GESI approach, and stakeholders’ 
recommendations for future programming. The ET discusses findings regarding the target setting process
under EQ4. The following sections outline the findings to each of these components of EQ3.
PROGRAM DESIGN & IMPLEMENTATION
Strengths
• Strong SAEP Staff
• Strong Communication
Weaknesses
• Staffing Concerns
• Insufficient Local Input
All stakeholder groups provided feedback regarding the strengths and weaknesses of the SAEP program.
Overwhelmingly, they noted the presence of strong SAEP staff (24 interviews) as a key strength. As one
USAID respondent indicated, “I think Deloitte has brought really strong technical expertise to their team.
And so when they make recommendations, we feel confident that they’re being based on some of the best
practices.” Respondents considered good communication with the SAEP team as a strength (23 interviews).
One partner respondent noted, “The communication was transparent with SAEP. We worked together
without hierarchy for the electrification in rural areas. The assistance for information was there for
operators. We always worked together.” 
In terms of perceived weaknesses, respondents highlighted staffing concerns as a consistent theme (14 
interviews) that COVID-19 exacerbated. As discussed in EQ2, the pandemic greatly affected SAEP staff’s 
17 | SAEP EVALUATION REPORT USAID.GOV
ability to travel and provide in-person implementation. Some respondents noted COVID-19 contributed to
turnover and limited staff’s ability to conduct their work as effectively as before. Other respondents noted
concerns with turnover were present throughout program implementation. One partner respondent stated,
“Perhaps a point of clarity on SAEP’s side would be to better define roles so that the POC [point of contact]
isn’t shifted and someone else from SAEP is brought in when headway being made is with a different person.
I’m not sure the internal dynamics, but that stymied our progress.” 
Respondents also noted insufficient local input as a challenge (12 interviews). IP respondents, in particular,
indicated this was a primary challenge. One IP respondent stated, “The weaknesses from where I’m sitting
could have been that the program was designed without the participation of the prospective clients…the real
needs of the clients are so that you’re working to give things that people actually want.” A partner
respondent indicated their concern with, “The program design from the start. From my understanding, it’s 
something decided, and a quick study done by SAEP and proposed to our principals for it to be agreed on.
Technical officials were never involved in that initial concept.” As noted in EQ2, alignment between program
and partner priorities is key to success; one of the best ways to ensure maximum alignment is to engage local
stakeholders throughout the project lifecycle, according to respondents.
OPERATIONAL DECISION MAKING
SAEP
Decision Making Perceived Positively
USAID
Decision Making Perceived Negatively Due To:
• Frequent Request for Added Activities
• Operational Challenges
As effective decision making is a key component to effective implementation, the ET asked IP and USAID
respondents about the internal decision-making processes within the SAEP team and within USAID. Both IP
and USAID respondents viewed SAEP decision-making positively (13 interviews), with one IP respondent
stating operational decision-making was “Very easy. I can phone the [Chief of Party] or [Deputy Chief of
Party] and we’ll have a discussion, and we’ll have a plan. We’ll make sure we can deliver. Open door policy.
OPEN communication!” 
On the other hand, both IPs and USAID staff provided predominantly negative feedback regarding USAID’s 
decision-making processes (11 interviews). Primary issues included scope creep and overpromising of
activities to stakeholders, as well as internal USAID dynamics such as delays in approvals, staff turnover, etc.
Respondents identified scope creep on numerous occasions, highlighting negative impacts from frequent
USAID Offices and Missions and/or Power Africa requests for SAEP to add additional activities or partners
into SAEP’s scope. Respondents believed that these new requests were often politically motivated,
representing opportunities for the embassy and/or USAID to expand their work with new partners.
Respondents noted that these new activities did not always align with SAEP’s overall focus and pulled the
SAEP team in multiple, sometimes competing, directions, which could distract from their core objectives. 
Respondents flagged as challenges operational issues within USAID such as staff turnover and delayed
approvals. One IP respondent noted that it, “can be [a] hindrance,” while a USAID respondent stated that
due to inadequate resourcing in various offices, “[USAID hasn’t] cleared activities and made approvals as
quickly as we could have.” It is worth noting that some IP respondents reflected positively on USAID
decision-making (four interviews). One IP respondent stated, “There was an open stream of communication
that made a bit of difference in delivery.” 
USAID.GOV SAEP EVALUATION REPORT | 18
ITERATIVE APPROACH
Benefits
• Flexibility to Adapt
Challenges
• Staffing and Resourcing Concerns
A key aspect of SAEP’s implementation approach was an adaptable and iterative process for work planning.
USAID/Power Africa and the SAEP team regularly updated the work plan, using an iterative approach to
adjust or incorporate additional activities. SAEP reevaluated its work plan on a quarterly basis, allowing for
shifts on the country and regional level based on USAID requests, changing country landscapes, or country
and regional needs. IP and USAID respondents overwhelmingly found flexibility as the primary benefit to the
iterative program design approach (15 out of 16 interviews). This positive feedback included the ability to
adapt to changing program landscapes (e.g., political climates, embargos, etc.), identification of additional
funding streams, and the ability to adapt to USAID and Power Africa activity requests. As one IP respondent
highlighted, “The iterative process actually made things better. It made the elements of the activities that
were covered within the work plan, they made them much better and easier to implement… I think that
worked well.” A USAID respondent said, “I appreciated that they were willing to update the work plan
throughout the year. We valued that.” The Angola and South Africa cases demonstrate the benefits of this
flexibility, enabling SAEP to modify and expand its approach as the contexts changed. At the start of the
program, SAEP was unable to work in Angola; however, conditions improved over time and SAEP began
engaging stakeholders there midway through implementation. As for South Africa, SAEP staff reported that
their ability to work in the country shifted over time and SAEP’s flexible approach allowed them to shift
effectively. 
In only two interviews (of seven) did USAID respondents remark on any weaknesses to SAEP’s iterative
approach, suggesting largely positive feedback from USAID. Among the challenges noted, IP respondents
cited staffing and resourcing concerns (four interviews) and that the approach enabled the addition of
extraneous activities (three interviews) as its primary drawbacks. The staffing and resourcing concerns
included issues with continuity as well as managerial challenges in quickly, and sometimes significantly, shifting
the program’s approach. As one IP respondent noted, “…from the management perspective, it’s massively
difficult. The team structure we proposed in the RFP [request for proposals] does not fit what we’re
currently delivering now. We had to change and modify what we have to what doesn’t fit. It honestly never
fit for [Outcome 4] because transactions were different. So, we worked for the structure that we have.”
Regarding the latter concern with enabling the addition of extraneous activities, respondents focused on how
the iterative approach enabled USAID to request frequent additions to SAEP’s work plan (in line with
negative feedback about USAID decision-making processes).
GENDER AND SOCIAL INCLUSION 
GESI Interventions
• Mostly Gender-Focused Activities
• Trainings
• GESI-Related Reports and Recommendations to Partners
GESI Strengths
• Integration of Women into Programming
• Strong Stakeholder Engagement
GESI Challenges
• Male-Dominated Sector
The program documents identified GESI as a priority within SAEP, though different documents used differing
language. Some documents, such as the SAEP Gender Action Plan, only referred to gender while other
19 | SAEP EVALUATION REPORT USAID.GOV
documents utilized the broader GESI terminology, which goes beyond gender to include equality and social
inclusion. SAEP’s GESI work included ensuring the involvement of women in the power sector, providing
training and gender-focused activities to stakeholders, and working to provide strategic planning to
stakeholders in the form of reports and recommendations. According to the ET’s desk review, prior to the
pandemic, SAEP had a Gender Specialist who provided regional support and assisted in the development of a 
Gender Action Plan in Year 1. This Gender Action Plan detailed the scope of planned GESI activities across
SAEP’s five-year programming, including timelines, activities by country, staff responsible for activity
implementation, and related deliverables. In this Gender Action Plan, a Gender Specialist was responsible for
the majority of activities, along with the respective Outcome Leads and Country Teams. However, during the
pandemic, the Gender Specialist transitioned off the team, and SAEP contracted three GESI Advisors for ad
hoc tasks, such as support to partners for Gender Action Plans, stakeholder reports and recommendations
on the incorporation of women, and other activities. Despite the turnover in the SAEP team, IP respondents
noted that GESI activities continued. The ET assessed the perception of GESI activities directly with IP and
USAID respondents. 
GESI Interventions. When asked about the types of GESI-related interventions SAEP conducted,
respondents almost exclusively highlighted gender-focused activities (20 interviews) vs. activities addressing
other social groups (one interview). Concretely, these activities included trainings (17 interviews) and GESI￾related stakeholder reports and recommendations (13 interviews). One private sector respondent
remarked, “…One thing they just did…was to train our team in terms of gender mainstreaming and how we
can apply gender in our processes, starting from recruitment, decision making, and how [we] really could
customize and how we can make sure there’s also gender in terms of the beneficiaries...It gives us a picture
to say, just recruiting more women, it doesn’t end there, but also looking at their input and voices and all that
is part of gender, which is a good thing…we appreciate that as well.” 
GESI Strengths. IP respondents cited the integration of women into programming as the primary strength
of the GESI approach within SAEP (eight interviews). One IP respondent said of activities, “I think even in the
development of our scope of work, we always made sure that every activity highlights a gender component.” 
Another IP respondent stated, “We have a really good gender team. We did many things you could say.
Importantly, for gender within off-grid, finding opportunities where companies are interested in getting
involved and getting women or minorities involved in their work. We did several value chain assessments
across mini-grids and solar home systems for
understanding how these things work across markets
and economies. And how can you pull in women or
minorities along the way. It’s a voice that needs to be 
heard.” Among other strengths, IP respondents
highlighted strong stakeholder engagement (six
interviews). One IP respondent said, “Whenever we
engage a counterpart, you make sure [gender is]
centered. As an example… I went to [one country’s]
utility board, and I strongly recommended they appoint
a lady as the head of a project implementation unit.
And they listened to us and made that appointment
and she’s excelling…I’m feeling very good…about
making a change in this difficult area of gender.” 
GESI Challenges. SAEP’s GESI work faced a number of challenges. In particular, respondents noted the
male-dominated power sector (12 interviews). As an example, one IP respondent noted that women are
more “downstream” and have not been empowered or advocated for to take more technical power sector
Figure 10: Seamstress using SAEP-supported off-grid solar
lights. PHOTO CREDIT: ENGIE ENERGY ACCESS MOZAMBIQUE
USAID.GOV SAEP EVALUATION REPORT | 20
positions, so “there are no women to elevate.” Another IP commented on the lack of women in the sector,
noting, “Female engineers…were given the option to attend meetings and trainings. Unfortunately, the [city]
doesn’t have any female engineers to [attend these meetings and trainings].” As an additional challenge,
respondents noted that SAEP’s GESI activities were fairly superficial (seven interviews). As one USAID
respondent noted, “For me, it’s done at a superficial level. To show actual evidence for supporting women in
energy or showing the links between any training that’s done and improvement of women’s position. I don’t 
think there’s strong enough evidence to support that. You’re doing a once-off training and [determining] how
the training is implemented and… [its contribution to] elevating their status and women of value. I’ve never
seen those links made. The links between gender and women in energy is very weak.” An IP respondent
echoed this by saying, “in most of our activities, we did not have a strong gender component, with the
exception…[of] when we are interacting with our counterparts, we make them aware that…part of our
objective is to promote gender inclusion. But we didn’t have an active program designed where we work
with the counterparts to address this aspect.” This IP respondent continued to state that though work
continued on an activity basis, because of the Gender Specialist’s attrition in Year 3, work was incomplete. 
GESI Opportunities. Respondents highlighted ways in which future programs could improve the GESI
approach (seven interviews). These opportunities include incorporating GESI into the overarching program
design so that expectations and scope are clear, integrating GESI across the program rather than as
standalone tasks, clarifying USAID’s GESI expectations, and going beyond sex disaggregation to integrative
programming. To improve GESI within the sector, respondents recommended workshops with partners for
additional training and recruitment of women, activities to increase female employment within utilities, longer
timelines for GESI programming to allow for related social and behavior change, activities focused on persons
with disabilities, and local community engagement.
FUTURE RECOMMENDATIONS
Technical Recommendations
• Continue Existing Work/Focus
• Improve Sectoral Collaboration
Program Design/Implementation
Recommendations
• Localize Programming
• Provide Financial Support/Address Funding Gaps
In addition to understanding stakeholder perceptions regarding current SAEP implementation, the ET
explored potential improvements to the SAEP approach. The ET divided recommendations from
stakeholders into technical recommendations and recommendations for design or implementation
approaches. All respondent types provided recommendations. 
Technical Areas. Across stakeholder groups, respondents recommended continuing to focus on many of
the same technical areas as SAEP. Some respondents indicated generically that a future program should
continue existing work (15 interviews), while others called out specific components that SAEP already
focused on such as new connections (29 interviews), renewable energy (22 interviews), capacity building (21 
interviews), more generation (19 interviews), and more transmission (16 interviews). In another top
recommendation, respondents suggested improving collaboration and coordination between sector
stakeholders (25 interviews). This included collaboration between donors, collaboration across sector actors
at the country or regional level, benchmarking and knowledge hub projects, and co-creating projects. 
Program Design and Implementation. The ET noted a consistent theme across respondents around
localizing programming (24 interviews), ensuring approaches account for local stakeholder needs as well as
the country context, culture, and language skills. Related themes included providing more, early input on
program design and planned activities from local stakeholders (14 interviews), ensuring active engagement of
local staff (12 interviews), and ensuring adequate language and cultural skills on the SAEP team (three
21 | SAEP EVALUATION REPORT USAID.GOV
interviews each). To this end, an IP respondent stated, “The first is that there would have to be a clear
scoping mission. That would help first. Not just a concept note. A real scoping mission. To make sure the
local consent and input is there. The buy-in is there.” A partner respondent similarly noted that, “If they are
thinking of any program, engage with us to see if we agree. If SAEP comes up with their own program and
brings it to us, it is challenging.” Another partner respondent concurred, saying, “For an organization like
SAEP, the sensibilities, the organizations, beneficiaries, you have to have the localization and have the updates.
It’s not something that’s extremely hard. It ensures you have a successful project. Ensure you have
communication.” As a part of this localization process, respondents recommended ensuring that future
programs consider what other donors are doing to avoid programming duplication and to facilitate
collaboration. 
The other key recommendation for future program design was to provide financial support (19 interviews).
Partner respondents noted this recommendation most frequently. Though some respondents requested
direct funding of power infrastructure, others spoke about how future programs could facilitate financing via
connections with financiers or the creation of new financial models. These respondents focused on the
financing gap in the power sector, particularly for new connections.
CONCLUSIONS
Overall, SAEP’s design and implementation approach was well regarded and appeared to support effective
and efficient implementation, likely aiding SAEP’s achievement of results. Despite the overall positive
feedback, several areas for improvement exist. Conclusions regarding specific aspects of SAEP design or
implementation are noted below.
Throughout SAEP’s implementation, strong staff and communication remained notable strengths, with many
respondents highlighting the SAEP team’s transparency and willingness to engage as a key factor driving
activity achievements. The ET found challenges with staffing and insufficient localization as noted above. 
Internal SAEP decision making was also a program strength. The “open door policy” was a highlight of the
program, with many praising SAEP’s “above and beyond” approach to partner engagement and
implementation. However, USAID’s decision-making processes often caused delays and acted as roadblocks
to efficient and effective service delivery.
The iterative approach to work planning proved a success, offering significant flexibility and enabling active
adaptation to changing circumstances. It did cause some challenges, however, including issues with continuity
or requiring rapid shifts in resources.
SAEP interventions employed a GESI lens. However, SAEP’s approach to GESI focused heavily on gender
rather than other social strata. Its activities tended to be superficial in nature. IP respondents noted that the
strongly male-dominated sector hampered GESI activities and results. Additional opportunities for GESI work
include incorporation of GESI programming during the program design phase and adjustment of activities to
include tailored GESI activities and social inclusion approaches.
To improve on the SAEP approach, stakeholders provided numerous recommendations for future
programs. On the technical side, most respondents wanted a future program to continue focusing on areas
similar to those on which SAEP focused, particularly on new connections and power access. On design and
implementation approaches, improved localization was the top recommendation, followed by a desire that
future programs address the financing gap in the power sector.
USAID.GOV SAEP EVALUATION REPORT | 22
EQ4: WHAT WERE THE BENEFITS AND DRAWBACKS OF POWER AFRICA SETTING THE
CONNECTIONS AND MW TARGETS FOR THE PROGRAM?
FINDINGS
The first step to answering this EQ involved determining how, exactly, the original SAEP targets were set. To
do so, the ET examined program documents and asked IP and USAID respondents about the process. The
ET then asked respondents about benefits and/or drawbacks to the indicator selection and target setting
processes. The ET asked respondents about their recommendations for improving the key indicators and/or
targets for future programming. Each aspect of this EQ is discussed in turn below.
ORIGINAL INDICATOR AND TARGET SETTING PROCESS
Indicator
Selection
Process
SAEP RFP Required Some Minimum Results
• 1,000MW of new, renewable energy generation
• 3 million new connections
Deloitte Team’s Proposal/Eventual Contract
• Expanded the above 1,000MW of new, renewable energy generation to
4,000MW of combined generation and transmission capacity
• Selected the indicators tied to fee payments
The framing of this EQ assumed that Power Africa itself set the targets for MWs of new generation and
transmission as well as for new connections. However, shortly after award of this evaluation and after initial
discussions with both USAID and IP staff, the ET received differing perspectives on how the original
indicators were selected and the corresponding targets set. Thus, to ensure that the ET had an accurate
understanding of what happened in the first place, the ET reviewed both the original SAEP solicitation and
the resulting technical portions (Section C) of the SAEP contract. In interviews, the ET began EQ4 discussions
by asking USAID and IP respondents whether they were familiar with the indicator selection and target
setting process and, if so, what that process had entailed.
In only four out of 23 IP and USAID interviews, respondents could recall how this original indicator and
target setting process occurred. Even these respondents were unclear as to how exactly indicators and
targets came to be. One USAID respondent said, “I literally have not been able to find any documentation,
and I went looking hard within Power Africa or within the [regional] mission to understand how that number 
got set.” Due to the limited amount of information from interviewees, the ET relied heavily on document
review for understanding the indicator selection and target setting process.
The RFP for SAEP included several required indicators (outlined in RFP Attachment 4), and its statement of
objectives (SOO) outlined some minimum results for each of SAEP’s five outcome areas. USAID then
required bidders to: 1) propose indicators and targets that at least met the minimum result requirements,
and 2) using a subset of the proposed indicators and targets, design a fee schedule tying fee payments to
achievements of the targets (hereafter referred to as the “fee indicators”). The only guidance regarding
which indicators to select as fee indicators appears in Section L.11.3: “Offerors should also propose a fee
schedule based on indicators and targets proposed in Annex B: Life of Project Results Framework.” Bidders
had to submit their planned Monitoring, Evaluation, and Learning (MEL) approach in Annex B with their
proposal. Thus, USAID left it largely to bidders to decide which indicators would measure success and, within
that, which indicators would be fee indicators. The fee indicators and their associated targets were
embedded in the executed contract.
23 | SAEP EVALUATION REPORT USAID.GOV
The SOO set two minimum results related to generation and new connections, both under Outcome Four:
Renewable Energy and Energy Efficiency Technologies and Practices Locally Demonstrated and Scaled:
● “At least 1,000MW of new on-grid and off-grid renewable power generation projects brought to
financial close as a result of Contractor assistance.
● At least 3 million new connections added in Southern Africa as a result of Contractor assistance.” 
Based on Section C of the SAEP contract (and corroborated in interviews), Deloitte maintained the 3 million
minimum for new connections in its proposal but expanded on the MWs of new renewable power
generation by proposing 4,000MWs of generation and transmission, combined. Deloitte selected the
following as fee indicators (per the Year Four Annual Report, Annex G):
● Capacity (MW) from transactions SAEP supported that achieved financial closure
● Direct electricity access: Number of new grid and off-grid actual direct connections
● Number of laws, policies, strategies, plans, or regulations officially proposed, adopted, or
implemented
● Submission of required deliverables as per Section F of the Contract
● Generation and transmission capacity (MW) pending financial closure
Based on discussions and interviews, the fee indicators have driven and focused SAEP’s interventions due to
Deloitte’s financial incentive to ensure achievements. Thus, these fee indicators set the stage for SAEP to
place emphasis on these results. However, the original solicitation did not focus heavily on new generation,
transmission, or even new connections. Rather the SOO focused on the following five outcomes (which are
used to structure the program):
● Improved regulation, planning and procurement for energy;
● Improved commercial viability of utilities;
● Improved regional harmonization and cross border trade;
● Renewable energy and energy efficiency technologies and practices locally demonstrated and scaled;
and
● Increased human and institutional capacity. 
The SOO (and Section C of Deloitte’s contract) placed the new generation/transmission and new connection
indicators under Outcome Four for renewable energy. Thus, the original RFP appears to place greater
emphasis on the five enabling outcomes above rather than on the longer-term results of new generation and
new connections. 
At some point early in implementation, however, the generation, transmission, and connection indicators
shifted from being lower-level indicators under Outcome Four to being impact-level indicators, as illustrated
in SAEP’s first Performance Monitoring and Evaluation Plan (PMEP), from 2017. The PMEP places new
generation and transmission and new connections at the Project Impact level rather than under Outcome
Four. This structure continues in the most recent PMEP version (version 10). 
INDICATOR/TARGET BENEFITS
Benefits
• Targets Focused Implementation on
Core Goals
• Regional Targets Offered Flexibility
• Targets Pushed SAEP to Strive for Higher
Achievement
USAID.GOV SAEP EVALUATION REPORT | 24
When asked about the benefits of the existing fee indicators, respondents were largely positive regarding the
use of fee indicators for managing the contract, though there was notable room for improvement. Overall,
respondents felt that the existence of fee indicators helped keep the team focused on the core goals of the
program (five interviews) and that the fee indicators pushed the team to keep trying to achieve even higher
results (five interviews), since there was a financial incentive for them to do so. Respondents appreciated the
flexibility target setting at the regional rather than country level (three interviews) offered. 
Results Orientation. As noted above under EQ3, USAID often approached the SAEP team with new
intervention ideas and requests. These requests could sometimes pull the SAEP team in multiple directions.
Five interviewees noted the fee indicators helped keep the program focused. One USAID respondent noted
SAEP’s flexibility in, “Being able to ramp up based on USAID requests and needs. And yet, they still seem to
remain very committed to core targets… Their fee was attached to targets, and I think that’s why they were
looking to address their performance indicators… I think it was keeping the eye on the prize.” Thus,
respondents said that the fee indicators helped ensure SAEP’s focus on the core goals and lessened
distractions toward less relevant interventions.
Striving for Higher Achievement. Five
respondents noted that the fee indicators helped push
the SAEP team to keep achieving results, even in the
face of challenges. Even after noting that some of the
targets were unrealistic, one USAID staff member
remarked that, “It kept them hungry. It kept them
working… We were just very open about that… at the
beginning. They’re like, we’re going to go after it. If
everything goes well, maybe we’ll get it… I think it did
push them.” Another USAID staff person similarly noted
that, “They seem to have done a great job pushing to try
and beat the numbers that we set for them.” 
Regional Flexibility. Four respondents appreciated the regional nature of the fee indicators, which
allowed SAEP to adapt over time, particularly in response to major shifts such as the challenges in Zambia or
new opportunities in Angola. Regional targets helped ensure that SAEP focused their resources on the most 
fruitful efforts, regardless of where those opportunities were located. One USAID staff member noted
about the regional approach that, “I think it was effective because it helped SAEP see where they could be 
most impactful… That work could have more impact per dollar.”
Figure 11: Wind turbines from an SAEP partner. 
PHOTO CREDIT: ENEL GREEN POWER
INDICATOR/TARGET DRAWBACKS
Despite largely positive feedback regarding the utilization of fee indicators in general, respondents noted
several specific challenges. First, most respondents believed that at least some of the targets were unrealistic
(11 out of 21 interviews). Additionally, respondents noted concerns regarding the attribution of outcomes to
the SAEP program (seven interviews), that the selection criteria for the indicators and targets were unclear
(six interviews), concerns regarding the timelines for achieving intended results (five interviews), insufficient
Drawbacks
• Unrealistic Targets
• Concerns with Attributing Results to
SAEP
• Lack of Clear Criteria
• Timeline Concerns for Achieving Results
• Insufficient Localization
• Regional Targets Emphasized Big Projects
over Small Projects
25 | SAEP EVALUATION REPORT USAID.GOV
localization (four interviews), and that the focus on big regional targets led the team to prioritize big
interventions over smaller, but still beneficial, ones (four interviews).
Target Feasibility. Whether the fee indicator targets were realistic was central to answering this EQ.
Most USAID and IP respondents felt that the targets, especially the target for new connections, were
unrealistic (11 interviews). Respondents may have emphasized the new connections target because this is the
only fee indicator SAEP has not yet achieved. However, respondents flagged the financial challenges in
increasing new connections, particularly for on-grid connections. SAEP did not directly fund grid extension
programs and only provides technical assistance. But, as one USAID respondent noted, grid extension is
inherently costly. In many countries, the “low hanging fruit” of connecting densely populated areas where the
potential income from new customers may cover the costs of grid extension, is largely complete. Connecting
rural areas is expensive and often not commercially viable and must be subsidized. As one IP staff member
said, “If I had been a part of target setting, knowing the situation in Southern Africa, I would have said that the
3 million connections in four years was a little bit ambitious. Yes, it was a good target to aim for, but knowing
the problems that are in the region generally, I thought from the start 3 million new connections was a bit
ambitious and 2 million, which is where we are now, [could be feasible]. I would have said maybe 2.5 million, 
but definitely not more.” USAID staff made similar points, with one stating specifically that, “I think 2 million
[new connections] would probably have been the ambitious yet achievable number.” 
Attribution of Results. Seven respondents critiqued the attribution of results to SAEP. Here,
respondents noted two concerns: 1) whether SAEP could really claim credit for all the MWs and new
connections when SAEP was often supporting other entities such as the World Bank or African
Development Bank, and 2) the tenuous links between SAEP’s technical assistance and the fee indicators of
MWs and connections. 
Under the first concern, respondents focused on what “counted” toward SAEP’s indicators. SAEP counted
results whether they provided extensive advisory support, or they supported just a small component of a 
much larger intervention. One USAID respondent recounted an incident in which SAEP had claimed MWs
of new generation, but the country government counterparts disputed this claim, saying that SAEP’s support
had been insufficient for them to claim the MWs as their own results. 
The second concern about attribution focuses on the theory of change and the necessary assumptions
required to move between the technical advisory services SAEP provided and the fee indicators of
generation, transmission, and new connections (five interviews). As noted above, this challenge may have
roots in the original design of the RFP, which did not originally focus heavily on new generation, transmission,
or connections. Some respondents felt that the link between technical assistance and the fee indicators is not
straightforward and often only exists in the longer-term, only to be realized after SAEP ends. For example,
interventions to improve the commercial viability of utilities would likely result in benefits such as reduced
losses, more efficient processes, improved cost recovery, etc. And, in the long term, this could contribute to
more financial resources to support the purchase of additional power or to expand the grid. But that change
would not likely occur within the five-year timeframe of a program like SAEP. As one IP respondent noted,
“How did… enabling environment work help with MWs and connections?”
Unclear Criteria. The next most frequently cited concern with the indicators and targets was that it was
unclear how they were set or selected. As noted above, very few respondents had any idea where USAID’s 
original minimum requirements came from or understood how the indicators and targets shifted during the
proposal process. The ET received the output from a McKinsey-led study of potential new connections in
Africa by 2030, which USAID may have used as a part of determining the new connections target. However,
the ET does not have specific evidence that this data were used or how it was used to determine the new
USAID.GOV SAEP EVALUATION REPORT | 26
connection target for SAEP.
5Reflecting the overall concern about the lack of clear criteria in the process, an
IP respondent noted that, “It was difficult for me to assess whether they were realistic or not because one of
the things I would have expected to see… was how the metrics broke down into performance areas for
each of the outcomes and for each of the key staff within each of the outcomes. And that framework didn’t 
exist.” 
Timeline for Results. Next, respondents cited concerns with the timelines needed to achieve the
indicator targets, particularly for the fee indicators. Here, respondents expressed concern about two aspects:
1) that realizing MWs and connections after technical assistance takes longer than five years, and 2) that
power transactions have very long timelines. The first concern echoes the above attribution concerns and
the time necessary to move from technical assistance outcomes to new generation, transmission, or
connections. The second concern relates to the time it takes for a power transaction to achieve financial
close. Even under the best circumstances, generation and transmission transactions take many years. Unless a 
program like SAEP starts supporting a power transaction right away in its period of performance or the
transaction is already in process, it is unlikely to achieve financial close before the end of five years. This
disincentivizes a program like SAEP, whose fee is tied to achieving these outcomes, from supporting longer￾term efforts that could be very beneficial, but that will not achieve financial close before the end of the
program. One respondent expanded this concern to new connections, saying, “The number of connections
that the program is going to actually achieve often come after the period of performance. For example, if
we’re bringing liquidity to a solar home system company, it takes a while for them to actually get to the point
of selling those solar home systems and then working with families to make sure that they’re productive.”
Respondents noted that the long timelines for achieving the fee indicators sometimes drove which
interventions to pursue.
Insufficient Localization. Next, respondents (particularly IP respondents) highlighted insufficient local
input and/or calibration of indicator selection and targets setting to local needs and realities (four interviews). 
As one USAID respondent framed it, “But clearly the targets were unrealistic. And I think… setting huge
generation targets without understanding the market… I think the [countries in the region] should have been
consulted. I think the targets were not necessarily in line with the realities of the region.” 
Emphasis on Larger Power Markets. Finally, respondents highlighted the concern that high, regional
targets pushed the SAEP team to focus on the largest interventions so they could reach the targets and claim
the associated fee (four interviews). But, given the very different sizes of the power markets across Southern
African countries, this meant SAEP emphasized countries with larger power markets. Even though a small
power plant could have a huge impact on a country with a small power market, SAEP was less likely to
support it than a much larger power plant in a much larger market like South Africa, respondents said. One
IP staff member noted, “[With a] straight number like 3,000MW [of generation], in that case, you’re
scrambling looking for that number. So, you look at the transaction costs of 5MW here vs. 50MW in the
other country. But you might have had more impact with that 5MW.” This finding is further supported by
SAEP’s transaction reporting, which shows that 95 percent of new generation capacity and 66 percent of
new transmission capacity reaching financial close with SAEP support occurred in the four largest power
markets (by installed generation capacity).
6
 
5 McKinsey Connections Data Spreadsheet provided by USAID. 6 Installed generation capacity estimates based on data from the Power Africa website: https://www.usaid.gov/powerafrica/wherewework. The
four largest power markets under SAEP are: South Africa, Angola, Zambia, and Mozambique. 
27 | SAEP EVALUATION REPORT USAID.GOV
RESPONDENT RECOMMENDATIONS FOR FUTURE INDICATOR AND TARGET SETTING
Recommendations • Improve Localization
• Increase Stakeholder Engagement
• Establish Country-Level Targets
In addition to asking respondents about the challenges encountered with the SAEP indicators and targets, the
ET asked about their recommendations for improving on the process for future programs. The top
respondent recommendations were interrelated: to improve localization in indicators selection and target
setting (eight interviews), increase stakeholder engagement in the process (seven interviews), and create
country-specific targets (six interviews).
These interrelated recommendations address some of respondents’ central concerns with the SAEP targets:
that they were unrealistic, they did not reflect local needs or priorities, and that they incentivized larger
interventions over smaller, potentially more beneficial ones. The desire for more local engagement mirrors
feedback under EQ3 and reflects a desire to ensure that what a future program aims to achieve is linked
with what local stakeholders actually need. As one USAID staff member noted, “If you focus on technical
capacity in a country that doesn’t need it because you’re trying to achieve the target, then you’re focusing on
a country that doesn’t need it. Country by country context, I think, is key.” The desire for some country-level
targets built on this need for local input in target setting and attempted to address the size imbalances
between countries in the region to avoid the larger power markets from attracting all the attention at the
expense of the smaller markets. As one example, an IP respondent suggested making the targets a 
percentage of the market rather than just a fixed number. This would right-size, he said, the targets to the
individual markets. Instead of saying 200MW of new generation, which could double the generation capacity
in some countries but be a drop in the bucket in others, he suggested the target could be a 20 percent
increase in generation in each country (or a certain number of countries).
CONCLUSIONS
Overall, the fee indicators and targets steered SAEP, keeping the program focused and pushing the team to
maximize achievements. However, the ET found several issues with the existing indicators and targets. Most
significantly, the selected indicators and the set targets did not appear based on any clear or documented
criteria. Indeed, the EQ itself focused on the benefits and drawbacks of USAID selecting the indicators and
setting the targets. However, it appears that both USAID and Deloitte played significant roles in indicator
selection and target setting. The ET found that the focus of the program shifted over time. Though new
generation and connections were lower-level results in the RFP (situated under Outcome 4), they rose to
impact-level outcomes during implementation. As a result, the structure of SAEP, which centered around the
five core outcomes from the RFP (which did not include generation or new connections) became a 
somewhat awkward fit. This contributed to respondent impressions that SAEP’s technical assistance was
sometimes only loosely related to the fee indicators of new generation, transmission, and connections.
The fee indicator targets, in particular for new connections, were unrealistic. In the case of new connections,
USAID set the 3 million target in the RFP. Though time remains for SAEP to achieve this goal before the end
of their period of performance, it was an ambitious goal; often, stakeholders noted that 2 million or even 2.5 
million new connections would have been an ambitious but more realistic goal.
Additional challenges in indicator selection and target setting centered around SAEP’s ability to truly claim the
results represented in their fee indicators, the misalignment of timeframes between when SAEP needs to
“achieve” results vs. how long power transactions actually take in practice, insufficient localization and local
input in target setting, and how the large, regional fee indicator targets pushed SAEP to focus on larger
interventions at the expense of smaller ones.
USAID.GOV SAEP EVALUATION REPORT | 28
Finally, respondents provided recommendations that all centered on improving local input and localization in
the indicator selection and target setting process, including increasing stakeholder input and introducing
country-level targets to mitigate concerns about how regional targets favor larger power markets.
EQ5: IN WHICH COUNTRIES DID SAEP HAVE THE MOST SUCCESS VERSUS THE LEAST
SUCCESS, AND WHY?
FINDINGS
Given that SAEP worked across ten countries, with varying levels of engagement across countries, EQ5
endeavored to understand how success varied across these contexts and why. To this end, the ET asked IP
and USAID respondents about which countries SAEP had the most success in versus the least success and
why. In 22 interviews, respondents felt sufficiently familiar with SAEP’s activities across countries to respond
to these questions.
MOST SUCCESSFUL COUNTRIES
Most Successful
Countries
• Malawi
• Mozambique • Zambia
According to respondents, SAEP achieved the most success in Malawi, Mozambique, and Zambia (15, 12, and
nine interviews, respectively). When asked why these were the most successful countries, the factors
respondents cited mirrored the factors noted under EQ2.
Malawi. Respondents focused on stakeholder’s positive attitudes and responsiveness (their willingness to
engage with SAEP), that SAEP met a clear and present need and at the right time (alignment of priorities),
and that SAEP had strong staff in-country (strong technical capabilities). 
In terms of stakeholder receptiveness, one USAID staff
member stated, “I’m going to say the same thing for
Malawi… there were willing and stable stakeholders
who were there for SAEP to engage. Not just the
government but other [development finance
institutions] to build an environment that was active
and opportunities for the project to participate in.” A 
partner respondent echoed this sentiment, “I think
Malawi, because I think, with a positive mind, there’s 
been much more impact in Malawi on the support
from SAEP. I think these successes have also been
driven by the need because I think the Malawi [grid] is
not yet interconnected and there is much more benefit
when Malawi gets connected.” In addition to
stakeholders’ willingness to engage, respondents felt
that success in Malawi was due to good timing of the
program. As one respondent noted, “...SAEP
encountered the right projects at the right time to partner or feed into [in Malawi]. And I think the support
for SAEP came at a good time for them, it’s actually clear objectives and indicators of how they can benefit
from the interconnected grid to the power pool. So, I think it came at the right time in terms of the support
for the tools that they actually recommended. The support also on the interconnection itself, the
Figure 12: SAEP-supported Solar Home System (SHS) Kick￾Starter Program in Malawi. 
PHOTO CREDIT: ZUWA ENERGY
29 | SAEP EVALUATION REPORT USAID.GOV
transmission line and also various works in terms of the capacity building that they are also offering to Malawi.
So, I think more success was in Malawi.” 
Mozambique. While less specific about what led to SAEP’s success, respondents’ feedback focused on
how embedded SAEP was in the country and how the program built strong relationships with stakeholders.
One IP staff member noted, “We’ve been very involved in Mozambique. We had an embedded advisor. Like
we have in Angola now for the transmission line. And we’ve been there every month and preparing to help
them with interconnection. So, we have a very strong relationship with the people there.” 
Zambia. Following Malawi and Mozambique, the third most frequently cited successful country was Zambia
(nine interviews). However, Zambia offers a unique case in that respondents also cited it as one of the least
successful countries, placing third on that list as well (three interviews). On its success, respondents cited
SAEP’s intensive engagement and strong staff, as well as counterparts’ receptiveness to SAEP. Factors
contributing to a lack of success centered on significant shifts in Zambia’s political climate. This caused
significant change in SAEP’s counterparts and resulted in shifts in priorities. Zambia pulled out of a major loan
from the World Bank to which SAEP was providing technical assistance. Thus, though SAEP’s work in Zambia
started strong, these shifts significantly affected SAEP’s ability and effectiveness in the country. 
LEAST SUCCESSFUL COUNTRIES
Least Successful
Countries
• Madagascar
• Lesotho
• Botswana
• Eswatini
• Namibia
Fewer respondents provided feedback about which countries experienced the least success (14 interviews
versus 22 that discussed the most successful countries). Respondents most frequently cited Madagascar and
Lesotho as the least successful (six and four interviews, respectively). Botswana, Eswatini, Namibia, and
Zambia tied for third most cited (three interviews each).
Madagascar. In Madagascar, respondents noted challenges with inconsistent and complicated bureaucracy
leading to slow response times, SAEP’s difficulties navigating the culture and language, which contributed to
communication challenges, a changing political environment, and Madagascar generally being a difficult place in 
which to work. Inconsistent and excessive bureaucracy significantly affected success in Madagascar, according
to respondents. Several respondents that mentioned Madagascar discussed SAEP’s grant program for mini￾grids. The program aimed to alleviate financing constraints and expand new connections by supporting
renewable energy companies with a subsidy to expand their mini-grid networks and connect households,
clinics, and other facilities. However, the government changed their regulations that had previously exempted
renewable energy products from import taxes. Thus, the government expected the companies to pay taxes.
This led to back-and-forth discussions between the government, the developers, and SAEP, which led to
significant delays. With the delays came mounting costs (i.e., storage costs on top of the import taxes). An IP
partner expressed the impact of this change in rules, “Materials were ordered but there is a problem with
clearance. The Ministry of Finance changed the rules. So, they had to pay instead of being free of duty, and
they cannot pay. The goods have been in storage for one year at the port. So now there are storage costs.
We are trying to intervene with the Ministry of Finance and Ministry of Energy but now storage is
US$150,000, duty is US$60,000, the goods were about US$40,000. No ministry wants to take the
[responsibility]. And they [the developers] can’t pay…” To date, the materials have been unable to clear the
port.
USAID.GOV SAEP EVALUATION REPORT | 30
Lesotho, Botswana, Eswatini, and Namibia. In these countries, respondents noted that a key driver
limiting success was the small size of their power markets, limiting the effort that SAEP focused there. As 
discussed under EQ4, the regional nature of the fee indicators pushed SAEP to focus on interventions that
would help them achieve their targets. This encouraged SAEP to focus, for example, on a 100MW
generation plant in a larger market like South Africa vs. a 10MW plan in Lesotho, even though the latter
could potentially have a more significant social benefit than the former. One USAID respondent noted, “But
then the other ones like Botswana, like Lesotho, like Swaziland, we were never going to be able to, just from
a budget perspective, the ultimate need, to get to the targets. So, letting the targets drive our presence in
these countries [that] had just such little population that we were trying to do, high value or innovative,
interesting things... But they were never going to be the driver of where the projects were going just because
of the targets.” Other interviews echoed this sentiment. One USAID respondent noted, “Eswatini and
Lesotho. I didn’t think there was much target[ed] effort to do much work there. The level of engagement
and investment relative to stand up relationships and/or an activity wasn’t there.” This respondent believed
that the effort needed to establish effective partnerships relative to the size of the potential interventions
was not sufficient to encourage SAEP to work there.
CONCLUSIONS
SAEP achieved the most success in Malawi and Mozambique, mostly due to the strength of the relationships
they built, the strength of the SAEP technical team, and stakeholders’ willingness to engage. These factors
mirror the general factors influencing success noted under EQ2.
Respondents ranked Zambia third on both the most and least successful country lists. Zambia proved to be a 
unique case in that SAEP worked heavily there in the first few years of implementation before a significant
political shift led to changes in stakeholder priorities and the cancelation of a large power transaction that
SAEP was supporting. After this point, SAEP struggled to achieve significant progress in Zambia and thus
shifted its focus to work in other countries, a possibility that the flexible, regional targets enabled. 
SAEP achieved the least success in Madagascar, Botswana, Eswatini, Lesotho, and Namibia. In Madagascar,
SAEP encountered significant bureaucratic challenges as well as difficulties navigating the cultural and linguistic
environment, which was very different from the rest of the region. In the remaining countries, the small size
of their power markets played a significant role in the lack of success. As discussed in EQ4, the fee indicators
encouraged SAEP to focus on large interventions that could help them achieve the indicator targets. Though
SAEP had some successful interventions in these countries, they generally received less attention than larger
countries. Where successes occurred, they did not substantially contribute to achieving fee indicator targets. 
EQ6: HOW IS THE SAEP PERCEIVED AND VALUED BY THE STAKEHOLDERS OF THE
PROGRAM?
FINDINGS
Perceptions
• SAEP is improving the sector, particularly in
terms of power access
• Satisfaction with SAEP’s work
• USAID staff rated SAEP more
positively than partners did (though
partners were also positive, overall)
Stakeholder
To assess stakeholder satisfaction and perceptions regarding SAEP, the ET asked respondents about the
extent to which they thought SAEP contributed to improving the power sector in both the quantitative
31 | SAEP EVALUATION REPORT USAID.GOV
survey and the qualitative interviews. The survey asked respondents how SAEP contributed to particular
outcomes in the sector and overall satisfaction with SAEP. The following sections discuss findings from each. 
The quantitative survey asked respondents to rate the extent to which they felt SAEP had improved the
power sector on a scale of 1 to 3, with 1 being no improvements and 3 being significant improvements. The
survey first asked about SAEP’s contributions to the overall sector, followed by questions about specific
components, such as generation or transmission. On the overall sector, respondents provided SAEP an
average rating of 2.3, indicating that respondents view the improvements as somewhat significant. There
were notable disparities by respondent type, however. USAID respondents reported the most positive
sentiments while partner organizations reported the lowest ratings. IP respondents fell in the middle (see 
Table 4). 
In the qualitative data, respondents shared similar views about SAEP’s contributions to the sector.
Respondents in the vast majority of interviews reported positive feedback, noting either significant
improvements (19 interviews) or some improvements (21 interviews) to the sector. 
Table 4: SAEP Contributions to the Sector(Scale of 1-3; 3=Significant)
SAEP CATEGORY USAID IP PARTNERS OVERALL
Power Sector (Overall) 3.0 2.6 1.9 2.3
Generation Capacity 3.0 1.9 1.4 1.7
Transmission Capacity 3.0 2.3 1.3 1.8
Power Access 2.5 2.5 2.3 2.4
Energy Efficiency 1.0 2.0 2.0 1.9
Power Sector Policy 2.0 2.8 1.8 2.2
Power Sector Regulation 3.0 2.7 1.5 2.1
Power Sector Coordination 3.0 2.2 1.8 2.0
Private Sector Investment in Power 3.0 2.0 2.2 2.2
Greenhouse Gas Emissions 2.0 1.8 2.0 1.9
In addition to rating the overall improvements that SAEP made to the power sector, the survey asked
respondents about SAEP’s contributions to nine sectoral areas. As Table 4 shows, respondents rated SAEP’s 
contributions to improving electricity access the highest, with an average rating of 2.4. On the other hand, the
other key outcomes of generation and transmission received the lowest overall ratings (1.7 and 1.8,
respectively). These findings are interesting to note, as EQ1 showed that SAEP has achieved the latter targets
but not for new connections. Thus, stakeholder perceptions of success are opposite what the performance
indicators show. However, the survey data mirror EQ1 interviewee feedback, where respondents noted
new connections as a top success, despite not meeting the anticipated target. As highlighted under EQ1, this
dichotomy may be related to stakeholders’ perceptions that the new connection target was unrealistic and
that SAEP may not be able to claim attribution for all its generation and transmission results.
As with SAEP’s overall contributions to the sector, disaggregated sub-sector ratings by respondent type
highlight that USAID respondents felt most positive about SAEP’s contributions while partners felt the least
USAID.GOV SAEP EVALUATION REPORT | 32
positive in almost all cases (contributions toward greenhouse gas emissions was an exception). Female
respondents tended to provide more positive ratings than did male respondents.
SATISFACTION WITH SAEP
The survey asked about respondents’ level of satisfaction with the SAEP program, on a scale of 1 to 5, with
higher scores indicating they were more satisfied. Most respondents were satisfied with SAEP, with an overall
average of 4.0 (see Figure 13). 
Figure 13: Satisfaction with SAEP by Respondent Type (Scale of 1-5; 5=Very Satisfied)
4.5
4.1
3.8 4.0
3.5
4.0
4.5
5.0
Level of 
Satisfaction
USAID/USG IP Partners Overall
 Similar trends across respondent types appear. USAID staff were the most satisfied with SAEP while
partners were least satisfied, though it is important to note that with an average score of 3.8, even partners
were fairly satisfied with the program. 
CONCLUSIONS
Overall, respondents and interviewees felt that SAEP made notable improvements to the power sector, and
they had high levels of satisfaction with the program. Contrary to performance indicator data, stakeholders
were happier with SAEP’s work to increase new connections (for which SAEP has not yet met its target)
than they were with SAEP’s contributions to new generation and transmission capacity (where SAEP has met
its targets). This is likely due to a combination of poorly calibrated targets and the attribution concerns
respondents raised regarding the fee indicators under EQ4.
The differences in the perceptions between USAID respondents and partner respondents are notable and
show a likely gap in USAID’s understanding of partner needs and sentiments regarding the program. 
RECOMMENDATIONS
Based on the findings and conclusions of this evaluation, the ET has the following recommendations for
USAID as it proceeds with designing the SAEP follow-on program:
1. USAID should ensure the localization of future programs. Localization involves active
engagement with local stakeholders throughout the project life cycle so that the activities meet the
needs of those it is intended to help. Findings across EQs highlighted the importance of localization in
design, indicator target setting, and staffing. Localization affected both stakeholder buy-in and
engagement with SAEP and the achievement of results. In some cases, SAEP navigated local needs
well and managed the cultural and linguistic differences across the region, but room for
improvement remains. Though both USAID and Deloitte played parts in the localization of SAEP, for
future programming, USAID should begin the local engagement process early to take into account
local priorities and needs even at the proposal stage. If engagement could not begin before the
solicitation phase, USAID should maintain sufficient flexibility to allow for anticipated partners to
influence the design and implementation approach after award. Additionally, USAID should ensure
that IPs staff future programs in a way that addresses the different cultural and linguistic needs across
33 | SAEP EVALUATION REPORT USAID.GOV
the region, such as through more extensive engagement of local staff and experts. USAID can
provide this oversight through both contractual requirements, such as requirements for linguistic
skills on the team and/or for inclusion of regional experts, as well as through its ongoing oversight of
the contract during implementation, where USAID can help ensure that the IP updates its staffing to
align with any needed changes. Finally, future program design and target setting should account for
the work other donors have done in each country to avoid duplication and maximize benefits to
partner countries by involving other donors as stakeholders as well. For example, if another donor is
already supporting a country’s planned generation activities, USAID’s programs could focus on
another area of the sector that is in more need of assistance. This recommendation responds to
findings from EQ2 noting the importance of aligning programming with local priorities and to findings
from EQ3 and EQ4 related to insufficient localization under SAEP. 
2. USAID should ensure that clear criteria drive future indicator selection and target
setting. Given that the fee indicators drive program focus, USAID should ensure that the fee
indicators represent the intended focus of future programs and that the resulting interventions
clearly link to achieving those indicators. For example, if USAID’s intention is to focus on new
generation and new connections, then USAID should clearly link the structure of the program and
the anticipated interventions to achieving those outcomes during the life of the program. If the
intended focus of the program is more on the enabling environment, building capacity, and
institutional strengthening, then USAID should focus the fee indicators on these outcomes instead, to
ensure the maximum linkage between program efforts and achievements. Additionally, USAID and
the IP should set the targets using clear justifications and realistic criteria that incorporate local input,
needs, and priorities. This ensures that targets are ambitious, but realistic, given regional realities.
USAID should explore possibilities for setting targets that balance the flexibility of regional targets
while adjusting to the widely varying power markets in the region. Multiple options are possible but 
could include setting a high-level regional target while requiring a subset of country-level targets. As 
an illustrative example, setting a target of 1,000MW of new generation regionally could be 
accompanied by a requirement to increase capacity of at least 100MW or 10 percent of the
country’s total installed capacity in at least four countries. Including both a “hard” figure and a 
percentage option could help adjust for wide variances between countries: 10 percent in Eswatini
would be five to six MW and is achievable while 10 percent in South Africa would be nearly
6,000MW and would likely be unrealistic (USAID, of course, would need to set the specific
thresholds using their own specific criteria and goals). Including both thresholds mitigates this
challenge while ensuring that smaller, high-impact interventions receive attention. This
recommendation responds to findings from EQ4 regarding the lack of clear criteria for indicator
selection and target setting under SAEP and the important, driving role of the fee indicators as well
as EQ4 findings regarding the inclusion of local input and blending the flexibility of regional targets
with country-level targets.
3. USAID should ensure that future programming improve on the GESI approach. To
ensure a cohesive GESI approach, USAID should first define its GESI goals for the activity. It can then
hold IPs accountable to that vision. USAID should ensure that future program design enables
achievement of that vision. For example, if the vision involves increasing women’s participation in the
power sector, then USAID would need to anticipate interventions to support this goal, which would
look very different from interventions designed to ensure that women already in the field receive
adequate opportunities. USAID should ensure that the GESI approach goes beyond gender toward
intersectionality and social inclusion, accounting for other marginalized groups and gaps within social
equity (in line with USAID’s GESI guidance). USAID should ensure that performance indicators and
action plans adequately capture this updated approach and integrate the approach throughout the
eventual programs. Finally, USAID should integrate the GESI approach across country programs and
USAID.GOV SAEP EVALUATION REPORT | 34
leads, allowing for synergy across programming and interventions. This recommendation responds to
EQ3 findings regarding ways to improve on SAEP’s GESI approach for future programs.
4. USAID should ensure that future programming addresses the financing gap in the
power sector. Across stakeholder groups, respondents flagged a lack of financing for power
projects as a key constraint. Though some stakeholders requested that USAID provide direct
funding for infrastructure investments (which was particularly important for extending the grid in
rural areas as doing so is not commercially viable), others referred more generally to ways in which
USAID could help connect utilities, governments, and private sector firms with financiers (either
commercial banks or development banks/funds). USAID should base such programming on
thorough studies assessing the primary constraints to power sector financing in Southern Africa.
Such design may require specialized programming such as credit guarantees or other lending
derisking mechanisms to meet the needs of both credit and financiers. This recommendation
responds to findings from EQ3 regarding how financing remains a significant need in the sector and
can act as a roadblock to progress, and how addressing this need in future programming could
improve outcomes. 
35 | SAEP EVALUATION REPORT USAID.GOV
ANNEX 1: STATEMENT OF WORK
SECTION C—STATEMENT OF WORK
C.1. Purpose
USAID Power Africa Program Office requires an external performance evaluation of a USAID-funded
project to improve program performance management and efficiency. This evaluation will also help Power
Africa meet Agency requirements, make improved informed strategic and management decisions around
program implementation, and advance Agency learning.
C.2. Background of the Project to Be Evaluated
Project Name Southern Africa Energy Program (SAEP)
Implementing Organization Deloitte
Contract # AID-674-C-17-00002
Project Funding $69,995,372
Period of Performance March 2017-March 2022
Geographic Scope Southern Africa- Angola (new in FY19); Botswana;
Eswatini; Lesotho; Madagascar; Malawi;
Mozambique; Namibia; South Africa; Zambia
USAID Office Power Africa
C.3. Description of the Project and Theory of Change
SAEP’s overarching goal is to overcome the challenges of energy access and power sector development in
Southern Africa. SAEP aims to increase electricity availability and access in Southern Africa, specifically
through:
● 3,000 MW of new generation capacity;
● 1,000 MW of new transmission capacity;
● 3 million new connections
SAEP was designed to increase electricity generation and to improve access to power in 1137 countries
located throughout Southern Africa while objectively quantifying and measuring progress toward five key
outcomes or work streams of the Program. The outcomes are outlined in the Figure below, which
demonstrates SAEP’s Development Hypothesis and Results Framework.
The approach SAEP used to increase investment in electricity supply and access in Southern Africa was to
strengthen the regional enabling environment and facilitate transactions. SAEP addresses key constraints to
7 Botswana; Eswatini; Lesotho; Madagascar; Malawi; Mozambique; Namibia; South Africa; Zambia
USAID.GOV SAEP EVALUATION REPORT | 36
energy sector investment, through strengthening regulation, improving planning and procurement, improving
management of electricity trade, demonstrating and scaling energy efficiency technologies and
practices, and providing capacity building to institutions and human resources for energy sector management.
C.4. Evaluation Objective
The evaluation seeks to understand how effective the SAEP activity’s efforts were/are to increase electricity
availability and access in Southern Africa by facilitating transactions and strengthening the enabling
environment for private sector investment in power, with a focus on clean energy. Power Africa technical 
teams and Power Africa management will use the results of this evaluation to make programmatic
adjustments, inform future project redesign efforts and strategies, and support evidence-based decision
making.
C.5. Evaluation Questions
Evaluation questions will be finalized in collaboration with USAID post-award. The following evaluation
questions are illustrative:
i. To what extent was SAEP able to meet its purpose & related outputs & outcomes (MWs,
Connections, Enabling Environment)?
ii. What enabled and/or hindered achievements and how were any hindrances addressed? Provide
specific examples.
37 | SAEP EVALUATION REPORT USAID.GOV
iii. How did the program design, target setting and operational decision-making shape and
contribute to the achievements of the Activity?
iv. What were the benefits and drawbacks of Power Africa setting the connections and MW
targets for the program (which may have been too ambitious?)
v. In which countries did SAEP have the most success* vs. the least success, and why? (focus on off￾grid interventions; unlocking financing for private sector companies; improving the enabling
environment?)
vi. How is the SAEP activity perceived and valued by the stakeholders of the project?
(Implementers, including transaction advisors; utilities; regulators; host country governments;
private sector partners; Mission POCs)
*For the purpose of this evaluation, “success” is defined as having met the intended outputs, outcomes, and,
where identifiable, impacts, of the SAEP activity. Outputs, outcomes, and impacts are outlined in the figure
above.
C.6. Evaluation Design and Methodology
The following is a guideline for the proposed methodology and data analysis to be used in the performance
evaluation:
● The contractor should propose an evaluation design that employs mixed methods for data
collection. These should include key informant interviews, which may be conducted remotely via
video conferencing (e.g., Google Meet; Zoom; Microsoft Teams etc.) due to COVID restrictions or
precautions.
● A reasonable and appropriate data analysis method (discussed and agreed upon with the Power
Africa MEL Team), that will yield credible results must be proposed. With quantitative results, a 
margin of error less than 10 percent is permissible or deemed reasonable. Disaggregated data may
be required (e.g., country, type of connection, type of technology, transaction type, type of energy
source, by sex or other relevant aspects of the program).
● Data collection should include - document/desk review, key informant interviews/focus group
discussions, review of Power Africa Information System (PAIS) and Power Africa Tracing Tool
(PATT) data, quantitative tools.
● Data sources should include regular implementing mechanism reports, programmatic documents,
and relevant stakeholders’ documents.
Any known limitations in data collection should be included in the evaluation report.
USAID.GOV SAEP EVALUATION REPORT | 38
ANNEX 2: BIBLIOGRAPHY
● Electricity Generation Company Malawi Limited GESI Document Review List
● Electricity Generation Company Malawi Limited GESI Findings Validation Workshop Final
Presentation
● Patton, Michael Q. (2002). Qualitative Research and Evaluation Methods. Thousand Oaks, Calif: Sage
Publications
● Power Africa Portfolio Review Presentation, November 2016
● Power Africa Portfolio Review Presentation, August 2021
● Power Africa Portfolio Review Presentation, May 2022
● Southern African Development Community (SADC) Gender Mainstreaming Baseline Survey Report
● SADC Gender Action Plan 2020
● SAEP Activity Deliverable Tracker
● SAEP Annual Report Year 1 
● SAEP Annual Report Year 2 
● SAEP Annual Report Year 3 
● SAEP Annual Report Year 4 
● SAEP Contract, Section C 
● SAEP GESI Timelines Spreadsheet
● SAEP Monitoring & Evaluation Standard Operating Procedures
● SAEP Monitoring Indicator Actuals and Targets, Q2 2022
● SAEP Monitoring Indicator Actuals and Targets, Q3 2022
● SAEP Performance Management & Evaluation Plan (PMEP) Version 3, 2018
● SAEP PMEP Version 4, 2018
● SAEP PMEP Version 5, 2018
● SAEP PMEP Version 6, 2020
● SAEP PMEP Version 7, 2020
● SAEP PMEP Version 8, 2021
● SAEP PMEP Version 9, 2021
● SAEP PMEP Version 10, 2021
● SAEP Regional Course Catalogue, October 2017
● SAEP Request For Proposals
● SAEP Work Plan Year 1 
● SAEP Work Plan Year 3 
● SAEP Work Plan Year 4 
● SAEP Work Plan Year 5 
● SAEP Year 1 Gender Action Plan
● SAEP Year 2 Gender Action Plan Timelines
● SAEP Year 3 Gender Action Plan Timelines
● Southern Africa Gas Roadmap for SAEP
● Southern Africa Gas Roadmap to Guide USAID SAEP Gas Work
39 | SAEP EVALUATION REPORT USAID.GOV
ANNEX 3: INFORMED CONSENT STATEMENTS
QUALITATIVE CONSENT FORM
Hello, my name is ____________ and I am here on behalf of Social Impact, an evaluation company based in
the United States. We are independent evaluators working on behalf of the United States Agency for
International Development (USAID). We are evaluating the Southern Africa Energy Program (SAEP), which
is being implemented by Deloitte. We are not directly affiliated with either USAID or Deloitte, as we are an
independent evaluator. Thus, we appreciate your candid responses. In this evaluation, we are assessing the
extent to which SAEP achieved its goals, what worked well in implementation, and what the lessons learned
might be that could benefit similar, future projects. This evaluation will help USAID understand how to
improve its activities in the future.
We are asking you to participate as one of at least 100 stakeholder interviews because of your and/or your
organization’s involvement in the SAEP program. We kindly request approximately [ two hours (for USAID
and IP staff)/60 minutes (for all other stakeholders)] of your time so we can hear about your experiences
and opinions. We will ask for details about SAEP’s activities as well as your perceptions of the interventions. 
We do not anticipate any risks to participating in this interview, and there are no direct benefits, though your
participation may broadly inform improvements in future USAID power programs. Your participation in this
study is completely voluntary and you are under no obligation to participate. If you don’t know the answer or
do not wish to answer a question it is fine just tell us and we will move on. If you start the interview and wish
to stop at any time for any reason, simply let us know. 
Only a handful of SI researchers on this study will have access to your personal information. Your name,
address, contact information, and other identifiers will not be shared with anyone outside of the research
team. Your personal information will be recorded on a separate sheet of paper from the rest of the
information and will be kept separately. While all identifying information will be kept confidential, de￾identified data may be shared with USAID and/or publicly for future research purposes.
If you have any concerns, you may contact Kari Nelson at knelson@socialimpact.com or +1 952 446 5440 or
the Social Impact Institutional Review Board at irb@socialimpact.com or +1 703 465 1884 with questions
about the study or results. I will leave a copy of this form with you. 
Do you have any questions about this interview? 
Do you agree to participate? Yes/No
Do you agree to us recording this interview? Yes/No
QUANTITATIVE CONSENT FORM
Hello, this survey comes from Social Impact, an evaluation company based in the United States. We are
independent evaluators working on behalf of the United States Agency for International Development
(USAID) to evaluate the Southern Africa Energy Program (SAEP), which is being implemented by Deloitte.
We are not directly affiliated with either USAID or Deloitte, as we are an independent evaluator. Thus, we
appreciate your candid responses. In this evaluation, we are assessing the extent to which SAEP achieved its
goals, what worked well in implementation, and what the lessons learned might be that could benefit similar,
future projects. This evaluation will help USAID understand how to improve its activities in the future.
USAID.GOV SAEP EVALUATION REPORT | 40
We are sharing this survey with several hundred key stakeholders involved in the SAEP program. We kindly
request 10-15 minutes of your time to complete this survey and for us to better understand your
perspective of the SAEP program. 
We do not anticipate any risks to participating in this survey, and there are no direct benefits, though your
participation may broadly inform improvements in future USAID power programs. Your participation in this
study is completely voluntary and you are under no obligation to participate. If you don’t know the answer or
do not wish to answer a question, you can skip that question. You can also choose to stop the survey at any
time without consequence. 
Only a handful of SI researchers on this study will have access to your personal information. The survey will
not ask for your name or other direct identifiers, however, we will ask for indirect information such as your
role related to SAEP, your gender, etc. After data analysis, the collected data will be deidentified (information
potentially linking you to your responses will be modified and/or removed) and will be shared publicly on
USAID’s Development Data Library for use by other researchers.
If you have any concerns, you may contact Kari Nelson at knelson@socialimpact.com or +1 952 446 5440 or
the Social Impact Institutional Review Board at irb@socialimpact.com or +1 703 465 1884 with questions
about the study or results. I will leave a copy of this form with you. 
Do you agree to participate? Yes/No
41 | SAEP EVALUATION REPORT USAID.GOV
ANNEX 4: DATA COLLECTION INSTRUMENTS
KEY INFORMANT INTERVIEW GUIDES
USAID/EMBASSY STAFF (KIIS)
Introduction:
1. What is your role at USAID/within the US government?
2. How have you been involved in the design or implementation of SAEP and/or other Power Africa initiatives?
EQ1:
3. How would you describe “success” for SAEP? 
a. What would “success” look like?
b. How, if at all, does your definition of success differ from the official definition (as set by SAEP’s 
program logic and performance indicators)?
4. From your perspective, to what extent has SAEP achieved your definition of success?
a. Are there certain outcomes where SAEP has been more successful than others?
b. To what extent does your perspective on SAEP success match with the extent to which SAEP has
met its performance indicators (which is potentially a different definition of “success”)?
5. From your perspective, are there any major intended outcomes that SAEP has been unable to achieve? If so,
which ones and why haven’t they been achieved?
6. Have there been any unintended outcomes or outcomes that were not expected? If so, what are they?
EQ5: 
7. To what extent are you familiar with SAEP’s work across multiple countries?
8. If familiar with SAEP’s work in multiple countries: In which countries has SAEP had the most success? Why? 
a. In what ways was SAEP more successful?
9. If familiar with SAEP’s work in multiple countries: In which countries has SAEP had the least success? Why?
a. In what ways was SAEP less successful?
EQ2:
10. What factors have helped facilitate SAEP achievements? How so and why?
a. Types of factors to consider: country context, willingness/cooperativeness of partners,
implementation approach, SAEP management approach, etc.
11. What factors have hindered SAEP achievements? How so and why?
USAID.GOV SAEP EVALUATION REPORT | 42
a. Types of factors to consider: country context, willingness/cooperativeness of partners,
implementation approach, SAEP management approach, COVID-19, etc.
12. How were any hindrances addressed?
13. How, if at all, has COVID-19 affected SAEP implementation and/or success?
14. Some SAEP partnerships have been more successful than others. Which have been most successful and what
factors do you think are driving these differences?
EQ3:
15. From your perspective, what were the strengths and weaknesses of the SAEP program design?
16. How has the original program design affected SAEP’s ability to achieve its intended outcomes?
a. What aspects of design were most impactful? Why?
17. How effective has operational decision-making been under SAEP?
a. Has it differed across entities (Deloitte, USAID, specific sub-teams, etc.)? If so, how?
b. How, if at all, has it impacted the achievement of expected outcomes?
18. We understand that SAEP took an iterative approach to interventions, with frequent updates to the work
plan. From your perspective, how effective was this approach for improving outcomes and why?
19. What approaches has SAEP used for gender and social inclusion in its activities?
20. What worked well for supporting gender and social inclusion?
21. What challenges were encountered in trying to support gender and social inclusion?
22. What, if any, adjustments would you suggest for the program design or approach moving forward?
EQ4:
23. Are you familiar with how the targets for key monitoring indicators were set for SAEP? If so, how were they
set and by whom?
a. If familiar with target setting: What worked well in regard to how key monitoring targets were set?
b. If familiar with target setting: What challenges were encountered either in the target setting process
or because of it?
24. How realistic do you think the key performance targets of MW of new generation and transmission and
number of new connections were? Why or why not?
25. In a future activity like SAEP, what, if any, changes would you propose to how key indicators were selected
and/or how the targets were set?
EQ6:
26. As a whole, how successful has SAEP been in improving the power sector?
43 | SAEP EVALUATION REPORT USAID.GOV
27. If you were to design a future activity like SAEP, what would you recommend focusing on in terms of
remaining sectoral challenges?
Closing: 
28. Is there anything additional you’d like to share about SAEP?
USAID.GOV SAEP EVALUATION REPORT | 44
IP STAFF AND CONSULTANTS (KIIS)
Introduction
1. What is your role within the SAEP program?
2. How have you been involved in the design or implementation of SAEP?
EQ1:
3. How would you describe “success” for SAEP? 
a. What would “success” look like?
b. How, if at all, does your definition of success differ from the official definition (as set by SAEP’s 
program logic and performance indicators)?
4. From your perspective, to what extent has SAEP achieved your definition of success?
a. Are there certain outcomes where SAEP has been more successful than others?
b. To what extent does your perspective on SAEP success match with the extent to which SAEP has
met its performance indicators (which is potentially a different definition of “success”)?
5. From your perspective, are there any major intended outcomes that SAEP has been unable to achieve? If so,
which ones and why haven’t they been achieved?
6. Have there been any unintended outcomes or outcomes that were not expected? If so, what are they?
7. Which specific types of interventions under SAEP have you been involved in?
a. Potential examples: transaction advising, capacity development, utility support, engineering studies,
tariff studies, policy reform, power regulation, etc.
8. Regarding the specific interventions that you have been involved in, what are the biggest successes achieved
and what factors drove those successes?
9. What have been the biggest challenges in your specific activities?
10. If you were to engage in a future activity like SAEP, what changes would you propose for the new activity?
EQ5:
11. What contextual or country-related factors have varied the most between countries? How have these factors
impacted relative success between countries? 
12. In which countries has SAEP had the most success? Why? 
a. In what ways was SAEP more successful?
13. In which countries has SAEP had the least success? Why?
a. In what ways was SAEP less successful?
45 | SAEP EVALUATION REPORT USAID.GOV
EQ2:
14. What factors have helped facilitate SAEP achievements? How so and why?
a. Types of factors to consider: country context, willingness/cooperativeness of partners,
implementation approach, SAEP management approach, etc.
15. What factors have hindered SAEP achievements? How so and why?
a. Types of factors to consider: country context, willingness/cooperativeness of partners,
implementation approach, SAEP management approach, COVID-19, etc.
16. How were any hindrances addressed?
17. How, if at all, has COVID-19 affected SAEP implementation and/or success?
18. Some SAEP partnerships have been more successful than others. Which have been most successful and what
factors do you think are driving these differences? 
EQ3:
19. From your perspective, what were the strengths and weaknesses of the SAEP program design?
20. How has the original program design affected SAEP’s ability to achieve its intended outcomes?
a. What aspects of design were most impactful? Why?
21. How effective has operational decision-making been under SAEP? 
b. Has it differed across entities (Deloitte, USAID, specific sub-teams, etc.)? If so, how?
c. How, if at all, has it impacted the achievement of expected outcomes?
22. We understand that SAEP took an iterative approach to interventions, with frequent updates to the work
plan. From your perspective, how effective was this approach for improving outcomes and why?
23. What approaches has SAEP used for gender and social inclusion in its activities? 
24. What worked well for supporting gender and social inclusion? 
25. What challenges were encountered in trying to support gender and social inclusion?
26. What, if any, adjustments would you suggest for the program design or approach moving forward?
EQ4: 
27. Are you familiar with how the targets for key monitoring indicators were set for SAEP? If so, how were they
set and by whom?
a. If familiar with target setting: What worked well in regard to how key monitoring targets were set?
b. If familiar with target setting: What challenges were encountered either in the target setting process
or because of it?
USAID.GOV SAEP EVALUATION REPORT | 46
28. How realistic do you think the key performance targets of MW of new generation and transmission and
number of new connections were? Why?
29. In a future activity like SAEP, what, if any, changes would you propose to how key indicators were selected
and/or how the targets were set?
EQ6:
30. As a whole, how successful has SAEP been in improving the power sector?
31. If you were to design a future activity like SAEP, what would you recommend focusing on in terms of
remaining sectoral challenges?
Conclusion: 
32. Is there anything additional you’d like to share about SAEP?
47 | SAEP EVALUATION REPORT USAID.GOV
UTILITY AND REGULATOR REPRESENTATIVES (KIIS)
Introduction
1. What is your role within your organization?
2. How have you been involved in the SAEP program?
EQ1:
3. How would you describe “success” for SAEP? 
a. What would “success” look like?
4. From your perspective, to what extent has SAEP achieved your definition of success?
a. Are there certain outcomes where SAEP has been more successful than others?
5. From your perspective, are there any major intended outcomes that SAEP has been unable to achieve? If so,
which ones and why haven’t they been achieved?
6. Have there been any unintended outcomes or outcomes that were not expected? If so, what are they?
7. How, specifically, has SAEP supported your [utility/organization]?
a. Potential examples: transaction advising, capacity development, reducing losses, engineering studies,
tariff studies, policy reform, customer service improvements, etc.
8. What, if anything, did SAEP’s support help your [utility/organization] achieve that it may not have been able to 
achieve without SAEP’s support?
a. How did SAEP’s support help achieve this outcome?
9. Is there anything you had hoped your [utility/organization] could achieve with SAEP support that you were not
able to achieve? If so, what wasn’t achieved and why?
EQ2:
10. What went well about working with SAEP?
11. What have been the biggest challenges you’ve encountered in working with SAEP?
a. Did SAEP take any specific actions to address the challenges that were encountered? If so, what did
they do, and was it effective?
12. How, if at all, has COVID-19 affected SAEP implementation and/or success?
13. If you could change anything about the type of support or the manner in which SAEP supports
[utilities/organizations] like yours, what would it be? Why?
14. What changes would you suggest for a future activity like SAEP that might facilitate engagement with your
[utility/organization]?
USAID.GOV SAEP EVALUATION REPORT | 48
EQ6:
15. As a whole, how successful has SAEP been in improving the power sector?
16. If you were to design a future activity like SAEP, what would you recommend focusing on in terms of
remaining sectoral challenges?
Conclusion: 
17. Is there anything additional you’d like to share about SAEP?
49 | SAEP EVALUATION REPORT USAID.GOV
PARTNER GOVERNMENT REPRESENTATIVES (KIIS)
Introduction
1. What is your role within your organization?
2. How have you been involved in the SAEP program?
EQ1:
3. How would you describe “success” for SAEP? 
a. What would “success” look like?
4. From your perspective, to what extent has SAEP achieved your definition of success?
a. Are there certain outcomes where SAEP has been more successful than others?
5. From your perspective, are there any major intended outcomes that SAEP has been unable to achieve? If so,
which ones and why haven’t they been achieved?
6. Have there been any unintended outcomes or outcomes that were not expected? If so, what are they?
7. How, specifically, has SAEP supported the power sector in your country and your [office/ministry], specifically?
a. Potential examples: increasing generation, improving transmission, transaction advising, capacity
development, reducing losses, engineering studies, tariff studies, policy reform, customer service
improvements, etc.
8. What, if anything, did SAEP’s support help you/your [office/ministry] achieve that it may not have been able to 
achieve without SAEP’s support?
a. How did SAEP’s support help achieve this outcome?
9. Is there anything you had hoped you/your [office/ministry] could achieve with SAEP support that you were not
able to achieve? If so, what wasn’t achieved and why?
10. What, if any, challenges remain to be addressed in the power sector in your country?
11. How might SAEP (or another future program) support you in addressing these challenges?
EQ2:
12. What worked well about working with SAEP?
13. What have been the biggest challenges in working with SAEP?
a. Did SAEP take any specific actions to address the challenges that were encountered? If so, what did
they do, and was it effective?
14. How, if at all, has COVID-19 affected SAEP implementation and/or success?
15. If you could change anything about the type or manner of SAEP support, what would it be? Why? (What type
or manner of support would you have wanted SAEP to provide that was not provided?)
USAID.GOV SAEP EVALUATION REPORT | 50
16. What changes would you suggest for a future activity like SAEP that might facilitate engagement with your
[utility/organization]?
EQ6:
17. As a whole, how successful has SAEP been in improving the power sector?
18. If you were to design a future activity like SAEP, what would you recommend focusing on in terms of
remaining sectoral challenges?
Conclusion: 
19. Is there anything additional you’d like to share about SAEP?
51 | SAEP EVALUATION REPORT USAID.GOV
REGIONAL POWER ENTITIES (SOUTHERN AFRICAN POWER POOL, SADC, ETC) (KIIS)
Introduction
1. What is your role within your organization?
2. How have you been involved in the SAEP program?
EQ1:
3. How would you describe “success” for SAEP? 
b. What would “success” look like?
4. From your perspective, to what extent has SAEP achieved your definition of success?
b. Are there certain outcomes where SAEP has been more successful than others?
5. From your perspective, are there any major intended outcomes that SAEP has been unable to achieve? If so,
which ones and why haven’t they been achieved?
6. Have there been any unintended outcomes or outcomes that were not expected? If so, what are they?
7. How, specifically, has SAEP supported your organization’s work?
b. Potential examples: increasing generation, improving transmission, transaction advising, capacity
development, reducing losses, engineering studies, tariff studies, policy reform, customer service
improvements, etc.
8. What, if anything, did SAEP’s support help you/your organization achieve that it may not have been able to 
achieve without SAEP’s support?
b. How did SAEP’s support help achieve this outcome?
9. Is there anything you had hoped you/your organization could achieve with SAEP support that you were not
able to achieve? If so, what wasn’t achieved and why?
10. What, if any, challenges remain to be addressed in the power sector in the region?
11. How might SAEP (or another future program) support you in addressing these challenges?
EQ5:
12. How familiar are you with SAEP’s specific power sector activities in its ten partner countries?
13. If familiar with country-level work: In which countries has SAEP had the most success? Why? 
a. In what ways was SAEP more successful?
14. If familiar with country-level work: In which countries has SAEP had the least success? Why?
b. In what ways was SAEP less successful?
USAID.GOV SAEP EVALUATION REPORT | 52
EQ2:
15. What worked well about working with SAEP?
16. What have been the biggest challenges you’ve encountered working with SAEP?
a. Did SAEP take any specific actions to address the challenges that were encountered? If so, what did
they do, and was it effective?
17. How, if at all, has COVID-19 affected SAEP implementation and/or success?
18. If you could change anything about the type or manner of SAEP support, what would it be? Why?
19. What changes would you suggest for a future activity like SAEP that might facilitate engagement with your
[utility/organization]?
EQ6:
20. As a whole, how successful has SAEP been in improving the power sector?
21. If you were to design a future activity like SAEP, what would you recommend focusing on in terms of
remaining sectoral challenges?
Conclusion: 
22. Is there anything additional you’d like to share about SAEP?
53 | SAEP EVALUATION REPORT USAID.GOV
PRIVATE SECTOR ACTORS (KIIS)
Introduction
1. What is your role within your organization?
2. How have you been involved in the SAEP program?
EQ1:
3. How would you describe “success” for SAEP? 
a. What would “success” look like?
4. From your perspective, to what extent has SAEP achieved your definition of success?
a. Are there certain outcomes where SAEP has been more successful than others?
5. From your perspective, are there any major intended outcomes that SAEP has been unable to achieve? If so,
which ones and why haven’t they been achieved?
6. Have there been any unintended outcomes or outcomes that were not expected? If so, what are they?
7. How, specifically, has SAEP supported your company’s work?
a. Potential examples: capacity development, market studies, customer service improvements, etc.
8. What, if anything, did SAEP’s support help your company achieve that it may not have been able to achieve
without SAEP’s support?
a. How did SAEP’s support help achieve this outcome?
9. Is there anything you had hoped your company could achieve with SAEP support that you were not able to 
achieve? If so, what wasn’t achieved and why?
10. Is there anything further that SAEP could help your company (or companies like yours) to achieve? If so, what?
EQ2:
11. What worked well about working with SAEP?
12. What have been the biggest challenges you’ve encountered working with SAEP?
13. How, if at all, has COVID-19 affected SAEP implementation and/or success?
14. Did SAEP take any specific actions to address the challenges that were encountered? If so, what did they do,
and was it effective?
a. If you could change anything about the type or manner of SAEP support, what would it be? Why?
15. What changes would you suggest for a future activity like SAEP that might facilitate engagement with your
[utility/organization]?
USAID.GOV SAEP EVALUATION REPORT | 54
EQ6:
16. As a whole, how successful has SAEP been in improving the power sector?
17. If you were to design a future activity like SAEP, what would you recommend focusing on in terms of
remaining sectoral challenges?
Conclusion: 
18. Is there anything additional you’d like to share about SAEP?
55 | SAEP EVALUATION REPORT USAID.GOV
OTHER DONORS (KIIS)
Introduction
1. What is your role within your organization?
2. How have you been involved in the SAEP program?
3. How has your organization engaged with SAEP?
EQ1:
4. How would you describe “success” for SAEP? 
a. What would “success” look like?
5. From your perspective, to what extent has SAEP achieved your definition of success?
a. Are there certain outcomes where SAEP has been more successful than others?
6. From your perspective, are there any major intended outcomes that SAEP has not been able to achieve? If so,
which ones and why haven’t they been achieved?
7. Have there been any unintended outcomes or outcomes that were not expected? If so, what are they?
8. How, specifically, have you partnered with SAEP and/or has SAEP supported your organization’s work?
a. Potential examples: increasing generation, improving transmission, transaction advising, capacity
development, reducing losses, engineering studies, tariff studies, policy reform, customer service
improvements, etc.
9. What, if anything, did this partnership and/or support help you/your organization achieve that it may not have
been able to achieve without SAEP’s support?
a. How did SAEP’s support help achieve this outcome?
10. Is there anything you had hoped you/your organization could achieve in collaboration with SAEP that you were
not able to achieve? If so, what wasn’t achieved and why?
11. What, if any, challenges remain to be addressed in the power sector in the country/region where you operate?
12. How might SAEP (or another future program) support you in addressing these challenges?
EQ2:
13. What worked well about your work with SAEP?
14. What have been the biggest challenges you’ve encountered in working with SAEP?
a. Did SAEP take any specific actions to address the challenges that were encountered? If so, what did
they do, and was it effective?
15. How, if at all, has COVID-19 affected SAEP implementation and/or success?
USAID.GOV SAEP EVALUATION REPORT | 56
16. If you could change anything about SAEP, what would it be? Why?
17. What changes would you suggest for a future activity like SAEP that might facilitate engagement with your
[utility/organization]?
EQ6:
18. As a whole, how successful has SAEP been in improving the power sector?
19. If you were to design a future activity like SAEP, what would you recommend focusing on in terms of
remaining sectoral challenges in the region?
Conclusion: 
20. Is there anything additional you’d like to share about SAEP?
57 | SAEP EVALUATION REPORT USAID.GOV
ALL STAKEHOLDER GROUPS (SURVEY)
Consent
1. [insert consent language] Do you agree to participate in this survey?
a. Yes
b. No [skip to end] 
Demographics
2. What type of organization do you belong to?
a. USAID or Embassy Staff
b. SAEP Staff (and/or SAEP subcontractors/consultants)
c. A Utility
d. A Power Regulator 
e. A SAEP Partner Country Government Agency/Ministry
f. A Regional Organization (e.g. Southern African Power Pool, SADC, etc.)
g. A Private Sector Company
h. Another Donor
i. Other
j. Do Not Wish to Answer
3. What country do you work in (select “regional” if you work across multiple countries)?
a. Angola
b. Botswana
c. Eswatini
d. Lesotho
e. Madagascar
f. Malawi
g. Mozambique
h. Namibia
i. South Africa
j. Zambia
k. Regional
l. Do Not Wish to Answer
4. What is your gender?
a. Male
b. Female
c. Other
d. Do Not Wish to Answer
SAEP Engagement
5. To what extent did you/your organization work with the SAEP program?
a. Extensively (for example, working together on multiple interventions, working together over multiple
years, etc.)
b. Somewhat (for example, working together on a few interventions, working together sporadically but
not continuously, etc.)
c. Only a Little (for example, only working together on one or two interventions, working together for
only a short period of time, etc.)
d. Do Not Know
e. Do Not Wish to Answer
6. What types of activities did you and/or your organization engage with SAEP on? (Check all that apply)
a. Generation Transactions or Procurements
b. Transmission Transactions or Procurements
c. Financing of Power Projects
d. New On-grid Connections (power access)
e. Mini-grid Systems
USAID.GOV SAEP EVALUATION REPORT | 58
f. Household or Small Solar/Wind Systems
g. Energy Efficiency Initiatives
h. Power Policy Reform
i. Power Regulatory Reform
j. Capacity Development/Training
k. Utility Commercial Improvements (reducing losses, improving customer service, process
improvements, etc.)
l. Technical and/or Engineering Studies
m. Sectoral Collaboration/Coordination Efforts (Working Groups, Task Forces, etc.)
n. Other, please specify: ______________
o. Do Not Know
p. Do Not Wish to Answer
7. For the interventions on which you partnered with SAEP, what were the intended outcomes (in the short- or
long-term)? [select all that apply] 
a. Increased Generation Capacity
b. Increased Transmission Capacity
c. Add New Power Connections (on- or off-grid)
d. Other, please specify: ___________
e. Do Not Know
f. Do Not Wish to Answer
8. Please indicate the extent to which SAEP has helped improve each of the listed outcomes in your country or
region:
Outcome Barely or 
Not at All Somewhat Significantly Do Not
Know 
Do Not
Wish to
Answer
Power Sector (Overall)
Generation Capacity
Transmission Capacity
Power Access
Energy Efficiency
Power Sector Policy
Power Sector Regulation
Power Sector Coordination
Private Sector Investment in
Power
Greenhouse Gas Emissions
9. How satisfied are you, overall, with the SAEP program?
a. Very Satisfied
b. Satisfied
c. Neither Satisfied nor Dissatisfied
d. Dissatisfied
e. Very Dissatisfied
f. Do Not Wish to Answer
10. What worked well in SAEP implementation? [open text box] 
11. What do you think the major challenges were in SAEP implementation? [open text box] 
59 | USAID SAEP EVALUATION DESIGN REPORT USAID.GOV
ANNEX 5: CONFLICT OF INTEREST DISCLOSURES
USAID.GOV USAID SAEP EVALUATION DESIGN REPORT | 60
61 | USAID SAEP EVALUATION DESIGN REPORT USAID.GOV
USAID.GOV USAID SAEP EVALUATION DESIGN REPORT | 62
63 | USAID SAEP EVALUATION DESIGN REPORT USAID.GOV
USAID.GOV USAID SAEP EVALUATION DESIGN REPORT | 64
65 | USAID SAEP EVALUATION DESIGN REPORT USAID.GOV
USAID.GOV USAID SAEP EVALUATION DESIGN REPORT | 66
ANNEX 6: EVALUATION DESIGN REPORT
SOUTHERN AFRICA ENERGY PROGRAM 
EVALUATION 
EVALUATION DESIGN REPORT
67 | USAID SAEP EVALUATION DESIGN REPORT USAID.GOV
JUNE 10, 2022
CONTENTS
Acronyms ...................................................................................................................................................... 68
Introduction And Purpose .......................................................................................................................... 69
Context .....................................................................................................................................................................................................................................69
Program Description..........................................................................................................................................................................................................69
Theory of Change................................................................................................................................................................................................................69
Evaluation Purpose..............................................................................................................................................................................................................69
Design and Methodology............................................................................................................................. 70
Evaluation Questions..........................................................................................................................................................................................................70
Methodology...........................................................................................................................................................................................................................70
Data Analysis...........................................................................................................................................................................................................................75
Quality Assurance................................................................................................................................................................................................................75
Ethical Considerations........................................................................................................................................................................................................75
Gender and Social Inclusion Plan..................................................................................................................................................................................76
Limitations................................................................................................................................................................................................................................76
Dissemination and Utilization .................................................................................................................... 77
Annex 1: Timeline........................................................................................................................................ 78
Annex 2: Bibliography .................................................................................................................................. 79
Annex 3: Informed Consents ...................................................................................................................... 80
Annex 4: Data Collection Instruments...................................................................................................... 80
Annex 5: Scope of Work ............................................................................................................................. 80
Annex 6: Conflict of Interest Disclosures.................................................................................................. 80
USAID.GOV USAID SAEP EVALUATION DESIGN REPORT | 68
ACRONYMS
COVID-19 Novel Coronavirus of 2019
ET Evaluation Team
EQ Evaluation Questions
GESI Gender, Equality, and Social Inclusion
IP Implementing Partner
IRB Institutional Review Board
KII Key Informant Interview
MW Megawatt
PAIS Power Africa Information System
PATT Power Africa Tracing Tool
PMEP Performance Management & Evaluation Plan
RERA Regional Energy Regulators Association
SADC Southern Africa Development Community 
SAEP Southern Africa Energy Program
SI Social Impact
USAID United States Agency for International Development
USG United States Government
69 | USAID SAEP EVALUATION DESIGN REPORT USAID.GOV
INTRODUCTION AND PURPOSE
CONTEXT
Power Africa, a U.S. government-led public-private partnership that launched in 2013, was created to
connect 60 million new homes and businesses in sub-Saharan Africa to power by 2030. The Southern
African Energy Program (SAEP), a flagship initiative of this partnership, aims to increase investment in
electricity supply and access in Southern Africa by strengthening the regional enabling environment and
facilitating power transactions. 
PROGRAM DESCRIPTION
A lack of adequate and reliable power in the Southern Africa region limits competitiveness and affects the
livelihoods and health of the population, particularly those living outside urban areas. The five-year SAEP
program, implemented by Deloitte, launched in 2017 with activities in ten countries: Angola, Botswana,
Eswatini, Lesotho, Madagascar, Malawi, Mozambique, Namibia, South Africa, and Zambia. SAEP received an
extension to its work, making it a six-year program that is expected to end in June 2023.
Though the goals of SAEP are to increase generation and access to electricity, SAEP does not directly fund
major infrastructure investments. Rather, SAEP works to incentivize, facilitate, and enable investments
through technical assistance, policy reform, capacity development, and transaction facilitation.
To identify interventions during the annual work planning process, the SAEP team met with key stakeholders
in each partner country to discuss potential interventions and ways in which SAEP could support their
electrification, generation, and other sectoral improvement goals. SAEP took a flexible and adaptable
approach to implementation, which involved iterative changes and adaptations to the work plan. 
THEORY OF CHANGE
SAEP’s stated theory of change says that by strengthening regulation, improving planning and procurement,
management of electricity trade, demonstrating and scaling energy efficiency technologies and practices, and
providing capacity building to institutions and human resources for energy sector management, SAEP aims to
increase investment in electricity supply and access in Southern Africa. In countries like Eswatini and Lesotho,
SAEP’s support works to address reliance on electricity/energy imports. Long-term program targets include
4,000 megawatts (MW) of new power generation and transmission capacity and three million new
connections (on- and off-grid). 
EVALUATION PURPOSE
The purpose of this evaluation is to provide USAID (United States Agency for International
Development)/Southern African and the USAID Power Africa team with feedback and recommendations
for improving on the SAEP activity and to inform future programming in the energy sector. This evaluation
will provide USAID teams with evidence to support programmatic adjustments, inform future design efforts
and strategies, and make informed decisions. Other potential users of the evaluation include implementers,
regional organizations, national ministries, private sector partners, and regulators, who can adapt based on the
lessons learned from SAEP.
Social Impact (SI) will conduct a utilization-focused, mixed methods evaluation with an emphasis on broad
stakeholder engagement to promote use of the evaluation and evidence-based decision making. Our
USAID.GOV USAID SAEP EVALUATION DESIGN REPORT | 70
technical design for the SAEP evaluation keeps quality, utilization, and feasibility firmly in focus while ensuring
broad coverage of all ten SAEP countries. 
DESIGN AND METHODOLOGY
EVALUATION QUESTIONS
The Evaluation Team (ET) worked closely with USAID to refine and finalize the Evaluation Questions (EQs)
to ensure mutual understanding and maximize utility and learning from the evaluation:
1. To what extent was SAEP able to meet its purpose, related outputs and outcomes (MWs, Connections,
Enabling Environment)? 
2. What enabled and/or hindered achievements and how were any hindrances addressed?
3. How did the program design, target setting and operational decision-making shape and contribute to the
achievements of the program? 
4. What were the benefits and drawbacks of Power Africa by setting the connections and MW targets for the
program?
5. In which countries did SAEP have the most success versus the least success, and why? 
6. How is the SAEP perceived and valued by the stakeholders of the program? 
METHODOLOGY
The SAEP evaluation will rely on multiple methods to obtain valuable data across the ten active SAEP 
countries. On the qualitative side, the ET will conduct Key Informant Interviews (KIIs) and a desk review of
relevant documents and literature. On the quantitative side, the ET will conduct a short, online survey for key
stakeholders and a review of the Power Africa Information System (PAIS), Power Africa Tracing Tool
(PATT), as well as activity monitoring data. 
SI will conduct remote data collection to reduce potential risks related to the novel coronavirus of 2019
(COVID-19). The survey will aim to ensure that stakeholders in all ten countries can be included in data
collection and have their perspectives heard. Under the COVID-19 pandemic, SI established measures and
processes to effectively conduct evaluations remotely and ensure each member of the ET has the ability and
familiarity with remote platforms to collect remote data effectively and efficiently. 
The following sections describe SI’s mixed methods approach to the evaluation, which includes document
and desk review, secondary data, short quantitative surveys, and KIIs, as outlined in Table 5, below.
Table 5: Evaluation Design Matrix 
Evaluation Question Anticipated Methods
Document/D
esk Review
Secondary
Data
Quantitative 
Survey KIIs
Question 1: To what extent was SAEP able to 
meet its purpose and related outputs and
outcomes (MWs, Connections, Enabling
Environment)?
X X X X 
71 | USAID SAEP EVALUATION DESIGN REPORT USAID.GOV
Evaluation Question Anticipated Methods
Document/D
esk Review
Secondary
Data
Quantitative 
Survey KIIs
Question 2: What enabled and/or hindered
achievements and how were any hindrances
addressed?
X X X 
Question 3: How did the program design, target
setting and operational decision-making shape and
contribute to the achievements of the Program?
X X X X 
Question 4: What were the benefits and
drawbacks of Power Africa by setting the
connections and MW targets for the program?
X X 
Question 5: In which countries did SAEP have the
most success versus the least success, and why?
X X X X 
Question 6: How is the SAEP perceived and valued
by the stakeholders of the Program?
X X 
Document/Desk Review. The ET will conduct a systematic document/desk review to understand the
availability and quality of secondary data in each of the ten SAEP countries, how the data relates to the
evaluation questions and to identify key issues and constraints to examine further in primary data collection.
The desk review will include SAEP background documents including quarterly and annual reports,
programmatic documents, and relevant stakeholder documentation. 
Secondary Data. The ET will review PAIS, PATT, and SAEP monitoring data. Where useful, the ET will
request other secondary data such as utility data. The ET will use data from the PAIS to understand SAEP
partners, commitments made (and progress made toward them), private sector engagement, and a broad
range of other work completed through Power Africa. Data from the PATT will provide insights on power
transactions, the energy mix, and generation capacity. As in the document/desk review, the data will be 
examined for trends over time and to better understand progress achieved and where challenges may
remain. These databases will be particularly useful in answering the first EQ on SAEP’s ability to meet its
purpose and objectives. The SI team will pay special attention to variations across countries, types of
connections, types of technology, transaction types, types of energy sources, gender, and other applicable
points of data disaggregation.
Quantitative Survey. The SI team will design and implement a short, online quantitative survey to assess
achievements, benefits, successes, and challenges during implementation. Given the short period of
performance for this evaluation, a short survey will enable rapid collection of key perspectives from a broad
number of stakeholders in the ten active SAEP countries that may not otherwise all be reached through the
proposed interviews. The survey tool will contain questions to better understand perceptions of SAEP
implementation, achievements reached, and challenges faced, with a particular focus on EQs one, two, five,
and six. These data will complement the secondary data and document/desk review and will be triangulated
with the more in-depth data collected from stakeholders through the interviews. The final survey will be 
translated from English into Portuguese and French prior to distribution to ensure all stakeholders can
participate.
SI intends to electronically administer the survey to as broad a range of stakeholders as possible, including
implementing partner (IP) staff, regional bodies (Southern Africa Power Pool [SAPP], Regional Energy
USAID.GOV USAID SAEP EVALUATION DESIGN REPORT | 72
Regulators Association [RERA], Southern Africa Development Community [SADC] Centre for Renewable
Energy & Energy Efficiency), regulators and utilities engaged with SAEP, private sector partners, government
partners, and representatives from other donor agencies. 
Online surveys often receive low response rates. To increase participation in the survey and mitigate this
concern, the ET will seek the IP’s support in distributing the survey to all relevant stakeholders, with a note
encouraging them to participate, share their feedback, and help inform future programming decisions. The
short survey length, which will be translated into French and Portuguese, will help encourage participation.
Additionally, based on input from the IP, we anticipate that the majority of stakeholders should have
adequate internet access to open and respond to a short survey such as this. Applying this approach allows
for a broad reach and appropriate representation. 
Key Informant Interviews. The SI team will hold KIIs with a sample of stakeholders to provide context
and document stakeholder experiences related to SAEP’s interventions. Employing a semi-structured
interview approach, qualitative data collection will include questions to examine stakeholder experiences with
program implementation and perceived program effects, including respective contribution of different
activities to USAID’s results. The ET will tailor interview protocols to the different stakeholder types to fit
their roles. KIIs will last approximately 60-90 minutes with most stakeholders; however KIIs with USAID and
IP staff are expected to be a bit longer (1.5-2 hours), given that these key stakeholders hold significant
insights for all EQs and thus will take longer to administer. The tools will be tested with initial interviewees to
ensure all questions can be answered within the estimated time frame. The ET will work with professional
interpretation and translation services as needed. This data will allow the SI team to identify lessons and best
practices that could be used to improve program delivery and effectiveness across the region. 
Targeted stakeholder groups for KIIs include USAID and embassy staff, IP staff, utilities, regulators,
government representatives, representatives of other donors, and private sector actors. 
Because SAEP’s work varied significantly from one country to the next, the first round of KII sampling will
begin at the country level. The overall distribution of KII respondents will be weighted based on the relative
level of activity in each country (those with more activity will have more interviews). 
The below chart summarizes how feedback from each respondent group will be used for answering each of
the EQs:
Table 6: Relevant EQs for Each Respondent Group
Respondent Group EQ1 EQ2 EQ3 EQ4 EQ5 EQ6
USAID/USG Staff X X X X X X
IP Staff X X X X X X
Utility Representatives X X X
Regulator Representatives X X X
Partner Government Representatives X X X
Regional Entities (SAPP, SADC, RERA, etc.) X X X X
73 | USAID SAEP EVALUATION DESIGN REPORT USAID.GOV
Respondent Group EQ1 EQ2 EQ3 EQ4 EQ5 EQ6
Private Sector Actors X X X
Other Donors X X X
At the second stage of sampling, since SAEP engaged in different interventions in each country, the ET will
then purposively select KII respondents from each relevant stakeholder group within each country. Thus, for
example, if SAEP did not engage with private sector companies in a given country, the ET’s sample would
exclude this group and rather focus on stakeholders related to the interventions SAEP did engage in.
Qualitative research uses purposive sampling for the identification of information-rich cases and is an
effective use of limited resources.8 Stakeholders at the regional level will be sampled in this manner. The ET
anticipates the below respondent sample sizes, distributed across the ten active SAEP countries and
stakeholder types, as outlined below:
Table 7: Anticipated Total Sample Sizes for the Evaluation
Stakeholder Group Total Anticipated Number of Respondents
USAID and Mission Staff 10-12
IP Staff 20-30
Utilities 20-30
Regulators 20-30
Government Stakeholders 30-40
Regional Entities 6-10
Other Donors 10-15
Private Sector Actors 20-30
Total 136-192
Below, we outline each of the anticipated stakeholder groups, the discussion topics, and the team’s approach
to sampling respondents within each category.
USAID and Embassy Staff. USAID, the Power Africa Program Office, and other Mission staff who have
been directly involved in the implementation of SAEP are important stakeholders, as these individuals are
heavily involved in designing, implementing, and monitoring the activity. Interviews with these stakeholders
will cover all six of the evaluation questions. The ET will target those staff members who have been most 
heavily involved in the activity and related negotiations with government and other national stakeholders.
IP Staff. IP staff have detailed knowledge and understanding of how the activity has been implemented as
well as the successes and challenges to date. Thus, this group of stakeholders will have valuable information
for answering all six of the evaluation questions. In targeting IP staff, the ET will focus on the staff who have
been most heavily involved in implementation while ensuring that the ET speaks to staff members from
across discipline areas and project components. For instance, the ET would interview those involved in each
of the partner countries, those at both the managerial as well as technical levels, and those covering cross￾cutting specialties like monitoring and evaluation (M&E) and gender, equality, and social inclusion (GESI). This
category will include staff from any supporting sub-contractors and/or consultants who helped implement
specific interventions. It will include SAEP transaction advisors.
8 Patton, Michael Q. (2002). Qualitative Research and Evaluation Methods. Thousand Oaks, Calif: Sage Publications
USAID.GOV USAID SAEP EVALUATION DESIGN REPORT | 74
Utility Representatives. The ET will sample a variety of utility representatives from each of the partner
countries. Interviews with utility representatives will aid the ET in answering EQs two through six, helping the
ET to understand how policy and regulatory reform efforts have affected their operations as well as service
delivery to customers. The ET will target a variety of utility representatives to ensure robust findings,
including those of each SAEP country, different sized utilities, those who have more/less experience with
private sector engagement, with utilities operating under different regulatory structures, both managerial as
well as technical utility staff, and (to the extent feasible) both male and female representatives.
Regulators. KIIs with regulators will explore the policy and regulatory environment in each of the partner
countries and how SAEP’s activities have affected their work and will contribute to answering EQs two
through six. The ET will target regulator representatives in each of the partner countries, will include
individuals at multiple levels with the organizations, those in a variety of regulatory environments, and (where
feasible) both male and female representatives, ensuring that all perspectives are considered.
Partner Government Stakeholders. Policy or regulatory reform efforts require extensive
collaboration with government stakeholders. These representatives will be able to share their insights
regarding the strengths of SAEP’s interventions as well as the challenges faced. Interviews with government
stakeholders will contribute to answering EQs two through six. The ET will target representatives that have
been most heavily involved in SAEP’s activities and those with oversight over the energy sector. To the
extent feasible, the ET will target representatives from across different government entities within each
country and will include both managerial as well as technical staff of the ministries/agencies. To the extent
feasible, the ET will include both male and female stakeholders.
Regional Entities (Southern Africa Power Pool, Southern Africa Development Community
(SADC), etc.). In addition to utilities and regulators within specific countries, SAEP intervened at the
regional level, supporting reforms and providing technical assistance to regional entities such as the SAPP and
SADC. Thus, the ET will explore their perceptions of SAEP’s work in the power sector. Where possible and
relevant, the ET will seek to interview representatives from a cross-section of regional entities and will
include both managerial as well as technical staff.
Representatives of Other Donors. In many cases, SAEP has supported and assisted other donor￾funded activities and projects. In some cases, these were other USAID activities, but SAEP supported work
by donors such as the World Bank and the African Development Bank. In countries where this support
occurred, the ET will interview representatives from these other donor institutions to assess their
perceptions of SAEP’s support and what that support did or didn’t help them to achieve. Where possible and
relevant, the ET will seek to interview representatives from a cross-section of engaged donors and will
include both managerial as well as technical representatives. 
Private Sector Actors. Policy and regulatory reform in the energy sector often includes a goal of
increasing private sector engagement in the sector. Thus, KIIs with private sector representatives will offer
key insights into the extent to which SAEP has been effective at attracting and facilitating private sector
engagement. These KIIs will assist the ET in answering EQs two through six. The ET will target private sector
representatives in each of the partner countries, private companies involved across the spectrum of power
services (generation, transmission, distribution, maintenance, etc.), and will aim to include both smaller and
larger companies. The ET may seek out relevant industry and trade group representatives. Where feasible,
both male and female representatives will be included.
75 | USAID SAEP EVALUATION DESIGN REPORT USAID.GOV
DATA ANALYSIS
The ET will analyze data through an iterative process during and after the data collection. Triangulation of
emerging evidence from the literature, quantitative survey and the KIIs will allow the SI team to check for
trends, consistency, affinity, and contradiction between quantitative and qualitative data, aiming to better
answer the EQs, generate more reliable findings, and reach well-founded conclusions. The SI team will apply
a GESI lens and employ data triangulation to understand if and how findings differ by gender or social strata.
Quantitative Data Analysis. The ET will perform rigorous data quality checks and review the data using
SI’s high frequency check protocols. These checks will allow the SI team to identify survey logic, data range,
missing values, implausible values, digit preference (for numeric responses), and outlier errors in the data. If
such issues exist and survey modifications are deemed necessary, survey changes will be documented. The SI
team will execute a final cleaning process before analysis to review potential data quality concerns in the final
dataset. The ET will utilize Stata software to explore trends and disaggregate by country, sex, type of energy
source, and other relevant key variables to understand key outputs and outcomes. 
Qualitative Data Analysis. During data collection, the ET will take detailed notes, and if consent is
provided, record all discussions. Notes will be translated into English, as necessary. Once data collection is
completed, the SI team will utilize a structured coding approach to conduct content and thematic analysis
using Dedoose software for identification of emergent themes and contextual factors upon which to draw
findings and conclusions and to develop evidence-based recommendations. The ET will utilize a Findings,
Conclusions, and Recommendations Matrix to triangulate data across data sources and to ensure that analysis
is evidence-based, triangulated, and systematic. This will identify any challenges and suggest improvements
that can enhance the impact of the program, reduce inefficiencies, and enhance learning.
QUALITY ASSURANCE
Making evidence-based decisions only works well when the data are of high quality and can be trusted. The
ET will implement SI’s proprietary EQUI® system to ensure the evaluation meets rigorous international
standards for technical excellence. The system includes seven utilization- and quality assurance checkpoints
that include detailed guidance notes, checklists, and templates for the ET to use to ensure consistency with
SI’s best practices. As a part of this system, each evaluation deliverable—including all drafts of the evaluation
design and final reports, qualitative and quantitative instruments and data, and presentations—undergoes a 
rigorous quality review, using checklists to guide development of usable, high-quality products that consider
gender and social inclusion considerations. 
ETHICAL CONSIDERATIONS
SI has an in-house institutional review board (IRB) that is available to review data collection instruments for
evaluations, assessments, and surveys when working with vulnerable populations. SI’s IRB will review and
provide approval prior to the start of data collection. Data security is a critical aspect of data collection and is
closely related to other efforts to safeguard respondents. All SI evaluation personnel—including
headquarters staff and consultants—will follow professional and ethical guidelines to ensure that the
evaluation is carried out with honesty and integrity, respondents are protected, and data security is ensured.
Before commencing data collection, the ET leader will conduct a meeting with the ET to ensure that the
team is familiar with the ethical standards laid out in the IRB application, especially COVID-19 related
protocols and agree on interviewing techniques. This includes ensuring that respondents’ confidentiality and
privacy will be protected during data collection through procedures for obtaining informed consent to collect
data in any medium, including informing respondents about the potential uses and any potential sharing or
publication of their data. All members of the ET who will be engaged in data collection with human subjects
USAID.GOV USAID SAEP EVALUATION DESIGN REPORT | 76
will be required to complete a certificate course on essential elements of ethics prior to the start of data
collection.
GENDER AND SOCIAL INCLUSION PLAN
The evaluation will aim to assess to what extent inclusion of planned activities, such as gender specific
trainings, guidelines, indicators, policies, and strategies, have been implemented and/or adopted. With an eye
toward assessing SAEP with a GESI lens, the ET will review data through the document review process and
aim to maintain gender balance in the KIIs (to the extent practical given stakeholder distributions), identifying
any inequalities and differences in outcomes. The ET will ensure USAID’s Automated Directives System 205
standards are maintained using a GESI scorecard. This scorecard will allow the ET to assess gender sensitivity
throughout the evaluation process, including within the submitted data collection instruments. Collected data
and analysis will be disaggregated by sex, socio-economic status, age, geographic region, and other
characteristics as necessary and possible to discern differences and similarities in experiences, perceptions,
needs, barriers, and other findings.
LIMITATIONS
The potential limitations as well as mitigation measures for the SAEP evaluation include:
Short Evaluation Timelines. To deliver the SAEP evaluation on the necessary timelines, SI will combine 
methods to quickly collect data from a broad range of stakeholders. SI will conduct a remote approach to all
data collection, which will speed data collection.
Large Geographic Dispersion. SAEP’s coverage of ten countries introduces a unique challenge for
ensuring that the views and perspectives of all stakeholders are included in the evaluation. The dispersion
creates both logistical challenges for reaching stakeholders in each country and poses challenges for
aggregating findings across widely varying contexts. SI’s remote approach to the evaluation design will help
mitigate the logistical challenges while ensuring data can be collected without delay. With regard to
aggregating findings, the ET will speak to a variety of stakeholders in each country and will take a close look at
potential inter-country differences when analyzing the data.
Response Bias. SAEP stakeholders may carry systematic but possibly unobserved biases specific to their
role, cultural considerations, and/or other factors. This bias could influence their responses (such as being
overly positive about achievements or saying what they think the ET or the Power Africa Program Office
wants to hear). The SI team will design the instruments carefully and triangulate evidence across methods to
minimize potential bias and ensure validity and reliability.
Selection Bias. Because the qualitative component relies on purposive sampling, it is possible that the
selection could introduce bias to the evaluation. Thus, it is possible that the selected individuals may not
represent all relevant views. The ET will mitigate this limitation by ensuring a wide range of views from
stakeholders across partner countries as well as from diverse contexts. 
Similarly, for the quantitative survey, though all relevant stakeholders will be included in the survey invitation,
individuals will decide whether or not to respond to the survey. It is possible that those who do respond
could be different than those who do not. To the extent possible, the ET will mitigate this by encouraging all
stakeholders to complete the short survey and to collect basic demographic data to assess any potential
skewing of the respondent pool.
77 | USAID SAEP EVALUATION DESIGN REPORT USAID.GOV
DISSEMINATION AND UTILIZATION
To ensure maximum utility of evaluation findings, the ET will build in several checkpoints during which the ET
can share emerging findings, conclusions, and recommendations with USAID. Wherever possible, the ET will
share emerging findings in an ad hoc manner through weekly reports and/or during weekly check-ins. The
formal checkpoints include an out brief at the end of data collection, a mid-term briefing/collaborative
workshop after substantive data analysis is complete but before report writing, provision of the Draft Final
Report, and a Final Dissemination Presentation.
At the end of data collection, the ET will meet with core USAID staff to provide an update on the data that
were collected and to provide an initial view of emerging findings. Emerging findings at this stage are subject
to change and/or refinement during data collection. But, often, there are some clear trends emerging from
data collection that would be useful for USAID to be aware of, and these will be shared at this time.
Following initial analysis, preliminary findings and conclusions will be shared with USAID via a mid-term
briefing/collaborative workshop. 
The Draft Final Report will provide the next check in point during which the ET will be able to formally
present its full findings, conclusions, and recommendations. This draft report will build on the feedback
already received during the out brief and the collaborative workshop to ensure that the report is tailored to
focus on areas of most interest to USAID and other key evaluation users.
The final version of the Final Report will be submitted to the Contracting Officer Representative for
approval and the approved version will be submitted to the USAID Development Exchange Clearinghouse.
Deidentified quantitative data will be shared for public use on the Development Data Library.
Once the Final Report is approved (or before, if preferable to USAID), the ET will organize a final
dissemination presentation to stakeholders to increase uptake and encourage learning.
USAID.GOV USAID SAEP EVALUATION DESIGN REPORT | 78
ANNEX 1: TIMELINE
DELIVERABLE DUE DATE
Evaluation Work Plan May 27, 2022
Evaluation Design Report June 10, 2022
Collaborative Workshop o/a August 10, 2022
Mid-Term Report o/a August 17, 2022
Draft Evaluation Report September 9, 2022
Final Evaluation Report October 7, 2022
Weekly Reports October 12, 2022
79 | USAID SAEP EVALUATION DESIGN REPORT USAID.GOV
ANNEX 2: BIBLIOGRAPHY
• Patton, Michael Q. (2002). Qualitative Research and Evaluation Methods. Thousand Oaks, Calif: Sage
Publications
• Power Africa Portfolio Review Presentation, November 2016
• Power Africa Portfolio Review Presentation, August 2021
• Power Africa Portfolio Review Presentation, May 2022
• SAEP Annual Report Year 1 
• SAEP Annual Report Year 2 
• SAEP Annual Report Year 3 
• SAEP Annual Report Year 4 
• SAEP Monitoring & Evaluation Standard Operating Procedures
• SAEP Performance Management & Evaluation Plan (PMEP) 2018
• SAEP PMEP 2020
• SAEP PMEP 2021
• SAEP Regional Course Catalogue October 2017
• SAEP Work Plan Year 1 
• SAEP Work Plan Year 3 
• SAEP Work Plan Year 4 
• SAEP Work Plan Year 5 
• Southern Africa Gas Roadmap for SAEP
• Southern Africa Gas Roadmap to Guide USAID SAEP Gas Work
USAID.GOV USAID SAEP EVALUATION DESIGN REPORT | 80
ANNEX 3: INFORMED CONSENTS
ANNEX 4: DATA COLLECTION INSTRUMENTS
ANNEX 5: SCOPE OF WORK
ANNEX 6: CONFLICT OF INTEREST DISCLOSURES
Annexes 3-6 have been removed here, as they are duplicates of the annexes in the Evaluation Report.