DISCLAIMER 
This report is made possible by the support of the American People through the United States Agency for International Development (USAID). 
The contents of this report are the sole responsibility of Deloitte Consulting LLP and do not necessarily reflect the views of USAID or the 
United States Government. This report was prepared under Contract Number AID-674-C-17-00002.
SOUTHERN AFRICA ENERGY PROGRAM
UTILITY MONITORING AND EVALUATION (M&E) MANUAL
FOR NEW UTILITY PERFORMANCE MANAGEMENT PROCESSES 
September 21, 2020
TIER 5: Public
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | i
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | ii
TABLE OF CONTENTS
1 BACKGROUND.......................................................................................................................................2
1.1 OVERVIEW..............................................................................................................................................................................2
1.2 M&E PROCESS MANUAL DEVELOPMENT...................................................................................................................4
2 INTRODUCTION....................................................................................................................................6
2.1 ROADMAP FOR THE PROCEDURES MANUAL.....................................................................................................6
2.2 THE TIMEFRAME ...............................................................................................................................................................9
2.3 OVERVIEW OF APPLICATION TO UTILITY...........................................................................................................9
3 THE NEW PERFORMANCE MANAGEMENT PROCESS........................................................... 11
3.1 M&E RESPONSIBILITIES: ROLES AND ACTIVITIES..............................................................................................11
3.2 EXECUTION TASKS AND RESPONSIBILITIES......................................................................................................20
3.3 REPORTING REQUIREMENTS ...................................................................................................................................28
3.4 DATA HANDLING PROCESS REQUIREMENTS...................................................................................................30
3.5 PROPOSED TRAINING REQUIREMENTS...............................................................................................................31
APPENDIX A: DEFINITIONS & MODELS.............................................................................................. 34
A1 Definitions & Concepts.......................................................................................................................................................34
A2 International Models and Equations.................................................................................................................................36
APPENDIX B: PM TOOLS .......................................................................................................................... 41
B1 Linkage to Strategic Plan & Timeframe Assessment tool: ...............................................................................................41
B2 Metric, Process and Organizational Capability Assessment tools:..................................................................................45
B3 Strategic KPIs - M&E Assessment tool:..............................................................................................................................50
B4 Operational KPIs - M&E Assessment Models: ..................................................................................................................53
B5 Service Providers KPIs - M&E Assessment tool:................................................................................................................56
B6 Utility KPIs – Balanced Scorecard (BSC)...........................................................................................................................56
APPENDIX C: M&E FUNDAMENTAL SKILLS & SOFTWARE.......................................................... 58
APPENDIX D: SAMPLE FUNCTIONAL SPECIFICATION TEMPLATE FOR PROCUREMENT 
OF SOFTWARE AND IT DEVELOPMENT............................................................................................ 59
REFERENCES.................................................................................................................................................. 68
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | iii
LIST OF FIGURES
Figure 1 Utility Performance Management Guide......................................................................................................3
Figure 2: Performance Measurement Framework.....................................................................................................7
Figure 3: Roadmap for this Manual................................................................................................................................8
Figure 4: KPIs Categorization Map ..............................................................................................................................13
Figure 5: Detailed and Simplified Timeline for Utility M&E ...................................................................................14
Figure 6: Schematic of a Generation Utility ..............................................................................................................15
Figure 7: Conversion Model onto a Likert Score ....................................................................................................17
Figure 8: Gap Assessment Heat Map ..........................................................................................................................22
Figure 2 Data Warehouse .............................................................................................................................................30
Figure A1-1: KPI Distribution – Illustration of definitions 3 to 13 ......................................................................35
Figure A2-1: Subset of Statham’s Family of Curves.................................................................................................36
Figure A2-2: UCF Illustration of Power Stations Performance ............................................................................40
Figure A2-3: Dashboard illustration............................................................................................................................40
Figure B2-1: Heat Map....................................................................................................................................................49
Figure B3-1: M&E Assessment form/template for Strategic KPIs.........................................................................50
Figure B3-2: M&E Assessment for Utility Operations Div. Strategic KPIs.........................................................51
Figure B3-3: M&E Assessment Formula for Strategic KPIs....................................................................................51
Figure B3-4: Strategic KPIs Conversion Model onto a Likert Score...................................................................52
Figure B4-1: Operational KPI: PCLF Conversion Model onto a Likert Score ..................................................53
Figure B4-2: Operational KPI: UCLF Conversion Model onto a Likert Score .................................................54
Figure B4-3: Operational KPI: UAGS/7000 Conversion Model onto a Likert Score......................................55
Figure B5-1: Service Providers KPI: Overall BSC for a Service Provider...........................................................56
Figure B5-2: Service Providers KPI: Section of BSC for the Operations Division...........................................56
LIST OF TABLES
Table 1: Criteria for Strategic KPIs.............................................................................................................................16
Table 2: Roadmap for selected utility .........................................................................................................................20
Table 3: Summary of Reporting Requirements ........................................................................................................29
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | iv
ACRONYMS
Acronym Definition
BoD: Board of Directors
BSC Balanced Scorecard 
CDS: Consistent Data Set
CEO: Chief Executive Officer
CI: Confidence Intervals - unless stated these are two-sided 95% CI
CM&E: Chief of the M&E Department
ECP: Error Correction Process governed by a Committee, aka Technical Audit Committee
EDA: Exploratory Data Analysis
ESO: Energy Sent Out in MWh
ExCo: Executive Committee
FinY-E: Financial Year-End
FinYr: Financial Year for the Utility as stipulated by the BoD
GoM: Goverment of Malawi
GU Generation Units
GWh: GigaWatt-hour = 1,000 MWh
HR&A: Human Resources & Administration
IBQ: International Best Quartile
IC: Installed Capacity
ICT: Information and Communications Technology
KFAs: Key Focus Areas aka Key Performance Areas
KPIs: Key Performance Indicators
M&E FDC: M&E Framework Development Committee
M&E: Monitoring and Evaluation
MD: Maximum Demand in MW
MW: Megawatt, measure of Power
MWh: MW-hour, measure of Energy
NESO: Net Energy Sent Out aka ESO
OCLF: Other Capability Loss Factor
PCLF: Planned Capability Loss Factor
P&D: Planning & Development
PM: Performance Management
POC: Point of Contact
QA&S: Quality Assurance and Safety
RM: Risk Management
SEO: Senior Evaluation Officer
Serv. Pr.: Service Provider
SMO: Senior Monitoring Officer
SOW: Scope of Work
SP: Strategic Plan aka Integrated Strategic Plan (ISP)
UCLF: Unplanned Capability Loss Factor
UNIPEDE: International Union of Producers and Distributors of Energy
References are in brackets, e.g. [3] and listed in the REFERENCE Section of this Report
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 1
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 2
1 BACKGROUND
1.1 OVERVIEW
The objective of USAID’s Southern Africa Energy Program (SAEP) is to increase electricity availability and 
access in Southern Africa. This is done by facilitating transactions and strengthening the enabling 
environment for public and private sector investment in the power sector. SAEP will address five key 
constraints to such investment: 
i) ineffective regulation, planning and procurement for energy; 
ii) low commercial viability of utilities;
iii) limited regional harmonization and cross-border electricity trade;
iv) lack of demonstrated and scaled clean and renewable energy and energy efficiency 
technologies and practices, and
v) weak institutional and human resource capacity for energy sector management.
The objective of SAEP’s Outcome 2 is to improve the commercial viability of utilities to increase their 
capacity to operate as off-takers and increase access to electricity. As an exemplary guide for other utilities 
undergoing the Monitoring and Evaluation (M&E) process, SAEP outlines six tasks conducted previously 
for an M&E process for a utility in Africa:
• Task 1: Assessing the current PM culture, organizational structure and process
• Task 2: Developing the new monitoring and evaluation system including an utility performance 
scorecard
• Task 3: Developing the new PM process
• Task 4: Developing the new PM organizational structure
• Task 5: Communicating the new utility PM structure, process and scorecard
• Task 6: Implementing the improved utility PM structure, process, tools, and scorecard
As a brief overview in how these tasks could relate to another applicable utility:
Task 1 involved a review of the SP, the organizational structure, the PM process, and an assessment of the 
management of data. A gap assessment within the utility was used to identify gaps between the current 
situation and an acceptable desired state for PM at the utility. 
Task 2 intended to design a performance indicator capture and reporting mechanism (scorecard) which 
supports the utility SP. Together with the utility, SAEP developed an Inquiry Document and an Inception 
Report.
Task 3 was segmented into two parts: Task 3A dealt with the workshops and Task 3B dealt with the 
procedures for the new PM process (this manual). The SAEP team held workshops with the Southern 
Africa utility to develop and expand on the development of the required processes which underline the 
operation of the M&E process.
For Task 4, a close interaction between SAEP and the Southern Africa utility needed to be established. 
SAEP facilitated this by supplementing and leveraging the functionality of an existing M&E Framework and 
Development Committee (M&E FDC) through the development of a Terms of Reference (ToR) which set 
out the operational framework for the Committee. These changes were approved and mandated by the 
CEO. The outcome was the establishment of an M&E Department in the Office of the CEO, supported 
by a workforce with detailed Job Output Models.
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 3
In Task 5, SAEP delivered a presentation to the utility’s EXCO of the New Performance Management 
Process, and proposed implementation of a Pilot Phase with the SAEP assistance to seek approvals for the 
former and the latter.
Task 6 addressed the following:
• Identification of tools needed for successful data capture and reporting
• Implementation of the data warehouse platform as per a Performance Measurement Framework 
• M&E Department has capability, is functional and operational as per expectation, 
• Software tools required for M&E staff for the devolution of their duties
• Facilitation of a Capacity Building workshop
This manual pairs well with SAEP’s leading practice guide in the brief Utility Performance Management 
Guide, outlined below.
Figure 1 Utility Performance Management Guide
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 4
1.2 M&E PROCESS MANUAL DEVELOPMENT
1.2.1 M&E Process Workshops
SAEP conducted three workshops at the utility to:
A. Assess the ICT systems and dovetail them onto a data warehouse platform
B. Identify goals, objectives and outputs for the new PM process
C. Identify phases, determine the progress on M&E assessment plan, as well as the progress on data 
analyst recruitment
1.2.2 Procedures Manual
With coordinated inputs from the workshops, SAEP commenced the development of the M&E procedures 
manual. The purpose of this manual is to provide a “living document” which sets out the actions, 
responsibilities and data requirements, and the reporting required consistent with the utility timeframe 
for PM. Essentially, this manual concludes Task 3, regarding the development of the new performance 
management process, procedures and tools to implement the monitoring and evaluation function; this is 
critical to the next phase of the project, which will involve a board meeting in early 2019 to review, agree, 
approve the M&E processes and organizational structure and endorse the setting up of the pilot project. 
This document serves as a guidebook which will be used by managers and staff to understand and 
implement the M&E process. It will be an integral part of the M&E training. 
This manual is the first edition, it is intended for readers with some fundamental utility knowledge and it 
is expected to be upgraded from time to time. From Section 3 onward, an M&E background is a 
requirement. 
The purpose of this manual is:
• Define the new PM roles and activities, particularly in the required M&E (new) Department, 
the Planning and Development (P&D) Division and the ICT Department.
• Provide the interactions with the divisional and departmental Point-of-Contact 
(PoC). These interactions need to coincide chronologically with deliveries
• Prepare the M&E Department with all skills needed for set-up, which serves as a guideline 
when the practical operation of the department will be needed.
• Detail information on the required M&E tools and interlinking communications with
the Information and Communications Technology (ICT) Division for the data capturing and
storage. Both tools and communications are outlined in this manual and essential for the final 
report to stakeholders.
• Provide a step-by-step process in line with a reporting schedule as determined by ExCo 
and BoD
• Provide a template of the functional specification for the data warehouse, in compliance 
with one of the tasks resulting from the workshops
All the above must be coordinated with a utility’s reporting timeframe. The skills and the tools need to 
go hand in hand with the training aspect of a utility’s initiative. A fundamental training program is described 
in Section 3.5 of this document.
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 5
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 6
2 INTRODUCTION
The goal of this assistance is to develop a new PM capability within a utility to monitor and report the 
progress towards the utility’s Strategic Plan (SP).
In this section, the duality of the PM Process Map and the Performance Measurement Framework is 
explained as it applies to any utility. Synchronizing deliverables relative to timing requirements is important.
The new PM process represents our recommended approach to ensure that the objectives of the M&E 
process will be achieved.
2.1 ROADMAP FOR THE PROCEDURES MANUAL
This manual provides the fundamental procedures for the M&E Department in the execution of its duties.
This section sets out the elements of the roadmap so that a logical step-by-step process is followed when 
going through the manual.
A Performance Measurement Framework is necessary for the effective performance of a utility. This 
framework was dealt with in detail in Task 2 and is illustrated in Figure 2 below. Suffice it to say that this 
framework depicts the different environments where accountability resides. For instance, in the decision￾making environment, the accountability rests in the BoD and ExCo; in the SP and PM environment, the 
accountability resides in the respective divisions. The accountability of the M&E and analytical environment 
resides with the M&E Department reporting to the CEO, while the data management and reporting 
environment accountability is shared between the M&E Department, ICT Department primarily, and
Quality Assurance and Safety (QA&S) and Risk Management (RM), secondarily (see Figure 2 below).
Divisions must achieve their strategic and operational Key Performance Indicators (KPIs) outlined in Figure 
2, as well as provide their own targets and agreed upon variations. KPIs serve as inputs to the respective 
division’s business plans. This is an iterative process which involves determining, according to the initial 
strategic objectives of the SP, the feasibility of the outcomes, based on the financial, staff, and asset 
constraints. The results based on the feasibility of the outcomes then feeds back and updates the SP 
accordingly. This may result in recalibrating the KPIs’ targets and spreads. In Figure 2, this repetitive
process is reflected in the arrow from the final environment, “5 Data Management and Reporting 
Environment” back to “1 Decision Making Environment” as the user re-calibrates the utility business 
requirements and needs.
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 7
Figure 2: Performance Measurement Framework
As illustrated in Figure 3 below, as a roadmap for developing a PM Procedures Manual, one needs to adopt
a bifocal approach; that is, to focus on both the PM Process Map (left side, figure 1) and the Performance 
Measurement Framework (right side, figure 2).
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 8
Figure 3: Roadmap for this Manual
The following provides an overview on how this roadmap is applied based on the SAEP SOW:
A. On the left-hand-side, the PM Process Map of Figure 3 focuses on:
1. The defined goals of the company and the application of the gap analysis provide a present 
and a desired state of the utility. The gap analysis provides the foundation of what, where 
and who is required to achieve the desired state (Task 1)
2. Tasks 2, 3 and 4 define the specifications for the what, where and responsibility of the 
desired state
3. Both tasks 5 and 6 place into action the results of 1 and 2 above by means of managing 
this process, prototyping, validating, reporting and communicating
B. On the right-hand-side, the Performance Measurement Framework of Figure 3, based on the 
results of A.2 above, focuses on:
1. The ‘what’, which is essentially what needs to be measured (e.g. Strategic & Operational 
KPIs). The ‘what’ aspect also determines what data needs to be collected and stored
(which addresses the ‘where’ to collect it from and ‘where’ to store it – the data 
warehouse at the center of the framework)
2. The ‘where’ addresses where in the Performance Measurement Framework the process 
takes place and is very much dependent on the process’ requirement and the four main 
environments that play a part in this process (e.g. as a requirement, the decision-making 
environment in Figure 2 might require a variance report on three categories of a 
maintenance plan: 1i) the original one at the beginning of the Financial Year, ii) the 
adjusted one during the Financial Year and iii) the actual one at Financial Year End (see 
Section 3.2.2.1). The repositories in the data warehouse would be: for i) in the CDS 
source layer, in the raw rata and CDS source layers for ii) and iii). The Maintenance Plan 
Variance Report would be stored in the results/reporting layer of the data warehouse
3. The areas of responsibility, in turn, are dependent on the ‘where’ (e.g. an evaluation 
specialist would be producing energy forecasts in the M&E analytical environment)
4. The remaining element is the ‘when’ and this is addressed in paragraph 2.2 below and 
provides the third dimension of the roadmap
a utility
Utility
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 9
2.2 THE TIMEFRAME
The process timeframe within the Performance Measurement Framework plays an important role in the 
process. The synchronization of the outputs from the SP, to the development of models (e.g. the KPIs 
targets and spreads, forecasts of energy, predictions of forced outages, etc.) to the reporting, need to 
occur at planned times for the framework to operate not only efficiently, but also effectively. Accordingly, 
a detailed schedule for the process needs to be determined and agreed upon to ensure all aspects of the 
process happen when designated.
2.3 OVERVIEW OF APPLICATION TO UTILITY
As indicated in their SP, this example utility’s overarching objective is to generate sufficient, reliable power 
which requires a means to ensure its performance capability. An M&E Department whose goal (see Figure 
3 – left hand side) is to accomplish the performance capability measuring function can assist. Based on the 
information acquired from the M&E Department, the utility is empowered to make knowledgeable 
decisions (see Figure 3 – right hand side). Given the above, the roadmap for this manual is to work 
clockwise within the Performance Measurement Framework (Figure 3 – Right hand side), starting at the 
“Utility business requirements and needs”, working clockwise through the “SP alignment via KFAs and 
Outputs” and finishing off at “Reporting, Displaying”.
The mode of operation for implementing the M&E process working tasks is addressed in Section 3 of this
manual. There are 6 appendices which are integral to this manual and provide detailed information and 
techniques required to achieve the results for the various tasks:
• Appendix A contains the Definitions & Models 
• Appendix B provides the PM Tools utilized
• Appendix C sets out the M&E skill and software requirements
• Appendix D provides a functional specification template for the utility
The utility’s timeframe is defined by its annual financial cycle (sample period1 July to 30 June) and 
requirement from other critical stakeholders, examples could be the Ministry of Finance (Treasury) of the 
country’s government . The utility must ensure the complete set of deliverables work within these
constraints.
Please note the abbreviations mentioned in the acronyms section at the beginning of this report. In this report,
“upload” or “uploading” means storing of data/information in the data warehouse.
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 10
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 11
3 THE NEW PERFORMANCE MANAGEMENT PROCESS
This section sets out the development of the new PM process as introduced in Section 2 above. It is highly 
recommended that readers familiarize themselves with the references here cited either prior or in parallel 
with reading this document which will provide useful context. 
This process manual focuses on the responsibilities, activities and outputs required to manage the M&E 
process. Accordingly, the manual is based on the assumption that an M&E Department has been constituted and 
fully complemented with the necessary members of staff, as well as that the KPIs and Balanced Scorecard (BSC)
mentioned in Appendix B, and with the fundamental assumption is that the Data Warehouse and the Error 
Correction Process (ECP) Committee are operational.
Given the above, the new PM process is focused on collecting, assessing and reporting on the KPI and 
BSC performance and corrective actions that need to be taken. In this regard, the following items are 
discussed below:
• Responsibilities in M&E: roles and activities
• Responsibilities in M&E: working tasks required
• Reporting requirements
• Data handling process
• Entailed training
We commence this process by first setting out the responsibilities in defining the roles and activities of 
the divisions (particularly the P&D Division) and the M&E Department. For a newly formed utility (like
our example utility), this initial step is determined through an initial review of existing capabilities versus 
a “to be state”. This was accomplished in the first task of the SAEP assistance. As the utility matures, this 
Inception Report [1] of this first task becomes a status report to assess actual state versus a desired state
of PM. This is detailed later in this manual. 
3.1 M&E RESPONSIBILITIES: ROLES AND ACTIVITIES
Usually, a utility’s business requirements and needs are the fundamental results emanating from a SP or 
Integrated Strategic Plan (ISP). The SP would be constructed by a P&D Division with the assistance mainly 
of an M&E Department, an ICT Department, a RM Department and the inputs from other divisions. In 
this case, the Operations Division (which manages the generation assets of the utility), the Finance and 
HR & A Divisions would be supplementing the required inputs. The roles and activities provided below 
are by no means exhaustive and are more likely to be modified as the utility matures. 
a. P&D Division
The role of this division provides the strategic projects in the SP that need to be completed by the utility 
to achieve the requirements of ExCo and the BoD mandates. These projects need to be aligned with 
fundamental PM principles [8], so that they can be monitored and evaluated; this is where the close 
relationship with the M&E Department takes place. For the latter to perform its functions, the progress 
of these projects needs to be measured in an acceptable format, which will be covered later under the 
M&E Department paragraph of Section 3.1.
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 12
Once developed by the P&D Division, the SP should have the following components as a minimal 
requirement:
i. Four Fundamental Key Focus Areas (KFAs) (a.k.a. Key Performance Areas) based on the seminal
works of Kaplan and Norton [8]; these are given below (in { } are the utility’s named KFAs):
o Customer: “To achieve the utility’s vision, how should the utility appear to its
customers?” {Customer, Stakeholder and Corporate Image}.
o Finance: “To succeed financially, how should the utility appear to its shareholders?” 
{Revenue Generation and Financial Stewardship}
o Internal Business Process: “To satisfy its shareholders and customers, what business 
processes must the utility excel at?” {Business Process Excellence}
o Learning and Growth: “To achieve the utility’s vision, how will the utility sustain its 
ability to change and improve?” {Learning, Growth and Innovation}.
ii. The strategic objectives dealing with the answers to the KFA questions, as adapted from [9] p 19, 
we point out that before any metric selection, utilities should describe what they are attempting 
to achieve with their strategies, and, furthermore, we point out that the four fundamental KFAs 
provide a robust structure for companies/utilities to express their strategic objectives. Hence, 
strategic objectives, in a SP, need to be the minimal requirement for having line-of-sight between 
the strategies and the KFAs.
iii. The third minimal requirement are the Key Tasks or Activities that are needed to be carried out 
in order to achieve the strategic objectives.
iv. The fourth essential requirement is the output expected with its corresponding starting and 
completion time targets and the division/department responsible for that output deliverable.
All these points are the application of the elements mentioned in 2.1
b. M&E Department
The functions of this department are essentially interacting in two modes of operation: the monitoring 
function and the evaluation function. It is important to note that the divisions are the owners of their data sets 
and therefore responsible for the collection and provision of the PM-required data. Hence a link or POC between 
each division and the M&E Department is essential.
The M&E Department is responsible for the wholistic PM requirements of the utility. As a reference, using 
the Heat Map in Figure 8, this would mean ensuring that the linkages to the SP, the metrics, the processes 
are in place and related to the HR&A, the organizational requirements for an effective PM process. Given 
the linkage with the P&D Division in terms of the SP, the M&E Department needs to give guidance and 
strategic direction in the PM requirements. This department reports on monitoring and evaluation results 
(from BSCs/Dashboard) and would be responsible for the KPIs & BSCs structure, design (usually carried 
out by means of participative workshops with the divisions) and reporting thereof. The M&E Department
contributes to performance improvement mechanisms of the utility by assisting the CEO in making 
information-driven decisions. The department presents to the CEO the PM Targets for submission to the 
Stakeholders (in the utility’s case, the GoM – Treasury).
The KPIs structure is illustrated in diagrammatic form in Figure 4.
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 13
Figure 4: KPIs Categorization Map
The following paragraphs detail the “what” and the “how” for each of the functions in the M&E 
Department
b1. The Monitoring Function
The principal tasks for this function in the M&E Department are as follows (see Section 3.2 Working Tasks
1 to 16 and Annex 0 & Ax3):
1. Collection: of data from assigned POCs across the utility by using appropriate techniques (data 
capture and storage, surveys, interviews) to monitor performance
2. Coordination: by synchronizing the data collection from the divisions at predetermined times
(Figure 6 below)
3. Validation: by substantiating the data provided by the divisions & SPs, with the support of the 
evaluation function, in terms of:
i. Preparation: Prepare the data so that outliers, mistakes are eliminated
ii. Formatting the “clean data” into an analytical protocol (e.g. variables in columns-observations 
in rows)
iii. Aggregation in predetermined tranches (e.g. hourly data, weekly data)
4. Structuring: by organizing the validated data in the required Reporting formats (e.g. Monthly 
reports on Planned Outages) – The reporting formats are dictated by ExCo
5. Storing: by gathering the processed data (as per 2 and 3 above), in a Consistent Data Set (CDS)
repository, for analytical and for reporting purposes at predetermined times (Figure 6 below)
6. Reviewing: by modifying and developing evaluation tools, mechanisms, with the assistance of the 
evaluation function, modalities (review of PM Standards, PM processes) and providing comprehensive 
source of evidence documentation
7. Verification: by assessing the reporting reliability of the divisions and SPs in terms of PM
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 14
Assessing: by appraising the periodic performance of the utility’s strategic KPIs and, secondly, at FinY-E, the 
performance of all the utility’s KPIs
8. Informing: by providing monthly reports developed above to the utility’s ExCo, divisions and 
departments
9. Monitoring: by reviewing the current situation (vis-à-vis PM) in the utility and performing a gap 
assessment as mentioned in the status report paragraph above.
Tasks 1 to 9 above are regulated by the Chronological Timing Cycle shown in Figure 5 below.
Figure 5: Detailed and Simplified Timeline for Utility M&E
b2. The Evaluation Function
This function is essential for the performance assessment in any utility. It provides an assessment of the
robustness of the PM results for the decision-making environment (see Annex Ax.1). It is recognized as a 
highly specialized function and necessitates particularly skilled resources. In todays’ terminology these 
resources are referred to as Data Scientists and more in this regard is covered in Section 3.2 below.
Figure 6 below provides a representation of a generation utility having a mix of hydro-reservoir and gas 
generating plants.. This figure illustrates the Primary Energy (Water, Gas) being converted into the Energy 
Sent Out (ESO in MWh) for each of the Power Stations (Station 1, Station 2, …, Station). Each of these 
stations are subject to energy losses due to Forced Outages, Planned Outages and Other Losses outside 
management control (such as droughts). Most utilities use these as critical operational KPIs for the energy 
capability and availability. When utilizing these operational KPIs it is vital to utilize a recognized 
Start 
financial 
cycle
End
financial 
cycle
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 15
international system. In this manual the “International Union of Producers and Distributors of Energy” 
(UNIPEDE) system is used.
The corresponding terminology is as follows (see Appendix A points 18 to 26 for detailed Definitions and 
Equations):
Forced Outages ➔ UCLF
Planned Outages ➔ PCLF
Other Losses ➔ OCLF
In this manual from here onwards the UNIPEDE definitions will be used.
It is clear that the Unplanned Capability Loss Factor (UCLF) and Other Capability Loss Factor
(OCLF) are random variables which need to be predicted, while the Planned Capability Loss Factor
(PCLF) follows a certain maintenance plan for a particular period. This period is usually ringfenced to one 
year.
Figure 6: Schematic of a Generation Utility
The principal tasks of the M&E Department for this function are as follows (see Annex Ax2 & Ax3):
1. Forecasting: of time-related variables (with random noise)
Usually the first step is to forecast the Net ESO (NESO aka ESO); this is the total energy required
by the grid in different time tranches (daily, weekly, monthly, quarterly and for the FinYE) usually 
measured in GWh. This entails the development of forecasting models (e.g. ForecastPro – see 
Appendix C).
2. Predictions:
i. Hand in hand with the ESO Forecast goes the Prediction of the Maximum Demand (MD)
required by the grid in different time-tranches (daily, weekly, monthly, quarterly and for the 
FinYE) generally measured in MW. Usually, the MD is linearly highly correlated (the Evaluation 
Function needs to verify this correlation regularly for different Time-Tranches) and hence, given 
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 16
a forecasted NESO, a MD, with corresponding confidence intervals (CI) needs to be predicted.
This entails the development of Predictions models (say, using Statistica – see Appendix C).
ii. Being random variables the UCLF and OCLF needs to be predicted using Statistical techniques. 
The latter are highly dependent on the type of Plant mix in a Generating Utility mix and the 
statistical models vary significantly and a comprehensive Exploratory Data Analysis (EDA) needs 
to be performed. This entails the development of Statistical Distributions models (say, using 
Statistica – see Appendix C).
iii. This UCLF and OCLF are weighted-Averaged across the Stations (the weight being the Installed 
Capacity (IC) of the station divided by the Total IC of Utility fleet), to obtain the expected Utility 
Total UCLF and Total OCLF. This entails the development of Simulation models (say, using 
@Risk – see Appendix C).
iv. Based on the UCLF prediction, the corresponding Total PCLF, under average conditions, should 
be between 2.33 to 3.33 times the UCLF to keep the Plant under expected running conditions.
v. Given the Total expectation of the planned maintenance (PCLF) according to iv. above and with 
its corresponding CIs, these values are then submitted to the P&D and Operations Divisions as 
constraints (or goals – depending on the Optimization Model used or scenarios – depending on 
the Simulation Models used) for the most likely outcome of the Maintenance Plan.
3. Development: of PM models across the Utility by using appropriate techniques. The development 
of these models is by no means exhaustive. However, there are basically three fundamental PM 
Models, the Strategic KPIs Model, the Operational KPIs Models (note the plural) and the Service 
Providers KPIs Model.
i. The PM model for the Strategic KPIs; there is essentially one model for these KPIs which is based 
on the criteria classification shown in Table 1 and its conversion is illustrated in Figure 7 below.
Table 1: Criteria for Strategic KPIs
< 25% of Norm is achieved
25% ≤ Norm achieved < 75%
Norm is achieved 
(Completed by expected date)
Facilitation provided to functional 
areas to enable delivery against 
Norm
Delivery realisation enabled 
against SP objectives (tracking 
& progress verified and reported)
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 17
Figure 7: Conversion Model onto a Likert Score
The details of the Assessment Tool and Application are provided in Appendix B
ii. The PM models for the Operational KPIs (Appendix B4 for details)
Operations (aka Generation) Division
o Planned Outage Model: PCLF (see Appendix A definition-point 19)
o Forced Outage Model: UCLF (see Appendix A definition-point 20)
o Reliability Model: UAGS/7000(see Appendix A definition-point 26)
Finance Division
o Control on Cost of Production: Cost of Production Variance (%)
o Return on Capital Employed: ROCE (%)
o Gearing Ratio: Total Debt/Total Equity ratio
Planning & Development Division
o Planned Project Control: Abs. Time to delivery variation (%)
o Project Cost Control: Abs. Cost Variance (%)
o Implementation Failure: No. of Projects not implemented (relative to Plan)
Human Resource & Administration Division
o Organizational Churn: Employee Retention Index (%)
o Hiring Performance: 100% - Hire Failure rate (%)
o Organizational Climate: Leave Frequency (excl. Annual Leave) per person
iii. The PM Model for the Service Providers KPIs
The BSC design for the Service Providers (Horizontal KPIs) is based on 4 criteria: 
Timeliness, Responsiveness, Deliverables and Professionalism. The highest weight is given 
to the Deliverables, this includes the quality of the deliverable and the achievement of 
their mandates. The second highest weight is given to Responsiveness (e.g. answering 
emails, phone calls, etc.). The rationale being that this determines the effectiveness of the 
Service Provider, whereas efficiency is measured by Timeliness. Since Responsiveness can 
affect timeliness (changes due to unforeseen circumstances), being efficient in delivering 
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 18
at the stipulated time might be counterproductive due to the lack of Responsiveness; in 
turn, Timeliness can be affected by constraints beyond their control and hence the weight 
of Timeliness should be less than Responsiveness. Finally, Professionalism is the way the 
service providers conduct the activities within the aspect of the other 3 criteria.
4. Validations:
i. The Evaluation Function performs validations (see Appendices C1 and C2). The primary 
validation is on the SP (fundamental document for PM). This validation is used to support 
the Monitoring Function (as mentioned in paragraph b1-3 above) in terms of 
Consistency, Omissions and Duplications.
o Consistency is mostly achieved when:
• data is reasonably structured (e.g. in ASCII, CSV formats)
• data values conform with expectations or to conventional models
• a particular field that is supposed to be a function of n others is dependable
• information is specified during the data collection planning phase (as opposed to extra 
data that were not planned for)
• outliers are identified and managed
• ease of manipulation of data for analytical purposes is ensured
• comparative checks (assessing additivity e.g. No of Strategic Objectives < Activities 
< Outputs) are performed
• importing, exporting data files (e.g. data losses, data corrupted due to pauses during 
up/downloads) are dependable transactions
• human errors (e.g. especially with spreadsheets and manual entries) are minimized
• in analytical mode, sensitivities of the models are checked.
o Omissions, check for:
• missing data
• mislaying data (e.g. in the Responsibility field of the SP, “responsible person” is 
entered instead of, say, “Director of Finance”)
• planned omission (omission of data for a specified period, with valid reasons; e.g. 
during equipment change and testing thereof, such as a turbine shaft replacement and 
a turbine-acceptance-test)
o Duplications, check for:
• single: usually with mislabeling of Outputs (e.g. with abbreviations such as Mb for both 
Mabeele and Mayabongwe for a Power Station [fictitious names])
• multiple: generally due to copy-and-paste portions of a document/spreadsheet
ii. Once the Maintenance Plan has been finalized by the Operations Division (see 2 v. above) 
the expected NESO needs to be reconciled with the Forecasts obtained in 1 above.
iii. Validation of Weights
Elicitation of Weights via Divisional Workshops
Ensuring that Weights balancing equations are bias-free
5. Reviewing: The Evaluation Function reviews by modifying and developing evaluation tools, 
mechanisms
i. Reviews of Models adopted as mentioned in paragraphs 1 to 4 above
ii. Review of Consistent Data Set (CDS) in Data Warehouse
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 19
c. ICT Department
The ICT Department relationship with M&E is critically important. Data is collated by the systems 
supported by ICT in the utility’s divisions (e.g. the Operations Division’s Data is presently collected 
manually and will soon import via SCADA). The ICT is responsible for providing and maintaining the Data 
Warehouse.
d. Divisions & Service Providers
The divisions and SPs are responsible to provide a link to the M&E Department via a POC residing in their 
respective division or department. The POCs are responsible to upload the information required to 
populate the BSCs (KPIs, Targets, Weights, Actual values, Sources of Evidence). It is the POC’s
responsibility to ensure that these uploads are effectively implemented (accessible, the information is 
available on demand and reliable).
The divisions and departments are responsible for the reliability of the BSCs (refer to paragraph 3.2.2.1).
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 20
3.2 EXECUTION TASKS AND RESPONSIBILITIES
Table 2: Roadmap for selected utility
Work 
Task Requirement When 
required
Responsible 
party
Remarks
WT1 SP Review Every 3-5 yrs. CM&E Validation & Verification of 
Strat Objectives & Outputs
WT2 Review of Divisional 
& Serv. Prov. BSCs
FinY-E SMO Particularly Validation & 
Verification of Strategic KPIs 
and Weights
WT3 BSCs Assessments 3rd Week Aug 
FinYr
SMO Divisions & Serv. Pr.
WT4 Updated Forecasts &
Predictions
3rd Week Sep 
FinYr
SEO ESO, MD, UCLF, Revenue, and 
KPIs
WT5 PM1 Report to CEO Last week Sep 
FinYr
CM&E BSC Assessments; Forecasts & 
Predictions
WT6 PM1 Presentation to 
ExCo
End Sep FinYr CM&E Elicit ExCo for directives on 
PM1 results
WT7 Upload values of 
Financial KPIs and 
data from Financial 
Division
End Oct FinYr SMO Required for WT10
WT8 Semi-annual 
Validation & Review 
KPIs and Weights
Mid Nov FinYr SEO
WT9 Validation, Review 
and Upload 
Maintenance Plan
End Nov FinYr SMO
WT10 Revise Divisional 
KPIs 
End Nov FinYr SEO Subject to Financial, 
Maintenance, Forecast & 
Predictions constraints
WT11 PM2 Report to CEO Mid Dec FinYr CM&E Results from WT10
WT12 Revisions of WT11 End Jan FinYr SMO/SEO Based of feedback from the 
CEO
WT13 Utility KPIs and BSC 
Review Workshop
Mid Feb FinYr CM&E Div. & Utility KFAs, KPIs, 
Weights, Targets, Spreads, 
BSCs
WT14 PM3 Report to CEO 1 week after 
WT13
CM&E Results of WT13
WT15 Presentation to 
ExCo
End Feb FinYr CM&E Results of WT14
WT16 Presentation to BoD Mid Mar FinYr CEO / CM&E Results of WT14
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 21
Refer to Appendix C for skills and basic software requirements
The roadmap guide for selected utility in Table 2 above provides:
o The required Work Tasks (WT) that need to be performed
o When these WT need to occur
o The responsible party in the M&E Department for the WT
o Remarks summarizing the description of the WT
3.2.1 Work Task 1: Strategic Plan Review – Timing: every 3 to 5 years
Responsible Party: CM&E
The Inquiry Document
At this point, an Inquiry document, by either an external party to the Utility, or by a member of 
the P&D Division should have a checklist confirming the minimal requirements above. Such a 
checklist is provided in Appendix B1. It queries the state of the SP. A positive result in this regard 
would reinforce that the sequence of processes from there on having a robust foundation. 
In addition, a validation of the SP (Outputs particularly), is also necessary. This can be carried out 
with the assistance of the M&E Department whereby the Evaluation Specialist would check for 
issues (detailed in paragraph b. M&E Department, below) of:
o Consistency: e.g. data sequential in time (common error: Start and Completion times are 
not in sequence)
o Omissions: e.g. missing data
o Duplications: e.g. double labelling
The Status Report
This report provides the information of the PM status based on the information obtained from 
the Inquiry Document and a review of: 
o the Strategic Plan,
o the Organizational structure, 
o the Performance Management (PM) process 
o the management of Data,
A Situation Assessment (the Status) is evaluated based on the four points above. This evaluation 
constitutes the input for a Gap Assessment which is then used to identify gaps between the 
current situation and an acceptable desired state for Performance Management.
The Gap Assessment, in terms of PM, is measured by tools such as Heat Maps as illustrated in 
Figure 8 below. A detailed derivation of the Heat Map is provided in Appendix B.
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 22
Figure 8: Gap Assessment Heat Map
Each arrow indicates where each Division/Office stands (current situation) in terms of the SP
(KFAs and Time-Frame), the metrics (KPIs), the Processes in place (BSC and the Dashboard) and
Organizational capability to carry out the PM requirements. The example above (Figure 8), 
illustrates that, apart from the KFAs, all other PM requirements need attention.
The Situation Review and Gap Assessment are usually performed by the Monitoring Function of 
the M&E Dept or an external Specialized Entity (e.g. SAEP); every 3 to 5 years, this task should be 
performed by an independent Specialized Entity. Details of such workings are reflected in [1].
3.2.2 Work Task 2: Review of Divisional & Service Providers BSCs –
Timing: Fin Y-E
Responsible Party: SMO
At the Financial Year-End (e.g. if the Financial Year spans from 1 July 2018 to 30 June 2019, then Financial 
Year End is 30 June 2019) the SMO needs to action the monitoring of performance of the Utility.
The SMO needs to validate and verify the Divisional and Service Providers BSCs, to ensure that the 
performance assessments are reliable.
3.2.2.1 Validation of Divisional and Service Providers BSCs by acting on the following:
o With the POC, the SMO ensures that the Divisions and Department have uploaded 
their respective populated with Actuals KPIs values (of the BSCs) into the Raw Data 
layer of the Data Warehouse for the FinYE (these are the non-validated BSCs)
o The SMO, assisted by the SEO, analyze the Actual reported KPIs values to identify 
mistakes, outliers in the BSCs
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 23
o Report these findings, via the CM&E to the Division / Department copy to the ECP 
Committee, for a corrective (documented) response (if no findings: then report 
such to the ECP Committee)
o The SMO applies the documented changes as per ECP Committee resolution to the 
BSCs (if any) and uploads these into the Results/Reports layer of the Data 
Warehouse (these are the validated BSCs).
3.2.2.2 Verification of Divisional and Service Providers BSCs by acting on the following:
o The SMO performs a comparative analysis of the BSCs submissions (non-validated 
and validated) with the originally uploaded BSCs regarding the KPIs and the Weights
(both Strategic and Operational for the Divisions and for the Service Providers)
o The SMO assesses the BSCs reliability based on the following scoring model:
From the comparative analysis record the number of differences between the 
validated and non-validated BSCs for each Division / Department This number 
constitutes the number of findings; then apply the following Likert Scale:
Likert Scale: 5- No findings; 4- 1 finding; 3- 2 findings; 2- 3 findings; 1- > 3 findings
The score determines the level of reliability of the BSCs for each 
Division/Department
After completion of the validations and verifications (3.2.2.1 and 3.2.2.2), the SMO, via the CM&E, 
issues a Technical Audit Report, based on the findings above, to the CEO and the ECP Committee 
Chair. This report is uploaded in the Results/Reports layer of the Data Warehouse.
3.2.3 Work Task 3: BSCs Assessments – Timing: 3rd Week Aug FinYr
Responsible Party: SMO
During the third week in August of the FinYr, the SMO needs to assess the performance of the utility.
The SMO needs to assess the utility’s divisional and service providers’ performance by using the validated
BSCs (see Work Task 2), to ensure that the performance assessments are reliable.
3.2.3.1 For the assessment of the Strategic KPIs particular to each Division, the appraisal needs to be 
done at the end of each month of the FinYr (see Appendix B: Assessment Template B3 and 
Example B3-1). Hence, the SMO needs to work with the Assessment Plan that was submitted 
to the CM&E the previous FinYr (see Work Task 14) and during the previous FinYr, have 
assessed all the Strategic KPIs for all the Divisions. The SMO would then populate the
consolidated Strategic KPIs assessments in the validated BSCs of the respective Divisions
3.2.3.2 For the assessments of the Operational KPIs particular to each Division, the appraisal is done 
at FinY-E (the Operational KPIs values are uploaded by the respective Divisions POCs in the 
Raw Data layer of the Data Warehouse and the results are populated in the validated BSC 
of the respective Divisions. This task is very dependent on task 3.2.2.1 (first bullet) above.
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 24
3.2.3.3 For the assessment of the Service Providers KPIs the appraisal needs to be done at the end 
of each month of the FinYr (see Appendix B: Assessment Template B5). Similarly, to 3.2.3.1 
above, the SMO needs to work with the Assessment Plan that was submitted to the CM&E 
the previous FinYr (see Work Task 14) and during the previous FinYr, have assessed all the 
Service Providers KPIs for all the Departments.
3.2.3.4 For the assessment of the utility out of the Divisional and Service Providers BSCs automated 
final scores (as per 3.2.3.1 to 3.2.3.3 above), these must feed directly into the utility’s BSC 
(see Appendix B – B6). The SMO is responsible for this transition of information. 
After completion of tasks 3.2.3.1 to 3.2.3.4), the SMO, via the CM&E, issues a Performance 
Assessment Report, based on the findings above, to the CEO by the 3rd week in August of the 
FinYr. This report is uploaded in the Results/Reports layer of the Data Warehouse.
3.2.4 Work Task 4: Updated Forecasts & Predictions – Timing: 3rd 
Week Sep FinYr
Responsible Party: SEO
From the 1 July FinYr to mid-Sep FinYr, the SEO performs an Exploratory Data Analysis (EDA) on the 
historical Data of the KPIs. The SEO also models the behavior of the KPIs, Energy Sent Out (ESO) and 
Maximum Demand (MD) and utilizes those models to forecast and predict the next FinYr estimates with 
their corresponding spreads. The SEO then provides the following to the CM&E:
3.2.4.1 A Variance Report (historical) providing the difference between actuals and expected values
as well as the difference between the actual and expected variances of the KPIs, for the 
previous FinYr (using inferential methods). This report needs to include the significance level 
of the differences. Identify critical KPIs. Upload in Results/Reports layer of the Data 
Warehouse.
3.2.4.2 A Forecast & Predictions Report (for the FinYr) which contains the following: 
3.2.4.2.1 As per tasks in Section 3.1.b2.1 Forecasting and 3.1.b2.2 Predictions
3.2.4.2.2 Use the above and, provide simulations for the Revenue. Estimate the Costs of 
Production (these will differ from Utility to Utility and may change from FinYr to 
FinYr). Compute Financial KPIs and other requested Indicators (by the CM&E), 
based on the simulations.
3.2.4.2.3 Based on 3.2.4.21 and Status Report, model the 5year Forecasts using Statham’s 
Curves (with CIs)-See Appendix A1 and Appendix A2. Provide Sustainability 
curves for critical KPIs (as per Variance Report in 3.2.4.1). 
3.2.4.2.4 A summary of 3.2.4.2.1 to 3.2.4.2.4
3.2.4.2.5 Upload in Results/Reports layer of the Data Warehouse.
3.2.5 Work Task 5: PM1 Report to CEO – Timing: Last Week Sep 
FinYr (prior to ExCo)
Responsible Party: CM&E
From the 3rd Week Sep FinYr to the last week Sep FinYr, the CM&E collates the Variance Report and the 
Forecasts & Predictions Report and, by summarizing them, produces the First Performance Management 
Report (PM1) in tandem with a slide presentation for Exco.
The CM&E, then: 
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 25
3.2.5.1 Sends the PM1 report and presentation to the CEO for changes (if any) and approval to 
present to ExCo
3.2.5.2 Once approved, uploads in Results/Reports layer of the Data Warehouse.
3.2.6 Work Task 6: PM1 Presentation to ExCo – Timing: End Sep 
FinYr
Responsible Party: CM&E
Presentation and hand-over of PM1 Report to ExCo. Depending on the results, the CM&E would elicit 
comments/directives on the past FinYr Performance and on the Targets (with Spreads) that ExCo may 
wish to set for the next FinYr. The Record of Decisions are documented and uploaded in the Results/Reports 
layer of the Data Warehouse.
3.2.7 Work Task 7: Upload values of Financial KPIs and data from 
Financial Division – Timing: End Oct FinYr
Responsible Party: SMO
By the end of October FinYr, the SMO assures that the POC (Finance) has uploaded Financial Data (incl. 
KPIs data) in the Raw Data layer Data Warehouse. As per the M&E Monitoring functions, i.e. 3.1.b1, 
the SMO performs Tasks 1, 3, 4 and 5 and may require assistance from the SEO. After all validations are 
checked, the information is the Financial Data and KPIs Report which is uploaded in the CDS layer of the 
Data Warehouse.
3.2.8 Work Task 8: Semi-annual Validation & Review KPIs and 
Weights – Timing: Mid Nov FinYr
Responsible Party: SEO and CM&E
By mid-November FinYr, the SEO repeats tasks 3.2.4.1 and 3.2.4.2 and then provides the following to the 
CM&E:
3.2.8.1 Semi-annual Variance Report
3.2.8.2 Semi-Annual Forecasting & Predictions Report for FinYE
The above, once ratified, are then uploaded in the Results/Reports layer of the Data Warehouse.
Then, the CM&E:
3.2.8.3 Collates the Semi-annual Variance Report and the Semi-annual Forecasts & Predictions Report 
and, by summarizing them, produces the Mid-FinYr Performance Management Report and 
summarizes it into a presentation for the CEO
3.2.8.4 Sends the Mid-FinYr Performance Management Report and presentation to the CEO for 
changes (if any) and directives therefrom. Once ratified by the CEO the report and 
presentation are uploaded in the Results/Reports layer of the Data Warehouse.
3.2.9 Work Task 9: Validation, Review and Upload Maintenance Plan 
– Timing: End Nov FinYr
Responsible Party: SMO
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 26
By the end of November FinYr, the SMO assures that the POC (Operations) has uploaded the 
Maintenance Plan in the Raw Data layer Data Warehouse.
As per the M&E Monitoring functions, i.e. 3.1.b1, the SMO performs Tasks 1, 3, 4 and 5 and may require 
assistance from the SEO. After all validations are checked, the information is compiled the Planned 
Maintenance Original M&E Report which is delivered to the CM&E for ratification and then uploaded in 
the CDS Source layer of the Data Warehouse.
3.2.10 Work Task 10: Revise Divisional KPIs – Timing: End Nov FinYr
Responsible Party: SEO
By the end of November FinYr, the SEO needs to assess the feasibility of the performance expectations, 
based on the information provided so far in the FinYr by:
3.2.10.1 Collates the information from Work Tasks 4 to 9 for the Divisions
3.2.10.2 Analyzes the obtained information as per the functions in 3.1.b2
3.2.10.3 Designs either an optimization model subject to constraints given in Work Tasks 7 and 9 or 
constructs various scenarios (based on 3.2.10.2) by using different simulation models.
3.2.10.4 Produces a Planned Maintenance Adjusted M&E Report to the CM&E and, once ratified, 
uploads it the Results/Reports layer of the Data Warehouse.
3.2.10.5 Consolidates the methodologies in 3.2.10.2 and the results obtained from 3.2.10.3, into a 
Revised Divisional KPIs Report to the CM&E and, once ratified, uploads it in the 
Results/Reports layer of the Data Warehouse.
3.2.11 Work Task 11: PM2 Report to CEO– Timing: Mid Dec FinYr
Responsible Party: CM&E
From the End November FinYr to the Mid December FinYr, the CM&E utilizes the Revised Divisional 
KPIs Report outcomes, and: 
3.2.11.1 Produces the Second Performance Management Report (PM2) in tandem with a slide 
presentation for the CEO. 
3.2.11.2 Presents to the CEO and the Record of Decisions are documented and uploaded in the 
Results/Reports layer of the Data Warehouse. 
3.2.12 Work Task 12: Revisions of Work Task 11– Timing: End Jan
FinYr
Responsible Parties: SMO and SEO
Depending on the results of the optimization and/or simulation results the CEO would provide certain 
directives (e.g. tighten the PCLF spreads, decrease the UCLF expectation with a 5-year strategic plan 
mapping, etc.); this means to:
3.2.12.1 SMO to inform the Divisional Directors and POCs as well as the HODs of the COE decisions
3.2.12.2 SMO to provide them with a Template which will consist at least of the following fields for 
preparing information and presenting at the Workshop (Work Task 13) on:
3.2.12.2.1 Historical Performance for Strategic and Operational KPIs and BSCs scores as well 
as descriptive reasons for such performance for the Divisions. For the Service 
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 27
Providers, their Historical Performance for the BSCs based on their Service Level 
Agreements with the Divisions and other Departments.
3.2.12.2.2 The next FinYr Strategic KPIs and Operational KPIs with corresponding spreads and 
weights (with balancing checks)
3.2.12.3 The SEO might be required to repeat Working Task 10
3.2.12.4 The SMO and SEO need to produce a Workshop Facilitating Presentation: Utility KPIs and 
BSC Review, for the CM&E and upload in the Compacting layer of the Data Warehouse.
3.2.13 Work Task 13: Utility KPIs and BSC Review Workshop –
Timing: Mid Feb FinYr
Responsible Party: CM&E and SMO
For this important strategic Workshop to be conducted at latest Mid-February FinYr, the CM&E is 
responsible for:
3.2.13.1The logistics (allocated suitable dates and times, suitable venue to accommodate participants, 
catering, travelling and accommodation requirement, laptop with projector, white-board with 
markers, etc.)
3.2.13.2 Officially informing the Divisional Directors and HODs of the logistics (latest Mid-Jan FinYr) 
via the Office of the CEO
3.2.13.3 Obtain the populated Divisional and Service Providers Presentations as per 3.2.12.2 and 
upload them into a folder “Utility KPIs and BSC Review Workshop / Original Submissions” in the 
Compacting layer of the Data Warehouse. 
3.2.13.4 Instructs the SMO to keep records of attendance and outcomes from the Workshop. The 
records of attendance and outcomes are collated into a Utility KPIs and BSC Review 
Workshop Report and uploaded by the SMO into a folder “Utility KPIs and BSC Review 
Workshop / Workshop Outcomes” in the Compacting layer of the Data Warehouse. 
3.2.14 Work Task 14: PM3 Report to CEO – Timing: 1 week after 
Work Task 13
Responsible Party: CM&E
Within one week after Work Task 13, the CM&E deals with:
3.2.14.1 The SMO submission to the CM&E, for the following FinYr, of the Assessment Plan for the 
Strategic KPIs that were derived from the Utility KPIs and BSC Review Workshop (see Work 
Task 13, also Appendix B3 and Appendix B3-1 Example)
3.2.14.2 The Utility KPIs and BSC Review Workshop Report by summarizing its outcomes 
3.2.14.3 Consolidating 3.2.14.1 and 3.2.14.2 into the Third Performance Management Report (PM3)
in tandem with a slide presentation for the CEO
3.2.14.4 Sending the PM3 report and presentation to the CEO for changes (if any) and approval to 
present to ExCo
3.2.14.5Once approved, uploads in Results/Reports layer of the Data Warehouse.
3.2.15 Work Task 15: Presentation to ExCo – Timing: End Feb FinYr
Responsible Party: CM&E
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 28
Presentation and hand-over of PM3 Report to ExCo. Depending on the results, the CM&E would elicit 
comments/directives on the next FinYr Performance Targets (with Spreads) that ExCo needs to ratify for 
the next FinYr BoD approval. The Record of Decisions are documented and uploaded in the Results/Reports 
layer of the Data Warehouse.
3.2.16 Work Task 16: Presentation to BoD – Timing: Mid Mar FinYr
Responsible Parties: CEO and CM&E
Presentation and hand-over of PM3 Report by the CM&E to BoD. Depending on the discussions, the 
CM&E would defend by substantiating the ExCo decisions (supported by the CEO and Director of 
Finance), elicit (if necessary) comments/directives on the next FinYr Performance Targets (with Spreads) 
that ExCo would need to change to seek BoD approval. The Record of Decisions are documented and 
uploaded in the Results/Reports layer of the Data Warehouse.
3.3 REPORTING REQUIREMENTS
In Section 3.2, each type of report is submitted by the responsible party. For example, it is the 
responsibility of the CM&E to ensure that the Technical Audit Report is in order, uploaded and delivered 
to the CEO and the ECP Committee Chair. Refer to Section 3.2.x for the details required in each Work 
Task x report; for instance, Section 3.2.2 will contain the details of Work Task 2 and its related report, 
i.e. the Technical Audit Report. All mentions of reports in 3.2.x are underlined.
The format and structure of each report does not form part of this Manual; the CM&E would structure 
each report according to the recipient requirements, and inclusive of the 3.2.x specified details.
A summary for the Work Tasks, Description, Due Time/Period, Report Type(s) and the Responsible Party 
is given in Table 3 below.
There are five critical Reporting requirements in the New Performance Management Process. These are 
best explained by the deliverables, namely (refer to Table 3 below): 
a. The Status Report (Work Task 1b)
b. The Performance Management Report 1 to the CEO (Work Task 5)
c. The Mid-FinYr Performance Management Report to the CEO (Work Task 8c)
d. The Performance Management Report 2 to the CEO (Work Task 11)
e. The Performance Management Report 3 to the CEO (Work Task 14)
Other Reporting requirements are the Inputs of the KPIs (all KPIs or sets thereof) and other Indicators, 
as required by ExCo, which would need displaying on a Dashboard(s) and would most likely change from 
time to time during the FinYr (especially the Strategic KPIs).
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 29
Table 3: Summary of Reporting Requirements
Work 
Task No. Description Due 
Time/Period Report Type(s) Responsible 
Party
1 Strategic Plan 
Review
every 3 to 5 
years
a. Inquiry Document
b. Status Report
a. CM&E
b. CM&E
2 Review of Divisional 
& Service Providers 
BSCs
Fin Y-E Technical Audit Report CM&E
3 BSCs Assessments 3rd Week Aug 
FinYr
Performance Assessment Report CM&E
4 Updated Forecasts 
& Predictions
3rd Week Sep 
FinYr
a. Variance Report
b. Forecast & Predictions Report
a. CM&E
b. CM&E
5 PM1 Report to 
CEO
Last week Sep 
FinYr
First Performance Management Report 
(PM1)
CM&E
6 PM1 Presentation 
to ExCo
End Sep FinYr PM1 ExCo Record of Decisions CM&E
7 Upload values of 
Financial KPIs and 
data from Financial 
Division
End Oct FinYr Financial Data and KPIs Report SMO
8 Semi-annual 
Validation & Review 
KPIs and Weights
Mid Nov FinYr a. Semi-Annual Variance Report
b. Semi-Annual Forecasting & 
Predictions Report
c. Mid FinYr Performance Management 
Report
a. SEO
b. SEO
c. CM&E
9 Validation, Review 
and Upload 
Maintenance Plan
End Nov FinYr Planned Maintenance M&E Report SMO
10 Revise Divisional 
KPIs
End Nov FinYr a. Planned Maintenance Adjusted M&E 
Report
b. Revised Divisional KPIs Report
SEO
11 PM2 Report to 
CEO
Mid Dec FinYr a. Second Performance Management 
Report (PM2)
b. CEO Record of Decisions
CM&E
12 Revisions of Work 
Task 11
End Jan FinYr Workshop Facilitating Presentation: 
Utility KPIs and BSC Review
SMO/SEO
13 Utility KPIs and BSC 
Review Workshop
Mid Feb FinYr a. Divisional and Service Providers 
Presentations
b. Utility KPIs and BSC Review 
Workshop Report
a. Divisional 
and Service 
Providers 
POCs; CM&E
b. SMO
14 PM3 Report to 
CEO
1 week after 
WT 13
Third Performance Management 
Report (PM3)
CM&E
15 Presentation to 
ExCo
End Feb FinYr PM3 ExCo Record of Decisions CM&E
16 Presentation to 
BoD
Mid Mar FinYr PM3 BoD Record of Decisions CEO / CM&E
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 30
3.4 DATA HANDLING PROCESS REQUIREMENTS
3.4.1 Data Warehouse
Links, uploads and downloads events are highlighted as ‘Data Warehouse’ at various points in 
this Manual.
o Regular Data Inputs for Operational KPIS (daily, weekly, monthly, quarterly, FinYE)
o Intermittent Data Inputs for Strategic and Service Providers KPIs
The two above are entered in the Raw Data layer of the Data Warehouse.
Figure 2 Data Warehouse
3.4.2 Data Connectors Required for:
o The aggregation from external data sources
o Queries
o Handling Big Data
o Analytical software interfacing
o The Inputs to Dashboard software
3.4.3 Outputs
o Structured Files
o Routine Reports (see Section 3.5 Reports Requirements)
o Customized Reports
3.4.4 P&D Division
The P&D Division relationship with ICT, is fundamentally important. Data is collated by the 
systems supported by ICT for the construction of the SP. This interface needs to be automated, 
and linked to the Data Warehouse.
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 31
3.4.5 M&E Dept
The M&E Department relationship with ICT, is also critically important. Data is collated by the 
systems supported by ICT in the Utility’s Divisions (e.g. the Operations Division’s Data is presently 
collected manually and soon it will be imported via SCADA).
3.4.6 Software
3.4.6.1 Information Management software needs to be produced to handle the Data Warehouse 
inputs/Outputs, the Documentation tracking, the BSCs assessments and the Dashboards 
populations. Interfaces with various platforms (as per 3.3.2) need also to be produced.
3.4.6.2 The term ‘produced’ is as per coding definition (i.e. design, testing, debugging, proto-typing 
and launching into production).
3.4.6.3 A decision will be taken after the Technical Specification (see Appendix C) is received on the
repository for the software; for instance, it could reside in the Raw Data, Analysts, 
Compacting and Results/Reports Layers or have its own layer in the Data Warehouse.
The platform used is illustrated in Annex Ax.3
3.5 PROPOSED TRAINING REQUIREMENTS
While not part of the SAEP scope, the Training should have its particular SoW and could be structured 
in three phases as a possible high-level plan:
Phase 1: ExCo training 
Phase 2: Management training 
Phase 3: Operating Staff & Analysts training
All 3 phases would start with an Induction particular to the targeted audience, This Induction would have 
an introduction to KPIs (types of measures [Qualitative & Quantitative]) and the KPIs elements of risk to 
achieve their expectations. Phase 3 Induction will include some Division Specifics (requirements). Roles 
of: M&E, Analysts, Service Providers.
After the Induction:
Phase 1 would include Interpretations consisting of: 
• KPIs (types of Measures: e.g. Scores, Units [UCLF%], …)
• Weights (constraints: e.g. sums must balance to 100%...)
• Targets & Spreads (strategic curves for Scores, for Units…)
• Sources of Evidence (for each KPI, auditable, …)
• Contracts & Compacts
• Practical examples within the utility
This would follow with Benchmarking consisting of:
• Internal Benchmarks (e.g. based on Statham’s Curves)
• International Benchmarks (e.g. based on International Best Quartiles)
• Practical examples within the utility
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 32
Phase 2 would include Interpretations & Implementation consisting of: 
• Understanding the BoD & Exco directives and Mapping these onto each Division BSC
• KPIs (types of Measures: e.g. Scores, Units [UCLF%], …)
• Weights (constraints: e.g. sums must balance to 100%...)
• Targets & Spreads (strategic curves for Scores, for Units…)
• Sources of Evidence (for each KPI, auditable, …)
• Contracts & Compacts (if required)
• Practical examples within the utility
This would follow with Benchmarking & Reporting consisting of:
• Internal Benchmarks (e.g. based on Targets and Spreads above)
• Reporting high-level formats
• Practical examples within the utility
Phase 3 would include Analyses, Construction of KPIs and BSCs consisting of: 
• Analyses of KPIs datasets
• Modelling and derivation of Targets & Spreads for KPIs
• Mapping these onto each Division KPIs & Service Providers
• Construction of New KPIs (types of Measures: e.g. Scores, Units [UCLF%], …)
• Construction of Weights (from SP, modifications and constraints: sums to balance 100%...)
• M&E information acquisition and scoring (evaluation)
• Practical examples for each Division & Service Providers
This would follow with Reporting consisting of:
• General reporting structural requirements
• Reports design
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 33
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 34
APPENDIX A: DEFINITIONS & MODELS
A1 Definitions & Concepts
1. Contracts: As per SP, a Division is responsible for targeting the Objectives’ outputs. Once this 
is done, then a measuring instrument is necessary to monitor the progress of particular outputs. 
Since this should be of meaningful substance, it ought to be a binding agreement (like a “promise”) 
between the responsible Division and the utility (represented by the BoD and ExCo). These 
instruments are called Contracts and span over a year (business or calendar).
2. Compacts: Compacts are aggregated Contracts to establish over a period (which normally 
corresponds to the tenure of the responsible Division – usually 5 years) how attainable the 
proposed targets are over time. The form part of the quantification of the Sustainability of the 
utility. 
Both Contracts and Compacts are quantitative demonstrations of the seriousness in the commitments 
made by the various Divisions responsible to the utility. They provide also a measure of feasibility in 
what can be achieved by when; in doing so, they enable the utility to make quantifiable changes to 
become progressively better at performing.
3. Likert scale: Here, it is adopted as a rating scale, to elicit information about a qualitative or 
quantitative KPI, for the purpose of quantifying (for a qualitative KPI) the measure, or reducing 
the KPI to a common scale (quantitative KPI), enabling weighted aggregation of the KPIs’. The 
scale utilized here is 1 to 5 where, the performance is: 
a. 1: Very poor
b. 2: Worse than expected
c. 3: As expected
d. 4: Better than expected
e. 5: Excellent
Down to 2-decimals values are permitted (e.g. 4.32). These are usually a result when the scores 
are a result of a weighted average.
Performance Statistical Model:
4. Norm: This is the expectation or estimated value that would be the median (quantitative KPI) or 
the expectation (qualitative KPI) that has a 50 : 50 chance of being achieved.
5. Floor: The KPI value (quantitative KPI) or event (qualitative KPI) that has a 95% chance of being 
bettered; it also provides a peg to evaluate overestimation of the Norm.
6. Kick-in: The KPI value (quantitative KPI) or event (qualitative KPI) that has an 80% chance of 
being bettered (close to the worse quartile).
7. Stretch: Given the Norm, the KPI value (quantitative KPI) or event (qualitative KPI) that would 
reflect a stretch on the Norm. This would have a 20% chance of being bettered (close to the best 
quartile).
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 35
8. Ceiling: The KPI value (quantitative KPI) or event (qualitative KPI) that has a 5% chance of being 
bettered. It constitutes the capping of the KPI; it also provides a peg to evaluate underestimation 
of the Norm.
Definitions 4 to 8 provide the elements that define the areas of performance or performance zones. 
These zones provide an additional dimension of awareness of the KPI’s condition. They provide the 
building blocks for a reporting dashboard.
Performance Zones:
9. Red Zone: Any KPI at or below the Floor, is deemed to affect the strategy and sustainability of 
the Company; could flag overestimation of the KPI.
10. Orange Zone: Value between Floor (excl.) and Kick-in (incl.), is intended as an alarm zone, to 
flag the state of the indicator.
11. Green Zone: Value between Kick-in (excl.) and Stretch (excl.). This zone shows that the KPI is 
meeting the agreed/negotiated expectation; it can signal the start of a proportional reward.
12. Blue Zone: Value between Stretch (incl.) and Ceiling (incl.). Within this zone, the KPI is 
exceeding the expectation and the reward is accrued at a higher rate.
13. Purple Zone: Any KPI with a value greater than the Ceiling (capped value) does not accrue a 
reward; could flag underestimation of the KPI.
CoB: Chance of Betterment
Spread: The tolerance of an Operational KPI around the Norm, based on definitions 5 to 8 
above.
 Figure A1-1: KPI Distribution – Illustration of definitions 3 to 13
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 36
SMART definition for a KPI: Simple, clear and understandable; Measurable; Achievable and Agreed 
between Stakeholders; Realistic – within the control of accountable Division and within the ambit of 
its spread; and Time-bound.
A2 International Models and Equations
Statham’s Family of Curves:
Here we illustrate the application of these curves to an Operational KPI with the intention of achieving 
the International Best Quartile (IBQ [target]) within or at the 5th Year.
Figure A2-1: Subset of Statham’s Family of Curves
14. Linear Strategy: This strategy is the neutral one, progressing at a constant rate towards the 
target (black dotted line).
15. Push Strategy: This is the riskiest strategy as, usually, the intended time window (in this case of 
5 years) is exceeded (cyan line). However other external factors may prevent the adoption of 
other curves.
16. Pull Strategy: This is the strategy (red line) usually adopted by BoD’s and ExCo’s to drive the 
business. Under the right conditions, the likelihood is that the target will be achieved earlier than 
expected (red dash-dot line).
17. Hybrid Strategy: Pull-Push (purple line), is a strategy that somewhere within the time window 
there is an expected (or planned) constraint that would invert the condition of the strategy. Other 
hybrid strategies are Pull-Pull, Push-Push and Push-Pull.
International Union of Producers and Distributors of Energy – UNIPEDE:
The implementation Operational KPIs is usually linked to the Country’s Energy Regulating Body’s
mandate of Setting of rules, guidelines and codes for the regulation of the Electrical Energy
industry. It is therefore important to utilize Operational KPIs that are Internationally accredited 
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 37
and can be benchmarked. Such indicators can be obtained from the UNIPEDE definitions and 
formula. The main relevant ones for this project are:
18. REG: Reference Energy Generation is the energy that could be produced if the unit were 
operated continuously at reference unit power under reference ambient conditions throughout 
the period. Reference ambient conditions are environmental conditions representative of the 
annual mean (or typical) ambient conditions for the unit.
𝑅𝐸𝐺 = 𝑁𝑒𝑡 𝑀𝑎𝑥𝑖𝑚𝑢𝑚 𝐶𝑎𝑝𝑎𝑐𝑖𝑡𝑦 (𝑀𝑊) . 𝑇
where T represents the total hours in the time-window under measurement
19. PCLF: Planned Capability Loss Factor is defined as the ratio of the planned energy losses during 
a given period of time, to the reference energy generation expressed as a percentage. Planned 
energy loss is energy that was not produced during the period because of planned shutdowns or 
load reductions due to causes under plant management control. Energy losses are considered to 
be planned if they are scheduled at least four weeks in advance. The following data is required to 
determine each unit's value for this indicator: Planned energy loss, expressed in units of megawatt￾hours (electric) and REG.
𝑃𝐶𝐿𝐹 =
𝑃𝑙𝑎𝑛𝑛𝑒𝑑 𝐸𝑛𝑒𝑟𝑔𝑦 𝐿𝑜𝑠𝑠 (𝑀𝑊ℎ)
𝑅𝐸𝐺 %
20. UCLF: Unplanned Capability Loss Factor is the percentage of maximum (or reference) energy 
generation that a plant is not capable of supplying to the electrical grid because of unplanned 
energy losses (such as unplanned shutdowns, outage extensions or load reductions). Energy losses 
are considered unplanned if they are not scheduled at least four weeks in advance. A low value 
for this indicator indicates that important plant equipment is reliably operated and well maintained.
The following data is required to determine each unit's value for this indicator: Unplanned energy 
loss, expressed in units of megawatt-hours (electric) and REG.
𝑈𝐶𝐿𝐹 =
𝑈𝑛𝑝𝑙𝑎𝑛𝑛𝑒𝑑 𝐸𝑛𝑒𝑟𝑔𝑦 𝐿𝑜𝑠𝑠 (𝑀𝑊ℎ)
𝑅𝐸𝐺 %
21. UCF: The Unit Capability Factor monitors the progress in attaining high unit and industry energy 
production availability. This indicator reflects effectiveness of plant programs and practices in 
maximizing available electrical generation, and provides an overall indication of how well plants 
are operated and maintained.
𝑈𝐶𝐹 = 100% − 𝑃𝐶𝐿𝐹 − 𝑈𝐶𝐿𝐹
22. OCLF: Other Capability Loss Factor is the percentage of energy generation that a plant is not 
capable of supplying to the electrical grid due to causes beyond the control of plant management 
including internal non-engineering constraints.
𝑂𝐶𝐿𝐹 =
𝑂𝑡ℎ𝑒𝑟 𝑈𝑛𝑝𝑙𝑎𝑛𝑛𝑒𝑑 𝐸𝑛𝑒𝑟𝑔𝑦 𝐿𝑜𝑠𝑠 (𝑀𝑊ℎ)
𝑅𝐸𝐺 %
23. EAF: Energy Availability Factor monitors the progress in attaining high unit and industry energy 
production availability after the losses due to causes beyond the control of plant management 
including internal non-engineering constraints have been deducted.
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 38
𝐸𝐴𝐹 = 𝑈𝐶𝐹 − 𝑂𝐶𝐿𝐹
𝐸𝐴𝐹 = 100% − 𝑃𝐶𝐿𝐹 − 𝑈𝐶𝐿𝐹 − 𝑂𝐶𝐿𝐹
24. LF or GLF: Load Factor or Generation Load Factor is the ratio of the energy utilized to the 
reference energy generation expressed as a percentage.
𝐿𝐹 =
𝐸𝑛𝑒𝑟𝑔𝑦 𝑈𝑡𝑖𝑙𝑖𝑧𝑒𝑑 𝑜𝑟 𝐿𝑜𝑎𝑑𝑖𝑛𝑔 (𝑀𝑊ℎ)
𝑅𝐸𝐺 %
25. EUF: Energy Utilization Factor is the ratio, expressed as a percentage, of the energy utilized (load) 
to the available energy after the losses due to causes beyond the control of plant management 
including internal non-engineering constraints have been deducted.
𝐸𝑈𝐹 =
𝐿𝐹
𝐸𝐴𝐹 %
For a Power Station (PS): the Operational KPIs (definitions 19 to 25) need to be aggregated as a weighted 
average of the generating units; where the weights are the ratio of Net Maximum Capacity of the 
generating unit, to the total Net Maximum Capacity of the PS. Similarly, the Utility Operational KPIs 
(definitions 19 to 25) are aggregated as a weighted average of the generating units of the Utility; where 
the weights are the ratio of Net Maximum Capacity of the generating unit, to the total Net Maximum 
Capacity of the Utility.
Weighting in terms of the Generating Units Net Maximum Capacity over the Total Net Maximum Capacity 
is used to compute the Power Station’s or the utility overall KPI as given in definitions 20 to 25 above, 
but checks and balanced must be performed to assure the values produced.
26. UAGS/7000: The purpose of this indicator is to enable monitoring of an important aspect of the 
reliability of service provided to the electrical grid. It considers the success in improving reliability 
by reducing the number of the generating unit trips. It also provides an indication of plant 
operation and maintenance performance. It is defined as corresponding to the number of 
unplanned (unintentional) automatic grid separations of internal origin, which occur per 7000 
operating hours.
"Unplanned" means that grid separation is not an anticipated part of a planned test, nor part of an 
operating program designed to adjust output to demand (e. g. a controlled shutdown). Controlled 
shutdowns of units where the line circuit breaker is manually or automatically opened at loads 
equal to or below the first automatic synchronizing load should not be considered when 
computing this indicator.
"Grid separation" means the opening of the generator breaker or HV yard breaker where no 
generator breaker exists. This could be an opening signal actuated by overshooting of a safety 
threshold, or a spurious trip. Grid separation can only occur during grid service of the units. Grid 
service is obtained when the start-up is successful for requested start-ups or when loads in excess 
of the first synchronizing load is reached for “contracted” or “other start-ups” (i.e. house or block 
load as programmed into the automatic synchronizing equipment). “Automatic” (unintentional) 
means that the grid separation is not the result of an action by the operator either on one of the 
switches to trigger a unit trip or grid separation, or to simulate operation of a protection system. 
(Note: Generation Utilities have used the word “manual” to distinguish trips initiated by the 
operator to protect the plant. However, trips caused by the operator in error e.g. opening the 
wrong switch leading to a trip are excluded from the “manual” category. Controlled shutdowns 
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 39
of units where the line circuit breaker is manually or automatically opened al loads equal to or 
below the first automatic synchronizing load should not be considered when computing this 
indicator).
“Operating" means that the generator set is connected to the off-site grid (transmission of 
generated power) even if the alternator is operating in synchronous motor mode due to 
exceptional circumstances.
"Of internal origin" means that the trip is due to an unspecified internal installation failure resulting 
in a loss of reliability - even if the initial event can be traced to an off-site cause. The signal, which 
triggered grid separation, must originate from one of the sensors (or protection logic) for 
monitoring unit parameters (generator set and power transmission, up to and including the 
generator transformer HV breaker, and boiler). Grid separations actuated by protection systems 
for the physical parameters of the grid are not included unless they were incorrectly controlled.
The selected figure of 7000 hours represents the typical number of on-line hours for most plants 
operating at base load or semi-base load. The indicator thus represents an approximate value of 
the actual number of grid separations occurring in one year.
The following data are required to determine the value for this indicator: the number of unplanned 
automatic grid separations (U) with the generator circuit breaker (or HV breaker where no 
generator breaker exists) in initially closed position and the number of operating hours (OPH).
It is expressed as the average number of ‘trips’ per unit per year (rate).
𝑈𝐴𝐺𝑆 𝑝𝑒𝑟 7000 =
𝑈
𝑂𝑃𝐻 . 7000 𝑟𝑎𝑡𝑒
For a Power Station (PS): the UAGS/7000 need to be calculated as follows: the number of unplanned 
automatic grid separations (Ut) and the number of operating hours (OPHt), need to be summed for 
all the PS generating units the PS UAGS/7000 = ∑(Ut.7000) / ∑(OPHt). The same would apply for the
Utility (no weighting applies).
27. Variance: Statistical Variance, that is the averaged (unbiased) squared measure of how values are 
spread from their expectation. Its squared root, i.e. returning to the values unit-of- measure, is 
the Standard Deviation. There is a fundamental mathematical reason why the square is used rather 
than the simple difference from the expectation: it can be proved that the average of this difference 
is always equal to zero. This is the reason why here we would work with Statistical Variance 
rather than the Accounting Variance (which is the arithmetic difference).
28. Performance Dashboard Examples
Figure A2-2 below shows an example for a dashboard Bar-Chart on how each Power Station is 
performing vis-à-vis the Unit Capability Factor for a particular time of the year (e.g. monthly or 
quarterly; at Financial Year-End it will be the final Contractual result for each of the Power 
Stations). It permits to benchmark the Power Stations within the utility. If monitored monthly, it 
provides a visual on how each Power Station is likely to fulfill its contractual obligations at Financial 
Year-End (sustainability)
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 40
Figure A2-2: UCF Illustration of Power Stations Performance
Dashboards provide high level user interfaces are typically used for executive level information [10]. The 
Dashboard will be designed, tested and implemented in the final task of this assistance- which will be a 
pilot for the utility M&E program. Each Division as well as Service Providers would have a similar one. In 
the final task, these will be upgraded as a direct mapping from the BSCs, in form of gauges / dials;.
The final Dashboard would be colour coded in the same type palette as a Likert Scale (defined in Section 
4) and have as components the following:
i. Strategic KPIs
ii. Operational KPIs
iii. Service Providers KPIs
An illustrative example is shown in Figure A2-3 below.
Figure A2-3: Dashboard illustration
Scale
…
PS1 PS2 PS3 … PSn
Unit Capability Factor
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 41
APPENDIX B: PM TOOLS
This Appendix provides guidance to the Evaluation Officer in the M&E Department in the application of 
the tools in the reviews.
B1 Linkage to Strategic Plan & Timeframe Assessment Tool:
ASSESSMENT Date: June 2018 By: SAEP
Key Focus Areas (KFAs) Consistency Omissions Duplications
“To achieve the Utility’s Vision, how should the utility appear to 
its Customers?” SP **
“To succeed financially, how should the utility appear to its 
Shareholders?” SP **
“To satisfy its Shareholders and Customers, what Business 
Processes must the utility excel at?” SP **
“To achieve the utility’s Vision, how will the utility sustain its 
ability to change and improve SP **
Strategic Objectives (SO) Consistency Omissions Duplications
“Have the Strategic Objectives been adequately described what 
they are attempting to achieve vis-à-vis the Key Focus Areas?” SP 
***
Key Tasks / Activities (KA) Consistency Omissions Duplications
“Are the Activities adequately aligned to the Strategic Objectives?” 
SP ***
Outputs (O/P) Consistency Omissions Duplications
“Are the Outputs adequately aligned to their corresponding 
Activities?” SP *
“Is the responsible Division/Department aligned to each Output?
SP *
“Are the Start and Completion times for each Output adequate?” 
Addresses the Time-Frame ***
the Evaluation Specialist checks for issues (detailed in Section 3.1, paragraph b. M&E Department) of:
o Consistency: e.g. data sequential in time (common error: Start-Completion times are not in sequence)
o Omissions: e.g. missing data
o Duplications: e.g. double-labelling
* M&E Department Accountability; ** Divisional Accountability; ***Both M&E Department and Divisional 
Accountability
Consistency, Omissions and Duplications: 
Likert Scale: 5- No findings; 4- 1 finding; 3- 2 findings; 2- 3 
findings; 1- > 3 findings 
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 42
B1-1 SP Validation – Specific Utility Application
REVIEW: when the Outputs were scrutinized, the following findings were observed in the SP.
Key Focus Area 1: Customer, Stakeholders and Corporate Image
a) p31 (C): Tedzani IV EPC contract in place Ongoing Jul 2017
DOPD
Is this correct? Was this task completed in 2017? If yes, the should all the completed projects be 
reflected in the SP? This would no longer make it a Strategic Plan. Is it a typo?
b) p32 (C): Wovwe Contract for consultancy in place Ongoing Dec 2017
DOPD
Is this correct? Was this task completed in 2017? If yes, the should all the completed projects be 
reflected in the SP? This would no longer make it a Strategic Plan. Is it a typo?
c) p34 (C): STAGE 1A (20MW) Leasehold to the site concluded Ongoing
Mar 2018 DOF
In subsequent tasks, when the “Leasehold to the site concluded” appears the Responsible Office is 
DOHRM. Is it correct to have this one for the DOF? 
d) p34 (D): STAGE 1A (20MW)
Financial closure Jan 2018 Jun 2018 DOF 
EPC contract in place Feb 2018 Aug 2018 DOPD
20 MW installed and commissioned Sep 2018 Dec 2019 DOPD
This Output seems a duplicate with the Output above it. Is this correct? 
e) p34 (O): 30 MW installed and commissioned Jul 2021 Dec 2022 Responsible 
Office
Responsible Office is missing. Can DOPD be assumed?
f) p36 (C): Wind – Stage 1 Financial closure May 2019 Dec 2019 DOPD
In these tasks, when the “Financial closure” appears the Responsible Office is DOF. Is it correct to 
have this one for the DOPD?
p36 (C): Wind – Stage 2 ditto
g) p36 (O): Wind – Stage 2 Carry out feasibility studies Sep 2018 Apr 2019
Responsible Office
Responsible Office is missing. Can DOPD be assumed?
Key Focus Area 3: Business Process Excellence
a) p46 (C): Involve key users in procurements Users involved in procurement 
Procurement 100% 100% DOF
Should this be consistent with the term “Ongoing” used in other tasks? Or is there another 
meaning to “100%”?
b) p49 (O or D): Power Stations & Liwonde Quarterly 1st, 2nd, 3rd & 4th DOO
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 43
What is the meaning of “Power Stations & Liwonde” as an “Output”. Is it a duplicate of 
the previous Output “Hold Safety Drills” or is there an omission of “Hold Safety Drills”?
c) p51 (C): Upgrade existing Power station access roads
Nkula Access road upgraded Jul 2018 Dec 2018 DOHRM
Tedzani Access road upgraded Jan 2019 Jun 2019 DOHRM
Kapichira Access road upgraded Jul 2019 Dec 2020 DOHRM
Wovwe Access road upgraded Jan 2021 Dec 2022 DOHRM
Is the DOHRM responsible for the upgrading of access roads? Would the DOO or DOPD be 
more appropriate?
FINDINGS on Outputs (KPIs): Consistency (C) = 12; Omissions (O) = 3; Duplications (D) = 4
FINDINGS on Strategic Objectives: In ‘purple: 1 and 2’ (C) = 2; Omissions (O) = 2; Duplications (D) = 0
B1-2 SP Assessment – Utility Application
Scored using Likert Scale across the DIVISIONS – Summarized in Figure B2-1 
ASSESSMENT Date: June 2018 By: SAEP
Key Focus Areas (KFAs) Consistency Omissions Duplications
“To achieve the Utility’s Vision, how should the Utility appear to 
its Customers?” Addresses the Linkage to SP (5+5+5)/3 = 5**
Consistent 
throughout: 5
No
omissions: 5
None: 5
“To succeed financially, how should the Utility appear to its 
Shareholders?” Addresses the Linkage to SP (5+5+5)/3 = 5**
Consistent 
throughout: 5
No
omissions: 5
None: 5
“To satisfy its Shareholders and Customers, what Business 
Processes must the Utility excel at?” Addresses the Linkage to SP 
(5+5+5)/3 = 5**
Consistent 
throughout: 5
No
omissions: 5
None: 5
“To achieve the Utility’s Vision, how will the Utility sustain its 
ability to change and improve Addresses the Linkage to SP 
(5+5+5)/3 = 5**
Consistent 
throughout: 5
No
omissions: 5
None: 5
Strategic Objectives (SO) Consistency Omissions Duplications
“Have the Strategic Objectives been adequately described what 
they are attempting to achieve vis-à-vis the Key Focus Areas?”
Addresses the Linkage to SP (3+4+5)/3 = 4***
2 findings: 3 2 findings: 4 None: 5
Key Tasks / Activities (KA) Consistency Omissions Duplications
“Are the Activities adequately aligned to the Strategic 
Objectives?” Addresses the Linkage to SP (5+5+5)/3 = 5***
Consistent 
throughout: 5
No
omissions: 5
None: 5
Outputs (O/P) Consistency Omissions Duplications
“Are the Outputs adequately aligned to their corresponding 
Activities?” Addresses the Linkage to SP (1+2+1)/3 = 1.3*
8 findings: 1 3 findings: 2 4 findings: 1
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 44
“Is the responsible Division/Department aligned to each Output?
Addresses the Linkage to SP (1+2+1)/3 = 1.3*
8 findings: 1 3 findings: 2 4 findings: 1
“Are the Start and Completion times for each Output 
adequate?” Addresses the Time-Frame (1+5+1)/3 = 2.3***
4 findings: 1 None: 5 4 findings: 1
* M&E Department Accountability; ** Divisional Accountability; ***Both M&E Department and Divisional 
Accountability
Consistency, Omissions and Duplications:
Likert Scale: 5- No findings; 4- 1 finding; 3- 2 findings; 2- 3 findings; 1- > 3 findings 
Linkage to SP
SCORE for Divisions: KFAs: (5x4)/4=5, SO: 4, KA: 5; Overall each Div. SCORE: (5+4+5)/3 = 4.7
SCORE for M&E Dept: SO: 4, KA: 5, O/P: (1.3x2)/2 = 1.3; Overall M&E SCORE: (4+5+1.3)/3 = 3.4
Time-Frame
Both, M&E Department and Div. have structured the deliverables with a workable Time-Frame in terms 
of PM which scales it between at “On its Way” and “Near To-be State” (score 3.5); but the Start and 
Completion times have issues with C and D and score 2.3; Overall Time-Frame SCORE: (3.5+2.3)/2 = 
2.9
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 45
B2 Metric, Process and Organizational Capability Assessment Tools:
B2-1 Metric: KPIs (Vertical & Horizontal)
𝐌𝐞𝐭𝐫𝐢𝐜 𝐀𝐬𝐬𝐞𝐬𝐬𝐦𝐞𝐧𝐭 =
𝐒−𝐒𝐜𝐨𝐫𝐞 + 𝐌−𝐒𝐜𝐨𝐫𝐞 + 𝐀−𝐒𝐜𝐨𝐫𝐞 + 𝐑−𝐒𝐜𝐨𝐫𝐞 + 𝐓−𝐒𝐜𝐨𝐫𝐞
5
Simplicity, clarity and comprehensive:
S-Score:
Measurability:
M-Score:
Attainability and Approved by Stakeholders:
A-Score:
Realistic – within the control of accountable Division and within the ambit of its spread:
R-Score:
Time-bound:
T-Score:
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 46
B2-2 Process: BSCs and Dashboards
𝐏𝐫𝐨𝐜𝐞𝐬𝐬 𝐀𝐬𝐬𝐞𝐬𝐬𝐦𝐞𝐧𝐭 =
𝐃−𝐒𝐜𝐨𝐫𝐞 + 𝐀−𝐒𝐜𝐨𝐫𝐞 + 𝐖−𝐒𝐜𝐨𝐫𝐞 + 𝐒−𝐒𝐜𝐨𝐫𝐞
4
Data Management (acquisition, collation, validation, storage and extraction, BSCs & 
Dashboards) D-Score:
Alignment to SP’s KFAs and Outputs per Division (Strategic KPIs mainly, but also 
Operational KPIs)
A-Score:
Weights are Balanced (sum up to 100%):
➢ For KFAs
➢ For KPIs (within & between [final weight: product of KPI within weight and relative KFA weight])
Weights Inertia: These are the minimum weights that KFAs or KPIs need to exercise on the BSC
➢ For KFAs: Each KFA weight must be greater than or equal to 8%
➢ For KPIs (final weight [between]) must be greater than or equal to 4%
➢ Subject to balancing constraint above (sum to 100%)
These two Weights criteria (Balanced and Inertia) are assessed equally in strength, meaning that the
failing of one is a failure for all and that the overall score is the smallest score in anyone of them.
W-Score:
Source of Evidence: Substantiation, in document form (incl. electronic), in place for each 
KPI assessment (per Division)
S-Score:
This process should be optimised based on the Metric Assessment and Data, Alignment, Weight and 
Source of Evidence assessments. This usually takes place in the form of KPIs Workshops
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 47
B2-3 Organizational Capability in terms of PM
This scoring system sets out the methodology for scoring the capabilities within 
the Divisions using the Likert Scale (1 to 5) and the criteria below.
All the Divisions require a POC to interface with the M&E Dept (refer to task 4 report)
M&E Department: as per points i. to iv. below
ICT Department: as per points i. to iv. below
Human Resources & Administration Division: as per point iii. below + Capability to interface PM & 
Individuals Performance Appraisals (Carving of Bonus Pools) 
*POC: Point of Contact
i. Capability Related Diagnostics (subjective & objective) C-Score: Average of below
Developed: 
o Skills ➔ Score: 
o Instruments ➔ Score: 
o Methods ➔ Score: 
o Procedures ➔ Score: 
ii. Alignment to Other Divisions (Points of Contact – POCs) A-Score: 
iii. Timing processes synchronization plan (includes Communication processes)
T-Score: 
iv. Storage of Data, Assessments, Models (CDS) S-Score: 
𝐎𝐫𝐠𝐚𝐧𝐢𝐳𝐚𝐭𝐢𝐨𝐧𝐚𝐥 𝐂𝐚𝐩𝐚𝐛𝐢𝐥𝐢𝐭𝐲 𝐀𝐬𝐬𝐞𝐬𝐬𝐦𝐞𝐧𝐭 =
𝐂 − 𝐒𝐜𝐨𝐫𝐞 + 𝐀 − 𝐒𝐜𝐨𝐫𝐞 + 𝐓 − 𝐒𝐜𝐨𝐫𝐞 + 𝐒 − 𝐒𝐜𝐨𝐫𝐞
4
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 48
B2-4 Metric, Process and Organizational Capability (i.t.o. PM) Assessment 
– Specific Utility Application
Scored using Likert Scale across the DIVISIONS and SERVICE PROVIDERS
METRIC
Simplicity, clarity and comprehensive S-Score: 1
Measurability M-Score: 1
Attainability and Approved by Stakeholders A-Score: 1
Realistic – within the control of accountable Division and within the ambit of its spread
R-Score: 1
Time-bound T-Score: 1
𝐌𝐞𝐭𝐫𝐢𝐜 𝐀𝐬𝐬𝐞𝐬𝐬𝐦𝐞𝐧𝐭 =
𝐒 − 𝐒𝐜𝐨𝐫𝐞 + 𝐌 − 𝐒𝐜𝐨𝐫𝐞 + 𝐀 − 𝐒𝐜𝐨𝐫𝐞 + 𝐑 − 𝐒𝐜𝐨𝐫𝐞 + 𝐓 − 𝐒𝐜𝐨𝐫𝐞
5
= (1+1+1+1+1)/5 = 1
PROCESS
Data Management (acquisition, collation, validation, storage and extraction, BSCs & 
Dashboards) D-Score: 2 (some manual acquisition from Gen. Units)
Alignment to SP’s KFAs and Outputs per Division (Strategic KPIs mainly, but also 
Operational KPIs) A-Score: 1
Weights Balancing & Inertia W-Score: 1
Source of Evidence: Substantiation, in document form (incl. electronic), in place for each 
KPI assessment (per Division) S-Score: 1
𝐏𝐫𝐨𝐜𝐞𝐬𝐬 𝐀𝐬𝐬𝐞𝐬𝐬𝐦𝐞𝐧𝐭 =
𝐃 − 𝐒𝐜𝐨𝐫𝐞 + 𝐀 − 𝐒𝐜𝐨𝐫𝐞 + 𝐖 − 𝐒𝐜𝐨𝐫𝐞 + 𝐒 − 𝐒𝐜𝐨𝐫𝐞
4
= (2+1+1+1)/4 = 1.25
ORGANIZATIONAL CAPABILITY (i.t.o. PM)
P&D Division: POC in M&E FDC ➔ Score: 3
M&E Department: Organizational Capability Assessment (see below) ➔ Score: 1.3
Finance Division: POC in M&E FDC ➔ Score: 3
ICT Department: Organizational Capability Assessment (see below) ➔ Score: 1.3
Operations Division: POC in M&E FDC ➔ Score: 3
Human Resources & Administration Division: POC in M&E FDC ➔ Score: 3, Timing processes 
synchronization plan (includes Communication processes) ➔ T-Score: 1, Individuals Performance 
Appraisals (Carving of Bonus Pools) ➔ Score: 1; Overall ➔ Score: 1.7
Capability Related Diagnostics (subjective & objective) C-Score: (2+1+1+1)/4 = 1.25
Developed: 
o Skills ➔ Score: 2
o Instruments ➔ Score: 1
o Methods ➔ Score: 1
o Procedures ➔ Score: 1
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 49
Alignment to Other Divisions (Points of Contact – POCs) A-Score: 2*
Timing processes synchronization plan (includes Communication processes) T-Score: 1
Storage of Data, Assessments, Models (CDS) S-Score: 1
𝐎𝐫𝐠𝐚𝐧𝐢𝐳𝐚𝐭𝐢𝐨𝐧𝐚𝐥 𝐂𝐚𝐩𝐚𝐛𝐢𝐥𝐢𝐭𝐲 𝐀𝐬𝐬𝐞𝐬𝐬𝐦𝐞𝐧𝐭 =
𝐂 − 𝐒𝐜𝐨𝐫𝐞 + 𝐀 − 𝐒𝐜𝐨𝐫𝐞 + 𝐓 − 𝐒𝐜𝐨𝐫𝐞 + 𝐒 − 𝐒𝐜𝐨𝐫𝐞
4
= (1.25+2+1+1)/4 = 1.3
*Members of M&E FDC but M&E Dept non-existent
SUMMARY HEAT MAP for Specific Utility
Figure B2-1: Heat Map 
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 50
B3 Strategic KPIs - M&E Assessment Tool:
Figure B3-1: M&E Assessment form/template for Strategic KPIs.
Template used by the M&E Department to monitor and evaluate a Division’s & Service Provider’s Strategic 
KPIs.
ASSESSOR: M&E
DATE:
Fin. Year:
Outputs Targeted NORM
< 25% of Norm 
is achieved
25% ≤ Norm 
achieved < 
75%
Norm is 
achieved 
(Completed 
by expected 
date)
Facilitation 
provided to 
functional areas 
to enable 
delivery against Norm
Delivery 
realisation 
enabled against SP objectives 
(tracking & 
progress 
verified and 
reported)
Source of Evidence: Documents, 
Reports,etc. reference that can substantiate 
assessment
KFA1: Customer, Stakeholders and Corporate Image
Strategic Objective 1: Meet customer expectations
Objective
Strategic Objective 2: Enhance corporate image and visibility
Objective
KFA2: Revenue Generation and Financial Stewardship
Strategic Objective 1: Improve Financial Performance
Objective
Strategic Objective 2: Achieve Cost Optimization
Objective
KFA 3: Business Process Excellence
Strategic Objective 1: Improve Operational Efficiency
Objective
Strategic Objective 2: Enhance quality of internal processes
Objective
Strategic Objective 3: Improve Technological Infrastructure
Objective
Key focus area 4: Learning, Growth and Innovation
Strategic Objective 1: Cultivate a High Performance Culture
Objective
DIVISION / SERVICE PROVIDER :
Assessment
STRATEGIC KPIs: INPUTS Highlighted in 'yellow'
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 51
B3-1 Strategic KPIs Utility Application: Operations Division Assessment (Example)
Figure B3-2: M&E Assessment for Utility Operations Div. Strategic KPIs.
Figure B3-3: M&E Assessment Formula for Strategic KPIs.
ASSESSOR: M&E
DATE: 15-Jul-19
Fin. Year: 2018 / 19
Outputs
Targeted NORM
< 25% of Norm 
is achieved
25% ≤ Norm 
achieved < 
75%
Norm is 
achieved 
(Completed 
by expected 
date)
Facilitation 
provided to 
functional areas 
to enable 
delivery against Norm
Delivery 
realisation 
enabled against SP objectives 
(tracking & 
progress 
verified and 
reported)
Source of Evidence: 
Documents, Reports,etc. 
reference that can 
substantiate the assessment
Value
KFA1: Customer, Stakeholders and Corporate Image
Strategic Objective 1: Meet customer expectations
Objective
Nkula trash diversion barrier constructed
Sep-18 15% 15% M&E Report OpsA1
Tedzani trash diversion barrier constructed
Jun-19 20% 107% M&E Report OpsA7
Disposal site constructed for Kapichira HPP Oct-18 100% 102% M&E Report OpsA2
Kapichira Power Plant fully rehabilitated Dec-18 100% 142% M&E Report OpsA3
Tedzani I & II Power Plants fully 
rehabilitated – Stage I Mar-19 70% 70% M&E Report OpsA6
2 dams on Rivirivi: Feasibility study Jun-19 5% 103% M&E Report OpsA8
2 dams on Lirangwe: Feasibility study Jun-19 0% 0% M&E Report OpsA9
2 dams on Lisungwi: Feasibility study Jun-19 30% 30% M&E Report OpsA10
2 dams on Mkulumadzi: Feasibility study Jul-19 100% 126% M&E Report OpsA11
KFA 3: Business Process Excellence
Strategic Objective 1: Improve Operational Efficiency
Objective
ePAC Implemented Dec-18 100% 142% M&E Report OpsA4
TQM systems in place Dec-18 50% 114% M&E Report OpsA5
Strategic Objective 3: Improve Technological Infrastructure
Objective
Acquired up-to-date software: generation 
planning tools e.g. plexos, digsilent 
(stability component)
Jun-19 100% 102% M&E Report OpsA12
Staff trained on generation planning e.g. 
plexos, digsilent Jun-19 5% 5% M&E Report OpsA13
2. Enhance power plant operations
STRATEGIC KPIs: INPUTS Highlighted in 'yellow'
DIVISION / SERVICE PROVIDER : Operations DIVISION
Assessment
2. Provide reliable power
2. Optimize management of assets
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 52
In this Assessment (Figure B3-2, Example as in 15 Jul 2019), the 1st Output (‘Nkula trash…’) was reportedly 
assessed in Sep 2018 and the evidence is in Report ‘M&E Report Ops A1’. It indicates that it performed 
less than 25% of expectation (i.e. 15%) was accomplished by Sep-18. 
For the 2nd Output (‘Tedzani trash…’), it was reportedly assessed in Jun 2019 and the evidence is in Report 
‘M&E Report Ops A7’. It indicates 20% of the Facilitation provided to functional areas to enable delivery 
against Norm, was accomplished. The Assessment Model computes the performance Value according to 
the formulæ shown in Figure B3-3, and results in 107%.
Conversion onto the Common Performance Scale (Likert Scale: Figure B3-4)
1st Output: 15% ➔ Likert Score: 1
2nd Output: 107% ➔ Likert Score: 3.21
Figure B3-4: Strategic KPIs Conversion Model onto a Likert Score
The Strategic KPIs Likert Scores (i.e. 1 for 1st Output, 3.21 for 2nd Output, etc.) are then 
entered in the Operations Division BSC.
This Methodology is consistently applied across all the Divisions for the Performance Scoring 
of the Strategic KPIs
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 53
B4 Operational KPIs - M&E Assessment Models:
Each and every Division in a Utility has different Operational KPIs and each KPI needs to be analyzed and 
Workshopped with the corresponding Division. Here a typical Operations (sometimes aka Generation) 
Division is used to provide guidance in the methodology for the Evaluation Function of an M&E 
Department
CAPABILITY
The preferred method is usually to measure the Capability of the Generating Units (or the UCF, see 
Appendix A definition-point 21 and corresponding formula); but since this indicator is a computed value 
derived from the maintenance (PCLF) and forced (UCLF) outages, the focus, as Operational KPIs should 
be on the PCLF and UCLF. 
The PCLF should measure the ability to have the maintenance plan under control and hence a measure of 
deviation from plan is appropriate. An accepted practice for the PCLF is an expected 2 percentage points 
deviation from plan, with a spread of 1 percentage point either side (i.e. a deviation of 1% or less from 
plan would be deemed as excellent and a deviation of 3% or more from plan would be unacceptable.
Figure B4-1: Operational KPI: PCLF Conversion Model onto a Likert Score
The UCLF should measure the ability to benchmark against peer Utilities. A Utility that has a fleet of 
Hydro-Reservoir and Gas-Turbines mix should have a UCLF of less than 3%. Using the Performance
Statistical Model given in Appendix A definition-points 4 to 8 (see also Figure A1-1) which provides the 
same chances of betterment for different Operational KPIs within and between Divisions, the following is 
obtained:
For the UCLF as a variable expressed as a percentage, a logit model may be used to arrive at required 
probabilities. 
For a value of a UCLF expected at 3%, the spread (as per Figure A1-1) would be: 
C = 1.0% with a 5% chance of being bettered
S = 1.9% with a 20% chance of being bettered
Norm = 3% as per expectation
K-I = 4.7% with an 80% chance of being bettered
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 54
F = 7.2% with a 95% chance of being bettered
The above values are entered in the Divisional BSC
The Corresponding Likert Score conversion model for the above values is illustrated in Figure B4-2.
Figure B4-2: Operational KPI: UCLF Conversion Model onto a Likert Score
The OCLF (see Appendix A definition-point 22) is deemed outside Management control (such as a 
drought). This Operational KPI does not affect the Capability of the Utility, but affects the Availability 
of the Energy dispatched and hence the Utility’s Revenue. If hedging (Risk Management) is plausible, 
this is an indicator that may be used for Energy Derivatives (Options).
RELIABILITY
The usual manner to measure the Reliability of the Generating Units (or the UAGS/7000, see Appendix 
A definition-point 26 and corresponding formulæ); this indicator provides a demonstration of the 
Generating Units tripping when they are capable (available) of supplying energy. It is essentially a “Trip 
Rate”; an accepted benchmark for a Utility that has a fleet of Hydro-Reservoir and Gas-Turbines mix, 
would be an expected value of 1 Trip/Generating Unit/Year.
By keeping the Generating Units’ Operating Hours constant (as per the computations of UCLF and PCLF), 
using Poisson Statistical models and the Performance Statistical Model given in Appendix A definition￾points 4 to 8 (see also Figure A1-1), the following is obtained:
For a value of UAGS/7000 expected at 1 Trip/Generating Unit/Year, the spread (as per Figure A1-1) would 
be: 
C = 0.43 with a 5% chance of being bettered
S = 0.78 with a 20% chance of being bettered
Norm = 1.01 as per expectation
K-I = 1.21 with an 80% chance of being bettered
F = 1.39 with a 95% chance of being bettered
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 55
The above values are entered in the Divisional BSC
The Corresponding Likert Score conversion model for the above values is illustrated in Figure B4-3.
Figure B4-3: Operational KPI: UAGS/7000 Conversion Model onto a Likert Score
The models given above (Figures C4-1 to C4-3) enable the BSCs to be automated within a Data Warehouse . 
Other Divisional Operational KPIs are modelled in similar fashion and then transferred to their corresponding 
BSCs
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 56
B5 Service Providers KPIs - M&E Assessment Tool:
Figure B5-1: Service Providers KPI: Overall BSC for a Service Provider
Figure B5-2: Service Providers KPI: Section of BSC for the Operations Division
B6 Utility KPIs – BALANCED SCORECARD (BSC)
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 57
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 58
APPENDIX C: M&E FUNDAMENTAL SKILLS & SOFTWARE
It is important to note that the Management of the M&E Department requires a good understanding of 
the data management, analytical tools and the presenting of information in an M&E environment 
application.
The minimal requirement is a formal four-year University degree in Statistics, Computer Science, or 
relevant discipline inclusive of registration with ICCSSA or RSS as a Certificated Statistician.
Preferably, a master’s degree in Statistics, Computer Science, or relevant discipline inclusive of registration 
with ICCSSA or RSS as a Chartered Statistician. Experience in the following sections of Statistical Sciences 
would be required in the Monitoring and Evaluation Functions.
o Data Formatting, Data Storage and Data Mining
o A good knowledge in EDA (Univariate and Multivariate)
o A clear understanding of the difference between Prediction (as in Regression) and Forecasting 
(as in Time-Series models) and the ability to model in these two fields
o Statistical modelling skills (distributional, linear, non-linear, time-series, transformational, etc.)
o Machine Learning
o Optimization and Simulation
Programming:
R-Coding (preferred)
Python-Coding
Data Management:
Data Handling and Querying: MySQL
Reporting: Spotfire
Analytical:
EDA, Distributional, Regression: Statistica
Forecasting: Forecast-Pro for Windows (Professional)
Optimization: Evolver, RISKOptimizer (Palisade)
Simulation: @Risk (Palisade)
Decision Analysis: DecisionTools Suite (Palisade) which includes Evolver, @ Risk, RISKOptimizer and 
is supplemented by PrecisionTree, TopRank, NeuralTools, StatTools and BigPicture
Reporting/Communicating:
Microsoft Office
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 59
APPENDIX D: SAMPLE FUNCTIONAL SPECIFICATION
TEMPLATE FOR PROCUREMENT OF SOFTWARE AND IT 
DEVELOPMENT
<UTILITY LOGO HERE>
FUNCTIONAL SPECIFICATION
SECTION 1: COLLABORATIVE BACKGROUND
[City], _________________ YYYY
The [Name of Counterpart/Organization], hereinafter referred to as the Developer, represented by the 
[Name and Official Title], on the one hand, and the Utility, represented by [Name and Official Title], on 
the other hand, hereinafter collectively referred to as the “Parties,” completed this Functional Specification, 
hereinafter referred to as the “FSpec”, deliberated during the meeting at Utility’s Offices in [City], dated 
_______2019, on the following:
1. Utility Performance Management Initiative BACKGROUND
One of Utility’s objectives is to improve the commercial viability of the utility so as to increase their 
capacity to operate as off-takers and increase access to electricity. In this regard, support to the 
Monitoring and Evaluation (M&E) Department in Utility and the development of a performance 
management (PM) capability within the Utility, as well as the monitoring and reporting the progress 
toward obtaining the objectives of the Utility’s Strategic Plan (SP), is required. Presently a Data 
Warehouse (DW) addressing the PM objectives is non-existent. 
2. Acknowledgement
2.1 Intent of the FSpec
The Parties specifically acknowledge that this FSpec expresses the present intentions and 
expectations of the Parties with respect to the principles and goals specified in the FSpec, and 
that this FSpec is not an obligation of funds, nor does it constitute a legally binding agreement 
by any Party.
In view of the above, this document defines the basic principles of the requirements as 
delineated in Section 2 below and that the Developer is stated as the official counterpart to the 
said requirements.
2.2 Confidentiality
The Parties specifically also acknowledge that all elements in this document and further 
submitted documents by each of the two Parties to the other are treated with the highest 
confidentiality, may be legally privileged and is intended solely for both Parties. Any use, 
disclosure or distribution of the contents of this document and/ any other submitted documents 
by each of the two Parties to the other is unauthorized and may be unlawful.
3. Overarching goal of the FSpec 
The Parties mutually agree that this FSpec expresses the elicitation for a Technical Specification, 
hereinafter referred to as the “TechSpec”, addressing the requirements in the FSpec. 
4. Process Management
The process is shown in Figure 1 of Section 2 below.
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 60
Upon completion of the meeting the Developer will submit the TechSpec and Quotation to Utility. 
Once mutual agreement is reached on the Quotation, the Utility will exercise control and delegate the 
overseeing for the implementation of the TechSpec to a suitable competently qualified supervisor.
The tasks addressing the FSpec requirements and detailed in the TechSpec may be revised, from time 
to time, by mutual Parties agreement during the period of implementation; these revisions are 
documented, submitted to the utility for approval and, if approved, included in the Appendix of the 
FSpec and TechSpec.
5. Liability Limitation
This is not a contract, but a FSpec. Either party can discontinue involvement in the activities as described 
herein by giving a curtesy notice. Any and all legally binding obligations shall be codified in separate 
documentation (legal contracts) pertaining the TechSpec and Quotation thereof. 
While both parties shall execute the activities in accordance with the highest professional standards, neither 
Party shall be held liable to the other for any loss, or damage arising, in connection with or relating in any 
way to this FSpec, including, but not limited to, indirect, special, incidental, consequential, punitive, or 
exemplary loss or damages. The FSpec signatories have no financial and material liabilities to each other.
On behalf of the Developer
______________________________________
On behalf of <UTILITY NAME HERE>
______________________________________
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 61
MAIN ELEMENTS
This FSpec is identical to what is also known as a Functional Requirement Document (FRD). As a primer 
the Developer is encouraged to get acquainted with utility M&E documents <insert document names here>.
This specification expresses the required basic features from a User perspective.
Functional Requirements
The FSpec addresses the M&E and Analytical Environment as well as the Performance Measurement 
Framework illustrated in Figure 2 in the context of PM. 
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 62
Figure 2: Performance Measurement Framework
The Data Warehouse (DW) is represented schematically at the center of the Framework (Figure 2). This 
schematic is further expanded in Annex A1 and provides a high-level outline of the components of the Data 
Warehouse with the corresponding activities and interactions of the M&E Department Staff. 
In Annex A1, the diagram shows that Raw Data enters the DW whereby it goes through a process of 
verification and validation and other modes of preparation (such as filtering, formatting, aggregating). 
Subsequent to that process, the clean and reliable Data is stored in the Consistent Data Set (CDS) Source 
(read-only: access restricted). The Analysts work with their respective Data Marts from the CDS Source. 
The information produced from the analytics has two outputs: either in the KPIs Layer or in the 
Results/Reporting Layer.
The final product needs to address the access and storage of Data and Information in terms of authorization.
The main users are the Divisions and Service Providers Points-of-Contact (POCs) as well as Members of 
staff in the M&E Department of the Utility.
Annex A2 shows the Data and Information flow in terms of inputs, processing and information outputs 
elements in the DW. Information queries could emerge at any of the elements in Annex A2.
Non-Functional Requirements
The performance of the DW, the security of the DW as well as its usability and maintainability are also in 
consideration.
Other Requirements
The following requirements will be addressed at the second meeting as per the Process Management route 
for the FSpec:
• User Requirements (Agile protocol expressed as “User Stories”)
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 63
• Product Requirements in terms of Data Connectors and Off-the-Shelf Software interfaces
• Results/Reports Templates format details
• System Requirements
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 64
ANNEXES
Annex 1. Figure A1 provides the cross-cutting responsibilities or the M&E Department Staff
Figure A1: M&E Functional Responsibilities in the Data Warehouse
MS: Monitoring Specialist; SMO: Senior Monitoring Officer ES: Evaluation Specialist; SEO: Senior Evaluation Officer
CM&E: Chief of Monitoring & Evaluation Department
Dash-Dot lines indicate responsibility cross-cuts
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 65
Annex 2. the figure below provides a schematic diagram of the data inputs into the Data Warehouse, the processing of the data, the storage into a 
Consistent Data Set (CDS) repository, the analytical platforms, the KPIs historical information and their forward-looking expectations. 
Figure A2: Data and Information Flow Diagram
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 66
ANNEX 3. DATA WAREHOUSE
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 67
ANNEX 4. M&E FUNCTIONAL RESPONSIBILITIES IN THE DATA WAREHOUSE
MS: Monitoring Specialist; SMO: Senior Monitoring Officer ES: Evaluation Specialist; SEO: Senior Evaluation Officer 
CM&E: Chief of Monitoring & Evaluation Department
Dash-Dot lines indicate responsibility cross-cuts
USAID SOUTHERN AFRICA ENERGY PROGRAM (SAEP) UTILITY M&E MANUAL | 68
REFERENCES
[8] R. S. Kaplan and D. P. Norton, "The Balanced Scorecard - Measures that Drive Performance," Harvard 
Business Review, vol. 70(1), pp. 71-79, 1992. 
[9] R. S. Kaplan, "Conceptual Foundations of the Balanced Scorecard," Harvard Business School Publication, 2010. 
[10] S. Negash and P. Gray, "Business Intelligence," in Handbook on Decision Support Systems 2, Berlin, Springer￾Verlag, 2008, pp. 175-193.
Additional References
V Micali, “Business Performance Measurement and Management”, Chap. 11, p255, Ed. V Charles and M Kumar, 
Cambridge Scholars Publishing, 2014
WEC, "World Energy Trilemma," World Energy Council, London, UK, 2017.
J. Fleiss, Statistical Methods for Rates and Proportions, John Wiley & Sons, 1981.