Power Distribution Program: Interim Performance Evaluation Report November 24, 2014 This publication was produced for review by the United States Agency for International Development by Michael Philips, Shahab Qureshi, Jacob Laden, Zameer Haider, Zaheer Ahmed Ather, and Anser Ali. It was prepared by Management Systems International (MSI) under the Monitoring and Evaluation Program (MEP). ACKNOWLEDGMENTS The authors would like to thank all those who facilitated the team’s work team and enabled us to complete this evaluation, including the following:  The implementing partner International Resources Group (IRG/Engility) facilitated the team’s fieldwork, coordinated with participants for surveys and provided a wealth of information to the evaluation team including interviews, discussions and relevant documents. We also appreciate the implementing partner’s feedback on the preliminary findings and conclusions prior to the drafting of this report.  The officials from the USAID/ Pakistan Energy Office provided input on the evaluation process and valuable coordination with the implementing partner. We also appreciate the input on the preliminary findings and conclusions prior to the drafting of this report.  The managers and supervisors from power distribution companies and the officials from National Electric Power Regulatory Authority (NEPRA) shared their knowledge and understanding of the program with the evaluation team.  The principals of girls’ colleges that hosted the Energy Conservation Campaigns for students and teachers for their valuable comments and thoughts on this activity. POWER DISTRIBUTION PROGRAM INTERIM PERFORMANCE EVALUATION REPORT November 24, 2014 Contracted under Order No. AID-391-C-13-00005 Monitoring and Evaluation Program DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. i CONTENTS Executive Summary................................................................................................................................................................................. 1 Program Summary .................................................................................................................................................................................. 7 Program Background .............................................................................................................................................................................. 9 Evaluation Purpose and Questions ...................................................................................................................................................12 Evaluation Methodology ......................................................................................................................................................................13 Findings and Conclusions by Activity and Question ....................................................................................................................17 Summary of Findings and Conclusions ............................................................................................................................................39 Summary of Recommendations .........................................................................................................................................................43 Annexes ...................................................................................................................................................................................................45 Annex 1: Evaluation Statement of Work ..............................................................................................................................46 Annex II: Bibliography ................................................................................................................................................................76 Annex III: Data Collection Instruments ................................................................................................................................77 Annex IV: List of Individuals Interviewed ........................................................................................................................... 127 Annex V: Detailed Results of Demand Side Management (DSM) Program .............................................................. 142 Annex VI: Detailed Results of Linemen Safety Training ................................................................................................. 150 Annex VII: Evaluation Team Bios ......................................................................................................................................... 162 Annex VIII: Disclosure of Conflict of Interest .................................................................................................................. 164 Tables and Figures Table 1: Program Summary .................................................................................................................................................................. 7 Table 2: Group Interviews Conducted at Each DISCO ..............................................................................................................14 Figure 1: Map of PDP Program Area .................................................................................................................................................. 8 Figure 2: Industrial Participation in Survey, by DISCO Territory.............................................................................................27 Figure 3: Industrial Participation in Survey, by Subsector ...........................................................................................................27 Figure 4: Major Causes of Linemen Accidents ..............................................................................................................................33 Figure 5: Is Cost of Electricity a Great Concern for Respondents? ..................................................................................... 142 Figure 6: Proportion of Respondents Subjected to Maximum Demand and Power Factor Charges .......................... 143 Figure 7: Main Reasons for Participation in EE Motors Program .......................................................................................... 144 Figure 8: Energy Saving Measures and Their Effectiveness, Used by Respondents ........................................................... 144 Figure 9: Top Reasons for Having Satisfaction with Equipment Installed Under Program ............................................. 145 Figure 10: Whether or Not the Installed Equipment Achieved Intended Results ............................................................ 146 Figure 11: Satisfaction with Various Attributes of Equipment ................................................................................................ 146 Figure 12: Future Intention Concerning the EE Equipment .................................................................................................... 147 Figure 13: Rating of Program by Its Attributes........................................................................................................................... 148 Figure 14: Respondents’ Recommendations about What to do for Widespread Adoption of EE Equipment ......... 149 Figure 15: Sample Composition by DISCO ................................................................................................................................. 150 Figure 16: Safety Programs Attended by Linesmen from Different DISCOs ..................................................................... 151 Figure 17: Most Recent Safety Programs Attended by Survey Respondents ..................................................................... 152 ii Figure 18: Accidents Experienced by Survey Participants by DISCO .................................................................................. 153 Figure 19: Type of Accident Experienced by Survey Participants by DISCO..................................................................... 154 Figure 20: Results of Accidents Experienced by Survey Participants by DISCO ............................................................... 155 Figure 21: Nature of Services Provided by Survey Participants when Accident Happened by DISCO ...................... 156 Figure 22: Type of Accidents Witnessed by Survey Participants by DISCO ...................................................................... 157 Figure 23: Results of Accidents Witnessed by Survey Participants by DISCO .................................................................. 158 Figure 24: All Accidents Experienced or Witnessed by Respondents ................................................................................. 158 Figure 25: Types of Accidents Experienced or Witnessed by Respondents ...................................................................... 159 Figure 26: Result of All Accidents Experienced or Witnessed by Respondents ............................................................... 160 Figure 27: Major Factors Contributing to Linesmen Accidents, by DISCO ....................................................................... 161 iii ACRONYMS AEB Area Electricity Board AMR Automatic Meter Reading BOD Board of Directors CIS Customer Information System COR Contracting Officer’s Representative COS Cost of Service COSs Cost-of-Service Study CTO Current Transformer Operated DISCO Government-Owned Power Distribution Company DO Development Objective DSM Demand Side Management EE Energy Efficiency / Energy Efficient ERP Enterprise Resource Planning FATA Federally Administered Tribal Areas FESCO Faisalabad Electric Supply Company GEPCO Gujranwala Electric Power Company GIS Geographic Information System GOP Government of Pakistan GPS Global Positioning System HESCO Hyderabad Electric Supply Company HHU Hand-Held Unit IESCO Islamabad Electric Supply Company IR Intermediate Result IRG International Resources Group IT Information Technology KESC Karachi Electric Supply Company LDI Load Data Improvement LESCO Lahore Electric Supply Company LOE Level of Effort M&E Monitoring and Evaluation MEP Monitoring and Evaluation Program MEPCO Multan Electric Power Company MSF Mission Strategic Framework MSI Management Systems International MW Megawatt MWP Ministry of Water and Power NEPRA National Electric Power Regulatory Authority OAPA Office of Afghanistan and Pakistan Affairs PDP USAID Power Distribution Program PEPCO Pakistan Electric Power Company PESCO Peshawar Electric Supply Company PMU Performance Management Unit PPE Personal Protective Equipment QESCO Quetta Electric Supply Company SEPCO Sukkur Electric Power Company SOW Statement of Work SRAP Special Representative for Afghanistan and Pakistan TPW Team Planning Workshop VSD Variable Speed Drive WAPDA Water and Power Development Authority (WAPDA) 1 EXECUTIVE SUMMARY BACKGROUND USAID started the Power Distribution Program (PDP) in 2010 with the intent of providing technical and managerial assistance to Pakistan’s 10 government-owned electricity distribution companies (DISCOs), as well as the National Electric Power Regulatory Authority (NEPRA) and the Ministry of Water and Power (MWP). The long-term objective of PDP is to help prepare the DISCOs for privatization. The nearer-term objective is to implement a set of activities that: reduce power losses, improve accuracy in meter reading and billing, enhance planning and engineering modernization, improve safety practices, link tariffs to the costs of providing services and improve overall DISCOs governance. Although USAID wanted to address all 52 project activities, due to time and resource constraints, they selected 17 activities across nine DISCOs that reflect the broad range of project activities.1 Three of the activities were two-part activities, so in effect, the evaluation addressed 14 activities. These were contained in the following seven components: 1. Commercial Performance: The evaluation reviewed 1.) Hand-Held Units, 2.) Hand-Held Units and Improved Meter Reading, 3.) Electronic Metering, 4.) Automatic Meter Reading and 5.) Customer Information System￾Phase II and III. 2. Communication and Outreach with Consumers: The evaluation focused on one set of activities, including DISCO’s Outreach Activities and Anti-theft Campaigns. 3. Technical Loss Reduction: The evaluation focused on 1.) Planning and Engineering Modernization (GIS Mapping, System Analysis, and Training), and 2. Demand Side Management (energy efficient industrial motor) program. 4. Financial Management: The evaluation included Enterprise Resource Planning (ERP) and implementation (documentation for ERP manual and ERP implementation). 5. Governance: The evaluation focused on 1.) Assistance to NEPRA (Multiple Activities) and 2.) Cost of Service Study– Phase II and Phase III. 6. Human Resources and Change Management: The evaluation examined 1.) Lineman Training, Tools, and Training Aids and 2.) The Utility Exchange Program. 7. Gender: The evaluation focused on 1.) Energy Conservation Campaigns in girls colleges2 and 2.) Gender Equity Training. EVALUATION QUESTIONS USAID identified five evaluation questions to assess the effectiveness and sustainability of the 17 activities at nine DISCOs, as well as the effect on gender equity and governance among public sector entities in the energy arena. The evaluation questions were the following:3 1 TESCO was recently added as a DISCO to PDP program; however, it was not involved long enough to be considered for the evaluation. 2 Colleges in Pakistan have students age 15-17, similar to high schools age groups in the U.S. 3 While questions 1, 2, and 4 were applicable to all activities, questions 3 and 5 applied only to those activities where governance and gender were relevant. 2 1. How has the project achieved its planned results to date? Explain the results and net effects of selected PDP activities, including any unintended (both positive and negative) consequences. 2. What are the prospects for sustainability of the results achieved thus far? Which results appear to be less sustainable (e.g., revenue increase activity)? 3. Did the project make any difference in the “governance” element of the power sector entities such as MWP, NEPRA, and the DISCOs? What are some of the accomplishments? What areas still need extensive work? 4. How valid are the current project design, development theory and framework? Identify additional approaches or activities recommended, if any, to achieve the program objectives. 5. How has the project performed in increasing women’s participation in the energy sector? EVALUATION METHODOLOGY PDP’s evaluation methodology focused on data collection and analysis from four sources of information to gain firsthand information from project beneficiaries, including PDP documents, group interviews, individual interviews and surveys. The process is summarized below and can be seen in detail on pages 6–9 of the full report:4 1. Reviewed PDP documents; 2. Conducted group interviews with DISCO staff responsible for implementing and overseeing programs under PDP; 3. Conducted group and individual interviews with staff at PDP, USAID, and NEPRA; and 4. Conducted one survey regarding lineman training and another regarding the Demand Side Management (energy efficient industrial motor) program. The team organized the data by the 17 activities and five evaluation questions. Through a process of triangulation, the team analyzed the data for findings and then formed conclusions that drew upon themes arising across multiple data sources and stakeholders. Recommendations were formed based on the conclusions. FINDINGS AND CONCLUSIONS The following findings, conclusions, and recommendations are organized by evaluation question and address all 17 PDP activities. Findings 1. Results of activities: The evaluation team found that all activities are ongoing at varying degrees of completion. Some are still at an early stage and their effectiveness cannot be fully determined. Training and technical assistance, the key features of many of the activities, have been effectively provided, according to respondents from DISCOs across all activities. Staff at all nine DISCOs are optimistic about the activities and said that GIS mapping and advanced metering techniques are resulting in improved accuracy of meter readings and billings, enabling consumers to better understand the bills. Across all DISCOs, the Cost of Service Study (COSs) have given DISCO staff a better understanding of the cost of serving each consumer class. This information is being used to prepare tariff petitions submitted to NEPRA. PDP has effectively provided technical assistance to NEPRA on the COSs so that NEPRA is able to make tariff determinations. 4 More on methodology, including the data collection process, qualitative and quantitative data analysis methods used and strengths and limitations can be found on pages 6–9 of the full report. 3 The linemen trainings have resulted in thousands being trained on improved safety practices. The lineman survey found that those who completed training believe it has improved awareness of safety practices, although there are questions about whether overall accidents and fatalities have declined and whether personal protective equipment is available and being used. Results for some activities are difficult to discern. These activities include Anti-Theft Campaigns, Gender Equity Trainings, and Energy Conservation Campaigns at girls’ colleges and the Utility Exchange Program. The Anti-Theft Campaigns are difficult to measure, especially in the short term. DISCO respondents indicated that campaigns need to be one element in a broader anti-theft effort. Further study is required to determine if the campaigns have affected energy theft. In the case of the Energy Conservation Campaigns at girls’ colleges, little evidence showed that energy savings or conservation practices resulted. Anecdotal information suggests that students are turning lights off, but no savings were measured or even estimated. While this was an activity for women’s colleges in the area of energy conservation, its effects in promoting women’s participation in the energy sector were unclear. PDP energy conservation presentations were well received, but did not yield many results. The Gender Equity Trainings were of high quality and the DISCOs deserve praise for hosting them; however, little evidence showed that they improved gender equity at the DISCOs. 2. Prospects for sustainability: It is impossible to predict with certainty which PDP activities will be sustainable. However, we used three main criteria to determine the likelihood that any activity would continue once PDP technical and financial assistance has ended. First, sustainability is likely if there is clear support for the activity from top DISCO management, as is the case with COSs. Second, sustainability is likely if the DISCOs have already expanded the activity on their own without PDP involvement, as is the case with lineman training. Third, sustainability is likely if the activity will not involve major capital outlays. The PDP activities that appear to have the greatest prospects for sustainability are the lineman training, Cost of Service Studies, and Enterprise Resource Planning. Some DISCOs have already integrated lineman training into their Regional Training Center curricula, making it more likely the trainings will continue without USAID support. Likewise, the DISCOs understand the Cost of Service Study well, the DISCO staff say senior management is supportive, and, except for some possibly inadequate staffing, the activity appears to be well-positioned to continue and to form the basis for tariff petitions into the future. Currently supplied Hand-Held Units (HHUs) are expected to require only minimal maintenance costs and therefore, may have sustained use. Some activities, in contrast, may require additional steps to ensure sustainability. Based on the successes of PDP activities and the lessons learned, some activities may be ideal for expansion, but any such expansion would require additional funds and management support. Activities with potential for expansion include: Automatic Meter Reading, Electronic Metering, GIS mapping, Demand Side Management (energy efficient industrial motor) program and the use of HHUs. Considering the support for GIS mapping among operating staff, prospects are strong for the activity’s institutionalization within the DISCOs. However, sustainability of the GIS mapping activity is dependent on support from top management and continued training, particularly during staff turnovers, which occur frequently. Some activities that do not require large investment (e.g., Energy Conservation Campaigns, Anti-Theft Campaigns and Gender Equity Training) likely will be unsustainable. The DSM (energy efficient industrial motor) program is not sustainable because the DISCOs did not administer it and no alternative institutional structure, such as a non-profit energy center or energy efficiency fund, was established to maintain it. Additionally, no apparent source of funding exists to continue providing subsidies of up to 50 percent for the new motors. Due to lack of funding, respondents said the utility exchanges are unlikely to continue without PDP support. 4 It should be noted that DISCO staff expressed uncertainty about whether all PDP activities would continue without funding and support.5 3. Effect on governance: Some of the activities hold promise for greatly improved DISCO governance. This is particularly true of ERP, COSs, CIS, and GIS mapping and advanced metering. In particular, COSs have significant potential to improve governance, as they are a method for a cost basis for establishing tariffs. However, it is too early in the implementation of these activities to discern the specific effects on governance. DISCO staff members responsible for managing and implementing PDP activities were generally uncertain about the effects of their activities on DISCO or power-sector governance. 4. Proposed modifications by respondents: PDP interventions have been largely technical, with engineering, financial and accounting support provided for the most part. While PDP staff explained that they seek management support for all PDP activities, many respondents at DISCOs urged modification of activities to gain greater support from upper management for some of these initiatives. The activities needing the most redesign are the Energy Conservation Campaigns, the DSM (energy efficient industrial motor) program and the Utility Exchange Program. 5. Gender equity: Evidence shows that the Gender Equity Trainings have resulted in enhanced awareness of gender equity issues among training participants. There are more facilities for women (washrooms, child care centers, etc.) and sexual harassment committees established at the DISCOs. However, only a fraction of the DISCO staff received the training and the evaluation team found no evidence that women’s participation in the power sector has increased, or that female recruitment and promotion had increased at the DISCOs. The effort may be too limited in scope to have a real and lasting effect on the environment at the DISCOs. RECOMMENDATIONS 1. HHUs, Improved Meter Reading, Electronic Metering, and Automatic Meter Readings (AMRs) – To improve the efficiency and reliability of the meter reading and other benefits, HHUs should be a temporary step in the transition to AMRs. AMRs would reduce human error in the reading process and automatically transmit the reading to a billing system. In the meantime, PDP should encourage DISCOs to expand HHU use and support the availability of these devices throughout the service territory. PDP should also advocate for PESCO management and NEPRA to provide financial support for installing Electronic Metering across the service territory, with the intention to eventually transition to AMRs. PESCO metering staff should be recruited by PDP to train metering staff at other DISCOs and get them started on pilots of their own. The evaluation team recommends that USAID consider broader use of HHUs as a preliminary step in the transition to AMR throughout the DISCOs’ service territories. HHU use should be considered for the longer term in locations where consumers have lower connected load levels. These initiatives should be included in future USAID programming. DISCOs and Pakistani government agencies should be asked to cost-share for greater sustainability.6 5 PDP’s 2014 fiscal year work plan states that sustainability of PDP interventions at the two “turnaround DISCOs” will be addressed through improvements in organizational structure at the DISCOs that will increase staff capacity. In addition, the costs associated with continuing the activities will be calculated and shared with the two DISCOs so that the costs can be incorporated into the DISCO budgets. 6 USAID will need to consult NEPRA and the DISCOs about what cost share percentage would be feasible for these activities. While 50 percent cost share may not yet be feasible, some will be necessary to ensure that there is GOP buy-in of the process and that the metering systems are maintained and sustained by DISCOs over the long term. 5 2. Customer Information System (CIS) – USAID should continue to support CIS development at PESCO and the Multan Electric Power Company (MEPCO), at least until it is operational and has a track record that can be evaluated. Assessments should be used to develop a plan to ensure there are resources and a process to support the system, including training staff in system maintenance, and training users – such as customer service staff – to ensure that the CIS is maintained long-term. As it reaches full implementation, it will be possible to see if the CIS ultimately enables DISCOs to better serve their customer base. In the meantime, PDP should ensure that the system is as user-friendly as possible for DISCO staff to fully learn and use. The roles and responsibilities for migrating data to the CIS need to be clarified and senior DISCO management should take greater ownership of the initiative. 3. Outreach Activities and Anti-Theft Campaigns – By design, Anti-Theft Campaigns are just one of several activities intended to reduce power theft. PDP should monitor and assess the effectiveness of the campaigns to determine the activity’s effectiveness in changing attitudes or behaviors toward theft over time. The Anti-Theft Campaigns should continue, but only as part of the overall, comprehensive anti-theft effort, which has been evaluated and found to be effective. USAID should develop a formal plan for ongoing coordination of anti-theft activities by providing direction for the last phase of the PDP and ongoing programming. 4. GIS mapping – During the final year of the project, PDP should encourage DISCO management to hire GIS unit staff and appropriately compensate them to retain them. In addition, PDP should use GIS quality-assurance best practices, through which mapped data can be verified by supervisors. USAID and PDP should support a forum/network among GIS staff across the DISCOs to share knowledge and experience. Ongoing training and refresher courses on GIS mapping for existing and new GIS staff are also needed. Moreover, USAID and PDP should obtain support from NEPRA to ensure middle and senior management at the DISCOs are aware of the importance of this initiative, will be engaged in the GIS mapping activity and will allocate sufficient budget and staffing to it. 5. Demand Side Management (energy efficient industrial motor) program – USAID should continue the work to improve the energy efficiency of industrial motors. However, the activity should be implemented through the DISCOs and function like a Demand Side Management (DSM) program designed to reduce electricity demand during peak periods. The continuation of the program should include a financing and incentive strategy that can continue implementation without indefinite reliance on government subsidies. The continued program should also include adequate measurement and verification of savings. USAID and PDP should work with NEPRA to reinforce, acquire or incentivize utilities to engage in DSM and increase DISCO support to improve energy efficiency practices. 6. ERP implementation – A plan or roadmap for continuing ERP implementation after PDP technical support ends is critical; PDP should generate this action plan as soon as possible in the final phase of the program. The plan should include a clear definition of how costs associated with ERP implementation will be shared by USAID and the DISCOs. It should also include a strong counterpart arrangement for ERP vendor oversight. Before the project’s completion, PDP should strengthen DISCO staff project management capabilities to enable adequate oversight of implementation and vendor performance. 7. Cost of Service Study – PDP should ensure that DISCO staff are trained on the COS software, specifically how to modify and adapt it for ongoing tariff-setting and other purposes. PDP should establish a plan and budget for the DISCOs to ensure that adequate and fully trained staff are available to continue the COS studies. PDP should also consider developing a COS user’s manual to properly guide DISCOs on how to maintain and use this analysis for ongoing tariff-setting. COS information could also be integrated into ERP design. 8. Assistance to NEPRA – USAID should provide ongoing assistance to NEPRA to pursue options for setting economical tariffs that consider the social needs of consumers. PDP should particularly continue assistance in its determination of cost-based tariffs. NEPRA staff should receive training to build their capacity for cost monitoring and cost-based tariff determination. This will better ensure that NEPRA continues to update and use the COS model in the future. 9. Lineman training – PDP staff should conduct a thorough analysis of serious accidents and fatalities before and after training activities to see if there is any reduction in accidents and fatalities among trained linemen and determine why accidents among linemen persist. USAID and PDP should support ongoing follow-on training in collaboration with the DISCOs. PDP should work with human resources departments and unions at DISCOs to 6 ensure compliance and enforcement of safety equipment use, procedures and practices. It should also work with DISCOs to establish procurement rules pertaining to Personal Protective Equipment (PPE) so that only high-quality safety equipment is purchased.7 Safety equipment use should be established in DISCO policy and enforced by supervisors and DISCO leadership. 10. Utility Exchange Program – In the final phase of the program and in future programming, exchange visits should be held with utilities with similar socioeconomic conditions as Pakistani DISCOs or those that have emerged from similar conditions to build modern energy distribution systems. There should be procedures for participants to present ideas from these exchanges to management staff and other leadership at their home DISCOs. USAID should also consider establishing a domestic personnel exchange program among Pakistani DISCOs to increase sharing of ideas and establish ongoing dialogue on best practices. The domestic and international exchanges should be part of a larger capacity-building effort to help DISCOs identify and apply lessons. Also, exchange programs should include DISCO leadership and senior managers, as they have more power to implement changes. 11. Energy Conservation Campaigns – This activity should be framed around achieving energy savings. As such, the campaign should include energy audits, audit review, financial incentives, financing, post-installation inspections and overall support to the colleges throughout the project process. The Energy Conservation Campaigns should be structured as DSM activities with DISCOs’ direct involvement. With USAID support, future programming should advocate that provincial education departments include topics related to energy conservation in the provincial curriculum to make energy conservation awareness a more central tenet of all students’ education. USAID Energy Office should consult with USAID Education to identify strategies for this crosscutting effort. 12. Gender Equity Training – Improving participation and the working environment for women in the energy sector involves long-term cultural changes, so PDP’s gender equity intervention should be a longer-term and more comprehensive priority for the program and future USAID gender equity activities in the DISCOs. As the training currently reaches only a small percentage of DISCO staff, PDP should expand training and provision of materials to reach more staff. Training should reach senior management with a focus on the benefits of gender equity to the organizations’ productivity. The activity should also include training for trainers (as found in the training for line work), so they can better replicate the training and lessons learned beyond the life of PDP and future programming. These efforts would help create champions within the DISCOs to potentially mobilize a critical mass of people who can effect change within the DISCOs. Follow-on activities and training should be included in this activity’s design to reinforce learning and long-term change in the DISCOs’ workplace environments. Staff and consultants hired to design and manage such interventions should be experts on gender equity in the workplace. Furthermore, these initiatives should be coordinated with USAID’s Gender Equity Program, which may be better equipped to support these activities in the DISCOs over the long term. 7 “Personal Protection Equipment” refers to the helmets, protective gloves, belts, boots and other equipment to help protect linemen from injury and death. 7 PROGRAM SUMMARY The Power Distribution Project (PDP) of the United States Agency for International Development (USAID) works with Pakistan’s 10 government-owned electricity distribution companies to improve their technical, financial and managerial performance. It also works with the National Electric Power Regulatory Authority (NEPRA8 ) to improve the agency’s regulatory performance. TABLE 1: PROGRAM SUMMARY 8 PDP also supports the Ministry of Water and Power, however, these activities are not included in the evaluation. Title / Field Program Information Contract Number EPP-I-13-03-00006 Contract Officer’s Representative (COR) Nadeem Habib Start Date September 17, 2010 Completion Date September 16, 2015 Location Lahore, Faisalabad, Gujranwala, Islamabad, Multan, Hyderabad, Peshawar, Sukkur, and Quetta Name of Implementing Partners (IPs) International Resources Group (IRG)/Engility USAID Objective Addressed Development Objective: Increased Sustainable Energy Supplied to the Economy Budget $230 million 8 FIGURE 1: MAP OF PDP PROGRAM AREA 9 PROGRAM BACKGROUND Pakistan’s electric power sector is in a state of crisis, negatively affecting economic growth, employment and social stability. Meanwhile, energy demand has grown about 6 percent over the past six years, putting additional strain on an already overloaded electric system.9 Energy shortages range from 6,000–7,000 megawatts (MW), a third of peak demand for electricity, and up to 1,500 million cubic feet per day of natural gas.10 The resulting energy shortages cause rolling blackouts known as “load shedding,” which can last up to 12 hours a day for some consumers. The shortages also affect employment, exports and private-sector growth. The sector’s technical and financial constraints prevent electric utilities from covering costs. By one estimate, the power sector is losing about $175 million each month.11 As a result, massive energy subsidies (which amount to 2 percent of GDP, according to the World Bank) are covering both operating and most capital costs.12 Recent electricity price increases have fueled inflation and the cost of everyday goods and services. Political interests and weak rule of law in the sector have resulted in difficulty implementing much-needed reforms. PROBLEM OR OPPORTUNITY ADDRESSED Pakistan’s power sector is in a state of transition from being owned and managed by the government to fully autonomy, where companies independently purchase, generate, transmit, dispatch and distribute electricity. The energy sector has been government-run, and the Water and Power Development Authority (WAPDA) Power Wing provided the line and functional control of the Distribution Department, directing the operation of eight area electricity boards (AEBs) across the country. Power sector reforms in 1998 led to the restructuring of WAPDA and the conversion of AEBs into stock electric power distribution companies referred to as DISCOs, with the Government of Pakistan (GOP) holding all shares. Regulatory agencies, like the National Electric Power Regulatory Authority (NEPRA) and the Pakistan Electric Power Company (PEPCO), were charged with supervising the DISCOs’ transition to full autonomy. Over the past 16 years, although reform measures have made these companies more independent, only one DISCO has been privatized. The other 10 remain government-owned and have not been in a condition to be transferred to private ownership.13 In response to conditions in the power sector, USAID established programs to address the inefficiencies in the institutions responsible for power generation, transmission and distribution. To address issues in distribution, USAID designed the Power Distribution Program (PDP). In September 2010, USAID/Pakistan awarded a three￾year, $60 million task order to Washington, D.C.-based International Resources Group (IRG, now known as “Engility”) as the Implementing Partner (IP) for PDP. A task order amendment later increased the budget to $230 million and the performance period to five years. The program is scheduled to end in 2015, although USAID expects to continue some PDP activities under new programs. FOCUS AREAS AND GROUPS PDP works with Pakistan’s 10 government-owned DISCOs in Lahore, Faisalabad, Gujranwala, Islamabad, Multan, Hyderabad, Peshawar, Sukkur and Quetta to improve their operational and financial performance. The program, which also works with the Ministry of Water and Power and the National Electric Power Regulatory Authority 9 Engr Hussain Ahmad Siddiqui. “Rising electricity load-shedding,” Dawn.com, Oct. 20, 2014. http://www.dawn.com/news/1139012. 10 Rachid Bennmessaoud, Uzma Basim, Anthony Cholst, and José R. López-Calix. Pakistan – The Transformative Path, World Bank, Pakistan, October 2014. 11 Ahmad Fraz Kahn. “Disastrous situation in power sector,” Dawn.com, Aug. 1, 2014. http://www.dawn.com/news/1122459 12 Rachid Bennmessaoud, Uzma Basim, Anthony Cholst, and José R. López-Calix. Pakistan – The Transformative Path, World Bank, Pakistan, October 2014. 13 Statement of Work, Interim Performance Evaluation of the Power Distribution Program, June 2014. 10 (NEPRA), focuses on reducing losses, increasing revenues and improving customer service so that the companies can perform at levels commensurate with well-run utility companies around the world. INTENDED RESULTS The Mission Strategic Framework (MSF) Development Objective 1 is “Increased Sustainable Energy Supplied to the Economy.” The MSF Intermediate Results (IR) to which PDP contributes are: IR 1.1: Increased Energy Supply 1.1.2: Improved Efficiency of Consumption and the Distribution Systems 1.1.3: Increased Financial Sustainability of Power Supply 1.1.4: Increased Non-U.S. Government Investment in the Energy Sector IR 1.2: Improved Energy Sector Governance 1.2.1: Improved Policy Implementation 1.2.2: More Autonomous Energy Sector Entities 1.2.3: Improved Capacity of USAID-Supported Energy Public Sector Entities PDP APPROACH AND IMPLEMENTATION PDP consists of seven main components and 52 activities14 . Although USAID wanted to evaluate all of them, this was not possible due to time and resource constraints. Therefore, this evaluation focuses on 17 activities selected by USAID across the seven component areas due to their importance to future programming. Three of the activities are parts of the same activity, so in effect, the evaluation addresses 14 activities from the seven components at nine out of 10 DISCOs. Particular emphasis is given to the two “turnaround” DISCOs, PESCO and MEPCO, which received substantial investments as compared to other DISCOs. The specific activities addressed in this evaluation, organized by component, include: 1. Commercial Performance: Activities implemented under this component are intended to introduce new and improved technologies, practices and procedures to reduce commercial losses, especially at the meter and meter-reading level. The evaluation focuses on the following activities: 1.) Hand-Held Units, II.) Hand-Held Units and Improved Meter Reading, III.) Electronic Metering, IV.) Automatic Meter Reading and V.) Customer Information System– Phase II and III. 2. Communication and Outreach with Consumers: This component aims to improve the DISCOs’ “brand awareness” and increase consumers’ awareness of ethical electricity use, on-time bill payment and the DISCOs’ roles as private distribution service providers. The evaluation focuses on DISCOs’ Outreach Activities and Anti-Theft Campaigns. 3. Technical Loss Reduction: Technical losses are caused by line losses, which result from energy flows through medium- and low-voltage distribution systems, including conductor, transformer and other electric power distribution system components. The evaluation focuses on two activities: I.) Planning and Engineering Modernization (GIS mapping, system analysis and training) and II.) Demand Side Management (energy efficient industrial motor) program. 14 Due to time and resource limitations, the evaluation could not cover all 52 PDP activities. Therefore, USAID selected the 17 major activities to be addressed. 11 4. Financial Management: To ensure higher collection rates, PDP provides guidance and training on financial management and encourages DISCOs to use collection agencies to gather private sector consumer receivables. PDP also focuses on the collection of accurate and timely data to monitor decision-making progress and achieve financial planning and business results with greater accuracy, accountability, transparency and reliability. The evaluation focuses on Enterprise Resource Planning (ERP) implementation (development of ERP manual and training in nine DISCOs). 5. Governance: One of MWP’s cornerstones in the power-sector reform process was redesigning the DISCOs’ governance structure to improve policy implementation and decision-making. This has involved changes in leadership and improvement in organizational policies and work processes. The evaluation focuses on the following activities: 1.) Assistance to NEPRA (Multiple Activities), and 2.) Cost of Service Study – Phase II and III. 6. Human Resources and Change Management: Training and capacity-building of DISCO staff aim to improve management, efficiency, innovation and staff knowledge and awareness. This component is designed to elevate their business performance standards and ultimately improve the services they provide to consumers. PDP also emphasizes employee safety by incorporating safety measures into human resource (HR) management in addition to the current practice of quantifying fatalities for staff and the public, including “lost-time” accidents. In an effort to reduce fatal and non-fatal accidents, PDP has launched the Quick Effect Safety Training Program for two DISCOs’ linemen and is in the process of replicating this training in the remaining DISCOs. The evaluation examines the following activities: 1.) Lineman Training, Tools, and Training Aids and 2.) The Utility Exchange Program. 7. Gender: The program emphasizes women’s participation in the energy sector by offering Energy Conservation Campaigns at girls’ colleges and a training component devoted to gender equity in the workplace. The evaluation focuses on both these activities. 12 EVALUATION PURPOSE AND QUESTIONS The USAID/Pakistan Mission commissioned Management Systems International’s (MSI) Monitoring and Evaluation Program (MEP) to systematically assess the effectiveness of PDP’s various interventions to improve the performance of the nine DISCOs and the governance and policy environment of public sector entities in the energy arena.15 This evaluation assesses the effectiveness of the program approach and its components with the aim of identifying opportunities for improvement. Through evidence gathered on the project’s seven major components, particularly the 17 activities mentioned, the evaluation assesses the program’s effect on electricity distribution performance and governance to assist USAID, the Special Representative for Afghanistan and Pakistan (SRAP), the GOP and the USAID/Washington Office of Afghanistan and Pakistan Affairs (OAPA) to understand the outcomes achieved as well as areas for improvement in the sampled activities. The evaluation addresses the following questions, which have been articulated and explained in the evaluation statement of work (SOW): 1. How has the project achieved its planned results to date (estimated level of effort at 50 percent)? Explain the results and net effects of selected PDP activities, including any unintended (both positive and negative) consequences. 2. What are the prospects for sustainability of the results achieved thus far? Which results appear to be less sustainable (e.g., revenue increase activity)? 3. Did the project make any difference in the “governance” element of the power sector entities such as MWP, NEPRA, and the DISCOs? What are some of the accomplishments? What areas still need extensive work? 4. How valid is the current project design, development theory and framework? Identify any additional approaches or activities recommended to achieve the program objectives. 5. How has the project performed in increasing women’s participation in the energy sector? 15 Statement of Work, Interim Performance Evaluation of the Power Distribution Program, June 2014. 13 EVALUATION METHODOLOGY EVALUATION METHODOLOGY The evaluation team applied a mixed-methods approach, using both quantitative and qualitative techniques to collect data from multiple sources in a variety of locations to ensure multiple levels of triangulation. PDP evaluation methodology consisted of data collection and analysis based on four sources of information: 1. Review of PDP documents; 2. Group interviews with DISCO staff responsible for implementing and overseeing the implementation of programs under PDP; 3. Interviews with staff at PDP, USAID, and NEPRA; and 4. Two surveys conducted by a subcontractor: one regarding lineman safety training and the other regarding the DSM (energy efficient industrial motor) program. DATA COLLECTION The evaluation began with a desk review of project documents including quarterly reports; an annual report; the Office of the Inspector General audit report; various documents pertaining to individual PDP activities, technical assessments, and studies; and policy and procedure manuals generated by PDP for the DISCOs. The team conducted fieldwork over a three-week period. This included seven group interviews with key PDP implementers at IRG and one group interview with officials from the USAID/Pakistan Energy Office. The team also conducted six group interviews with 10 principals from girls’ colleges and 51 group interviews with 132 individuals from nine DISCOs, including Islamabad Electric Supply Company (IESCO), Multan Electric Power Company (MEPCO), Hyderabad Electric Supply Company (HESCO), Peshawar Electric Supply Company (PESCO), Lahore Electric Supply Company (LESCO), Faisalabad Electric Supply Company (FESCO), Gujranwala Electric Power Company (GEPCO), Sukkur Electric Power Company (SEPCO) and Quetta Electric Supply Company (QESCO). The team interviewed various project stakeholders including managers and operational staff in each of the nine DISCOs, as well as key officials at the National Electric Power Regulatory Authority (NEPRA). The evaluation team used a semi-structured guide on each PDP activity included in the evaluation and wrote detailed summary notes for each interview. The entire evaluation team traveled to conduct the interviews in Peshawar and Multan. They then split into two teams, one with three team members and the other with two team members, to conduct the fieldwork in all other cities and DISCOs to expedite the data collection process. Table 2 shows which activities the team examined at each of the DISCOs. 14 TABLE 2: GROUP INTERVIEWS CONDUCTED AT EACH DISCO PDP Activity DISCO FESCO GEPCO HESCO IESCO LESCO MEPCO PESCO QESCO SEPCO Total Hand-Held Units and Improved Meter Reading 1 1 1 3 Electronic Metering 1 1 AMR Metering 1 1 2 Customer Information System 1 1 2 Outreach and Anti-Theft Campaigns 1 1 2 Planning and Engineering Modernization (GIS Mapping, System Analysis, and Training) 1 1 1 1 1 1 1 1 8 ERP Implementation 1 1 2 Cost-of-Service Study 1 1 1 1 1 1 1 1 1 9 Lineman Training 1 1 1 1 1 1 1 1 1 9 Utility Exchange Program 1 1 1 1 1 1 1 1 1 9 Gender Equity Training 1 1 1 1 4 Total Group Interviews 4 4 6 5 4 10 9 5 4 51 15 The team started by conducting interviews with PDP implementing partner IRG and with USAID’s energy office. After completing the DISCO interviews, the team followed up by interviewing IRG senior staff to discuss a broad range of the activities. This allowed for further data triangulation since the evaluation team could compare IRG’s statements with those of DISCO staff. The team also conducted two surveys to obtain in-depth data on the lineman safety training and the DSM (energy efficient industrial motor) program. The evaluation team prepared the structured survey questionnaires with assistance from MEP staff. A subcontractor fielded the surveys. The surveys included face-to-face interviews of linemen in five DISCOs with relevant safety trainings in Peshawar, Multan, Hyderabad, Gujranwala and Lahore. The lineman survey was developed to gather feedback on safety and best practices in line work (drawing from a sample of 1,075 potential participants); 333 linemen completed the survey, exceeding a 95 percent confidence level with a 5 percent margin of error. For the Demand Side Management (energy efficient industrial motor) program survey, the subcontractor conducted phone interviews with representatives of participating firms. The survey drew on a sample of 156 industry representatives across 52 industry locations in eight DISCO service territories. Using a snowball sampling method, the team reached 48 industry representatives from the 52 firms that participated in the project to gauge their views on their new motors and the motor program. DATA ANALYSIS The first step in the data analysis process entailed identifying common themes that emerged from the various interviews that addressed each of the evaluation questions. To maximize investigator triangulation, each of the two sub-teams reviewed and coded the other team’s notes on a tally sheet organized by themes that emerged from the interviews. The tally sheet allowed them to discern trends and dominant responses among the respondents. It also facilitated triangulating interview results across various stakeholder groups and DISCOs. In this report, the qualitative findings reported are those with which a majority of DISCO respondents generally agreed, with two exceptions. For some activities, only one or two DISCOs were interviewed. (For example, only MEPCO and PESCO were engaged for some activities.) In such cases, the findings are reported when the DISCOs were in agreement or, in some cases, when respondents at just one stated something the evaluators found significant. These single-perspective findings, of which there are few, are clearly stated as such in the text. The second exception to the approach of reporting only majority views is when respondents made statements that were not in direct response to a question. Some of these were, in the judgment of evaluators, significant enough to be reported. The minority views expressed in such cases reflect not disagreement, but the fact that other respondents were not asked about these topics. For the industrial motors and lineman training sections, the analysis relied more heavily on the quantitative results of the two surveys. In each, qualitative data is triangulated with quantitative findings when possible. METHODOLOGICAL STRENGTHS AND LIMITATIONS The greatest strength of this evaluation methodology is its use of multiple levels of triangulation across information sources and researchers, ensuring the validity and reliability of findings and conclusions. Specifically, the data were gathered from multiple information sources with varying perspectives, including USAID, the implementing partner and key personnel within each of the nine DISCOs. These data were comparatively analyzed and triangulated against the document reviews and survey results. This approach is critical to mitigating potential selection bias inherent in a sampling approach associated with the qualitative data collection methods. The main methodological limitation is that the interview data were largely self-reported based on the individuals’ personal experience and perspective. Much of it lacked substantiation through secondary documentation verifying respondents’ statements. For example, if a DISCO interviewee stated that the Anti-Theft Campaign was reducing power theft or that the Gender Equity Training changed how women are treated at the DISCO, no secondary data 16 was available to substantiate these assertions. To address this, the team sought out and used any available secondary sources against which to triangulate interview data. Additionally, although the evaluation’s main focus was on program activity implementers and practitioners, it would have been useful to interview more top management and the CEOs at the DISCOs. While that was not a requirement in the SOW, it would have facilitated better understanding of some of the governance issues and the degree to which PDP activities are shaping the DISCOs’ overall response to Pakistan’s energy crisis. Finally, with respect to gender, the evaluation team consisted solely of men, charged with evaluating an activity to improve the status of women within the DISCOs under two of the 17 activities evaluated. While men are capable of assessing gender equity, there would optimally be at least one woman on the team to lead interviews with women to ensure respondents’ comfort in sharing their views on gender issues. Furthermore, the men selected for the evaluation were not gender experts. Generally, MEP strives to achieve gender balance on its evaluation teams. However, the low number of available female energy sector experts did not make this possible for this evaluation. 17 FINDINGS AND CONCLUSIONS BY ACTIVITY AND QUESTION The findings, conclusions and recommendations are organized by each of the 17 activities relevant to the evaluation. For each activity, the findings are reported for each relevant evaluation question as follows:16 1. Results of Activities 2. Prospects for Sustainability 3. Effect on Governance 4. Modifications Proposed by Respondents 5. Gender Equity A. COMMERCIAL PERFORMANCE I. & II. HAND-HELD UNITS/IMPROVED METER READING The evaluation team combined two activities: a) use of HHUs and b) the use of HHUs and IMR. These activities were both used to improve the DISCOs’ meter-reading process. The HHU activity was initially designed to distribute the HHU technology in Phase II. The IMR activity supported HHU use and improved the accuracy of the metering process in Phase III. The findings, conclusions and recommendations outlined below are for both HHUs and IMR activities. The team interviewed respondents from three DISCOs that were directly involved in managing these activities. Findings 1. Results of Activities: Evaluation results revealed that the HHUs and IMR activities worked well at the three participating DISCOs. All respondents from the groups interviewed at three DISCOs reported that the HHUs are resulting in more accurate meter readings and fewer incorrect billings. Generally, customers appreciated the improved accuracy and transparency in the meter-reading process. Specifically, respondents indicated that these improvements resulted in a number of other positive effects, including reduced undercounting of electricity usage by improving meter reader and customer interaction; reduced customer complaints/disputes; improvement to the utilities’ public image; and increased revenue. Further, respondents from all three DISCOs mentioned that HHU use has resulted in greater meter reading efficiency as reflected in reduced paper work and time savings. 2. Prospects for Sustainability: The staff interviewed at all three DISCOs were enthusiastic about the success of the HHU pilot and hoped that it would be expanded to and replicated in other subdivisions. PDP-trained meter readers have trained other meter readers on HHU use. Respondents from two groups reported that they are starting to expand HHU use to other subdivisions, independent of PDP support. However, they also mentioned that DISCOs lack financial resources to 16 Not all evaluation questions are relevant to each of the 17 sampled activities. Therefore, findings are reported for pertinent evaluation questions only. 18 purchase more HHUs. Moreover, there is no plan or budget for maintenance or replacement of broken HHUs. Respondents from two groups mentioned that they would like PDP to financially support expansion of HHU use throughout the DISCOs’ service territories. 3. Effect on Governance: The interviewed groups at two DISCOs said that HHU use would improve DISCO governance. All three groups agreed that meter reading transparency was improved (or would be improving) and that there is improved monitoring of meter readers’ performance. 4. Modifications Proposed by Respondents: The respondents from two of the three participating DISCOs said that HHU use should be regarded as an intermediate step toward improved efficiency and productivity of meter reading. The ultimate goal, respondents suggested, should be a transition to Automatic Meter Reading (AMR), which allows for off-site readings and reduces the need for physical visits to sites. Conclusions The activity has strengthened participating DISCOs’ meter-reading capability and appears to be sustainable within the subdivisions in which it was initially rolled out, with expansion to some additional subdivisions. However, the use of HHUs across the three DISCO service territories, as well as those of other DISCOs, will require additional financial resources that do not appear to be available. While respondents expressed enthusiasm for the expansion of HHUs, they did not indicate that there was a high likelihood that the DISCOs would be able to provide matching funds or finance this initiative themselves. The effects of more transparent meter-reading methods, reduced dependence on meter readers for accurate readings and reduced billing disputes include an improved public perception of the DISCOs. The more accurate and transparent information that improved metering produces can enable management and DISCO leadership to make more informed decisions. Recommendations As suggested by respondents, to improve the efficiency and reliability of meter reading, the use of HHUs should be a temporary step in the transition to AMRs, which can read meters remotely and do not require meter readers. The success of this HHU pilot may provide useful information to advocate for such financial support to top management. AMRs would reduce human error in the reading process and automatically transmit the readings to the billing system. However, AMRs are expensive, as they require replacing current metering infrastructure with new electronic meters and communication systems, and may not be affordable for many DISCOs, even on a pilot basis. Considerations for PDP: In the near-term, PDP should increase its assistance to DISCOs to expand HHU use as a preliminary step and support the availability of these devices throughout the DISCOs’ service territories. In particular, HHUs should be considered for consumers who have a connected load of less than 5 kilowatts (KW) as HHUs are more cost-effective for these load levels. In all — or nearly all — of the DISCOs, the largest numbers of consumers fall in this category. At the same time, AMRs would be best suited for loads of 20 KW or more. PDP should also help the DISCOs develop plans for how to address HHU repair and replacement. Considerations for USAID: The expanded use of HHUs is an appropriate activity for further USAID programming. However, DISCOs and Pakistani government agencies should also be encouraged to provide matching funds to reinforce buy-in for maintaining the devices and to ensure greater sustainability. USAID should encourage NEPRA to allow DISCOs to include the costs of AMRs in the setting of tariffs. As stated, HHUs are more appropriate in the long term for consumers with lower load levels. 19 III. ELECTRONIC METERING Findings Electronic Metering is a basic requirement of modern utility operations. Pakistani DISCOs mainly use electromagnetic meters, which are inefficient, relatively easy to tamper with and generate limited data. Under PDP, USAID initiated a pilot activity to install electronic meters in one PESCO subdivision to monitor the performance of the new meters. 1. Results of Activities: Only one DISCO is participating in this activity, so available evaluation data were limited to relevant staff from PESCO. So far, out of a projected 100,000 to be installed by the time PDP ends, 14,500 electronic meters have been installed. The activity has resulted in more accurate meter readings and an improved billing process, according to the four respondents in one group interview. They further report that as a result, DISCO revenue has increased, but no substantiating data was available. A feature of electronic meters is that they can record a customer’s maximum demand, which has helped the DISCO understand load requirements of individual customers. Respondents indicated that Electronic Metering had helped in streamlining the planning process and load augmentation. 2. Prospects for Sustainability: Meter procurement requires financial resources and PESCO lacks the funds to buy the thousands of units necessary to cover its service territory. Additional training is needed to increase the number of staff with the skills to install the meters. Respondents mentioned several additional challenges to sustainability, including buy-in from management, resource allocation and consumer resistance to installation in areas with limited rule of law. One respondent said, “Sustainability is not an impossible target, [but it] requires substantive efforts both from the management and the staff. The commitment and allocation of resources for purchase of these meters on a large scale is required.” Another said, “Sustainability is a grave concern, particularly keeping in mind the poor law-and￾order situation in the areas where staff cannot go to install these meters and there is a lot of resistance from consumers.” 3. Effect on Governance: Use of electronic meters improves the transparency and accuracy of billing, which increases DISCOs’ revenues. It also improves public regard for the DISCO, which can enhance overall DISCO governance. Conclusions Electronic Metering has provided PESCO a chance to switch to a modern, accurate metering system. The electronic meters help prevent electricity theft and improve accuracy in readings and billings. However, given the high price of the new meters and replacement process, the sustainability of this program is unclear. While the decision to use Electronic Metering has yielded some improvements, it is a step short of the more accurate AMR meters. Recommendations Consideration for PDP: PDP should advocate for PESCO management and NEPRA to provide financial support for installing Electronic Metering across the service territory, with the goal of eventually transitioning to AMRs. PDP should start by installing Electronic Metering in PESCO’s immediate service territories in proximity to Peshawar so their full implementation can be a model for expanding Electronic Metering to other parts of KP. In time this could serve as a best practice example for other DISCOs for Electronic Metering implementation across Pakistan. PDP should recruit PESCO metering staff to train at other DISCOs to enable them to start their own pilots. 20 IV. AUTOMATIC METER READING (AMR) PDP’s AMR activity involved AMR metering activity in the residential, commercial , agricultural and industrial sectors (Component 2) as well as upgrading the standard electrostatic meters at the facilities of larger energy users (> 20 KW) within the MEPCO and HESCO service territories (Component 3). The conversion to AMR will automate the meter reading process, improve reading accuracy, reduce meter tampering and allow two-way communication between the meters and DISCOs. For example, meters can send electricity usage recorded in short intervals and supply quality information to the DISCO, which can remotely turn off electricity supply to meet load management targets. Findings 1. Results of Activities: Respondents from two groups interviewed at MEPCO and HESCO anticipated benefits from AMR meter use. These included reduction in electricity theft, improved meter reading accuracy, improved billing process and improved demand management. For example, the two-way communication feature of AMR will enable DISCOs to switch off electricity to large tube well users during peak periods, and cut off the subsidized load coming from the DISCO to reduce rolling blackouts. At the time of this evaluation, both HESCO and MEPCO had ordered that AMR cells be established to manage the AMR activity. AMR devices were in the process of being installed at the time of the evaluation, as a result the automation capability of this technology was not being fully used; meters are still read manually at the meter location. The contractor had not yet completed delivery and implementation of the software for the activity and, as a result, AMR analysis reports were not yet being generated automatically as intended. 2. Prospects for Sustainability: Respondents at both DISCOs expect AMR meter installation to continue after PDP ends, but only if sufficient financial resources exist, and they have doubts about whether such resources will be available. The DISCO staff mentioned there is support from senior management for the AMR program, although the cost of re-metering will be high.17 Also, opposition may come from customers, particularly tube well owners, who do not want the utilities to be able to cut off their electricity. 4. Modifications Proposed by Respondents: According to two respondents interviewed from HESCO on AMR, these meters should be installed in tandem with Current Transformer Operated (CTO) meters (commonly known as check meters) on transformers. DISCOs are already implementing another program through which CTO AMR meters are being installed on transformers. The team recommends a linkage between these two activities: GPS/GPRS/AMR on the consumer premises and CTO/AMR on feeders and transformers. Conclusions The installation of AMR meters will likely have a range of benefits, as identified by the respondents. AMR meters would reduce theft, reduce human error in the reading process and automatically transmit the reading to billing system. However, compared with standard electrostatic meters, AMR meters are expensive. While senior management is reportedly supportive of AMR metering, this activity’s continuation without PDP support will hinge on the availability of funds. 17 In response to this finding, USAID has stated that PDP will also make sure that the opex cost for AMRs is made part of HESCO’s tariff petition for FY 2016. 21 Recommendations USAID and PDP will need to work with senior management at the DISCOs on a plan to fund/finance the AMR program across the DISCOs’ service territories, using the pilot projects in MEPCO and HESCO as examples. Support from NEPRA should be sought for including AMR meters and program costs in the tariffs. A detailed cost– benefit assessment must be conducted to justify the investment and should identify financing options for senior management to consider. As recommended under HHUs, the transition to AMR metering should be gradual. HHUs are more appropriate than AMR in locations with lower load levels. Considerations for PDP: The installation of AMR meters at the consumer end should coincide with the installation of AMR metering on feeders and transformers. Although feeders and transformers are being outfitted with meters under a separate activity, the synchronization of the two activities could yield immediate benefits in terms of operational efficiency. Considerations for USAID: USAID should ensure that the DISCOs educate consumers about the AMR meters. If there is a need to switch power off during peak periods, consumer participation should be voluntary. Consumers should be incentivized to participate, possibly through billing credits, which would require NEPRA approval. V. & VI. CUSTOMER INFORMATION SYSTEM (CIS) – PHASES II & III State-owned DISCOs in Pakistan use legacy computer systems operating on the COBOL language, which has become obsolete elsewhere in the world, to maintain customer-billing data. The inherent limitations of the system mean that customer-billing data is managed at each customer service center and cannot be accessed easily when a customer raises a dispute or query. A customer of one service center cannot be serviced at another center, and customers wait hours to have DISCO staff access their data to resolve any dispute or issue. PDP is, therefore, in the process of developing a modern Customer Information System (CIS) for the two turnaround DISCOs, PESCO and MEPCO. The legacy data is being migrated from old databases into more commonly used systems. The CIS is being designed as one unified customer database across each DISCO so that customer data can be accessed from any location within the DISCO service territory. Findings 1. Results of Activities: The respondents involved in the development and implementation of the CIS at both PESCO and MEPCO were optimistic about the system’s full rollout, anticipating that it will decrease billing disputes, enhance customer service and improve overall administrative efficiency. At PESCO, the “As-Is” analysis, which identifies gaps and opportunities in customer service, is near completion. When completed, data will be migrated and end users will be trained to use the system. At MEPCO, the CIS has now been developed and is being piloted in parallel to the old system. The MEPCO CIS is expected to go live by the end of December 2014. However, although both DISCOs are in the process of migrating from an old COBOL-based system to the new Oracle-based CIS, neither DISCO’s CIS has yet achieved outcomes that can be appraised and evaluated. Although both DISCOs appear to be making headway on CIS implementation, full assessment of the activity’s effectiveness is not possible at this time. The remote disconnect feature is supposed to be part of Component 3 of the AMR program, yet we were informed by one DISCO that AMRs installed on agricultural tube wells were being fitted with remote disconnect features. This was being done, according to the DISCO, without informing the tube well owners. 2. Prospects for Sustainability: Although the CIS is not yet operational at either PESCO or MEPCO, respondents from both DISCOs mentioned that there will be a need to ensure that adequate staff capacity and information technology (IT) infrastructure, both hardware and software, are in place and operating correctly. The respondents at PESCO mentioned that the roles 22 and responsibilities for migrating data from the legacy system to the new CIS need to be clarified and the management should take greater ownership of the initiative. These measures were reported as necessary steps to ensure the CIS will be sustained. One indicator of CIS sustainability is that in 2013, with a pilot CIS set to deploy at the sub-division level, MEPCO requested PDP to expand CIS deployment to the full circle level. Such support from MEPCO management indicates management understands the value of CIS and will likely continue to support its deployment after PDP closes out. 3. Effect on Governance: The CIS is intended to enhance governance at the DISCOs by improving customer relations. As the CIS is not yet operational, its effect on governance cannot be determined. However, staff at both DISCOs anticipate a positive effect because of the promise to improve administrative efficiency and customer service due to fewer billing disputes and timely complaint resolution. 4. Modifications Proposed by Respondents: Staff at both DISCOs mentioned that the IT infrastructure is inadequate and suggested that such hardware and software be made available earlier in the start-up of the activity. The staff at one DISCO suggested that further management support would be needed to fully operationalize the system. 5. Gender Equity: The two groups of respondents interviewed at MEPCO and PESCO said the new CIS has a great potential for female participation. One of the two groups of respondents interviewed at MEPCO mentioned that a number of women are working in their IT department and they will be involved in using and managing the CIS in future. Conclusions The relevant staff at both DISCOs were optimistic about the implementation and use of the CIS and its prospects for success. As the CIS is yet to be implemented, it is too soon to conclude that it will achieve its promised results. Similarly, keeping in view the staff’s concerns relating to the lack of IT infrastructure and necessary senior management support, it is too early to judge whether the CIS activity will be sustainable. Recommendations USAID should continue supporting CIS development at PESCO and MEPCO until it is fully implemented and should establish performance indicators so that the systems can be more fully evaluated. USAID should support the development of a plan to ensure that adequate financial and human resources are available and that a process exists to support the system, including staff training in system maintenance and training for users (such as customer service staff). This will help to ensure that the CIS is maintained over the long-term. As it reaches full implementation, it will be possible to see if the CIS ultimately enables the DISCOs to better serve their customers. Considerations for PDP: The roles and responsibilities for migrating data to the CIS should be clarified and senior DISCO management should be encouraged to take greater ownership of the initiative. As the system is optimized, its interface should be simplified and made as user-friendly as possible. In some countries, the complexity of CIS systems has been a concern. As the CIS programs are being rolled out, PDP should ensure that user friendly off the shelf products are provided so to that they are easy to use. Ensuring that there is a framework of resources and support for users to support the system over the longer term is of critical importance before PDP close out in the final phase of the program. 23 B. CONSUMER COMMUNICATION & OUTREACH VII. OUTREACH ACTIVITIES AND ANTI-THEFT CAMPAIGNS Power theft has long been an issue for all DISCOs, but its severity is much higher for some (HESCO, PESCO, QESCO, SEPCO and MEPCO) than others.18 The poor law-and-order situation in certain areas of the country prevents DISCO staff from taking punitive measures to discourage power theft. As such, using anti-theft messaging appears to be one viable option to discourage and reduce power theft. The Anti-Theft Campaigns consist of media messaging to consumers about electricity theft. Messages that reinforce social and religious values against theft and outlining legal punishment of theft were circulated to deter actual and potential offenders. The messaging was distributed via various modalities, including radio, TV, newspapers and posters, as well as through presentations at schools. Findings 1. Results of Activities: According to five respondents in two group interviews at PESCO and MEPCO, PDP has provided guidance on the campaign and its materials, but the DISCOs themselves created the educational and media materials. The staff at one of the two DISCOs stated that the campaign is having an effect on theft. One PESCO manager stated that, “Over 2 percent line losses have been reduced, for which Anti-Theft Campaigns may have contributed.” Unfortunately, no data are available to verify these claims. Others stated that any decreases in theft are due to a combination of factors, including increased theft detection, physical removal of illegal connections and increases in fines for stealing, in addition to the campaigns. The other DISCO’s staff said its public image improved as a result of the messaging. 2. Prospects for Sustainability: Staff at both DISCOs said they did not think the campaigns would continue after PDP ends, due to lack of financial resources. Respondents indicated that these activities will not be sustained without financial and technical assistance from USAID or other donors. Some managers reported a sense of entitlement to energy as a free resource, which is a challenge to the sustainability of these campaigns. With little capacity for enforcement, deterrence becomes difficult. As one DISCO manager from PESCO described, “To be honest, we cannot stop theft permanently, as after some time, people will start stealing it again. The theft is a continuous process and the thieves have a habit of stealing…campaigns do not work on these people”. Conclusions The anti-theft campaigns are up against some strong cultural attitudes and political opinions. The sense of entitlement to electricity as a public good and the number of rural poor who have illegal connections because they cannot afford legal ones is a challenge to enforcing payment for electricity. Although staff at one of the two DISCOs said the campaigns have had some success in reducing theft, it is difficult to substantiate this. There is little or no evidence that the effectiveness of the Anti-Theft Campaigns is being monitored at this time, although it is understandable that measurable results would not be expected in the short term. 18 The State of Industry Report 2013, p. 17, NEPRA. While other DISCOs experienced combined losses of about 10 percent, HESCO and PESCO experienced losses of about 35 percent, and SEPCO around 40 percent. QESCO’s losses are officially around 16–17 percent (likely more severe than this reported percentage because most of the theft is in tube well consumption, which is not metered or included in this figure). 24 The campaigns seem to be conveying the right messages, although the effect on the attitudes and behaviors of electricity thieves and would-be thieves is unknown. Due to the sensitivity of the subject of theft, it is difficult to measure in affected communities directly. At the same time, it is impossible to discern the overall effectiveness of Anti-Theft Campaigns without this information. Recommendations Considerations for PDP: Data collected over a longer term are needed to assess whether these campaigns are having any effect. Performance indicators should be developed and monitored quarterly to assess the effectiveness of the Anti-Theft Campaigns on changing attitudes and behaviors. Without this information, evaluators were not able to determine the campaign’s effectiveness in reducing theft. Assessments should first determine if the anti￾theft campaign is successful and it should guide the development of a formal plan for ongoing coordination of anti￾theft activities to provide direction for the last phase of the PDP and ongoing programming. Considerations for USAID: By design, Anti-Theft Campaigns are just one of several activities intended to reduce power theft. The Anti-Theft Campaigns should continue, but only as part of the overall, comprehensive anti-theft effort, which has been evaluated and found to be effective. While other anti-theft activities do exist, the campaign staff seemed unaware of them except in a very general sense – so better integration of the various anti-theft activities is probably warranted. Future programming will require other resources and activities to deter illegal activities and law enforcement to identify perpetrators and reinforce payments, as well as technologies to detect and prevent theft (e.g., use of aerial bundled cables). Furthermore, institutional support and resources from DISCO management for the campaigns will sustain these and other anti-theft activities. C. TECHNICAL LOSS REDUCTION VIII. PLANNING AND ENGINEERING MODERNIZATION (GEOGRAPHIC INFORMATION SYSTEM (GIS) MAPPING, SYSTEM ANALYSIS AND TRAINING) GIS mapping provides a utility with a holistic picture of large parts of its network, including the ability to zoom in and out at desired resolutions to see details and distances in a given region. GIS mapping, combined with load-flow analysis, provides a better picture of how electricity is flowing through distribution lines to avoid overburdening certain lines and to improve overall system efficiency. The GIS mapping and load-flow analysis provide an electrical engineering platform that helps model, optimize and balance the electrical load, reduce overloading feeder lines and perform load flow and voltage drop analysis. Under this activity, PDP trained DISCO staff to conduct GIS mapping and load-flow analysis and to use the Global Positioning System (GPS) instrument and modern load-flow analysis software. Findings 1. Results of Activities: Staff at all eight DISCOs with GIS implementation said they have received training on GIS mapping and conducting load flow analysis and are in the process of transitioning their utilities away from the “inefficient legacy drawing sheet system” to GIS. They also said GIS mapping is useful for visualizing large parts of the distribution system. Moreover, staff at six of the DISCOs said they expect technical oversight of the distribution system to improve. Two DISCOs specifically mentioned that using load-flow analysis will reduce feeder overloading, thereby reducing line losses and outages. Staff training was an important topic in the interviews. GIS staff at seven of the eight DISCOs also suggested holding a forum among GIS staff at all DISCOs to share knowledge and experience on mapping techniques and software usage. Staff at five DISCOs expressed the need for ongoing training and refresher courses for staff. Although they represented a minority of the DISCOs, GIS staff at three of the DISCOs said they were not trained on all aspects of the SynerGee software for load-flow analysis and three said real-world, on-the-job training should have been provided, not just classroom instruction. GIS staff at seven of the eight DISCOs indicated that training 25 could be enhanced through a forum among all DISCOs’ GIS staffs to share knowledge and experience on mapping techniques and software usage. At three of the eight DISCOs, staff expressed concern that management was not sufficiently supportive or even aware of the GIS mapping initiative. While this concern was expressed by fewer than half of respondent groups, operational staff’s general hesitance to criticize management makes any such mention significant. At one DISCO, a middle-management supervisor of the GIS operation joined the evaluation team’s interview in progress and was clearly unfamiliar with the GIS initiative. At another, the chief engineer was somewhat familiar with GIS, but unaware of the components and progress of the GIS activity. For GIS mapping and load-flow analysis to be effective, physical changes made to the distribution system must be incorporated into the GIS. Staff at four of the eight DISCOs expressed the need for a systematic approach or protocol for linemen and other DISCO field staff who make changes to communicate them to GIS staff. 2. Prospects for Sustainability: While relevant staff at most of the participating DISCOs believed the entire system would be mapped even without PDP, this would require substantial additional financial resources. A strong point in favor of sustainability, as mentioned by staff at one DISCO, is that distribution network digitalization is a regulatory requirement with which DISCOs will need to comply. Prospects therefore exist for continuation of this activity in the future. However, frequent transfers of trained GIS staff to non-GIS positions within the DISCOs is hampering the activity at three DISCOs and is undermining its sustainability, according to three of the eight groups interviewed.19 For example, at one DISCO where two engineers had been trained on the GIS software, one engineer was subsequently transferred to an unrelated department. Three DISCOs mentioned a shortage of trained linemen for GIS mapping, which could undermine full implementation as well as the sustainability of the activity. 4. Modifications Proposed by Respondents: Highlighting the lack of training and preparation for the use of GIS mapping, five of the eight groups suggested that GIS staff receive training, software orientations and follow-on training. For future training programs, staff at four of the eight DISCOs agreed that actual data should be used instead of mock data. Four out of eight groups of GIS users reported the lack of a mechanism for incorporating changes made on the mapped feeders. GIS staff of three DISCOs said more advanced software training is required. Conclusions As a result of this activity, DISCOs are close to being able to correctly measure the length and location of feeders and account for feeder inventory. Better visualization of assets and the overall distribution systems will follow when the GIS is fully operational. The GIS promises to reduce human error inherent in the manual system, improving the accuracy and reliability of system data. Although the GIS is in its pilot phase, DISCO staff are confident that it will enable them to understand the technical limits of and maintain their operational systems with greater accuracy and efficiency. Considering the support for GIS mapping among operating staff, prospects are strong for the activity’s institutionalization within the DISCOs. This will hinge on, however, DISCO management providing adequate financial and human resources and an ongoing GIS basics training for new staff and on more advanced topics for existing staff. 19 Most of the views about GIS sustainability were expressed by a minority of staff interviewed at the eight DISCOs. They are reported here because of their potential significance. 26 Evidence suggests that as GIS mapping activities expand system wide, DISCOs will need strengthened technical capability in overall distribution planning. GIS staff expressed hope that as GIS use becomes more widespread, it will improve DISCOs’ technical governance and decision-making. Recommendations Considerations for PDP: During the final year of the project, PDP staff should advocate to DISCO management that they hire GIS unit staff at appropriate compensation levels to incentivize them to stay in their jobs. In addition, PDP should develop a mechanism based on best practices for GIS data-quality assurance, with a percentage of mapped data verified by supervisors. Members of DISCOs’ middle and senior management need to know the GIS mapping activity and its significance in reducing losses through load-flow analysis and improved oversight of the distribution systems. Their support can ensure sufficient budget and staffing for GIS operations. USAID and PDP should seek support from upper management, as well as from NEPRA, so that GIS mapping costs can be recovered through tariffs. On an immediate and pragmatic level, DISCOs need, with PDP assistance, to devise a system for reporting ongoing changes in the already mapped feeders to GIS personnel. Also, it appears that more SynerGee software licenses are needed. Even if this is an internal DISCO matter, PDP should assist. Considerations for USAID and PDP: In the remaining phase of implementation and follow-on activity, USAID and PDP should support a forum/network among GIS staff across the DISCOs to share knowledge and experience. Both existing and new staff also need follow-on training and refresher courses on GIS mapping. Such training workshops should be held in conjunction with the GIS forum. IX. DEMAND SIDE MANAGEMENT (ENERGY EFFICIENT INDUSTRIAL MOTOR) PROGRAM The industrial motors activity involved installing energy efficient (EE) motors and variable-speed drives (VSD) at industrial facilities to improve energy efficiency. Industries were motivated to participate through a 50 percent subsidy on the motors and drives. PDP staff directly conducted the activity. Although it was called a Demand Side Management activity, the DISCOs were not involved in the activity. Findings 1. Results of Activities: MEP conducted a phone survey of representatives from 48 out of 52 firms that participated in the motor program. A majority (85 percent) said they participated in the program to save energy and reduce bills. According to data provided by PDP, 52 firms within all DISCOs operating areas, except QESCO (See Figures 2 and 3), installed EE motors and VSDs. Two firms within the Karachi Electric Supply Company (KESC) service territory participated, even though KESC is not an official PDP beneficiary. Textiles (36 percent), food production (27 percent) and cement production (21 percent) were the main industrial subsectors represented in the EE motor program. A majority of participants (75 percent) were satisfied with the equipment price and 87 percent reported that the equipment was operating normally. However, 21 percent (nine representatives) complained that the installed equipment was affecting their industrial processes, with reduced production being the main complaint, and 19 percent (eight representatives) said the equipment failed to achieve its intended benefits (i.e., energy and money savings). The majority of respondents (83 percent) said they would most likely install more EE motors and VSDs. For the 12 percent (five representatives) who would not, most cited the perceived inability of equipment to achieve intended results. 27 A large number of respondents (87 percent) said they believed use of EE motors and VSDs among industries in Pakistan is widespread, although this does not appear to be accurate because high-efficiency motors and VSDs are expensive. Experience around the world shows that they are not purchased in large numbers without educational and financial incentive programs in place to promote the motors and assist with their selection and installation. This also contradicted the belief by 54 percent that awareness about EE motors and their benefits is lacking in the country. FIGURE 2: INDUSTRIAL PARTICIPATION IN SURVEY, BY DISCO TERRITORY FIGURE 3: INDUSTRIAL PARTICIPATION IN SURVEY, BY SUBSECTOR HESCO 15% FESCO 13% IESCO 6% KESC 7% LESCO 33% PESCO 11% MEPCO 15% Food Production 27% Auto Parts 2% Refrigeration 2% Cement Production 21% Chemicals and Fertilizer 6% Textile Fabrics Spinning 36% Non Metalic Production 4% Metal Production 2% 28 Conclusions The EE motors and drive installations likely saved energy. PDP used an acceptable methodology that involved an initial inspection immediately after installation. A re-assessment of the data collection and calculation methodology was conducted as part of a Data Quality Assessment (DQA), but further inspections using alternative methodologies that include more extended and/or regular measurement and verification may provide more accurate assessments of energy savings. The activity is unlikely to be sustainable beyond the conclusion of PDP because no plan is in place to continue the 50 percent subsidies or establish a follow-on activity to promote and finance the motors and drives. Only 4 percent of survey respondents said the 50 percent subsidy motivated them to participate. Further, since the activity was conducted independently of the DISCOs, no DISCO staff are trained and ready to continue the activity. The lack of an established institutional infrastructure undermines the sustainability of the program. Recommendations Considerations for USAID: Improving the energy efficiency of industrial motors in Pakistan is an important and worthwhile activity and should be continued by USAID. However, the activity should be implemented through the DISCOs, which can structure it as a Demand Side Management (DSM) program designed to reduce electricity demand during peak periods. The continuation of the program should also include a financing or financial incentive strategy that can keep implementation going without indefinite reliance on government subsidies and donor funding. The continued program should also include adequate measurement and verification of savings. Utilities are not always supportive of end-use energy efficiency, but once they understand how it can help reduce supply constraints and be used to manage peak demand, they should be quite supportive. Considerations for USAID and PDP: USAID and PDP should work with NEPRA and the DISCOs to reinforce and incentivize the establishment of the EE motor program as a DSM activity. Similar DSM programs should develop in response to other EE opportunities in other sectors. In the near term, PDP should measure and verify energy savings at a statistically valid sample of installed motors. It should also reach out to all program participants and seek to resolve complaints from a small percentage of firms about motor performance. D. FINANCIAL MANAGEMENT X. ENTERPRISE RESOURCE PLANNING Successful management of a large enterprise like a utility requires reliable, timely and accessible information on revenues, costs, number of employees and geographic dispersion of assets and customers. USAID’s operational audit reports of the DISCOs pointed out that modernization was long overdue for the legacy financial and management information practices. Enterprise Resource Planning (ERP) systems were selected as a method for tracking and organizing all DISCOs’ major operational elements. Findings 1. Results of Activities: ERP development under PDP began at IESCO and FESCO, while LESCO chose to pursue its own ERP implementation arrangements. Starting in PDP Phase III, the ERP focus was on just the two “turnaround” DISCOs: MEPCO and PESCO. Their ERP systems are being rolled out and therefore not fully functional. According to respondents at the two DISCOs, PDP has developed ERP accounting and implementation manuals, approved by the board of directors at one DISCO and awaiting formal approval at the other. At PESCO, the Board of Directors decided in April 2014 to establish an IT infrastructure for ERP and recruit staff. Respondents indicated that PDP has documented all necessary processes and steps to initiate ERP implementation at both DISCOs. Specifically the program has completed the vendor selection process and basic staff IT training. However, respondents at both DISCOs expressed concerns about the lack of operational and implementation plans for ERP implementation. 29 Respondents at PESCO and MEPCO said ERP, once fully implemented, will reduce the time and steps for recording and accessing information for decision-making on DISCO finance and will improve accuracy of the information. As one MEPCO participant stated, “ERP implementation will address the issues of timeliness and accuracy of information, leading to better control and improved decision-making at DISCO.” 2. Prospects for Sustainability: Respondents at both DISCOs said senior management support, and will pay ongoing costs for, ERP implementation, which will ensure its continuation after PDP ends. But they indicated a lack of trained staff, which will hinder full-scale ERP implementation. Both MEPCO and PESCO have this sustainability issue, as staff lack capacity to lead implementation without PDP technical oversight. Both DISCO respondents and PDP staff indicated that, although ERP implementation activities are under way, the remaining program duration is insufficient for its full implementation. DISCO respondents also stated that no plan exists for continuation of ERP implementation after PDP ends. However, an indication of senior management support at PESCO is that it included ERP operating costs in its tariff petition, which was approved by NEPRA in April 2014. Likewise MEPCO’s 2014 tariff petition included a line item for ERP, and the petition was approved by NEPRA. Conclusions ERP implementation is moving forward and the approval/adoption of process documentation and allocation of implementation funds shows a willingness to keep transitioning from the manual system to the automated one. However, while some training on ERP has occurred, the limited staff and management capacity poses a great challenge for eventual full implementation. PDP and the DISCOs need to make a concerted effort to mitigate this. No clear action plan is in place about how to continue with implementation after PDP ends. Recommendations Considerations for PDP: A plan or roadmap for continuing ERP implementation after the completion of PDP is critical. PDP should generate an action plan clearly defining how to divide costs associated with ERP implementation between the program and the DISCOs. It should also include a strong counterpart arrangement for oversight of ERP vendors. PDP should strengthen DISCO staff project management capabilities before PDP operations end to ensure adequate ongoing independent oversight of implementation and vendor performance. E. GOVERNANCE XI. & XII. COST OF SERVICE STUDY (COS) – PHASE II & III Financial deficits at the DISCOs have resulted in part from their inability to charge customers the full price of supplying electricity. The deficits have, in turn, constrained the DISCOs’ capacity to invest in system improvements.20 As part of PDP, USAID developed an activity to help DISCOs determine the cost of providing service to different customer classes and to use that information in their tariff petitions to NEPRA. If NEPRA approves, tariffs will recover the costs of providing service and DISCOs’ financial conditions will improve. PDP trained DISCO staff in the use of a COS model to prepare future tariff petitions. 20 USAID EPP Report on Pakistan Power Sector Subsidies, 2010. 30 Findings 1. Results of Activities: Staff at all nine DISCOs understand and appreciate how the COS studies identify the cost of serving each customer class. Tariff petitions are already being prepared on the basis of the COS at five of the nine DISCOs. All respondents in the nine group interviews said tariff petitions reflect, or are expected to reflect, the true costs of service. However, tariff setting typically considers more than just the costs of serving each customer class. It also considers affordability of electricity, particularly for lower-income residential users. Currently, the Government of Pakistan subsidizes residential tariffs. Industrial users also cross-subsidize them.21 Respondents at the five DISCOs where COS studies are already being prepared, said elimination of cross-subsidies in the tariffs would be politically difficult to implement because residential tariffs would have to increase. The COS studies and PDP COS training with NEPRA and DISCOs are already having results in setting more accurate tariffs based on the DISCOs’ cost of service to different classes of customers. Staff interviewed at NEPRA stated that some 2014–15 tariff determinations would be made on a cost-of-service basis for DISCOs submitting COS-based tariff petitions. Tariff determinations for the remaining DISCOs would be COS-based the following year. NEPRA has informed DISCOs that after COS-based tariffs are determined, multiyear tariff determinations, instead of annual determinations, will be initiated. 2. Prospects for Sustainability: NEPRA’s support of COS-based tariffs should improve the likelihood that DISCOs will continue to submit COS￾based tariff petitions. Another indication that the COS studies will be sustainable is senior management’s active support. Respondents at the five DISCOs where the COS studies were already being prepared said senior management were either actively involved or committed to the COS studies. Few majority themes emerged from the COS interviews, but one recurrent concern related to sustainability was the belief that staff capacity is limited to handle the COS studies independently. Four groups mentioned the need for ongoing staff training to ensure that COS studies are sustainable. One respondent said that while PDP trained the COS staff well, the training addressed only COS model use, not how to modify it if needed. 4. Modifications Proposed by Respondents: A majority of DISCO respondents did not explicitly suggest modifications. However, as mentioned, four of the nine DISCO respondents expressed concern about inadequate human resources and training. Presumably these staff would support a modification to the program that involves additional staffing and training. Three groups of respondents suggested that PDP provide a COS user’s manual. Such a manual has indeed been prepared by PDP, but at the time of this evaluation (September 2014), it had only been provided to one DISCO, IESCO. Conclusions DISCO management and NEPRA support for COS studies by is a good indicator that PDP’s COS activity is a success and will likely be sustainable after the program ends. However, insufficient human resources and training could undermine these. The extent to which the COS studies will be the basis for proposed tariffs is not clear; tariff determinations will eliminate cross-subsidies, which may be a tough political choice. As five respondents indicated, it will also be 21 Cross-subsidization is the practice of charging a higher price to one group of consumers for a good or service to subsidize another group so it can pay a lower price. 31 difficult to fully remove cross-subsidies for the residential sector. It appears that a larger factor than COS studies in tariff determination will be the degree to which the GOP decides to pass on full costs to customers. Recommendations The COS studies represent an opportunity beyond tariff-setting; they can be the basis to institutionalize accounting practices that accurately capture and record all DISCO costs. The studies’ findings can initiate cost containment programs that minimize waste while optimizing cost-effective practices. Considerations for PDP: DISCO staff should receive training on how to use the COS software, specifically to modify and adapt it for these other purposes. Ensuring adequate staff resources and training is crucial for COS to be fully utilized and sustained in the long term. PDP should reexamine the COS staffing and training regimen at each of the nine DISCOs and establish a plan and budget for them to ensure that adequate and fully trained staff are available to continue the COS studies in the future. Also, a COS user’s manual would be beneficial. PDP should provide guidance materials of this kind before the program closes out. XIII. ASSISTANCE TO NEPRA PDP assisted NEPRA in organizational strengthening and improvements in its regulatory functions and capacity to perform. This support included capacity building for organizational and functional restructuring, development of an electricity market framework, technical performance standards for DISCOs and developing detailed procedures for tariff-setting. Similar to PDP assistance provided to DISCOs, it also provided assistance to NEPRA on how to incorporate COS studies in tariff-setting procedures. Findings 1. Results of Activities: Respondents in the group interview at NEPRA expressed satisfaction with the PDP support received and showed strong support for NEPRA’s resulting achievements. For example, PDP developed the methodology, process documentation, and guidelines for NEPRA’s new tariff determination procedures; case management procedures to follow for tariff petition filings; rules for electricity wheeling;22 rules for housing societies that distribute electricity to their customers;23 and rules pertaining to DISCO compliance with technical performance standards. In addition, they said PDP conducted an IT infrastructure needs assessment, employee compensation procedures, development of staff job descriptions for 190 positions, and staff performance evaluations. 2. Prospects for Sustainability: NEPRA respondents said the new procedures will remain after PDP ends and that NEPRA will update and revise the rules and procedures prepared with PDP’s support. According to PDP, NEPRA has approved tariff guidelines prepared by PDP. However, they did not share specific steps they will take to sustain the various activities. 3. Effect on Governance: NEPRA respondents stated that the COS studies will have a major effect on power-sector governance. Tariffs based on the real cost of service provision to different customer classes will also bring transparency to the tariff￾setting process, resulting in greater public support for the tariffs. COS-based tariffs will also increase DISCOs’ revenues, enhancing their ability to cover their costs and allowing for improved performance. 22 “Wheeling” refers to the transfer across transmission and distribution lines of electricity from one area to another. 23 Nonprofit cooperative housing societies in Pakistan develop land and provide municipal services to residents, including electricity services. 32 Conclusions PDP support has helped NEPRA address basic tariff review procedures and improve its organizational structure and capacity to effectively regulate the power sector. However, it will take strong leadership at NEPRA to fully institutionalize the reforms. The first major test will come with NEPRA’s rulings on the new tariff petitions, which are starting to be based on the COS studies. NEPRA will need to balance the priorities of DISCOs’ economic viability with the social and political considerations regarding lowering residential energy costs through the cross￾subsidy system. NEPRA may need additional assistance or advice as the reforms become operational and questions emerge on improving or modifying the new procedures. Recommendations Considerations for USAID: PDP has been assisting NEPRA on a variety of fronts, but from the standpoint of this evaluation, the most relevant assistance is in tariff-setting. There is a continuing need for USAID to support NEPRA in its determinations of cost-based tariffs. NEPRA staff should receive training on regular cost monitoring and ongoing calculation of cost-based tariffs to ensure that NEPRA continues to update and use the COS model in the future. Elimination of cross-subsidies based on the cost-of-service model is likely to become a difficult and controversial decision that can result in substantial increases in residential tariffs. USAID should ensure ongoing assistance to NEPRA after PDP ends on options for setting economically rational tariffs, while considering consumers’ economic needs. F. HUMAN RESOURCES & CHANGE MANAGEMENT XIV. LINEMAN SAFETY TRAINING The lineman safety program was designed to promote awareness and provide in-field training on safe work practices and methods. PDP conducted a number of trainings at each DISCO for linemen and, on some occasions, for linemen managers and supervisors. This activity was assessed through a mixed-methods approach including both staff interviews at nine DISCOs and results of an in-person survey of 333 trained linemen across five DISCOs. Findings 1. Results of Activities: The lineman survey found that 20 percent of respondents had an accident during their professional career. This finding is skewed toward two DISCOs with high occurrence of linemen’s accidents: PESCO (29 percent) and HESCO (28 percent). Of the 20 percent of surveyed linemen in all DISCOs who have had accidents, electric shock was the most common type (51 percent), followed by falls from heights (22 percent) and electric spark (12 percent). Surprisingly, 87 percent of the surveyed linemen said there were Standard Operating Procedures (SOP) in place at DISCOs with 84 percent stating that such procedures covered safety-related issues, and 68 percent maintaining that linemen followed the procedures diligently. Nearly all survey participants found the trainings useful, with 98 percent that said the trainings have improved awareness about safety. Respondents also rated the trainings highly: 96 percent rated the overall contents of the trainings to be excellent or very good; 93 percent rated the trainers to be excellent or very good; and 97 percent of the respondents rated the training contents to be extremely or very useful. These results are consistent with findings from the DISCO staff interviews. Staff at eight of nine DISCOs said the trainings resulted in improved field safety practices. While linemen and staff said safety practices improved, the program’s effect on serious accidents and fatalities since the safety trainings began is questionable. According to data cited by respondents at MEPCO and HESCO, serious 33 accidents and fatalities have not decreased since the lineman trainings. However, no fatalities of trained linemen have occurred. DISCO managers are divided on the effect of the trainings on the frequency of accidents. Five of nine managers interviewed reported no change in the number of accidents, while the other four said linemen injuries/fatalities have decreased. Conversely, 98 percent of the trained linemen surveyed said accidents among trained linemen have declined as a result of the trainings. Regardless of whether the accident rate has decreased or not, DISCO staff offered several possible reasons for the number of ongoing accidents. The most likely reason offered by managers is that linemen are under pressure from their supervisors to respond to a large number of service calls and are overconfident about their ability to do so. As a result, a great hurry, thus increasing the likelihood of safety-related mistakes. Figure 4 indicates the range of reasons surveyed linemen gave for accidents. The two most common were “Linemen are careless” (69 percent) and “Not enough safety gear” (40 percent). FIGURE 4: MAJOR CAUSES OF LINEMEN ACCIDENTS 2. Prospects for Sustainability: Of all PDP activities, the lineman training may be the one that is most likely to be continued after PDP closes out. In fact, at six of nine DISCOs, respondents said their DISCO was conducting ongoing safety training for existing and newly hired linemen, and the safety training has been incorporated in the overall training curriculum for linemen. Seven of nine interviewed indicated that management supports lineman training. Six of nine managers said 0 100 200 300 400 500 600 700 800 900 1000 GEPCO HESCO LESCO MEPCO PESCO ALL No Clear Guidelines Management Not Implementing Safety Linemen are Not Trained Linemen are Not Aware of Safety Measures "Macho" Mindset No Transport for Safety Gear No Enough Safety Gear Linemen are Careless 34 the trainings will continue after PDP ends. Six of nine DISCO respondents suggested refresher safety courses for linemen. 3. Effect on Governance: Managers at seven of nine DISCOs said high-quality personal protection equipment (PPE) is required to be available. USAID respondents suggested that the DISCOs or the GOP should adopt strict PPE standards to ensure procurement of better equipment for linemen in the future. However, no signs of progress on improved PPE standards have surfaced. Conclusions While all DISCOs found the trainings useful, timely and highly relevant, it is not clear that they have resulted in a decrease in accidents among linemen. Some DISCOs have mainstreamed safety training as a regular DISCO activity, while others have not continued with the trainings. Nonetheless, the activity is very likely to be sustainable, as awareness created through this activity has resulted in linemen facing a safer work environment with the benefit of having received safety training. DISCO management has been generally supportive of the training and institutionalization of safety procedure at DISCOs. Their regional training centers are providing safety trainings independent of PDP funding. While the quality of training provided by DISCOs is unexamined, this indicates that the DISCOs can sustain safety training after the PDP ends. Questions remain about the effectiveness of the trainings in decreasing the number of accidents. There are four categories to consider: 1) overall fatalities; 2) overall accidents; 3) trained linemen fatalities; 4) trained linemen accidents. Categories 1 and 2 have not decreased. Category 3 is zero. However, the data are unclear with respect to category 4. Although no trained linemen have died on the job, they may still have accidents. The linemen surveyed by MSI thought accidents had decreased among trained linemen as well, however, it is not clear whether the frequency of these accidents has indeed decreased. Part of the problem may be that linemen do not always use the PPE provided. Managers and PDP staff also mentioned that the culture among linemen does not always encourage safety equipment use and management and senior leadership rarely enforce PPE safety regulations. Recommendations Considerations for PDP: PDP staff should conduct a thorough analysis of serious accidents and fatalities before and after the trainings to see if they have reduced accidents and fatalities. The examination should look at all linemen, but focus on those who received safety training. It should also seek to determine why accidents among trained linemen continue. PDP should continue to support follow-on training in collaboration with the DISCOs and their RTCs. PDP should work with HR departments and unions at DISCOs to ensure employee compliance in the use of safety equipment, procedures and practices and enforcement by supervisors. It should also work with DISCOs to establish PPE procurement rules to require purchase of only high-quality safety equipment. Follow-on training will be effective in reducing accidents only if DISCO policy requires safety equipment use and supervisors and DISCO leadership enforce it. XV. UTILITY EXCHANGE PROGRAM In this activity, DISCO staff were selected for exchange visits to electric utilities in other countries. The objective was to expose these staff to ideas and best practices by utilities in other countries that might be useful in Pakistan. 35 Findings 1. Results of Activities: Evidence suggests that utility exchanges helped staff acquire new ideas and increase awareness of best practices in other countries. Staff at eight of nine DISCOs indicated that while the utility exchanges were very informative for participants, the knowledge gained was not frequently applied. Six said the knowledge gained was difficult to implement and has resulted in little change at their DISCOs. Just two groups of exchange participants said they used the knowledge gained to apply new ideas at their home DISCO. 2. Prospects for Sustainability: Six of nine respondents said their DISCOs are unlikely to continue the utility exchanges on their own. All nine respondents said their DISCOs did not have resources to continue the exchanges. Seven said their DISCOS have no arrangement with foreign utilities for such programs, so the exchange is unlikely to be sustained. 4. Modifications Proposed by Respondents: The participants proposed that any future exchanges should be conducted on a regular basis over longer periods of time. They also suggested the exchanges be coupled with follow-up activities at the home DISCO to enable participants to share knowledge gained and increase the likelihood that good ideas from other utilities are implemented at the home DISCOs. Four of the nine respondents suggested visiting utilities that operate in similar socioeconomic conditions as Pakistani DISCOs, where the innovations and ideas learned at such utilities are probably more relevant and implementable at home than best practices undertaken by utilities in advanced countries. 5. Gender Equity: By design, female employees were included on the Pakistani exchange teams. Participants reported seeing female employees at the host utilities, many in middle and top management positions. Two respondents and PDP staff said these observations helped sensitize male participants to the idea that women can perform in management positions at Pakistani DISCOs. Conclusions The activity partially achieved its intended objectives of exposing DISCO staff to practices that could be implemented at the participants’ home utilities. While participants’ personal knowledge improved, in most cases the DISCOs have not applied practices learned. The exchanges are unlikely to be sustainable after PDP ends, due to financial constraints. An unanticipated benefit of the exchanges was the improved gender sensitization among male participants who saw women in high-level positions at host utilities. Recommendations Considerations for USAID and PDP: If continued, exchange visits in the final phase of the program and in future programming should be occur at utilities with similar socioeconomic conditions as Pakistani DISCOs or those that have emerged from similar conditions to build modern energy distribution systems. For example, more exchanges like the one that PDP conducted in Meralco (Philippines) or Energy Generating Authority of Thailand (EGAT) might be appropriate to replicate. There should also be preset procedures for participants to convey ideas to the appropriate management staff at their home DISCOs. Considerations for USAID: USAID should consider a domestic personnel exchange program among Pakistani DISCOs to increase cross-fertilization of ideas and establish ongoing dialogue on best practices. The domestic and 36 international exchanges should be part of a larger capacity-building effort to help DISCOs identify and apply lessons. Also, exchanges should include DISCO leadership and senior managers, who have more power to implement change. XIV. ENERGY CONSERVATION CAMPAIGN The Energy Conservation Campaign sought to spread awareness on energy use and conservation at home and on college campuses. This activity was conducted under the gender component, which is aimed at increasing women’s participation and especially career selection within the power sector. This activity is formally referred to as a “campaign,” but it involved only one-off presentations by PDP staff at girls’ colleges about energy conservation measures and career opportunities for women in the power sector. Findings 1. Results of Activities: Participants in all six group interviews said the trainings were informative, and increased awareness on energy saving practices. Respondents in five interviews said energy efficiency measures, such as turning off lights and unplugging unused devices and appliances on standby, were undertaken at their colleges. Respondents in five of the interviews said the presentations created awareness about energy conservation among participants, which included faculty, staff and students. While some behavioral changes were cited, there was no evidence that institutions introduced significant changes to make their facilities more energy efficient. 2. Prospects for Sustainability: The respondents in five of six group interviews said their college will not continue this activity on its own. In fact, since the original presentations, neither the colleges nor PDP has followed up. The activity did not include direct support from DISCOs, and PDP and the DISCOs have not followed up to ensure that the campaigns continue. As one principal from Hyderabad said, “Without the support from PDP or HESCO, it is very difficult — rather impossible — to conduct such activities in the future.” 4. Modifications Proposed by Respondents: Four of the six college respondents called for more and regular energy conservation presentations. Some principals recommended more incentives for using energy efficient alternatives, such as energy efficient bulbs and appliances. Principals from two of the six girls’ colleges suggested PDP advocate that provincial education departments include energy conservation-related topics in curricula to ensure long-term awareness-raising among students and teachers. Conclusions While the workshops raised awareness among participants, they had only a limited effect on energy use at the schools. Respondents reported some energy conservation measures, such as turning off lights, taking place. PDP did not attempt to integrate the presentations into any larger EE strategy for the colleges. Recommendations Considerations for USAID and PDP: This was intended as a gender-related activity, but is an energy efficiency activity as well; its objective should be framed around achieving energy savings. If the result is to increase energy efficiency, the activity should expand into a real campaign, with energy audits, audit review, financial incentives, financing, post-installation inspections and overall support of the colleges throughout the project. This activity should be part of a broader public outreach and communication activity with a dedicated focus on energy savings. Given that PDP focuses on improving technical and financial capacity at DISCOs, EE programs, including the Energy Conservation Campaign, should be structured as Demand Side Management activities operated by the DISCOs. 37 Greater coordination with the provincial education departments would be required to have a significant effect on educational institutions and a broader effect on energy conservation. With USAID support, programming should advocate that provincial education departments include topics related to energy conservation in the provincial curriculum to make awareness a more central tenet of all students’ education. USAID ENR should consult with USAID Education to identify strategies for this crosscutting effort. XVII. GENDER EQUITY TRAINING This activity involved one-off workshops at the DISCOs to raise awareness of gender equity issues among staff. Topics included women’s rights, common offenses against women in the workplace and how women are protected constitutionally from harassment in the workplace. It also addressed male colleagues’ role in discouraging discrimination against women in the workplace. The workshops intended to improve conditions for DISCOs’ female workforce, specifically the handling of sexual harassment complaints, and increase recruitment and promotion of women within the DISCOs. Findings 1. Results of Activities: Staff at four of six DISCOs involved in this project activity said the workshops raised awareness of gender issues and relevant laws among DISCO staff. However, the attendees represented a small percentage of all DISCO employees. For example, only 22 of the 13,500 employees at IESCO attended the gender training. Some interviewees explained that while they got a lot out of the training, it was not enough to change the workplace environment at the DISCOs. As a female manager and training participant explained, “The empowerment of women is being denied in a male￾dominant power sector. The promotion and induction policies are not being adhered to properly and women are not being provided a congenial work environment to unleash their full potential.” No other majority views emerged about the results or effectiveness of the trainings. When asked about greater sensitivity among male colleagues, improved cooperation between male and female colleagues, greater staff concern with gender issues or a better workplace environment for women, just one respondent replied in the affirmative. When asked about improved recruitment and promotion opportunities, no respondents replied in the affirmative. As one female participant explained, “Those who attended this training have become more respectful toward their female colleagues and there is a positive change in their attitude,” However, she also mentioned that “only a few staff members have been trained from the DISCO” so “we did not observe substantial changes in the workplace as far as women’s rights are concerned.” According to USAID analyses conducted in connection with the Energy Conservation Campaign activity, a common attitude exists that “the utility sector is a men’s domain.”24 Women constitute less than 1 percent of the workforce at all nine DISCOs and are thus marginalized as a “special category.” 2. Prospects for Sustainability: No follow-up gender equity workshops have taken place and they are unlikely without PDP organizing them. However, PDP reports that staff has written to the DISCOs requesting additional trainings and the DISCOs have responded positively. 24 FESCO College Campaign Impact Assessment, USAID 2012 p. 4. 38 3. Effect on Governance Improving gender sensitivity among male employees and management at the DISCOs is a requirement to enforce HR regulations, fair hiring practices and the rights of women in the workplace. DISCO management has become somewhat more gender sensitive partly as a result of the training workshops and partly because of other gender￾related measures the DISCOs have had to undertake, such as establishing sexual harassment committees and building women’s washrooms. However, only one respondent said top management is committed to improving gender equity. Conclusions The training workshops were well received and raised awareness among participants, but as one-off events, they are unlikely to result in long-term changes at the DISCOs in female recruitment, promotion or handling of harassment complaints. Recommendations Considerations for USAID: Given that improving participation and the working environment for women in the energy sector involves long-term cultural changes, PDP’s gender equity intervention should be a longer-term and more comprehensive priority for the program. As the training reaches only a small percentage of DISCO staff, PDP should expand training and provision of materials to reach more staff. Training should reach senior management, with a focus on the benefits gender equity can bring to the organizations’ productivity. The activity should also include training for trainers (as in the training for lineman work) so they can better replicate the training and lessons learned. These efforts would help create champions within the DISCOs that can build a critical mass of people to effect change in the DISCOs. In addition, follow-on activities and continued training should be part of the design of this activity to reinforce learning and long-term change in the DISCOs’ workplace environments. Program staff and consultants hired to design and manage such interventions should be experts on gender equity in the workplace. Furthermore, these initiatives should be coordinated with USAID’s national gender equity program. 39 SUMMARY OF FINDINGS AND CONCLUSIONS 1. RESULTS OF ACTIVITIES Most activities addressed by this evaluation have achieved, or started to achieve, positive results. Training and technical assistance, the key features of many of the activities, have been effectively provided, according to many DISCO respondents across all activities. Other activities are new or have not started, so outcomes are not evident. To make a lasting effect, some efforts should be more targeted in their design, reach a critical mass of stakeholders and integrate more follow-on activity. Although the respondents made a number of suggestions about improving PDP activities, none complained about PDP performance directly. The lineman training activity has been particularly successful, heightening awareness of safety practices; the DISCOs have embraced it and are continuing the trainings, a sign they may support the trainings after PDP is completed. While no evidence shows that the training has led to a decrease in accidents, linemen overall now understand safe practices and the personal protection equipment required for working on power lines. While implemented only on a pilot basis so far, the advanced metering technologies — HHUs, AMRs and Electronic Metering — are already improving the accuracy of meter reading while reducing paperwork for meter readers and reducing fraudulent readings. Challenges exist to these activities’ sustainability due to the high capital costs of the technologies involved. Many hope NEPRA will approve passing these costs onto the tariffs, but upfront funding and strong management support will be needed to expand these metering techniques across the service territory. The COS studies and PDP COS training with NEPRA and DISCOs are already having results in setting more accurate tariffs based on the DISCOs’ cost of service to different classes of customers. Staff interviewed at NEPRA for this evaluation said some tariff determinations at DISCOs that are submitting COS-based tariff petitions will be made on the basis of cost of service for the year 2014–15. The ERP and CIS show great promise but it is too early to determine their effects on the financial system and customer service at the DISCOs. PDP has developed accounting and ERP manuals and DISCOs have adopted them. MEPCO and PESCO are piloting the CIS and it is expected to go live by the end of December 2014. Although both turnaround DISCOs appear to be making headway on CIS implementation, full assessment of effectiveness is not possible at this time. GIS mapping capability has significantly improved and has provided greater ability to visualize the distribution system; however, it is too early to see concrete results on operations. Nonetheless, DISCO staff are optimistic about the system’s prospects for improving management of the distribution systems and preventing overloading of feeders. Sustainability of the activity is in question, however, because of inadequate training, relocation of GIS staff and insufficient support from top management. Results from the survey of industrial representatives suggest that the industrial motor and VSD program has resulted in significant energy savings, although the savings are not consistently measured. Furthermore, since the activity was conducted independently of the DISCOs, it will be difficult to continue after PDP ends because no institution is ready to provide both the technical expertise and subsidies required to maintain it. The Utility Exchange Program had positive results in teaching participants about practices at other utilities that could be used at Pakistani DISCOs. However, few learned practices have been adopted and opportunities to share lessons learned with DISCO management are few and far between. Some respondents expressed that exchange locations were not always a good match with the kinds of challenges and level of development experiences at DISCOs in Pakistan. 40 Three activities have not achieved clearly positive results to date. These are the Anti-Theft Campaigns, the Energy Conservation Campaigns at girls’ colleges and the Gender Equity Trainings. Results for the Anti-Theft Campaigns are extremely difficult to measure, especially in the short-term. DISCO staff talked about how the campaigns need to be just one element of a broader anti-theft effort, but it would still be difficult to separate out the reduced theft resulting from just one component of that effort. Little evidence showed energy savings resulting from the Energy Conservation Campaigns. The actual energy conservation presentations by PDP staff were well received, but as a one-off activity, it did not achieve much. The faculty, administration and students at these schools suggested that to make a difference, more in-depth approaches to conservation would be needed, such as incentives for energy saving technology at the institution and integration of energy conservation in the curriculum. The Gender Equity Trainings received high scores and the DISCOs and PDP deserve praise for working to repeat them. But they alone have not spurred many gender equity improvements at the DISCOs. Admittedly, like the Anti-Theft Campaigns, the results of this activity are difficult to measure at this stage. These are issues that will not easily change in a short period and with reach to a fraction of the DISCO population. Without a critical mass of trained staff, it is hard to substantiate any change in awareness resulting from the activity or effect on the gender composition of the DISCO’s workforce or workplace environment. Not enough DISCO management has been trained or involved with institutionalizing gender equity measures, such as ensuring women are recruited and promoted within the utilities and ensuring that the establishment of harassment committees are effectively protecting women from harassment. 2. PROSPECTS FOR SUSTAINABILITY It is impossible to predict with certainty which PDP activities will be sustainable. However, we used three main criteria to determine the likelihood that any activity would continue once PDP technical and financial assistance has ended. First, sustainability is likely if there is clear support for the activity from top DISCO management, as is the case with COSs. Second, sustainability is likely if the DISCOs have already expanded the activity on their own without PDP involvement, as is the case with lineman training. Third, sustainability is likely if the activity will not involve major capital outlays. The PDP activities that appear to have the greatest prospects for sustainability are the lineman training, Cost of Service Studies, and Enterprise Resource Planning. Some DISCOs have already integrated lineman training into their Regional Training Center curricula, making it more likely the trainings will continue without USAID support. Likewise, the DISCOs understand the Cost of Service Study well, the DISCO staff say senior management is supportive, and, except for some possibly inadequate staffing, the activity appears to be well-positioned to continue and to form the basis for tariff petitions into the future. Currently supplied Hand-Held Units (HHUs) are expected to require only minimal maintenance costs and therefore may have sustained use. Some activities, in contrast, may require additional steps to ensure sustainability. Based on the successes of PDP activities and the lessons learned, some activities may be ideal for expansion and any such expansion would require additional funds and management support. Activities with potential for expansion include: Automatic Meter Reading, Electronic Metering, GIS mapping, Demand Side Management (energy efficient industrial motor) program and the use of HHUs. Considering the support for GIS mapping among operating staff, prospects are strong for the activity’s institutionalization within the DISCOs. However, sustainability of the GIS mapping activity is dependent on support from top management and continued training, particularly during staff turnovers which occur frequently. In contrast, some activities that do not require large investment (e.g., Energy Conservation Campaigns, Anti-Theft Campaigns and Gender Equity Training) likely will be unsustainable. The DSM (energy efficient industrial motor) program is not sustainable because the DISCOs did not administer it and no alternative institutional structure, such as a non-profit energy center or energy efficiency fund, was established to maintain it. Additionally, no apparent source of funding exists to continue providing subsidies of up to 50 percent for the new motors. Due to lack of funding, respondents said the utility exchanges are unlikely to continue without PDP support. 41 Sustainability of Energy Efficiency and Demand Side Management activities: The energy efficiency activities, which included the DSM and the Energy Conservation Campaigns, were carried out well, but lost a major opportunity in not involving DISCOs in their implementation. These activities should be designed as DSM programs and should be monitored by utilities so they can see the value to their systems in decreased peak demand and increased ability to manage the timing and size of demand. To improve the effectiveness of all DSM and energy efficiency initiatives, USAID should support the diagnostic analysis such as breakdown of energy consumption by sector, segment and end-uses, as well as load research, to better appreciate the energy efficiency and DSM potential for this activity. USAID should also support the training of upper management at DISCOs on the benefits of energy efficiency and DSM through approaches reflecting cost– benefit analysis and providing economic value. Sustainability of training: PDP effectively provided most training of DISCO staff. Many trainees are now training others within the DISCOs. But while the activities are still in their first phase, DISCO staff will need ongoing training and capacity building as well as an ongoing expert presence. Training and follow-on training is insufficient for the cost-of-service studies, Electronic Metering, Automatic Meter Reading, ERP, GIS mapping (especially on the SynerGee software), and the Customer Information System.25 3. EFFECT ON GOVERNANCE Some of the activities hold out promise for greatly increased governance at the DISCOs. This is particularly true of ERP, COSs, CIS, GIS mapping, and advanced metering. However, these activities are too new to discern their effects on governance. Respondents were generally unsure of the effects of their activities on governance or (in retrospect) didn’t necessarily understand what was meant by “governance.” The cost-of-service studies may generate the clearest example of improved governance. PDP has worked with both DISCOs on modeling the cost of serving each customer class and including that information in tariff petitions. At the same time, PDP has worked with NEPRA on determining tariffs based on cost of service. DISCOs are starting to submit COS-based tariff petitions to NEPRA. As the DISCOs become more comfortable with COS￾based tariff petitions and NEPRA becomes more comfortable approving COS-based tariffs, the financial governance of the electricity distribution sector will become more transparent and efficient, and DISCOs will be able to cover more of their costs and move toward eliminating circular debt and operating in the black. The DISCOs will still need to reduce losses, but moving toward a COS-based system holds out the promise of greatly improved DISCO performance, management and governance. 4. MODIFICATIONS PROPOSED BY RESPONDENTS Given the difficulty of implementing a full complement of activities at all nine DISCOs, the decision to modify the program to focus some activities at just two DISCOs made pragmatic sense. Activities such as ERP and AMR installation require large amounts of time, training and human resources; after PDP ends, USAID should continue to focus the more complex and expensive activities on just a few DISCOs. Other activities that are relatively less complex, such as CIS and DSM/energy efficiency programs, can be implemented across all the DISCOs. PDP interventions have been largely technical, with engineering, financial and accounting support provided for the most part. But activities should be modified, according to many respondents, to do more to gain support from upper management on some of these initiatives. PDP staff have explained that they do indeed work to gain management support for all PDP activities. That may be true, but for some reason the word is not reaching many of the people on the front lines who are responsible for managing and implementing the activities. 25 PDP’s 2014 fiscal year work plan states that sustainability of PDP interventions at the two “turnaround DISCOs” will be addressed through improvements in organizational structure at the DISCOs that will increase staff capacity. In addition, the costs associated with continuing the activities will be calculated and shared with the two DISCOs so that the costs can be incorporated into the DISCO budgets. 42 Some of the activity designs are problematic. In particular, the gender training, Anti-Theft Campaigns and Energy Conservation Campaigns do not appear to be well thought out as far as effectiveness and sustainability. 5. EFFECT ON WOMEN’S PARTICIPATION IN THE POWER SECTOR The evaluation team found no evidence that women’s participation in the power sector has increased. The Gender Equity Trainings seem to have resulted in enhanced awareness of gender equity issues among participants, and DISCOs have established women’s washrooms and sexual harassment committees. But it is not clear if the committees are active and effective. The interviews conducted and documents reviewed did not provide evidence that female recruitment and promotion had increased at the DISCOs. 43 SUMMARY OF RECOMMENDATIONS 1. HHUs, Improved Meter Reading, Electronic Metering and AMRs: To improve the efficiency and reliability of the meter reading and other benefits, use of HHUs should be a temporary step in the transition to AMR, which would reduce human error in the reading process and automatically transmit the reading to a billing system. In the near term, PDP should encourage DISCOs to expand HHU use and support the availability of these devices throughout the service territory. PDP should also advocate for PESCO management and NEPRA to provide financial support for installing Electronic Metering across the service territory, with the aim of eventually transitioning to AMRs. PDP should recruit PESCO metering staff to train metering staff at other DISCOs to help them start pilots of their own. Broader use of HHUs is recommended as a preliminary step in the transition to AMR throughout the DISCOs’ service territories. HHUs should be considered in the longer term in locations where consumers have lower connected load levels. These initiatives should be included in future USAID programming; DISCOs and Pakistani government agencies should be asked to cost share for greater sustainability. 2. Customer Information System (CIS) USAID should continue supporting CIS development at PESCO and MEPCO, at least until it is operational and has a track record that can be evaluated. PDP should ensure that the system is as user-friendly as possible for DISCO staff to fully learn and use. The roles and responsibilities for migrating data to the CIS need to be clarified and senior DISCO management should take greater ownership of the initiative. 3. Outreach Activities and Anti-Theft Campaigns: Data collected over a longer term are needed to assess whether these campaigns are having an effect. Performance indicators should be developed and monitored quarterly to assess the effectiveness of the Anti-Theft Campaigns on changing attitudes and behaviors. Without this information, evaluators were not able to determine the campaign’s effectiveness in reducing theft. Assessment should first determine if the campaign is successful and should guide a formal plan for ongoing coordination of anti-theft activities to provide direction for the last phase of the PDP and ongoing programming. Considerations: By design, Anti-Theft Campaigns are just one of several essential efforts required to reduce power theft. The Anti-Theft Campaigns should continue, but as part of the broader overall, comprehensive anti-theft effort. Furthermore, institutional support and resources from DISCO management for the campaigns are needed to sustain these and other anti-theft activities. 4. GIS mapping: During the final year of the project, PDP staff should encourage DISCO management to hire GIS unit staff and appropriately compensate them to retain them over the long-term. In addition, PDP should use GIS quality-assurance best practices, through which mapped data can be verified by supervisors. USAID and PDP should support a forum/network among GIS staff across the DISCOs to share knowledge and experience. Ongoing training and refresher courses on GIS mapping for existing and new GIS staff are needed. Moreover, USAID and PDP should obtain support from NEPRA to ensure middle and senior management at the DISCOs understand the importance of this initiative, will be engaged in the GIS mapping activity and will allocate sufficient budget and staffing to it. 5. Demand Side Management (energy efficient industrial motor) program: USAID should continue to improve the energy efficiency of industrial motors. However, the activity should be implemented through the DISCOs and function like a DSM program designed to reduce electricity demand during peak periods. The continuation of the program should also include a financing and incentive strategy that can continue implementation without indefinite reliance on government subsidies. The continued program should also include adequate measurement and verification of savings. USAID and PDP should work with NEPRA to reinforce, acquire or incentivize utilities to engage in DSM and increase DISCO support to improve energy efficiency practices. 6. ERP implementation: A plan or roadmap for continuing ERP implementation after completion of PDP technical support is critical. PDP should generate this action plan as soon as possible in the final phase of the program. The plan should include a clear definition of how costs associated with ERP implementation will be shared by 44 USAID and the DISCOs. It should also include strong counterpart arrangement for ERP vendor oversight. Before the project’s completion, PDP should strengthen DISCO staff project management capabilities to enable adequate oversight of implementation and vendor performance. 7. Cost of service: DISCO staff should be trained on how to use the COS software, specifically to modify and adapt it for these other purposes. PDP should establish a plan and budget for the DISCOs to ensure that adequate and fully trained staff are available to continue the COS studies in the future PDP should also consider developing a COS user’s manual to properly guide DISCOS on maintaining and using this analysis for ongoing tariff-setting. COS information could also be part of ERP design. 8. Assistance to NEPRA: USAID should provide ongoing assistance to NEPRA to pursue options for setting economical tariffs that consider the social needs of consumers. PDP should particularly continue assistance in its determination of cost-based tariffs. NEPRA staff training should be provided to build their capacity for cost monitoring and cost-based tariff determination. This will better ensure that NEPRA continues to update and use the COS model in the future. 9. Lineman training: PDP staff should conduct a thorough analysis of serious accidents and fatalities before and after the trainings to see if accidents and fatalities have declined among trained linemen and determine why accidents among linemen persist. USAID and PDP should support ongoing follow-on training in collaboration with the DISCOs. PDP should work with HR departments and unions at DISCOs to ensure compliance and enforcement of safety equipment use, procedures and practices. It should also work with DISCOs to establish procurement rules pertaining to personal protective equipment (PPE) so that only high-quality safety equipment is purchased. Safety equipment use should be established in DISCO policy and enforced by supervisors and DISCO leadership. 10. Utility Exchange Program: Exchange visits in the final phase of the program and in future programming should be held with utilities with similar socioeconomic conditions as Pakistani DISCOs or those that have emerged from similar conditions to build modern energy distribution systems. Procedures should exist for participants to present ideas learned from these exchanges to management and other leadership at their home DISCOs. USAID should also consider establishing a domestic personnel exchange program among Pakistani DISCOs to increase cross-fertilization of ideas and establish ongoing dialogue on best practices. The domestic and international exchanges should be part of a larger capacity-building effort to help DISCOs internally to identify and apply lessons. Also, exchanges should include DISCO leadership and senior managers in the program, as they have more power to implement changes based on what they learn. 11. Energy Conservation Campaigns: The objective of this activity should be framed around achieving energy savings. As such, the campaign should include energy audits, audit review, financial incentives, financing, post￾installation inspections and support the colleges overall throughout the project process. The Energy Conservation Campaigns should be structured as DSM activities with DISCOs’ direct involvement. With USAID support, future programming should advocate that provincial education departments include topics related to energy conservation in the provincial curriculum to make energy conservation awareness a more central tenet of the education of all students. USAID ENR should consult with USAID Education to identify strategies for this crosscutting effort. 12. Gender Equity Training: Given that improving participation and the working environment for women in the energy sector involves long-term cultural changes, PDP’s gender equity intervention should be a more long￾term and comprehensive priority for the program. As the training reaches only a small percentage of DISCO staff, PDP should expand training and provision of materials to reach more staff. Training should reach senior management with a focus on the benefits of gender equity on the organizations’ productivity. The activity should also include training for trainers (as in the training for line work) so they can better replicate the training and lessons learned. These efforts would help create champions within the DISCOs that can build a critical mass of people to effect change. In addition, follow-on activities and training should be included in the design of this activity to reinforce learning and long-term change in the DISCOs’ workplace environments. Staff and consultants hired to design and manage the interventions should be experts on gender equity in the workplace. Furthermore, these initiatives should be coordinated with USAID’s national gender equity program. 45 ANNEXES 46 ANNEX 1: EVALUATION STATEMENT OF WORK JUNE 4, 2014 POWER DISTRIBUTION PROGRAM INTERIM PERFORMANCE EVALUATION EVALUATION STATEMENT OF WORK 47 POWER DISTRIBUTION PROGRAM INTERIM PERFORMANCE EVALUATION EVALUATION STATEMENT OF WORK 48 ACRONYM LIST ADS Automated Directives System AEB Area Electricity Boards BOD Board of Directors COR Contracting Officer’s Representative DISCO Government-Owned Power Distribution Company DO Development Objective FATA Federally Administered Tribal Areas FESCO Faisalabad Electric Supply Company GEPCO Gujranwala Electric Power Company GOP Government of Pakistan HESCO Hyderabad Electric Supply Company IESCO Islamabad Electric Supply Company IMF International Monetary Fund IR Intermediate Result IRG International Resources Group KESC Karachi Electric Supply Company LDI Load Data Improvement LESCO Lahore Electric Supply Company LOE Level of Effort MEPCO Multan Electric Power Company MSF Mission Strategic Framework MSI Management Systems International MWP Ministry of Water and Power NEPRA National Electric Power Regulatory Authority NPCC National Power Control Center OAPA Office of Afghanistan and Pakistan Affairs PIRS Performance Indicator Reference Sheets PDP USAID Power Distribution Program PEPCO Pakistan Electric Power Company PESCO Peshawar Electric Supply Company PML-N Pakistan Muslim League-N PMU Performance Management Unit PQM Power Quality Monitoring QUESCO Quetta Electric Supply Company SEPCO Sukkur Electric Power Company SOW Statement of Work SRAP Special Representative for Afghanistan and Pakistan TA Technical Assistance TPM Team Planning Meeting USG United States Government USAID United States Agency for International Development WAPDA Water and Power Development Authority (WAPDA) 49 I. BACKGROUND INFORMATION A. IDENTIFYING INFORMATION ABOUT THE PROJECT/ACTIVITY TABLE 1: PROJECT SUMMARY B. DEVELOPMENT CONTEXT Pakistan’s electric power sector is in a state of crisis, negatively affecting economic growth, employment and social stability: demand for power outpaces supply by up to 5,500 MW, causing rolling black-outs called ‘load shedding,’ lasting up to 12 hours a day26. It is widely accepted that the energy demand has grown at about a 6 percent rate over the past six years27 . The energy crisis manifests in load-shedding, is pervasive and affects livelihoods and private sector growth in Pakistan. Political interests affect decision-making and reform. Rule-of-law is weak, so there are difficulties implementing reforms. The population and users of electricity are dynamic. Increases in prices related to energy drive up inflation. Managerially, Pakistan’s power sector is in a state of transition from one wholly government￾owned and managed to one that is fully autonomous, where companies operate independently with regard to their ability to purchase, generate, transmit, dispatch and distribute electricity. Initially, the power sector was run as a monolithic organization under the Water and Power Development Authority (WAPDA) whose Power Wing provided the line and functional control of the Distribution Department, directing the operation of eight area electricity boards (AEBs) across the country. In 1998 under power sector reforms, WAPDA was restructured where generation, transmission and distribution were separated into the power sector as it exists today. At the same time, AEBs were converted into stock companies called electric power 26 Government of Pakistan Energy Policy 2013 Report. Title / Field Project/Activity Information Contract/Agreement Numbers EPP-I-13-03-00006 Contracting/Agreement Officer’s Representative (COR/AOR) Nadeem Habib Start Date 09/17/2010 Completion Date 09/17/2010 Location Lahore, Faisalabad, Gujranwala, Islamabad, Multan, Hyderabad, Peshawar, Sukkur, and Quetta Name of Implementing Partners (IPs)  International Resources Group (IRG) USAID/Pakistan Mission Strategic Framework Linkages Development Objective 1: Increased Sustainable Energy Supplied to the Economy Budget $230,000,000 50 distribution companies (DISCOs) with all shares held by the Government of Pakistan (GOP). The power sector reforms also established the regulatory agency, National Electric Power Regulatory Authority (NEPRA) and the Pakistan Electric Power Company (PEPCO) to supervise the transition to full autonomy of the DISCOs. Sixteen years later, the ‘transition’ continues, and autonomy remains an objective rather than a reality. Critical Assumptions The energy crisis and load-shedding will last three-to-four years. Self-interest is interfering with progress. USAID support in collaboration with other donor programs can have an impact on the Development Objective (“Increased Sustainable Energy Supplied to the Economy”) after several years. Political will is needed to resolve the energy crisis. Focus areas and groups PDP works with Pakistan’s nine government-owned DISCOs to improve their operational and financial performance. The program focuses on reducing losses, increasing revenues, and improving customer service, so that the companies can achieve levels of performance commensurate with those of well-run utility companies around the world. PDP works with DISCOs in Lahore, Faisalabad, Gujranwala, Islamabad, Multan, Hyderabad, Peshawar, Sukkkur and Quetta, as well as with the Ministry of Water and Power, and National Electric Power Regulatory Authority (NEPRA). PDP works to: (1) conduct operational audits of DISCOs in governance, engineering, operations, financial management, human resources management (including activities focused on women), communications and customer service; (2) develop and implement performance improvement action plans for each DISCO; (3) provide training for DICSCOs’ staff in strategic planning, engineering, human resources, customer service, linesmen safety, financial management, and communications; (4) introduce new approaches and equipment for improving power distribution and revenue collection; (5) provide technical assistance to the Ministry of Water and Power (MWP) to introduce ‘best practices’ and strengthen the ministry’s planning and policy-making role; (6) implement a national metering program resulting in improved system control and decreased unscheduled load shedding; (7) prepare two DISCOs for commercialization; institute reform in two turn-around DISCOs; (8) implement energy conservation and demand management programs; (9) carry out cost-of-service studies at all nine DISCOs to determine cost-reflective tariffs, (10) Provide technical assistance to NEPRA to improve its performance . Challenges There are several challenges and constraints that PDP faces; these relate to the capacity and willingness of the counterparts to support real reform, and stakeholder engagement. PDP operates with nine different GOP-owned DISCOs spread the breadth and width of the country. In addition, PDP advises the Ministry of Water and Power and NEPRA. Each of these three groups has competing visions as to what the future state of Pakistan’s power sector should look like. Coordination has been a challenge to date. The GOP lacks the leadership to curb both energy theft and non-payment of bills. The biggest offenders are: the federal and provincial governments, FATA, the security services, the military, 51 the military-owned corporations, and the politically well-connected. Circular debt is still an issues, even though it was paid off during the summer of 2013 (by using a tranche of IMF money plus a wave of rupees printing), it is building once again. There appears to be an absence of political will within the government to address the fundamental drivers of the circular debt. The power sector is beholden to a group of ministries that do not understand their role to develop and implement policy to entities in their domain, and do not address the issues to the proper authorities (i.e., the Boards of Directors of the DISCOs). Finally, effecting change during a period when the current PML-N is still settling in after last year’s elections, as the new government pursues a series of short-term interventions to ensure popular credibility, and compliance with meeting quarterly IMF targets and milestones, rather than pursuing a broader strategic visions requiring more deliberate, long-term change. C. DEVELOPMENT HYPOTHESIS Pakistan’s energy crisis will be alleviated by USAID’s assistance to achieve Development Objective 1: Increased sustainable energy supplied to the economy, by providing support for an increased energy supply and improvements to energy sector governance. PDP helps to: improve performance of power distribution companies, enhance the leadership role of the Ministry of Water and Power, support a better managed WAPDA, and strengthen the regulatory framework of the sector, thereby eliminating the sector’s need for government subsidies. PDP also assists the Government of Pakistan in improving their performance in the sector by introducing best practices, new systems (financial and technical) and improved practices to increase revenues. D. INTENDED RESULTS The Mission Strategic Framework (MSF) Development Objective 1 is: Increased Sustainable Energy Supplied to the Economy. The MSF Intermediate Results (IR) which PDP contribute to are: IR 1.1: Increased Energy Supply 1.1.1 Increased Generation and Transmission Capacity 1.1.2: Improved Efficiency of Consumption and the Distribution Systems 1.1.3: Increased financial Sustainability of Power Supply 1.1.4: Increased non-USG investment in the Energy Sector IR 1.2: Improved Energy Sector Governance 1.2.1: Improved Policy Implementation 1.2.2: More Autonomous Energy Sector Entities 1.2.3: Improved Capacity of USAID-Supported Energy Public Sector Entities See Annex 5 for the MSF DO1 energy results framework for more details. E. APPROACH AND IMPLEMENTATION 52 PDP is designed to introduce improved technology, DISCO management practices and to build internal capacity by DISCOs to produce specific and sustainable performance improvements. The primary areas that have been targeted for performance improvement are as follows: 1. Commercial Performance The nine DISCOs can be divided into two groups: those that are relatively high-functioning in terms of commercial performance, and those that have significant commercial and technical losses. As a group, DISCOs have failed to automate their commercial practices and this has led to significant losses with meter reading, data transfer, energy accounting, and ultimately in major deficiencies in commercial practices. PDP performance improvement activities and training programs are focused on introducing improved technologies, practices and procedures to reduce commercial losses (non￾technical) through improved meter reading and data transfer procedures; increased quality control in commercial practices; aggressive theft-detection; and organizational changes designed to support decision-making where it is most appropriate within the commercial directorates of the DISCOs. 2. Technical Loss Reduction DISCO line losses consist of a combination of technical and non-technical components. Non-technical losses are due to commercial inefficiencies and consumer theft, addressed above. Technical losses are the result of line losses resulting from energy flows through the medium voltage and low-voltage distribution systems, including conductor, transformer and other electric power distribution system components. PDP Team conducts the following loss reduction demonstration projects to illustrate how technical losses can be minimized in very densely-populated urban centers, as well as in rural areas where irrigation and agricultural processing loads dominate energy consumption:  Power Factor Correction (improvement to tube well motors)  Automatic Meter Reading (removing the human factor)  Electronic Meters (less expensive and more effective in reducing losses)  Congested Area Strategies and Network Modifications (physical improvements to the facilities in high-loss areas)  Planning and Engineering Capacity-building (establishing Power Quality Monitoring - PQM - units within DISCOs); and  Municipal Water Pumps and Industrial Motors (replacing old pumps and motors) 3. Consumer Outreach Program The primary objective of this project component is to improve the ‘brand awareness and identity’ of DISCOs in order to foster improved understanding by electricity consumers of 53 the DISCOs’ roles as distribution service providers, and ensure that consumers understand that DISCOs are local utilities that do not represent the central government, and encourage consumers to use electricity rationally and pay their bills on time. 4. Financial Management Two important drivers of DISCOs’ financial sustainability are its relationships with private customers and government clients.  Given the low likelihood that DISCOs can independently make significant progress to ensure higher collection rates from government entities, PDP has chosen, under its Financial Management rubric, to concentrate on its private customers – particularly on utilizing collection agencies to collect private sector consumer receivables.  Another important driver of financial sustainability targeted by PDP’s Financial Management intervention is the collection of accurate and timely data to monitor the progress of decision-making and achieving financial goals. Toward this end, DISCOs are assisted by PDP in replacing and modernizing their enterprise resource planning and to ensure higher accuracy, accountability, transparency and reliability of their business data. 5. Governance One of the cornerstones of the power sector reform process, as designed by the Ministry of Water and Power (MWP), was to redesign the governance structure of the DISCOs with the intent of ensuring higher degrees of independence and professionalism in order to improve policy implementation and decision-making. Existing Boards of Directors (BODs) of DISCOs were dissolved and new members were selected from private sector entities, along with highly respected leaders from key institutional beneficiaries. Upon the completion of the selection and appointment process, the MWP requested PDP to design training and mentoring processes for the newly appointed BOD members by means of workshops and competency training in ‘boardmanship.’ 6. Human Resources an Change Management Training and capacity-building of DISCO staff is one of the most prominent contributions PDP is making in sector reform. In addition to training activities, PDP has initiated a ‘change management’ program to facilitate a change in the institutional culture of DISCOs. Changing this culture focuses on providing high quality service and managing the DSICOs with world standards of professionalism directly related to improved corporate performance. A second area of emphasis is on employee safety. Existing safety procedures and training are not standard-based, while the safety reporting system needs revamping. PDP is 54 incorporating additional safety measures in HR management in addition to the current practice of quantifying fatalities for staff and the public to include ‘lost time’ accidents. PDP has launched the Quick Impact Safety Training program for linesmen of two DISCOs and is in the process of replicating this training in the remaining DISCOs, in order to reduce fatal and non-fatal accidents, thereby improving overall operational productivity. F. CURRENT STATUS OF ACTIVITIES Increased Energy Supply  Saved 119 MWs of power through installation of capacitors, electricity meters, pumps and motors, as well as improvement in the DISCOs’ commercial procedures. (This amount of saved power is sufficient to supply electricity to 1.4 million beneficiaries.)  Helped the nine (still) public sector DISCOs increase their income by $133 million through loss reduction and the introduction of modern technology;  Trained over 10,000 linesmen on proper safety techniques, reducing fatal and near-fatal accident by 70 percent in some DISCOs Improved Energy Sector Governance  Improved governance and regulation of the country’s power sector through the development of the National Power Plan of 2013, and the introduction of ‘best practices’ in performance measurement at the MWP;  Reduced unscheduled load-shedding by introducing modern technologies for tracking near real-time power levels in the DISCOs;  Conducted a survey of 50,000 electricity consumers to increase the accuracy of billing for electricity and eliminate illegal connections. The survey identified 1,100 electricity theft cases and 146 wrong tariff cases;  Helped DISCOs adopt “Protection from Harassment in the Workplace Act” to improve the working environment of female staff II. RATIONALE FOR EVALUATION USAID’s evaluation policy encourages independent external evaluation to increase accountability to inform those who develop programs and strategies, and to refine designs and introduce improvements into future efforts. In keeping with that aim, this evaluation will be conducted to review and evaluate the performance of the USAID-funded Power Distribution Program activities implemented by IRG. The evaluation will focus on assessing the program’s performance 09/17/2010 – 04/30/2014 in achieving its program goal, objectives, and results. 55 A. PURPOSE AND USE OF THE EVALUATION The recommendations from this external evaluation of the Power Distribution Project will help USAID/Pakistan’s energy staff and other stakeholders to determine what activities are working well and why, which ones perhaps are not and why, and to make modifications and course corrections, if necessary, to help guide the Power Distribution Project over its remaining implementation time. The flexibility and adaptability of the project should be examined, as typified by the Ministry of Water and Power, WAPDA, NEPRA and the DISCOs where PDP’s work has been concentrated. The evaluation should provide pertinent data analysis and recommendations to assist USAID, ASSIST, SRAP, the Government of Pakistan (GOP), and the USAID/Washington Office of Afghanistan and Pakistan Affairs (OAPA) to understand what has been accomplished organizationally that is attributable to the Power Distribution Project. This should also include relevant management, financial, and cost-efficiency findings present themselves. In summary, the evaluation will help all stakeholders involved to better understand the initial results and contributions of the project achieved from September 2010 to May 2014. It will help reinforce and more closely focus it for the remaining project period (September 16, 2015). B. AUDIENCE AND INTENDED USE The audience of the evaluation will be the USAID/Pakistan Mission, specifically the Energy Office team, OAPA, and the implementing partner IRG. An Executive Summary and recommendations will be provided to the GOP. USAID will use the Evaluation Report to consider making changes to its current energy sector strategy of providing support to the central level and to share ‘lessons learned’ with other stakeholders; IRG will learn about their strengths and weaknesses and adjust their project as necessary; and the GOP (primarily the MWP, NEPRA and WAPDA) will learn how to better benefit from IRG’s TA. It is expected that the DISCOs will have the opportunity to discuss how IRG’s PDP project is assisting them and how this type of project could better assist them in the future to meet their goals. The information will also inform the energy team as the MSF Development Objective results framework is updated based on evidence and lessons learned in the energy portfolio. C. EVALUATION QUESTIONS 1. How has the project achieved its planned results to date (estimated level of effort at 50 percent)? Explain the results and net effects of PDP activities, including any unintended (both positive and negative) consequences. 2. What are the prospects for sustainability of the results achieved thus far; which results appear to be less sustainable? e.g revenue increase activity. 3. Did the project made any difference in the “governance” element of the power sector entities such as MWP, NEPRA, DISCOs? What are some of the accomplishments? What areas still need extensive work? 4. How valid is the current project design development theory and framework. Identify additional approaches or activities recommended, if any, to achieve the program objectives. 56 5. How has the project performed in increasing women’s participation in the energy sector? III. EVALUATION DESIGN AND METHODOLOGY The evaluation team will be responsible for developing an evaluation strategy and methodologies that include a mix of qualitative and quantitative data collection and analysis approaches. The methodology will be presented as part of the draft work plan as outlined in the deliverables below and included in the final report. The evaluation team will have available for their analysis a variety of program implementation documents, and reports. Methodology strengths and weaknesses should be identified as well as measures taken to address those weaknesses. All data collected and presented in the evaluation report must be disaggregated by gender and geography. A. DATA COLLECTION METHODS The evaluation team will apply a mixed-methods approach, using both quantitative and qualitative techniques to collect data from multiple sources in a variety of locations to ensure multiple levels of triangulation. The broad areas addressed by the evaluation include the effectiveness of the approach and results, especially regarding GOP governance. Each of these will be examined through a unique set of methodologies. The overall effectiveness can then be examined by synthesizing the findings for each component. The evaluation team should begin with a ‘desk review’ of all documents cited in the “Sources of Information” section. It should also be prepared to conduct interviews with a sample of assisted DISCOs, the MWP, NEPRA, NPCC and other relevant government institutions in PDP’s six assistance intervention areas (Section D-2 above). The Mission expects the evaluation team to present strong quantitative analyses, within data limitations, that clearly address key issues found in the research questions, such as direct and indirect effects and cost-effectiveness of PDP. The Mission is looking for new, creative suggestions regarding this evaluation, and it is anticipated that the evaluator will provide a more detailed explanation of the proposed methodology for carrying out the work. The methodology will most likely consist of a mix of tools appropriate to each of the evaluation’s questions. These tools may include a combination of the following:  Review PDP documentation  Review GoP’s energy policy and NEPRA reports  Qualitative methods including: o focus group discussions with PDP, DISCOs, MWP, NEPRA and NPCC o stakeholder interviews  Quantitative analyses (USAID is investing about 70-80 million in Multan Electric Power Company (MEPCO) and Peshawar Electric Supply Company (PESCO). We would like to know if these funds have made a difference or not) 57 Overall Assessment In addition to examining each of these project components, the team will need to speak with key personnel from both USAID/Pakistan, particularly the energy office, involved in the project and PDP, the implementing partner. These key informant interviews will provide both project background and facilitate the assessment of overall project effectiveness. Table 1 below details the complete proposed key informant interview plan. As noted above document review is a key component of the evaluation methodology. The required documents include, but are not limited to the following:  PDP annual work plans, quarterly/ annual reports, Inspector General audit report  Operations plan audits of distribution companies by PDP  Other technical reports/ assessments by PDP USAID/Pakistan and PDP will provide all the agreed-upon secondary data in advance of the evaluation team’s arrival in Islamabad, except for those noted. 58 TABLE 2: PROPOSED KEY INFORMANT INTERVIEW PLAN Organization Key Contact IP Type of Beneficiary Budget (US$) City/ Province Total Interviews USAID/Pakistan (Energy Office) ENR Islamabad 5 Power Distribution Program IRG Yes Implementer Islamabad 15 Seven Distribution Companies (LESCO, FESCO, GEPCO, HESCO, QUESCO, SEPCO, IESCO) IRG No Direct Different locations 42 Two Distribution Companies (PESCO, MEPCO) IRG No Direct Peshawar and Multan 22 Ministry of Water and Power IRG No Direct Islamabad 3 National Electric Power Regulatory Authority (NEPRA) IRG No Direct Islamabad 3 National Power Control Center (NPCC) IRG No Direct Islamabad 3 Total 93 59 B. DATA ANALYSIS METHODS Prior to the start of data collection, the evaluation team will develop and present, for USAID review and approval, a Data Analysis Plan that details how interviews will be conducted, transcribed and analyzed; what procedures will be used to analyze the qualitative data from key informant and other stakeholder interviews; and how the evaluation team will weigh and integrate qualitative data from these sources with quantitative data from indicators and project performance monitoring records to reach conclusions about the effectiveness and efficiency of the interventions to date. For analysis of qualitative data, the evaluation team will:  Prepare summary reports of the document desk review.  Summarize key informant interview notes and code them according to themes relevant to the evaluation questions.  Prepare tally sheets identifying the themes that emerge in the document review and key informant interviews to facilitate systematic and rigorous data analysis aimed at identifying key evaluation findings.  Prepare individual reports for each case study summarizing key findings. The team will analyze the quantitative data by preparing cross-tabs and frequency distributions from the online survey. Finally, the team will prepare a detailed outline summarizing key findings, based on all the data analysis, and conclusions for each evaluation question and overall recommendations. C. METHODOLOGICAL STRENGTHS AND LIMITATIONS Key informant, stakeholder interviews and focus group discussions are likely to be major data sources for this evaluation. It is anticipated that some of these interviews will be conducted through translators by the international team required for this evaluation. As a result, some differences in language could enter the interview process and interview notes taken and analyzed by the evaluators that may not fully capture the full intent or meaning offered by the interviewees. It is anticipated that some interviews may be conducted in the presence of at least one or more outside observers, including project and USAID staff, and that the interview responses and group discussions could be affected by the presence of these observers. The evaluation methodology relies on triangulation of sources and methods to ensure the validity and reliability of results. The scope of this evaluation is limited to answering the evaluation questions to the extent that they can be operationally defined and data are available. The methods proposed for collecting and analyzing data are potentially subject to selection bias, which occurs when the subjects of surveys or interviews are not representative of the population of interest. In this case, selection bias is most likely to result from the online survey when some respondents choose to participate while others do not. 60 D. EXISTING DATA AND INFORMATION SOURCES A summary of the documents available to the evaluation team is listed below. A complete list of documents that will be reviewed by the evaluation team will be provided prior to the first evaluation meeting. Some baseline data is available for selected indicators. Program Documents  PDP contract  PDP work plan  PDP M&E Plan (PMP) including PIRS  Annual and quarterly reports  Technical assessments and studies  Audit report of PDP GOP Documents:  GOP National Power Policy 2013  State of Industry Report by NEPRA  Tariff determinations and petitions on NEPRA’s website PDP Work Plan is available both at the Mission and at IRG’s PDP Office in Islamabad (Sector F￾6/2, Street 19, House # 23). PDP performance indicators are described in the Project M&E plan. The M&E plan includes key performance indicators for each key process area, along with indicator definitions, rationales, data sources, collection frequencies and targets. PDP has created Performance Indicator Reference Sheets (PIRS) for each indicator in the M&E plan. The PIRS is a summary resource that describes each indicator in detail and includes information on indicator definitions, units of measurement, data sources, data collection methods, collection and reporting frequency, persons responsible for data coaction, data analysis methods, data quality and data safety procedures, and performance targets. PDP has established a Monitoring and Evaluation (M&E) system that tracks and reports the results of selected performance indicators. The M&E unit has baseline data, quarterly, annual and special reports as well as audit reports. D. EVALUATION PROCESS The evaluation process consists of five main stages: Stage 1: Preparation and planning by the consultant’s staff. 61 Stage 2: Initial review of priority documents by the evaluation team. Working from their home stations, the evaluation team members will review all project background documents. Stage 3: Team work in Islamabad. Upon deployment in Islamabad, the evaluation team will participate in a team planning meeting facilitated by the consultant and undertake specific preparatory tasks as a team. A two-day Team Planning Meeting (TPM) will be held in Islamabad, Pakistan before the evaluation begins. This meeting will allow USAID to present the team with the purpose, expectations, and agenda of the assignment. In addition, the team will: - Clarify team members’ roles and responsibilities - Establish a team atmosphere, share individual working styles, and agree on procedures for resolving differences of opinion - Review and develop final evaluation questions (work out realistic expectations of the team within each of the topic areas during meetings with IRG, MWP, and USAID) - Review and finalize the assignment timeline and share with USAID - Develop data collection methods, instruments, tools and guidelines - Review and clarify any logistical and administrative procedures for the assignment - Develop a preliminary draft outline of the team’s report; and - Assign drafting responsibilities for the final report. Stage 4: Fieldwork. The evaluation team will begin its fieldwork after it completes its team work in Islamabad. Team members will conduct key informant interviews and on-the-spot document review at different distribution companies. Stage 5: Data analysis and report writing. IV. TEAM COMPOSITION A. EVALUATION TEAM POSITIONS AND SKILLS The Evaluation Team will consist of a team of independent international evaluators and Pakistani nationals. A statement of potential bias or conflict of interest (or lack thereof) is required of each team member. A Team Leader/Evaluator will lead a four-person evaluation team to conduct this mid-term performance evaluation in accordance with the USAID Evaluation Policy and directives. S/he is expected to possess at least fifteen years of experience in international development, experience of evaluating USAID projects, short-term consulting experience in Pakistan or neighboring countries, a master’s degree in business/economics or another relevant field and demonstrated leadership and report writing abilities and communication skills. 62 Ideally, we would like to have international and Pakistani evaluations with following skills sets: The international evaluators technical expertise:  A track record of significant professional expertise and experience;  Familiar with USAID evaluation policy and requirements;  One of the team members must have professional experience in evaluation methodology (incl. quantitative and qualitative methods in impact assessment);  Member should be specialized in the energy sector governance (incl. management and technical reform);  Field experience in developing/transition countries (preferably in Pakistan);  Strong writing and editing skills;  Strong analytical skills and ability to synthesize;  Excellent inter-cultural skills and the ability to interact with a multitude of partners and beneficiaries at government, private sector and donor levels;  Fluency in oral and written English. The Pakistani national evaluators technical expertise:  Specific in-country experience in the energy sector in Pakistan;  Sound knowledge of Pakistani energy sector issues and systems;  strong professional experience in power sector engineering tools and techniques;  Not have been closely related with any USAID-funded energy projects in Pakistan;  Fluent in oral and written English, and able to serve as an interpreter. B. STAKEHOLDER PARTICIPATION From USAID, the Office of Energy will be the lead for the evaluation with PMU in a facilitation role. All other actors – IRG, GOP, distribution companies – will be a focus of the evaluation. V. EVALUATION MANAGEMENT The evaluation team will officially report to MSI, who is responsible for all direct coordination with the USAID/Pakistan, through the Contract Officer’s Representative. From a technical management perspective, the evaluation team will work closely with following staff members of Energy Office: Michael Curtis, Energy Office Director Timothy Moore, Deputy Energy Director Nadeem Habib, COR for PDP 63 The Energy Office will provide guidance for the participation of key partners. In order to maintain objectivity, key decisions about the evaluation will involve USAID/Pakistan’s Performance Management Unit. A. LOGISTICS USAID/Pakistan The Program Office’s Performance Management Unit (PMU) will facilitate the preparation of the evaluation SOW in accordance with USAID standards and good practices, review the instruments and the draft report, and provide technical inputs on the contractual matters. The energy office will extend support in conducting meetings with different GOP officials and will provide technical input to the evaluation. Energy Office will also provide different source of information as and when required. Power Distribution Program IRG will provide all relevant information required for the evaluation and facilitate meetings and interviews which the evaluation team may require with their staff and beneficiary organizations. Beneficiary Organizations The staff from selected beneficiary organizations is expected to cooperate with the evaluation team by giving time for meetings and interviews and providing relevant documents. MSI MSI will provide support for travel, lodging and other arrangements related to evaluation team’s work. B. SCHEDULING The estimated time period for undertaking this evaluation is 70 working days, of which at least 40 days will be spent in different DISCOs in all four provinces of Pakistan. The start date will be determined in consultations with USAID/ Pakistan Mission. The evaluation team is required to work six days a week. The team is required to travel to selected provinces in each region where program activities are being implemented. At least 50 percent of the consultants’ time will be spent outside Islamabad to conduct interviews with DISCO officials, project staff, and the public. The evaluation team will prepare and submit a draft report 24 hours in advance of the exit briefing and presentation of the findings, which it will deliver to USAID Mission. Comments from USAID will be incorporated before the submission of the final draft. The evaluation should be completed by the end of October in order to incorporate the findings in PDP workplan and M&E Plan. 64 B. BUDGETING Level of Effort (LOE) in Days: Activity Expat Team Leader Expat Evaluation Specialist Expat Evaluation Specialist Pakistani Evaluation Specialist Pakistani Evaluation Specialist Document review, work plan, draft questions, data analysis plan, suggested list of interviewees, finalized questions for the survey 3 3 3 3 3 In-briefing with USAID 1 1 1 1 1 Interviews in Islamabad 4 4 4 4 4 Interviews or survey work in provinces 20 30 30 40 40 Mid-term briefing and interim meetings with USAID 2 2 2 2 2 Data analysis, preliminary report and presentation preparation 10 10 10 10 10 Draft evaluation report 6 6 6 6 6 Final exit presentation to USAID (with PowerPoint presentation and draft evaluation report) 1 1 1 1 1 Final evaluation report 2 2 2 2 2 Totals 49 59 59 69 69 65 VI. EVALUATION DELIVERABLES 1. Evaluation Work Plan: During the Team Planning Meeting (TPM), the evaluation team will prepare a detailed Work Plan which will include the methodologies to be used in the evaluation. The Work Plan will be submitted to the COR at USAID/Pakistan for approval no later than the sixth day of work. USAID will share the revised work plan with GOP for comment, as needed, and will revise accordingly. The initial work plan will include (a) the overall evaluation design, including the proposed methodology, data collection and analysis plan, and data collection instruments; (b) a list of the team members indicating their primary contact details while in-country, including the e-mail address and mobile phone number for the team leader; and (c) the team’s proposed schedule for the evaluation. The revised work plan shall include the list of potential interviewees, sites to be visited, and evaluation tools. 2. Discussion of Preliminary Draft Evaluation Report: The Team will submit a draft report to the USAID COR and ENR Office team, who will provide preliminary comments prior to final Mission debriefing. This will facilitate preparation of a more final draft report that will be left with the Mission upon the evaluation team’s departure. 3. Debriefing with USAID: The team will present the major findings of the evaluation to UAID/Pakistan by means of a PowerPoint presentation after submission of the draft report and before the team’s departure from the country (if the evaluation team is based in Pakistan, this ‘departure’ obviously does not apply). The debriefing will include a discussion of achievements and activities only, with no recommendations for possible modifications to project approaches, results, or activities. The team will consider USAID comments and revise the draft accordingly, as appropriate. 4. Debriefing with IRG: The team will present the major findings of the evaluation to IRG through a PowerPoint presentation. The debriefing will include a discussion of achievements and activities only, with no recommendations for possible modifications to project approaches, results, or activities. The team will consider IRG comments and revise the draft report accordingly, as appropriate. 5. Draft Evaluation Report: The content of the draft evaluation report is outlined in Annex 1, and all formatting shall be consistent with the USAID branding guidelines. The focus of the report is to answer the evaluation questions and may include factors the team considers to have a bearing on the objectives of the evaluation. Any such factors can be included in the report only after consultation with USAID. The draft evaluation report will be submitted by the evaluation team leader to PMU 24 hours in advance of the exit briefing for review and comments by USAID. USAID’s PMU and ENR office will have ten calendar days in which to review and comment and PMU shall submit all comments to the evaluation team leader. 66 6. Final Evaluation Report: The final evaluation report will incorporate final comments provided by the PMU. USAID comments are due within ten days after the receipt of the initial final draft. The final report should be submitted to the PMU within three days of receipt of comments by the evaluation team leader. All project data and records will be submitted in full and shall be in electronic form in easily readable format; organized and fully document for use by those not fully familiar with the project or evaluation; and owned by USAID and made available to the public barring rare exceptions. 7. One-page briefer on key qualitative and quantitative findings and conclusions relative to the evaluation questions for each municipality is included in the evaluation’s scope— to be given to the appropriate municipal government, provincial government, and/or GOP representative(s), so that they have the opportunity to review evaluation findings and share them with the larger community. Each briefer shall be translated in English and Urdu and provided to PMU and the Energy Office. 67 SOW ANNEX 1: REPORT CONTENT (ADAPTED AGENCY STANDARD LANGUAGE) The evaluation report will follow standard guidelines as laid out in Appendix 1 of USAID’S Evaluation Policy and operationalized in ADS 203.3.1.8 (Documenting Evaluations), reproduced in Annex 2. The evaluation report will follow the structure given below (the section titles and order are illustrative):  Title page  Table of Contents;  Table of tables and figures;  List of acronyms  Acknowledgements or preface (optional);  Program summary  Map showing the location of program activities  Executive summary which will be 3-5 pages in length that summarizes key points (project purpose and background, key evaluation questions, methods, findings, etc.)  Introductory chapter;  The Development Problem and USAID’s Response (1-3 pages): This section will describe the development problem USAID wants to address. This will include USAID’s response to the problem, the development hypothesis and theory of change, results framework, and project implementation (including the current status of the project or activity);  Purpose of the evaluation and evaluation questions (1-2 pages): This section will include the purpose of the Study and state all questions;  Evaluation Design, Methodology and Limitations (1-3 pages): A written design which includes key questions, methods, main features of data collection instruments; an explanation of why these methods were chosen, with additional information in the annex as necessary; limitations of the methodology (e.g., selection bias, recall bias, unobservable differences between comparator groups, etc.), and how these have been accounted for; and data analysis plan with discussion relevant to the analysis;  Findings and Conclusions: This section will include the findings and conclusions related to each evaluation question. If there are a large number of findings, there will be a synthesis or summary of findings for each question that establishes the connection with the conclusions that follow. The conclusion must answer each evaluation question based on the evidence provided in the findings.  Recommendations: Based on the conclusions, this section must include actionable statements that can be implemented into the existing program or included into future program design. Recommendations are only valid when they specify who does what, and relate to activities over which the USAID program has control.  References; and  Annex  Evaluation Statement of Work  Evaluation Methods and Limitations  Table of evaluation question by data sources, collection and analysis methodologies 68  Data Collection Instruments (all survey instruments, questionnaires, discussion guides, checklists, etc)  Bibliography of Documents Reviewed  List of individuals and agencies contacted and places visited  Meeting notes of all key meetings with stakeholders.  Disclosure of Any Conflicts of Interest  Statement of Differences (only if applicable)  Evaluation Team Bios 69 SOW ANNEX 2: TABLE OF STUDY QUESTIONS BY DATA SOURCES, COLLECTION AND ANALYSIS METHODOLOGIES Data Collection Study Question Activities Covered Type of Answer/ Evidence Method Sources Sampling/ Selection Data Analysis Methods 1. How has the project achieved its planned results to date (estimated level of effort at 50 percent)? Explain the results and net effects of PDP activities, including any unintended (both positive and negative) consequences. 17 key activities identified by USAID related to: 6-commercial performance 1-Communications and outreach 2-Technical loss reduction 1-financial management 3-governance 2-HR and change management 2- Gender Descriptive  Description of overall trends, with illustrative examples  Objective assessment with respect to study question, with illustrative examples  Individual interviews  Focus groups  Survey  Review of background data and documents Interview participants  3-5 managers/ExE’s from each of 9 DISCOs-number of DISCOs varying by activity (estimated total = 60-100 interviews) Focus groups (15 FGDs with 10- 12 participants each):  Planning engineering and modernization trainees -45 total participants (4 FGDs with 10-12 participants each)  Energy conservation campaign participants- 35 total from (reached with 3 regional FGDs)  Participants in utility exchange program-Reached with 4 focus groups (45 participants total)  Gender equity training- (45 participants from 8 discos in 4 FGDs) Survey participants  Survey of demand side management (156 participants, 3 from each DISCO)  Survey of linesmen/ trainers trained by PDP (2241 potential survey participants)  Purposive sampling for selection of interview and focus group participants  Random sample of training participants for survey  Identification of trends and themes across data sources, noting patterns by disaggregates  Linking primary qualitative data findings to existing data and available documents 2. What are the prospects for sustainability of the results achieved thus far; which results appear to be less sustainable? e.g All 17 activities (as above) Descriptive  Description of overall trends, with illustrative examples  Individual interviews  Focus groups  Survey  Review of Interview participants  3-5 managers/ExE’s from each of 9 DISCOs-number of DISCOs varying by activity (estimated total = 60-100  Purposive sampling for selection of interview and focus group  Identification of trends and themes across data sources, and noting patterns 70 SOW ANNEX 2: TABLE OF STUDY QUESTIONS BY DATA SOURCES, COLLECTION AND ANALYSIS METHODOLOGIES Data Collection Study Question Activities Covered Type of Answer/ Evidence Method Sources Sampling/ Selection Data Analysis Methods revenue increase activity.  Objective assessment with respect to study question, with illustrative examples background data and documents interviews) Focus groups (15 FGDs with 10- 12 participants each):  Planning engineering and modernization trainees -45 total participants (4 FGDs with 10-12 participants each)  Energy conservation campaign participants- 35 total from (reached with 3 regional FGDs)  Participants in utility exchange program-Reached with 4 focus groups (45 participants total)  Gender equity training- (45 participants from 8 discos in 4 FGDs) Survey participants  Survey of demand side management (156 participants, 3 from each DISCO)  Survey of linesmen/ trainers trained by PDP (2241 potential survey participants) participants  Census sample of all training participants for survey by disaggregates  Linking primary qualitative data findings to existing data and available documents 3. Did the project made any difference in the “governance” element of the power sector entities such as MWP, NEPRA, DISCOs? What are some of the accomplishments? What areas still need extensive work? 3 governance activities: 1-Assistance to NEPRA 2-Cost of service studies (for phases 2 and 3) Descriptive  Description of overall trends, with illustrative examples  Objective assessment with respect to study question, with illustrative  Focus groups  Literature review of project docs  Individual interviews  3-5 managers/ExE’s from each of 9 DISCOs-number of DISCOs varying by program (estimated total = 60-100 interviews)  7 interviews with NEPRA on specific governance assistance activities  Focus groups or interviews with 45 managers from 9  Purposive sampling for selection of interview and focus group participants  Identification of trends and themes across data sources, and noting patterns by disaggregates  Linking primary qualitative data findings to existing data and 71 SOW ANNEX 2: TABLE OF STUDY QUESTIONS BY DATA SOURCES, COLLECTION AND ANALYSIS METHODOLOGIES Data Collection Study Question Activities Covered Type of Answer/ Evidence Method Sources Sampling/ Selection Data Analysis Methods examples DISCOs. available documents 4. How valid is the current project design development theory and framework. Identify additional approaches or activities recommended, if any, to achieve the program objectives. All 17 activities Descriptive  Description of overall trends, with illustrative examples  Objective assessment with respect to study question, with illustrative examples  Individual interviews  Focus groups  Survey  Review of background data and documents Interview participants  3-5 managers/ExE’s from each of 9 DISCOs-number of DISCOs varying by activity (estimated total = 60-100 interviews) Focus groups (15 FGDs with 10- 12 participants each):  Planning engineering and modernization trainees -45 total participants (4 FGDs with 10-12 participants each)  Energy conservation campaign participants- 35 total from (reached with 3 regional FGDs)  Participants in utility exchange program-Reached with 4 focus groups (45 participants total)  Gender equity training- (45 participants from 8 discos in 4 FGDs) Survey participants  Survey of demand side management (156 participants, 3 from each DISCO)  Survey of linesmen/ trainers trained by PDP (2241 potential survey participants)  Purposive sampling for selection of interview and focus group participants  Census sample of all training participants for survey  Identification of trends and themes across data sources, and noting patterns by disaggregates  Linking primary qualitative data findings to existing data and available documents 72 SOW ANNEX 2: TABLE OF STUDY QUESTIONS BY DATA SOURCES, COLLECTION AND ANALYSIS METHODOLOGIES Data Collection Study Question Activities Covered Type of Answer/ Evidence Method Sources Sampling/ Selection Data Analysis Methods 5. How has the project performed in increasing women’s participation in the energy sector? 3 gender activities 1-internship progam 1-Energy conservation campaigns in girls colleges 1-Gender equity training Descriptive  Description of overall trends, with illustrative examples  Objective assessment with respect to study question, with illustrative examples  Individual interviews  Focus groups  Survey  Review of background data and documents Individual interviews  Interviews with 10 participants, 5 each in the internship programs at MEPCO and PESCO + 6-10 interviews with Managers (3-5 from MEPCO and 3-5 from PESCO) Focus groups  3 FGDs with 35 teachers (3-5 from each of 7 DISCOs) that helped organize women that participated in energy conservation campaigns in girls colleges.  4-8 FGDs or Group Interviews with Participants in the gender equity training (10-12 participants per FGD)  Purposive sampling for selection of interview and focus group participants  Identification of trends and themes across data sources, and noting patterns by disaggregates  Linking primary qualitative data findings to existing data and available documents 73 SOW ANNEX 3: REPORTING GUIDELINES (AGENCY STANDARD LANGUAGE) According to ADS 203.3.1.8 (Documenting Evaluations), evaluation reports must meet the following criteria: 1. Evaluation reports must represent a thoughtful, well-researched, and well-organized effort to objectively evaluate what worked in the project, what did not work, and why. 2. Evaluation reports must address all evaluation questions included in the Statement of Work. The evaluation report should include the evaluation statement of work as an annex. The technical officer (who is the COR when the evaluation is conducted by a contractor) must agree upon, in writing, all modifications to the statement of work, whether in technical requirements, evaluation questions, evaluation team composition, methodology or timeline. 3. Evaluation methodology must be explained in detail and all tools used in conducting the evaluation such as questionnaires, checklists, and discussion guides will be included in an annex in the final report. 4. When evaluation findings address outcomes and impact, they must be assessed on males and females. 5. Limitations to the evaluation must be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.). 6. Evaluation findings must be presented as analyzed facts, evidence, and data and not based on anecdotes, hearsay, or simply the compilation of people’s opinions. Findings should be specific, concise, and supported by strong quantitative or qualitative evidence. 7. Sources of information must be properly identified and listed in an annex. 8. Recommendations must be supported by a specific set of findings and should be action-oriented, practical and specific, with defined responsibility for the action 74 SOW ANNEX 4: LIST OF AVAILABLE DOCUMENTS A summary of the documents available to the evaluation team is listed below. A complete list of documents that will be reviewed by the evaluation team will be provided prior to the first evaluation meeting. Some baseline data is available for selected indicators. Program Documents  PDP contract  PDP work plan  PDP M&E Plan (PMP) including PIRS  Annual and quarterly reports  Technical assessments and studies  Audit report of PDP GOP Documents:  GOP National Power Policy 2013  State of Industry Report by NEPRA  Tariff determinations and petitions on NEPRA’s website PDP Work Plan is available both at the Mission and at IRG’s PDP Office in Islamabad (Sector F￾6/2, Street 19, House # 23). PDP performance indicators are described in the Project M&E plan. The M&E plan includes key performance indicators for each key process area, along with indicator definitions, rationales, data sources, collection frequencies and targets. PDP has created Performance Indicator Reference Sheets (PIRS) for each indicator in the M&E plan. The PIRS is a summary resource that describes each indicator in detail and includes information on indicator definitions, units of measurement, data sources, data collection methods, collection and reporting frequency, persons responsible for data coaction, data analysis methods, data quality and data safety procedures, and performance targets. PDP has established a Monitoring and Evaluation (M&E) system that tracks and reports the results of selected performance indicators. The M&E unit has baseline data, quarterly, annual and special reports as well as audit reports. 75 SOW ANNEX 5: MSF DO1 RESULTS FRAMEWORK DO 1: Increased Sustainable Energy Supplied to the Economy a) gigawatt-hours (GW∙h) of energy sold b) hours of unplanned load shedding c) tariff differential subsidy IR 1.1.1: Increased Generation and Transmission Capacity a) megawatts (MW) of electrical power added or saved as a result of United States Government supported construction, rehabilitation, and other generation and transmission improvements b) efficiency of thermal power plants (British thermal units of input heat energy per kilowatt-hour of electrical output energy (Btu/kW∙h)) c) number of United States Government supported installations and operations and maintenance improvements of generation plants and transmission networks d) number of transmission bottlenecks resolved IR 1.2.1: Improved Policy Implementation a) number of key policies and regulations in development stages of analysis, drafting, stakeholder consultation, legislative review, approval, or implementation as a result of United States Government assistance b) TBD IR 1.1.2: Improved Efficiency of Consumption and the Distribution Systems a) megawatts (MW) of electrical power saved as a result of United States Government support to distribution companies b) number of installations and operations and maintenance improvements IR 1.1.3: Increased Financial Sustainability of Power Supply a) revenue at distribution companies b) number of days in the distribution company billing and collections cycle c) number of days for fuel adjustment process IR 1.2.2: More Autonomous Energy Sector Entities a) number of policies following international best practices developed and implemented b) number of board recommendations following international best practices implemented by public sector entities IR 1.2.3: Improved Capacity of USAID-Supported Energy Public-Sector Entities a) a) number of best practice-driven systems created, improved, and implemented IR 1.2.4: Increased constructive civil society engagement in the energy sector a) number of civil society organizations receiving United States Government assistance engaged in advocacy interventions (energy disaggregate of the democracy and governance indicator) b) number of public forums resulting from United States Government assistance in which government officials and citizens interact IR 1.1: Increased Energy Supply a) number of beneficiaries with improved energy services due to United States Government assistance (4.4.1-31) b) gigawatt-hours (GW∙h) of energy availability c) power (megawatts) available to meet power sector demand as result of IR 1.2: Improved Energy Sector Governance a) number of positive responses from citizens surveyed on transparency in the energy sector in Pakistan b) percent change in the gross annual accumulation of circular debt IR 1.1.4: Increased Non-USG Investment in the Energy Sector a) public and private funds leveraged by the United States Government for energy infrastructure projects (alternative F indicator 4.4.1-32) 76 ANNEX II: BIBLIOGRAPHY  Task Order USAID/Pakistan Power Distribution Improvement Program (PDIP).  Modifications in Task Order USAID/Pakistan Power Distribution Improvement Program (PDIP)  PDP Performance Management Plan (PMP), February, 2014  PDP Annual Report, 1 October 2012 – 30 September 2013.  PDP Quarterly Reports, Financial Year 2013 and 2014.  PDP Annual Work Plan, Financial Year 2013 and 2014.  Operational Audit Reports of FESCO, GEPCO, HESCO, IESCO, LESCO, MEPCO, PESCO and QESCO.  Performance Improvement Action Plans of FESCO, GEPCO, HESCO, IESCO, LESCO, MEPCO, PESCO and QESCO.  Project Overview of Planning & Engineering (P&E) Based on GIS Mapping.  Gallup Pakistan Final Report of Energy Conservation Survey: 2012 dated 22 November 2012.  Technical & Operational Handbook on Hand-Held Unit Application used for Meter reading.  User Functional Requirements, Customer Information System (CIS), 28 March 2012.  User’s Manual to the Fully Allocated Cost of Service Study, August, 2013.  ERP Documentation Manual for Power Distribution Companies (DISCOs) in Pakistan, 12 July 2012.  State of Industry Report 2013, National Electric Power Regulatory Authority (NEPRA).  National Power Policy, 2013, Government of Pakistan.  Gender Strategy – Gender Equity Training Needs Assessment Report, May 2013.  Gender Strategy – Gender Equity Training Report for Power Distribution Companies of Pakistan, May, 2013.  Gender Strategy – Gender Equity Training Post Impact Assessment Report, August 2013.  Narrative on Commercial Procedure Optimization Project (CPOP) / Improved Meter Reading (IMR).  Narrative on Policy Development and Implementation for DISCOs.  Narrative on Lineman Training Program.  Narrative on Methodology for Calculating MW & Energy Saved by Installation of LT Capacitors.  Narrative on Methodology for Calculating MW & Energy Saved by Installation of HT Capacitors.  Narrative on Methodology for Calculating MW & Energy Saved by Installation of Pumps and Motors. 77 ANNEX III: DATA COLLECTION INSTRUMENTS Power Distribution Program (PDP) Interim Performance Evaluation Instrument for Group Interviews with IRG/PDP Management (Overall Program) Introduction Thank you very much for meeting us today. My name is __________________ and I represent the USAID/ Pakistan Monitoring and Evaluation Program (MEP). This program is being implemented by Management Systems International, an international consulting firm. The program helps the USAID/ Pakistan Mission monitor and evaluate a range of its programs, including those in the areas of science, innovation and higher learning. USAID/ Pakistan has contracted MSI to conduct an interim evaluation of the USAID Power Distribution Program (PDP) and systematically assess the effectiveness of PDP’s interventions to improve the performance of the DISCOs and the governance and policy environment of public sector entities in the energy arena. We would like to ask you a few questions about the program and Mr./Ms. _____________________ will take notes. We will need to cover several aspects of PDP program with your help. Confidentiality With your permission, we would like to record this discussion so that we accurately capture your feedback and do not miss any important points. Please be assured that your responses will be kept confidential. In case we use quotations from this interview in our evaluation report, you will not be identified by name, or official title, but in general terms as a researcher, official or manager. Group Interview Code: XXX Number of Participants Date and time of Interview Interview Location: Component/Activities covered through this Interview: Commercial Performance: 1. Customer Information System- Phase II 2. Customer Information System- Phase III 3. Electronic Metering 4. Hand-Held Units 5. Hand-Held Units and Improved Meter Reading 6. Automatic Meter Readers Governance: 7. Assistance to NEPRA (Multiple Activities) 8. Cost of Service Study – Phase II 9. Cost of Service Study – Phase III Communication and Outreach with Consumers: 10. Outreach Activities and Anti-theft Campaigns Human Resources and Change Management: 11. Lineman Training, Tools, and Training Aids 12. Utility Exchange Program Technical Loss Reduction: 13. Planning and Engineering Modernization (GIS Mapping, System Analysis, and Training) 14. Demand-side Management (Industrial Motors) Gender: 15. Energy Conservation Campaigns in Girls Colleges 16. Gender Equity Training Financial Management: 17. Enterprise Resource Planning (ERP) Implementation (development of ERP manual and training in nine DISCOs) 78 QUESTIONS: Overview of Implementation: 1. Please give us an overview of design and implementation experience of PDP. 2. Please comment on GoP’s buy-in and interaction with counterparts for design and implementation of PDP activities. Implementation Design: 3. What were the key considerations that informed the design of PDP? 4. What is your opinion about choice of alternatives selected to design PDP activities with regard to intended results? 5. What were the anticipated and unanticipated risks that materialized during implementation? 6. What are some of the important lessons - that can be drawn from your experience from PDP activities – to inform future intervention in power distribution sector? Expectations and Results: 7. How has different component helped in achieving its planned results? (Prompts: Contribution of PDP deliverables to achievement of target results for the recipient of assistance). 8. In what significant ways were the activities expected to change, improve or strengthen processes, practices, capacities of DISCOs and other GoP entities? 9. What are the most significant achievements of PDP? 10. What challenges were in achieving program’s intended results? Sustainability: 11. To what extent was there buy-in from DISCOs and GOP agencies on PDP activities and deliverables? 12. Please tell us about instances where buy-in was difficult or was not achieved. Are there any lessons and way forward on weak buy-in? 13. Which GoP entities plan to continue to finance these activities without donor assistance? 14. What are the long-term changes resulted from implementation of these activities? (Such as changes in policies to institutionalize these changes). Governance: 15. Are there any examples of performance improvement as result of strengthening of governance in power distribution sector? 16. Please identify and describe policy change as a direct or indirect result of PDP activities. (Prompts: enactments; amendments in regulation, laws or rules; improvements in processes and practices adopted by executive decisions). 17. What are some of challenges that need to be addressed to improve governance of power distribution? Gender: 18. In what ways have PDP activities promoted inclusion of women in the power sector? 19. What were the main gender related issues identified in the GoP entities (NEPRA, MWP and DISCOs)? 20. What further interventions are planned to improve gender equity in the power sector? 79 Power Distribution Program (PDP) Interim Performance Evaluation Instrument for Group Interviews with IRG/PDP Activity Managers Introduction Thank you very much for meeting us today. My name is __________________ and I represent the USAID/ Pakistan Monitoring and Evaluation Program (MEP). This program is being implemented by Management Systems International, an international consulting firm. The program helps the USAID/ Pakistan Mission monitor and evaluate a range of its programs, including those in the areas of science, innovation and higher learning. USAID/ Pakistan has contracted MSI to conduct an interim evaluation of the USAID Power Distribution Program (PDP) and systematically assess the effectiveness of PDP’s interventions to improve the performance of the DISCOs and the governance and policy environment of public sector entities in the energy arena. We would like to ask you a few questions about the program and Mr./Ms. _____________________ will take notes. We will need to cover several aspects of PDP program with your help. Confidentiality With your permission, we would like to record this discussion so that we accurately capture your feedback and do not miss any important points. Please be assured that your responses will be kept confidential. In case we use quotations from this interview in our evaluation report, you will not be identified by name, or official title, but in general terms as a researcher, official or manager. Group Interview Code: XXX Number of Participants Date and time of Interview Interview Location: Component/Activities covered through this Interview: Commercial Performance: 1. Customer Information System- Phase II 2. Customer Information System- Phase III 3. Electronic Metering 4. Hand-Held Units 5. Hand-Held Units and Improved Meter Reading 6. Automatic Meter Readers Governance: 7. Assistance to NEPRA (Multiple Activities) 8. Cost of Service Study – Phase II 9. Cost of Service Study – Phase III Communication and Outreach with Consumers: 10. Outreach Activities and Anti-theft Campaigns Human Resources and Change Management: 11. Lineman Training, Tools, and Training Aids 12. Utility Exchange Program Technical Loss Reduction: 13. Planning and Engineering Modernization (GIS Mapping, System Analysis, and Training) 14. Demand-side Management (Industrial Motors) Gender: 15. Energy Conservation Campaigns in Girls Colleges 16. Gender Equity Training Financial Management: 17. Enterprise Resource Planning (ERP) Implementation (development of ERP manual and training in nine DISCOs) 80 QUESTIONS: Overview of Implementation: 1. Please give us an overview of your experience of the design and implementation of this activity at various DISCOs and other entities (NEPRA, Industrial Units and Girls Colleges etc.). 2. Please comment on buy-in and participation from stakeholders for design and implementation of this activity. Implementation Design: 3. How did this activity support transition from previous implementation to new approaches in order to achieve intended results? 4. What were the challenges in achieving the objectives of PDP? 5. How do you think this activity could have been implemented differently? Expectations and Results: 6. How has this activity helped in achieving its planned results? (Prompts: Change in unplanned load shedding, reduction in line losses, reduction in power theft, increase in capacity to meet demand, increase in electricity available to consumers and reduction in line men incidents and fatality). 7. In what ways this activity was expected to change the processes, practices and capacities in its functional areas? 8. What feedback have you received about this activity from the beneficiaries? 9. What were the challenges in achieving the expected results of this activity? 10. How were you able to address these challenges to achieve the desired results? Sustainability: 11. How have stakeholders and decision makers taken ownership of this activity and its results? 12. What are the prospects for continuation and/or follow up of this activity without support from PDP and other donor? 13. What are the long-term changes resulted from implementation of this activity? (such as changes in policies to institutionalize these changes). Governance: 14. How has the governance of the host organizations changed as a result of implementation of this activity? (Please give examples) 15. What are the challenges that need to be addressed to improve governance related to your functional areas in host organizations? Gender: 16. In what ways have this activity helped addressing gender related issues in beneficiary organizations? 17. How are the female staff involved in implementation of this activity? 18. How has this activity resulted in encouraging women’s participation in the power sector? 81 Power Distribution Program (PDP) Interim Performance Evaluation Instrument for Group Interviews with DISCO Staff on i) Electronic Metering ii) Hand-Held Units and Improved Meter Reading iii) Automatic Meter Readers under Commercial Performance component Introduction Thank you very much for meeting us today. My name is __________________ and I represent the USAID/ Pakistan Monitoring and Evaluation Program (MEP). This program is being implemented by Management Systems International, an international consulting firm. The program helps the USAID/ Pakistan Mission monitor and evaluate a range of its programs, including those in the areas of science, innovation and higher learning. USAID/ Pakistan has contracted MSI to conduct an interim evaluation of the USAID Power Distribution Program (PDP) and systematically assess the effectiveness of PDP’s interventions to improve the performance of the DISCOs and the governance and policy environment of public sector entities in the energy arena. We would like to ask you a few questions about the program and Mr./Ms. _____________________ will take notes. We will need to cover several aspects of PDP program with your help. Confidentiality With your permission, we would like to record this discussion so that we accurately capture your feedback and do not miss any important points. Please be assured that your responses will be kept confidential. In case we use quotations from this interview in our evaluation report, you will not be identified by name, or official title, but in general terms as a researcher, official or manager. Note for the interviewer: Objectives: In C-2 PDP aimed to focus on installing new electronic meters and re-fixing customer services—mainly to replace antiquated, broken and inaccurate electro-mechanical meters. Hand-Held Units (HHUs) were also provided at specific subdivisions to have improved control over meter readings. Under C-2 PDP also installed Automatic Meter Reading (AMR) at distribution feeders. Installation of AMR, removes the human factor from the meter reading process, thereby eliminating the opportunity for corruption and increasing the accuracy of customer billings. DISCOs can closely monitor the distribution feeders where automatic meters are installed, which will allow them to measure directly the effect of this intervention. Power savings resulting from the AMR intervention is measured in KWh savings and increased revenue. Group Interview Code: XXX Number of Participants Date and time of Interview Interview Location: 82 QUESTIONS: Overview of Implementation: 1. How has this activity (HHU/AMR/EM/IMR) been developed and implemented at your DISCO? 2. Describe the engagement of your DISCO staff in implementation of this activity (HHU/AMR/EM/IMR)? Implementation Design: 3. How has the program introduced improved technologies, practices and procedures to implement (HHU/AMR/EM/IMR)? 4. Please describe how the knowledge and skills learned under IMR will be adopted by all meter readers. (Only for IMR participants). 5. What issues were faced during the implementation of this activity? 6. What could be improved in the design of this activity (HHU/AMR/EM/IMR) to improve its effectiveness? Expectations and Results: 7. What were your expectations from implementation/installation of this activity (HHU/AMR/EM/IMR)? 8. What changes did you observe before and after the (HHU/AMR/EM/IMR) was implemented in your DISCO? 9. What feedback about the utility of (name of component/activity) have you received from your staff? 10. Please describe your experience with the operation and performance of the meters/units. 11. Please describe how are you using data from HHU and AMR for billing purposes. 12. What were the challenges of the (HHU/AMR/EM/IMR) in achieving its expected results? 13. How can (HHU/AMR/EM/IMR) be improved to better achieve its planned results? 14. How do you think that the use of HHU and AMR has changed billing errors and revenue assurance? (Prompt: estimated financial gains). 15. How do you think that the HHUs and AMR has resulted in reduction of non-technical losses, please describe how? (Prompt: error in recording reduced). Sustainability: 16. What are the prospects for sustainability are there for the maintenance and use of (HHU/AMR/EM/IMR) after support from IRG is terminated? (repair and maintenance, expansion of technology to remaining network areas) Governance: 17. How has the governance of the DISCO changed as a result of implementation of (HHU/AMR/EM/IMR) and other PDP activities? (Prompt: reduction in billing errors and disputes and reduction of non-technical losses) Gender: 18. How are the female staff involved in implementation of (HHU/AMR/EM/IMR)? 19. How can gender equity be further enhanced in implementation of this activity? 83 Power Distribution Program (PDP) Interim Performance Evaluation Instrument for Group Interviews with DISCO Staff on Customer Information System (CIS)- Phase II and III under Commercial Performance component Introduction Thank you very much for meeting us today. My name is __________________ and I represent the USAID/ Pakistan Monitoring and Evaluation Program (MEP). This program is being implemented by Management Systems International, an international consulting firm. The program helps the USAID/ Pakistan Mission monitor and evaluate a range of its programs, including those in the areas of science, innovation and higher learning. USAID/ Pakistan has contracted MSI to conduct an interim evaluation of the USAID Power Distribution Program (PDP) and systematically assess the effectiveness of PDP’s interventions to improve the performance of the DISCOs and the governance and policy environment of public sector entities in the energy arena. We would like to ask you a few questions about the program and Mr./Ms. _____________________ will take notes. We will need to cover several aspects of PDP program with your help. Confidentiality With your permission, we would like to record this discussion so that we accurately capture your feedback and do not miss any important points. Please be assured that your responses will be kept confidential. In case we use quotations from this interview in our evaluation report, you will not be identified by name, or official title, but in general terms as a researcher, official or manager. Group Interview Code: XXX Number of Participants Date and time of Interview Interview Location: QUESTIONS: Overview of Implementation: 1. How has Customer Information Service (CIS) been developed and implemented at your DISCO? Can you please describe the work completed under CIS1 and CIS2? Implementation Design: 2. Please describe the extent of CIS developed and implemented at your DISCO. 3. Please describe your DISCO’s involvement/inputs to the development of the CIS. 4. What could be improved in the implementation of the CIS to improve its effectiveness? Expectations and Results: 5. Please describe your understanding of the purpose and outcomes, and benefits of the CIS, during design stage? 6. What changes did you observe before and after the CIS was implemented in your DISCO with the support of PDP? 7. How has the CIS helped in achieving its planned results? How do the results match with your expectations? 8. What feedback about the utility of CIS the CIS have you received from your staff? 9. Describe any noticeable changes in approach to work as a result of CISCIS 84 10. What were the challenges of the CIS in achieving its expected results? Sustainability: 11. How are the DISCO staff operating and maintaining the CIS? Or have outsourced it? 12. How do you plan using the CIS after support from IRG is terminated? 13. What measures have been taken to institutionalize CIS at your DISCO? What was the outcome of those measures? (Prompts: maintenance, trouble shooting, regular updates and upgrades etc.). Governance: 14. How has the governance of the DISCO changed as a result of implementation of CIS and other PDP activities? 15. What role has CIS has played in improving governance at your DISCO? What efforts are required to improve governance in your DISCO? Gender: 16. How has CIS resulted in encouraging women’s participation in the power sector? 17. How are the female staff involved in implementation of CIS? 18. How female workers can contribute greater role in CIS? 85 Power Distribution Program (PDP) Interim Performance Evaluation Instrument for Group Interviews with DISCO Staff on Outreach Activities and Anti-theft Campaigns under Communication and Outreach with Consumers Introduction Thank you very much for meeting us today. My name is __________________ and I represent the USAID/ Pakistan Monitoring and Evaluation Program (MEP). This program is being implemented by Management Systems International, an international consulting firm. The program helps the USAID/ Pakistan Mission monitor and evaluate a range of its programs, including those in the areas of science, innovation and higher learning. USAID/ Pakistan has contracted MSI to conduct an interim evaluation of the USAID Power Distribution Program (PDP) and systematically assess the effectiveness of PDP’s interventions to improve the performance of the DISCOs and the governance and policy environment of public sector entities in the energy arena. We would like to ask you a few questions about the program and Mr./Ms. _____________________ will take notes. We will need to cover several aspects of PDP program with your help. Confidentiality With your permission, we would like to record this discussion so that we accurately capture your feedback and do not miss any important points. Please be assured that your responses will be kept confidential. In case we use quotations from this interview in our evaluation report, you will not be identified by name, or official title, but in general terms as a researcher, official or manager. Group Interview Code: XXX Number of Participants Date and time of Interview Interview Location: QUESTIONS: Overview of Implementation: 1. How has the outreach and anti-theft campaign been developed and implemented at your DISCO? 2. What other similar activities has your DISCO conducted previously? 3. How have DISCO staff been trained to design, develop and implement outreach material? Implementation Design: 4. How have customers responded to DISCOs Outreach Activities and Anti-theft campaigns? 5. Please describe the engagement of DISCO staff in these activities (third party implementers?) 6. What issues did you face during the implementation of this activity with IRG? Expectations and Results: 7. What were your expectations when IRG engaged you into this activity? 8. What results have been achieved as a result of implementation of this activity? (theft reduction, customer satisfaction, customer relationships) 9. What feedback about the utility of (name of component/activity) have you received from your staff and target consumers? 86 10. How has this activity helped in achieving its planned results? (Prompts: Change in unplanned load shedding, reduction in line losses, reduction in power theft, increase in capacity to meet demand, increase in electricity available to consumers and reduction in line men incidents and fatality). 11. What were the challenges of the (name of component/activity) in achieving its expected results? 12. How could the Anti-theft Campaigns been designed and implemented differently to better achieve their planned results? Sustainability: 13. How do you plan to continue conducting anti-theft activities, after the close of PDP intervention? Gender: 14. How are the female staff involved in implementation of (name of component/activity)? 15. How has this activity resulted in encouraging women’s participation in the power sector? 16. How can gender equity be further enhanced in implementation of this activity? 17. How has your DISCO considered engaging more female workers for these types of activities? 87 Power Distribution Program (PDP) Interim Performance Evaluation Instrument for Group Interviews with DISCO Staff on Planning and Engineering Modernization (GIS Mapping, System Analysis, and Training) under Technical Loss Reduction Introduction Thank you very much for meeting us today. My name is __________________ and I represent the USAID/ Pakistan Monitoring and Evaluation Program (MEP). This program is being implemented by Management Systems International, an international consulting firm. The program helps the USAID/ Pakistan Mission monitor and evaluate a range of its programs, including those in the areas of science, innovation and higher learning. USAID/ Pakistan has contracted MSI to conduct an interim evaluation of the USAID Power Distribution Program (PDP) and systematically assess the effectiveness of PDP’s interventions to improve the performance of the DISCOs and the governance and policy environment of public sector entities in the energy arena. We would like to ask you a few questions about the program and Mr./Ms. _____________________ will take notes. We will need to cover several aspects of PDP program with your help. Confidentiality With your permission, we would like to record this discussion so that we accurately capture your feedback and do not miss any important points. Please be assured that your responses will be kept confidential. In case we use quotations from this interview in our evaluation report, you will not be identified by name, or official title, but in general terms as a researcher, official or manager. Group Interview Code: XXX Number of Participants Date and time of Interview Interview Location: QUESTIONS: Overview of Implementation: 1. Please describe details of implementation of activities associated with GIS Mapping, System Analysis, and Training, at your DISCO 2. Describe the level of engagement of your staff in the design and implementation of these trainings? Implementation Design: 3. Describe the process of implementation of these activities. Describe your level of satisfaction with implementation approach? 4. How would describe the quality of the training in the field of GIS and System Analysis? (Prompt: ability to perform GIS operations and system analysis) 5. How the design and implementation of these trainings could have been improved? 6. What were the obstacles faced in transition from a manual system to a GIS system in the selected areas? How these challenges were addressed? Expectations and Results: 7. How far do you think the productivity and efficiency of the system has improved because of GIS adoption? 8. How far this activity has helped you in the fault analysis and day to day operational problems? 88 9. How far are you utilizing this facility in system augmentation and load flow studies? 10. How far adoption of the GIS facility has resulted in cost containment at your DISCO? 11. How this activity has helped you in load management? 12. What proportion of your network in covered by this system and to what extent it is being used for the intended purposes? (Prompt: being used for planning, trouble shooting, maintenance and operations). Sustainability: 13. Describe the initiatives your DISCO has taken to ensure continuity of use of resources developed under this activity, after conclusion of PDP? Governance: 14. How have the newly acquired skills and knowledge resulting from this activity, improved governance at your DISCO? (Prompt: improved monitoring, improved detection of non-technical losses, better planning). Gender: 15. How are GIS and other technologies promising to encourage and empower women in power sector? 16. What has been the participation of female workers in this activity? 89 Power Distribution Program (PDP) Interim Performance Evaluation Instrument for Group Interviews with DISCO Staff on Enterprise Resource Planning (ERP) Implementation under Financial Management component Introduction Thank you very much for meeting us today. My name is __________________ and I represent the USAID/ Pakistan Monitoring and Evaluation Program (MEP). This program is being implemented by Management Systems International, an international consulting firm. The program helps the USAID/ Pakistan Mission monitor and evaluate a range of its programs, including those in the areas of science, innovation and higher learning. USAID/ Pakistan has contracted MSI to conduct an interim evaluation of the USAID Power Distribution Program (PDP) and systematically assess the effectiveness of PDP’s interventions to improve the performance of the DISCOs and the governance and policy environment of public sector entities in the energy arena. We would like to ask you a few questions about the program and Mr./Ms. _____________________ will take notes. We will need to cover several aspects of PDP program with your help. Confidentiality With your permission, we would like to record this discussion so that we accurately capture your feedback and do not miss any important points. Please be assured that your responses will be kept confidential. In case we use quotations from this interview in our evaluation report, you will not be identified by name, or official title, but in general terms as a researcher, official or manager. Note for the interviewer: Objectives: In particular, DISCOs need to replace and modernize their legacy financial systems with modern Enterprise Resource Planning (ERP) systems so as to enhance the accuracy, accountability, transparency, and reliability of business data. Before an ERP system can be put into place, however, it is necessary to evaluate current business systems to determine the adaptations required for ERP implementation. Toward this end, has assisted the DISCOs by documenting current and future business processes and creating a roadmap for successful ERP implementation. In C-2, PDP produced a comprehensive business blueprint for ERP implementation and made available to all DISCOs. The ERP modules focused on financial, materials management, project management and payroll applications and can be used as a model for other platform applications as well. The project provided technical assistance to DISCOs for the implementation of financial ERP applications which consolidated the various financial reporting requirements of the organization. Group Interview Code: XXX Number of Participants: Date and time of interview: Interview Location: QUESTIONS: Overview of Implementation: 1. What were the main conclusions of the need assessments undertaken to inform the design of the ERP manual development activity? 2. What progress has been made on ERP implementation at your DISCO? What still needs to be done? Implementation Design: 90 3. What are the key issues in current financial management practices and how do you expect ERP implementation to address them? 4. How many trainings were conducted and how many of your colleagues participated? What was the feedback of the participants about the quality and usefulness of the training? Expectations and Results: 5. How have the ERP manual development and training activities helped in achieving outcomes related to improved financial management in DISCOs? 6. What significant changes has ERP implementation made to your financial management practices? 7. How did your DISCOs ERP activities support transition from the previous financial management system to the ERP based system? (Prompts: such as any change management plan or activity) 8. How did the trainings improve performance of financial management staff at the DISCO? 9. Please describe the measurable and attributable improvement in financial performance at your DISCO resulting from the ERP manual and trainings Sustainability: 10. What measures have been taken to institutionalize the ERP manual? 11. What next steps in ERP implementation are planned? How are those steps proposed to be financed? 12. Which actions or decisions of DISCOs or MWP best reflect their interest and ownership of ERP activities? Governance: 13. How has corporate governance of the DISCO changed as a result of implementation of ERP? What are the likely benefits of the ERP based information to decision makers and policy makers? 91 Power Distribution Program (PDP) Interim Performance Evaluation Instrument for Group Interviews with NEPRA Staff on Assistance to NEPRA (Multiple Activities) and Cost of Service Study – Phase II and III under Governance component Introduction Thank you very much for meeting us today. My name is __________________ and I represent the USAID/ Pakistan Monitoring and Evaluation Program (MEP). This program is being implemented by Management Systems International, an international consulting firm. The program helps the USAID/ Pakistan Mission monitor and evaluate a range of its programs, including those in the areas of science, innovation and higher learning. USAID/ Pakistan has contracted MSI to conduct an interim evaluation of the USAID Power Distribution Program (PDP) and systematically assess the effectiveness of PDP’s interventions to improve the performance of the DISCOs and the governance and policy environment of public sector entities in the energy arena. We would like to ask you a few questions about the program and Mr./Ms. _____________________ will take notes. We will need to cover several aspects of PDP program with your help. Confidentiality With your permission, we would like to record this discussion so that we accurately capture your feedback and do not miss any important points. Please be assured that your responses will be kept confidential. In case we use quotations from this interview in our evaluation report, you will not be identified by name, or official title, but in general terms as a researcher, official or manager. Notes for the interviewer: Objectives: PDP has designed the Cost of Service (CoS) component to specifically focus on the needs of the DISCOs, the role of NEPRA as the regulator, and the urgent need to implement cost reflective tariffs for the DISCOs. Under C-2, PDP developed an allocated CoS Model and held training workshops to train future users of the model. The model was customized for IESCO but can easily be revised for use by the other DISCOs. PDP is holding additional training workshops to train future users of the model at all DISCOs. Under C-3, PDP will undertake detailed CoS calculations for the rest of the 8 DISCOs (LESCO, FESCO, GEPCO, MEPCO, PESCO, HESCO, SEPCO and QESCO) and assist them in preparation of their tariff petition. Group Interview Code: XXX Number of Participants: Date and time of interview Interview Location: QUESTIONS: Overview of Implementation: 1. What methods were employed to undertake the cost of service study for your DISCO? (Prompts: such as data collection, verification, financial modeling etc.) 2. What were the main findings of these studies? What is your opinion about the usefulness of those findings? Implementation Design: 3. Please give an overview of interactions and knowledge sharing between CoS experts and your DISCOs. Expectations and Results: 92 4. What errors in billing have been detected as a result of cost of service study? (Prompt: i.e. $47 million identified nationally by all DISCO, method of calculation). 5. How is your DISCO using the cost of service study model? (Prompts: basis of tariff petitions, financial control, cost optimization etc.). 6. What do you consider important next steps in adoption of cost-reflective tariffs? 7. What changes did you observe before and after the Cost of Service Study was implemented in your DISCO with the support of PDP? 8. How does your DISCO plan to optimize cost structure and cost-linked tariffs? What do you foresee as major constraints in this area? Sustainability: 9. What further steps at DISCOs and NEPRA are planned on the basis of the Cost of Service Study? 10. What measures have been taken to adopt new procedures for cost management and tariff petitions on the basis of cost of service methodology at your DISCO? Governance: 11. What changes in DISCO’s operations and financial management practices are made/to be made on the basis of cost of service study? 12. What are the most important areas of improvement in financial management and governance that you feel need technical assistance? 13. What type of support is required to improve governance of financial management? Gender: 14. How is cost of service study likely to benefit low-income households? (poverty being a proxy for gender focus) 93 Power Distribution Program (PDP) Interim Performance Evaluation Instrument for Group Interviews with DISCO Staff on Lineman Training, Tools, and Training Aids under Human Resources and Change Management: Introduction Thank you very much for meeting us today. My name is __________________ and I represent the USAID/ Pakistan Monitoring and Evaluation Program (MEP). This program is being implemented by Management Systems International, an international consulting firm. The program helps the USAID/ Pakistan Mission monitor and evaluate a range of its programs, including those in the areas of science, innovation and higher learning. USAID/ Pakistan has contracted MSI to conduct an interim evaluation of the USAID Power Distribution Program (PDP) and systematically assess the effectiveness of PDP’s interventions to improve the performance of the DISCOs and the governance and policy environment of public sector entities in the energy arena. We would like to ask you a few questions about the program and Mr./Ms. _____________________ will take notes. We will need to cover several aspects of PDP program with your help. Confidentiality With your permission, we would like to record this discussion so that we accurately capture your feedback and do not miss any important points. Please be assured that your responses will be kept confidential. In case we use quotations from this interview in our evaluation report, you will not be identified by name, or official title, but in general terms as a researcher, official or manager. Notes for the Interviewer: Objectives: Linemen Training, Tools and Equipment is an essential intervention for transforming DISCOs front-line operations to match that of well-run utilities. Tools and equipment will be procured (in conjunction with the transportation intervention) and consolidated at central training locations. The DISCO will be required to provide some equipment, which has been determined to be of acceptable quality. Groups of line staff from the sub-divisions will be brought to the location and trained in the use and care of the tools. Improved customer satisfaction and decreased technical losses, are the direct result from the proper tooling and training of linemen as workmanship in line maintenance leads to improved system performance. Under this program, PDP will also provide the DISCOs with new connectors for HT and LT extensions, which when installed will improve the binding of joints in the distribution system. This in turn will not only save MWs and improve revenues for the DISCOs, but also improve upon the safety (employees and general public), reliability of supply and customers satisfaction. Group Interview Code: XXX Number of Participants Date and time of Interview Interview Location: QUESTIONS: Overview of Implementation: 1. How has linemen safety training activity been developed and implemented at your DISCO? Implementation Design: 2. How has the activity introduced safety practices and procedures for linemen at your DISCO? 3. What feedback have you received from the participants of linemen safety trainings? 4. How do you think that the effectiveness of these trainings can be improved? Expectations and Results: 5. Please describe the usefulness and relevance of the trainings to your operations/business 94 6. What changes in safety practices and procedures have you observe as a result of this training? 7. How far this training has been helpful in reducing the frequency and severity of accidents among linemen of your DISCOs. Sustainability: 8. How do you plan to provide similar trainings on your own? What resources have you allocated for this purpose? 9. How do you anticipate ensuring training of remaining and newly hired linemen? Governance: 10. How has this training activity helped in meeting regulatory requirements associated with the safety of employees and citizens? 95 Power Distribution Program (PDP) Interim Performance Evaluation Instrument for Group Interviews with DISCO Staff on Utility Exchange Program under Human Resources and Change Management: Introduction Thank you very much for meeting us today. My name is __________________ and I represent the USAID/ Pakistan Monitoring and Evaluation Program (MEP). This program is being implemented by Management Systems International, an international consulting firm. The program helps the USAID/ Pakistan Mission monitor and evaluate a range of its programs, including those in the areas of science, innovation and higher learning. USAID/ Pakistan has contracted MSI to conduct an interim evaluation of the USAID Power Distribution Program (PDP) and systematically assess the effectiveness of PDP’s interventions to improve the performance of the DISCOs and the governance and policy environment of public sector entities in the energy arena. We would like to ask you a few questions about the program and Mr./Ms. _____________________ will take notes. We will need to cover several aspects of PDP program with your help. Confidentiality With your permission, we would like to record this discussion so that we accurately capture your feedback and do not miss any important points. Please be assured that your responses will be kept confidential. In case we use quotations from this interview in our evaluation report, you will not be identified by name, or official title, but in general terms as a researcher, official or manager. Group Interview Code: XXX Number of Participants: Date and time of interview: Interview Location: QUESTIONS: Overview of Implementation: 1. Please describe your DISCO’s participation in Utility Exchange Program. Implementation Design: 2. How has the program introduced improved policies, practices or procedures as a result of exposure received through these exchange visits? 3. What type of follow-up activities have been undertaken to maximize benefits of these exchange visits? 4. Please describe the functional focus of Utility Exchange Program. (Prompts: Administration, operations, management, technical, customer service etc.). 5. What feedback did the DISCOs receive from the participants about the exchange visits? 6. What was their contribution in terms of knowledge sharing upon their return? Expectations and Results: 7. What were your expectations as a result of participation in this activity? 8. What changes did you observe before and after the utility exchange visits was implemented in your DISCO with the support of PDP? 96 9. What benefits, if any, were achieved as a result of implementation of the utility exchange at your DISCO? 10. Describe any noticeable changes in approach to work as a result of the utility exchange? 11. How can utility exchange visits be improved to better achieve its planned results? Sustainability: 12. What are the prospects for continuation of this activity at your DISCO after PDP has ended? 13. What is the contribution of the exchange program in institutional learning at your DISCO? Governance: 14. How has the governance of the DISCO changed as a result of the knowledge gained as a result of the exchange visits? Gender: 15. How are the female staff involved in these exchange program? 16. In what ways these visits sensitized the participants about the gender related concerns and issues at your DISCO? 97 Power Distribution Program (PDP) Interim Performance Evaluation Instrument for Group Interviews with Principals of Girls Colleges regarding Energy Conservation Campaign Activity under Gender Component Introduction Thank you very much for meeting us today. My name is __________________ and I represent the USAID/ Pakistan Monitoring and Evaluation Program (MEP). This program is being implemented by Management Systems International, an international consulting firm. The program helps the USAID/ Pakistan Mission monitor and evaluate a range of its programs, including those in the areas of science, innovation and higher learning. USAID/ Pakistan has contracted MSI to conduct an interim evaluation of the USAID Power Distribution Program (PDP) and systematically assess the effectiveness of PDP’s interventions to improve the performance of the DISCOs and the governance and policy environment of public sector entities in the energy arena. We would like to ask you a few questions about the program and Mr./Ms. _____________________ will take notes. We will need to cover several aspects of PDP program with your help. Confidentiality With your permission, we would like to record this discussion so that we accurately capture your feedback and do not miss any important points. Please be assured that your responses will be kept confidential. In case we use quotations from this interview in our evaluation report, you will not be identified by name, or official title, but in general terms as a researcher, official or manager. Group Interview Code: XXX Number of Participants Date and time of interview Interview Location: 98 Questions: 1. How would you describe the implementation of energy conservation campaigns and its outcomes at your colleges? 2. What, if any, interaction was there between the DISCO and your college during the energy conservation awareness campaign? 3. How did faculty acquire the information on energy conservation shared through this campaign? What sources were used as materials? 4. What other topics/activities were covered during these campaigns? 5. In your opinion what are the core benefits of these awareness campaigns? (Such as Energy savings awareness at home and at workplace). 6. How did the campaign change awareness of energy conservation among students and teachers of your colleges? 7. In what ways did students and faculty change their energy practices? 8. To what extent and in what ways have students and faculty introduced what they learned to their peers and relatives? 9. How do you think these campaigns have affected your students’ ability to enter the power industry as career? 10. How will your college (or department of education) continue to promote energy conservation awareness information in the future after PDP is completed? 11. How do you think these campaigns could have been implemented differently? 99 Power Distribution Program (PDP) Interim Performance Evaluation Instrument for Group Interviews with DISCO Staff on Gender Equity Training under Gender Component Introduction Thank you very much for meeting us today. My name is __________________ and I represent the USAID/ Pakistan Monitoring and Evaluation Program (MEP). This program is being implemented by Management Systems International, an international consulting firm. The program helps the USAID/ Pakistan Mission monitor and evaluate a range of its programs, including those in the areas of science, innovation and higher learning. USAID/ Pakistan has contracted MSI to conduct an interim evaluation of the USAID Power Distribution Program (PDP) and systematically assess the effectiveness of PDP’s interventions to improve the performance of the DISCOs and the governance and policy environment of public sector entities in the energy arena. We would like to ask you a few questions about the program and Mr./Ms. _____________________ will take notes. We will need to cover several aspects of PDP program with your help. Confidentiality With your permission, we would like to record this discussion so that we accurately capture your feedback and do not miss any important points. Please be assured that your responses will be kept confidential. In case we use quotations from this interview in our evaluation report, you will not be identified by name, or official title, but in general terms as a researcher, official or manager. Note for the interviewer: Objectives: Currently, less than one percent of employees in the power sector are female. USAID-organized trainings will help improve working conditions for female employees while simultaneously building awareness on the importance of equal opportunity for all staff irrespective of gender. Group Interview Code: XXX Number of Participants Date and time of Interview Interview Location: QUESTIONS: Overview of Implementation: 1. How was the gender equity training activity designed and conducted at your DISCO? 2. Please describe the involvement of male and female employees in the training. Implementation Design: 3. How would you describe the contents and delivery of the training? 4. What was the feedback from training participants? (Prompt: contents, trainer’s skills, schedule and timing, atmosphere, special arrangement for female participants). Expectations and Results: 5. How have these trainings achieved the intended objectives? Please elaborate. 100 6. What changes, if any, have you noticed in the behavior of male employees towards their female colleagues as a result of this training? 7. To what extent and in what ways have your DISCOs policies and procedures changed in response to these trainings? (Prompts: more facilitating work environment for female workers, extended maternity leaves, child care centers, separate toilets for women, and flexible working hours). 8. Please describe any gaps or issues in the training contents and trainer’s skills. 9. How can such training activities be designed differently to produce desired results? 10. What have been the effects of the trainings on women at their workplace? 11. To what extent and in what ways have trainings influenced female employees’ career prospects? 12. What changes have you observed in the intake of women interns as a result of this activity? 13. To what extent and in what ways do you think this activity could result in encouraging women in the power sector job market? 14. Please describe how, if at all, these trainings have changed the efficiency and productivity of the female employees? 15. What changes would you advocate in this training? Why? Sustainability: 16. How will your DISCO continue to conduct gender equity trainings in the future after PDP is completed? 17. What is the level of ownership of this training activity at your DISCO? Governance: 18. How has this activity helped your DISCO comply with existing government policies and laws on gender equity and empowerment? (i.e. reserve quota for women in public sector employment, laws relating to conditions at workplace and harassment). 101 LINEMEN SAFETY TRAINING SURVEY تاروں کی دیکھ بھال کرنیوالوں کی حفاظتی تربیت کا سروے For Linesmen and Managers/Supervisors تعارف۔ Introduction “Dear Sir/Madam, recently under a USAID funded program, linemen safety trainings were conducted. Our records indicated that you participated in the in the said trainings. USAID is now in the process of conducting evaluation of this important activity. We need your cooperation to this regards by providing us with relevant information and data so that USAID can establish if the project achieved its intended objectives and outcomes. We therefore, request you to kindly spare some of your valuable time for an interview, which is likely not to take more than 1 hour. USAID very much appreciates your cooperation and support to this regards” محترم/محترمہ، حالیہ دنوں میں امریکی ادارہ برائے بین االقوامی امداد کی جانب سے الئن مینوں کی حفاظت سے متعلق ایک تربیتی نشست کا اہتمام کیا گیا تھا ، ہمارے ریکارڈ کے مطابق آپ نے اس تربیتی پروگرام میں شرکت کی تھی ۔ یو۔ایس۔ایڈ ان دنوں اس اہم پر ِ وگرام کے تجزیاتی جائزہ میں مصروف ہے۔ اس امر میں ہمیں آپکے تعاون اور متعلقہ معلومات کی ضرورت ہے تا کہ یو۔ایس۔ایڈ یہ جان سکے کہ آیا اس پروگرام نے اپنے مطلوبہ مقاصد اور نتائج حاصل کر ٰ ہم آپ سے درخواست کرتے ہیں کہ برائے مہربانی ا لئے ہیں کہ نہیں۔ لہذا پنے قیمتی وقت سے کچھ وقت ایک انٹرویو کیلئے مختص کیجئے اور یہ انٹرویو ای ک گھنٹے سے اوپر نہیں ہوگا۔ یو۔ایس۔ایڈ اس عمل میں آپکے تعاون اور مدد کو انتہائی قدر کی نگاہ سے دیکھتا ہے۔ ِ )عمومی/توضیحی/ پیشہ وارانہ پس منظر( BACKGROUND PROFESSIONAL/DESCRIPTIVES/GENERAL .A )بجلی کے تقسیم کار ادارے کا نام( :Name DISCO 1.A )جواب دہندہ کا نام( :Name Respondent .b1A A 2. Division/Circle: )سرکل/ڈویژن) A 3. Job Title: )عہدہ) )کام کی تفصیل( :Description Job 4. A )بجلی کی تقسیم کے ادارے کیساتھ وابستگی کے سال( :DISCO with Years of Number 5. A A 6. Number of Years on Current Position: )عرصہ کا تعیناتی پر عہدہ موجودہ) A 7. Age: )عمر) A8. Did you ever have any accident(s), while performing your duties? (CIRCLE ONE NUMBER) )کیا پیشہ وارانہ ذمہ داریوں کی ادائیگی کے دوران کبھی آپکا کوئی حادثہ ہوا؟( Yes )ہاں )1 GO TO A9 No )نہیں )2 SKIP TO A10 Don’t know (DO NOT READ) نہیں۔ معلوم 98 SKIP TO A10 Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 SKIP TO A10 )نمبر شمار( number Serial (FOR OFFICE USE ONLY) )انٹرویو کی تاریخ( :date Interview 102 A9. If YES, please provide type of accident, resulting impact and nature of services (Multi responses allowed) برائے) مہربانیبتائیےکہوہ کس قسم کا حادثہ تھا اور اسکے اثرات کیاتھے اور وہ کس طرح کی خدمات کی ادائیگیکے دوران پیش آیاتھا؟( A 9a. Type of Accident A 9b. Resulting Impact Major Injury (Incapacitated to move by himself) = 1 )بڑی چوٹ۔ خود حرکت کرنے سے قاصر( Minor Injury (Able to move by himself)= )چھوٹی چوٹ۔ خود حرکت کرنے کے قابل( 2 )کوئی چوٹ نہیںلگی( 3 = Injury No A 9c. Nature of Service Presumed dead line (Lineman thought line is dead when it was live = 1 )الئن مین کے خیال میں تاریں ڈیڈ تھیں( Live Line (Lineman knew the line is live = 2 )الئن مین جانتا تھا کہ تاریں ڈیڈ نہیں ہیں( )دیگر( 90 = :Other 1 )بجلیکا جھٹکا( Shock Electric 2 )بجلیکا شعلہ( Spark Electric )کرنٹ لگنے سے Burn Electric 3 جلنا( Fall from Height )گرنا اونچائیسے) 4 )کسیشئے کا اوپر Object Falling 5 سے گرنا( Breakdown of safety equipment 6 )حفاظتی آالت کا خراب ہونا( 90 )دیگر۔۔۔۔۔۔۔۔۔۔۔۔( :Other Don’t know )نہیں معلوم )98 Refused to answer سے جواب) 99 انکار( A10. Have you witnessed injury or fetal accidents involving linemen performing their routine jobs? (CIRCLE ONE )کیا آپ نے کبھی الئن مینوں کو اپنے فرائض کی ادائیگی کو دوران کسی خونی حادثے یا چوٹ کا شکار ہوتے (NUMBER دیکھا ہے؟( Yes )ہاں )1 GO TO A11 No )نہیں )2 SKIP TO A12 Don’t know (DO NOT READ) نہیں۔ معلوم 98 SKIP TO A12 Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 SKIP TO A12 A11. If YES, please provide descriptions of accident and nature of services (Multi response) )اگر ہاں، تو برائے مہربانی بتائیے کہ وہکس قسم کا حادثہ تھا اور اسکے اثرات کیا تھے اور وہ کس طرح کی خدمات کی ادائیگی کے دوران پیش آیا تھا؟( 1 AAa. Type of Accident 1 AAb. Resulting Impact Major Injury (Incapacitated to move by )بڑی چوٹ۔ خود حرکت کرنے سے 1 = (himself قاصر( Minor Injury (Able to move by himself)= 2 )چھوٹی چوٹ۔ خود حرکت کرنے کے قابل( )کوئی چوٹ نہیں لگی( 3 = Injury No )موت واقع ہونا( 4 =Death 1 AAc. Nature of Service Presumed dead line (Lineman thought line is dead when it was live = 1 الئن مینکے خیال میںتاریں ڈیڈتھیں( Live Line (Lineman knew the line is live = 2 الئن مین جانتا تھا کہ تاریں ڈیڈنہیںہیں( 103 )دیگر( 90 = :Other 1 )بجلی کا جھٹکا( Shock Electric 2 )بجلی کا شعلہ( Spark Electric )کرنٹ لگنے سے Burn Electric 3 جلنا( )اونچائی سے گرنا( Height from Fall 4 )کسی شئے کا اوپر Object Falling 5 سے گرنا( Breakdown of safety equipment 6 )حفاظتی آالت کا خراب ہونا( 90 )دیگر۔۔۔۔۔۔۔۔۔۔۔۔( :Other Don’t know )نہیں معلوم )98 Refused to answer سے جواب) انکار( 99 A12. Based on your experience, on an average how many accidents, of the following nature, occur in a year in your )اپنے تجربات کی بنیا د پر بتائیے کہ آپکے تقسیم کار ادارے میں اس نوعیت کے حادثات کی تعداد ایک سال میں ?DISCO اوسطاً کتنی ہے؟( Type of Accident )اقسام کی حادثات )Count )حادثہ جس سے موت واقع ہو( death in resulting Accidents .a )حادثہ جس میں بڑی چوٹ لگے( injuries major involving Accidents .b )حادثہ جس میں چھوٹی چوٹ لگے( injuries minor involving Accidents .c )حادثہ جس میں کوئی چوٹ نہ لگے( injuries no involving Accidents .d e. Other (Specify hereدیگر۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔) A13. Is there any Standard Operating Manual (SOP) for linesmen on conducting their normal duties? (CIRCLE ONE )کیا الئن مینوں کو معمول کے فرائض سرانجام دینے کےلئےکوئی معیاری ضابطہ ٴ کار موجود ہے؟( (NUMBER Yes )ہاں )1 GO TO A14 No )نہیں )2 SKIP TO B1 Don’t know (DO NOT READ) نہیں۔ معلوم 98 SKIP TO B1 Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 SKIP TO B1 A14. Does the SOP cover safety related aspects? (CIRCLE ONE NUMBER) ٴ )کیا وہ معیاری ضابطہ ں کا احاطہ کرتا ہے؟( ٴ کار حفاظت سے متعلقہ پہلوو 1 )ہاں( Yes 2 )نہیں( No Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 A15. Do linesmen follow SOP diligently? (CIRCLE ONE NUMBER) کار ٴ )کیا الئن مین باقاعدگی سے معیاری ضابطہ پر عمل کرتے ہیں؟( 104 1 )ہاں( Yes 2 )نہیں( No Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 B. ENGAGEMENT IN TRAINING(S) شمولیت میں نشستوں تربیتی B1. Did you attend any of the Linesmen Safety Training course? (CIRCLE ONE NUMBER) )کیا آپ نے کسی ' الئن مینوں کی حفاظت کی تربیتی ورکشاپ' میں شرکت کی تھی؟( Yes )ہاں )1 SKIP TO B3 No )نہیں )2 GO TO B2 B2. If NO, what was the reason? (CIRCLE ONE NUMBER) )تھی؟ وجہ کیا اسکی تو نہیں اگر) I am manager/supervisor سپروائزر/منیجر میں) )ہوں 1) GO TO SECTION C) I was not eligible )تھا نہیں اہل میں )2) TERMINATE INTERVIEW) 90 )دیگر۔۔۔۔۔۔۔۔۔۔۔۔۔specify Please (انٹرویو ختم کردیں۔ Other B3a. Which Safety Program(s) did you attend? (Multiple Selection Allowed) )کون سے حفاظتی پروگراموں میں آپ نے شرکت کی تھی؟( a. Quality and Safety Monitors 1 )معیار ار سالمتی کے نگران( f. Lineman Safety Training Program for PESCO Change Management ) OCSEPتبدیلی کے انتظام پر الئنوں مینوں کی حفاظت کا تربیتی پروگرام( 6 b. Refresher - Lineman Safety and Performance Improvement Training Program )اعادہ۔ الئن مینوں کی حفاظت اور کارکردگی کو بہتر کرنے کی تربیت( 2 g. Lineman Safety and Performance Improvement Training Program )الئن مینوں کی حفاظت اور کارکردگی کو بہتر کرنے کی تربیت( 7 c. One Day Safety Training Program 90 )دیگر۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔ specify (other .h 3( حفاظت کی تربیت کا ایک روزہ پروگرام( d. Lineman Safety and Hazard Identification Training Program )الئن مینوں کی حفاظت اور خطرات کی نشاندہی کا تربیتی پروگرام( 4 i. Don’t know (DO NOT READ) نہیں۔ معلوم 98 e. Quick Impact Safety Training Program )فوری اصابت۔ حفاظت کے تربیتی Orientation پروگرام پر رہنمائی( 5 j. Refused to answer/No reply (DO NOT READ) 99 جواب سے انکار۔ )درج ذیل میں سے کون سی تربیتی نشست میں آپ نے حال ہی ?attended you training recent most the was What . b3B میں شرکت کی ہے؟( Please answer the rest of the survey (Section B) with the most recent training in mind. (Circle the most recent training only.) a. Quality and Safety Monitors 1 )معیار ار سالمتی کے نگران( f. Lineman Safety Training Program for PESCO Change Management ) OCSEPتبدیلی کے انتظام پر الئنوں مینوں کی حفاظت کا تربیتی پروگرام( 6 b. Refresher - Lineman Safety and Performance 2 g. Lineman Safety and Performance Improvement 7 105 Improvement Training Program )اعادہ۔ الئن مینوں کی حفاظت اور کارکردگی کو بہتر کرنے کی تربیت( Training Program )الئن مینوں کی حفاظت اور کارکردگی کو بہتر کرنے کی تربیت( c. One Day Safety Training Program 90 )دیگر۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔ specify (other .h 3( حفاظت کی تربیت کا ایک روزہ پروگرام( d. Lineman Safety and Hazard Identification Training Program )الئن مینوں کی حفاظت اور خطرات کی نشاندہی کا تربیتی پروگرام( 4 i. Don’t know (DO NOT READ) نہیں۔ معلوم 98 e. Quick Impact Safety Training Program )فوری اصابت۔ حفاظت کے تربیتی Orientation پروگرام پر رہنمائی( 5 j. Refused to answer/No reply (DO NOT READ) 99 جواب سے انکار۔ B4. How did you get engaged in Linesmen Safety Training program? (MULTIPLE RESPONSES ALLOWED) آپ) 'الئن مینوں کی حفاظت کے تربیتی پروگرام میں کیسے شامل ہوئے تھے؟( )میں نے اسکے بارے میں سنا تھا اور اس کیلئے درخواست دی it for applied and heard I .a 1 تھی( )میرے سپروائزر نے میری نامزدگی کی تھی( me nominated Manager/supervisor My .b 2 c. Other (Please specifyدیگر۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔ )90 d. Don’t know (DO NOT READ) نہیں۔ معلوم 98 e. Refused to answer/No reply (DO NOT READ) انکار۔ سے جواب 99 )آپ نے کتنے دن وہ تربیت حاصل کی تھی؟( ?training the attend you did days many How 5.B یہاں اندراج کریں۔ (HERE RESPONSE RECORD( For any other response circle the relevant response below دائرہ نیچے کیلئے جوابات دیگر) لگائیں( Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer / No reply (DO NOT READ) انکار۔ سے جواب 99 B6. Did you attend entire duration of the training? (CIRCLE ONE NUMBER) )کیا آپ تربیت کے سارے وقت شریک رہے؟( Yes )ہاں )1 SKIP TO B8 No )نہیں )2 GO TO B7 B7. If NO, please describe the reasons? (MULTIPLE RESPONSES ALLOWED) بتا وجوہات اسکی آپ کیا تو نہیں اگر) سکتے ہیں؟( )تربیت اس طرح کی نہیں تھی، جیسی میں نے توقع کی expected I what not was Training .a 1 تھی( b. I had personal engagements )باعث کے مصروفیات ذاتی )2 3 )پیشہ وارانہ مصروفیات کے باعث( engagements professional had I .c d. I could not understand what was done in training میری ،تھا جارہا کیا میں تربیت کچھ جو) 4 سمجھ سے باہر تھا( e. I did not find the training useful )تھی لگی نہیں کارآمد تربیت مجھے )5 6 )میں تربیت دینے والے کو سمجھ نہیں سکا( trainer the understand not could I .f g. Other (specify hereدیگر۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔ )90 h. Don’t know (DO NOT READ) نہیں۔ معلوم 98 i. Refused to answer / No reply (DO NOT READ) دیا۔ نہیں جواب 99 B8. Who conducted the training? (CIRCLE ONE NUMBER) )تھا؟ کیا نے کس اہتمام کا تربیت) 106 1 )غیر ملکیوں نے( Foreigners 2 )مقامی لوگوں نے( Locals 3 )غیر ملکیوں اور مقامی لوگوں کی ٹیم نے( Locals and Foreigners of Team Other (Please specifyدیگر۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔ )90 Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer / No reply (DO NOT READ) دیا۔ نہیں جواب 99 B9. What was the training mode? (CIRCLE ONE NUMBER) )تھا؟ کیا اسلوب کا تربیت ُ) ٴ جماعت میں علمی کام( Only Theory Classroom 1 )صرف کمرہ ٴ جماعت میں عملی مظاہرہ اور Only Demonstration and Theory Classroom )صرف کمرہ علمی کام( 2 جماعت پھر فیلڈ میں عملی مظاہرہ( Only Demonstration Infield then Classroom ٴ 3 )کمرہ Classroom then Infield Demonstration and Hands On Training ٴ جماعت پھر فیلڈ میں عملی مظاہرہ اور اپنے ہاتھوں سے عملی مہارت کا مظاہرہ( )کمرہ 4 5 )صرف فیلڈ میں عملی مظاہرہ( Only Demonstration Infield )فیلڈ میں عملی مظاہرہ اور اپنے ہاتھوں سے عملی Training On Hands and Demonstration Infield مہارت کا مظاہرہ( 6 Other (Please specifyدیگر۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔ )90 B10. What components were most useful to you? (CIRCLE ONE NUMBER) )آپکی تربیتی نشست میں شمولیت کو کیسے بیان کیا جا سکتا ہے؟( ٴ جماعت میں علمی کام( Theory Room Class 1 )کمرہ 2 )اپنے ہاتھوں سے عملی مہارت کا مظاہرہ(Training on Hands 3 )فیلڈ کے دوران عملی مظاہرہ( Demonstration Infield Other (Please specifyدیگر۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔ )90 Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer / No reply (DO NOT READ) دیا۔ نہیں جواب 99 )بعد از تربیت کےاثرات( IMPACTS TRAINING POST .C C1. Is there any appreciable decrease in the number of fatal/nonfatal accidents? (CIRCLE ONE NUMBER) یہاں اب کیا) ِ خطرناک حادثات کی تعداد میں کوئی قابل تعریف کمی ہوئی ہے؟( Yes )ہاں )1) SKIP TO C3) No )نہیں )2) GO TO C2) Don’t know (DO NOT READ) نہیں۔ معلوم 98) SKIP TO C3) Refused to answer / No reply (DO NOT READ) دیا۔ نہیں جواب 99) SKIP TO C3) C2. If NO, what could be the reason? (MULTIPLE RESPONSES ALLOWED) سکتی ہو وجہ کیا اسکی تو ،نہیں اگر) ہے؟( 1 )تمام الئن مین تربیت یافتہ نہیں ہیں( trained are linemen all Not .a b. Training was not effective )تھی نہیں ٴثر 2 )تربیت مو )تربیت یافتہ الئن مین عملی طور پر تربیت کے practice in training apply not do linemen Trained .c 3 اصولوں پر عمل پیرا نہیں ہوتے( d. The management has not applied processes and procedures as recommended under training 4 )انتظامیہ نے تربیت کے دوران سفارش کردہ طریقوں اور عملیات کا نفاذ نہیں کیا( e. Other (specifyدیگر۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔ )90 107 f. Don’t know (DO NOT READ) نہیں۔ معلوم 98 g. Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 C3. Is there a marked difference in the performance of those linesmen who have been trained against those who have not in observing safety habits? (CIRCLE ONE NUMBER) ِ کے درمیان کوئی قابل )کیا تربیت یافتہ اور ان الئن مینوں جو حفاظت کے اصولوں پر عمل پیرا نہیں ہیں، کی کارکردگی ذکر فرق ہے؟( Yes )ہاں )1) SKIP TO D1) No )نہیں )2) GO TO C4) Don’t know (DO NOT READ) نہیں۔ معلوم 98) SKIP TO D1) Refused to answer / No reply (DO NOT READ) دیا۔ نہیں جواب 99) SKIP TO D1) C4. If NO, what could be the main reason? (CIRCLE ONE NUMBER) )ہے؟ سکتی ہو کیا وجہ اہم اسکی تو نہیں اگر) a. Training was not effective )تھی نہیں ٴثر 1 )تربیت مو b. Trained linemen do not apply training in practice کا طریقوں گئے سکھائے مین الئن یافتہ تربیت) 2 استعمال نہیں کرتے( c. The management does not support application of training knowledge and skills in practice انتظامیہ) 3 تربیت میں سکھائے گئی مہارتوں اور علم کے نفاذ کی حوصلہ افزائی نہیں کرتی( d. Other (specifyدیگر۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔ )90 e. Don’t know (DO NOT READ) نہیں۔ معلوم 98 f. Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 D. PERCEPTIONS ABOUT ROOT CAUSES OF ACCIDENTS (PRE AND POST TRAINING) )حادثات کی بنیادی وجوہات کے بارے میں تصورات۔ ما قبل و بعد از تربیت( D1. In your opinion what are main reasons for the accidents, involving linespersons performing their routine duties? (MULTIPLE RESPONSES ALLOWED) )آپکی رائے میں الئن مینوں کی فرائض کی ادائیگی کے دوران ہونیوالے حادثات کی اہم وجوہات کیا ہیں؟( 1 )الئن مین عمومی طور پر الپروا ہوتےہیں( careless generally are Linespersons .a 2 )الئن مین حفاظتی اصولوں سے ال علم ہیں( measures safety of aware not are Linespersons .b 3 )الئن مین درست طور پر تربیت یافتہ نہیں ہیں( trained properly not are Linespersons .c d. Linespersons are not supplied with appropriate safety equipment/gear آالت حفاظتی مناسب پاس کے مینوں الئن) 4 نہیں ہیں( e. Linespersons do not have access to appropriate transportation to bring their protection gear )الئن مینوں کو اپنے حفاظتی آالت کو ساتھ النے کیلئے مناسب ذرائع نقل وحمل تک رسائی حاصل نہیں ہے( 5 f. Management has not implemented safety procedures e.g. issuance of work order 6 )انتظامیہ نے حفاظتی ضابطوں مثالً ورک آرڈر کے اجراٴ کا نفاذ نہیں کیا ہے۔( g. Management has not provided clear guidelines and standard procedures 7 )انتظامیہ نے میعاری ضابطہ کار پر واضح ہدایات جاری نہیں کیں۔( h. The macho mindset among linemen encourages to take risks )الئن مینوں میں پائی جانیوالی بہادری دکھانے کی ذہنیت انہیں خطرات سے کھیلنے پر اکساتی ہے( 8 i. Other (Please specifyدیگر۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔ )90 j. Don’t know (DO NOT READ) نہیں۔ معلوم 98 k. Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 D2. In your opinion, occurrence of accidents involving Linespersons performing their routine work, is a serious issue? )الئن مینوں کی فرائض کی ادائیگی کے دوران ہونیوالے حادثات آپکے مطابق کیا ایک سنجیدہ (NUMBER ONE CIRCLE( مسئلہ ہے؟( 1 )ہاں( Yes 108 )نہیں( No 2 (SKIP TO QTS D4) Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer / No reply (DO NOT READ) دیا۔ نہیں جواب 99 D3. Please provide 3 main reasons for your answer above, in the order of importance? (Mark 1, 2, 3 against each )اہمیت کی ترتیب کے حساب سے اپنے گذشتہ جواب کی روشنی میں تین اہم وجوہات بیان کیجئے۔( (reason possible a. There are just too numerous accidents )ہے زیادہ بہت تعداد کی حادثات) )بہت زیادہ اموات واقع ہوتی ہیں( deaths many too are There .b )الئن مین اچھی کارکردگی کا مظاہرہ نہیں well perform cannot Linemen .c کرتے( d. The workplace practices encourages risk taking )کام کرنیوالی جگہ کی عمومی عادات خطرات سے کھیلنے کی حوصلہ افزائی کرتی ہیں( e. Other (specify here۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔ دیگر) f. Don’t know (DO NOT READ) نہیں۔ معلوم 98 g. Refused to answer/No reply (DO NOT READ) انکار۔ سے جواب 99 D4. Do you think that the Linesmen Safety Training was needed? (CIRCLE ONE NUMBER) )کیا آپ سمجھتے ہیں کہ الئن مینوں کی حفاظتی تربیت کی ضرورت تھی؟( 1 )ہاں( Yes )نہیں( No 2 (SKIP TO QTS D6) D5. Please provide 3 main reasons for your answer above, in the order of importance? (Mark 1, 2, 3 against each )اہمیت کی ترتیب کے حساب سے اپنے گذشتہ جواب کی روشنی میں تین اہم وجوہات بیان کیجئے۔( (reason possible )الئن مینوں کے درمیان آگاہی پیدا کرنا( linesmen among awareness create To )انتظامیہ کے درمیان آکاہی پیدا کرنا( management among awareness create To )الئن مینوں کے درمیان حفاظتی عادات کو بڑھانا( LM among practices safe promote To )الئن مینوں کے درمیان بہادری دکھانے والی ذہینت linesmen among mindset macho change To کو تبدیل کرنا( Other (specify hereدیگر۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔) f. Don’t know (DO NOT READ) نہیں۔ معلوم 98 g. Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 D6. Do you think that the Linesmen Safety Training has improved Safety awareness and safety related skills of linesmen at your DISCO? (CIRCLE ONE NUMBER) )کیا آپ سمجھتے ہیں کہ حفاظتی تربیتی پروگرام نے آپکے تقسیم کار ادارے کے الئن مینوں کی حفاظت سے متعلق آگاہی اور مہارتوں کو بہتر کیا ہے؟( 1 )ہاں( Yes 2 )نہیں( No Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 D7. Were there any negative impacts of the training? (CIRCLE ONE NUMBER) )کیا اس تربیتی پروگرام کے کوئی منفی اثرات ہیں؟( 109 Yes )ہاں )1 Go to D8 No )نہیں )2 Skip to D9 Don’t know (DO NOT READ) نہیں۔ معلوم 98 Skip to D9 Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 Skip to D9 D8. If YES, please provide main negative impact of the safety training? (CIRCLE ONE NUMBER) )اگر ہاں تو برائے مہربانی حفاظتی تربیت کے تین منفی اثراتبتائیے۔( 1 )ادارے کو چھوڑ جانے کا رجحان( organization leave to Tendency 2 )مغرور ہوجانا( arrogant Become 3 )حد سے زیادہ پراعتماد ہو جانا( confident over Become Other (specify hereدیگر۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔ )90 Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer/No reply (DO NOT READ) انکار۔ سے جواب 99 D9. Please provide overall rating on the Importance and Benefits of the training? (CIRCLE ONE NUMBER IN EACH )معتدل( Moderately( بہت( Very( انتہائی( Extremely( تربیت کی اہمیت اور فوائد کے حساب سے درجہ بندی کیجئے( (ROW )عام سے( Fairly Not at all )نہیں بالکل) Don’t know )نہیں معلوم) a. Importance (Important) اہمیت۔ 1 2 3 4 5 98 b. Benefits (Beneficial) فوائد۔ 1 2 3 4 5 98 E. TRAINING EXPERIENCE (Managers/Supervisors not required to complete this section) تجربہ کا تربیت E1. During the training, were you required to complete some written tasks? (CIRCLE ONE NUMBER) ِ )دوران تربیت کیا آپ سے کوئی تحریری کام مکمل کرنے کو کہا گیا تھا؟( 1 )ہاں( Yes 2 )نہیں( No Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 E2. During the training, were you required to complete some practical tasks? (CIRCLE ONE NUMBER) ِ )دوران تربیت کیا آپ سے کوئی عملی کام کروایا گیا تھا؟( 1 )ہاں( Yes 2 )نہیں( No Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 E3. Did you find the training Interesting? (CIRCLE ONE NUMBER) )تھی؟ لگی دلچسپ نشست تربیت آپکو کیا) 1 )ہاں( Yes 2 )نہیں( No Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 110 E4. Did you find the training Useful? (CIRCLE ONE NUMBER) )کومفیدپایا؟ نشست تربیتی نے آپ کیا) 1 )ہاں( Yes 2 )نہیں( No Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 E5. Do you think you have gained additional knowledge and skills from the training? (CIRCLE ONE NUMBER) کیا) آپ سمجھتے ہیں کہ تربیت سے آپ نے کوئی اضافی علم اور مہارتیں حاصل کی ہیں؟( 1 )ہاں( Yes 2 )نہیں( No (SKIP TO QTS E7) Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 E6. If YES, do you think that additional knowledge and skills gained from the training will be useful for you while performing your routine work at the DISCO? (CIRCLE ONE NUMBER) )اگر ہاں تو کیا آپ سمجھتے ہیں کہ تربیت سے جو علم اور مہارتیں آپ نے حاصل کی ہیں، وہ آپکو اپنے تقسیم کار ادارے ِ میں روزمرہ کے کام سرانجام دینے میں مفید ہونگے؟( 1 )ہاں( Yes 2 )نہیں( No Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 E7. Do you think more training should be organized for linespersons in future? (CIRCLE ONE NUMBER) کیا آپ) سمجھتے ہیں کہ مستقبل میں الئن مینوں کیلئے مزید تربیتی نشستیں ہونی چاہییں؟( 1 )ہاں( Yes 2 )نہیں( No Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 )تربیتی اجزاٴ کی درجہ بندی( COMPONENTS TRAINING FOR RATING .F F 1. Please rate the following components of the training. (CIRCLE ONE NUMBER IN EACH ROW) )معلوم نہیں( Know t’Don سکتی( )درجہ بندی نہیں ہو Rate Cannot( ٹھیک ہے( Fair( اچھا( Good( بہت اچھا( Good Very( انتہائی شاندار( Excellent( برائے مہربانی تربیت کے مندرجہ ذیل اجزاٴ کی درجہ بندی کیجئے( 111 a. Overall Contents of the training )مندرجات مجموعی کے تربیت )1 2 3 4 99 98 98 99 4 3 2 1 )علم کی تفویض کا طریقہ( delivery of Mode .b 98 99 4 3 2 1 )تربیت کار( Trainers .c d. Theoretical Knowledge )علم نظری )1 2 3 4 99 98 )اپنے ہاتھوں سے عملی مہارت کے مظاہرے کا Exercises On Hands .e 98 99 4 3 2 1 تجربہ( F2. Please rate various components of the training, by level of usefulness. (CIRCLE ONE NUMBER IN EACH ROW) )مفید( Useful( بہت مفید( Useful Very( انتہائی مفید( Useful Extremely( برائے مہربانی افادیت کے حساب سے تربیت کے مختلف مندرجات کی درجہ بندی کیجئے( )غیر مفید( Useful Not Cannot Rate )ردجہ بندی نہیں کرسکتا/سکتی( Don’t know )معلوم نہیں( a. Overall Contents of the training مجموعی کے تربیت) 98 99 4 3 2 1 مندرجات( b. Theoretical Knowledge )علم نظری )1 2 3 4 99 98 )اپنے ہاتھوں سے عملی مہارت کے Experience On Hands .c 98 99 4 3 2 1 مظاہرے کا تجربہ( F3. Please rate various components of the training, by level of relevance to your job at your DISCO. (CIRCLE ONE NUMBER IN EACH ROW) )برائے مہربانی تربیت کے مختلف مندرجات کی درجہ بندی اپنے تقسیم کار ادارے میں اپنی الزمت سے انکی مطابقت کے لحاظ سے کیجئے( Extremely Relevant )انتہائی متعلقہ( )بہت متعلقہ( Relevant Very )متعلقہ( Relevant )غیر متعلقہ( Relevant Not Cannot Rate )ردجہ بندی نہیں کرسکتا/سکتی( Don’t know )نہیں معلوم) a. Overall Contents of the training مجموعی کے تربیت) مندرجات( 1 2 3 4 99 98 b. Theoretical Knowledge )علم نظری )1 2 3 4 99 98 )اپنے ہاتھوں سے عملی مہارت کے Experience On Hands .c مظاہرے کا تجربہ( 1 2 3 4 99 98 )استقامت اور پائیداری( SUSTAINABILITY .G G1. Do you think your DISCO will continue providing similar trainings in future? (CIRCLE ONE NUMBER) آپ کیا) سمجھتے ہیں کہ آپکا تقسیم کار ادارہ مستقبل میں ایسی تربیتی نشستوں کی فراہمی جاری رکھے گا( 112 1 )ہاں( Yes 2 )نہیں( No Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 G2. Do you think your DISCO will ask you to train other linespersons in your DISCO? (CIRCLE ONE NUMBER) آپ) کیا سمجھتے ہیں کہ آپکا تقسیم کار ادارہ دیگر الئن مینوں کی تربیت کرانے کیلئے آپکو کہے گا؟( 1 )ہاں( Yes 2 )نہیں( No Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 G3. Will you transfer to your colleagues and subordinate the knowledge and skills that you gained from training? )کیا آپ نے تربیت سے جو علم اور مہارتیں حاصل کیں ہیں کیا وہ آپ اپنے ساتھیوں اور (NUMBER ONE CIRCLE( ماتحتوں کو منتقل کریں گے؟( 1 )ہاں( Yes 2 )نہیں( No Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 G4. In your opinion, what do you think should be done to ensure that the knowledge and skills, gained from this training are transferred to other remaining and new linespersons at your DISCO? (MULTIPLE RESPONSES ALLOWED) )آپ کے خیال میں اس بات کو یقینی بنانے کیلئے کہ آپکے تقسیم کار ادارے میں تربیت حاصل کرنے سے رہ جانیوالے اور نئے بھرتی ہونیوالے الئن مینوں کو تربیت سے حاصل ہونیواال علم اور مہارتیں منتقل ہو جائیں، کیا کرنا چاہیئے؟( 1 )مزید تربیتی نشستوں کااہتمام( trainings More .a کار کو متعارف اور نافذ کیا جائے( SOP implement and Introduce .b ٴ 2 )معیاری ضابطہ ٴ ں کے گروپ کا قیام( groups mentor Create .c 3 )تربیت کاررہنماو )تربیت یافتہ لوگوں کے ذریعے تربیت کے نظام trained through training of system a Introduce .d 4 کو متعارف کیا جائے( 5 )حفاظتی آالت کی فراہمی( equipment safety Provide .e )الئن مینوں کے درمیان حفاظتی رواجاتکا نفاذ( linesmen among practices work safe Enforce .f 6 g. Other (specify hereدیگر۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔ )90 h. Don’t know (DO NOT READ) نہیں۔ معلوم 98 i. Refused to answer/No reply (DO NOT READ) انکار۔ سے جواب 99 G5. In your opinion, what do you think should be done to make safety a key management consideration at your DISCO? (MULTIPLE RESPONSES ALLOWED) )آپ خیال میں ایسا کیاکرنا چاہئیے کہ الزمین کی حفاظت آپکے تقسیم کار ادارے کی انتظامی ترجیح بن جائے؟( a. Training of management )تربیت کی انتظامیہ )1 2 )انتظامیہ کو آگاہی فراہم کرنا( management of awareness/Education .b 3 )قانون کی پابندی کو متعارف کرانا( law binding Introduce .c d. Regulation from NEPRA )قوانین کے نیپرا )4 5 )حفاظتی آالت کی فراہمی( equipment safety Provide .e f. Enforce safe work practices among linesmen رواجاتکا حفاظتی درمیان کے مینوں الئن) 6 سختی سے نفاذ( 113 g. Other (specify hereدیگر۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔ )90 h. Don’t know (DO NOT READ) نہیں۔ معلوم 98 i. Refused to answer/No reply (DO NOT READ) انکار۔ سے جواب 99 )صرف سپروائزرز کیلئے( Only Supervisors For G6. Based on your experience with trainings provided in the past, do you think that the trained linesmen will continue to use the knowledge and skills in their routine work? (CIRCLE ONE NUMBER) )گزشتہ دنوں فراہم کی گئیں تربیتی نشستوں کے حوالے سے آپ سمجھتے ہیں کہ کیاتربیت یافتہ الئن مین، سکھایا گیا علم اور مہارتوں کو اپنے روزہ مرہ کے فرائض میں استعمال کرنا جاری رکھیں گے؟( 1 )ہاں( Yes 2 )نہیں( No Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 G7. Do you think that other linesmen, who were not included in the training, will also, learn the new knowledge and skills from their trained colleagues? (CIRCLE ONE NUMBER) )کیا آپ سمجھتے ہیں کہ وہ الئن مین جو تربیتی پروگرام کا حصہنہیں تھے، کیا اپنے تربت یافتہ ساتھیوں سے علم اور مہارتیں سیکھیں گے؟( 1 )ہاں( Yes 2 )نہیں( No Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 G8. Do you think that the trained linesmen will actively share the knowledge and skills from training with their untrained colleagues? (CIRCLE ONE NUMBER) )کیا آپ سمجھتے ہیں کہ تربیت یافتہ الئن مین فعال طور پر علم اور اپنی مہارتیں اپنے دیگر غیر تربیت یافتہساتھیوں کو سکھائیں گے؟( 1 )ہاں( Yes 2 )نہیں( No Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 G9. What the DISCO management should do to ensure that new knowledge and skills from this training becomes standard work practice among all linesmen at your DISCO? (MULTIPLE RESPONSES ALLOWED) کار تقسیم) ادارے کی انتظامیہ کو تربیت کے دوران سکھائے گئے علم اور مہارتوں کو ادارے کے الئن مینوں کے درمیان ایک معیار بنانے کیلئے کیا کرنا چاہئیے؟( 1 )مزید تربیتی نشستوں کااہتمام( trainings More .a کار کو متعارف اور نافذ کیا جائے( SOP implement and Introduce .b ٴ 2 )معیاری ضابطہ 3 )تربیت کاروں کے گروپ کا قیام( groups mentor Create .c )تربیت یافتہ لوگوں کے ذریعے تربیت کے نظام trained through training of system a Introduce .d 4 کو متعارف کیا جائے( 5 )حفاظتی آالت کی فراہمی( equipment safety Provide .e )الئن مینوں کے درمیان حفاظتی رواجاتکا سختی linesmen among practices work safe Enforce .f 6 سے نفاذ( g. Other (specify hereدیگر۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔ )90 114 h. Don’t know (DO NOT READ) نہیں۔ معلوم 98 i. Refused to answer/No reply (DO NOT READ) انکار۔ سے جواب 99 H. GAP IDENTIFICATION AND RECOMMENDATIONS )سفارشات اور الکینشاندہی) H1. Do you think the training covered everything that you expected from it? (CIRCLE ONE NUMBER) ٴ ں کا احاطہ کیا ہے جن کی توقع کی جارہی تھی؟( )کیا آپ سمجھتے ہیں کہ تربیتی پروگرام نے ان تمام پہلوو Yes )ہاں )1)SKIP TO QTS H3) 2 )نہیں( No Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 H2. If NO, what areas you think were not covered in the training? (MULTIPLE RESPONSES ALLOWED) نہیں اگر) تو آپکے خیال میں وہ کون سے پہلو ہیں جن کا تربیت کے دوران احاطہ نہیں ہو سکا؟( 1 )نظری علم( Knowledge Theoretical 2 )جامع نصاب( Curriculum Comprehensive 3 )عملی رہنمائی کی تربیت( Training on Hands 4 )معیاری ضابطہ ٴ کار اور ہدایت نامے( guidebooks and SOPs 5 )طباعت شدہ آگاہی کا مواد( material awareness/education Printed 6 )آڈیو اور وڈیو میں آگاہی کا مواد( material awareness/education AV g. Other (specify hereدیگر۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔ )90 h. Don’t know (DO NOT READ) نہیں۔ معلوم 98 i. Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 115 H3. If similar trainings are organized in future, what additional components would you like in them? (MULTIPLE )اگر اسی طرح کی تربیت کا مستقبل میں اہتمام کیا جائے تو مزید کون سے مندرجات آپ چاہیں گے اس میں (ALLOWED RESPONSES شامل ہوں( 1 )نظری علم( Knowledge Theoretical 2 )جامع نصاب( Curriculum Comprehensive 3 )عملی رہنمائی کی تربیت( Training on Hands 4 )معیاری ضابطہ ٴ کار اور ہدایت نامے( guidebooks and SOPs 5 )طباعت شدہ آگاہی کا مواد( material awareness/education Printed 6 )آڈیو اور وڈیو میں آگاہی کا مواد( material awareness/education AV g. Other (specify hereدیگر۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔ )90 h. Don’t know (DO NOT READ) نہیں۔ معلوم 98 i. Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 THANKS )شکریہ( 116 ) انٹرویو کی تاریخ( : Date Interview _____________ INDUSTRIAL MOTOR SURVEY انڈسٹریل موٹر سروے For End-users (buyers/beneficiaries) )آخر صارف، خیدار/مستفید کنندہ کیلئے( )تعارف( Introduction Dear Sir/Madam, recently under a USAID funded program, energy efficient motors were introduced to you. Our records indicated that you participated in the program and some motors were installed at your facility. USAID is now in the process of conducting evaluation of this important activity. We need your cooperation to this regards by providing us with relevant information and data so that USAID can establish if the project achieved its intended objectives and outcomes. We therefore, request you to kindly spare some of your valuable time for an interview, which is likely not to take more than 1 hour. USAID very much appreciates your cooperation and support to this regards. محترم/محترمہ، حالیہ دنوں میں امریکی ادارہ برائے بین االقوامی امداد کی جانب سے فراہم کی گئیں کم توانائی والی موٹریں آپکو متعارف کرائی گئیں تھیں، ہمارے ریکارڈ کے مطابق آپ نے اس پروگرام میں شرکت کی تھی اور کچھ موٹریں آپکے ادارہ میں نصب کی گئیں ہیں۔ یو۔ایس۔ایڈ ان ِ دنوں اس اہم پروگرام کے تجزیہ میں مصروف عمل ہے۔ اس عمل میں ہمیں آپکے تعاون اور متعلقہ معلومات کی ضرورت ہے تا کہ یو۔ایس۔ایڈ یہ ٰ ہم آپ سے درخواست کرتے ہیں جان سکے کہ آیا اس پروگرام نے اپنے مطلوبہ مقاصد اور نتائج حاصل کر لئے ہیں کہ نہیں۔ لہذا کہ برائے مہربانی اپنے قیمتی وقت سے کچھ وقت ایک انٹرویو کیلئے مختص کیجئے اور یہ انٹرویو ایک گھنٹے سے اوپر نہیں ہوگا۔ یو۔ایس۔ایڈ اس عمل میں آپکے تعاون اور مدد کو انتہائی قدر کی نگاہ سے دیکھتا ہے۔ )عمومی اعداد وشمار( DATA GENERAL .A _______________________________________________________________سہولت کار ادارہ :Facility 1. _______________________________________________________ادارہ جس کی شاخ ہے :Organization Parent 2. (NUMBER ONE CIRCLE(توانائی فراہم کرنیوالے ادارے کا نام :Company Supply Power 3. Islamabad Electric Supply Company(IESCO) 1 Karachi Electric Supply Corporation (KESC) 2 Quetta Electric Supply Corporation (QESCO) 3 Lahore Electric Supply Company(LESCO) 4 Peshawar Electric Supply Company(PESCO) 5 Multan Electric Power Company(MEPCO) 6 Hyderabad Electric Supply Company(HESCO) 7 Faisalabad Electric Supply Company(FESCO) 8 Gujranwala Electric Supply Company(GEPCO) 9 Other (please specify): 90 Don’t know (DO NOT READ) 98 ___________________________________________________________جواب دہندہ کا نام :Respondent of Name 4. ________________________________________________________ جواب دہندہ کا عہدہ :Respondent of Position 5. ___________________________________________________________________________ رابطہ نمبر :Phone 6. 7. Main Business/Industrial Activity: سرگرمی صنعتی اہم_________________________________________________ 1. What is your average monthly electricity usage (kWh), during last 1 year? )گزشتہ ایک سال کے دوران، آپکی بجلی کا اوسطاً ماہانہ کتنا استعمال ہوتا ہے ؟( یہاں اندراج کریں۔ (HERE RESPONSE RECORD( )کسی دیگر جواب کی صورت میں متعلقہ جواب نیچے below response relevant the circle response other any For دائرہ لگاکر دیں( Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer / No reply (DO NOT READ) دیا۔ نہیں جواب 99 2. What is the average monthly cost of electricity (Rs.), during last 1 year? ماہانہ ًاوسطا کی بجلی ،دوران کے سال ایک گزشتہ) قیمت کیا تھی؟( Serial number نمبر شمار )دفتری استعمال کیلئے( (FOR OFFICE USE ONLY) )بجلی کا استعمال( USAGE ELECTRICITY .B 117 یہاں اندراج کریں۔ (HERE RESPONSE RECORD( For any other response circle the relevant response below جواب متعلقہ میں صورت کی جواب دیگر کسی) نیچے دائرہ لگاکر دیں( Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer / No reply (DO NOT READ) دیا۔ نہیں جواب 99 3. Is your electricity tariff subjected to Maximum Demand Charges? (CIRCLE ONE NUMBER) )کیا آپکی بجلی کے نِرخ " زیادہ سے زیادہ طلب کے چارجز" پر منحصر ہوتے ہیں؟( Yes )ہاں )1 GO TO B4 No )نہیں )2 SKIP TO B5 Don’t know (DO NOT READ) )نہیں معلوم )98 SKIP TO B5 Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 SKIP TO B5 )اگر ہاں، تو آپکی ایک سال کی زیادہ سے زیادہ طلب کتنی ہوتی ہے؟( ?(kW (year a in demand maximum your is what ,YES If 4. یہاں اندراج کریں۔ (HERE RESPONSE RECORD( For any other response circle the relevant response below جواب متعلقہ میں صورت کی جواب دیگر کسی) نیچے دائرہ لگاکر دیں( Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer / No reply (DO NOT READ) دیا۔ نہیں جواب 99 5. Is your electricity tariff subjected to Power Factor Charges? (CIRCLE ONE NUMBER) )کیا آپکی بجلی کے نِرخ "پاور فیکٹر چارجز" پر منحصر ہوتے ہیں؟ Yes )ہاں )1 GO TO B6 No )نہیں )2 SKIP TO B7 Don’t know (DO NOT READ) نہیں۔ معلوم 98 SKIP TO B7 Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 SKIP TO B7 6. If YES, what has been your average Power Factor value (%), during last 1 year? )اگر ہاں تو گزشتہ ایک سال کے دوران آپکا اوسطاً " پاور فیکٹر "کتنے فیصد تھا؟( یہاں اندراج کریں۔ (HERE RESPONSE RECORD( For any other response circle the relevant response below جواب متعلقہ میں صورت کی جواب دیگر کسی) نیچے دائرہ لگاکر دیں( Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer / No reply (DO NOT READ) دیا۔ نہیں جواب 99 7. Have you estimated or have rough idea of energy consumed by large industrial end-uses? )کیا آپکو کچھ اندازہ ہے کہ بڑے صنعتی )آخر( صارف کتنی بجلی خرچ کرتے ہیں؟( (NUMBER ONE CIRCLE( Yes )ہاں )1 GO TO B8 No )نہیں )2 SKIP TO B9 Don’t know (DO NOT READ) نہیں۔ معلوم 98 SKIP TO B9 Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 SKIP TO B9 8. If YES, please provide below approximate percentage share, in annual electricity consumption, of various end-uses; )اگر ہاں تو برائے مہربانی ہمیں بتائیے کہ بجلی کے ساالنہ مصرف کی مد میں درج ذیل مختلف آخر صارفین کااندازا ًکتنا فیصد حصہ ہے؟( End-use )صارفین آخر )% Share a. Industrial Motors )موٹریں صنعتی ) )ریفریجریشن یا سرد سازی( Refrigeration .b )ائر کنڈیشننگ یا باد گردگی بشمول چلرز (AHU and Chillers Including (Conditioning Air .c ِ کے( ِ )کمپریسڈ ہوا یا باد فشاری( Air Compressed .d )پانی کی پمپنگ( Pumping Water .e f. Elevators/Lifts )مصاعد لفٹیںیاِ) 118 g. Office Equipment )آالت دفتری) )روشنی کے آالت( Lighting .h i. Industrial Processes (e.g. heating, cooling, drying, cooking, etc.) صنعتی عملیات مثالً حرارت یا ٹھنڈک پیدا کرنے کا عمل، خشک کنندگی اور کھانا پکانا وغیرہ )پانی کو گرم کرنے کا عمل( Heating Water .j k. Other (Please describeعملیات۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔ دیگر) 9. Is cost of electricity a great concern for your business? (CIRCLE ONE NUMBER) معاملہ اہم کارکا و کسب آپکے قیمت کی بجلی کیا) ہے؟( 1 )ہاں( Yes 2 )نہیں( No Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 )کیا آپ نے بجلی کے نرخوں میں کمی کرنے کے کوئی (NUMBER ONE CIRCLE? (costs electricity reduce to ways try you Did 10. طریقے آزمائے ہیں؟( Yes )ہاں )1 GO TO B11 No )نہیں )2 SKIP TO B13 Don’t know (DO NOT READ) نہیں۔ معلوم 98 SKIP TO B13 Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 SKIP TO B13 11. If YES, Please describe what ways you have done and their level of effectiveness (CIRCLE ONE NUMBER IN EACH ROW OF INITIATIVE WHICH WAS ADOPTED). )اگر ہاں، تو کیا آپ مجھے بتا سکتے ہیں کہ وہ کون سے طریقے تھے جو آپ نے آزمائے اور اُن کی فعالیت کیا تھی؟( Energy Efficiency (EE) Initiative )تونائی کی فعالیت کیلئے اٹھائے گئے اقدامات( Extremely Effective ٴ )انتہائی موثر( Very Effective ٴ )بہت موثر( Effective ٴ )موثر( Fairly Effective ٴ )کچھ حد تک موثر( Not Effective ٴ )غیر موثر( Don’t Know )معلوم نہیں( a. Conducted Energy Audits )پڑتال کی تونائی )1 2 3 4 5 98 b. Education/Awareness of Workers on EE )تونائی کی فعالیت پر کارکنان کی تربیت/آگاہی( 1 2 3 4 5 98 )توانائی کی فعالیت ک الئٹنگ کی تنصیب( Lighting EE Installed .c 1 2 3 4 5 98 98 5 4 3 2 1 )تونائی کی فعالیت کی موٹروں کی تنصیب( Motors EE Installed .d )توانائی کی فعالیت کے دیگر technologies EE other Installed .e طریقوں کی تنصیب( 1 2 3 4 5 98 f. Other (Please Describe):دیگر۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔ 1 2 3 4 5 98 13. Have you ever used the services of professional energy efficiency professionals? (CIRCLE ONE NUMBER) )کیا آپ نے کبھی تونائی کی فعالیت کے ماہرین کے پیشہ وارانہ خدمات استعمال کی ہیں؟( Yes )ہاں )1 GO TO B14 No )نہیں )2 SKIP TO C1 Don’t know (DO NOT READ) نہیں۔ معلوم 98 SKIP TO C1 Refused to answer/No reply (DO NOT READ) نہیں۔ معلوم 99 SKIP TO C1 14. If YES, please provide below, which ones and their level of effectiveness. (CIRCLE ONE NUMBER IN EACH ROW OF SERVICE WHICH WAS USED). )اگر ہاں، تو برائے مہربانی درج ذیل میں سے کون سی پیشہ وارانہ خدمات حاصل کی گئیں اور انکی فعالیت کس درجہ کی تھی؟ ہر قطار میں دی گئی اس خدمت کو دائرہ لگائیں جو استعمال کی گئی۔( 119 Energy Efficiency Professional Services )تونائی کی فعالیت کی پیشہ وارانہ خدمات( Extremely Effective )انتہائی موئثر( Very Effective )بہت موئثر( Effective )موئثر( Fairly Effective )کچھ حدتک موئثر( Not Effective موئثر( )غیر Don’t know )معلوم نہیں( 98 5 4 3 2 1 )تونائی کی پڑتال( Audits Energy .a 98 5 4 3 2 1 )تونائی کی صرفیت کا جائزہ( consumption energy of Review .b 98 5 4 3 2 1 )تونائی کی فعالیت کے طریقوں کی تنصیب( technologies EE Install .c 98 5 4 3 2 1 )تونائی کی فعالیت کے اقدامات پر تجاویز( measures EE on advice Provide .d )تونائی کی فعالیت کے اقدامات کی measures EE of effectiveness Evaluate .e اثرپذیری کا تجزیہ( 1 2 3 4 5 98 f. Other (Please Describeدیگر۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔ )1 2 3 4 5 98 )فعال موٹروں کے پروگرام میں شمولیت( PROGRAM MOTORS EFFICIENT IN ENGAGEMENT .C 1. How did you become engaged in Industrial Efficient Motors Program? (Multiple responses allowed) )آپ فعال صنعتی موٹروں کے پروگرام میں کیسے شامل ہوئے تھے؟( a. Saw the advertisement and contacted program پروگرام کر دیکھ اشتہار) سے رابطہ کیاتھا( 1 2 )پروگرام ٹیم نے خود رابطہ کیا تھا( team program by approached Was .b 3 )کسی نے پروگرام کی سفارش کی تھی( recommended Someone .c d. Other (Specifyدیگر۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔ )90 e. Don’t know (DO NOT READ) نہیں۔ معلوم 98 f. Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 2. What was the main reason for participation in the program? (CIRCLE ONE NUMBER) )پروگرام کی شمولیت کی سب سے اہم وجہ کیا تھی؟( 1 )تونائی کی قیمت کی بچت( costs energy Save 2 )پیداوار کی بڑھوتری( production Improve )تونائی کی فراہم کے معیار کو بہتر کرنا( quality supply power Improve 3 )بجلی کے تقسیم کار اداروں سے DISCO with relationship Improve تعلقات کو بہتر کرنا( 4 5 )مالی فوائد کی ترغیب( Incentives Financial Other (Specifyدیگر۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔ )90 Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 3. Please mark what has been installed at your premises and their power ratings under the program. (Multiple responses )برائے مہربانی بتائیے کہ آپکے احاطہ میں کیا کچھ نصب ہے اور پروگرام کے تحت انکی درجہ بندی کیا ہے؟( (allowed 3a. Equipment Installed )آالت شدہ نصب )3b. Power Rating (kW)بندی درجہ Unit1 Unit2 Unit3 Unit4 Unit 5 a. Only energy efficient motors Installed 1 )صرف تونائی کی فعالیت کی موٹریں نصب ہیں( b. Only Variable Speed Drives (VSD) Installed 2 )صرف "متغیر رفتار کے ڈرائیورز" نصب ہیں( c. Both EE Motors and VSDs Installed 3 )درج اال موٹریں اور ڈرائیورز دونوں نصب ہیں( 120 d. None (IF ANSWER IS NONE FOLLOW 4 کچھ بھی نہیں۔ (INSTRUCTION TERMINATE INTERVIEW )انٹرویو کا اختتام کردیں( e. Other (Specifyدیگر۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔ )90 )آپکے احاطہ میں کام کا اجراٴ کب ہوا تھا( ?premises your at implemented work the was When 4. (RECORD RESPONSE HERE): MM/YY )کریں درج یہاں) _______/_______ For any other response circle the relevant response below )کسی دیگر جواب کی صورت میں متعلقہ جواب نیچے دائرہ لگاکر دیں( Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer / No reply (DO NOT READ) دیا۔ نہیں جواب 99 )آالت کی تنصیب کس نے کی تھی؟( (allowed responses Multiple? (equipment the installed Who 5. 1 )آالت کے فراہم کنندگان نے( Supplier Equipment The .a 2 )پروگرام کے نفاذ کنندگان نے( Implementer Program .b 3 )آپکے اپنے تکنیکی عملے نے( Staff technical Your .c 4 )کسی دیگرشخص ثالث نے( Party Third .d e. Other (Please specifyدیگر۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔ ) 90 Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer / No reply (DO NOT READ) نہیں جواب دیا۔ 99 6. Were you satisfied with installation? (CIRCLE ONE NUMBER) )ہیں؟ مطمئن سے تنصیبات آپ کیا) 1 )ہاں( Yes 2 )نہیں( No Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer / No reply (DO NOT READ) نہیں جواب دیا۔ 99 7. Please rank in the order of preference the key benefits of the equipment? (Mark 1, 2, 3 against each benefit) )برائے مہربانی ترجیح کےحساب سے آالت کے اہم فوائد کی درجہ بندی کریں؟( )تونائی کی بچت( Savings Energy .a b. Monetary Savings )بچت مالی) )طلب کی بچت( Savings Demand .c )لوڈ شیڈنگ میں کمی( shedding load Reduce .d )توانائی کے معیار میں بہتری( quality power Improve .e )پیداوار میں اضافہ( production Improve .f g. Other (Specifyدیگر۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔) 8. Did you pay for the equipment? (CIRCLE ONE NUMBER) )کی ادائیگی کوئی کیلئے آالت نے آپ کیا) 1 ُ جی ہاں۔ کلی ادائیگی۔ (price full (Yes Yes (part of the price) ادائیگی۔ جزوی ہاں۔ جی ُ 2______ % 3 )نہیں( No Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer/No reply (DO NOT READ) نہیں جواب دیا۔ 99 )پروگرام میں شمولیت کا تجربہ( EXPERIENCE PARTICIPATION PROGRAM .D 1. Has the equipment, under the program, fully installed and tested? (CIRCLE ONE NUMBER) )کیا پروگرام کے تحت دیئے گئے آالت مکمل طور پر آزمائے گئے تھے اور تنصیب کر دیئے گئےہیں؟( Yes )ہاں )1 SKIP TO D3 No )نہیں )2 GO TO D2 121 Don’t know (DO NOT READ) نہیں۔ معلوم 98 SKIP TO D3 Refused to answer/No reply (DO NOT READ) نہیں جواب دیا۔ 99 SKIP TO D3 2. If NO, when do you expect the equipment to be installed and tested? )اگر نہیں، تو کیا آپ بتا سکتے ہیں کہ آپ کب تک ان کی آزمائش اور تنصیب کی توقع کرتے ہیں؟( (RECORD RESPONSE HERE): MM/YY )کریں درج یہاں_______ )/_______ For any other response circle the relevant response below )کسی دیگر جواب کی صورت میں متعلقہ جواب نیچے دائرہ لگاکر دیں( Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer / No reply (DO NOT READ) دیا۔ نہیں جواب 99 3. Was the equipment of your choice installed? (CIRCLE ONE NUMBER) )ہیں؟ گئے کئے نصب آالت کے پسند آپکی کیا) 1 )ہاں( Yes 2 )نہیں( No 3 )انتخاب کا اختیا ر نہیں تھا( choice a have not Did Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 4. Are you satisfied with the quality of equipment? (CIRCLE ONE NUMBER) )ہیں؟ مطمئن سے معیار کے آالت آپ کیا) 1 )ہاں( Yes 2 )نہیں( No Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 5. Are you satisfied with the price of equipment? (CIRCLE ONE NUMBER) )ہیں؟ مطمئن سے قیمتوں کی آالت آپ کیا) 1 )ہاں( Yes 2 )نہیں( No 3 )آالت کی قیمت ادا نہیں کی( equipment for pay not Did Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 6. Please describe the intended outcomes of the work carried out at your premises (Multiple responses allowed) )آپکے احاطہ میں جو کام کیا جا رہا ہے، برائے مہربانی اسکے مطلوبہ نتائج بیان کیجئیے۔( 1 )تونائی کی بچت( Savings Energy .a b. Monetary Savings )بچت مالی )2 3 )طلب کی بچت( Savings Demand .c 4 )لوڈ شیڈنگ میں کمی( shedding load Reduce .d 5 )توانائی کے معیار میں بہتری( quality power Improve .e 6 )پیداوار میں اضافہ( production Improve .f g. Other (Specifyدیگر۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔ )90 h. Don’t know (DO NOT READ) نہیں۔ معلوم 98 i. Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 )بعد از شمولیت( Participation Post )کیا نصب شدہ آالت ٹھیک طور پر کام کررہے (NUMBER ONE CIRCLE? (normally operating equipment installed the Is 7. ہیں؟( 1 )ہاں( Yes 2 )نہیں( No Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer/No reply (DO NOT READ) نہیں جواب دیا۔ 99 122 8. Has the installed equipment affected your processes/production? (CIRCLE ONE NUMBER) )کیا نصب شدہ آالت آپکی پیداوار اور کار کردگی پر اثرانداز ہوئے ہیں؟( Yes )ہاں )1 GO TO D9 No )نہیں )2 SKIP TO D10 Don’t know (DO NOT READ) نہیں۔ معلوم 98 SKIP TO D10 Refused to answer/No reply (DO NOT READ) نہیں جواب دیا۔ 99 SKIP TO D10 9. If you answered YES, please rank below in the order of severity? (Mark 1, 2, 3 against each option) )اگر ہاں، تو شدت کے حساب سے درج ذیل کی درجہ بندی کریں۔ نوٹ: A، ،2 3کے درجے تحریر کریں۔( )پیداوار میں کمی( production Reduced .a )مشینوں کے ما حاصل میں کمی( machines of output Reduced .b )بہت زیادہ شور شرابہ( Noise High .c )قابو کرنے میں دشواری( Control to Difficult .d )کام میں تعطل کا باعث( disruptions Causes .e f. Other (Specifyدیگر۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔) 123 10. Has the work done under the project achieved claimed/intended results? (CIRCLE ONE NUMBER) )کیا پروگرام کے تحت کیئے گئے کام نے اپنے مطلوبہ نتائج حاصل کئے ہیں؟( Yes )ہاں )1 SKIP TO D12 No )نہیں )2 GO TO D11 Don’t know (DO NOT READ) نہیں۔ معلوم 98 SKIP TO D12 Refused to answer/No reply (DO NOT READ) نہیں جواب دیا۔ 99 SKIP TO D12 )اگر نہیں، تو ابھی ت ک کیا کچھ حاصل نہیں کیا جا سکا؟( (allowed responses Multiple? (achieved been NOT has what ,NO If 11. 1 )تونائی کی بچت( Savings Energy .a b. Monetary Savings )بچت مالی )2 3 )طلب کی بچت( Savings Demand .c 4 )لوڈ شیڈنگ میں کمی( shedding load Reduce .d 5 )توانائی کے معیار میں بہتری( quality power Improve .e 6 )پیداوار میں اضافہ( production Improve .f g. Other (Specifyدیگر۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔ )90 12. Please rate the following attributes of the program? (CIRCLE ONE NUMBER IN EACH ROW) )برائے مہربانی پروگرام کی درج ذیل خوبیوں کی درجہ بندی کیجئے( )پروگرام کی خوبیاں( Attributes Program Excellent Very Good Good Fair Poor Cannot Rate a. Marketing and Promotion )ترویج اور بازاریابی )1 2 3 4 5 6 b. Interaction and Communication with program team رابطہ سے ٹیم پروگرام) کاری( 1 2 3 4 5 6 c. Process of Registration and Participation )عملیات کے شمولیت اور اندراج )1 2 3 4 5 6 d. Process of Equipment Commissioning and Testing طالقیت اور آزمائش کی آالت) کے عملیات( 1 2 3 4 5 6 6 5 4 3 2 1 )نگرانی اور تجزیہ کاری( Evaluation and Monitoring .e f. Reporting of Results to You )رپورٹنگ کی نتائج )1 2 3 4 5 6 g. Overall Program as a Whole )پر طور کلی پروگرام )1 2 3 4 5 6 E. SUSTAINABILITY )استقامت) 1. Would you continue using the installed equipment? (CIRCLE ONE NUMBER) جاری استعمال کا آالت شدہ نصب آپ کیا) رکھیں گے؟( 1 )ہاں( Yes 2 )نہیں( No Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer/No reply (DO NOT READ) نہیں جواب دیا۔ 99 2. Will you install more equipment of the same type? (CIRCLE ONE NUMBER) کرو نصب آالت اور کے نوعیت اسی آپ کیا) گے؟( Yes )ہاں )1 SKIP TO E4 No )نہیں )2 GO TO E3 124 Don’t know (DO NOT READ) نہیں۔ معلوم 98 SKIP TO E5 Refused to answer/No reply (DO NOT READ) نہیں جواب دیا۔ 99 SKIP TO E5 )اگر نہیں، تو کیا آپ بتا سکتے ہیں کہ کیوں؟( (allowed responses Multiple? (why describe please ,NO If 3. 1 )اضافی آالت کی ضرورت نہیں ہے( equipment additional for need No .a )ماہرین کے مدد کے بغیر نصب نہیں کر support expert without install Cannot .b سکتے( 2 3 )معلوم نہیں کہ کیسے خریدے جاتے ہیں( procure to how know t’Don .c 4 )قیمت کے متحمل نہیں ہوسکتے( price the afford Cannot .d e. It does not yield claimed results )کرتے نہیں مہیا نتائج مطلوبہ یہ )5 f. There are no professionals to design and install ماہرین کرنیوالے نصب اور بنانے) موجود نہیں ہیں( 6 g. Other (specifyدیگر۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔ )90 h. Don’t know (DO NOT READ) نہیں۔ معلوم 98 i. Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 4. If you answered YES to E2, please describe why? (Multiple responses allowed) )کیوں؟ کہ ہیں سکتے بتا آپ کیا تو ،ہاں اگر) 1 )یہ بجلی کے بِلوں میں کمی التے ہیں( bills electricity reduces It .a 2 )یہ توانائی کی رسد کے معیا کو بہتر کرتے ہیں( quality supply power improves It .b c. It improves the life of our equipment )ہیں بڑھاتے عمر کے آالت یہ )3 4 )یہ پیداواری فعالیت کو بہتر کرتے ہیں( efficiency production improves It .d )صرف اس لئے کہ تقسیم کار ادارہ ایسا کرنے کا کہتا so do to us asks DISCO if Only .e ہے( 5 6 )صرف تب جب یہ مفت میں فراہم کئے جائیں( free for offered is it if Only .f g. Other (specify دیگر۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔ )90 h. Don’t know (DO NOT READ) نہیں۔ معلوم 98 i. Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 5. In your opinion, is there a widespread use of energy efficient motors among industries in Pakistan? (CIRCLE ONE )آپکی رائے کے مطابق، کیا پاکستان کی صنعتوں میں تونائی کی فعالیت کی موٹروں کی بڑے پیمانے پر ضرورت ہے؟( (NUMBER 1 )ہاں( Yes 2 )نہیں( No Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer/No reply (DO NOT READ) نہیں جواب دیا۔ 99 6. In your opinion is there sufficient awareness, about energy efficient motors and their benefits, among industries in )آپکی رائے کے مطابق،کیا پاکستان کی صنعتوں میں توانائی کی فعالیت کی موٹروں سے متعلق کافی (NUMBER ONE CIRCLE? (Pakistan آگاہی ہے؟( 1 )ہاں( Yes 2 )نہیں( No Don’t know (DO NOT READ) نہیں۔ معلوم 98 Refused to answer/No reply (DO NOT READ) نہیں جواب دیا۔ 99 125 7. What are the main reasons for low penetration of energy efficient motors in Pakistani industries? (Multiple responses )پاکستا ن کی صنعتوں میں تونائی کی فعالیت کی موٹروں کے کم نفوذ کی سب سے اہم وجوہات کون کون سی ہیں؟( (allowed a. Lack of Awareness )کمی کی آگاہی )1 b. Lack of availability of Equipment )دستیابی غیر کی آالت )2 3 )بڑھی ہوئی قیمتیں( price High .c d. Lack of availability of professional/expert services غیر کی ماہرین) دستیابی( 4 e. Other (specify دیگر۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔ )90 f. Don’t know (DO NOT READ) نہیں۔ معلوم 98 g. Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 8. Will you recommend installation of equipment to your peers? (CIRCLE ONE NUMBER) )کیا آپ اپنے ساتھیوں کو ان آالت کی تنصیب کی سفارش کرو گے؟( Yes )ہاں )1 SKIP TO E10 No )نہیں )2 GO TO E9 Don’t know (DO NOT READ) نہیں۔ معلوم 98 SKIP TO E10 Refused to answer/No reply (DO NOT READ) نہیں جواب دیا۔ 99 SKIP TO E10 9. If NO, please describe why? (Multiple responses allowed) )کیوں؟ تو،نہیں اگر) a. It does not yield claimed results and is not worth installing انکو اور کرتے مہیا نہیں نتائج مطلوبہ یہ) نصب کرنا منافع بخش نہیں( 1 b. Too costly for the benefit they produce )ہے پڑتی مہنگی بہت پیداوار انکی )2 c. Require professional support that is not available in Pakistan میسر میں پاکستان امداد ماہرانہ مطلوبہ) نہیں ہے( 3 d. Other (specifyدیگر۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔ )90 e. Don’t know (DO NOT READ) نہیں۔ معلوم 98 f. Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 10. In your opinion what needs to be done to promote adaptation of energy efficient motors by other industries? ؑ )آپکی رائے میں دیگر صنعتوں میں توانائی کی فعالیت والی موٹروں کی سازگاری کی ترویج کیلیے کیا کرنے (allowed responses Multiple( کی ضرورت ہے؟( a. Actively promote them through marketing campaigns سے ذریعہ کے مہمات کی مارکیٹنگ) محرک انداز میں ترویج( 1 b. Government should waive taxes to reduce prices محصوالت گورنمنٹ کیلئے کمی میں قیمتوں) میں چھوٹ دے( 2 c. Someone (government or DISCOs) should provide expert advice and services )گورنمنٹ یا تقسیم کاری کے ادارے ماہرانہ رائے اور خدمات فراہم کریں( 3 d. Other (specifyدیگر۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔ )90 e. Don’t know (DO NOT READ) نہیں۔ معلوم 98 f. Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 )خالٴ کی نشاندہی اور سفارشات( RECOMMENDATIONS AND IDENTIFICATION GAP .F 1. Do you feel that the industrial energy efficient motor program was properly designed and executed? (CIRCLE ONE )کیا آپ محسوس کرتے ہیں کہ تونائی کی فعالیت والی موٹروں کا صنعتی پروگرام درست طور پر ترتیب دیا گیا تھا اور نافذ ہوا ہے؟( (NUMBER Yes )ہاں )1 SKIP TO F3 No )نہیں )2 GO TO F2 Don’t know (DO NOT READ) نہیں۔ معلوم 98 SKIP TO F3 126 Refused to answer/No reply (DO NOT READ) نہیں جواب دیا۔ 99 SKIP TO F3 2. If NO, which areas below do you think need improvement? (Multiple responses allowed) )اگر نہیں تو آپکی رائے میں کس چیز کو بہتر کرنے کی ضرورت ہے؟( 1 )مارکیٹنگ اور ترویج( Promotion /Marketing .a 2 )صارفین کا اندراج( Registration Customer .b 3 )آالت کا معیار( Quality Equipment .c 4 )آالت کی مختلف اقسام( Variety Equipment .d 5 )الت کی تنصیب( Installation Equipment .e 6 )آالت کی ضمانت( Warranty Equipment .f g. Technical Support )امداد تکنیکی )7 h. Verification of benefits )تصدیق کی فوائد )8 i. Reporting of Results )رپورٹنگ کی نتائج )9 j. Better Price )قیمتیں بہتر )10 k. Other (specifyدیگر۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔ )90 l. Don’t know (DO NOT READ) نہیں۔ معلوم 98 m. Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 3. If similar energy efficiency programs are offered in future, what additional features/aspects do you wish to be )اگر اسی طرح کا فعالیت پروگرام مستقبل میں پیشکش کی جائے تو کیا اضافی خصوصیات آپ (allowed responses Multiple? (included چاہیں گے کہ اس میں شامل کی جائیں؟( 1 )آالت کا مزید بہتر معیار( Quality Equipment Better .a 2 )آالت کی وسیع اقسام( Variety Equipment Wide .b c. Better Information )معلومات بہتر )3 4 )آالت کی باقاعدہ ضمانت( Warranty Equipment Proper .d e. Technical Support )مدد تکنیکی )5 f. Training )تربیت )6 g. Reporting of Verified Results )رپورٹنگ کی نتائج مصدقہ )7 h. Other (specifyدیگر۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔۔ )90 i. Don’t know (DO NOT READ) نہیں۔ معلوم 98 j. Refused to answer/No reply (DO NOT READ) دیا۔ نہیں جواب 99 THANKS شکریہ 127 ANNEX IV: LIST OF INDIVIDUALS INTERVIEWED List of Interviewees from DISCOs: Group Interview No. No. of Participants in Each Group Names of the Interviewee/Participants Intervention/ Activity DISCO Location Contact Details Date of Interview 1. 1 Amana Rizwan, Deputy Manager (Training and Planning) Gender Equity Training IESCO Islamabad 0323-5155441 11-Sep-14 2. 2 Fayyaz Hussain Siddiqui, Chief Engineer Operations Islamabad Bakht Zaman, Superintendent Engineer, Operations Islamabad Lineman Training IESCO Islamabad 0333-8562009 03335123848 12-Sep-14 3. 2 Waheed Akram, Manager (Customer Services) Abid Tiwana, Assistant Manager, Management Information Systems Cost of Service Study IESCO Islamabad 0331-5210 351 0336-5135 687 11-Sep-14 4. 2 Riaz Qadir Bukhari, Customer Services Department Riaz, Sub Division Officer, Kamalabad, Rawalpindi Hand Held Units + Improved Meter Reading IESCO Islamabad 11-Sep-14 5. 2 Mushtaq Ahmed, DG Surveillance Pervaiz Iqbal, DG Administration Utility Exchange Program IESCO Islamabad 0333-5788881 051-2287059 12-Sep-14 6. 4 Fida Ahmed Khan, General Manager Technical Dr. Muhammad Amjad, Superintendent Engineer, Peshawar Circle Jamshed Ali Khan, Executive Engineer Cantonment Division, Peshawar Asif Khan, Sub Divisional Officer, Kohat Electronic Metering PESCO Peshawar 0315 8300601 03005976843 03005937365 03339146998 15-Sep-14 128 Group Interview No. No. of Participants in Each Group Names of the Interviewee/Participants Intervention/ Activity DISCO Location Contact Details Date of Interview Road, Peshawar 7. 2 Jamshed Ali Khan, Executive Engineer, Cantonment, Peshawar Abdullah Shah, Executive Engineer Operations, Rural Cantonment, Peshawar Lineman Training PESCO Peshawar 0313-9819900 0300-5937265 0312-5933268 15-Sep-14 8. 3 Mashkoor Khan, Manager Management Information Systems Syed Mohammad Hassan, CIS Core Team Member Haider Ali, CIS Core Team Member Customer Information System PESCO Peshawar 0315 8300401 03335998316 03336167776 16-Sep-14 9. 4 Shaukat Afzal, DG Public Relations Salma Gul, Commercial Officer Gul Nabi Syed, Additional Director PDC Saad ud Din Khilji, Deputy Director Commercial DISCO outreach and antitheft campaigns PESCO Peshawar 0311 5812000 0346 6695707/0332931 8415 0315 8300433 0315 8300537 16-Sep-14 129 Group Interview No. No. of Participants in Each Group Names of the Interviewee/Participants Intervention/ Activity DISCO Location Contact Details Date of Interview 10. 2 Mussarat Gul, Customer Services Director Adil Rehman, Deputy Manager Tariff Cost of Service Study PESCO Peshawar 0333-9152739 0315 8300523 0334-9002391 15-Sep-14 11. 5 Fida Ahmed Khan, General Manager Technical Anwar-ul-Haque Yousafzai, General Manager Finance Khurshid Ahmad Orakzai, Director General, HR Musarrat Gul, Chief Commercial Officer Muhammed Maskoor Khan, Management Information Systems, Manager ERP Implementation PESCO Peshawar 0315 8300601 0313 9999906 0300 5941974 0315 8300523 0315 8300401 15-Sep-14 12. 4 Musarrat Gul, Customer Services Department Dr. Muhammad Amjad, Superintendent Engineer, Peshawar Circle Jamshed, Executive Engineer Cantonment Division Asif Khan, Sub Divisional Officer Kohat Road S/division Hand Held Units PESCO Peshawar 0315 8300523 03005976843 03005937365 03339146998 15-Sep-14 130 Group Interview No. No. of Participants in Each Group Names of the Interviewee/Participants Intervention/ Activity DISCO Location Contact Details Date of Interview 13. 3 Khayyam Ilyas, Assistant Director Zeeshan Farid, Assistant Director Ali Raza, Assistant Director Planning & Engineering Modernization (GIS Mapping, System Analysis & Training) PESCO Peshawar 03339198143 03158300517 03349124173 16-Sep-14 14. 1 Ishrat Rashid, Commercial Assistant Utility Exchange Program PESCO Peshawar 03355950394 0336-9261482 091-9212179 16-Sep-14 15. 2 Fazal Ullah Durani, Director, Customer Services Head Quarters Shaukat Bukhari, Additional Manager/ Principal Regional Training Center Lineman Training MEPCO Multan 0345-8189017 0345-8189333 18-Sep-14 16. 2 Malik Imtiaz Ahmad, Manager Commercial Jahangir Bhutta, Manager Finance (CPC) Cost of Service Study MEPCO Multan 0300-6304407 0345-3944446 0333-6104991 0345-2004991 18-Sep-14 131 Group Interview No. No. of Participants in Each Group Names of the Interviewee/Participants Intervention/ Activity DISCO Location Contact Details Date of Interview 17. 4 Malik Imtiaz Ahmad, Add. Director General Muhammad Umer Lodhi, Executive Engineer City Division Saqib Inam, Sub Divisional Officer Pak Gate Subdivision Shahid Iqbal Chishti, Sub Divisional Officer Shamsabad Subdivision AMR Metering MEPCO Multan 0300-6304407 0300-6327727, 0345-5542777 0302-3575496, 0345-8540215 0333-6102259, 0345-8540257 19-Sep-14 18. 4 Mian Nadeem Ahmed, Additional Director General, Management Information Systems Hasan Tauqeer Bokhari, Chief Executive Officer, PITC Arshad Mahmood, Additional Director General, PITC Humayun Zafar, Assistant Manager, MIS Customer Information System MEPCO Multan 03453005253 03354260000 03003065886 03136321120 19-Sep-14 19. 3 Mian Ansar, Finance Director Liaquat Memon, Manager HR Naimat Ullah Qureshi, Manager Material Management ERP Implementation MEPCO Multan 0345-394 4447 0345-930 4444 0300-634 0693 18-Sep-14 20. 1 Jamshed Niazi, Public Relations Officer DISCO outreach and antitheft campaigns MEPCO Multan 18-Sep-14 132 Group Interview No. No. of Participants in Each Group Names of the Interviewee/Participants Intervention/ Activity DISCO Location Contact Details Date of Interview 21. 2 Irum Saba, Deputy Director Commercial Assad Hammad, Revenue Officer, Mumtazabad Gender Equity Training MEPCO Multan 03458441545 03006359088 18-Sep-14 22. 3 Mushtaq Ahmed, Executive Engineer Musapak Saad Shafiq, Sub Divisional Officer Nawan Shahar Rehan Ali Chohan, Sub Divisional Officer Gulgasht Emanul Sardar, Deputy Director Rana Tanveer, Sub Divisional Officer, Cantonment Hand Held Units MEPCO Multan 03458540181 03447006451 03456300999 03458540195 03007333369 03458540258 03006902789 03346057564 03006379740 18-Sep-14 23. 3 Asghar Khan, Manager Rafiq Bari, Assistant Manager Sadia Javed, Assistant Director Planning & Engineering Modernization (GIS Mapping, System Analysis & Training) MEPCO Multan 0345-7734444 0321-6304060 0345-8189153 19-Sep-14 24. 2 Naeem Ullah Khan, DG HR Syed Shaukat Hussain, Principal, Regional Training Center Utility Exchange Program MEPCO Multan 0345-4075555 0345-8189333 19-Sep-14 25. 2 Mehmood Ali Qaimkhani, Manager Commercial Hina Talpur, Deputy Manager Tariff (CPC) Cost of Service Study HESCO Hyderabad 0300-3020855 0335-8125044 22-Sep-14 133 Group Interview No. No. of Participants in Each Group Names of the Interviewee/Participants Intervention/ Activity DISCO Location Contact Details Date of Interview 26. 3 Najamuddin Abro, PD (Construction) Hyderabad Manjee Khan Deputy Manager TMP Pashmeena Shaikh, Deputy Manager Services Lineman Training HESCO Hyderabad 0300-0800415 0300-0333346 0300-0800410 22-Sep-14 27. 2 Rashid Ansari Executive Engineer - Qasimabad Sub Division Rana Shafiq, Sub Divisional Officer Qasimababd Sub Division AMR Metering HESCO Hyderabad 0301-3972176 0300-8374640. 23-Sep-14 28. 3 Jamil Ahmed Qaim Khani, Manager MIS Hina Talpur, Dy Manager Tariff Ms Ambar Shah, Assistant Manager Customer Service Gender Equity Training HESCO Hyderabad 03008372139 03008372561 03008372826 23-Sep-14 29. 6 Jahangir Amir, Senior Engineer Ahmed Khan, Senior Engineer Tajammul Hussain, Assistant Manager Abdul Qayyum, Junior Engineer Abdul Ghafoor, Manager Planning Pervez Afzal, Manager Planning & Engineering Modernization (GIS Mapping, System Analysis & Training) HESCO Hyderabad 0333-2604258 0300-3414259 0321-3016784 0300-3055788 23-Sep-14 30. 2 Deen Muhammad, Manager Finance Tajammul Hussain, Assistant Manager Engineering and Planning Utility Exchange Program HESCO Hyderabad 0300-8372542 0321-3016784 24-Sep-14 134 Group Interview No. No. of Participants in Each Group Names of the Interviewee/Participants Intervention/ Activity DISCO Location Contact Details Date of Interview 31. 2 Syed Muhammad Abbas Shah, Customer Services Director Imad Ali Mirani, Finance Director Cost of Service Study SEPCO Sukkur 0331-8937995 0300-0881101 25-Sep-14 32. 2 Deepak Kumar, Deputy Manager HR Sukker Khalid Jamil, Deputy Director Safety Lineman Training SEPCO Sukkur 0333-7592946 0300-9311980 25-Sep-14 33. 4 Rafique Ahmed, Additional Manager Iftikhar Hussain, Deputy Manager Ghulam Ali, Assistant Director Dedar Hussain, Assistant Director Planning & Engineering Modernization (GIS Mapping, System Analysis & Training) SEPCO Sukkur 0300-8313953 0301-2245760 0300-6661914 25-Sep-14 34. 2 Ahmed Solangi, Manager HR Noor Soomro, Assistant Manager Utility Exchange Program SEPCO Sukkur 0300-8319973 071-9310805 26-Sep-14 35. 2 Rana Muhammad Saleem, Additional Manager Tariff Ali Muhammad, Deputy Manager Tariff Cost of Service Study FESCO Faisalabad 0345-1500409 0345-1500305 29-Sep-14 36. 1 Syed Ahmed Ali Shah, Deputy Manager Planning and Engineering Lineman Training FESCO Faisalabad 0345-1501020 0347-5500012 29-Sep-14 135 Group Interview No. No. of Participants in Each Group Names of the Interviewee/Participants Intervention/ Activity DISCO Location Contact Details Date of Interview 37. 3 Muhammad Nawaz, Addtl Manager Zahid Latif, AD Muhammad Abdullah, Assistant Director Planning & Engineering Modernization (GIS Mapping, System Analysis & Training) FESCO Faisalabad 0333-9944740 0345-1500279 0345-1500280 30-Sep-14 38. 1 Muhammad Saleem, Deputy Manager Planning Utility Exchange Program FESCO Faisalabad 0345-1500123 041-9220242 30-Sep-14 39. 2 Haifiz Muhammad Imran, Deputy Manager Commercial Syed Qurb-e-Mujtaba, Additional Manager MIS Cost of Service Study GEPCO Gujranwala 03400001088 03400001384 29-Sep-14 40. 2 Muhammad Tahir Ghazi, Principal RTC Nandipur Shahid Pervaiz, Deputy Manager (Operations & Maintenance) Lineman Training GEPCO Gujranwala 0340-0001314 0340-0001079 29-Sep-14 41. 2 Habibullah, Deputy Director Farhan Hassan, Assistant Director Planning & Engineering Modernization (GIS Mapping, System Analysis & Training) GEPCO Gujranwala 0335-4742483 0340-0001055 29-Sep-14 42. 2 Maria Zaheer, Assistant Manager CS Muhammad Asif, Deputy Manager Commercial Utility Exchange Program GEPCO Gujranwala 0340-0001464 0340-0001673 29-Sep-14 136 Group Interview No. No. of Participants in Each Group Names of the Interviewee/Participants Intervention/ Activity DISCO Location Contact Details Date of Interview 43. 2 Khalid Mehmood Mirza, Additional Director General Commercial Basharat Ali, Director Finance Cost of Service Study LESCO Lahore 0314-7170400 0314-7170055 042-99204816 22-Sep-14 44. 2 Tariq Wahid Khan, Additional Director General Admin Najm ul Hasan, Executive Engineer Gulberg Lineman Training LESCO Lahore 0347-0010102 0347-0011510 22-Sep-14 45. 4 Uzma Azher, Assistant Manager Muhammad Rizwan, Assistant Director Waheed Zafar, Assistant Manager Zainab Batool, GIS Specialist Planning & Engineering Modernization (GIS Mapping, System Analysis & Training) LESCO Lahore 0322-4540007 0321-4740013 0322-4607914 0313-9779792 23-Sep-14 46. 2 Saghir Ahmed, Director General HR Imtiaz Butt, Director General IT Utility Exchange Program LESCO Lahore 0347-0010011 042-99204811 0347-0010099 042-36636815 23-Sep-14 47. 2 Yasir Faheem, Deputy Manager Finance Muhammad Khalid, Director Commercial Cost of Service Study QESCO Quetta 0346-8337139 0321-8103243 0346-8337003 1-Oct-14 48. 2 Muhammad Naeem Ullah, Principal RTC Syed Uzair Ali Hasni, Manager (Admin) Lineman Training QESCO Quetta 0333-7807021 0301-3728655 1-Oct-14 137 Group Interview No. No. of Participants in Each Group Names of the Interviewee/Participants Intervention/ Activity DISCO Location Contact Details Date of Interview 49. 2 Imran Khan Jogezai, Deputy Manager MIS, QESCO Zeenat Qazalbash, Upper Division Clerk Gender Equity Training QESCO Quetta 03337865556 03337874511 2-Oct-14 50. 3 Syed Abdullah, Deputy Director Khalid Farman, Assistant Director Saeed Qamar, Assistant Director Planning & Engineering Modernization (GIS Mapping, System Analysis & Training) QESCO Quetta 0300-9383076 0345-8387217 0345-8349503 2-Oct-14 51. 2 Asghar Mengal, Director Finance Abdullah Syed, Deputy Manager Planning Utility Exchange Program QESCO Quetta 0300-3632106 081-9203308 0300-9383076 081-9201626 2-Oct-14 138 List of Participants Interviewed from NEPRA: Group Interview No. No. of Participants in Each Group Names of the Interviewee/Participants Intervention/ Activity DISCO Location Contact Details Date of Interview 1. 4 Hussain Zaigham Alvi, Senior Advisor Hammad Shamimi, DG Administration & Human Resources Abdul Ghafoor Solangi, Deputy Director Ahmad Nadeem, Deputy Director, Human Resources Assistance to NEPRA (Multiple Activities) NEPRA Islamabad 0300-8301437 0321-5089000 0300-5153511 0321-5567488 12-Sep-14 139 List of Principals Interviewed from Girls’ Colleges: Group Interview No. No. of Participants in Each Group Names of the Interviewee/Participants Intervention/ Activity DISCO Location Contact Details Date of Interview 1. 3 Talat Samiullah (Vice Principal) Khalda Makhdoom (Teacher) Sadaf Zehra (Teacher) of Model College for Girls F-7/2, Islamabad Energy Conservation Campaigns in Girls Colleges IESCO Islamabad 9102526 / 9102529 9102526 / 9102529 11-Sep-14 2. 3 Samina Iffat , Principal, Government City Girls College, Gulbahar, Peshawar Nadia, Lecturer Political Science, Bacha Khan City Girls College, Peshawar Farhat Shaheen, Principal, Government Girls Degree College, Gulshan Rehman Colony, Kohat Road, Peshawar Energy Conservation Campaigns in Girls Colleges PESCO Peshawar 03219043749/ 091 226317 0313-9171416 03018928935 15-Sep-14 3. 1 Zubaida Javed Dean Faculty of Language and Religion, The Women University, Multan Energy Conservation Campaigns in Girls Colleges MEPCO Multan 0334-6065577 /0619200191 18-Sep-14 4. 1 Gulshan Ara Comprehensive Girls Higher Secondary College, Unit No.5, Latifabad Energy Conservation Campaigns in Girls Colleges HESCO Hyderabad 022-9260269/ 0331-3598485 22-Sep-14 5. 1 Farah Malhi, Principal/Point of contact, Government Post Graduate College for Women, Wahdat Road, Lahore Energy Conservation Campaigns in Girls Colleges LESCO Lahore 042-99260116 22-Sep-14 6. 1 Kaukab Liaqat, Principal/Point of contact Shiblee Girls Degree College, Madina Energy Conservation FESCO Faisalabad 041-8720727 / 0322-6264642 29-Sep-14 140 Group Interview No. No. of Participants in Each Group Names of the Interviewee/Participants Intervention/ Activity DISCO Location Contact Details Date of Interview Town, Faisalabad Campaigns in Girls Colleges 141 List of Group Interview with USAID and Implementing Partner: Sr. No. Names of the Interviewee/Participants Intervention/Activity Organization Location Date of Interview 1. Timothy Moore Nadeem Habib Overall PDP Program USAID/Pakistan Energy Office Islamabad 15-Sep-14 2. Imran Akhtar Outreach Activities and Anti-theft Campaigns IRG/PDP Islamabad 12-Sep-14 3. Qasim Virk Khurram Ehtisham Craig Fenton Amer Zia Usman Malik Qasim Virk Customer Information System- Phase II and III Electronic Metering PESCO Hand Held Units and Improved Meter Reading Automatic Meter Readers IRG/PDP Islamabad 12-Sep-14 4. Shafiq ur Rehman Omar Malik Planning and Engineering Modernization (GIS Mapping, System Analysis, and Training) Demand Side Management (Energy Efficiency Industrial Motors) IRG/PDP Islamabad 12-Sep-14 5. Marry Webster Abid Latif Lodhi Tahir Ali Khan Syed Akhlaq Ahmad Assistance to NEPRA (Multiple Activities) Cost of Service Study IRG/PDP Islamabad 12-Sep-14 6. Robert Dalton, Tufail Ahmad Sheikh Edward Bayless Ahmad Kamal Lineman Training, Tools, and Training Aids Utility Exchange Program IRG/PDP Islamabad 12-Sep-14 7. Craig Fenton Zubair Mahmood Enterprise Resource Planning (ERP) Implementation IRG/PDP Islamabad 12-Sep-14 8. Qurat Ul Ain Ibrahim Energy Conservation Campaigns in Girls Colleges Gender Equity Training IRG/PDP Islamabad 14-Sep-14 142 ANNEX V: DETAILED RESULTS OF DEMAND SIDE MANAGEMENT (DSM) PROGRAM The Demand Side Management program facilitated the replacement of motors with energy efficient models in 52 industries. It also encouraged the use of, and facilitated the purchase and installation of, variable speed drives. The program was operated by PDP staff across seven DISCO service territories, including the privately-owned Karachi utility, KESC (now known as “K Electric”), although the DISCOs themselves were not involved in the program. MSI conducted a phone survey of representatives from 48 out of 52 industries that participated in the Demand Side Management program. Nearly all the respondents said the cost of electricity was a major concern (Figure 5). Most said they were paying a lot for electricity and faced high demand charges and power factor charges. This was particularly the case with industries in the IESCO, KESC and FESCO service territories (Figure 6). The high energy costs were the main reason for participating in the motor program (Figure 7). FIGURE 5: IS COST OF ELECTRICITY A GREAT CONCERN FOR RESPONDENTS? 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% IESCO KESC LESCO PESCO MEPCO HESCO FESCO Other ALL Cost of electricity a great concern? Yes No 143 FIGURE 6: PROPORTION OF RESPONDENTS SUBJECTED TO MAXIMUM DEMAND AND POWER FACTOR CHARGES 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% IESCO KESC LESCO PESCO MEPCO HESCO FESCO Other ALL 100% 100% 67% 80% 71% 100% 83% 50% 79% 100% 100% 53% 40% 57% 100% 100% 50% 71% Percentage of Respondents Subjected to PF and MD Charges Max Demand Charges PF Charges 144 FIGURE 7: MAIN REASONS FOR PARTICIPATION IN EE MOTORS PROGRAM So far, most of the respondents are satisfied with their new motors, and 75 percent said the motors are effective, very effective or extremely effective (Figure 8). FIGURE 8: ENERGY SAVING MEASURES AND THEIR EFFECTIVENESS, USED BY RESPONDENTS Save energy costs 89% Improve production 2% Improve power supply quality 5% Financial Incentives 4% Reasons for Participation in EE Motors Program 0% 10% 20% 30% 40% 50% 60% Energy Audits Edu/Awareness EE Lighting EE Motors Other EE Tech 20% 22% 29% 16% 22% 11% 22% 35% 31% 25% 60% 47% 32% 28% 38% 6% 6% 3% 13% 3% 3% 3% 0% 13% 13% Effectiveness of EE Measures Tried by Participants Extremely Effective Very Effective Effective Fairly Effective Not Effective 145 The main reasons respondents were satisfied with the new motors was the energy and monetary savings (Figure 9), though some also responded that lowered demand charges were a reason as well. FIGURE 9: TOP REASONS FOR HAVING SATISFACTION WITH EQUIPMENT INSTALLED UNDER PROGRAM The vast majority of respondents said the new motor equipment had achieved its intended results (Figure 10), and most said they were satisfied with the equipment because of its quality and price (Figure 11). However 56 percent of the respondents from MEPCO and 44 percent from PESCO service territories said their satisfaction was based on price. This is a significant percentage of industry representatives who were satisfied with the price. This may be due to the 50 percent cost share they received from PDP which reduced the market price of the equipment. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% IESCO KESC LESCO PESCO MEPCO HESCO FESCO Other ALL Top Reasons for Satisfaction with Installation Energy Savings Monetary Savings Demand Savings Improve power quality Improve production Reduce load shedding Other 146 FIGURE 10: WHETHER OR NOT THE INSTALLED EQUIPMENT ACHIEVED INTENDED RESULTS FIGURE 11: SATISFACTION WITH VARIOUS ATTRIBUTES OF EQUIPMENT Yes 80% No 20% Installed equipment achieved Intended Results 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% IESCO KESC LESCO PESCO MEPCO HESCO FESCO Other ALL Satisfaction with Installed Equipment Satisfied with Quality Satisfied with Price Eq operating Normally Eq Affected Process 147 The overwhelming majority said they would continue to operate the new equipment and nearly as many said they would install more of it (Figure 12). A majority (70 percent) of the respondents within the MEPCO service territory said they would continue using the equipment. It is not known why the remaining 30 percent said they would not. FIGURE 12: FUTURE INTENTION CONCERNING THE EE EQUIPMENT 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% IESCO KESC LESCO PESCO MEPCO HESCO FESCO Other ALL Future intentions concerning use of installed equipment Continue Using Eq Install More Eq 148 As for the motor program itself, a majority said it was either excellent, very good, or good (Figure 13). About 75 percent said the program’s communications were excellent, very good or good, and 65 percent said the commissioning and testing process was excellent, very good or good. The program scored the lowest on results reporting, where 45 percent said it was excellent, very good or good; but 35 percent said it was fair or poor.28 FIGURE 13: RATING OF PROGRAM BY ITS ATTRIBUTES Interestingly, monitoring and evaluation scored very well even though by its own design, the program did no measurement or verification of energy savings. Respondents may have felt other aspects of the program’s monitoring and evaluation were good, but in an energy efficiency program, energy savings over time is typically the most important factor in the evaluation. 28 Several respondents (20 percent) said they could not rate this item which is why the totals do not add up to a 100 percent. 0% 5% 10% 15% 20% 25% 30% 35% 40% Communication Rego and Particptn Comm & Testing M&E Result Reporting Overall Program Program Rating by various attributes Excellent Very Good Good Fair Poor Cannot Rate 149 Respondents were asked what things could be done to ensure widespread adoption of energy efficient motors. Tax breaks (or subsidies generally) and active promotion were supported by over 60 percent of the respondents (Figure 14). The provision of expert advice was not considered something that would help ensure widespread use. FIGURE 14: RESPONDENTS’ RECOMMENDATIONS ABOUT WHAT TO DO FOR WIDESPREAD ADOPTION OF EE EQUIPMENT 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% IESCO KESC LESCO PESCO MEPCO HESCO FESCO Other ALL Things to do to ensure widespread use of EE motors Actively Promote Waive Taxes Provide Expert Advice 150 ANNEX VI: DETAILED RESULTS OF LINEMEN SAFETY TRAINING PDP conducted training workshops for linemen at nine DISCOs. The trainings, organized by the NRECA, were attended by about 2,000 linemen. After that, through a train the trainer program, another 9,000 were trained.29 The trainings have been (or are being) incorporated in the normal lineman trainings conducted by the DISCOs through their Regional Training Centers. In October, 2014, MSI conducted a survey of 333 linesmen who participated in the trainings. The evaluation conducted surveys at 5 of the DISCOs that hosted trainings related to safety while working on the line. The survey was conducted through in-person interviews by enumerators with participants at the 5 DISCOs (Figure 15). FIGURE 15: SAMPLE COMPOSITION BY DISCO 29 This is as of April 2014. http://www.ect.coop/newsmakers/international/pakistan-lineman-safety-program-saves-lives/68376 GEPCO 14% HESCO 34% LESCO 14% MEPCO 26% PESCO 12% DISCO Share in Survey 151 FIGURE 16: SAFETY PROGRAMS ATTENDED BY LINESMEN FROM DIFFERENT DISCOS 0 50 100 150 200 250 GEPCO HESCO LESCO MEPCO PESCO Number of Accidents Number of Safety Trainings Attended by Survey Participants by DISCOs R-LSPITP QISTPO Q and SM ODSTP LSTPfPCM LSPITP LSHI 152 FIGURE 17: MOST RECENT SAFETY PROGRAMS ATTENDED BY SURVEY RESPONDENTS 0 20 40 60 80 100 120 GEPCO HESCO LESCO MEPCO PESCO Number of Accidents Most Recent Trainings Received by DISCO Other R-LSPITP QISTPO Q and SM ODSTP LSTPfPCM LSPITP LSHI 153 Twenty percent of the interviewed linemen have experienced on-the-job accidents in their career, particularly at HESCO and PESCO (Figure 18). FIGURE 18: ACCIDENTS EXPERIENCED BY SURVEY PARTICIPANTS BY DISCO 15% 28% 18% 12% 29% 20% 0% 5% 10% 15% 20% 25% 30% 35% GEPCO HESCO LESCO MEPCO PESCO ALL Proportion of Survey Participants, by DISCO, involved in an Accident 154 The main types of accidents reported by the respondents were electric shock, followed by falling from a height (Figure 19). FIGURE 19: TYPE OF ACCIDENT EXPERIENCED BY SURVEY PARTICIPANTS BY DISCO 0 10 20 30 40 50 60 GEPCO HESCO LESCO MEPCO PESCO Total Number of Accidents Number of accidents by Type of Accident by DISCO Other Falling Object Fall from Height Electric Burn Electric Spark Electric Shock Breakdown of safety equipment 155 Of the accidents experienced by survey respondents, there were almost equal numbers of major injuries, minor injuries and no injuries, although major injuries were the most commonly experienced/observed type of injury (Figure 20). The most common accidents occurred with live lines or with live lines that were thought to be dead (Figure 21). FIGURE 20: RESULTS OF ACCIDENTS EXPERIENCED BY SURVEY PARTICIPANTS BY DISCO 0 10 20 30 40 50 60 GEPCO HESCO LESCO MEPCO PESCO ALL Number of Accidents Number of accidents by type of result No Injury Minor Injury Major Injury 156 FIGURE 21: NATURE OF SERVICES PROVIDED BY SURVEY PARTICIPANTS WHEN ACCIDENT HAPPENED BY DISCO 0 10 20 30 40 50 60 GEPCO HESCO LESCO MEPCO PESCO ALL Number of Accidents Nature of Service when accident happened Other Live Line Presumed dead line 157 The most common types of accidents were electric shock (over 60 percent of all accidents) followed by a fall from a height (over 40 percent). FIGURE 22: TYPE OF ACCIDENTS WITNESSED BY SURVEY PARTICIPANTS BY DISCO 0 20 40 60 80 100 120 140 160 180 GEPCO HESCO LESCO MEPCO PESCO ALL Number of Accidents Number of accidents by type , witnessed by participants, by DISCO Other Breakdown of safety equipment Falling Object Fall from Height Electric Burn Electric Spark Electric Shock 158 FIGURE 23: RESULTS OF ACCIDENTS WITNESSED BY SURVEY PARTICIPANTS BY DISCO FIGURE 24: ALL ACCIDENTS EXPERIENCED OR WITNESSED BY RESPONDENTS 0 20 40 60 80 100 120 140 160 GEPCO HESCO LESCO MEPCO PESCO ALL Number of Accidents Number of accidents by type of result, witnessed by survey participants Death No Injury Minor Injury Major Injury GEPCO, 19% HESCO, 41% MEPCO, 21% LESCO, 26% PESCO, 35% ALL, 30% Percent of Respondents Experiencing or Witnessing Accidents 159 FIGURE 25: TYPES OF ACCIDENTS EXPERIENCED OR WITNESSED BY RESPONDENTS 0 50 100 150 200 250 GEPCO HESCO LESCO MEPCO PESCO ALL Number of Accidents Number of All accidents by type by DISCO Other Breakdown of safety equipment Falling Object Fall from Height Electric Burn Electric Spark Electric Shock 160 FIGURE 26: RESULT OF ALL ACCIDENTS EXPERIENCED OR WITNESSED BY RESPONDENTS 0 50 100 150 200 250 GEPCO HESCO LESCO MEPCO PESCO ALL Number of Accidents Impact of Self Experienced or Witnessed Accident Death Major Injury Minor Injury No Injury 161 The major factors cited by linemen to be the cause of accidents were carelessness, and lack of safety gear (reported by 40 percent for both). FIGURE 27: MAJOR FACTORS CONTRIBUTING TO LINESMEN ACCIDENTS, BY DISCO 0 100 200 300 400 500 600 700 800 900 1000 GEPCO HESCO LESCO MEPCO PESCO ALL Major causes of Linesmen Accidents No Clear Guidelines Management not Implementing Safety Linesmen are not Trained Linesmen are not Aware of Safety Macho Mindset No transport for Safety Gear Not enough Safety Gear Linesmen are Careless 162 ANNEX VII: EVALUATION TEAM BIOS Michael Philips is the team leader for the evaluation of the Power Distribution Program. He has over 30 years of experience in the energy sector and is a recognized authority on the development and financing of sustainable energy programs and projects. He has published widely on structuring and implementation of energy efficiency and clean energy projects. He has been an advisor to municipal governments, electric utilities, private businesses, universities, international finance institutions, and national government agencies on the design and implementation of energy policies, regulations, and programs. He had a White House appointment at the US Department of Energy (1999-2001), was a professional staff member of the US House of Representatives Resources subcommittee on Oversight & Investigations (1985-1987), and was an instructor on renewable energy project development at the American Physical Plant managers Association (2008-2010). He is currently Senior International Specialist with the Quebec City-based consulting firm, Econoler. Jacob T Laden is the MSI’s Evaluation Advisor assigned to the Power Distribution Program evaluation. Jacob provided management support to the team as well as technical guidance on the design of the evaluation, instruments, and data analysis. Jacob also provided supplemental analysis and helped write the evaluation reports. While at MEP, Jacob has also served as advisor on the USAID Education Office’s evaluation of the Pakistan-US Science and Technology Cooperation Program, USAID EGA’s Firms Program and the Ex-post evaluation of the PAIMAN and FALAH programs conducted for USAID Health. Jacob has over 10 years of experience in analysis and program evaluation of international development programs with expertise in education, conflict and diplomacy, organizational capacity building, as well as science and technology programs. Before coming to MSI, Jacob was a Senior Evaluation Officer at the US Department of State in the Bureau of Educational and Cultural Affairs where he managed several evaluations and conducted performance measurement for the bureau’s international educational and cultural programs. Jacob also served as a Monitoring and Evaluation Specialist for Science Technology Engineering and Math (STEM) programs at the Society for Neuroscience. Prior to that he worked as a consultant for the World Bank’s learning and organizational effectiveness unit (HRSLO) where he coauthored three evaluations including an evaluation of the World Bank Language and Culture Program (2009), the Sustainable Development Learning Program (2009) and the 2008-2009 assessment of the HRSLO unit. He also co-authored the 2006 and 2007 evaluation of Project Unity an interfaith conflict resolution program in NYC. Zameer Haider is a full-time Evaluation Specialist at MEP. He was the task manager and a core team member for this evaluation. He has program management and implementation experience with national and international non￾governmental organizations engaged in USAID and other donor projects in the health and education sectors and in poverty alleviation. He has also managed donor-funded research and advocacy grants and led monitoring and evaluation activities in management positions. As part of his experience with MEP, Mr. Haider has managed evaluations of USAID-funded projects in Pakistan and conducted data collection and analysis. His educational qualifications include a master’s degree in international relations from the University of Punjab and a master’s of philosophy degree in public policy and governance from the National Defense University, Islamabad. Dr. Shahab Qureshi was the second team leader for the field work and was the key team member responsible for the two surveys conducted under this evaluation. He is an electrical engineer with a doctorate degree in Energy and Demand Side Management. He has demonstrated expertise in strategic policy development and planning, analytical business modeling, scenario simulations, data acquisition and management, monitoring and evaluation systems, complex statistical analysis, training, and capacity building. He has worked extensively on projects related to Load Research, DSM, energy efficiency, DSR, energy policies, alternate energies, and tariff design. Zaheer Ahmed Ather was a team member involved in the field work and had particular expertise in the use of advance meters and other power related technologies. He is an independent consultant working in the field of Power and Energy. His academic qualifications include Bachelor’s degree in Mechanical Engineering, Masters in Environmental Engineering and Masters in Business Administration. He has an experience of around 35 years in electric utility business. During this period he has worked on a number of cross functional areas involving Thermal Power Plant Management, Maintenance, Planning and Rehabilitation, OHSE, Procurement, Asset Management, Statistical Reports compilation and Management Studies. As a consultant he has worked with a number of international and local consultants in the area of Energy Planning, Energy Audit, Renewable Energy, Monitoring and Evaluation, Resettlement Issues, Training and Development. 163 Anser Ali was a team member involved in field work activities and was a subject matter expert on the finance and governance aspect of the evaluation. He is a power utility sector financial management specialist who has significant experience with USAID programs. He will support the Evaluator and the Local Technical Team Lead in the fieldwork planning and data collection process. 164 ANNEX VIII: DISCLOSURE OF CONFLICT OF INTEREST Name Michael Philips Title Team Leader Organization Management Systems International (MSI) Evaluation Position? Team Leader Team member Evaluation Award Number (contract or other instrument) USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) Power Distribution Program (PDP) International Resources Group (IRG) I have real or potential conflicts of interest to disclose. Yes No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date October 17, 2014   165 Name Jacob Laden Title Evaluation Advisor/ Team Member Organization Management Systems International (MSI) Evaluation Position? Team Leader Team member Evaluation Award Number (contract or other instrument) USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) Power Distribution Program (PDP) International Resources Group (IRG) I have real or potential conflicts of interest to disclose. Yes No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 7. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 8. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 9. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 10. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 11. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 12. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date November 11, 2014   166 Name Zameer Haider Title Evaluation Specialist/Team Member Organization Management Systems International (MSI) Evaluation Position? Team Leader Team member Evaluation Award Number (contract or other instrument) USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) Power Distribution Program (PDP) International Resources Group (IRG) I have real or potential conflicts of interest to disclose. Yes No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 13. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 14. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 15. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 16. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 17. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 18. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date July 14, 2014   167 Name Dr. Shahab Qureshi Title Team Member Organization Management Systems International (MSI) Evaluation Position? Team Leader Team member Evaluation Award Number (contract or other instrument) USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) Power Distribution Program (PDP) International Resources Group (IRG) I have real or potential conflicts of interest to disclose. Yes No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 19. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 20. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 21. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 22. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 23. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 24. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date October 30, 2014   168 Name Zaheer Ahmed Ather Title Team Member Organization Management Systems International (MSI) Evaluation Position? Team Leader Team member Evaluation Award Number (contract or other instrument) USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) Power Distribution Program (PDP) International Resources Group (IRG) I have real or potential conflicts of interest to disclose. Yes No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 25. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 26. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 27. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 28. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 29. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 30. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date October 31, 2014  a a a a   a a a a  169 Name Anser Ali Title Energy Sector Specialist/Evaluator Organization Management Systems International (MSI) Evaluation Position? Team Leader Team member Evaluation Award Number (contract or other instrument) USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) Power Distribution Program (PDP) International Resources Group (IRG) I have real or potential conflicts of interest to disclose. Yes No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 31. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 32. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 33. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 34. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 35. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 36. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date November 12, 2014  170 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523