[September 2013] This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Mendez, England & Associates. Mid-Term Performance Evaluation of USAID/Georgia Power and Gas Infrastructure Project (PGIP) MID-TERM PERFORMANCE EVALUATION OF USAID/GEORGIA POWER AND GAS INFRASTRUCTURE PROJECT (PGIP) Final Report September 2013 Prepared under RFTOP No. SOL-114-13-000005 Submitted to: USAID/Caucasus Submitted by: Peter Tal, Team Leader Nils Junge, Evaluation Specialist Murman Margvelashvili, Energy Expert Contractor: Mendez England & Associates 4300 Montgomery Avenue, Suite 103 Bethesda, MD 20814 Tel: 301- 652 -4334 www.mendezengland.com DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government CONTENTS EXECUTIVE SUMMARY...................................................................................................... I Evaluation Purpose and Evaluation Questions ................................................................................... i Evaluation Questions ............................................................................................................................... i Project Background ................................................................................................................................. ii Evaluation Design, Methods and Limitations ..................................................................................... ii Findings. ..................................................................................................................................................... iii Conclusions ............................................................................................................................................... v Recommendations .................................................................................................................................. vi Lessons Learned ...................................................................................................................................... vi 1.0 EVALUATION PURPOSE & EVALUATION QUESTIONS ................................ 1 1.1 Evaluation Purpose ................................................................................................................... 1 1.2 Evaluation Questions ............................................................................................................... 1 2.0 PROJECT BACKGROUND ..................................................................................... 2 3.0 EVALUATION METHODS & LIMITATIONS ...................................................... 4 3.1 Evaluation Methodology .......................................................................................................... 4 3.1.2 Qualitative Research and Analysis ...................................................................... 6 3.2 Evaluation Limitations .............................................................................................................. 6 4.0 FINDINGS, CONCLUSIONS, RECOMMENDATIONS ....................................... 7 4.1 Question 1: Were the PGIP Goals Accomplished? .......................................................... 7 4.1.1 Findings ..................................................................................................................... 7 4.1.2 Conclusions .............................................................................................................. 9 4.1.3 Recommendations .................................................................................................. 9 4.2 QUESTION 1.a: Economic benefits associated with gas pipeline construction under PGIP .............................................................................................................................. 11 4.2.1 Findings .................................................................................................................... 11 4.2.2 Conclusions ............................................................................................................ 21 4.2.3. Recommendations ...................................................................................................... 22 4.3 QUESTION 1.b: The overall quality of the design of the Senaki power transmission project .............................................................................................................. 22 4.3.1 Findings .................................................................................................................... 23 4.3.2 Conclusions ............................................................................................................ 23 4.3.3 Recommendations ................................................................................................. 23 4.4 QUESTION 1.c: The overall process of designing the Transmission project ......... 24 4.4.1 Findings .................................................................................................................... 24 4.4.2 Conclusions ............................................................................................................ 25 4.4.3 Recommendations ................................................................................................. 25 4.5 To what degree did USAID’s investment in CAPE and DGA promote a safer, more reliable Georgian power grid? ................................................................................. 25 4.5.1 Findings .................................................................................................................... 25 4.5.2 Conclusions ............................................................................................................ 26 4.5.3 Recommendations ................................................................................................ 26 4.6 QUESTION 2: Sustainability ............................................................................................... 26 4.6.1 Findings .................................................................................................................... 27 4.6.2 Conclusions ............................................................................................................ 27 4.6.3 Recommendations ................................................................................................ 28 4.7 QUESTION 2.a: the extent the operations and maintenance of Georgian gas pipeline and power transmission systems improved ..................................................... 28 4.7.1 Findings .................................................................................................................... 29 4.7.2 Conclusions ............................................................................................................ 29 4.7.3 Recommendations ................................................................................................ 30 4.8 Cross Cutting issue: Did the training and capacity building achieve its goals? ........ 30 4.8.1 Findings .................................................................................................................... 31 4.8.2 Conclusions ............................................................................................................ 31 4.8.3 Recommendations ................................................................................................ 31 5.0 LESSONS LEARNED ............................................................................................. 31 ANNEXES ANNEX A: Statement of Work ANNEX B: Work Plan, Evaluation Design & Methodology ANNEX C: List of Interviewees ANNEX D: List of Materials Consulted ANNEX E: Supporting Documentation ANNEX F: Household Questionnaire ANNEX G: Focus Group Guidelines ANNEX H: Focus Group Summaries FIGURES Figure 1: PGIP Pipeline Component – Senaki-Poti (operational), Senaki-Abasha (completed) Figure 2: PGIP Transmission Component - 220Kv line Tskaltubo-Menji Figure 3: Percentage HHs using fuel for cooking before and after switching to gas Figure 4: Percentage HHs using fuel for hot water before and after switching to gas Figure 5. Percentage HHs using fuel for space heating before and after switching to gas Figure 6: Reasons for Switching to Gas (%) Figure 7: Who benefits most from gas connection? (% respondents) Figure 8: Hypothetical expenditures (GEL/year) and savings (%) for a HH switching entirely to gas from other fuel TABLES Table 1: Beneficiary types and primary uses of natural gas Table 2: Comparison between Poti and Senaki – gas (preliminary) Table 3: Connection cost per HH (GEL) based on HH survey responses Table 4: HHs self-reported fuel expenditures (GEL/month) Table 5: Total Savings (per HH) ACRONYMS CAPE Computer Aided Protection Engineering software Com Commercial users of electricity and Natural Gas DGA Dissolved gas analyzers for large power transformers EECS Enhanced emergency control system for transmission Energo-Pro Regional Electricity Utility Company EPC Engineering – procurement – construction FGD Focus Group Discussion FIZ Poti Free Industrial Zone GGTC Georgia Gas Transportation Company GIPTP Georgia Improved Power Transmission Project GOG Government of Georgia and relevant ministries GOGC Georgian Oil and Gas Corporation GSE Georgian State Electric Systems HDD Horizontal Directional Drilling HH Households, residential consumers of electricity and gas HICD PLUS Human and Institutional Capacity Development Plus project IEC International Electrotechnical Commission – a standard setting and monitoring body Ind. Industrial users of electricity and natural gas IRMS IT, Research and Metadata Solutions kV kiloVolt MAOP Maximum Allowed Operating Pressure ME&A Mendez, England & Associates MVA Mega Volt Amper OGCT Oil and Gas Construction Trust (construction contractor) PGIP Power and Gas Infrastructure Project PLS- CADD Power Line System – Computer Aided Design Drafting software PGIOP Power and Gas Infrastructure Oversight Project PMP Performance Monitoring Plan PPM Parts per million PUE Soviet/Russian Rules of Installation for Electric Equipment, main regulating document for Electric equipment accepted in Georgia SER Sakenergoremonti SOCAR State Oil Company of Azerbaijan Republic, Regional Gas Utility Company SOW Scope of Work TT Tetra Tech USAID United States Agency for International Development ACKNOWLEDGMENTS We thank USAID-Georgia staff and specifically Mr. Sukru Bogut for guidance on the technical evaluation methodology, and Ms. Lela Kerashvili for her assistance with the direction and details required for the households survey. We would like to express our special thanks to: GGTC‘s Mamuka Kobakhidze,General Director, Irakli Okroshidze, Head of Commercial Department, and Zviad Rostomashvili, First Deputy Director; GOGC’s Zaqaria Avaliani, Technical Director, Temur Gochitashvilii, Advisor/Head Strategic Planning Department, Tato Goguadze, Head of Construction Supervision Department; GSE’s Maya Pitskhelauri, International Projects Manager, Sulkhan Zumburidze, Chairman of the Management Board/Rehabilitation Manager; Ucha Uchaneishvili, Board Member; Sakrenergoremonti’s David Kakabadze, General Director; Zviad Korshia, Deputy General Manager; Vakhtang Kelbakiani, Project Coordinator; and SOCAR Georgia Gas, Anar Mammadov, General Director. We would also like to thank Tamaz Jgharkava, Poti Business Manager; Ilia Eloshvili and Mariam Valishvili, Deputy Ministers of Energy; Patrick Lohmeyer, COP of USAID’s HICD project, David Managadze, Banker at EBRD; Enrico Spiller, KFW Director; and Kathie Julian, Resident Representative of ADB. Finally, this evaluation required extensive document review, numerous interviews, and site visits in few locations in Georgia. We would not have been able to complete the task without the generous assistance of Mr. Michael Hajny, COP Tetra Tech, as well as Ms. Sophie Berishvili, DCOP and Mr. Giorgi Beradze, who lead the Evaluation Team to sites. MID-TERM PERFORMANCE EVALUATION OF PGIP i | Page EXECUTIVE SUMMARY EVALUATION PURPOSE AND EVALUATION QUESTIONS This is a report on the mid-term performance evaluation of the Power and Gas Infrastructure Project (PGIP) funded by the United States Agency for International Development (USAID) Mission in Georgia. The PGIP project is being implemented during the period May 10, 2010 – September 30, 2014, by Georgia Oil & Gas Corporation (GOGC), Sakenergoremonti (SER), and Tetra Tech EM (TT). The mid-term evaluation of PGIP was conducted during the period June – August 2013, by a team assembled by Mendez England & Associates (ME&A) with headquarters in Bethesda, Maryland. The team consisted of three experts, two international and one local, all with experience in the energy infrastructure sector related to gas pipelines and power transmission projects. The main objective of the evaluation was to assess PGIP’s progress towards its goals and, as a secondary objective, provide lessons learned on the sustainability considerations of large-scale infrastructure projects, implementation modality, and other important aspects. The main thrust of the evaluation included PGIP activities from May 2010 to May 2013, and focused on: 1. End users’ measurable economic benefits that can be associated with the gas pipeline construction under PGIP. 2. Overall quality of the design and procurement of the Senaki power transmission project. 3. Overall process of designing the transmission project. 4. Extent the operations, maintenance, and sustainability of Georgian gas pipeline and power transmission systems improved, as well as the contribution that PGIP made (if any) towards such improvement. The findings and conclusions reached by the Evaluation Team could be used by USAID/Georgia and other USAID Missions in designing and implementing current and new programs in energy infrastructure construction and oversight services. EVALUATION QUESTIONS Per the SOW, the Evaluation Team was asked to answer a number of specific evaluation questions, outlined below: 1. Were the PGIP goals accomplished? These goals include: providing gas supplies to Western Georgia and increasing the reliability of Georgia’s power grid. a. What primary and secondary economic benefits, both current and projected, over the next ten years, can be associated with gas pipeline construction under PGIP? b. What is the overall quality of the design of the Senaki power transmission project prepared by the engineering oversight contractor, as perceived by host country stakeholders? c. What is the overall process of designing the transmission project? To what degree did USAID’s investments in sub-procurement, including Computer Aided Protection Engineering (CAPE) and Dissolved Gas Analyzers (DGA), promote or have the potential to promote a safer, more efficient, reliable, and robust Georgian power grid? 2. What is the sustainability of the infrastructure built with USAID support? a. To what extent have the operations and maintenance of Georgian gas pipeline and power transmission systems improved, and what was PGIP’s possible contribution toward such improvement? 3. What are the lessons learned from this mid-term evaluation of PGIP that can inform future program designs, procurements, executions, and oversight services that will utilize both private and host country methods that minimize bias and provide strong evidence? MID-TERM PERFORMANCE EVALUATION OF PGIP ii | Page PROJECT BACKGROUND PGIP was designed to: 1) promote energy security through greater access to electricity and natural gas supplies for households and businesses in Western Georgia; 2) promote the development of the Poti Free Industrial Zone (FIZ) on the Black Sea; and 3) secure power exports through reliable transmission infrastructure improvements domestically. PGIP is a portfolio of $115 million in energy infrastructure projects that include: i) main gas pipeline; ii) power transmission line construction; and iii) smart grid improvements. The project is unique because it is funded from “one-time” supplemental post-conflict resources, is the largest USAID-funded infrastructure project in Georgia, and utilizes an innovative mix of both private sector and host country-controlled organizations as implementers. GOGC was contracted to construct the Senaki-Poti (30 km), Senaki-Abasha (29 km), and Abasha￾Kutaisi (47 km) pipelines. SER was directly contracted by USAID for the construction of the transmission line Tskaltubo-Menji. TT was contracted for engineering oversight focused on all components (gas transit and transmission), as well as for assistance with design, engineering, and smart grid improvements. EVALUATION DESIGN, METHODS AND LIMITATIONS The Evaluation Team collected quantitative and qualitative data and information from a broad range of stakeholders and affected groups to obtain a balanced and thorough overview, as well as accuracy and completeness of the subsequent conclusions. Techniques that balance each other were utilized: quantitative vs. qualitative data; individual vs. group responses; semi-structured interviews vs. analysis of existing surveys; and data sets. The following main data sources of evidence were used:  Critical Desk Review of Materials related to PGIP, including project reports, annual work plans, project performance management plans, project design, communications among partners, billing data, Geostat reports, and USAID Monitoring and Evaluation (M&E) regulations.  Primary Data from TT’s performance reports and evaluations of implementing partners, as well as from drawings, measurements, manufacturers’ catalogs, and information received from GOGC, Georgia Gas Transportation Company (GGTC), Georgian State Electric Systems (GSE), SER, and the State Oil Company of Azerbaijan Republic (SOCAR).  Secondary Data from Geostat reports and GOCG, GSE and TT analyses and projections.  Program Outputs against objectives and performance indicators.  Field Visits to Poti and areas along the Senaki-Poti line, Senaki-Abasha segment, Tskaltubo and Menji sub-stations, transmission work-sites, SER contractor site, old Soviet pipelines, older transmission lines, and residential, commercial and industrial sites connected to gas.  Focus Groups Discussions (FGDs) with connected and non-connected (to gas) household (HH) members from Poti, Senaki, Abasha and Samtredia.  Key Informant Interviews, including open-ended and semi-structured interviews with USAID and PGIP implementers, (TT, SER, GOGC, GGTC), and program beneficiaries and stakeholders during site visits.  Questionnaires. Identical questionnaires were used to interview the main project’s stakeholders separately. Interviewees responded verbally and supplied written backup and supporting complementary information.  Mini-Survey of Households. Face to face interviews with a total of 100 HHs in Poti and Senaki were conducted in order to be able to address project impact at the HH level.  Direct Observations to verify the information. The Evaluation Team encountered some limitations inherent to the design of this evaluation and during its fieldwork in Georgia. Some of the more relevant limitations are listed below: MID-TERM PERFORMANCE EVALUATION OF PGIP iii | Page 1. Evaluation Period: This was a mid-term evaluation conducted before the power transmission component of the project was completed. Therefore, it might not be possible to determine the concrete results of the PGIP project. A number of PGIP’s key initiatives were still at too early a stage to expect any meaningful progress toward important performance indicators. 2. Resources Allocated for the Evaluation: The complexity of the project and amount of information produced during its implementation, combined with the limited time and resources allocated for the evaluation, limited the in-depth evaluation and the ability to follow up on the issues identified during the country visit. Therefore, the Evaluation Team relied mostly on data provided by the implementers, including utility billing records and data, where available. 3. Access to Technical Information: The Evaluation Team had some difficulties getting drawings of as-built structures or components prior to the field trips. In some cases they were not yet available, somewhat limiting the ability to assess design against construction. The above limitations did not prevent the Evaluation Team from gathering the information and data needed to draw conclusions and make recommendations. FINDINGS 1. Gas pipeline component and transmission component (which includes smart grid improvements through the sub-procurement of DGA, CAPE and Enhanced Emergency Control System for Transmission (EECS)), were designed according to the Ministry of Energy’s strategy and the needs of GSE, GOGC/GGTC to improve the power and gas infrastructure. The transmission line concept is sound as it is a reconstruction of a preexisting and needed backup alternative to transmission backbone. The gas pipeline extension addresses the need to supply gas to Poti for future economic development, while the smart grid improvements are strengthening the grid’s safety, robustness and management. 2. The completed activities of the gas pipeline component and smart grid have been implemented to satisfactory quality and provide the expected results to project stakeholders. The Evaluation Team reviewed and verified the project design assumptions for the pipeline and found the rationale for that design sound. The pipeline component is very close to be on time, below the budget, and fully on scope. In interviews with the Evaluation Team, the two Deputy Ministers of Energy, as well as representatives from GOGC and GSE, confirmed their satisfaction with the project’s results to date. 3. Senaki-Poti pipeline is fully operational and provides gas to Poti area. In addition to the Senaki-Poti completed pipeline segment, the Senaki-Abasha pipeline segment was in testing during the evaluation’s field survey. Abasha-Kutasi segment is in construction according to TT’s reports. The gas pipeline is extended to Poti Free Industrial Zone (FIZ) although is not used yet due to no industrial development going on in FIZ. The new pipeline is actively used by the gas distribution company SOCAR to provide access to the HHs and businesses in Poti at higher technical standards than elsewhere in Georgia. SOCAR provides only a single gas connection1 point for each HH included in the connection fee. An additional fee is required to provide a second gas connection for another appliance. 4. Although the design of the pipeline was sound, its cost-benefit analysis was not realistic in terms of expected consumption. A number of erroneous and overly optimistic assumptions were made. One of them was that consumption in 2013 will be 185 million m3, while in fact it is unlikely to be more than 1 million m3. Although this assumption is related to the GoG’s (optimistic) expectations that FIZ would materialize and attract thousands of energy intensive businesses, the Evaluation Team found it overly optimistic and not well-founded. 5. TT, as the oversight contractor, was instrumental in maintaining a steady and good quality flow of information and communication with and between all stakeholders. Furthermore, TT facilitated 1 SOCAR provides one single connection from the outside the dwelling gas meter to inside the dwelling. This connection is usually limited to the kitchen and used for cooking MID-TERM PERFORMANCE EVALUATION OF PGIP iv | Page decision-making processes and contributed to improved working relationships between the stakeholders on the transmission component. 6. New technology, Horizontal Directional Drilling (HDD), was introduced during construction of the pipeline that can be used elsewhere, and provide higher efficiency and better protection of the environment. Other new technologies introduced were related to control equipment in various distribution points. 7. The power transmission part of the project is in early stage of construction where only foundations have been installed. Delays were caused by the requirement to use a new, modern designing tool, Power Line Systems - Computer Aided Design and Drafting (PLS-CADD), which required skilled local professionals that could use it. SER did not have such professionals in its staff. 8. All 3 sub-procurements of smart grid were delivered and installed on time, and demonstrate a very solid contribution to the electrical transmission’s management and strengthening. All three are in full or partial operation. CAPE has achieved the goals of enhancing and strengthening a more reliable electrical grid. The software is in operation and provides a fast, efficient and reliable method for calculating the relay settings to reflect supply and demand changing requirements due to seasonal changes, as well as current conditions of the transmission grid and generating plants. In interviews with the Evaluation Team, GSE representatives confirmed the need for such a system, which improved the company’s performance by saving time modifying the transmission relays according to changing needs. The EECS system was also confirmed operational during the team’s visit at GSE headquarters (HQ) where the system is installed in GSE’s main national grid control room. The EECS system has been operational for several years and provides the important benefit of load shedding. According to a Board Member of the GSE, the system has reduced the probability of system blackouts. EECS is highly valued by GSE as an advanced modern system that places them as regional leaders in technology development. The Evaluation Team found that EECS has exceeded the expectations and it is a success story. A software upgrade is scheduled for September 2013, which will further boost the system’s functionality. 9. Businesses and social/government institutions connected to natural gas as a result of PGIP are able to realize considerable savings, from as much as two-thirds of their energy costs. These savings are used for other investments. The Evaluation Team did not find permanent employment impacts from connection to gas although, as part of project implementation, some temporary employment was created. HHs newly connected to gas use it predominantly for cooking. The average HH is able to save approximately 435 GEL ($263) per year in energy costs by connecting to the gas network. In areas where HHs have been connected for longer, their energy savings increase as they begin to use gas also for space heating. Beyond monetary savings, HHs placed a great value on convenience, comfort, lower pollution and nature conservation (preservation of forests from fewer trees being cut for firewood) by switching to gas. These are also included as economic benefits. On the other hand, if (and to the extent that) gas is subsidized by the Government of Georgia (GoG), total, economy-wide benefits would be offset by the subsidy’s share. Increases in gas consumption would result in increases in government outlays on subsidies. 10. Connection rates have slowed down in Poti, with only about one in ten HHs and businesses connected after 18 months of access. Although connecting HHs was not a PGIP goal, its broader impact on the economy at the HH level will be muted unless more HHs can connect. One time connection fee is a substantial barrier to HH connection to gas, especially in absence of a loan program. 11. The gas pipeline is extended to Poti FIZ, although is not used yet due to no industrial development going on in FIZ. The Evaluation Team confirmed this during a field trip to FIZ. This finding was further confirmed by TT reports. 12. FGDs conducted with HHs connected to gas indicated that women benefited more than men from the connection (less cleaning, more time to do other things, etc.). 13. PGIP is a large, complex project that includes 3 components. Although TT did an excellent work given the conditions, the technical burden imposed on it as the oversight contractor was much too heavy to allow efficient and best oversight performance with the limited staff. MID-TERM PERFORMANCE EVALUATION OF PGIP v | Page 14. The Evaluation Team found and learned during field trips on sub-stations that some critical transformers are operating without backup. An example is a single critical transformer in Tskaltobo 220V substation. 15. The condition of the new as well as old gas pipelines clearly indicate the deficiencies in pipeline preventive maintenance that can reduce the lifetime of the improvements provided by PGIP (see further details in section 4.4.) 16. The Evaluation Team found inconsistent and fading USAID sticker type branding. CONCLUSIONS 1. The gas pipeline and power transmission components of the project address the needs of GSE, GOGC/GGTC to improve the power and gas infrastructure. They are functioning properly and no significant problems were identified by stakeholders that would impact the performance of the completed infrastructure. 2. The gas pipeline component mid-term objectives were achieved by construction of envisaged parts of the pipeline and by providing economic measurable benefits and convenience to the consumers, as well as higher development opportunities to Poti FIZ through natural gas supply. 3. The project has led to positive impacts for all end-users who have connected to gas: HHs, businesses, social/pubic institutions and industry, recognize the economic advantage of switching to gas. Further utilization of the opportunities provided by PGIP gas pipeline component depends on economic growth and other factors outside of the project’s control. However, discussions with existing and potential consumers indicated the high potential of further economic development due to gas availability. 4. The cost-benefit analysis of the project was overly optimistic. The likelihood of industrial production being established depends on unpredictable economic factors and political decisions. For these reasons, the unknown element of large industrial production in Poti makes projecting costs and benefits a guessing game and requires a more conservative approach. 5. TT was highly instrumental in facilitation of communication between all the stakeholders, pushing hard for answers and allowing the project to move ahead. Both SER and GSE confirmed to the Evaluation Team the superior assistance they received from TT. 6. The project has the benefit of introducing modern technologies and equipment such as HDD, which provide higher efficiency and better protection of the environment. 7. The transmission line component is now moving ahead after some difficulties and delays caused by the introduction of sophisticated software such as PLS-CADD, as well as some Right of Way technical issues. 8. The smart grid improvements, through the installation of EECS, CAPE and DGAs, have increased reliability and improved maintenance of the transmission grid. All three sub-procurements, individually, have already demonstrated their contribution toward improving GSE transmission management and strengthening the grid security. The Evaluation Team monitored the smart grid’s sub-procurements operation in GSE HQ on two separate occasions. GSE personnel repeatedly stressed the contribution of these sub-procurements in time saving, efficiency, as well as strengthening the national electrical grid reliability. 9. The HH survey indicated that benefits are passed on through increased comfort, budget savings, and environmental benefits (HHs), as well as cost savings and business development opportunities (businesses and industry). 10. Total potential savings if all 17,773 HHs in Poti connected, would be a minimum $4.6 million per year. However, achieving this seems unlikely given the various barriers to connecting. Lack of modern, efficient and safe gas appliances, as well as their relatively high cost, limits the penetration of residential gas use for purposes other than cooking. 11. GoG’s expectations that FIZ would materialize and attract thousands of energy intensive businesses have proved to be overoptimistic, significantly reducing the benefits (to date) of the project. 12. Women feel that the project has provided them with great benefits, including comfort, less cleaning, and more time to do other things that they enjoy. MID-TERM PERFORMANCE EVALUATION OF PGIP vi | Page 13. Combining all projects under a single oversight roof, and without physical, local, and long-term availability of high caliber experts required to provide technical support, puts a heavy burden on the oversight contractor. 14. Some critical transformers in some sub-stations are operating without backup, which may be a problem in case the transformers suffer from winding short circuits, open circuits, or overheating. 15. Field visits indicated the lack of maintenance on the gas pipeline infrastructure, including old and new pipe, which is likely to affect the sustainability of the infrastructure. 16. Better type of USAID branding and marking is recommended – typically a non-fading metalized type of self-adhesive stickers as used for equipment rating plates would be a better solution. RECOMMENDATIONS 1. Establishing a training program designed for licensed gas technicians would further help boost both residential and commercial connections and use. Such training will help adding more than a single in-house connection – therefore allowing connection of more gas appliances, in addition to cooking ranges, such as heating and hot water appliances. Furthermore, private licensed gas technicians are expected to compete and reduce the cost of connection and therefore alleviate the financial barrier for a large number of potential users. 2. More conservative cost-benefit analysis is needed to avoid making unrealistic or erroneous assumptions. In the future, any cost-benefit analysis should be constructed, and thoroughly checked for validity with HH data and distribution company estimates to avoid making unrealistic or erroneous assumptions and reduce errors. 3. Encourage GoG to promote a policy of reducing residential connection barriers through re￾introducing financing programs. Consider introducing a small lifetime meter monthly flat fee instead of one time connection charge, or other options like offering customers a utility bond. This would increase the number of residential users benefiting from gas while further boosting the profits of GOGC/GGTC and municipal utilities. 4. Availability and physical presence of either local or foreign engineering experts to support highly complex technical projects is more efficient and advantageous than the present approach of utilizing mostly remote support. Such experts’ presence would allow better contractors support combined with improved field supervision and faster contractors’ design approval. 5. In order to minimize the probability of failure of critical transformers that are operating without backup, additional DGAs should be considered for installation. The Evaluation Team received feedback that GSE is committed to implement this recommendation. The final performance evaluation of the project should make sure that this is materialized. 6. GGTC should establish and finance an active program to improve the maintenance of the pipeline. It is recommended that USAID’s future infrastructure projects include a specific requirement from the beneficiary to prepare an active maintenance and monitoring program, allocation of skilled human resources to perform scheduled maintenance, and a method for verification that maintenance is performed. Such a requirement is likely to generate ownership of the infrastructure through maintenance while building a long-term sustainability. 7. Improve USAID branding stickers and prepare instructions for consistently marking equipment delivered under development programs. Furthermore, if possible, on computer equipment, display a continuous USAID logo during start-up and operation since this would be the best branding method. LESSONS LEARNED 1. Accurate assessment of the local technical and managerial capabilities is essential in determining the best working methodology for future infrastructure project designs. Specifically, such assessment needs to indicate whether the host country has skilled, experienced engineers and contractors required for the planned infrastructure project (covering all disciplines – civil, mechanical, electrical, electronic, etc.). 2. In the case that the host country lacks some of the skills and experience mentioned above, high calibre foreign experts should be made available locally to support, enhance and boost the local MID-TERM PERFORMANCE EVALUATION OF PGIP vii | Page capacity to reach the required competences and deliver a design-build infrastructure project. Building a local technical and managerial capacity for undertaking design-build projects is an important step of each country towards future self-sustainability. This would require not only training but also providing them with a high enough salary that will retain the skilled and qualified persons in the utility and in Georgia. 3. Complex requirements built into tenders are a serious impediment for small private companies who do not have much ability and resources to dedicate to complex procurement procedures, and therefore they do not participate. The possibility of dividing a large project into smaller projects with simpler requirements would enhance the ability of a bulk of the private sector small and medium size companies to participate in international tenders. 4. Cost-benefit analysis of a project should be performed before the project is approved and started. Care should be taken when developing cost-benefit analysis to adequately verify the input data, assumptions, and accuracy of calculations in order to avoid making unrealistic projects and inflating expectations. When cost-benefit analysis depends on the presence of large unknowns (direction of local economic development, decisions by large industrial consumers to build or not build), its value is considerably lowered. 5. It cannot be assumed that an infrastructure project such as PGIP will have immediate widespread benefits for end-users (HHs, businesses, industry) without ensuring that measures are in place to promote the use of said infrastructure. The investment costs borne by USAID or GoG are insufficient to achieve the economic benefits envisioned by the project – additional investment is needed by distribution companies and end-users. While a large company, such as SOCAR, faces few barriers to investing, low-income HHs need some type of support or incentives to invest. 6. New technologies like PLS-CADD should be considered as a component in local capacity building. In order to achieve the best results, a preferred option would be to keep those separate and fully implement prior to the project, rather than introducing in parallel with the implementation. In addition, more steps need to be included in the bid evaluation process to ensure that the bidders have in-house capability. 7. In terms of oversight, the company providing this service should have on staff qualified personnel. Oversight contractors should have a specific task, demonstrate availability of staff, including back-up personnel in case of vacations or sick leaves. Ideally the oversight contractor should have in office all required support technical personnel to provide engineering and field assistance to contractors rather than relying on remote engineering support. 8. Future technical training should cover the following areas: - Standards and compliance – specifically on drawings measurements uniformity (in millimeters including relevant tolerances). - Field measurements using advanced laser device for verification of dimensions accuracy. - Drawings evolution and revisions reflecting field progress towards as-built drawings. - Training at relevant levels in software application including manuals and local language operation instructions. 1 1.0 EVALUATION PURPOSE & EVALUATION QUESTIONS 1.1 EVALUATION PURPOSE This is a report on the mid-term performance evaluation of the Power and Gas Infrastructure Project (PGIP) funded by the United States Agency for International Development (USAID) Mission in Georgia. The PGIP project is being implemented during the period May 10, 2010 – September 30, 2014, by Georgia Oil & Gas Corporation (GOGC), Sakenergoremonti (SER), and Tetra Tech EM (TT). The mid-term evaluation of PGIP was conducted during the period June – August, 2013. According to the Statement of Work (SOW), the main goal of this assignment was to “evaluate the PGIP’s progress, and provide lessons learned for other USAID Missions in designing and implementing current and new programs in energy infrastructure construction and oversight services” (see Annex A). Therefore, the main objective of the evaluation was to assess PGIP’s progress towards its goals and, as a secondary objective, provide lessons learned on the sustainability considerations of large-scale infrastructure projects, implementation modality, and other important aspects. The main thrust of the evaluation included PGIP’s activities from May 2010 to May 2013, and focused on: 1. End users’ measurable economic benefits that can be associated with the gas pipeline construction under PGIP. 2. Overall quality of the design and procurement of the Senaki power transmission project. 3. Overall process of designing the transmission project. 4. Extent the operations, maintenance and sustainability of Georgian gas pipeline and power transmission systems improved, as well as the contribution that PGIP made (if any) towards such improvement. The evaluation was conducted by a team of three key experts: Mr. Peter Tal (Team Leader), Mr. Nils Junge (International Evaluation Specialist), and Mr. Murman Margvelashvili (Local Specialist). In addition, the team was assisted by Mr. Giorgi Giorgadze, head of the local firm IT, Research and Metadata Solutions (IRMS) who assisted with conducting the HH Survey, and Ms. Marika Gorgadze, ME&A Project Country Director and Interpreter. 1.2 EVALUATION QUESTIONS Per the SOW, the Evaluation Team was asked to answer a number of specific evaluation questions, outlined below: 1. Were the PGIP goals accomplished? These goals include: providing gas supplies to Western Georgia and increasing the reliability of Georgia’s power grid. a. What primary and secondary economic benefits, both current and projected, over the next ten years, can be associated with gas pipeline construction under PGIP? b. What is the overall quality of the design of the Senaki power transmission project prepared by the engineering oversight contractor as perceived by host country stakeholders? c. What is the overall process of designing the transmission project? To what degree did USAID’s investments in sub-procurement, including Computer Aided Protection Engineering (CAPE) and Dissolved Gas Analyzers (DGA), promote or have the potential to promote a safer, more efficient, reliable and robust Georgian power grid? 2. Sustainability of the infrastructure built with USAID support. a. To what extent have the operations and maintenance of Georgian gas pipeline and power transmission systems improved and what was PGIP’s possible contribution toward such 2 improvement? 3. What are the lessons learned from this mid-term evaluation of PGIP that can inform future program designs, procurements, executions and oversight services that will utilize both private and host country methods that minimize bias and provide strong evidence? 2.0PROJECT BACKGROUND PGIP was designed to: 1) promote energy security through greater access to electricity and natural gas supplies for households (HHs) and businesses in Western Georgia; 2) promote the development of the Poti Free Industrial Zone (FIZ) on the Black Sea; and 3) secure power exports through reliable transmission infrastructure improvements domestically. PGIP is a portfolio of $115 million in energy infrastructure projects that include: 1) main gas pipeline; 2) power transmission line construction; and 3) smart grid improvements of Georgia’s transmission network. The project is unique because it is funded from “one-time” supplemental post-conflict resources, is the largest USAID-funded infrastructure project in Georgia, and utilizes an innovative mix of both private sector and host country-controlled organizations as implementers. The map below (courtesy of GOGC) illustrates the planned pipeline projects and indicates the mid-term evaluated parts. Fig. 1: PGIP Pipeline Component – Senaki-Poti (operational), Senaki-Abasha (completed) Specifically, on each one of the project’s components, the following should be emphasized: 1. Main Gas Pipeline: Its objectives were based on Georgia’s national priorities as follows: a. Expand the gas supply to Western Georgia regions where no natural gas supply was previously available. The city of Poti and the expected to emerge FIZ were a priority. Therefore, a 700 mm diameter pipeline was constructed as an extension of the old 500 mm pipeline from Senaki to Poti. 3 b. Rebuild the old and reaching the end of life segments of pipeline, starting with Senaki-Abasha and continuing further eastwards, as long as there are funds available and USAID and the GoG agreed on the sections to bypass. 2. Power Transmission Line Construction: The objective is the reconstruction of 58.8 km of a double circuited 220 kV - Senaki 1 and 2 transmission lines and their associated bays at the Menji and Tskaltubo substations. The original transmission line was destroyed during the conflict years. The original double-circuit line was, and still is, a key part of the transmission system in Western Georgia. Without this line, the loss of the parallel 500 kV transmission line from Western to Eastern Georgia causes the underlying 220 kV and 110 kV systems to be overloaded and collapse, and thus cause a regional blackout in Eastern Georgia. Fig. 2: PGIP Transmission Component - 220Kv line Tskaltubo-Menji (courtesy of GSE) The Power Transmission Component includes three independent sub-procurements, which enhance Georgia’s electrical grid reliability, supervision and overall transmission, as well as load management. The sub-procurements are: a. CAPE, a software for relay and fuse settings. Under PGIP, USAID provided this software to the Georgian State Electrosystems (GSE), the transmission system operator of Georgia. The computer program enables faster calculation of more reliable settings of relays on the transmission system and, consequently, enables more reliable energy transmission systems and potential for increased regional trade to exporters of electricity from Georgia’s renewable hydro power. The software became operational in the fall of 2011. b. Enhanced Emergency Control System (EECS) for Transmission System: This smart grid sub￾procurement is an advanced system protection (relays, field units, and telecommunications equipment), which monitors the state of the GSE transmission system and takes rapid action to maintain the balance between load and frequency. The reaction time (from event to remedial action) of the system is less than fifty milliseconds, which is much faster than the existing load￾frequency control system. The system includes monitoring equipment (phasor measurement 4 units or phasors or synchrophasors) at 18 substations in Georgia. The system also includes monitoring equipment in the Borchka 400 kV and the Muratli 150 kV substations in Turkey, and in the Samux 500 kV substation in Azerbaijan. EECS supports the Black Sea Transmission Network (BSTN) and interconnection agreements among Azerbaijan, Georgia and Turkey. The system was originally scheduled to be fully in service by the end of September 2013, but was already in operation in July 2013. c. DGA: This activity covers the purchase and installation of on-line analyzers (field units), which measure the concentrations of the nine specific gases that are dissolved in the oil of GSE’s large power transformers on the 500 kV, 220 kV, and 110 kV transmission systems. The amount of the dissolved gas and their relative ratios indicate the health of the transformer. Rapid detection by the DGA of the build-up of the harmful gases in the transformer enables GSE to take rapid action to avoid failure of the transformer or, at least minimize the amount of damage to the transformer when it does fail. A typical size transformer is 500 MVA (megavolt amper) and the failure of such a transformer can affect about 100,000 customers. The DGA was scheduled to become operational by the end of April 2013. As of August 2013, it was partially operational due to problems not clearly disclosed to the Evaluation Team. 3. Engineering Oversight Component: This component provides resident professional engineering and other technical services to support power and gas transmission improvements being undertaken by USAID on behalf of the GoG. These services include a full range of expert engineering advice and oversight, organizational capacity building expertise, and the provision of analytical and technical support to USAID/Georgia. 3.0 EVALUATION METHODS & LIMITATIONS 3.1 EVALUATION METHODOLOGY The evaluation was organized around the two main components: gas pipe and electricity transmission. For each component, the Evaluation Team analyzed concept design; specifications; rationale for contractors’ selection; procurement; compliance; schedule and implementation; engineering oversight; capacity building; sustainability and operation cost; economic benefits; and project management. The Evaluation Team also incorporated TT’s oversight and training assistance, and analyzed oversight with engineering design and implementation of gas and electricity components, as well as oversight of procurement, training and capacity building. The Evaluation Team used a standardized questionnaire for interviewing TT, GSE, GOGC, GGTC and SER. Information included in this evaluation report is based on verbal or written answers. Specifically, the evaluation assessed: 1) economic benefits to the consumers that can be associated with gas pipeline construction under PGIP; 2) the overall quality of the design of the Senaki power transmission project; 3) the overall process of designing the transmission project; 4) the extent the operations and maintenance of Georgian gas pipeline and power transmission systems improved, as well as the contribution that PGIP made toward such improvement; and 5) measurable financial/economic benefits relevant to operations, maintenance and sustainability of the pipeline. Given the type of PGIP project activities, both qualitative and quantitative methods were used to conduct the evaluation. Data was collected from Tetra Tech, GOGC, GSE, SER, the State Oil Company of Azerbaijan Republic (SOCAR), and a broad range of stakeholders and beneficiaries (including HHs and businesses) to ensure independence of the evaluation process, as well as the accuracy and completeness of the subsequent conclusions, recommendations, and lessons learned. Techniques that balance each 5 other were utilized: quantitative vs. qualitative data; individual interviews vs. focus group discussions; questionnaires; HH survey; direct observations, etc. 3.1.1 Quantitative and Qualitative Analysis To assess the economic benefits that can be associated with gas pipeline construction under PGIP for end-users, the Evaluation Team collected quantitative and qualitative data. This mixed methods approach was used to assess socio-economic benefits accruing to residential and non-residential users in the region. The units of analysis were HHs and industrial/commercial consumers in the project region, i.e. end-users expected to experience a material change resulting from the project. In the absence of baseline data, comparisons were made between HHs connected and not connected (by choice) to gas, to better understand the relative benefits delivered by the project. HH data collection was conducted by the local subcontractor, IRMS. Focus Group Discussions (FGDs)  The Evaluation Team held two FGDs with HH members from the municipalities of Poti, Senaki, Abasha and Samtredia. HHs from the towns along the pipeline not expected to be affected by the project were also included, partly to verify that this was truly the case, and partly to determine whether there were any practical differences with Poti HHs.  FGD participants were selected by local staff of IRMS in each locality using the following criteria: knowledgeable regarding HH expenses, gender balance, and age diversity.  The purpose of the FGDs was to improve the Evaluation Team’s understanding of the welfare impact of connecting to gas, in order to analyze the project’s actual and potential impacts, as well as to inform the mini-HHs survey questionnaire.  FGDs were conducted in Georgian language. The Evaluation Team had FGD participants sign an informed consent form and ensured them of their anonymity.  One FGD was conducted with connected and one with un-connected HHs; different question sets were used for each group (see Annex G). The discussions allowed the Evaluation Team to probe the key evaluation questions, and better understand the context of energy use in the region.  The FGD sessions proved very useful for informing the HH questionnaires, which were revised accordingly. Mini-HH survey  Because of time and resource limitations, only a small HH survey was conducted. The sample size of the survey was 100, a number believed to be sufficient given the relative homogeneity of the population in question. While such a small sample allows only limited analysis of subgroups, the purpose of the evaluation did not extend to comparing subgroups beyond: i) connected/not￾connected; ii) Poti/Senaki; and iii) male/female. For all three dimensions, half of the respondents belonged to each group.  The relatively small sample size means that findings will be less statistically representative (the margin of error will be larger) than if the sample were larger.  HHs were sampled randomly in several clusters in each city. The questionnaires were pilot tested on 16 HHs and revised before final printing.  To conduct the survey, the Evaluation Team used random, purposive sampling. In Senaki and Poti, after interviewers selected those areas where most of the HHs were connected to gas, every third HH was selected to be interviewed. After selection, the respondent who had knowledge of HH energy expenses was selected using Kish Grid if the number of such HH members were more than 1.  Data was analyzed using Stata and NesStar publisher, and results were cross-checked by two data analysts.  Topics covered by the FGDs and the HH survey included service quality; payment; billing service; consumption (monthly) by heating and cooling, lighting, cooking, hot water heating; supply reliability (interruptions); tariffs; overall perceptions of value for money; abilities to pay bills, actual bill 6 payments; connection costs and payment methods; and potential gender impacts. Surveys were developed in English, and translated into Georgian after USAID approval. Key Informant Interviews • The Evaluation Team conducted a number of key informant interviews with non-residential customers and potential customers in Poti and Senaki to better understand the impact that connecting to gas has had (or may have) on their operations. • Key informants were not selected with a view to obtaining a statistical sample but rather to explore issues related by canvassing a diverse group of stakeholders. • While businesses are often reluctant to share commercial information, these interviews were nonetheless quite useful in obtaining estimates of the energy savings they were able to accrue by connecting to gas. 3.1.2 Qualitative Research and Analysis The qualitative evaluation began with a critical desk review of materials related to PGIP, including project reports and annual work plans, project performance management plans, project design, communications among partners, etc. The Evaluation Team conducted interviews with USAID and PGIP implementers, including TT, GOCG, GSE, GGTC, and SER; in-depth and semi-structured interviews with selected program beneficiaries and stakeholders such as the Ministry of Energy (MoE), GSE, EnergoPRO Georgia, SOCAR-Poti, Gasko+ (Senaki distribution company), Asian Development Bank (ADB), the European Bank for Reconstruction and Development (EBRD), KFW, and key informant interviews with businesses/industrial consumers (see Annex C). The Evaluation Team traveled to Poti, areas along the old Soviet pipeline, Senaki-Poti line, Senaki- Abasha line, Zestaphoni and Menji substations, transmission foundation sites, and old transmissions sites. The purpose of these site visits was to conduct fieldwork, as well as to observe Poti FIZ and the penetration of gas distribution/connection to residential and commercial users in Poti, Senaki and Kutaisi, and places adjacent to the main road. 3.2 EVALUATION LIMITATIONS The Evaluation Team encountered few limitations during its fieldwork in Georgia. Some of the more relevant limitations were: 1. Period Evaluated: This was a mid-term evaluation. A number of PGIP’s key initiatives were still at too early a stage to expect any meaningful progress toward important performance indicators and determine the concrete results of the PGIP project. 2. Resources Allocated for the Evaluation: The complexity of the project and amount of information produced during its implementation, combined with the limited time and resources allocated for the evaluation, limited the in-depth evaluation and the ability to follow up on the issues identified during the country visit. Therefore, the Evaluation Team relied mostly on data and information provided by the implementers, including utility billing records and data, where available. 3. Access to Technical Information: The team had some difficulties getting drawings of as-built structures or components prior to the field trips. In some cases they were not yet available, somewhat limiting the ability to assess design against construction. 7 4.0 FINDINGS, CONCLUSIONS, RECOMMENDATIONS QUESTION 1 - SUMMARY OF EVALUATION DESIGN AND METHODS Evaluation Question Type of Analysis Conducted Data Sources and Methods Used Type and Size Sample Limitations Were the PGIP goals accomplished? These goals include: providing gas supplies to Western Georgia; and increasing the reliability of Georgia’s power grid. Structured questionnaire (same) for all stakeholders GOGC, GSE, GGTC, SOCAR, SER, Tetra Tech – Structured interviews replies Technical and executive management made available by each stakeholder Time required to follow-up and verify answers Field trips Drawings, visiting construction sites, interviews with contractors Main gas pipeline, old pipeline, foundations for transmission line Weather and difficult terrain HH survey 4.1 QUESTION 1: WERE THE PGIP GOALS ACCOMPLISHED? This broad question encompasses: 1. Gas Pipeline Component 2. Transmission Component 3. Smart Grid Improvements - CAPE Software - EECS Software - DGA Hardware and Software The Evaluation Team finds it necessary to address all the above separately and consistently. 4.1.1 Findings 4.1.1.1 Gas Pipeline Component Based on the desk review of materials, direct observations and data collected in the field, the Evaluation Team found that:  The segment Senaki-Poti (30 km) was completed on time and it is fully operational, and gas is supplied to Poti’s residents and business.  The segment Senaki-Abasha (29 km) was completed on time and is presently under testing.  The three Cathodic Protection system stations are operational (range 5 km each-side, total 10 km protection) and covering the whole Senaki-Poti segment.  Approximately 48,000 inhabitants of Poti have or will have access to natural gas. The Evaluation Team interviewed SOCAR, the gas utility in Poti, and confirmed that works are in process to allow any potential user to connect to gas. The team completed a visit to street works site where gas pipelines are prepared for future connection of all potential users (see photo).  Poti FIZ has access to gas. The team visited FIZ and confirmed that gas is available for future user connections. However, at the time of the visit, there were no users in FIZ who might be in need of gas.  Visits in Poti, indicated that only a limited number of businesses were connected to gas (17). However, interviews conducted by the team with potential future gas users, indicated possible large 8 consumption. If all the estimated 153 (registered) Poti commercial organizations attained consumption levels similar to Senaki, consumption would potentially reach 1.3 million m3. (In Senaki, 76 businesses consumed up to 643, 795 m3 in 2012).  The Evaluation Team visited the main appliance retail store in Poti center to confirm whether sanitary hot water heaters and heating appliances are available. The team found that only electric water heaters were on display. 4.1.1.2 Transmission Component At the time of the evaluation, the transmission component was in early stages of construction. Only a part of the towers’ foundations along the way between Menji and Tskaltubo sub-stations was completed. The Evaluation Team found that:  The project is slightly behind the schedule but when completed, it will achieve one of its main goals: providing a transmission alternative in case of the 500kV line failure. The main reason for the delay, as confirmed by the Evaluation Team’s interviews with USAID, SER and TT, was caused by the contractual requirement to implement the design by using Power Line Systems - Computer Aided Design and Drafting (PLS-CADD), a modern software design tool used in the West that was not previously used in Georgia. SER, the transmission contractor, had experience in transmission project design and construction using Soviet/Russian standards and methodology2 . However, a new modern tool such as PLS-CADD, required time for learning, especially for the older and experienced SER personnel, trained in manual design. A younger team was introduced to work with the software tool and master the design work. This lack of local capacity in use of PLS-CADD, and PLS Towers software, caused delay in the implementation schedule. 4.1.1.3. Smart Grid Improvements CAPE Software. CAPE has achieved the goals of enhancing and strengthening a more reliable electrical grid. The specific findings are:  The system and software has been in full operation and use since 2011. The system provides a fast, efficient, and reliable method for calculating the relay settings to reflect changing requirements due to seasonal changes in power demand and supply, as well as current conditions of the transmission grid and generating plants. When maintenance and re-routing of power to alternative routing is needed, a fast setting change is also required.  In interviews with the Evaluation Team, GSE representatives confirmed the need and benefits for such system, which has enhanced the company’s performance by saving time and improving the accuracy in modifying the transmission relay settings according to changing needs. EECS System Software. EECS has not only achieved its objectives but it has exceeded the expectations and it is a success story. The specific findings are:  EECS is a superior transmission management tool allowing automatic load re-routing, as well as fast load shedding. This function is important and crucial in order to prevent, or at least minimize, the black-outs. The Evaluation Team monitored the operation and simulations performed by EECS during two separate visits at GSE and received a detailed overview from a GSE engineer.  EECS is praised by GSE as a major contributor in grid’s safety enhancement, including reduction of blackouts by allowing re-routing and power shedding. According to a Board Member of the GSE, “use of EECS software has already reduced the lost electricity due to Imereti outage by 70-80%. Since 2011, there has been no system blackout due to EECS equipment.” (Information provided by email to the Evaluation Team).  EECS became operational ahead of schedule and once the software upgrade is performed, additional benefits are likely to be attained. GSE’s personnel is continuously operating the EECS and maintains it 2 SER provided the Evaluation Team with a document detailing the company’s experience in transmission and electrical works for GSE as well as for other companies and donors. 9 up-to-date. A new software upgrade is scheduled by September 2013. Once the upgrade is completed, additional benefits are likely to be obtained. DGA Hardware and Software. Even as repairs and upgrades were in process at the time of the evaluation, the Evaluation Team found that the DGA has and will achieve its goals, specifically:  In operation, the system provides clear indication on vulnerability of transformers requiring service and maintenance. DGA detects excessive dissolved gas in the transformer oil and allows preventive maintenance to be performed timely on a transformer to keep it in continuous operation. The finding is based on monitoring DGA’s operation at GSE HQ.  The Evaluation Team found and learned during field trips on sub-stations that some critical transformers are operating without backup. The photo on the right shows a single critical transformer without backup in the Tskaltobo 220V substation.  By the end of the mid-term evaluation mission, GSE reported instances where DGA warned on immediate required maintenance and helped avoid transformer failure. DGA terminal is in operation at the GSE HQ in Tbilisi. 4.1.2 Conclusions Gas Pipeline Component  GOGC and GGTC have demonstrated good engineering and management, performing well and mostly on schedule. The evaluation team defines good engineering and management in a project as the ability of the contractor to deliver a good design and complete the project on or below the budget, as close as possible to the set schedule, and on the specified quality. GOGC met all above requirements. Senaki-Poti (30 km) was completed on time and it is fully operational. Senaki-Abasha segment was also completed on time and is being tested. The three Cathodic Protection system stations are operational.  While commercial and industrial customers take almost immediate advantage of the availability of gas, residential users are slow to connect to the municipal distribution network. However, in Poti, due to natural gas availability, significant economic development can potentially materialize.  There are very few available heating appliances and water heaters operating on gas in the retail outlets, which leads to use of other non-gas appliances for heating and sanitary hot water. Transmission Component The transmission component is slightly behind schedule, the main reason being the introduction of a new design software (PLS-CADD), which required skilled users. However, once the transmission will become operational, the specific objectives of this component will be achieved. Smart Grid Improvements Each and every one of the smart grid sub-procurements has a major contribution in: strengthening Georgia’s national grid reliability; reducing and using more efficiently the engineering time; and providing individual, as well as aggregated management benefits, including improved preventive maintenance (DGA only). EECS is a success story. 4.1.3 Recommendations Gas Pipeline Component  The Evaluation Team observed different ways of providing users access to gas. Compared with other locations such as Senaki and few others observed by the team, the methodology used by SOCAR appears to be the most efficient and reliable, even though it is more expensive. The Evaluation Team 10 could not obtain specific costs from different municipal gas utilities for connecting individual customers. Therefore, we looked at two main issues: 1) safety; and 2) easiness to connect new users. Substantial safety and reliability of the gas delivery network is achieved by constructing main delivery pipes underground (as done by SOCAR in Poti) since likelihood of breakage, vandalism, etc. is significantly minimized (the other municipal gas utilities built a less expensive aerial network. See pictures below). In terms of easiness to connect new users, the underground gas distribution network must prepare a potential connection for new users. Connecting a new user becomes a simple job and does not affect other users since the distribution line is not shut down. However, in the aerial distribution network, since all connected and new connections are on the same line, cutting and welding a new connection requires shutting off the whole line. Based on the above information, it is technically and commercially evident that the underground gas distribution system used by SOCAR is more effective while it is also more expensive because of the kind of work and resources it requires for building it.  Enhance the safety and number of residential and commercial gas connections, networks and appliances through introduction of courses for certified gas technicians, to be licensed nationally, in order to connect users to the municipal gas distribution networks. The proposed approach would reduce the connection costs, release the gas utility from the connection work required, and increase their revenues. Additional advantage for licensing gas technicians is in developing private sector competition together with enhanced safety and connection cost reduction. The rationale for this recommendation is driven by observations and research of national prevailing methodology. Currently, the gas utility delivers a single connection to gas inside the house. This single connection is typically in the kitchen and used for cooking, as clearly shown by the HH survey. Gas utilities charge more for additional gas connections in the house for use of heating gas appliances. The typical western approach is that the gas utility provides the connection to the house exterior (gas reducer and gas meter) only. From this point on, certified gas technicians prepare all required inside-the￾house gas connections to allow using the gas for multiple purposes such cooking, heating, drying, etc.)  Introduction and promotion of gas fired water heaters rather than electrical water heaters found in stores in Poti could save users a substantial amount of money. For example, an average summer shower of 50-75 liters of hot water would consume approximately 2.3 kWh, while the same shower in winter would require approximately 3 kWh to raise the water temperature. Taking into consideration about 40,000 showers/day in Poti only, by switching to gas fired hot water appliances, 11 between 92 MWh/day in summer and 120MWh/day in winter can be saved. The Evaluation Team estimated the savings of an average 36,500 MWH/yr due exclusively to the Poti population switching to gas for water heating only. Transmission Component  A better pre-project planning/training for introduction of new, sophisticated software could improve meeting implementation deadlines. Smart Grid Improvements  DGA, CAPE, and EECS have had a significant impact on GSE’s operations. According to GSE, these sub-procurements put the company well ahead of those in the closest neighbor countries, including Russia. Therefore, use of such tools should be continued wherever possible.  Due to a relatively large number of critical transformers in substations with no back-up in case of technical temporary failure, it is recommended to expand the DGA to all such transformers. The Evaluation Team was informed that, recently, GSE committed to install DGAs in all critical transformers. The team that will conduct the final project evaluation should verify whether this commitment materialized. 4.2 QUESTION 1.A: ECONOMIC BENEFITS ASSOCIATED WITH GAS PIPELINE CONSTRUCTION UNDER PGIP QUESTION 1.a - SUMMARY OF EVALUATION DESIGN AND METHODS Evaluation Question Type of Analysis Conducted Data Sources and Methods Used Type and Size Sample Limitations What primary and secondary economic benefits, both current and projected, over the next ten years, can be associated with gas pipeline construction under PGIP? GOGC/GGTC/GSE structured questionnaires interviews. Data received from GOGC, GGTC and GSE Reports, documents, stakeholders interviews, field survey, Geostat, operators and feedback to questions Executive management of stakeholders’ management. (2- 4 persons in each session) None encountered SOCAR and field surveys. Supply and consumption data. Demand forecast. Poti and Senaki utilities and users None encountered. HH, commercial and industrial users 4.2.1 Findings The economic impacts of PGIP, like any infrastructure project, are transmitted through those who use it, the beneficiaries, as well as those who manage the energy and infrastructure. Project implementers can also be considered beneficiaries. In the case of PGIP there are thus multiple levels of beneficiaries. Potential economic impacts can be broadly divided into two dimensions: - Widespread impacts at the HH level. While not of great significance to the national economy, these impacts are keenly felt and appreciated by individual HHs. - Major (but as yet unrealized) commercial and revenue impacts through the development of industry. In contrast to HHs, PGIP’s impact to industry has potentially large implications for the economy (dwarfing consumption by HHs) via job creation, revenue and taxation. In Poti, the local distribution company is SOCAR-Georgia Gas, which is also the largest distribution company (and main gas importer) in Georgia. SOCAR reports that it is investing $11.2 million in its Poti network, which is nearing completion this year. If all current Poti HHs and businesses connect, SOCAR- 12 Georgia (which distributes gas throughout Georgia) expects that it would increase its nationwide customer base by about 13%, from its current customer base of 153,000.3 In an economic analysis that SOCAR Georgia Gas shared with the Evaluation Team, the company has estimated an expected net cash flow of approximately $8 million over 10 years. The analysis factors in: average annual consumption (domestic and commercial customers); cost of goods sold; administrative expenses; irrecoverable debt allowance; and depreciation and taxes paid. The average domestic consumption of over 17,000 household and commercial customers was projected to grow from 685 m3 in year 1, to 894 m3 in year 10. It expects net cash flow turn positive in year 4 and grow to $2.5 million by year 10. SOCAR also reports that the total amount of taxes paid will be around $4 million over the 10-year period. It is clear that PGIP, through the Poti-Senaki line, has enabled the company to significantly expand its customer base and revenues. 4.2.1.1 Gas Customers For the analysis, gas customers were disaggregated into the consumer types and how they benefit from connecting to gas (see Table 1): Table 1. Beneficiary types and primary uses of natural gas Beneficiary Uses HHs Cooking, heating, hot water Businesses Production processes, same as HHs Social / government institutions Mostly heating, hot water Industry Manufacturing, production processes 4.2.1.2 Business and Industry After 18 months of gas availability, SOCAR has connected 17 non-residential customers in Poti, out of 279 commercial customers that it had identified in 2009. At the time of the evaluation, no permanent industrial users were identified in Poti. The only active company is Transmobil, a temporary asphalt factory, which consumes approximately 7,000 m3/day, or seven times as much gas as all Poti HHs consume combined, and provides a good sense of the potential impact of industry on consumption. Transmobil expects to be in operation for approximately two months to implement a project before closing down (it would only operate again if it were to win another contract). An employee reported that it had only applied for the bid because of the availability of gas. Potential industrial users for which information was obtained by the Evaluation Team include a smelter, which is ramping up operations, and a large fertilizer plant. Moulds and Metals Georgia, a company with 300 employees, noted that they would use gas for a rolling mill during phase II of their expansion (when employment would climb to 1,000). By far, the most significant user of gas, and potentially the greatest impact on the local economy, would come from SOCAR’s own industrial expansion plans, which include a fertilizer plant that would use 400-500 million m3/yr. In Poti, there is no evidence as of yet of permanent industry launching because gas has become available (with Transmobil as the only large-scale user, but temporary). For industry, numerous barriers to expansion and development exist; gas is only one out of numerous factors which influence a company’s 3 The figure of 13% is based on adding 20,000 customers (all businesses and HHs in Poti) to the 153,000. 13 business decisions. Others include the availability of other utilities (electricity, water, sewerage), roads and transport facilities, concessions, a qualified workforce, etc. However, any industrial production relying on gas that eventually launches in Poti will dwarf residential/business consumption, as the temporary asphalt factory demonstrates. Of the business and institutions interviewed, those that connected reported a decrease in energy expenditures by one half and two-thirds. Significant benefits to their bottom line were also reported. Connection costs (from 2,000 GEL to 15,000 GEL) were recouped in a matter of months. The impact of connecting to gas will depend on its relative importance as a business cost. The following small case studies of non-residential customers illustrate the impacts:  The Poti Medical Center switched to gas as part of a complete rehabilitation of the building, after which it reopened in March 2012. Previously, it used electricity for heating and had monthly costs of 7,000-8,000 GEL in the winter. With gas, its costs have fallen to 1,500 GEL. In addition, using gas is perceived as more comfortable (by 100% of household users surveyed) and efficient (according to several FGD participants.  The largest bakery in Poti, Tsiala Topuria, (part of a larger conglomerate, which includes 5 bakeries and a poultry operation) switched to gas from diesel in 2012. This reduced its energy costs from about 12,000 to 7,000 GEL. The bakery’s revenues are 1.5 million GEL. Connection costs were about 15,000 GEL. The cost of energy in its bread products has fallen from 6-7% to 3-4%, and profits have increased by about 7-8% as a result. Savings have made business expansion easier. Some were reinvested in new vehicles for the conglomerate.  Public School No. 12 in Poti (single building, 226 students) switched from diesel fuel to gas in December 2012. While previously its winter gas bills were 3,000 - 4,000 GEL, its maximum gas bill is now 1,200 GEL. However, the school administrator pointed out that other schools in Poti are unable to make the switch because they are significantly larger (and include multiple buildings) and thus cannot afford the connection cost. 4.2.1.3 Poti-FIZ In spite of the facilities, tax exemption and availability of electricity and gas, there is no major economic activity in FIZ, which was established in 2008 and opened in 2010. FIZ provides electricity and water only. There is no sewage yet in place. There are 300 plots with a surface of 5,000 m3 each or 1,500,000 m3 from which all are available for leasing. Approximately 225 companies expressed interest in starting business in FIZ or using their bonded warehouses. However, as of today, only three companies have set up shop. These are small assembly companies, which have no need for gas. Very few employees were seen (3 women in the perfume filling and 3 men working in the bonded warehouse). Expectations were that electricity and gas access will attract business and stimulate economic activity, job creation and manufacturing to FIZ. 4.2.1.4 Employment Based on interviews with Poti non-residential gas users, at this point in the project it is not possible to associate any long-term employment creation with the implementation of the Poti-Senaki gas pipeline. The reasons are related to the type of non-residential users in question, i.e. non-industrial who do not use gas in production processes. While the cost savings are significant, they have not yet led to expansion of business, which would require additional employees. This does not mean that employment will not be created. However, any significant changes in this area would most likely result from development in Poti’s industrial sector, which may still occur. Of course, future evaluations will need to take into account that not all new jobs created can be attributed to access to gas, as numerous other factors play an important role in decision-making: other infrastructure, economic environment, business￾friendly environment, etc. 14 The project created some temporary employment through implementation. SER, noted that they expect to generate approximately 300 jobs, through hiring and subcontracting, lasting from several months to more than a year. 4.2.1.5 Gas Network Access and Connections In Poti, there are an estimated 17,773 households who can potentially connect to gas. SOCAR Gas’ Poti distribution company reports that most of them already have access to their distribution network, and that virtually all will have access by October 2013. At present, only about 10 % of all Poti HHs (1,850) had been connected as of July 2013. The first HH connections took place in November 2011 and, within one year, 1,683 were connected, at a rate of 140/month. In 2013, the rate of connection was slow for the first six months, with only 112 new HHs added, a rate of less than 10 per month. This is likely to be related to the end of the installment plan promotion - 90% of unconnected HHs reporting this to be a barrier to connecting. According to the Performance Management Plan (PMP), 17,773 residential customers were expected to eventually connect to the network in Poti. The PMP estimates 15% to 20% of these potential customers to connect annually, suggesting all HHs and businesses would connect within 5 to 7 years of project start. This timeline was confirmed by the SOCAR Poti office. 4.2.1.6 Total Consumption Levels Gas consumption in Poti went from zero before Senaki-Poti gas line to 267,463 m3 in 2012, and 422,207 m3 in the first 6 months of 2013. The year-on-year increase reflects increasing customer numbers and a few larger customers.4 Based on company distribution data, average monthly consumption rates were estimated at 0.75 m3 for HHs in Poti, and 1.5 m3 per day for Senaki HHs. Higher consumption in Senaki suggests that HHs which have been connected longer tend to consume more gas, in line with findings that it takes time for the newly connected HHs to save funds to purchase new gas-reliant appliances. 4.2.1.7 Poti – Senaki Comparison Although Senaki is not directly affected by the Senaki-Poti line construction5 , Senaki is a municipality similar in size to Poti (differences in the number of potential customers appears to be linked to coverage of municipal center vs. outlying regions and villages), and may offer some useful lessons for the latter in terms of connections (see Table 2). Table 2. Comparison between Poti and Senaki – gas Indicators Poti Senaki General Gas company SOCAR Gasko+ Population a 47,700 52,300 Year connected to gas b 2011 2007 Share of city covered Complete coverage by October 2013 2/3; added 15 km in 2012, 2 km in 2013 Customers (all) with access b 17,773 9,250 Residential 4 GOGC data. 5 Although he was aware of the project, the director of Gasko+, the Senaki distribution company, did not know that it had come on line already, underlining the fact that Senaki is not noticably affected by PGIP. 15 Indicators Poti Senaki Tariffs – HH (GEL) b 0.53 0.483 HH customers connected b 1,850 > 2,700 Customers with access b approx. 13,000 approx. 5,500 HH consumption levels, m3 (2012) 267,463 1,877,480 HH consumption levels, m3 (2013) 422,207 945,257 Residential consumption m3/month (avg.) d 22.5 45.4 Residential consumption m3/day (avg.) d 0.75 1.5 Commercial Tariffs – commercial (GEL) b 0.89 (commercial) 0.80 (budget orgs) 0.86 Businesses connected (June 2013) b 17 76 Business consumption levels, m3 (2012) 28,557 643,895 Sources: a GeoStat, b SOCAR, Gasko+, c HH survey, d GoGC (Poti), Gasko+ (Senaki) Without a gas network since the early 1990s, Senaki was reconnected following the rehabilitation of the pipeline in 2007. Connection costs per HH are not uniform. They depend on a number of factors, including distance from the pipeline. Of the 50 connected HHs interviewed, the cost mostly ranged from 300 GEL to over 900 GEL (see Table 3). (Note: This sample is illustrative and not statistically representative. Conclusions cannot be made regarding the mean difference in connection costs between the two municipalities.) Table 3. Connection cost per HH (GEL) based on HH survey responses Range of cost per connection Poti (N) Senaki (N) 0 1 0 300‐399 1 7 400‐499 9 3 500‐599 7 2 600‐699 4 5 700‐799 2 0 800‐899 0 7 <900 1 1 total 25 25 Source: HH survey In Senaki, the rate of connections has been 51GEL per month in 2013, rising to 2,676 GEL on July 1, 2013. This represents about one third of Senaki HHs. Gasko+ reports that one third of HHs have access but have not connected, while the remaining third do not yet have access. Gasko+ built 15 km of distribution network in 2012 and has built 2 km so far in 2013. It is noteworthy that in Senaki, after 6 years of gas access, only one third of the population has connected. In contrast, SOCAR plans to have all of Poti connected by October 2013. Even with better access in Poti, however, it seems unlikely that all Poti HHs and businesses will connect without some sort of assistance in overcoming the barrier posed by connection costs. 16 4.2.1.8 HH Consumption Patterns The assessment of the impact of gas on HHs vividly underlines why the American expression “Now you’re cooking with gas” is used as a semi-humorous analogy when commenting on a marked improvement in someone’s performance. The ability to use gas instead of liquefied petroleum gas (LPG) or wood has a substantial impact on HH budgets, comfort, and convenience.6 This is a benefit which ties in clearly with the US’s interest of seeing “an economically prosperous Georgia.” The most common switching behavior is applied to energy used for cooking, from LPG cylinders to natural gas, which is significantly cheaper. Switching to gas is a potential cost-saving measure. Monthly energy expenditures go down. But switching is not costless, requiring an upfront investment to connect, acquire a meter, and equipment or appliances that use gas. As a result many potential customers delay connecting, while others never connect at all. Furthermore, limited evidence of safe, clean, and efficient gas fired appliances for space and hot water heating was found to be available in retail stores (water heaters on sale are electric). There are significant non-monetary benefits to switching to gas. The non-monetary benefits, in fact, appear to outweigh the monetary benefits. According to the survey conducted by the Evaluation Team, 100% of connected HHs, and 98% of unconnected said they would switch even without savings. The primary reasons were ‘comfort’. Other reasons given for switching were protecting the environment, and reduced pollution. Thus, it can be argued that even if gas were more expensive (wholesale gas prices are subsidized in Georgia), HHs would still receive a substantial benefit. 4.2.1.9 Energy Usage Patterns and Switching Behavior In Western Georgia, as elsewhere, HHs use multiple energy devices/appliances and multiple energy sources. Energy sources used by urban HHs in Poti and Senaki are primarily LPG, electricity, gas and wood. When HHs connect to the gas network, they substitute it for energy sources they had been using previously. Despite the clear benefits of using gas (in terms of convenience, comfort and cost), the switch to gas is generally found to be only partial. Many HHs take a phased approach to gas usage once they have connected. After connecting, they typically start using gas mainly for cooking and hot water while saving up funds to pay for a heating appliance. HHs that connected to gas were asked to list all the fuels they used before and after switching to gas. The figures below show the percentage of HHs using which types of fuels during the different periods: 6 In fact, because gas was the topic of the mini-HH survey, many HHs were happy to be interviewed, and expressed their satisfaction to the enumerators. None of the 100 HHs refused to be interviewed, an unusual response rate. Fig 4. Percentage HHs using fuel for hot water before and after switching to gas b f d f i hi Fig 3. Percentage HHs using fuel for cooking before and after switching to gas 17 Figure 5. Percentage HHs using fuel for space heating before and after switching to gas As seen above, the great majority of HHs use gas for cooking and hot water. About half of connected HHs (24) do not use gas for heating, most saying the heating appliance is expensive, and 13 (38%) mentioning safety concerns. 4.2.1.10 Reasons for Connecting To better understand how consumers perceive the benefits of switching, respondents were asked to explain their reasons for connecting to gas. All respondents reported that increased comfort as a reason, three quarters reported lower energy expenditures, and at least half reported less air pollution and nature conservation (from fewer trees being cut for firewood). From the results, it is clear that monetary benefits are only part of the story. The other perceived benefits, although more difficult to quantify, must also be taken into account when assessing economic benefits associated with the pipeline construction. Figure 6. Reasons for Switching to Gas (%) Source: Author calculations based on HH survey 4.2.1.11 Gender Benefits Do men or women benefit more from switching to gas? Intuitively, given that women are traditionally responsible for cooking and cleaning, it was expected that anything that increases convenience in the home would reduce the burden of work for women (see Fig. 7). 18 Figure 7. Who benefits most from gas connection? (% respondents) Overall, well over half of respondents felt that men and women benefited equally. During FGDs, it was pointed out that men no longer had to cut firewood or get the LPG cylinders refilled, while women did not have to deal with cleaning up ashes, etc. Of the 27 women who responded, 14 (52%) felt women benefited more, and 10 (37%) felt benefits were equally shared, and 2 (7%) felt men benefited more. The fact that women are more likely to feel benefits accrue more to them than their menfolk, suggests that they are more familiar with the burden of making do without gas. Therefore, it is likely that women enjoy greater benefits and the project is positively biased in their favor. 4.2.1.12 Quality of Supply In addition to expenditure and usage patterns, quality and reliability were evaluated. Out of 50 connected HHs, 41 (82%) reported that gas pressure was sufficient for their needs, 5 (10%) reported that it was not, and 4 (8%) refused to answer. In order to ascertain the pressure levels, HHs were also asked about the color of the flame. Forty-five (90%) reported it was blue, 3 (6%) yellow, and the others blue or bluish-red. 4.2.1.13 Expenditure Patterns Respondents were asked to estimate their monthly expenditures on all energy sources before and after connecting. Because energy expenditures differ significantly between summer and winter, they were asked about both seasons. A marked difference was found between HHs who connected and those who did not. Connected HHs reported HH expenditures almost twice as high as unconnected HHs. This pattern supports the finding that affordability is a significant barrier for many households. Table 4. HHs self-reported fuel expenditures (GEL/month) Connected HHs Unconnected HHs Before connecting Current Current Summer months Winter months Summer months Winter months Summer months Winter months Liquid gas (cylinders) 43 28 3 2 32 16 Wood 1 268 1 150 12 278 Electricity 37 59 31 46 22 33 Natural gas 13 42 Season duration 9 months 3 months 9 months 3 months 9 months 3 months Total per season 725 655 418 528 599 981 Source: Author calculations based on HH survey Note: Assumption is that the summer season lasts 9 months and winter season 3 months in Poti and Senaki municipalities. Therefore, avg. total summer months expenditures multiplied by 9 and avg. winter months multiplied by 3. 19 4.2.1.14 Savings To estimate savings, self-reported HH energy expenditures before connecting were compared with expenditures after connecting (both summer and winter months).7 Total average savings per annum are estimated at 435 GEL ($263) per connected HH. The survey found that HHs which did not connect were poorer than those who connected (i.e. their monthly expenditures were significantly lower). This underlines the issue of connection affordability. As of now, poorer HHs are unable to connect, so that the benefits are reaped mainly by better-off HHs. HHs also make an investment in connecting to gas, which includes the connection cost. Each HH spends on average 500 GEL to connect. Thus a HH can recoup its investment in less than a year. The average installment cost comes to less than 25 GEL/month over two years (depending on the amount) for customers using the plan. While many had access to the plan in the past (42% of survey respondents reported using it), most unconnected HHs reported that it is no longer available (90%). Table 5. Total Savings (per HH) HH type Savings N Household type Total HH expenses (GEL/month) Savings (GEL/yr) Savings(USD/ yr) % savings 50 Connected 611 435 263 61% 24 - gas for heating 683 410 248 69% 26 - no gas for heating 547 458 277 57% 50 Non-connected 383 Source: Author calculations based on HH survey Total potential savings if all 17,773 HHs in Poti connected, would be at a minimum $6 million per year. However, achieving this seems unlikely given the barriers to connecting, as discussed below. On the other hand, if (and to the extent that) gas is subsidized by the GoG, total, economy-wide benefits would be offset by the subsidy’s share. Increases in gas consumption would result in increases in government outlays on subsidies. 4.2.1.15 Fuel Efficiency In addition to reported savings, which reveal how HHs adjust to different fuels and the impact on their welfare and HH budgets, it is also instructive to look at how the efficiency of various energy sources compare, by converting fuels into a common energy unit and then comparing the costs per unit. Figure 8 compares the cost of different fuels based on their calorific values. It shows the hypothetical expenditures of average connected HHs in Poti (consuming the reported average of 133 GEL on gas per year) if they would switch entirely from one type of fuel to gas. It can be seen that by switching from LPG to gas, HHs could save 81% on energy costs, or 61%, if they switched from electricity. Wood, on the other hand, because it is cheaper than gas, ends up raising the cost of energy. For Senaki, where the average HH reported expenditures on gas of 240 GEL, the savings would be even higher under such a scenario. 7 Assumptions were made that winter lasted 3 months, and summer 9 months, and annual expenditures were weighted accordingly. 20 Figure 8. Hypothetical expenditures (GEL/year) and savings (%) for a HH switching entirely to gas from other fuel Source: Author calculations based on company billing data; energy equivalents from http://www.allsubjects4you.com/Fuels.htm 4.2.1.16 Willingness to Pay Above, it was noted that the non-monetary benefits to switching (comfort, conservation, pollution) are difficult to quantify. Nonetheless, consumers’ willingness to pay was estimated by asking a hypothetical question: “If there were no change in your total energy expenses after you were connected to gas, would you still want to use and pay for gas?” In other words, consumers were asked if they valued being connected to gas so much that they would give up whatever monetary savings they experienced. All respondents reported they would have connected, even without any cost savings. The implication of this finding is that the non-monetary benefits are worth at least as much as the amount HHs are saving, i.e. equivalent to at least twice the level of average savings per HH ($263 per year), or $526 per year. 4.2.1.17 Barrier to Connection and Usage The reason many HHs do not connect relates to high connection costs. Among unconnected HHs, 90 % said they could not afford the connection fee. During FGDs, many participants noted that the installment plan was no longer being offered, and this was delaying their decision to connect or preventing them from connecting all together. A number of barriers exist. The first and most obvious is that a HH or business simply does not have access to a pipeline. This barrier disappears once a gas line is installed (by the distribution company) along the street where the HH is located. Second, connection costs are unaffordable for many people. Meters (now included, but not in the past) have been a potential added cost, and heating appliances can also cause HHs to postpone using gas for heating (even if they are connected and using gas for cooking). 4.2.1.18 Cost-Benefit Analysis Although the SOW did not require it, the cost-benefit analysis developed for the project (before implementation) was reviewed. The review showed that projections were based on a number of assumptions that were unrealistic:  3-6% population growth rates, leading to Poti population almost doubling by 2019. In fact, although figures are not available for Poti, Georgia’s population declined in 2013 (by 0.33%).  Most HHs would connect immediately to the network. In fact, after two years, only 10% have connected (although demand to connect is high).  Mean annual expenditures on gas are much lower. 21  HHs would make a complete switch from other fuels to gas. In fact the majority of HHs only use gas for cooking and hot water, and only some for heating.  No investment costs for HHs in new heating appliances were assumed, when in fact new appliances are a significant additional expenditure, costing up to 700 GEL.  Expenditures on diesel fuel for generators are $22.39 million and will rise to $99.82 million by 2019. However, no HHs and few businesses use generators anymore.  In 2013, total gas consumption was projected (by the three different scenarios) at between 120.66 m3 and 192.72 m3 (depending on scenarios). In fact consumption is unlikely to be more than 1 million m3. (To consume just 120 million m3, Poti would need the equivalent of 60 asphalt factories consuming 10,000 m3 per day, operating 200 days per year). This is related to the GoG’s (optimistic) expectations that FIZ would materialize and attract thousands of energy intensive businesses. • Leaving aside projections for industrial gas consumption (since Poti FIZ has not come on line) and focusing only population projections, annual consumption in Poti by HHs (and 6-7 small businesses) was 267,430 m3 in 2012. It may rise to about 1 million m3, but this is still just a fraction of the 43.26 to 46.66 million m3 (depending on the scenario), as projected by the cost-benefit analysis. In addition, some errors were found:  Population was estimated at 40,000 HHs when in fact the number is closer to 18,000.  In the Excel spreadsheet cost-benefit analysis, for Gas Consumption, clearly million cubic meters per day is incorrect, and per year was intended. Revising the cost-benefit analysis is beyond the scope of the evaluation. In light of the large uncertainties in gas consumption, it would also be a risky endeavor. However, certain observations can be made:  As noted, any future Poti industrial firms that rely on gas will consume gas at levels that would far outweigh HH consumption. Even a single company’s consumption could dwarf the consumption of all HHs and businesses combined.  The most conservative (and pessimistic) cost-benefit projections would exclude industrial production all together. Using SOCAR Georgia’s projections, residential and commercial customers would be consuming on average 894 m3 by year 10 (and even this seems optimistic, given lower than expected HH consumption to date). If there are 20,000 such customers 10 years from now, this would result in gas consumption levels in the region of 17.8 million in year 10. This amount is many times lower than the original cost-benefit analysis figure, which projected the Poti population consuming 74 million m3 per day by 2019. 4.2.2 Conclusions  Residential, commercial and industrial users recognize the economic advantage of switching to gas. The project has led to positive impacts for all end-users who have connected to gas – HHs, businesses, social/pubic institutions and industry, recognize the economic advantage of switching to gas (See Annex H , Section D).  GoG’s expectations that FIZ would materialize and attract thousands of energy intensive businesses has proved to be overoptimistic, with implications for the cost-benefit scenario. This could significantly reduce the benefits of the project.  Without reducing the barriers facing HHs in connecting to gas, the impacts of the project will remain (unnecessarily, it can be argued) limited in scope. At some point, the number of connections is likely to plateau, well below the saturation point of 17,773 HHs. It cannot be assumed that making gas accessible, whether it be to HHs, business or industry, means it will actually be used.  Lack of modern, efficient and safe gas appliances, as well as their relatively high cost, limits the penetration of residential gas use for purposes other than cooking.  Making reliable projections of costs and benefits in an environment of great uncertainty (where the presence of a few large industries could completely alter the order of magnitude of benefits) is nearly impossible. 22 4.2.3. Recommendations  Although reducing the residential connection cost is not in PGIP scope, the Evaluation Team recommends that spreading the same cost of the gas utility investment over the lifetime of the equipment would increase the number of residential users benefitting from gas, and directly boost the profits of GOGC/GGTC and municipal utilities. According to the HH survey, when asked the question: “Please tell us the reasons why you are not connected to the gas network?” 45 out of 50 customers gave the reason as “not able to cover the connection fee.” Therefore, we recommend changing the connection fee to a monthly fee. This will make the fee very affordable while at the same time generating a regular revenue stream for the utility. Currently, the connection fee is spread evenly over about only 1/3 of the invested equipment (pressure reducer, gas meter, etc.) lifecycle. The equipment estimated lifecycle is 20 years. In the Georgian context, the approximately 600 GEL of connection fee could be evenly spread over 20 years. The amount of monthly fee can be determined by the utility. But in principle, over the lifecycle of the equipment, the utility will not only return its investment but it could double it or even triple it, while customers will pay a very affordable monthly connection fee.  Encourage GoG to promote a policy of reducing residential connection barriers through re￾introducing financing programs. Consider introducing a small lifetime meter monthly flat fee instead of one time connection charge. This would increase the number of residential users benefiting from gas while further boosting the profits of GOGC/GGTC and municipal utilities.  More conservative cost-benefit analysis should be constructed, and thoroughly checked for validity with HH data and distribution company estimates to avoid making unrealistic or erroneous assumptions, and reduce errors. The likelihood of industrial production being established depends on unpredictable economic factors and political decisions. For these reasons, the unknown element of large industrial production in Poti makes projecting costs and benefits a guessing game.  Establishing a training program designed for deployment of licensed gas technicians would further help boosting both residential and commercial connections and use by adding more than a single in￾house connection – therefore allowing connection of more gas appliances in addition to cooking ranges such as heating and hot water appliances. Furthermore, private licensed gas technicians are expected to compete and reduce the cost of connection and therefore alleviate the financial barrier for a large number of potential users. 4.3 QUESTION 1.B: THE OVERALL QUALITY OF THE DESIGN OF THE SENAKI POWER TRANSMISSION PROJECT QUESTION 1.b- SUMMARY OF EVALUATION DESIGN AND METHODS Evaluation Question Type of Analysis Conducted Data Sources and Methods Used Type and Size Sample Limitations What is the overall quality of the design and procurement of the Senaki power transmission project prepared by the engineering oversight contractor as perceived by host country stakeholders (GSE, GOG, etc.)? GSE/SER and Tetra Tech structured questionnaire and interviews. Drawings, reports. Specifications, documents, drawings and field surveys. Maps and PLS-CADD evaluation. Tenders and Procurement specifications. All tower and foundation drawings. Maps and explanations. None. MoE one-to-one interview. Feedback and personal interviews. Two Deputies of the MoE. Time allocated for discussion. Field survey. Contractor’s staging area. Foundations staging inspection. Foundations site in works. Completed foundations site. Sub-station with foundations. Sufficient number of concrete foundations (12). 2 completed foundation sites, 1 foundation site in works. 8 foundations in sub-station. None. 23 4.3.1 Findings The transmission project is in process. Therefore, the Evaluation Team was able to assess only the available design quality and limited implementation as follows:  The original transmission power project design is based on a previous 1987 Soviet design of the destroyed transmission line, which was re-designed with cutting-edge design tools (PLS-CADD) and superior modern materials.  Original concrete foundation designs were improved to reflect the superior type of new towers8 . New AutoCAD drawing for the new improved design were generated and shared with the Evaluation Team. According to the information reviewed, the towers were ordered from a Ukrainian manufacturer who could not meet the required delivery. Subsequently the towers were ordered from a Turkish supplier who committed to meet the requirements. Instead of non-galvanized standard Russian steel profiles tower construction, the Turkish supplier will deliver the same specifications but in galvanized steel profiles, which provides a better material and finish.  The final beneficiaries of the PGIP transmission line are aware of the design process, including the improvements and approved them. All stages of design and progress are coordinated between SER, GSE, TT and USAID.  Since GSE does not have transmission work experience and in-depth knowledge in the newest software used by SER, they had difficulties understanding the PLS-CADD reports.9 It should be also mentioned that GSE (as an operator) is neither required nor expected to have transmission knowledge and expertise.  During interviews with GSE management in relationship with the transmission project, GSE restated their goals to boost energy reliability and availability through reconstruction of an East-West 220 kV transmission line as a back-up of the 500 kV Imereti line from Enguri to Zestaphoni. The goal is going to be achieved by the present project design.  Interviews with SER, who is responsible for all components, indicated that all materials procured for the project were approved by TT and Power Engineers and are in compliance with specifications and budget. 4.3.2 Conclusions  Overall, the design quality implies a superior transmission line, which fully complies with GSE requirements to boost the energy reliability by re-constructing the East-West 220kV Senaki 1, 2 line as a back-up to the 500 kV backbone transmission line.  Introduction of PLS-CADD design and reporting tool provides a boost in the technical capacity building for future transmission lines. However, GSE, which does not have transmission work experience and in-depth knowledge in the newest software used by SER had difficulties understanding the PLS-CADD reports. • The higher quality material used for the towers is likely to minimize the maintenance costs while improving the operations. • The procurement process was smooth and all materials were of the required standards and specifications. 4.3.3 Recommendations  An introductory training seminar for GSE to explain and allow them to understand the advantages of working with and reading reports generated by PLS-CADD would allow the executive decision￾makers to take full advantage of this tool. The knowledge of PLS-CADD design and its simulation capabilities could have an important future use for modeling various transmission alternatives and therefore such training could be very important. 8 Mentioned earlier - Annex E. 9 Reported to the Evaluation Team by SER during interview. 24 4.4 QUESTION 1.C: THE OVERALL PROCESS OF DESIGNING THE TRANSMISSION PROJECT QUESTION 1.c - SUMMARY OF EVALUATION DESIGN AND METHODS Evaluation Question Type of Analysis Conducted Data Sources and Methods Used Type and Size Sample Limitations What is the overall process of designing the transmission project? (Evaluators should review progress reports and communications between the oversight contractor, GSE, USAID and other stakeholders) Interviews with GSE and Tetra Tech Interviews with SER. Interview with Ministry of Energy Review of documents and replies from interviewees Number of participants from Tetra Tech, GSE, SER and deputies minister None Field visits and evaluation of approved components Inspection and review of approved items Available stock and field sites visited None Field reports, measurements, drawings and communication procedures Kevin Franklin, Tetra Tech, data bank for drawings, reports, etc. Spot check 1-2 drawings, reports, field measurements Access limited to Tetra Tech office, time restrains 4.4.1 Findings After defining the objective of reconstructing the transmission line, USAID and TT subcontracted Power Engineers, a US company, for a preliminary design and oversight. Consequently, for PGIP transmission component, TT and Power Engineers undertook the transmission design in PLS-CADD. The design process included a number of steps, described below: Preliminary design by Tetra Tech/Power included the following steps:  Step I – Finalized the project design criteria based on field observations and studies of the data.  Step II – Led the process of designing the project’s technical specifications.  Step III - Completed design checks, constructability reviews and all quality control checks.  Step IV - Verified and completed the logistic issues (right of way technical issues, towers positions, etc.)  Step V - Conducted assessment of local capabilities to supply foundations and towers.  Step VI – Drafted the project construction planning and implementation plans and prepared the tender documents The tender was prepared on PLS-CADD and required the winning contractor to continue shop design in PLS-CAD. The contract was won by SER. Upon winning the transmission contract, SER’s role related to design only could be summarized as follows:  Acquired PLS-CADD software as part of the tender requirement in order to redesign and submit drawings and final design to TT for approval. Since all the design by PLS-CADD is electronic, drawings, bill of materials, and other digital data were transmitted by e-mail to TT.  TT uploaded the digital data into a special server to which Power Engineers had access.  Power Engineers, using PLS-CADD, checked the design made by SER in PLS-CADD.  Power Engineers either made comments and required modifications or retuned the approved design to the server.  Letters of NO OBJECTION were issued by TT. The Evaluation Team reviewed TT’s computerized database, design, communication process, reports, field measurements, etc., and found that TT, as the oversight contractor, was instrumental in maintaining 25 a steady and good quality flow of information and communication with and between all stakeholders. Furthermore, TT facilitated decision-making processes and contributed to improved working relationships between the stakeholders on the transmission component. SER and GSE executives, as well as the Ministry of Energy’ deputies that the team interviewed, uniformly confirmed that TT was highly instrumental in maintaining the flow of information, and pressuring for feedback when the communication temporarily broke down. 4.4.2 Conclusions • TT was highly instrumental in facilitating communication between all the stakeholders, pushing hard for answers, and allowing the project to move ahead. Both SER and GSE confirmed to the Evaluation Team the superior assistance they received from TT. • Introduction of PLS-CADD is an important technical capacity building towards future design of transmission projects. 4.4.3 Recommendations • Even though it is difficult to compare the GOGC contracts of gas pipeline design-build and SER hybrid contract for power transmission, the team found better schedule and budget results with GOGC. Therefore a complete design-build approach similar to GOGC approach appears to be smoother to implement, simplifies communication, and creates more project ownership from the contractor’s perspective. • While there is an important and measurable benefit with introduction of PLS-CAD as a modern design tool, it appears to the Evaluation Team that SER was practically obliged to acquire and use PLS-CADD in order to comply with the contract’s requirements, even though the company could design the transmission the same way they did for the previous projects. We recommend avoiding such situations in the future unless the capacity building of using a modern design tool is separated from the actual project design. It is recommended that such new tool and capacity building should be introduced before project start rather than in parallel with the project. 4.5 TO WHAT DEGREE DID USAID’S INVESTMENT IN CAPE AND DGA PROMOTE A SAFER, MORE RELIABLE GEORGIAN POWER GRID? QUESTION - SUMMARY OF EVALUATION DESIGN AND METHODS Evaluation Question Type of Analysis Conducted Data Sources and Methods Used Type and Size Sample Limitations To what degree did USAID’s investments in sub-procurements, including CAPE and DGA, promote or have the potential to promote a safer, more efficient, reliable and robust Georgian power grid? Structured interview with GSE and Tetra Tech Feedback to questions Number of participants in interviews None Site visits and discussions with operators Explanations from operators and simulations 2-3 simulations to visualize results None Manufacturers’ websites and catalogs Catalogs and field evaluation (DGA) in sub-station 4 DGA units installed. Display of field results Nonfunctioning DGA at time of field trip. None – after system activation at GSE HQ 4.5.1 Findings • The Evaluation Team found that the CAPE software was installed and in full operation and use since 2011. The system provides a fast, efficient and reliable method for relay setting calculations to timely react to seasonal changes of supply and demand, as well as current conditions of generation units and 26 the network. The above findings were derived from various documents delivered to the team, as well as from direct interviews with GSE and the system operators. • DGAs were originally installed on 11 critical transformers to verify concentration of dissolved gas, which affect the cooling oil and maintain steady operation of power transformers. Excessive concentration of dissolved gas components seriously endangers the transformer and requires maintenance intervention. DGA is providing remote, as well as local warnings of such dangerous situations. During the field visit, the Evaluation Team was told that all installed DGAs were subject to replacement. However, when the Evaluation Team visited the main terminal for DGA in GSE headquarters after two weeks, some of them had already been replaced. Unfortunately, the main terminal could not provide indication of how many DGAs were operational, and at the time of the visit, GSE could not provide such information either. TT informed the Evaluation Team that the DGA manufacturer is replacing all the faulty units. 4.5.2 Conclusions • CAPE software has a direct effect on fast and more accurate calculation and change of relay settings, which otherwise would take longer and might affect users getting electricity. Time saving, much more work efficiency, and improved safety are the typical characteristics of CAPE. While difficult to quantify CAPE’s contribution of making Georgia’s grid more robust and safer, we believe that such contribution exists. • Even though DGA was not fully operational, the team took notice of its operation in GSE HQ. Warning of dissolved gas components, which affect few transformers’ operation were received and corrective actions were initiated by GSE maintenance team. Prevention of transformer failures and maintaining their normal operation significantly contributes to the grid reliability and robustness. • Both CAPE and DGA are important procurements that independently and separately contribute to the Georgian grid safety and increased reliability. 4.5.3 Recommendations • The smart grid sub-procurements supplied by USAID are unique stand-alone systems, operating independently in the Georgian grid, where most of the new switches, electronics, controls, and some other smart grid components are delivered by Siemens. It is recommended requesting from the manufacturers of DGA and CAPE technical documentation and comparisons to emphasize the components’ abilities in operating smoothly with various suppliers such as Siemens in order to generate a safer and more reliable grid. With such documentation, USAID would be able to promote the same systems (CAPE and DGA) in other countries where similar power projects are planned. • The team recommends as a standard operating procedure the installation of DGA in all critical transformers in substations, especially in those places where there is no back-up transformer. 4.6 QUESTION 2: SUSTAINABILITY QUESTION 2 - SUMMARY OF EVALUATION DESIGN AND METHODS Evaluation Question Type of Analysis Conducted Data Sources and Methods Used Type and Size Sample Limitations The evaluation contractor should comment on the sustainability of the infrastructure built with USAID support. GOGC/GGTC and GSE interviews Feedback from stakeholders Executive participants None Field trips to existing structures and newly built structures Old and new structures comparisons Random checks None Interviews with donors and lenders Feedback. From EBRD, KFW Single interviewees None 27 4.6.1 Findings Gas Pipeline. Built with best modern materials as well as highest engineering standards, the pipeline sustainability and lifetime becomes a function of its regular maintenance. Without doubt, the Cathodic Protection would extend the lifetime of the underground pipeline but it cannot replace maintenance of above the ground pipe and aerial river crossing where rust damage was observed in spite of new coating. This further indicates that an active maintenance program implementation is utmost required. The Evaluation Team checked the bill of materials used for the project and concluded that the quality of the materials procured and delivered met the requirements. Further verification was conducted in the field to evaluate the maintenance and projected sustainability. The Evaluation Team surveyed visually and compared the maintenance of the old Russian pipeline and the new USAID-financed pipeline completed recently. We took notice of the lack of maintenance on both pipelines (missing supports and rust deterioration on the old pipeline as well as rust on the new pipeline). The photos below illustrate the poor maintenance practice on the old and the new pipelines. Old pipeline New pipeline before Poti distribution point Transmission. Since the project was not yet completed, the Evaluation Team was able to see only the future transmission’s towers foundations. However, the study and evaluation of design drawings indicated that superior materials will be used. The team was positively impressed by the quality of foundation studs and nuts, as well as the expected quality of the galvanized steel towers. All the above contribute to sustainability and longer life cycle and are likely to require limited active maintenance. The Evaluation Team surveyed older transmission lines during the field trip and took notice and documented the limited maintenance which leads to deterioration of towers and foundations. The photo on the right illustrates the observed situation. It must be emphasized that even though we found documents for maintenance standards in GSE, we certainly did not find supporting evidence that maintenance is performed at the needed level. 4.6.2 Conclusions  While there are maintenance manuals, the Evaluation Team did not find supporting evidence in the field that maintenance is actually performed. An active maintenance program would certainly contribute to both pipeline and transmission systems’ sustainability and extended life time.  At this stage, it is difficult to assess the sustainability of the transmission line since it is a project in its early stages. 28  The pipeline project various segments demonstrate ruggedness and functionality. However, without an active maintenance/repair program, as well as monitoring of small visible deteriorations and damages, the sustainability and lifecycle of the pipeline will be shortened. 4.6.3 Recommendations  The team recommends introduction of an active monitoring plan of both gas pipeline and transmission for deterioration (such as rust), for pipeline supports as well as tightening the towers’ fixtures and so on. Based on the period of monitoring – for example 6 month span between checks – a detailed and relevant maintenance plan, including implementation schedule should be developed and enforced to extend the systems sustainability as well as their life cycle.  It is further recommended that USAID infrastructure projects would include a specific requirement from the beneficiary to prepare an active maintenance and monitoring program. Such a requirement is likely to generate ownership of the infrastructure through maintenance while building a long-term sustainability.  We recommend the oversight contractor to require specific maintenance programs, including schedule, monitoring and reporting. 4.7 QUESTION 2.A: THE EXTENT THE OPERATIONS AND MAINTENANCE OF GEORGIAN GAS PIPELINE AND POWER TRANSMISSION SYSTEMS IMPROVED SUMMARY OF EVALUATION DESIGN AND METHODS Evaluation Questions Type of Analysis Conducted Data Sources and Methods Used Type and Size of Sample Limitations To what extent the operations and maintenance of Georgian gas pipeline and power transmission systems improved and what was PGIP’s possible contribution toward such improvement? a. Is it possible to compare a baseline situation before the project to mid-term and expected end of the project situation? b. How reliable is gas and electricity supply compared with pre￾project period? c. Were the privatization steps and progress achieved? Why and  GSE and GOGC past and current documentation and/or business processes associated with monitoring, maintaining and repairing power and gas infrastructure.  Tetra Tech Reports  GOG Reports  Gas and electricity supply and consumption statistics.  M&O Statistics  M&O manuals of GOGC and GSE  Document Review  Key Informant Interviews with GOG, GSE, GOGC, Tetra Tech and USAID  Analysis of targeted results, outputs, and outcomes  Verification of PGIP reporting  Quantitative and Qualitative Analysis  On-site observations  Critical synthesis and triangulation analysis of opinion data  Statistical data (supply, consumption, service, M&O analysis  The load of documents  The shortage of time  Ability to read voluminous Russian and Georgian documents in a short time 29 what can be done better? d. Was the energy security fully achieved? 4.7.1 Findings • Introduction of new elements such as Cathodic Protection, pigging, pipeline drying and HDD have a measurable contribution to ease maintenance and increase the lifetime of pipeline. • At the time of the mid-term evaluation the transmission component of PGIP was in early stages of construction. Consequently, no findings could be found about this component’s operation and maintenance or contribution of PGIP to them. • The Evaluation Team found from the pipeline component, that GOGC and GGTC have well established operation procedures. PGIP has not affected, improved or contributed measurably to the existing operations of the pipeline except by adding the new section Senaki-Poti. The new section of the pipeline, parallel to existing one, is not in operation yet and could not influence the operations on existing pipeline. Furthermore, the pipeline component constructed and being constructed under PGIP, is a small part of the overall Georgian gas network, which has been in operation for many years. • The Evaluation Team confirmed that GOGC has maintenance instructions; however, the field trip indicated an obvious lack of preventive maintenance on the exposed sections of both old and new pipeline segments. Similar lack of preventive maintenance was observed on older transmission lines. • Baseline comparison of PGIP between pre-project and mid-term evaluation, based on TT documents as well as field surveys, indicate the followings: - Pipeline component demonstrates a major progress and completion of the goals and expectations. It can be stated with confidence that there is compliance and results. - The transmission component is moving ahead at slower pace due to issues mentioned earlier. While progress is made, it is too early at this stage to mention more than reasonable progress. The operational improvements would be better assessed and confirmed after the construction completion. • At the time of mid-term evaluation, there is no impact on electricity supply directly connected to the PGIP transmission component but there is reduction of power disruptions and increase of reliability due to smart grid improvements. • The settings of GSE and GOGC as government-owned monopolistic companies do not leave room for privatization of neither pipeline nor transmission. No steps or initiatives for privatization were noticed nor do we believe that such steps are feasible in the short-term. However, works related to the pipeline and transmission components were subcontracted to private companies. • The energy security was strengthened with the gas pipeline and will be strengthened further as more segments of it under PGIP are completed, tested, and put in operation. No assessment can be made for the transmission components since the project is still in construction.  GOGC provided the Evaluation Team with important data on few old segments of the pipeline that require urgent repair or reconstruction since they have exceeded their lifecycle. (The expected lifetime of a pipeline is approximately 33-35 years). 4.7.2 Conclusions • Under PGIP, the scope of operations of GGTC has increased by adding a new section to their existing pipeline. Cathodic Protection, pigging and pipeline drying capacity at the new section of the pipeline add to improvement of operation and maintenance of the pipeline. The final evaluation of the project should verify to what extent these technologies will be used. • Smart grid improvements - through sub-procurements of EECS, DGA and CAPE - have achieved the objective of improving the operations of the power transmission grid by making equipment outages 30 and system disruptions less damaging (EECS), operation flexibility faster (CAPE), and potentially reducing transformer maintenance problems (DGA). • The existing operation instruction of GSE and of GOGC/GGTC should not be changed since the systems (electricity grid + transmission and pipeline) function is satisfactory. • In terms of maintenance, neither GOGC/GGTC nor GSE have a good track record on preventive maintenance; therefore, there is an urgent need for a measurable improvement in maintenance scope and schedule. • The transmission component, when completed, will have limited or no effect on electricity supply while it will have impact on the reliability since it would allow an alternative in two ways transmission to the 500 kV line. The effect of new Senaki 1, 2 transmission lines will be measurable through GSE dispatch records on operations of the network under Imereti line outage conditions. • Few segments of the older pipeline Saguramo-Kutaisi need urgent repairs or reconstruction since the pipeline has exceeded its lifecycle. The situation is further aggravated by lack of consistent preventive maintenance. 4.7.3 Recommendations  In order for PGIP to have a future impact on Georgia’s transmission and pipeline components, and beyond these projects, it is recommended working with all government organizations to require development and observance of relevant, active, preventive maintenance programs.  It is further recommended that all future USAID infrastructure development projects have a compulsory section requiring the beneficiary to deliver a sound maintenance program.  The systems like EECS and DGA might be recommended for other countries in South Caucasus – following comment of GSE of them being more technologically advanced than others.  The final evaluation of the project should confirm and document, with help of GSE, whether there are measurable dispatch records on actual contribution of Senaki 1, 2 transmission line.  In order to enhance further the energy security, it is recommended that repairs or reconstructions of few segments of the old pipeline should be undertaken by GOGC. Combining the repairs and reconstruction with an aggressive preventive maintenance would further enhance the lifecycle and the energy security. 4.8 CROSS CUTTING ISSUE: DID THE TRAINING AND CAPACITY BUILDING ACHIEVE ITS GOALS? SUMMARY OF EVALUATION DESIGN AND METHODS Evaluation Question Type of Analysis Conducted Data Sources and Methods Used Type and Size Sample Limitations a. Was capacity building segmented by topic (technical, managerial)? b. Is there a baseline of skills before the program started and how was the program tailored to meet the needs? c. How many trainees participated in the capacity building program? d. Was tests administered to trainees to assess the skills improvements? Key Informant Interviews Document Review Implementers reports List of participants Training materials USAID Training participants Evaluations of capacity building activities Quantitative and Qualitative Analysis of reports and results of interviews and FGDs. Critical synthesis and triangulation of data collected from various sources Lack of material (training manuals) and evaluations to review 31 4.8.1 Findings Capacity building is considered as one of the benefits of PGIP. It is to be provided by TT to GOGC and GSE as part of the PGIP’s SOW. The method of capacity building is not specified. The areas of focus are left up to TT, who is to review existing evaluations and plan accordingly. At the technical level, capacity building has been conducted on the transmission component through a learning by doing (or ‘direct contact’) approach as part of the project implementation. According to TT, 4 persons have benefited from this training. The provision of on-the-job training by TT for transmission project, but not for GOGC, appears related to the availability of expertise available, and perceived capacity of the companies rather than technical evaluation. For some of the new software products/systems the product suppliers provided technical training as part of the purchase. According to TT, 7 people were trained for DGA and 8 people for CAPE. PLS-CADD training has not been provided. Such training was considered during SER’s design phase but it was later set aside as a conflict of interest because it violated the arm’s length relationship between oversight contractor and construction contractor. Nonetheless, as noted elsewhere, DGA was implemented with flaws and there were no as-built drawings, raising questions about the adequacy of the training. At the time of the mid-term evaluation, an evaluation of GSE management practice at the upper and mid￾level management had been conducted by HICD PLUS, a USAID partner organization. The recommendations in the evaluation report are being used to build a capacity building program. The same process of evaluation and capacity building is now being applied to GOGC. The Evaluation Team notes that although TT cooperated with the team and provided comments, it is not responsible for the cross￾cutting management capacity building project. 4.8.2 Conclusions  Capacity building goals were not clearly stated and therefore were difficult to assess.  There is significant scope for strengthening capacity building activities. Challenges in evaluating this element resulted from lack of training evaluations and training manuals. Furthermore, limited adequate technical feedback, the lack of drawing standards applied, and limited assistance with field measurements provided, suggests that more can be done in this area.  Based on observations, it appears that the oversight contractor lacked sufficient local engineering expertise in the field of civil, mechanical, electronics and gas engineering required to best support the relevant project components. Reliance on remote engineering support substantially diminished the oversight contractor’s ability to provide best technical and engineering support coupled with an effective and sustainable capacity building. The Evaluation Team learned that TT’s initial proposal had 3 long-term experts (one gas expert, one electric expert, and one editor). All were cut at the start of contract negotiations. 4.8.3 Recommendations  As the oversight contractor is responsible for the technical aspects of the project, as well as project management, it would be desirable if capacity building focused on these specific fields.  Improve the quality of field and office technical support.  Provide training manuals and adequate installation instructions for Component 1, Electricity.  Document assessments and provide training agendas, and manuals 5.0LESSONS LEARNED • Accurate assessment of the local technical and managerial capabilities is essential in determining the best working methodology for future infrastructure project designs. Specifically, such assessment needs to indicate whether the host country has skilled, experienced engineers and contractors 32 required for the planned infrastructure project covering all disciplines – civil, mechanical, electrical, electronic, etc. • In the case that the host country lacks some of the skills and experience mentioned above, high calibre foreign experts should be made available locally to support, enhance and boost the local capacity to reach the required competences and deliver a design-build infrastructure project. Building a local technical and managerial capacity for undertaking design-build projects is an important step of each country towards future self-sustainability. This would require not only training but also providing them with a high enough salary that will retain the skilled and qualified persons in the utility and in Georgia. • Complex requirements built into tenders are a serious impediment for small private companies who do not have the ability and resources to dedicate to complex procurement procedures, and therefore they do not participate. The possibility of dividing a large project into smaller projects with simpler requirements would enhance the ability of a bulk of the private sector small and medium size companies to participate in international tenders. • Cost-benefit analysis of a project should be performed before the project is approved and started. Care should be taken when developing cost-benefit analysis to adequately verify the input data, assumptions, and accuracy of calculations to avoid making unrealistic projects and inflating expectations. When cost benefit analysis depends on the presence of large unknowns (direction of local economic development, decisions by large industrial consumers to build or not build) its value is considerably lowered. • It cannot be assumed that an infrastructure project such as PGIP will have immediate widespread benefits for end-users (HHs, businesses, industry) without ensuring that measures are in place to promote the use of said infrastructure. The investment costs borne by USAID or GoG are insufficient to achieve the economic benefits envisioned by the project – additional investment is needed by distribution companies and end-users. While a large company such as SOCAR faces few barriers to investing, low-income HHs need some type of support or incentives to invest. • New technologies like PLS-CADD should be considered as a component in local capacity building. In order to achieve the best results, a preferred option would be to keep those separate and fully implement prior to the project, rather than introducing in parallel with the implementation. • In terms of oversight, the company providing this service should have on staff qualified personnel. Oversight contractors should have a specific task, and demonstrate availability of staff, including back￾up personnel in case of vacations or sick leaves. Ideally the oversight contractor should have all required support technical personnel in office to provide engineering and field assistance to contractors rather than relying on remote engineering support. • Future technical training should cover the following areas: - Standards and compliance – specifically on drawings measurements uniformity (in millimeters including relevant tolerances). - Field measurements using advanced laser device for verification of dimensions accuracy. - Drawing evolutions and revisions reflecting field progress towards as-built drawings. - Training at relevant levels in software application including manuals and local language operation instructions. 33 ANNEXES 34 ANNEX A: STATEMENT OF WORK 35 STATEMENT OF WORK I. Summary Name of the Project (to be evaluated): Power and Gas Infrastructure Project (PGIP) Project Number: multiple contracts under Assistance Agreement AAG-114-G-10-00001 Project Dates: May 10, 2010 to September 30, 2014 Project Funding: $115,000,000 Implementing organizations: Georgia Oil & Gas Corporation (GOGC), and Tetra Tech EM, Inc. COR/ACOR: Sukru Bogut / Nick Okreshidze. The evaluation contractor must provide non-personal services for a mid-term evaluation of the Power and Gas Infrastructure Project (PGIP) during the first phase, which ended on November 1, 2012. The evaluation must include the PGIP activities from May 10, 2010 to May 2013 and must include the project execution phase and core assistance areas. This evaluation will provide lessons learned to USAID for future programming of infrastructure and oversight services. Summary of Specific Technical Requirements The evaluation contractor shall ensure that the evaluation team will complete the following tasks and provide the following deliverables within the terms defined by the contract: - Telecon with USAID/Mission to discuss the upcoming work - Detailed evaluation design and workplan to be submitted to the Task Order COR prior to the team’s visit to Georgia - Incoming briefing with USAID management to present the detailed evaluation design - Field work to conduct the evaluation of Power and Gas Infrastructure Project in accordance with the USAID-approved evaluation design - Outgoing briefing with USAID management to present the preliminary findings of the evaluation - Provide evaluation report to USAID in accordance with reporting guidelines - Submit USAID-approved evaluation report to DEC within 90 calendar days following the acceptance of the report by the COR. II. Background The development of Georgia’s energy sector is of strategic importance for the country’s emergence as a strong democratic nation with a vibrant economy. During Georgia’s slide into poverty after the collapse of the Soviet Union, nothing hurt the population more than the loss of reliable energy. However, various measures taken by the government with the assistance of the donor community have brought important improvements to the sector. To date, USAID’s assistance has contributed significantly to the restructuring of Georgia’s energy sector; key legal and policy reforms have been enacted; a critical part of the sector has been privatized; and several successful demonstration projects for management reforms in critical energy infrastructure and energy efficiency programs have been completed. The diversification of Georgia’s gas imports and efforts to make the country self-sustainable in terms of electricity supply have also had a strong positive impact. 36 Although much progress has been made, Georgia's power and gas infrastructure has not fully recovered from the devastation caused by the ravages of civil war of 90s, lack of regular maintenance, and scant investment in physical infrastructure. The involvement of USAID in power and natural gas transmission infrastructure fits squarely within the U.S. Government’s expressed interests in seeing an energy-secure, stable, and economically prosperous Georgia. As per the Assistance Agreement between the United States of America and Georgia, signed on February 25, 2010, USAID/Georgia is implementing a portfolio of approximately $115 Million in energy infrastructure projects in Georgia. The portfolio includes: 1) main gas pipeline construction through several fixed price contracts with the Georgian Oil and Gas Corporation (GOGC), a government-owned commercial organization charged with the import and transit of natural gas; 2) high voltage power transmission line construction through direct contracts issued by USAID with private commercial organizations; 3) smart grid improvements through procurements under the Engineering Oversight Task Order that include Computer Aided Protection Engineering (CAPE) software, Dissolved Gas Analyzers, and Enhanced Emergency Control Systems; and other projects as proper and subject to available funds in the Assistance Agreement. The goal of the assistance is to promote energy security through greater access to electricity and natural gas supplies for households and businesses in Western Georgia, promote the development of the Poti Free Industrial Zone (FIZ) on the Black Sea, and secure power exports through reliable transmission infrastructure improvements domestically. Gas Pipeline Component Construction of 148.8 km of 700 mm diameter 54 bar-Maximum Allowable Operating Pressure (783 psi) gas pipeline: This activity has already brought natural gas to the city of Poti, with a population of about 48,000. Poti did not have natural gas before the USAID-funded gas pipeline was completed. The other four sub-projects replace old gas pipelines, and increase the volume of gas that can be moved to western Georgia. Design and construction contracts are undertaken by GOGC through a host-country contracting mechanism, and contracts under GOGC are approximately $59 million. At the end of the program, by the Spring of 2014, the GOGC will have approximately 149 km of new 700 millimeter diameter pipeline with 54 bars of Maximum Allowable Operating Pressure. The piping will have a surface protective coating, electrochemical corrosion-protection, with a cathodic protection system. It will be able to accept pipeline in-line inspection tools allowing an extended period of operation and maintenance. • Under contract (106.0 km) • Poti – Senaki, 30 km; commissioned and accepted December 2011. • Senaki – Abasha 29 km; to be commissioned and accepted by June 2013 SOL-114-13-000005 • Abasha - Kutaisi 47 km; to be commissioned and accepted by December 2013 • In consideration (42.8 km) • Kutaisi – Zestaponi 23.1 km; to be commissioned and accepted by Summer 2014 • Kareli – Gori 19.7 km; to be commissioned and accepted by Spring 2014 a) Liakhvi – Ptsa I 4.6 km Section (replace old 500 mm with new 700 mm) b) Liakhvi – Ptsa II 6.6 km Section (replace old 700 mm with new 700 mm) c) Sveneti – Liakkhvi 8.5 km Power Transmission Component Computer Aided Protection Engineering (CAPE) Software for Relay and Fuse Settings: Under 37 PGIP, USAID provided this software to the Georgian State Electrosystems (GSE), the transmission system operator of Georgia. The computer program enables faster and more reliable settings of relays on the transmission system. The benefits to the people of Georgia are more reliable energy transmission systems and increased regional trade to importers of electricity from Georgia’s renewable hydro power. The software became operational in the fall of 2011. (Approximate cost: $98,000; implementer Tetra Tech) Reconstruction of 58.8 km of Double Circuited 220 kV Senaki 1 and 2 Transmission Lines and Associated Bays at the Menji and Tskaltubo Substations: The scope of work for this project includes all work required to rebuild the Senaki 1 & 2 220kV transmission line between the Tskaltubo Substation and the Menji Substation. The scope of work also includes the work to rebuild the associated termination bays at the Menji and Tskaltubo Substations. This 58.8 km (36.5 mile) line was originally commissioned in 1987. Over the course of several years of conflict, the line was knocked out of service, and most of the original parts have been stolen or salvaged. There were nine towers remaining of the original 211 towers. Although some towers and foundations remain, the Contractor was instructed to assume that all facilities would be replaced. No drawings of the original transmission line and bays remain. The intent is to rebuild the line to its original configuration with updates as needed to meet current local codes and industry standards. IEC and EN, or equivalent GOST standards will be used to supplement local design standards. This double-circuit line is a key part of the transmission system in western Georgia; without this line, the loss of the parallel 500 kV transmission line from western to eastern Georgia causes the underlying 220 kV and 110 kV systems to be overloaded, to collapse, and thus cause a regional blackout in eastern Georgia. The ground breaking ceremony occurred on November 2012; the work is to be commissioned and accepted at the end of 2013. (Approximate cost $18 million; implementer SakEnergoRemonti ) Enhanced Emergency Control System (EECS) for Transmission System: This smart grid project envisions the purchase and installation of advance system protection (that is, relays, field units, and telecommunications equipment) which monitors the state of the GSE transmission system and takes rapid action to maintain the balance between load and frequency. The reaction time (from event to remedial action) of the system is less than fifty milliseconds (that is, 2 ½ cycles at 50 Hertz), which is much faster than the load-frequency control system. The system includes monitoring equipment (phasor measurement units or phasors or synchrophasors) at 18 substations in Georgia. The system also includes monitoring equipment in the Borchka 400 kV and the Muratli 150 kV Substations in Turkey and in the Samux 500 kV Substation in Azerbaijan. This EECS supports the Black Sea Transmission System (BSTN) and supports interconnection agreements among Azerbaijan, Georgia and Turkey. The system is to be in service by the end of September 2013. (Approximate cost $1.6 million; implementer Tetra Tech) Dissolved Gas Analyzers (DGA): This activity covers the purchase and installation of on-line analyzers (field units), which measure the concentrations (expressed in parts per million) of nine specific gases that are dissolved in the oil of GSE’s large power transformers on the 500 kV, 220 kV, and 110 kV transmission systems. The amount of the dissolved gas and their relative ratios indicate the health of the transformer. Rapid detection by t he DGA of the build-up of the harmful gases in the transformer enables GSE to take rapid action to avoid the transformer failing or, to at least minimize the amount of damage to the transformer when it does fail. A typical size transformer is 500 MVA and the failure of such a transformer would affect about 38 100,000 customers. The DGA is to be commissioned, and will be operational by the end of April 2013. (Approximate cost $670,000; implementer Tetra Tech) Engineering Oversight Component This PGIP engineering oversight component provides resident professional engineering and other technical services to support power and gas transmission improvements being undertaken by USAID on behalf of the Government of Georgia (GOG). Tetra Tech is the implementing partner of the PGIP Oversight component that runs from May 2010 through September 2014, with two task orders (AID-114-TO-10-00003, which ended on November 1, 2012 and AID-114- TO-13-00001, which ends in September 2014) with a total budget of $12 Million. Tetra Tech subcontractors include Exp (formerly Trow), Power Engineers, from the U.S., and local subcontractors Gergili LLC, Basiani 93, etc. Under these task orders, Tetra Tech also provides smart grid procurements and design services under the Power Transmission Component (see above.) for the Senaki 1 and 2 twin chain 220 kV transmission project. This technical assistance spans the full range of expert engineering advice and oversight, organizational capacity building expertise, and the provision of analytical and technical support to USAID/Georgia. The oversight contractor provides full construction management services and engineering oversight for those sub-projects that have been awarded to third parties for construction and rehabilitation of power transmission infrastructure and power system upgrades. Major responsibilities of the oversight contractor are as follows: Power transmission oversight: The oversight contractor provides technical assistance, construction management services, and procurement and construction supervision services for subprojects approved for rehabilitation and construction. The oversight contractor examines the policy, regulatory, institutional, financial, commercial, and legislative environments governing activities in the Georgia power transmission sector, especially from the perspective of the preparation of bidding documents for procurement of equipment and material and the oversight of construction activities to ensure compliance with local law. The oversight contractor examines available design, reports, and other documents relating to the Senaki 1 and 2 power lines and the Menji and Tskhaltubo substations, and they provide procurement services for the dissolved gas analyzers and enhanced emergency control systems. The oversight Contractor is utilized for the supervision and oversight of these activities. The oversight contractor also completes site visits, as appropriate, and prepares appraisal reports. Gas transit oversight: The oversight contractor advises the GOGC on engineering services, design, and construction management services. The oversight contractor provides oversight services for subprojects approved for rehabilitation and construction. The oversight contractor examines the policy, regulatory, institutional, financial, commercial, and legislative environments governing activities in the Georgian gas transmission sector, especially from the perspective of the review of bidding documents for construction and design, and the oversight of procurement and construction activities to ensure compliance with local law. The oversight contractor examines available studies, reports, and other documents relating to the pipeline projects to be implemented. The oversight contractor completes site visits, as appropriate, and prepares or assists in the preparation of appraisal reports. Capacity building and management: The oversight contractor also provides direct capacity building assistance to GOGC and GSE to ensure the achievement of the objectives of USAID assistance to these organizations. Such assistance may include capacity building, strategic planning, organizational structure and performance, engineering capability, environmental 39 compliance and monitoring, procurement capability, transmission analysis and forecasting, leastcost planning, electrical grid and gas pipeline network optimization, and operational efficiency. The oversight contractor reviews existing evaluations of each organization, and if needed, conducts additional evaluations for each organization for potential capacity assistance needs, and proposes interventions to USAID for approval. Procurement and Design: The oversight contractor provides advisory assistance to GSE in its efforts to prepare engineering designs, plans and cost estimates for any project that is designated by the COR to ensure that GSE complies with appropriate national and international standards that are best applicable to these specific projects and reflect best engineering practices. The oversight contractor analyzes and evaluates GSE’s final designs, drawings, specifications, schedules, cost estimates, and lists of equipment requirements, and provides “No Objection” opinions to USAID once the GSE design is ready to be implemented. The oversight contractor prepares tender documents in accordance with the GSE design for USAID to procure required equipment and materials. 40 ANNEX B: WORK PLAN, EVALUATION DESIGN & METHODOLOGY 41 WORK PLAN & EVALUATION DESIGN In response to: Mid-Term Performance Evaluation of the USAID/Georgia Power and Gas Infrastructure Project (PGIP) July 4, 2013 Submitted by: Mendez England & Associates 4300 Montgomery Ave., Suite 103 Bethesda, MD 20814-4413 42 TABLE OF CONTENTS A. INTRODUCTION ......................................................................................... 1 B. BACKGROUND ............................................................................................ 1 C. EVALUATION TECHNICAL APPROACH ............................................... 2 C.1 EVALUATION GOALS AND PURPOSE ............................................................... 2 C.2 EVALUATION OBJECTIVES ..................................................................................... 2 C.3 PRIMARY COMPONENTS ....................................................................................... 2 C.4 EVALUATION QUESTIONS .................................................................................... 3 C.5 OVERVIEW OF EVALUATION METHODOLOGY ........................................... 4 C.6 TEAM COMPOSITION .............................................................................................. 5 C.7 EVALUATION DESIGN ............................................................................................. 5 C.8 EVALUATION PHASES ............................................................................................. 9 C.9 EVALUATION LIMITATIONS ................................................................................. 9 ANNEXES Annex 1: Statement of Work Annex 2: Preliminary Evaluation Schedule Annex 3: List of Documents Reviewed and Received Annex 4: Evaluation Matrix & Sources of Information to Respond To The Evaluation Questions Annex 5: Stakeholders List 43 Annex 6: Questionnaires Annex 7: Draft Report Outline 1 A. INTRODUCTION PGIP is a portfolio of $115,000,000 in energy infrastructure projects that include: 1) main gas pipeline; 2) 200 kV high-voltage power transmission line construction; and 3) smart grid improvements. PGIP’s main goal is to promote energy security through greater access to electricity and natural gas supplies for households and businesses in Western Georgia. The project is unique because: 1) it is funded from “one-time” supplemental post-conflict resources; 2) it is the largest USAID-funded infrastructure project in Georgia; and 3) it utilizes an innovative mix of both private sector and host country-controlled organizations as implementers. The ME&A evaluation team understands the importance of PGIP and its role in Georgia’s energy and economic development. Based on this understanding, we have developed the following evaluation methodology and design. B. BACKGROUND The development of Georgia’s energy sector is of strategic importance for the country’s emergence as a strong democratic nation with a vibrant economy. During Georgia’s slide into poverty after the collapse of the Soviet Union, nothing hurt the population more than the loss of reliable energy. However, various measures taken by the government with the assistance of the donor community have brought important improvements to the sector. To date, USAID’s assistance has contributed significantly to the restructuring of Georgia’s energy sector; key legal and policy reforms have been enacted; a critical part of the sector has been privatized; and several successful demonstration projects for management reforms in critical energy infrastructure and energy efficiency programs have been completed. The diversification of Georgia’s gas imports and efforts to make the country self-sustainable in terms of electricity supply have also had a strong positive impact. Although much progress has been made, Georgia's power and gas infrastructure has not fully recovered from the devastation caused by the ravages of civil war in the 90s, lack of regular maintenance, and scant investment in physical infrastructure. The involvement of USAID in power and natural gas transmission infrastructure fits squarely within the U.S. Government’s expressed interests in seeing an energy-secure, stable, and economically prosperous Georgia. As per the Assistance Agreement between the United States of America and Georgia, signed on February 25, 2010, USAID/Georgia is implementing a portfolio of approximately $115 Million in energy infrastructure projects in Georgia. The portfolio includes: 1) main gas pipeline construction through several fixed price contracts with the Georgian Oil and Gas Corporation (GOGC), a government￾owned commercial organization charged with the import and transit of natural gas; 2) high voltage 2 power transmission line construction through direct contracts issued by USAID with private commercial organizations; 3) smart grid improvements through procurements under the Engineering Oversight Task Order that include Computer Aided Protection Engineering (CAPE) software, Dissolved Gas Analyzers, and Enhanced Emergency Control Systems; and other projects as proper and subject to available funds in the Assistance Agreement. The goal of the assistance is to promote energy security through greater access to electricity and natural gas supplies for households and businesses in Western Georgia, promote the development of the Poti Free Industrial Zone (FIZ) on the Black Sea, and secure power exports through reliable transmission infrastructure improvements domestically. C. EVALUATION TECHNICAL APPROACH C.1 EVALUATION GOALS AND PURPOSE C.1.1. GOALS The goal of the evaluation is to assess the overall PGIP performance results in promoting energy security through greater access to electricity and natural gas supplies for households and businesses in Western Georgia, promote the development of the Poti Free Industrial Zone (FIZ) on the Black Sea, and secure power exports through reliable transmission infrastructure improvements domestically. C.1.2. PURPOSE The purpose of this contract is to perform the mid-term performance evaluation of the Power and Gas Infrastructure Project (PGIP). C.2 EVALUATION OBJECTIVES ‐ Assess and evaluate PGIP contributions to Georgia’s strategic energy sector on the country’s development – gas and electricity sectors. ‐ Evaluate the specific PGIP related components: 1. Main gas pipeline construction through several fixed price contracts with the Georgian Oil and Gas Corporation (GOGC), a government-owned commercial organization charged with the import and transit of natural gas; 2. High voltage power transmission line construction through direct contracts issued by USAID with private commercial organizations; 3. Smart grid improvements through procurements under the Engineering Oversight Task Order that include Computer Aided Protection Engineering (CAPE) software, Dissolved Gas Analyzers, and Enhanced Emergency Control Systems. C.3 PRIMARY COMPONENTS C.3.1. GAS PIPELINE AND ELECTRICITY TRANSMISSION COMPONENTS EVALUATION The evaluation is organized by the two main components: gas pipe and electricity. For each component, the team will analyze: 3  Concept Design  Specifications (related to international standards used and consistency)  Rationale for contractors’ selection  Procurement  Value and Compliance  Schedule and Implementation  Engineering Oversight  Capacity Building  Environmental  Sustainability and Operation Cost  Economic Benefits  Project Management  Lessons Learned C.3.2 OVERSIGHT AND PROCUREMENT COMPONENT EVALUATION Furthermore, the evaluation will also incorporate Tetra Tech’s oversight and training assistance where the team will analyze the followings:  Oversight with engineering design and implementation of gas and electricity components  Oversight with procurement, training and capacity building C.4 EVALUATION QUESTIONS The evaluation team will research and provide answers to USAID questions below. Annex 4 contains a detailed Evaluation Matrix with the questions and sub-questions. Evaluation Question 1: PGIP Goals Were the PGIP goals accomplished? These goals include; providing gas supplies to western Georgia; and increasing the reliability of Georgia’s power grid. 1.1 What primary and secondary economic benefits, both current and projected, over the next ten years, can be associated with gas pipeline construction under PGIP? Measures of such benefits include but are not limited to: the number of households and the number of businesses receiving access to gas services (both current and projected), and the amount of increased business activity and job-creation (disaggregated by sex.) Appropriate measures shall also be made for environmental considerations. The findings will be informed by qualitative data (e.g. key informant interviews) with GOGC officials, local gas distribution companies, energy sector experts, USAID, and host government entities. The desk review and analysis of quantitative information, statistical and monitoring data and various economic projections for Georgia will also be necessary. 1.2 What is the overall quality of the design of the Senaki power transmission project prepared by the engineering oversight contractor as perceived by host country stakeholders (GSE, GOG, etc.)? (The PGIP oversight contractor was originally tasked with the design duty and was responsible for producing a construction contract for the 4 implementation. However, the oversight contractor used a hybrid approach of a construction and a design-build contract, for which USAID issued a task order modification of the oversight contract. USAID is interested in learning about the implications of this change on the overall quality of the design, if any.) 1.3 What is the overall process of designing the transmission project? Evaluators should review progress reports and communications between the oversight contractor, GSE, USAID and other stakeholders. 1.4 To what degree did USAID’s investments in sub-procurements, including CAPE and DGA, promote or have the potential to promote a safer, more efficient, reliable and robust Georgian power grid? This question can be answered by interviewing stakeholders (GSE, GOG, Tetra Tech, USAID etc.), and/or reviewing various analyses and projections prepared by GSE (if available) regarding the number of power outages or transformer failures prevented, money saved, etc., due to the adoption of CAPE and DGA systems. Evaluation Question 2: Sustainability of the Infrastructure built with USAID Support The evaluation contractor must comment on the sustainability of the infrastructure built with USAID support. Under sustainability we imply host country’s operations and maintenance capacity (GSE/GOGC) and the ability/readiness to maintain and further invest in infrastructure improvements. To what extent the operations and maintenance of Georgian gas pipeline and power transmission systems improved and what was PGIP’s possible contribution toward such improvement? This will be informed by the review of companies’ past and current documentation and/or business processes associated with monitoring, maintaining and repairing power and gas infrastructure. Evaluation Question 3: Lessons Learned What are the lessons learned from this mid-term evaluation of the PGIP that can inform future infrastructure program designs, procurements, executions, schedule and oversight services that utilize both private sector and host country controlled organizations? C.5 OVERVIEW OF EVALUATION METHODOLOGY The evaluation methodology is designed to assess PGIP’s progress and provide lessons learned for other USAID missions in designing and implementing current and new programs in energy infrastructure construction and oversight services. In addition, the evaluation will determine whether: 1) PGIP has achieved its goals; and 2) the infrastructure built with USAID support is sustainable. Specifically, the evaluation will assess: 1) economic benefits that can be associated with gas pipeline construction under PGIP; 2) the overall quality of the design of the Senaki power transmission project; 3) the overall process of designing the transmission project; 4) the extent the operations and maintenance of Georgian gas pipeline and power transmission systems improved, as well as the contribution that PGIP (if any) made towards such improvement; and 5) financial/economic as well as O&M (Operations and Maintenance) sustainability. 5 The evaluation will distinguish between two principal evaluation types: 1) performance evaluation; and 2) beneficiary assessment. Performance evaluation. Performance evaluations focus on descriptive and normative questions: what a particular project or program has achieved (either at an intermediate point in execution or at the conclusion of an implementation period); how it is being implemented; how it is perceived and valued; whether expected results are occurring; and other questions that are pertinent to program design, management and operational decision making. Beneficiary assessment. The beneficiary assessment will use quantitative and qualitative methods (see description under Primary Data collection, below) to assess socio-economic benefits accruing to residential and non-residential users in the region. The units of analysis are household and industrial/commercial consumers in the project region, i.e. end-users expected to experience a material change resulting from the project. In the absence of baseline data, comparisons are made between households which are connected and those who are not connected (by choice) to gas, to better understand the relative benefits delivered by the project. A detailed matrix covering the relevant institutions/stakeholders by type, level of engagement during implementation, influence on and interest in the outcomes, and other indicators will be created. The preliminary list of stakeholders is in Annex 5. After a review of documents, criteria relating to their specific type of engagement, number of similar stakeholders, etc. will be used to decide on who will be interviewed and how many interviews will be conducted. We will ensure that the list of interviewees covers all relevant project dimensions and is representative of the stakeholders and institutions involved. C.6 TEAM COMPOSITION The team will be divided into two groups who, while supporting each other, will each focus on separate evaluation components. It will also allow us to increase the number of interviews conducted. Group 1 – Performance Evaluation Peter Tal, Team Leader Murman Margvelashvili, Georgian Energy Specialist Group 2 – Beneficiary Assessment Nils Junge, Evaluation Specialist Giorgi Gorgadze, Georgian Data Analyst and Survey Specialist (IRMS) C.7 EVALUATION DESIGN Given the type of PGIP project activities, both qualitative and quantitative methods will be used. A mixed-method evaluation systematically integrates evaluation methods, and includes data collection techniques such as structured key informant interviews, household surveys, and secondary data. The work for the qualitative evaluation will be primarily to conduct open-ended interviews with those organizations and individuals, as well as other stakeholders and partners, involved in the different 6 activities under the project. In addition, we will conduct a comprehensive review of information and reports pertaining to PGIP project since 2010. This information will be analyzed and the results will be tailored to answer the main evaluation questions outlined in the Scope of Work (SOW), as well as to provide lessons learned for future energy and infrastructure projects. Data will be collected by using a number of methods, some of which are mentioned below:  A critical desk-top review of materials related to PGIP, as well as any material that will be provided by USAID such as project reports and annual work plans, project performance management plan, project design, communications among partners, USAID and GOG, etc.  Interviews with USAID and PGIP’s implementers, including Tetra Tech, Georgia Oil and Gas Corporation (GOCG), and SakEnergoRemonti.  In depth, semi-structured interviews with selected program beneficiaries and stakeholders such as the Ministry of Energy, the Ministry of Economy, the Ministry of Environment, the Georgia State Electrosystems (GSE), EnergoPRO Georgia, SakEnergoRemonti, SOCAR Georgia, ADB, EBRD, KfW, EIB, etc. We believe that because PGIP is a complex project, interviewees may be intimidated by structured interviews and formal interviewing techniques. Accordingly, we will use semi-structured interviews, a more appropriate and valuable technique, because they will allow beneficiaries to present and explain points freely.  Focus Groups Discussions (FGDs): FGDs with household members in the project region to obtain qualitative data to provide context and background on the data obtained through the individual interviews. This FGD guideline will inform the household questionnaire to ensure that there is consistency across participants and locations.  Key informant interviews with businesses/industrial consumers  Site visits to Poti and areas along the Senaki-Poti line.  Direct observation to cross-check information (e.g. comparing statements to observed practice) and identification of factors not previously recognized Quantitative evaluation will consist of analyzing:  Data sourced from performance reports and evaluations of implementing partners;  Secondary data on Geostat reports and GOCG, GSE and Tetra Tech analyses and projections; Integrated Household Survey data; mini-household survey results;  Program outputs against objectives and performance indicators; and  Mini-survey of households. To conduct interviews, we will use a questionnaire (see Annex 6 for illustrative questions). All questionnaires will be developed and refined after the team completes the review of PGIP reports and materials and pilots them in the field. The draft questionnaire will be shared with USAID for comment. Raw data will be recorded on the individual questionnaires used for the interviews. Digital tape recorders will also be used, when appropriate and with participants’ permission, to record the conversations at various meetings. To facilitate the analysis, the responses to the questions will be 7 compiled and tabulated on a spreadsheet that facilitates the comparison of responses. This will help define response patterns and determine the similarity of the different responses. The final format will be decided after discussions with USAID. The Evaluation Design Matrix for this evaluation is presented in Annex 4. The matrix is organized by the two main components: gas pipe and electricity. For each component, the team will analyze:  Concept Design  Specifications (related to international standards used and consistency)  Rationale for contractors’ selection  Procurement  Value and Compliance  Schedule and Implementation  Engineering Oversight  Capacity Building  Environmental  Sustainability and Operation Cost  Economic Benefits  Project Management  Lessons Learned Primary data collection Collecting direct information from household and commercial/industrial consumers is necessary for assessing the project’s economic benefits. While the focus is on obtaining quantifiable information, because of the nature of the project (transmission lines rather than distribution networks, incomplete status, delays in development of Poti FIZ) we expect the results to be indicative, rather than authoritative. Commercial/industrial consumers’ survey (relating to gas) Approximately 10-20 key informant interviews will be conducted with commercial/industrial consumers in Poti (including Poti FIZ) with a sample of commercial/industrial consumers that use natural gas as an input. Because companies vary greatly by size, and obtaining interviews is more difficult than with households, they will not be selected randomly, nor will data collected and aggregated. Instead, a qualitative approach will be used: in-depth interviews will be conducted to gain an improved understanding of the importance and impact of the new gas infrastructure on their business. The following question topics will be explored during the KIIs:  Impact of connecting to gas on productivity, costs, expansion,  Relative importance of  Supply reliability (interruptions) 8  Impacts (actual and potential) of gas  Supply  Service quality  Billing service  Tariffs  Future needs – electricity, gas, related to air conditioning and heating Households To obtain data from households in the region, a mini, face-to-face survey will be conducted. The survey will have 25-30 questions and the sample size will be approximately 100 households, selected randomly from Poti (and possibly one or more of the settlements of Abasha, Samtredia, and Senaki, depending if any changes in gas access/supply were noted by FGD participants). Interviews will be conducted by trained interviewers. Data will be entered, cleaned and analyzed. All four settlements mentioned have substantial numbers of households both with and without gas. In Poti, data indicates that 1,795 HHs were connected as of June 1, while approximately 9,000 were unconnected. In the other settlements a much larger share of households are connected, but several thousand remain unconnected, even though they have access to the network. This an issue which will be explored further. Connected households will be compared with unconnected households to understand the impact of gas. Given the limited time and budget constraints for the evaluation, as well the fact that no impact (noticeable to households) from the electricity component is expected, the sample of 100 households was chosen as a reasonable number. The sample will allow for statistical analysis of two subgroups – connected and non-connected households concerning their fuel costs. The key, measurable economic impact for households is expected to be reduced energy expenditures as households switchto gas from less economical fuels. Data collection will be conducted by ME&A local subcontractor, IRMS. To develop the questionnaire, two FGDs will be held. The purpose of the FGD is to twofold. First, it will improve our understanding of the welfare impact of connecting to gas. The FGDs will be used as background for analysis of the project’s actual and potential impacts. Second, it will be used inform the mini-household survey questionnaire, covering broadly the same topics. FGDs will be conducted in the local language. We will solicit a signed Informed Consent before starting any FGDs and will ensure participants of their anonymity. Topics covered by the household survey (and focus group discussions) will include:  Service quality  Payment  Billing service  Consumption (monthly) by heating & cooling, lighting, cooking hot water heating,  Supply reliability (interruptions) 9  Tariffs  Overall perceptions of value for money, abilities to pay bills, actual bill payments  Connection costs  Potential gender impacts All surveys will be developed in English, and after being approved by USAID, will be translated into Georgian. The survey will clearly assure respondents that their responses will be anonymous to encourage frank and open answers. Primary data collection will be completed by July 27 and data analysis will completed by August 2. C.8 EVALUATION PHASES The evaluation is divided into the following phases I. Preparation a) Literature/document review b) Work plan preparation II. Exploratory research c) Consultation with stakeholders d) Document collection and review III. Primary data collection e) Development, refinement and piloting of research instruments (FGDs, survey) f) Conducting primary research IV. Analysis g) Data analysis from FGDs and households h) Synthesis of findings V. Report writing i) First draft of report j) Review by USAID k) Revisions to report C.9 EVALUATION LIMITATIONS There are several limitations inherent to the design of this evaluation. These are outlined below, together with measures to address them. 1. Selection Bias: As some key informants may decline to be interviewed, there is a possibility of selection bias, i.e. those respondents who choose to be interviewed might differ from those who do not in terms of their attitudes and perceptions, affiliation with government/non-government structures, and socio-demographic characteristics and experience. 10 2. Recall Bias: Since a number of questions raised during the interviews will deal with issues that took place in the past, recall bias cannot be excluded. As PGIP project activities were launched in May 2010, some respondents may find it difficult to accurately compare situations before and after the project. 3. Halo Bias: There is a known tendency among respondents to under-report socially undesirable answers and alter their responses to approximate what they perceive as the social norm (halo bias). The extent to which respondents will be prepared to reveal their true opinions may also vary for some questions that call upon the respondents to assess the performance of their colleagues or people on whom they depend upon for the provision of services. To mitigate this limitation, ME&A will provide the respondents with confidentiality and anonymity guarantees, where possible; conduct the interviews in the settings where respondents feel comfortable; and establish rapport between the interviewer and the respondent. 4. Period that will be Evaluated: This is a mid-term evaluation; therefore, it might be too early to determine the concrete results of the PGIP project. A number of PGIP’s key initiatives might still be at too early a stage to expect any meaningful progress toward important performance indicators. 5. Timing of the evaluation: The evaluation will take place in July, a period when many people in Georgia leave for vacation. We will take this into account when agreeing upon the timing of the trip, through careful advance planning/scheduling, and through telephone interviews if necessary. 6. Resources Allocated for the Evaluation: The time and resources allocated for the evaluation will not allow for a comprehensive household survey. Therefore, the evaluation team will have to rely on data provided by the implementers, including utility billing records and data, if available ANNEXES ANNEX 1: STATEMENT OF WORK STATEMENT OF WORK I. Summary Name of the Project (to be evaluated): Power and Gas Infrastructure Project (PGIP) Project Number: multiple contracts under Assistance Agreement AAG-114-G-10-00001 Project Dates: May 10, 2010 to September 30, 2014 Project Funding: $115,000,000 Implementing organizations: Georgia Oil & Gas Corporation (GOGC), and Tetra Tech EM, Inc. COR/ACOR: Sukru Bogut / Nick Okreshidze The evaluation contractor must provide non-personal services for a mid-term evaluation of the Power and Gas Infrastructure Project (PGIP) during the first phase, which ended on November 1, 2012. The evaluation must include the PGIP activities from May 10, 2010 to May 2013 and must include the project execution phase and core assistance areas. This evaluation will provide lessons learned to USAID for future programming of infrastructure and oversight services. Summary of Specific Technical Requirements The evaluation contractor shall ensure that the evaluation team will complete the following tasks and provide the following deliverables within the terms defined by the contract: - Telecon with USAID/Mission to discuss the upcoming work - Detailed evaluation design and workplan to be submitted to the Task Order COR prior to the team’s visit to Georgia - Incoming briefing with USAID management to present the detailed evaluation design - Field work to conduct the evaluation of Power and Gas Infrastructure Project in accordance with the USAID-approved evaluation design - Outgoing briefing with USAID management to present the preliminary findings of the evaluation - Provide evaluation report to USAID in accordance with reporting guidelines - Submit USAID-approved evaluation report to DEC within 90 calendar days following the acceptance of the report by the COR. II. Background The development of Georgia’s energy sector is of strategic importance for the country’s emergence as a strong democratic nation with a vibrant economy. During Georgia’s slide into poverty after the collapse of the Soviet Union, nothing hurt the population more than the loss of reliable energy. However, various measures taken by the government with the assistance of the donor community have brought important improvements to the sector. To date, USAID’s assistance has contributed significantly to the restructuring of Georgia’s energy sector; key legal and policy reforms have been enacted; a critical part of the sector has been privatized; and several successful demonstration projects for management reforms in critical energy infrastructure and energy efficiency programs have been completed. The diversification of Georgia’s gas imports and efforts to make the country self-sustainable in terms of electricity supply have also had a strong positive impact. Although much progress has been made, Georgia's power and gas infrastructure has not fully recovered from the devastation caused by the ravages of civil war of 90s, lack of regular maintenance, and scant investment in physical infrastructure. The involvement of USAID in power and natural gas transmission infrastructure fits squarely within the U.S. Government’s expressed interests in seeing an energy-secure, stable, and economically prosperous Georgia. As per the Assistance Agreement between the United States of America and Georgia, signed on February 25, 2010, USAID/Georgia is implementing a portfolio of approximately $ 115 Million in energy infrastructure projects in Georgia. The portfolio includes: 1) main gas pipeline construction through several fixed price contracts with the Georgian Oil and Gas Corporation (GOGC), a government-owned commercial organization charged with the import and transit of natural gas; 2) high voltage power transmission line construction through direct contracts issued by USAID with private commercial organizations; 3) smart grid improvements through procurements under the Engineering Oversight Task Order that include Computer Aided Protection Engineering (CAPE) software, Dissolved Gas Analyzers, and Enhanced Emergency Control Systems; and other projects as proper and subject to available funds in the Assistance Agreement. The goal of the assistance is to promote energy security through greater access to electricity and natural gas supplies for households and businesses in Western Georgia, promote the development of the Poti Free Industrial Zone (FIZ) on the Black Sea, and secure power exports through reliable transmission infrastructure improvements domestically. Gas Pipeline Component Construction of 148.8 km of 700 mm diameter 54 bar-Maximum Allowable Operating Pressure (783 psi) gas pipeline: This activity has already brought natural gas to the city of Poti, with a population of about 48,000. Poti did not have natural gas before the USAID-funded gas pipeline was completed. The other four sub-projects replace old gas pipelines, and increase the volume of gas that can be moved to western Georgia. Design and construction contracts are undertaken by GOGC through a host-country contracting mechanism, and contracts under GOGC are approximately $59 million. At the end of the program, by the Spring of 2014, the GOGC will have approximately 149 km of new 700 millimeter diameter pipeline with 54 bars of Maximum Allowable Operating Pressure. The piping will have a surface protective coating, electrochemical corrosion-protection, with a cathodic protection system. It will be able to accept pipeline in-line inspection tools allowing an extended period of operation and maintenance. • Under contract (106.0 km) • Poti – Senaki, 30 km; commissioned and accepted December 2011. • Senaki – Abasha 29 km; to be commissioned and accepted by June 2013 • Abasha - Kutaisi 47 km; to be commissioned and accepted by December 2013 • In consideration (42.8 km) • Kutaisi – Zestaponi 23.1 km; to be commissioned and accepted by Summer 2014 • Kareli – Gori 19.7 km; to be commissioned and accepted by Spring 2014 a) Liakhvi – Ptsa I 4.6 km Section (replace old 500 mm with new 700 mm) b) Liakhvi – Ptsa II 6.6 km Section (replace old 700 mm with new 700 mm) c) Sveneti – Liakkhvi 8.5 km Power Transmission Component Computer Aided Protection Engineering (CAPE) Software for Relay and Fuse Settings: Under PGIP, USAID provided this software to the Georgian State Electrosystems (GSE), the transmission system operator of Georgia. The computer program enables faster and more reliable settings of relays on the transmission system. The benefits to the people of Georgia are more reliable energy transmission systems and increased regional trade to importers of electricity from Georgia’s renewable hydro power. The software became operational in the fall of 2011. (Approximate cost: $98,000; implementer Tetra Tech) Reconstruction of 58.8 km of Double Circuited 220 kV Senaki 1 and 2 Transmission Lines and Associated Bays at the Menji and Tskaltubo Substations: The scope of work for this project includes all work required to rebuild the Senaki 1 & 2 220kV transmission line between the Tskaltubo Substation and the Menji Substation. The scope of work also includes the work to rebuild the associated termination bays at the Menji and Tskaltubo Substations. This 58.8 km (36.5 mile) line was originally commissioned in 1987. Over the course of several years of conflict, the line was knocked out of service, and most of the original parts have been stolen or salvaged. There were nine towers remaining of the original 211 towers. Although some towers and foundations remain, the Contractor was instructed to assume that all facilities would be replaced. No drawings of the original transmission line and bays remain. The intent is to rebuild the line to its original configuration with updates as needed to meet current local codes and industry standards. IEC and EN, or equivalent GOST standards will be used to supplement local design standards. This double-circuit line is a key part of the transmission system in western Georgia; without this line, the loss of the parallel 500 kV transmission line from western to eastern Georgia causes the underlying 220 kV and 110 kV systems to be overloaded, to collapse, and thus cause a regional blackout in eastern Georgia. The ground breaking ceremony occurred on November 2012; the work is to be commissioned and accepted at the end of 2013. (Approximate cost $18 million; implementer SakEnergoRemonti ) Enhanced Emergency Control System (EECS) for Transmission System: This smart grid project envisions the purchase and installation of advance system protection (that is, relays, field units, and telecommunications equipment) which monitors the state of the GSE transmission system and takes rapid action to maintain the balance between load and frequency. The reaction time (from event to remedial action) of the system is less than fifty milliseconds (that is, 2 ½ cycles at 50 Hertz), which is much faster than the load-frequency control system. The system includes monitoring equipment (phasor measurement units or phasors or synchrophasors) at 18 substations in Georgia. The system also includes monitoring equipment in the Borchka 400 kV and the Muratli 150 kV Substations in Turkey and in the Samux 500 kV Substation in Azerbaijan. This EECS supports the Black Sea Transmission System (BSTN) and supports interconnection agreements among Azerbaijan, Georgia and Turkey. The system is to be in service by the end of September 2013. (Approximate cost $1.6 million; implementer Tetra Tech) Dissolved Gas Analyzers (DGA): This activity covers the purchase and installation of on-line analyzers (field units), which measure the concentrations (expressed in parts per million) of nine specific gases that are dissolved in the oil of GSE’s large power transformers on the 500 kV, 220 kV, and 110 kV transmission systems. The amount of the dissolved gas and their relative ratios indicate the health of the transformer. Rapid detection by the DGA of the build-up of the harmful gases in the transformer enables GSE to take rapid action to avoid the transformer failing or, to at least minimize the amount of damage to the transformer when it does fail. A typical size transformer is 500 MVA and the failure of such a transformer would affect about 100,000 customers. The DGA is to be commissioned, and will be operational by the end of April 2013. (Approximate cost $670,000; implementer Tetra Tech) Engineering Oversight Component This PGIP engineering oversight component provides resident professional engineering and other technical services to support power and gas transmission improvements being undertaken by USAID on behalf of the Government of Georgia (GOG). Tetra Tech is the implementing partner of the PGIP Oversight component that runs from May 2010 through September 2014, with two task orders (AID-114-TO-10-00003, which ended on November 1, 2012 and AID-114- TO-13- 00001, which ends in September 2014) with a total budget of $12 Million. Tetra Tech subcontractors include Exp (formerly Trow), Power Engineers, from the U.S., and local subcontractors Gergili LLC, Basiani 93, etc. Under these task orders, Tetra Tech also providessmart grid procurements and design services under the Power Transmission Component (see above.) for the Senaki 1 and 2 twin chain 220 kV transmission project. This technical assistance spans the full range of expert engineering advice and oversight, organizational capacity building expertise, and the provision of analytical and technical support to USAID/Georgia. The oversight contractor provides full construction management services and engineering oversight for those sub-projects that have been awarded to third parties for construction and rehabilitation of power transmission infrastructure and power system upgrades. Major responsibilities of the oversight contractor are as follows: Power transmission oversight: The oversight contractor provides technical assistance, construction management services, and procurement and construction supervision services for subprojects approved for rehabilitation and construction. The oversight contractor examines the policy, regulatory, institutional, financial, commercial, and legislative environments governing activities in the Georgia power transmission sector, especially from the perspective of the preparation of bidding documents for procurement of equipment and material and the oversight of construction activities to ensure compliance with local law. The oversight contractor examines available design, reports, and other documents relating to the Senaki 1 and 2 power lines and the Menji and Tskhaltubo substations, and they provide procurement services for the dissolved gas analyzers and enhanced emergency control systems. The oversight Contractor is utilized for the supervision and oversight of these activities. The oversight contractor also completes site visits, as appropriate, and prepares appraisal reports. Gas transit oversight: The oversight contractor advises the GOGC on engineering services, design, and construction management services. The oversight contractor provides oversight services for subprojects approved for rehabilitation and construction. The oversight contractor examines the policy, regulatory, institutional, financial, commercial, and legislative environments governing activities in the Georgian gas transmission sector, especially from the perspective of the review of bidding documents for construction and design, and the oversight of procurement and construction activities to ensure compliance with local law. The oversight contractor examines available studies, reports, and other documents relating to the pipeline projects to be implemented. The oversight contractor completes site visits, as appropriate, and prepares or assists in the preparation of appraisal reports. Capacity building and management: The oversight contractor also provides direct capacity building assistance to GOGC and GSE to ensure the achievement of the objectives of USAID assistance to these organizations. Such assistance may include capacity building, strategic planning, organizational structure and performance, engineering capability, environmental compliance and monitoring, procurement capability, transmission analysis and forecasting, least- cost planning, electrical grid and gas pipeline network optimization, and operational efficiency. The oversight contractor reviews existing evaluations of each organization, and if needed, conducts additional evaluations for each organization for potential capacity assistance needs, and proposes interventions to USAID for approval. Procurement and Design: The oversight contractor provides advisory assistance to GSE in its efforts to prepare engineering designs, plans and cost estimates for any project that is designated by the COR to ensure that GSE complies with appropriate national and international standards that are best applicable to these specific projects and reflect best engineering practices. The oversight contractor analyzes and evaluates GSE’s final designs, drawings, specifications, schedules, cost estimates, and lists of equipment requirements, and provides “No Objection” opinions to USAID once the GSE design is ready to be implemented. The oversight contractor prepares tender documents in accordance with the GSE design for USAID to procure required equipment and materials. III. Purpose of the Evaluation and Its Intended use The results of this evaluation will be used to evaluate the PGIP’s progress, and provide lessons learned for other USAID missions in designing and implementing current and new programs in energy infrastructure construction and oversight services. Funded from “one-time” supplemental post-conflict resources and being the largest USAID-funded infrastructure project in Georgia, PGIP is very different from our traditional assistance programs in the country. Further, PGIP utilizes an innovative mix of both private sector and host country-controlled organizations as implementers, which makes the project unique. Therefore, we expect that this evaluation will highlight important lessons learned not only for the Mission but for the Agency as a whole. Lessons learned from the evaluation must focus on sustainability considerations of large-scale infrastructure programs, implementation modality and other important aspects as specified below. IV. Evaluation Questions and Methodology Key evaluation questions are: 1. Were the PGIP goals accomplished? These goals include: providing gas supplies to western Georgia; and increasing the reliability of Georgia’s power grid. Specific sub-questions are as follows: a. What primary and secondary economic benefits, both current and projected, over the next ten years, can be associated with gas pipeline construction under PGIP? Measures of such benefits include but are not limited to: the number of households and the number of businesses receiving access to gas services (both current and projected), and the amount of increased business activity and job-creation (disaggregated by sex.) Appropriate measures shall also be made for environmental considerations. The findings will be informed by qualitative data (e.g. key informant interviews) with GOGC officials, local gas distribution companies, energy sector experts, USAID, and host government entities. The desk review and analysis of quantitative information, statistical and monitoring data and various economic projections for Georgia will also be necessary. b. What is the overall quality of the design of the Senaki power transmission project prepared by the engineering oversight contractor as perceived by host country stakeholders (GSE, GOG, etc.)? (The PGIP oversight contractor was originally tasked with the design duty and was responsible for producing a construction contract for the implementation. However, the oversight contractor used a hybrid approach of a construction and a design￾build contract, for which USAID issued a task order modification of the oversight contract. USAID is interested in learning about the implications of this change on the overall quality of the design, if any.) c. What is the overall process of designing the transmission project? Evaluators should review progress reports and communications between the oversight contractor, GSE, USAID and other stakeholders. d. To what degree did USAID’s investments in sub-procurements, including CAPE and DGA, promote or have the potential to promote a safer, more efficient, reliable and robust Georgian power grid? This question can be answered by interviewing stakeholders (GSE, GOG, Tetra Tech, USAID etc.), and/or reviewing various analyses and projections prepared by GSE (if available) regarding the number of power outages or transformer failures prevented, money saved, etc., due to the adoption of CAPE and DGA systems. 2. The evaluation contractor must comment on the sustainability of the infrastructure built with USAID support. Under sustainability we imply host country’s operations and maintenance capacity (GSE/GOGC) and the ability/readiness to maintain and further invest in infrastructure improvements. To what extent the operations and maintenance of Georgian gas pipeline and power transmission systems improved and what was PGIP’s possible contribution toward such improvement? This will be informed by the review of companies’ past and current documentation and/or business processes associated with monitoring, maintaining and repairing power and gas infrastructure. 3. What are the lessons learned from this mid-term evaluation of PGIP that can inform future infrastructure program designs, procurements, executions and oversight services that will utilize both private sector and host country controlled organizations? Answers to the questions above may be obtained through a combination of several data collection methods. The evaluation contractor must suggest the best methods that minimize bias and provide strong evidence. The evaluation contractor must suggest the use of various data collection and analysis methods, both quantitative and qualitative, including document review, key informant interviews, focus group discussions, survey instruments, and others. The evaluation contractor must justify their inclusion of any data collection methodology as well as their selection process for all methodologies. For example, for a survey or mini-survey (if proposed), the number of respondents and their selection process should be explained and justified. The same is true for key informants, focus group discussions, and other methods as well. Selected respondents should be representative of women, youth, and vulnerable groups, where appropriate. The evaluation contractor must develop a detailed evaluation design, including a data collection plan and data collection tools. The evaluation design must explain how the evaluation contractor intends to conduct the study in detail, including a detailed description of one or more proposed methodologies as well as limitations of proposed methodologies. The proposed research design must explain in detail what methods will be used to obtain answers for each evaluation question. The evaluation contractor must explain in detail how the proposed methodology (mix of methods) to conduct the study generate evidence to ensure rigor and reliability of results; and how and why the proposed methodology will minimize bias. The evaluation design must include a detailed evaluation matrix (the illustrative matrix is given below). The design must also include the data analysis plan for each question, draft questionnaires (to be included as an attachment), and other data collection instruments or their main features, criteria for assessing responses to evaluation questions, known limitations to the evaluation design, and a dissemination plan. The evaluation design must also include specific sub-questions for each evaluation question, where needed. The evaluation matrix below is only illustrative. The evaluation contractor must suggest the best methods that would generate the most reliable and evidence-based answers to the key evaluation questions. Illustrative evaluation matrix: Research Question Data Source Methodology 1. Were the PGIP goals accomplished? These goals include: providing gas supplies to Western Georgia; and increasing the reliability of Georgia’s power grid. Assistance Agreement GoG reports Implementer reports (GOGC and Tetra Tech) Document Review Key Informant Interviews (USAID, GOG, GSE, GOGC, other donors a. What primary and secondary economic benefits, both current and projected, over the next ten years, can be associated with gas pipeline construction under PGIP? Measures of such benefits include but are not limited to: the number of households and the number of businesses receiving access to gas services (both current and projected), and the amount of increased business activity and job- creation (desegregated by sex.) Appropriate measures shall also be Implementer reports (GOGC and Tetra Tech) GOGC data /analyses/projections Geostat reports (Statistics Agency) Various economic reports and economic projections prepared Document Review Key Informant Interviews and/or focus group discussions with GOGC, GOG, USAID, Georgian energy sector experts/economis ts, other donors, etc. b. What is the overall quality of the design of the Senaki power transmission project prepared by the engineering oversight contractor as perceived by host country stakeholders? GSE Reports Implementer reports (Tetra Tech) GoG reports Communication between implementer, GSE, USAID and GOG Document Review Key Informant Interviews (USAID, Tetra Tech, GSE, GOG) Focus Group Discussions with power c. Evaluators should comment on the overall d process of designing the transmission project, by Communication between Document Review Research Question Data Source Methodology reviewing progress reports and communication trail between oversight contractor, GSE, USAID and other stakeholders. implementer (Tetra Tech), GSE, USAID and GOG Key Informant Interviews (GSE, Tetra Tech, USAID) d. To what degree have USAID’s investments in sub-procurements, including CAPE and DGA, promoted or have a potential to promote a safer, more efficient, reliable and robust Georgian power grid? GSE and Implementer (Tetra Tech) calculations/projectio ns regarding CAPE and DGA Key Informant Interviews with GSE, Tetra Tech and USAID Focus Group Discussions with power id 2. The evaluation contractor should comment on the sustainability of the infrastructure built with USAID support. Under sustainability we imply host country’s operations and maintenance capacity (GSE/GOGC) and the ability/readiness to maintain and further invest in infrastructure improvements. To what extent were the operations and maintenance of Georgian gas pipeline and power transmission systems improved and GSE Reports and Documentation GOGC Reports and Documentation Implementer Reports (Tetra Tech) GOG Document Review Key Informant Interview with GOG, GSE, GOGC, Tetra Tech and USAID Focus Groups 3. What will be lessons learned for future infrastructure program designs, procurements, executions and oversight services that will utilize both private sector and host country controlled organizations? GOG, GSE, GOGC, reports and documentation Implementer Reports (Tetra Tech) Document Review Key Informant Interviews Focus Groups V. Work Location The work will be performed in Tbilisi and selected Georgian regions and in the U.S. The teams will travel outside the capital as needed in order to meet with key players (GOGC, GSE, Tetra Tech, other host country partners) in diverse parts of the country, and to get a better sense of the overall context within Georgia. VI. Evaluation Team The evaluation contractor must propose the composition of the evaluation team. However, it is expected that the team will be comprised of a team leader, an evaluation expert, and a locally- hired expert/consultant, who should have experience conducting evaluations and assessments in pertinent areas related to energy infrastructure, preferably related to gas pipelines and power transmission projects. The team collectively must have expertise in conducting economic analysis using quantitative data and statistical projections. Experience in Georgia or in the Europe and Eurasia region is desirable. Strong analytical, communication and writing skills are also required. The following are key personnel skills required for the completion of the evaluation (this does not cover locally hired staff to collect data for the survey, if proposed): The Team Leader (international) will have experience conducting evaluations and assessments, including evaluations in the energy and energy infrastructure sectors. Experience in managing, implementing, or evaluating pipeline construction, power transmission line construction, and general electric power sector operation activities is highly desirable. Experience in Georgia or in the Europe and Eurasia region is desirable. The team leader will be responsible for the day-to-day management of the team, data collection and synthesis, presentations, and draft, interim, and final reports. Evaluation Expert (international) will have experience conducting evaluations and assessments using various data collection and analysis methods. Experience in evaluations of electrical engineering power systems construction, and control systems. Pipeline engineering design and construction is desirable. One locally hired expert/consultant with experience in the energy sector and experience participating as a team member conducting a USAID project-related assessment or evaluation. Prior work experience preparing economic analyses or statistical forecasts is preferable. The consultant should have deep knowledge of Georgia’s economic growth sector, is expected to provide valuable insights into Georgia’s development context, and may assist on a wide array of tasks, such as obtaining and analyzing statistical information on business activity in western Georgia, reviewing and analyzing documentation/business processes at GOGC and GSE, etc. English language knowledge is a requirement. In addition, if deemed appropriate, a translator can be hired. The proposed personnel must have considerable experience in designing, contracting, contract management, and evaluating development assistance programs. They must have excellent written and oral presentation skills. In case the implementation of a survey is proposed, the evaluation contractor must provide information on who will be collecting data in the field. All Team members must provide a signed statement attesting to a lack of conflict of interest, or describing an existing conflict of interest. The Evaluation team must demonstrate familiarity with USAID’s Evaluation Policy (http:/www.usaid.gov/evaluation/USAIDEvaluationPolicy.pdf) USAID will have an interview with the evaluation team leader through conference call/skype or any other means available. VII. Performance Period The following levels of effort are illustrative and should serve only as an example of the staff that may be mobilized under this Task Order. These levels may not reflect the actual level of effort contracted, and the evaluation contractor must submit its own estimate of the level of effort needed to fulfill the objectives. Total No of Days in Country /Consultant No of Work Days in Country /Consultant No of Days for preparation and Report Writing Total No of Work Days /Consultant International Technical Expert 32 10 40 Evaluation Expert 32 10 40 Local Consultant 35 35 The evaluation must be completed by September 30, 2013. VIII. Deliverables and Reporting Guidelines Draft and Final Work Plan and Evaluation Design A draft and final Work Plan and Evaluation Design document for the evaluation will be no more than ten pages, and shall be completed by the lead evaluator within two weeks of the award of the contract, and presented to the COR. The evaluation design will include a detailed evaluation design matrix (including the key questions, methods and data sources used to address each question and the data analysis plan for each question), draft questionnaires and other data collection instruments or their main features, known limitations to the evaluation design, and a dissemination plan. The final design requires COR approval. Unless exempted from doing so by the COR, the design will be shared with country-level stakeholders as well as with the implementing partners for comment before being finalized. The work plan will include the anticipated schedule and logistical arrangements, and delineate the roles and responsibilities of members of the evaluation team. Final Evaluation Report The evaluation contractor must submit a draft evaluation report to USAID at least two weeks prior to the completion of the Task Order. The report must explicitly respond to the requirements of the SOW, should answer the evaluation questions, be logically structured, and adhere to the standards of the USAID Evaluation Policy of January 2011 and the criteria to ensure the quality of the evaluation report. The report should not exceed 25 pages, excluding executive summary and annexes. The evaluation contractor must incorporate USAID’s comments, and submit the final report to USAID/Georgia within five working days following receipt of comments on the draft report. The evaluation final report should include an executive summary, introduction, background of the local context and the projects being evaluated, the main evaluation questions, the methodology or methodologies, the limitations to the evaluation, findings, conclusions, and recommendations and lessons learned (if applicable). The executive summary should be 3-5 pages in length and summarize the purpose, background of the project being evaluated, main evaluation questions, methods, findings, conclusions, and recommendations and lessons learned (if applicable). The evaluation methodology shall be explained in the report in detail. Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (e.g., selection bias, recall bias, unobservable differences between comparator groups, etc.) The annexes to the report shall at a minimum include:  The Evaluation Scope of Work  Any “statements of differences” regarding significant unresolved difference of opinion by funders, implementers, and/or members of the evaluation team  All tools used in conducting the evaluation, such as questionnaires, checklists, and discussion guides  Sources of information, properly identified and listed  Disclosure of conflicts of interest forms for all evaluation team members, either attesting to a lack of conflict of interest or describing existing conflict of interest. The evaluation contractor must make the final evaluation reports publicly available through the Development Experience Clearinghouse within 90 calendar days of final approval of the formatted report. In case the final evaluation report includes information protected from public disclosure, submission of a sanitized version, which can be used as a public document, will be necessary. Criteria to Ensure the Quality of the Evaluation Report Per the USAID evaluation policy, draft and final evaluation reports will be evaluated against the following criteria to ensure the quality of the evaluation report.1  The evaluation report should represent a thoughtful, well-researched and well organized effort to objectively evaluate what worked in the project, what did not and why.  Evaluation reports shall address all evaluation questions included in the scope of work.  The evaluation report should include the scope of work as an annex. All modifications to the scope of work, whether in technical requirements, evaluation questions, evaluation team composition, methodology or timeline need to be agreed upon in writing by the technical officer.  Evaluation methodology shall be explained in detail and all tools used in conducting the evaluation such as questionnaires, checklists and discussion guides will be included in an Annex in the final report.  Evaluation findings will assess outcomes and impact on males and females.  Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.).  Evaluation findings should be presented as analyzed facts, evidence and data and not based on anecdotes, hearsay or the compilation of people’s opinions. Findings should be specific, concise and supported by strong quantitative or qualitative evidence.  Sources of information need to be properly identified and listed in an annex.  Recommendations need to be supported by a specific set of findings.  Recommendations should be action-oriented, practical and specific, with defined responsibility for the action. IX. Other Requirements All records from the evaluation (e.g., interview transcripts or summaries) must be provided to the COR. All quantitative data collected by the evaluation team must be provided in an electronic file in easily readable format agreed upon with the COR. The data should be organized and fully documented for use by those not fully familiar with the project or the evaluation. USAID will retain ownership of the survey and all datasets developed. All modifications to the scope of work, whether in technical requirements, evaluation questions, evaluation team composition, methodology or timeline, need to be agreed upon in writing by the contracting officer. X. Projects Documents for Review and Logistics The CORs will put the evaluation contractor in contact with its implementing partners and will provide help with a small number of meetings (such as meeting with USG agencies where needed). Relevant reports and other project documentation will be provided by the Mission to the evaluation contractor prior to travel to Georgia. The evaluation contractor shall initiate Washington-based work by reading reports and familiarizing him/herself with the projects. These documents are:  Statement of work as is stated in the award;  Implementing partners Quarterly Reports;  Initial list of in-country contacts;  PMP indicator tables;  M&E plans submitted and approved by USAID;  Implemented monitoring reports;  Other deliverables (expert report, publications) produced by partner. The evaluation contractor must suggest how to provide translation, transportation, and logistical support to the evaluation team. ANNEX 2: PRELIMINARY EVALUATION SCHEDULE Evaluation Schedule (as of July 17) ◄ July 2013 ~ June 2013 ~ September 2013 ► Sun Mon Tue Wed Thu Fri Sat 26 27 28 29 30 31 1 2 3 4 5 6 7 8 9 10 11 12 13 24 15 16 17 18 19 20 21 22 23 ------------------------ 30 24 25 26 27 28 Review Materials Detailed Work Plan and Evaluation Methodology Home 29 ◄ June 2013 ~ July 2013 ~ August 2013 ► Sun Mon Tue Wed Thu Fri Sat 30 1 Review Materials Detailed Work Plan and Evaluation Methodology Home 2 Review Materials Detailed Work Plan and Evaluation Methodology Home 3 Review Materials Detailed Work Plan and Evaluation Methodology Home 4 Review Materials Detailed Work Plan and Evaluation Methodology Submit Detailed Draft Work Plan & Evaluation Design to COR Home 5 Review Materials Phone Call w/USAID/Georgia Home 6 ◄ June 2013 ~ July 2013 ~ August 2013 ► Sun Mon Tue Wed Thu Fri Sat 7 8 Review Materials Home 9 Travel to Georgia 10 Travel to Georgia 11 In-Briefing with USAID Discuss work plan and evaluation design Meet Tt; 4 PM at Tt Offices 12 Team Planning & Finalization of Work Plan and Evaluation Design Tbilisi 3:30 – meeting with mission director 13 Submit Detailed Final Work Plan & Evaluation Design to COR 14 15 Interviews with Stakeholders and Implementers 10:00 – USAID 12:00 Meeting with GGTC – Mamuka Kobakhidze , Irakli Okroshidze, Commercial Director and Zviad Rostomashvili 13:00 Meeting with GOGC- Zaqaria Avalian, Temur Gochitashvilii and Tato Goguadze Tbilisi 16 Interviews with Stakeholders and Implementers 11:00 T1/T2 Meeting with Tetra Tech Michael Hajny – PGIP COP Sophie Berishvili – PGIP DCOP Giorgi Beradze - Mid￾level Engineer, PGIP 15:00 Meeting with GSE – Maia Pitskhelauri and Sulkhan Zumburidze Tbilisi 17 Interviews with Stakeholders and Implementers 9:00 Meeting with David Kakabadze, Sakenergoremonti 10:30 Lela Kerashvili, Program development Specialist, USAID 14:00 – anar Mammadov – LLC Socar Georgia Gas Director 18:00 - Meeting with the deputy Minister of Energy 18 Field Work Travel to Kutaisi GOGC SER and Tt assistance T2: Kutaisi / Kut-Aba Gas / SMSM (Beradze) Tengiz dadiani, Iuri Tabukashvili T1: Samtredia/GIPTP Electric Tskaltubo SS (Zviad Khorshia and Vakhtang Kelbakiani) 19 Field Work GOGC SER and Tt assistance T2: Samtredia Gas Distribution Company / Gas (Beradze) T1: Abasha Senaki / GIPTP Electric Menji SS (Zviad Khorshia and Vakhtang Kelbakiani) 20 Field Work GOGC, GSE, SER and Tt assistance T2: Abasha Gas Distribution Company / Gas (Beradze) T1: GIPTP Overhead Line / SER (Zviad Khorshia and Vakhtang Kelbakiani) 21 22 Field Work GOGC, GSE, SER and Tt assistance T1: DGA at Zestaphoni Substation (Khorshia or Kelbakiani) T2: Senaki Gas Distribution Company / Gas (BeradzeP) 23 Field Work GOGC, GSE, SER and Tt assistance T1: EECS at Zestaphoni SS (Khorshia or Kelbakiani) T2: -Abasha – Gas Pipeline Project 24 Field Work GOGC, SER, GSE and Tt assistance T2: Senaki - Poti Gas Pipeline (Beradze) T1: GIPTP Overhead Line / SER River Crossings (Khorshia or Kelbakiani) 25 Field Work GOGC, GSE, SER and Tt assistance T2: Poti/ FIZ and Poti Gas Distribution Company (Beradze) T1: GIPTP Overhead Line SER problematic right of way areas (Khorshia or Kelbakiani) 26 Field Work Return to Tbilisis 27 11:00 – Patrick Lohmeyer, HICD COP Roman Tsutskiridze – HICD DCOP 28 29 Interviews with Stakeholders and Implementers GOGC and GGTC 2p.m mission direector Tbilisi 30 Interviews with Stakeholders and Implementers GSE and Tt Tbilisi 31 Interviews with Stakeholders and Implementers SER Data Analysis & Follow-Up Interviews Tbilisi 1 2 3 ◄ July 2013 ~ August 2013 ~ September 2013 ► Sun Mon Tue Wed Thu Fri Sat 28 29 30 31 1 Data Analysis & Follow-Up Interviews Tbilisi 2 Data Analysis & Follow-Up Interviews Tbilisi 3 Data Analysis & Prepare for Out￾Briefing Tbilisi 4 5 Out-Briefing with USAID 6 Travel Home 7 8 9 Write Draft Report Home 10 11 12 Write Draft Report Home 13 Write Draft Report Home 14 Write Draft Report Home 15 Write Draft Report Submit Draft Report Home 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 ◄ July 2013 ~ September 2013 ~ September 2013 ► Sun Mon Tue Wed Thu Fri Sat 1 2 3 4 5 6 7 ◄ July 2013 ~ September 2013 ~ September 2013 ► Sun Mon Tue Wed Thu Fri Sat 8 9 Receive Comments Integration of Comments Home 10 Integration of Comments Home 11 Integration of Comments Submit Final Report Home 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 1 2 3 4 5 ANNEX 3: LIST OF DOCUMENTS REVIEWED AND RECEIVED ANNEX 4: EVALUATION MATRIX & SOURCES OF INFORMATION TO RESPOND TO THE EVALUATION QUESTIONS PGIP Evaluation Matrix No. Evaluation Questions and Sub-Questions Data Source Methodology Data Analysis Evaluation Question 1: Were the PGIP goals accomplished? 1.1 What primary and secondary economic benefits, both current and projected, over the next ten years, can be associated with gas pipeline construction under PGIP? Supply-demand a. Is the baseline of gas pipe mapped on GIS or alternative drawings? b. Is the concept mapped? c. Is the present demand of gas fully satisfied? d. What volume of gas flow is planned? e. Are future needs of gas taken in consideration? f. Was private sector participation in works encouraged? End-user impacts  number of households which have gained access to gas (Poti)  potential and estimated number which can gain access to gas (Poti)  number of businesses with access to gas vs. connected to gas  increased business activity attributable to gas connection and projections for future activity  jobs attributable to gas (by gender)  Assistance Agreement  GOG Reports  Tetra Tech Reports  GOCG Reports/Analyses and Projections  Geostat Reports  Various Economic Reports prepared by GOG, other donors, or local experts  USAID Initial CBA  Public records  Statistics for baseline  Document Review  Key Informant Interviews (KIIs) with USAID, GOG, GOCG, Georgian energy sector experts, other donors  FGDs with HHs and businesses  Mini-survey  Private gas company data on users, connections, consumption levels Private distribution companies.  Document Review  Comparison of HHs and businesses connected to gas with those with access but not connected  Analysis of the amount of increased business activity and job creation, based on self-reporting by businesses  Analysis of project outputs and targeted results and verification of project reporting  Charting  Quantitative Analysis of the mini￾surveys   Gas meters and billing analysis  Qualitative Analysis of KIIs PGIP Evaluation Matrix No. Evaluation Questions and Sub-Questions Data Source Methodology Data Analysis 1.2 1.3 What is the overall quality of the design (compared to international standards) of the Senaki power transmission project prepared by the engineering oversight contractor as perceived by host country stakeholders (GSE, GOG, etc.)? a. Were the present and future needs for electricity properly assessed? b. Did GOG cooperate in providing feedback, determine needs, etc. for the project? c. Did the design provide best value for money? d. Did the BOM comply with local and international standards? e. Were provision taken for low O&M? Comment on the overall process of designing the transmission project? a. What were baseline conditions before project start and determination of needs? b. What codes were used for the design and refurbishment? c. Is there a program (design, implementation, capacity building) for the Enhances Emergency Control Sys? d. Was the design completed prior of contract award? e. Were there any land issues associated with the design? f. Was there a detailed schedule for oversights and monitoring? g. Was the capacity building program  Tetra Tech Reports  GSE reports  GOG reports  Communication between implementer, GSE, USAID and GOG  Other?  Communication between Tetra Tech, GSE, USAID and GOG.  Local and international Standards  Calculations, Legal (land issues), Drawings, Mapping.  Key Informant Interviews with USAID, Tetra Tech, GOG, GSE  Focus Group Discussions with Power Grid Professionals  Pictures and QA/QC  Document Review  Key Informant Interviews with GSE, USAID and Tetra Tech  Verify change orders and FGDs  Data charting and extrapolations  Critical analysis of the design process  Qualitative Analysis of KIIs; of communication between implementers, USAID and GOG; and of materials reviewed  Critical synthesis and triangulation analysis of opinion data.  Cost effectiveness of design PGIP Evaluation Matrix No. Evaluation Questions and Sub-Questions Data Source Methodology Data Analysis 1.4 1.5 planned during this phase and coordinated with GOG? h. How the design process should be improved? To what degree did USAID’s investments in sub￾procurements, including CAPE and DGA, promote or have the potential to promote a safer, more efficient, reliable and robust Georgian power grid? a. What are the CAPE Software concept and application overview, advantages, needs, applications, use, etc? b. What are the technical and economic advantages of DGA and how their feedback is monitored and acted upon? Transparency and openness to competition a. How many companies participated? Did winning companies meet all tender requirements? b. Were independent (not GOG connected) private sector businesses invited to participate? c. Which party was doing the procurement? d. Is the procurement documented in accord with USAID/GoG rules? e. Is there a tender document for each component of the electricity project? f. What are the details of contracts negotiations? g. Did the contractor provide updated drawings?  GSE and Tetra Tech calculations/projections regarding CAPE and DGA  USAID  GOG  Operators and Maintenance personnel   Tenders and offers  Contractors qualification and selection  GOG, Tetra Tech, GSE and GOCG relevant contract and contractors documents  Key Informant Interviews with GSE, Tetra Tech and USAID  Focus Group Discussions with Power Grid Professionals  Professional and operators’ views.  Building comparison spreadsheet.  Mini survey with winners and losers  Focus group discussions  Key Informant Interviews  Qualitative Analysis of KIIs and FGDs  Quantitative Analysis of reports expected safety and efficiency.  Critical synthesis and triangulation of data collected from various sources.  Field results and measurements.  Quantitative analysis  Qualitative analysis (BOM, standards, price offer)  Schedule, conditions of offer PGIP Evaluation Matrix No. Evaluation Questions and Sub-Questions Data Source Methodology Data Analysis 1.6 h. Are there receiving and testing documents? i. Was the procurement completed? j. Did the contractor delivered the BOM specifications or were modifications made? k. What better methodology should be used to make this part more efficient? Capacity Building e. Was capacity building segmented by topic (technical, managerial)? f. Were there high quality experts available to provide capacity building? g. Is there a baseline of skills before the program started and how was the program tailored to meet the needs? h. How many trainees participated in the capacity building program? i. Was the time, place, etc., sufficient to determine good results? j. Was tests administered to trainees to assess the skills improvements? k. How this program could have been improved?  Implementers reports  List of participants  Training materials  USAID  Training participants Evaluations of capacity building activities  Key Informant Interviews  Document Review  Quantitative and Qualitative Analysis of reports and results of interviews and FGDs .  Critical synthesis and triangulation of data collected from various sources. Evaluation Question 2: Sustainability of the Infrastructure built with USAID Support 2.1 To what extent the operations and maintenance of Georgian gas pipeline and power transmission systems improved and what was PGIP’s possible contribution toward such improvement?  GSE and GOGC past and current documentation and/or business processes associated with monitoring, maintaining  Document Review  Key Informant Interviews with GOG, GSE, GOGC, Tetra Tech and USAID  Analysis of targeted results, outputs, and outcomes  Verification of PGIP reporting  Quantitative and Qualitative Analysis PGIP Evaluation Matrix No. Evaluation Questions and Sub-Questions Data Source Methodology Data Analysis a. Is it possible to compare a baseline situation before the project to mid-term and expected end of the project situation? b. How reliable is gas and electricity supply compared with pre-project period? c. Were the privatization steps and progress achieved? Why and what can be done better? d. Was the energy security fully achieved? e. Split per value of procurement, contracting and oversight. f. Future expansion needs and repairing power and gas infrastructure.  Tetra Tech Reports  GOG Reports  Gas and electricity supply and consumption statistics.  M&O Statistics  Focus Group Discussions (FGDs)  Compare and plot statistical data from 2009 to present  On-site observations  Critical synthesis and triangulation analysis of opinion data  Statistical data (supply, consumption, service, M&O analysis Evaluation Question 3: Lessons Learned 3.1 What are the lessons learned from this mid-term evaluation of the PGIP that can inform future infrastructure program designs, procurements, executions, schedule and oversight services that utilize both private sector and host country controlled organizations? a. What lessons can be learned for improving project contracting transparency and open competition? b. How to design and implement realistic implementation schedules? c. Should contracts include a firm demand from each contractor to provide a performance guaranty? d. What were the inter-agency coordination methods between GOGC, GSE, and Georgian Railroad in the design and build of some of the work and how was this coordination?  GOG, GSE and GOCG reports and documentation  Tetra Tech reports  All technical and commercial documents  Field surveys  Users, stakeholders, GoG feedback, Georgian Railroad  Document Review  Key Informant Interviews  Focus Group Discussions  Systematic evaluation of each project’s component from design, implementation, schedule, etc.  Projections and extrapolations based on trends.  Critical synthesis and triangulation analysis of opinion data  Qualitative evaluations of results and progress so far.  Evaluations of Project’s expectations  Evaluation of project’s methodology.  Evaluation of private participation.  Evaluation of transparency  Evaluation of cooperation and coordination between agencies. Sources Of Information To Respond To The Evaluation Questions ANNEX 5: STAKEHOLDERS LIST ANNEX 6: QUESTIONNAIRES QUESTIONNAIRES DRAFT QUESTIONNAIRE - CONNECTED HOUSEHOLDS RECENTLY CONNECTED Gas Connection to the network 1. Are you connected to the gas network? 2. When did you connect to the gas network? 3. Why did you connect? 4. How much did it cost you to connect to the gas network? Reliability and supply 5. Have you had any supply interruptions in the last 6 months [that lasted for more than x hours]? 6. If yes, how many? 7. Can you consume as much gas as you need? 8. What color is the cooking flame? yellow, blue, other Cost 9. Please tell us what your gas bill was last month? 10. What is your average gas bill in winter in summer 11. Have you had trouble paying your bills in the last 3 months? 12. Please estimate how much money you save per month in summer since you connected to gas? 13. Please estimate how much money you save per month in winter since you connected to gas? Usage 14. What do you use gas for? cooking, heating, hot water, other 15. Before using gas, what fuel did you use? 16. Do you only use gas for these purposes? If yes, skip to Q14. 17. What else do you use? 18. Why do you use the other fuel? 19. Please tell us the main effect connecting to gas has had on your household? 20. How are men and women affected differently by a household’s access to gas? Electricity Reliability and supply 21. Do you use a generator? 22. Have you had any supply interruptions in the last 6 months [that lasted for more than x hours]? 23. If yes, how many? 24. Can you consume as much electricity as you need? Cost 25.Please tell us what your electricity bill was last month? 26.How many KwH electricity did you consume last month? 27.What is your average gas bill in winter? 28.Have you had trouble paying your bills in the last 3 months? Usage 29. What do you use electricity for? Lighting, TV, Computer, Music, Heating , Hot Water, Air Conditioning, other 30.Please look at this table on (summer) fuel expenses and tell us your expenses on each fuel for before you connected to the gas network and current. # Fuel expenses in summer Time 1 (before) Time 2 (after) 1 Liquid gas (in balloons) 2 Kerosene 3 Diesel 4 Wood 5 Coal 6 Manure 7 Other fuel for heating and lighting (specify) 8 Natural gas 9 Electricity Demographic information 31. What is your age? 32. How many persons live in your household? 33. Gender 34. HH income per month (current) 100 GEL or less 1 101 - 200 GEL 2 201 – 500 GEL 3 501 – 1000 GEL 4 1001 – 2000 GEL 5 2001 – 3000 GEL 6 More than 3000 7 (Don’t know) -1 (Refuse to answer) -2 DRAFT QUESTIONNAIRE – MINI-SURVEY FOR UNCONNECTED HOUSEHOLDS (TO GAS) Gas Connection to the network 1. How much does it cost to connect to the gas network? 2. Please tell us the reason you are not connected? 3. [if the reason is ‘connection fee is too expensive’] how much would be a reasonable connection fee in your opinion? 4. Do you plan to connect in the future? 5. if not why, not? Usage 9. What fuel do you use for: cooking, heating, hot water 10. Please tell us the main effect you expect connecting to gas would have on your household? 11. How are men and women affected differently by a household’s access to gas? Electricity Reliability and supply 12. Do you use a generator? 13. Have you had any supply interruptions in the last 6 months [that lasted for more than x hours]? 14. If yes, how many? 15. Can you consume as much electricity as you need? Cost 16. Please tell us what your electricity bill was last month? 17. How many KwH electricity did you consume last month? 18. What is your average electricity bill in winter? 19. Have you had trouble paying your bills in the last 3 months? 20. Please look at this table on (summer) fuel expenses and tell us your expenses on each fuel for before you connected to the gas network and current. # Fuel expenses in summer Time 1 (before) Time 2 (after) 1 Liquid gas (in balloons) 2 Kerosene 3 Diesel 4 Wood 5 Coal 6 Manure 7 Other fuel for heating and lighting (specify) 8 Natural gas 9 Electricity Demographic information 21. What is your age? 22. How many persons live in your household? 23. Gender 24. HH income per month (current) 100 GEL or less 1 101 - 200 GEL 2 201 – 500 GEL 3 501 – 1000 GEL 4 1001 – 2000 GEL 5 2001 – 3000 GEL 6 More than 3000 7 (Don’t know) -1 (Refuse to answer) -2 ANNEX 7: REPORT OUTLINE REPORT OUTLINE EXECUTIVE SUMMARY Evaluation Purpose and Evaluation Questions Project Background Evaluation Methodology and Limitations Findings and Conclusions 2 EVALUATION PURPOSE, OBJECTIVES & QUESTIONS 2.1 Evaluation Purpose 2.2 Evaluation Objective 2.3 Evaluation Questions 2.3.1 Were the PGIP goals accomplished? These goals include; providing gas supplies to western Georgia; and increasing the reliability of Georgia’s power grid. 2.3.1.1 What primary and secondary economic benefits, both current and projected, over the next ten years, can be associated with gas pipeline construction under PGIP? Measures of such benefits include but are not limited to: the number of households and the number of businesses receiving access to gas services (both current and projected), and the amount of increased business activity and job-creation (disaggregated by sex.) Appropriate measures shall also be made for environmental considerations. The findings will be informed by qualitative data (e.g. key informant interviews) with GOGC officials, local gas distribution companies, energy sector experts, USAID, and host government entities. The desk review and analysis of quantitative information, statistical and monitoring data and various economic projections for Georgia will also be necessary. 2.3.1.2 What is the overall quality of the design of the Senaki power transmission project prepared by the engineering oversight contractor as perceived by host country stakeholders (GSE, GOG, etc.)? (The PGIP oversight contractor was originally tasked with the design duty and was responsible for producing a construction contract for the implementation. However, the oversight contractor used a hybrid approach of a construction and a design-build contract, for which USAID issued a task order modification of the oversight contract. USAID is interested in learning about the implications of this change on the overall quality of the design, if any.) 2.3.1.3 What is the overall process of designing the transmission project? Evaluators should review progress reports and communications between the oversight contractor, GSE, USAID and other stakeholders. 2.3.1.4 To what degree did USAID’s investments in sub￾procurements, including CAPE and DGA, promote or have the potential to promote a safer, more efficient, reliable and robust Georgian power grid? This question can be answered by interviewing stakeholders (GSE, GOG, Tetra Tech, USAID etc.), and/or reviewing various analyses and projections prepared by GSE (if available) regarding the number of power outages or transformer failures prevented, money saved, etc., due to the adoption of CAPE and DGA systems. 2.4 The evaluation contractor must comment on the sustainability of the infrastructure built with USAID support. Under sustainability we imply host country’s operations and maintenance capacity (GSE/GOGC) and the ability/readiness to maintain and further invest in infrastructure improvements. To what extent the operations and maintenance of Georgian gas pipeline and power transmission systems improved and what was PGIP’s possible contribution toward such improvement? This will be informed by the review of companies’ past and current documentation and/or business processes associated with monitoring, maintaining and repairing power and gas infrastructure. 2.5 What are the lessons learned from this mid-term evaluation of PGIP that can inform future infrastructure program designs, procurements, executions and oversight services that will utilize both private sector and host country controlled organizations? 3 PROJECT BACKGROUND 4 EVALUATION METHODS & LIMITATIONS 4.1 Methodology 4.2 Limitations 5 FINDINGS, CONCLUSIONS AND RECOMMENDATIONS ANNEXES ANNEX C: LIST OF MEETINGS N CONTACT PERSON POSITION ORGANIZATION LOCATION DATE 1 Nick Okreshidze Keti Chogovadze T1/T2i USAID, office of energy and Environment USAID, Program Development Specialist USAID Tbilisi 07/11//2013 2 Michael Hajny Sophie Berishvili Giorgi Beradze Mamuka Kikalishvili T1/T2 TT, Director TT, DCOP TT, Mid-level Gaz Engineer TT, Manager Tetra Tech Tbilisi 07/11//2013 3 Stephen M. Haykin Lela Kerashvili T1/T2 USAID, Mission Director USAID, Program Development Specialist USAID Tbilisi 07/12//2013 4 Sukru Bogut Jonathan Chappell Nick Okreshidze Lela Kerashvili T1/T2 USAID, Senior Energy Infrastructure advisor USAID, office of energy and Environment USAID, office of energy and Environment USAID, Program Development Specialist USAID Tbilisi 07/15//2013 5 Mamuka Kobakhidze Irakli Okroshidze Zviad Rostomashvili T1/T2 GGTC GGTC, Commercial Director GGTC, Technical Director Georgian Gas Transportation Company Tbilisi 07/15//2013 6 Zaqaria Avaliani Temur Gochitashvilii Tato Goguadze T1/T2 GOGC, Technical Director GOGC, Strategic Development Department Head GOGC, PGIP project manager Georgian Oil and Gas Corporation Tbilisi 07/15//2013 7 Michael Hajny Sophie Berishvili Giorgi Beradze Sukru Bogut T1/T2 TT, Director TT, DCOP TT, Mid-level Gaz Engineer USAID, Senior Energy Infrastructure advisor Tetra Tech Tbilisi 07/16//2013 8 Maya Pitskhelauri Sulkhan Zumburidze Ucha Uchaneishvili T1/T2 Nils Junge GSE, International Projects and Reporting Department Manager GSE, General director GSE, Member of Management Board Georgian State Electrosystem Tbilisi 07/16//2013 9 David Kakabadze T1 Saqenergoremonti, General director Sakenergoremonti Tbilisi 07/17//2013 10 Lela Kerashvili T2 USAID, Program Development Specialist USAID Tbilisi 07/17//2013 11 Sukru Bogut Nick Higgins Lela Kerashvili T1 USAID, Senior Energy Infrastructure advisor USAID, Program Officer USAID, Program Development Specialist USAID Tbilisi 07/17//2013 12 Anar Mammadov Anton Samsonidze T2 SOCAR Georgia Gas Director SOCAR Georgia Gas, First Deputy Director SOCAR Tbilisi 07/17//2013 13 Mr. Eloshvili Ms. Mariam Valishvili T1/T2 Nils Junge Deputy MoE Deputy MoE Ministry of Energy Tbilisi 07/17//2013 14 DGA, EECS Zestaphoni s/s (George Beradze) GSE Georgian State Electrosystem Zestaphoni 07/18//2013 15 Vakhtang Kelbakiani George Beradze T1 Sakenergoreonti Project Coordinator TT, Mid-level Gaz Engineer SAKENERGOREMONTI Tetra Tech Tskhaltubo 07/19//2013 16 Tengiz Dadiani Luri Tabukashvili George Beradze T1 Kut-Aba Gas / SMSM TT, Mid-level Gaz Engineer Kut-Aba Gas / SMSM Tetra Tech Kutaisi 07/19//2013 17 Zviad Korshia Vakhtang Kelbakiani T1 Sakenergoreonti, Deputy General Director, Project manager Sakenergoreonti Project Coordinator SAKENERGOREMONTI Samtredia 07/19//2013 18 Gas pipeline – old pipeline river crossing, new pipeline HDD and aerial river crossing, Cathodic protection, gas distribution installations Gas Distribution Company TT, Mid-level Gaz Engineer Gas Distribution Company Tetra Tech Abasha 07/20//2013 George Beradze T1 19 Nino Kuprava FIZ, Sales Executive, Investment Promotion Coordinator Free Industrial Zone Poti 07/22//2013 20 Sulkhan Tolordava T1/T2 Head of Economic-Financial Department, Poti City Hall Poti City Hall Poti 07/22//2013 21 Tamaz Jgharkava T1/T2 SOCAR, Business Manager SOCAR Poti 07/22//2013 22 1. SOCAR-Gas Business/institutional consumers: 2. Poti Police department 3. Bakery “Shakro Kupunia” 4. Bakery “Tsiala Topuria” 5. Private Hospital 6. Restaurant “Wine House” 7. Scrap metal smelter “Moulds and Metals Georgia” 8. T1/T2 Poti 07/23//2013 23 David Bagaturia “LLC Gasko+” Business consumers: 1. Asphalt factory “Arkheopolis” 2. Bakery “Gabeda” T2, Nils Junge “LLC Gasko+”, Director Senaki Gas Distribution company “LLC Gasko+” Senaki 07/24//2013 24 SOCAR Gas Business Consumers: 1. Asphalt Factory “Transmobil” 2. Public School No. 12 3. Restaurant “ 4. Connection to main pipeline 5. Appliances shop T1/T2 Poti 07/24//2013 25 Patrick Lohmeyer Chemonics, HICD COP Chemonics Tbilisi 07/29//2013 Roman Tsutskiridze Ekaterine Leonidze T1 Peter Tal/T2 Nils Junge Chemonics, HICD DCOP Chemonics, HICD Senior Organizational Development Specialist 26 Enrico Spiller T1/T2 Nils Junge KFW, Director, Sector Coordination Energy and Transport KFW Development Bank Tbilisi 08/02//2013 27 David Managadze T1 EBRD, European Bank of Reconstruction and Development Tbilisi 08/02//2013 28 Kathie Julian T2 Nils Junge ADB, Resident Representative Asian Development Bank Tbilisi 08/02//2013 29 Stephen M. Haykin Nick Higgins Sukru Bogut Nick Okreshideze Lela Kerashvili T1/T2 USAID, Mission Director USAID, Program Officer USAID, Senior Energy Infrastructure advisor USAID, office of energy and Environment USAID, Program Development Specialist USAID Tbilisi 08/05//2013 ANNEX D: LIST OF MATERIALS CONSULTED ANNEX E: SUPPORTING DOCUMENTATION 1. AVERAGE POTI’S RESIDENTIAL CONSUMPTION The evaluation team utilized two methods to assess accuracy of an average Poti’s residential consumption. The methods were: A.1. – Meters reading (typical pictures reflecting the consumption are attached) A.2. – Theoretic calculation based on SOCAR’s data of residential consumption. (data attached). Both methods highlighted above concur and confirm the average residential consumption of gas to be between 0.70 to 0.75 m3 /day. 2. TOTAL POTI’S GAS CONSUMPTION The evaluation team used two methods to verify and confirm information regarding gas supply to Poti. The methods are explained below: B.1. During interviews with GOGC, we asked for information regarding the Senaki-Poti pipeline – one of the questions was related to gas supply to Poti. GOGC reply, extracted from the answers to our questionnaire is listed below. B.2. The data received from SOCAR in Poti was used again to re-calculate the consumption between the months reported by GOGC GOGC ANSWER ON SUPPLY TO POTI What amount of gas has flowed through Senaki‐  Poti section? During the period of September 1 2011 through September 1 2012, 144,235 cubic meters of natural gas has been processed. From September 1 2012 through 31 march 2013, 438,587 cubic meters have been processed. Compiling the data from GOGC and SOCAR, considering different starting dates, we have a very good concurrence and match. 3. STANDARD QUESTIONNAIRE TO STAKEHOLDERS 1 QUESTIONS RELATED TO CONCEPT AND DESIGN PRIOR PGIP START 1.1 What was the existing situation on the map, existing supplies, needs, etc. 1.2 What were the goals/objectives of the design? (mapped, supply, demand, etc) 1.3 With whom was the concept developed and finalized? (coordination) 1.4 What contingency plans/alternatives were considered if any? 1.5 Was the complete design completed before issuing tenders? 2 BOM & SPECIFICATIONS RELATED QUESTIONS 2.1 What international standards were used and why? 2.2 Was the BOM rigid or flexible allowing changes? 2.3 Were installation drawings and AS BUILT drawings required? 3 CONTRACTORS' SELECTION 3.1 How was the pre‐qualification list established? 3.2 What were the criteria of winning? 3.3 Were performance guaranties required and received? 4 PROCUREMENT RELATED QUESTIONS 4.1 Was the procurement in compliance with budget, specifications, standards, etc? 4.2 Was all the procurement completed? 5 IMPLEMENTATION SCHEDULE AND RELATED ISSUES QUESTIONS 5.1 Is the implementation on schedule? If not why? 5.2 Were there changes in the original design and why? 5.3 Did the contractors submit revised drawings and AS UILT? 5.4 Were there land releted problems (right of way) 5.5 Are all relevant drawings signed and approved? 6 ENVIRONMENTAL RELATED QUESTIONS 6.1 Was all construction in compliance with HSE regulations? 6.2 Was land remediation completed? 6.3 What are the safety devices implemented to prevent disasters? 7 TRAINING AND CAPACITY BUILDING 7.1 What training programs were implemented? 7.2 How many people were trained and for how long? 7.3 Did the training and capacity building achieve its goals? 8 SUSTAINABILITY AND O&M RELATED QUESTIONS 8.1 What cost effect on the commodity price has PGIP? 8.2 What cost effect on O&M has PGIP? 8.3 Are all the required technical skills fulfilled? 8.3 Are there operation, service, repair, spare parts, and drawings available?      9 ECONOMIC BENEFITS RELATED QUESTIONS 9.1 How did PGIP as a whole impact on your company's business model? 9.2 Did your company's profitability increase due to PGIP? 9.3 Did your company employ additional personnel (skilled and unskilled) due to PGIP? 10 PGIP MID‐TERM SUMMARY QUESTIONS? 10.1 What parts of your project could be done better and more efficient? 10.2 How would you rate the communication exchange with other stakeholders? SUB‐PROJECTS QUESTIONS A.1 DGA IMPLEMENTATION QUESTIONS A.1.1 Was training provided (installation, operation, monitoring, repair)? A.1.2 What are the results of the implementation? A.1.3 What are the direct and indirect benefits of the system? A.1.4 Visit a control room A.2 CAPE IMPLEMENTAION QUESTIONS A.2.1 Was training provided (installation, operation, monitoring, repair)? A.2.2 What are the results of the implementation? A.2.3 What are the direct and indirect benefits of the system? A.2.4 Visit a control room A.3 EECS IMPLEMENTAYTION QUESTIONS A.3.1 Was training provided (installation, operation, monitoring, repair)? A.3.2 What are the results of the implementation? A.3.3 What are the direct and indirect benefits of the system? Visit a control room 4. ECONOMIC ADVATAGE FOR SWITCHING TO GAS (COMMERCIAL) Through individual interviews with commercial users of gas in Poti, the team inquired on the fuel cost used prior of switching to gas and the current cost of using gas. The results are illustrated in the table below. 5. LACK OF ENERGY SAVINGS AWARENESS A typical low efficiency hot water atmospheric boiler used in a restaurant. Average expected thermal efficiency is below 75%. The boiler replaces the adjacent electric water heater on the left upper side of the picture. Even with such low efficiency, there are financial savings using gas. A modern wall-hung boilers (sealed combustion – modulating and condensing) - US and CE approved will have a minimal efficiency of 92% with a maximum efficiency of 98%. Energy saving on boiler efficiency are between 20 to 30%. A school heating system featuring a gas fired single stage boiler without controls other than the standard Hi Limit water temperature. According to the school principal, when rooms get too hot in the winter she and the teachers open the windows. Without thermostatic controls and outdoor temperature sensor to adjust water temperature, energy is wasted. Engineering data indicates that at least 30-40% of the energy is wasted by opening windows and overheating due to lack of proper controls. 6. DGA DGA MAIN COMPUTER DISPLAY IN ZESTAPHONI SUB-STATION12) 7. MODERN PIPELINE EQUIPMENT Modern monitoring, supervision and metering stations ANNEX F: HOUSEHOLD QUESTIONNAIRE HOUSEHOLD POWER AND GAS CONSUMPTION SURVEY QUESTIONNAIRE FOR CONNECTED HHS 2013 [Interviewer notes: Before starting the interview read the consent form to the respondent;] [INTERVIEWER INSTRUCTIONS: Before starting the interview find out the number of members who are aware about expenses associated with energy consumption. If such members’ number is equal to 1 – interview the respondent; If such members’ number is greater than one – use the Kish grid for respondent selection] Questionnaire Number Interviewer code: Day and time of interview: |___|___| |___|___| |___|___| : |___|___| Day Month HH MM Settlement: ____________________ respondent information name Telephone: 1st Contact Attempt Day and time of visit: |___|___| |___|___| Day Month |___|___| : |___|___| HH MM 2nd Contact Attempt Day and time of visit: |___|___| |___|___| Day Month |___|___| : |___|___| HH MM 3rd Contact Attempt Day and time of visit: |___|___| |___|___| Day Month |___|___| : |___|___| HH MM Household/family Visit Result Completed Interview 1 Partially completed 2 Rescheduled 3 No one at home 4 Household/family closed 5 Refusal (reason) ___________________ 6 Other (Specify ___________________ 7 Household/family Visit Result Completed Interview 1 Partially completed 2 Rescheduled 3 No one at home 4 Household/family closed 5 Refusal (reason) ___________________ 6 Other (Specify ___________________ 7 Household/family Visit Result Completed Interview 1 Partially completed 2 No one at home 4 Household/family closed 5 Refusal (reason) ___________________ 6 Other (Specify ___________________ 7 Interview Start Time_________________ Q1. [Don’t ask]Gender of Respondent Male 1 Female 2 Q2. Age of the respondent __________________ Q3. Education level of the respondent [Only one answer] ormal education 1 dergarten 2 mentary school (4-5 classes) 3 Incomplete secondary (5-9 classes) 4 Secondary (10-12 classes including general education, lyceum, gymnasium) 5 ndary vocational (technical or college) 6 Higher education diploma (Bachelor, Master) 7 Advanced higher education 8 (Don’t know) -1 (Refuse to answer) -2 Q4. Please think of the persons who currently live with you most of the time and share your budget when we refer to your household/family in the following questions. How many members are in your household, including you? __________________ Q5. Please tell me how many rooms are there in your apartment/house? a. Total number of rooms _______________ b. Number of bedrooms ________________ Q6. What is the area of apartment/house occupied by the household? __________________ sq.m Q7. [Show card] What were the HH expenses for the last month 100 GEL or less 1 101 - 200 GEL 2 201 – 500 GEL 3 501 – 750 GEL 4 751 – 1000 GEL 5 1001 – 2000 GEL 6 2001 – 3000 GEL 7 More than 3000 8 (Don’t know) -1 (Refuse to answer) -2 GAS – CONNECTED TO NETWORK Q8. Are you connected to the gas network? Yes 1 no 2 Q9. How long have you been connected? A Year B Month Q10. How much did it cost you to connect to the gas network [interviewer notes: if the installation fee included gas meter fee, write “0” in the meter field] __________________ (GEL) Meter Cost_______________(GEL) Q11. How did you pay? [Check all that apply] Household budget/savings 1 Government assistance 2 Borrowed from financial institution 3 Borrowed from friends/relatives 4 Other(specify) 5 (Don’t know) -1 (Refuse to answer) -2 Q12. Did you use an installment plan? Yes 1 no 2 Q13. How many gas supply interruptions have you had in the last month __________________ Q14. Is the gas pressure sufficient for your needs? Scale 1 – very satisfied 2 3 4 5 – not satisfied at all Q15. What color is the gas flame during cooking? __________________ Q16. What energy sources have you been using for the following purposes? A. Before you had gas B. Now Electricity Gas Balloon Wood Other Electricity Gas Balloon Wood Natural Gas Other Cooking          Heating          Hot Water          Other(specify )          Q17. If you aren’t using gas for heating, please name all possible reasons [Check all that apply] heating equipment cost to much 1 safety issue 2 other (specify) 3 Q18. If you use gas for heating, how many rooms do you heat? __________________ Q19. [Show card] Please tell us the main effect connecting to gas has had on your household? [Check all that apply] Decrease of expanses 1 More Comfort 2 Reduction of evoironment polution 3 Nature preservation (cutting trees) improvment 4 Other(specify) 5 (Don’t know) -1 (Refuse to answer) -2 Q20. Who was affected more in your household after connecting to the gas network Men 1 Women 2 Both 3 (Refuse to answer) 4 Q21. Please look at this table and tell us your average energy expenditures before and after connecting, for each season: # Fuel expenditures A Year Ago Current Cubic meter s Summer Winter Summer Winter 1 Liquid gas (in balloons) 2 price for 1 kg/lt liquid gas 3 Kerosene 4 Diesel 5 Wood 6 If wood obtained for free, write cubic meters 7 Coal 8 Manure 9 Electricity 10 Natural gas 11 Other fuel for heating and lighting (specify) Q22. If you know how many cubic meters of gas you consumed, please tell us the amount. _________cubic meters Q23. Have you not paid a gas bill on time in the last 6 months because of cash problems? Yes 1 No 2 Q24. Have you had to borrow in order to pay a gas bill in the last 6 months? Yes 1 No 2 Q25. If there were no change in your total energy expenses after you were connected to gas, would you still want to use and pay for gas? Yes 1 No 2 Go To Q27 Q26. [Show card] If yes, what are your reasons? [Check all that apply] More Comfort 1 Reduction of evoironment polution 2 Nature preservation (cutting trees) improvment 3 Other(specify) 4 (Don’t know) -1 (Refuse to answer) -2 Q27. Have you had any electricity supply interruptions during the past month? Yes 1 No 2 Go To Q29 Q28. If yes, how many? _________________ Q29. Please, tell us what was your electricity bill for the last month _________________ (GEL) Q30. If you happen to know how many KwH of electricity you consumed last month, can you tell us? [interviewer notes: put “-1” if respondent can’t recall the amount of electricity consumed] _________________ (KwH) Q31. What is your average electricity bill in winter? _________________(GEL) Q32. Have you had not paid an electricity bill on time in the last 6 months because of cash problems? Yes 1 no 2 Q33. Have you had to borrow to be able to pay an electricity bill in the last 6 months? Yes 1 no 2 Comments of Respondent ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ _____________________________________________________________________________ Interview End time_______________________ HOUSEHOLD POWER AND GAS CONSUMPTION SURVEY QUESTIONNAIRE FOR NON-CONNECTED HHS 2013 [Interviewer notes: Before starting the interview read the consent form to the respondent;] [INTERVIEWER INSTRUCTIONS: Before starting the interview find out the number of members who are aware about expenses associated with energy consumption. If such members’ number is equal to 1 – interview the respondent; If such members’ number is greater than one – randomly select the member and conduct the interview;] Questionnaire Number |___|___|___|___|___|___|___|___| |___|___| Interviewer code: Day and time of interview: |___|___| |___|___| |___|___| : |___|___| Day Month HH MM Settlement: ____________________ respondent information name Telephone: 1st Contact Attempt Day and time of visit: |___|___| |___|___| Day Month |___|___| : |___|___| HH MM 2nd Contact Attempt Day and time of visit: |___|___| |___|___| Day Month |___|___| : |___|___| HH MM 3rd Contact Attempt Day and time of visit: |___|___| |___|___| Day Month |___|___| : |___|___| HH MM Household/family Visit Result Completed Interview 1 Partially completed 2 Rescheduled 3 No one at home 4 Household/family closed 5 Refusal (reason) ___________________ 6 Other (Specify ___________________ 7 Household/family Visit Result Completed Interview 1 Partially completed 2 Rescheduled 3 No one at home 4 Household/family closed 5 Refusal (reason) ___________________ 6 Other (Specify ___________________ 7 Household/family Visit Result Completed Interview 1 Partially completed 2 No one at home 4 Household/family closed 5 Refusal (reason) ___________________ 6 Other (Specify ___________________ 7 Interview Start Time_________________ Q34. [Don’t ask]Gender of Respondent Male 1 Female 2 Q35. Age of the respondent __________________ Q36. Education level of the respondent [Only one answer] ormal education 1 dergarten 2 mentary school (4-5 classes) 3 Incomplete secondary (5-9 classes) 4 Secondary (10-12 classes including general education, lyceum, gymnasium) 5 ndary vocational (technical or college) 6 Higher education diploma 7 Advanced higher education 8 (Don’t know) -1 (Refuse to answer) -2 Please think of these persons who currently live with you most of the time and share your budget when we refer to your household/family in the following questions. Q37. How many members are in your household, including you? __________________ Q38. Write down how many rooms there are in your apartment/house? c. Total number of rooms _______________ d. Number of bedrooms ________________ Q39. What is the area of apartment/house occupied by the household? __________________ sq.m Q40. What were the HH expanses for the last month 100 GEL or less 1 101 - 200 GEL 2 201 – 500 GEL 3 501 – 750 GEL 4 751 – 1000 GEL 5 1001 – 2000 GEL 6 2001 – 3000 GEL 7 More than 3000 8 (Don’t know) -1 (Refuse to answer) -2 GAS – NOT CONNECTED TO NETWORK Q41. What energy sources did you use for the following purposes? [Check all that apply] A. Now Electricity Gas Balloon Wood Other Cooking     Heating     Hot Water     Other(specify)     Q42. Do you have an idea of how much it would cost you to connect to the gas network? __________________(GEL) Q43. Please, tell us the reasons why you aren’t connected to the gas network? [Check all that apply] Not able to cover installation fee 1 No gas coverage in our block/area 2 Gas not effective for heating/cooking 3 payment of fee in installments isn’t applicable 4 Other(specify) 5 (Don’t know) -1 (Refuse to answer) -2 Q44. [Show card] Please tell us the main effect connecting to gas would have on your household? Decrease of expanses 1 More Comfort 2 Reduction of evoironment polution 3 Nature preservation (cutting trees) improvment 4 Other(specify) 5 (Don’t know) -1 (Refuse to answer) -2 Q45. Please look at this table and tell us your average energy expenditures for each season: # Fuel expenditures Current Cubic meter Summer Winter s 1 Liquid gas (in balloons) 2 price for 1 kg/lt liquid gas 3 Kerosene 4 Diesel 5 Wood 6 If wood obtained for free, write cubic meters 7 Coal 8 Manure 9 Electricity 10 Other fuel for heating and lighting (specify) Q46. If there were no change in your total energy expenses after you were connected to gas, would you still want to use and pay for gas? Yes 1 no 2 Go To Q15 Q47. [Show Card] If yes, what are your reasons? [Check all that apply] More Comfort 1 Reduction of evoironment polution 2 Nature preservation (cutting trees) improvment 3 Other(specify) 4 (Don’t know) -1 (Refuse to answer) -2 Q48. Have you had any electricity supply interruption during the past month? Yes 1 No 2 Go To Q17 Q49. If yes, how many? _________________ Q50. Please, tell us what was your electricity bill for the last month _________________ (GEL) Q51. If you happen to know how many KwH of electricity you consumed last month, can you tell us? [interviewer notes: put “-1” if respondent can’t recall the amount of electricity consumed] _________________ (KwH) Q52. What is your average electricity bill in winter? _________________(GEL) Q53. Have you had not paid an electricity bill on time in the last 6 months because of cash problems? Yes 1 no 2 Q54. Have you had to borrow to be able to pay an electricity bill in the last 6 months? Yes 1 no 2 Comments of Respondent ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ Interview End time_______________________ ANNEX G: FOCUS GROUP GUIDELINES FOCUS GROUP DISCUSSION GUIDELINES – HOUSEHOLDS AND GAS Qualitative Assessment of PGIP Beneficiaries and Potential Beneficiaries Purpose and objectives The purpose of the focus group discussions is twofold. First, it will improve our understanding of the economic impact of connecting to gas. The FGDs will be used as a background for analysis of the project’s actual and potential impacts. Second, it will be used to inform the mini￾household survey questionnaire, covering broadly the same topics. That is, the HH questionnaire questions will be tested during the FGD for appropriateness and language, after which the HH questionnaire will be revised. INSTRUCTIONS FGD Composition Types: Two FGDs will be conducted with participants from: i) households connected to gas; and ii) unconnected households. (Virtually all households are expected to have electricity – over 90% of HHs in Georgia are connected). Size and gender: Each group will have 8-10 members, a mix of men and women. Location: Participants will be selected from the towns of Poti, Abasha, Samtredia, and Senaki. Other criteria: Age is not considered important, but the participants should be knowledgeable about their household’s budget. Duration: The group discussions will last approximately two hours. There should be approximately 8-10 persons per FGD. FGD practice has shown that if there are too many participants, it may be difficult for everyone to get a chance to speak. If there are too few, participants may feel uncomfortable, and a good discussion may not get off the ground. Preparation Before the FGD session starts, the facilitator should prepare all flipcharts to fill in as necessary.   Revisions to methodology Adjustments can be made to the methodology, based on observations made by the team, and discussed and agreed upon with the team’s Evaluation Specialist. Changes may relate to design, timing, appropriateness, sensitivity, clarity of the questions, etc. Research Questions The key issues to be addressed during the FGDs are:   i) Exploring and analyzing benefits and impact of connecting to the gas network, in relation to other fuels ii) Exploring and analyzing the costs of connecting Outputs  Brief summary report (2-3 pages) of each FGD  Recordings  Registration forms  Recommendations for HH questionnaires REGISTRATION FORM Before the session begins, participants will be asked to fill in a form with basic factual information: Name Age Number of household members Place of residence *What was your gas bill last month? *If you know how many cubic meters it was for, please write the amount *What is your average gas bill in the summer? *What is your average gas bill in the winter? Please tell us what was your electricity bill last month? If you know how many KwH electricity you consumed last month, please write the amount *Only for connected HHs. What is your income? 100 GEL or less 101 - 200 GEL 201 – 500 GEL 501 – 1000 GEL 1001 – 2000 GEL 2001 – 3000 GEL More than 3000 Please consider this table about summer period and tell us your expenses on each fuel before you connected to the gas network and what those expenses are now. # Fuel expenses during winter (monthly)* Fill in as applicable Time 1 (before connecting) Time 2 (after connecting) 1 Liquid gas (in balloons) 2 Kerosene 3 Diesel 4 Wood 5 Coal 6 Manure 7 Other fuel for heating and lighting (specify) 8 Gas 9 Electricity *Only for connected HHs. INTRODUCTION AND WARM-UP Thank the group for coming and accepting to spend time to have this discussion. We have asked you to come here today because we are interested in learning about your experience using energy sources in your home, as well as some other related questions. We are interested in these issues because a new gas pipeline has been constructed under a USAID-financed project. USAID has commissioned an evaluation of the work completed so far, and is interested in learning how and whether persons or businesses have been affected. The information you share with us will be very valuable for improving our understanding of how Georgian citizens can be better served by gas and electricity supply. Ask each member to introduce him or herself, please tell us: i) Name ii) Location iii) Profession iv) FOR CONNECTED GROUP ONLY: how long you have been connected to gas? QUESTIONS – CONNECTED HOUSEHOLDS Question Probes GAS 1 Why did you connect to the network? How long have you been connected to the gas network? Can you tell us how much it cost you to connect? (one time or installments) Do you know anyone who cannot connect because it is not affordable? What would help people connect more easily? 2 What do you use gas for? Do you only use gas for these purposes? Do you think gas is the best fuel for cooking, heating and hot water? If not, why not? What other fuels do you use and why? 3 How satisfied are you with using gas? Do you always use it when you can or do you use other fuels sometimes because they are cheaper? Have you had any supply problems with gas? Can you recall what color is of the gas cooking flame? Are there any negative impacts? 4 Are you satisfied with your gas provider? With repairs? With billing process? 5 What is your average monthly bill in summer and in winter? Have you had trouble paying your bills in the last 3 months? Please tell us how your energy expenditures have changed since you since you connected to gas? (savings?) Please estimate how much money you save per month in summer / winter since you connected to gas? Do you know anyone who has switched or wants to switch to gas, even though they are using a cheaper fuel? 6 What are the main effects that connecting to gas has had on your household? Are men and women affected differently? How is your total household budget affected? ELECTRICITY 7 What do you use electricity for? Lighting, TV, Computer, Music, Heating, Hot Water, Air Conditioning 8 Have you had any supply interruptions in the last 6 months? Do you know the reason? For how long? Do you use a generator? 9 Electricity costs Have you had trouble paying your bills in the last 3 months? GENERAL 10 Do you have any recommendations to the Government to improve the gas and electricity situation in Aside from reducing the tariff. Georgia? 11 Finally, what did you think of the questions we have just asked you? Did they make sense? Was the flow good? Would add anything? QUESTIONS – UNCONNECTED HOUSEHOLDS Question Probes GAS 1 Do you have access to a gas network? If yes, are you planning to connect to the gas network? Why/why not? Do you know how much it would cost you to connect to the network? If you connect, how will you pay? (all at once, installments) 2 What benefits would you expect to receive from gas if you connected? Lower cost? More convenience? Cleaner fuel? 3 Do you know anyone who has connected to gas? Do you know if they are they satisfied? And why? 4 If you connected, how much do you think you would be paying? In terms of monthly expenditures? Please estimate how much money you think you could save per month in summer / winter if you connected to gas? 5 What fuels do you use for different purposes cooking, heating, and hot water? 6 What would you expect the main effect connecting to gas Would men and women be affected differently? How is would you expect your total household budget would have on your household? to be affected? ELECTRICITY 7 What do you use electricity for? Lighting, TV, Computer, Music, Heating, Hot Water, Air Conditioning 8 Have you had any supply interruptions in the last 6 months? Do you know the reason? For how long? Do you use a generator? 9 Electricity costs Have you had trouble paying your bills in the last 3 months? GENERAL 10 Do you have any recommendations to the Government to improve the gas and electricity situation in Georgia? Aside from reducing the tariff. 11 Finally, what did you think of the questions we have just asked you? Did they make sense? Was the flow good? Would add anything? D. ANNEX H: FOCUS GROUP SUMMARIES FGD SUMMARY – CONNECTED HOUSEHOLDS July 20, 2013 Nils Junge Marika Gorgadze interpreted Notes verified with Giorgi Gorgadze Date: July 19, 2013 Participants: 8 Duration 1’15” GAS 1 Why did you connect to the network? How long have you been connected to the gas network? Can you tell us how much it cost you to connect? (one time or installments) Do you know anyone who cannot connect because it is not affordable? What would help people connect more easily?  Gas is more efficient, comfortable, cheaper, efficient, and clean.  Electricity is expensive.  Wood is inconvenient. Wood is better for your health (more oxygen, no fumes, and don’t face the risks of dying), but it is a hassle, and gas’s advantages outweigh all that.  Connections costs cited (GEL): 300, 600, 700, 400, 641, 550  Many used or are using the installment payment plan, of 25 GEL for up to 24 months. Installments make paying much easier.  Many cannot afford to connect. Many want to connect, however, the installment program is no longer available, and so they can’t afford it. Some couldn’t even afford it with the installment program. “I know 15 families who want to connect [but cannot]” (Abasha resident)  In some districts you can’t get connected at all (because network not available there).  General complaints about having to pay for a new meter yourself, if the company tells your current one is deficient (under-metering). “That’s how they make money.” Meters cost about 200 GEL.  To make gas more affordable, reduce the price.  When people install their meters, sometimes the gas company says they have to move it or replace it, and this incurs a cost. 2 What do you use gas for? Do you only use gas for these purposes? Do you think gas is the best fuel for cooking, heating and hot water? If not, why not? What other fuels do you use and why?  All use it for cooking, some for taking showers [respondents don’t talk about ‘heating water’ more generally, but about using it for showers], several for heating some rooms, just one for heating the (entire) house.  When you connect to gas, you have to buy new appliances.  Many who have gas still use wood for heating the house. Gas can’t heat large rooms if you have a single-family house. Some use electricity for heating showers.  To install ‘central heating’ [by which I understand water boilers] is very expensive, as much as $3000 and very few can afford that. But it is very efficient. 3 How satisfied are you with using gas? Do you always use it when you can or do you use other fuels sometimes because they are cheaper? Have you had any supply problems with gas? Can you recall what color is of the gas cooking flame? Are there any negative impacts?  “Sometimes pressure is weak, and it doesn’t get stronger when I turn it up.”  Flame color: blue, blue, bluish-red, it starts blue and turns red, mostly bluish,  No supply problems. Now and then we have a day of interruptions, for repairs to the network, we think, but they’re always announced on TV in advance.  May get an interruption once or twice per season.  One case of a weak-long interruption for repairs, was noted. 4 Are you satisfied with your gas provider? With repairs? With billing process?  Provider is very strict about not paying. If you’re one day behind, they cut you off. To reconnect, you pay a 5 GEL fine.  You can pay your bill anywhere, even at the pharmacy. The quality of the gas line is low, but the price is high. These companies (SOCAR, Gazprom) are monopolies, so there’s no competition and they charge high prices for low quality pipes. 5 What is your average Have you had trouble paying your bills in the last 3 monthly bill in summer and in winter? months? Please tell us how your energy expenditures have changed since you since you connected to gas? (savings?) Please estimate how much money you save per month in summer / winter since you connected to gas? Do you know anyone who has switched or wants to switch to gas, even though they are using a cheaper fuel?  Cost can be up to 150 GEL if you were to use it for everything you like.  In Poti, no one has more than 150 GEL gas bills.  Friends are paying 500-600 GEL for a load of wood per season, while our gas costs 450 GEL. You also need to have good windows, not to lose heat.  Summer/winter payments: 10-15/80; 20/100; 30/100; 15/15; 10/10 (for those who use it only for cooking and showers, there is no difference)  Tariffs are either 52 or 60 tetri per cm  LPG cylinders – cost is about 13-17 per refill, which you need to do twice or three times per month, depending on the season and HH size. So gas is about half as expensive.  Even if gas is not the cheapest option (in cases where wood is free, for example) it is still worth connecting, since it is much more convenient. So cost and savings is only side of the story. However, it was noted that the price of firewood is increasing rapidly. 6 What are the main effects that connecting to gas has had on your household? Are men and women affected differently? How is your total household budget affected?  Main benefits: comfort, efficiency, less work  We save money. “The more you save, the more you spend, though.” But even if it is not a large impact on your total budget, you are still much better off with gas.  No real gender impacts. Men had to bring in the wood, but women have to do the cleaning up afterwards. ELECTRICITY 7 What do you use electricity for? Lighting, TV, Computer, Music, Heating, Hot Water, Air Conditioning  We use electricity for everything you usually use it for; the usual purposes.  Out of the group, 4 had AC, but only one of them uses it, as it is too expensive. 8 Have you had any supply interruptions in the last 6 months? Do you know the reason? For how long? Do you use a generator?  Interruptions are very rare, last no more than 10 min. 9 Electricity costs Have you had trouble paying your bills in the last 3 months?  Ele bills are quite low: 10,5,10,20,10-15 GEL GENERAL 10 Do you have any recommendations to the Government to improve the gas and electricity situation in Georgia? Aside from reducing the tariff.  Allow people to buy meters with installment plans as well.  Gov or other donors should provide assistance to vulnerable households so they can connect. FGD SUMMARY – NON-CONNECTED HOUSEHOLDS Date: July 19, 2013 Participants: 8 Duration 1’15” GAS 1 Do you have access to a gas network? If yes, are you planning to connect to the gas network? Why/why not? Do you know how much it would cost you to connect to the network? If you connect, how will you pay? (all at once, installments)  Most would like to or are planning to connect. About half do not have access where they live, the other half would connect, but the installment program (spreading out the connection cost over 24 months) was stopped, and so cannot afford it.  It is cheaper to have gas, but it is just not accessible.  Heard that connection can cost up to 800, which is too much. It would cost 600-800, which is too much.  In our area (in Abasha), only about 1 in 10 HHs are connected.  On our street (in Senaki), only 2 out of 15 are connected  They have promised to bring the network to us in Poti, but we don’t know when, still waiting.  In our neighborhood (Samtredia) almost all our neighbors have access except us.  Some houses couldn’t even afford it if they had an installment plan. And the interest rates are high. 2 What benefits would you expect to receive from gas if you connected? Lower cost? More convenience? Cleaner fuel?  Heating rooms, greater comfort, ‘financial efficiency’  Expect to cut heating costs by half or by two-thirds  “Can you imagine how it is to cook in the summer using wood when it is 40 degrees outside?”  “It’s definitely better to have gas.”  “By using gas instead of firewood, we can maintain our natural resource, the trees.”  Perhaps if gas consumption increases, with more customers, the price will eventually fall.  Showering using electricity is very expensive.  I prefer to pay 150 and heat all my rooms than pay 120 now (for wood) and heat only one room!  Using electricity to heat in winter is very expensive. 3 Do you know anyone who has connected to gas? Do you know if they are they satisfied? And why?  Our connected friends and neighbors are very happy with it.  In Senaki, we have neighbors who pay 5-6 for gas (cooking) or up to 9 if they use it for showers.  We hear that payments are 6-8 per month.  I’ve heard from neighbors that gas pressure is low, red flame, and not strong enough to cook sometimes. 4 If you connected, how much do you think you would be paying? In terms of monthly expenditures? Please estimate how much money you think you could save per month in summer / winter if you connected to gas?  A neighbor used to pay 2000 per season for wood, now he’s connected and pays just 800.  “I’m ready to spend anything to get gas, once I am given access.”  “Installation is expensive, consumption is not.” 5 What fuels do you use for different purposes cooking, heating, and hot water? LPG for cooking, wood for heating, sometimes electricity 6 What would you expect the main effect connecting to gas would have on your household? Would men and women be affected differently? How is would you expect your total household budget to be affected? Convenience. ELECTRICITY 7 What do you use electricity for? Lighting, TV, Computer, Music, Heating, Hot Water, Air Conditioning  We use electricity for the usual things (like in the list)  Some have AC, but use it rarely. The old AC units are inefficient, expensive. Some only turn it on when they have guests, or for the comfort of the children. 8 Have you had any supply interruptions in the last 6 months? Do you know the reason? For how long? Do you use a generator?  Rarely. When there are repairs going on.  “In the old days, we used to be surprised when the electricity went on. Now we’re surprised when it goes off!”  Occasionally in the evening for an hour or so there are blackouts. Seems to becoming more frequent these days in Senaki.  We don’t need generators these days. 9 Electricity costs Have you had trouble paying your bills in the last 3 months? Bills: 80, 30, 27, 15 (more in winter), 60, 60-120, 25, 10 (25 in winter) GENERAL 10 Do you have any recommendations to the Government to improve the gas and electricity situation in Georgia? Aside from reducing the tariff.  We get an extra charge on our bill to cover ele losses. We should not be charged for these because they aren’t our fault.  Don’t like it when the tariff goes up if my consumption exceeds a certain amount.  Ele quality is not good enough. Voltage is too low. Usually in the evenings.  Would be better if the gas company offered the installment, not through the bank (because of interest rates). i T1 Peter Tal, team leader, ME&A and Murman Margvelashvili, Local expert, team member, ME&A T2 Nils Junge, International expert, team member, ME&A and Giorgi Giorgadze, ME&A