DISCLAIMER: The views of the expressed in this document do not necessarily reflect the views of the United States Agency for International Development or the United States Government. USAID/INDIA GREENING THE GRID (GTG) PROJECT – FINAL PERFORMANCE EVALUATION FINAL REPORT Collaborating, Learning & Adapting in India Mechanism (CLAIM), Task Order No. 1 IDIQ # 72038619D00001 Task Order # 72038620F00003 October 13, 2021 Source: USAID/India Flickr Contract Title: Collaborating, Learning and Adapting in India Mechanism (CLAIM) Contract Number: 72038619D00001; TO No. 72038620F00003 Submitted: October 13, 2021 Contractor: Panagora Group Sub-contractor: Grant Thornton India Author: Amit Bando (Team Leader), Velimir Lackovic (Energy Specialist), Antony Gerard (Regulatory Specialist) CONTENTS ABSTRACT......................................................................................................................................................................... 2 EXECUTIVE SUMMARY.................................................................................................................................................. 3 INTRODUCTION............................................................................................................................................................ 8 ANNEXES PROVIDE:.................................................................................................................................................. 8 OVERVIEW......................................................................................................................................................................... 9 PROJECT BACKGROUND ....................................................................................................................................... 9 EVALUATION PURPOSE........................................................................................................................................... 9 EVALUATION APPROACH...................................................................................................................................... 9 THE APPROACH USED BY PANAGORA FOR THE GTG PERFORMANCE EVALUATION IS SUMMARIZED IN THE FOLLOWING FIGURE: ................................................................................................. 9 EVALUATION QUESTIONS...................................................................................................................................10 METHODOLOGY ..........................................................................................................................................................13 REMOTE DATA COLLECTION............................................................................................................................13 DOCUMENT REVIEW AND PERFORMANCE INDICATOR ANALYSIS..................................................13 KEY INFORMANT INTERVIEWS AND FOCUS GROUP DISCUSSIONS .................................................13 DATA ANALYSIS METHODS.................................................................................................................................14 GUIDING PRINCIPLES .............................................................................................................................................15 LIMITATIONS AND POTENTIAL BIASES ..........................................................................................................16 WORK PLAN...................................................................................................................................................................17 TEAM COMPOSITION.............................................................................................................................................17 FINDINGS.........................................................................................................................................................................19 OVERVIEW OF FINDINGS AND CONCLUSIONS.........................................................................................21 STATUS OF PILOT PROJECTS ..............................................................................................................................23 BESS IN TRANSMISSION UTILITY........................................................................................................................26 BESS IN DISTRIBUTION UTILITY.........................................................................................................................28 COAL-BASED FLEXIBLE POWER GENERATION (NATIONAL AND STATE)......................................29 AUTOMATIC GENERATION CONTROL (AGC)...........................................................................................30 DEVELOPMENT AND INTEGRATION OF DYNAMIC REACTIVE POWER CONTROL...................33 REGIONAL PLATFORM RESERVE SHARING....................................................................................................35 STATUS OF CROSS-CUTTING, REGULATORY, AND SUPPORTING ACTIVITIES.............................38 CONCLUSIONS .............................................................................................................................................................45 RECOMMENDATIONS.................................................................................................................................................47 A. PILOT-SPECIFIC RECOMMENDATIONS:.....................................................................................................47 B. RECOMMENDATIONS RELATED TO OVERALL PROGRAM INITIATIVES.......................................51 C. RECOMMENDATIONS FOR FUTURE PROGRAMMING........................................................................53 ANNEXES......................................................................................................................................................................55 ANNEX 1: SUMMARY OF GTG PILOT PROJECTS IMPLEMENTED TO DATE......................................55 ANNEX 2: SUMMARY OF RECOMMENDATIONS FOR PILOT PROJECTS............................................62 ANNEX 3: DATA SOURCES, DATA COLLECTION AND ANALYTICAL METHODS TO BE USED TO ADDRESS EVALUATION QUESTIONS.......................................................................................................65 ANNEX 4: RESEARCH BIBLIOGRAPHY.............................................................................................................66 ANNEX 5: DRAFT DATA COLLECTION PROTOCOL AND LIST OF KEY STAKEHOLDERS ........78 ANNEX 6: LIST OF KEY STAKEHOLDERS INTERVIEWED..........................................................................87 LIST OF TABLES AND FIGURES TABLE 1 EVALUATION QUESTIONS AND SUB-QUESTIONS........................................................................ 9 TABLE 2 TEAM COMPOSITION, LOE AND DELIVERABLES ...........................................................................16 TABLE 3 SUMMARY OF PILOT FINDINGS.......................................................................................................... 23 TABLE 4: SUMMARY OF REGULATORY ENGAGEMENT............................................................................... 40 TABLE 5 ADDITIONAL INFORMATION REQUIRED ON PILOT PROJECTS.............................................57 TABLE 6 SUMMARY EVALUATION MATRIX........................................................................................................64 1 ACRONYMS ACEF Asia Clean Energy Forum AFC annual fixed charge AGC automatic generation control AS ancillary services BESS battery energy storage systems BRPL BSES Rajdhani Power Limited CAPEX capital expenditure CEA Central Electricity Authority CERC Central Electricity Regulatory Commission CLAIM Collaborating, Learning, and Adapting in India Mechanism DERC Delhi Electricity Regulatory Commission DISCOM distribution company DO Development Objective DPR Detailed Project Report DRPC dynamic reactive power control DSM deviation settlement mechanism EQ evaluation question EU European Union EV electric vehicle FAT factory acceptance test F&S forecasting and scheduling FGD focus group discussion FOR Forum of Regulators GENCOS Generation Companies GOI Government of India GSECL Gujarat State Energy Corporation Limited GTG Greening the Grid IAA Inter-Agency Agreement IAC Industry Advisory Council IEC International Electrotechnical Commission INR Indian Rupees IP implementing partner IRENA International Renewable Energy Agency IT information technology KEC KEC International KI key informant KII key informant interview KPCL Karnataka Power Corporation Limited KPTCL Karnataka Power Transmission Corporation Limited LDC Load Dispatch Centre LOE level of effort MBED Market Based Economic Dispatch MEL monitoring, evaluation, and learning MOP Ministry of Power MU million units MW megawatt 2 MWh megawatt hour NARUC National Association of Regulatory Utility Commissioners NLDC National Load Dispatch Centre NOAR National Open Access Registry NREL National Renewable Energy Laboratory NTPC National Thermal Power Corporation OA Open Access OPEX operating expenses or expenditures PAC Project Advisory Committee PCC Power Control Centre PF power factor PGCIL Power Grid Corporation of India Limited POSOCO Power System Operation Corporation PPA Power Purchase Agreement PSSE Power system simulator for engineering PV Photovoltaics PX power exchange QCA Qualified Coordinating Agency RE renewable energy RISE Renewable Integration and Sustainable Energy Initiative RLDC Regional Load Dispatch Centers RTM real-time market RTU remote terminal unit SAMAST Scheduling, Metering, Accounting and Settlement of Transactions in Electricity SAT site acceptance test SAWIE South Asia Women in Energy SCADA Supervisory Control and Data Acquisition SCED Security Constrained Economic Dispatch SLDC State Load Dispatch Center SOW scope of work STATCOM Static synchronous Compensator TA technical assistance T&D Transmission and Distribution TL team leader TOC Theory of Change TPP thermal power plant U.S. United States USAID United States Agency for International Development USEA U.S. Energy Association USG United States Government USISPF US-India Strategic Partnership Forum VRE variable renewable energy WIP work in progress 2 ABSTRACT This document reports on the findings, conclusions, and recommendations of a final performance evaluation of the Greening the Grid (GTG) project. GTG is a five-year initiative under the U.S-India Strategic Clean Energy Partnership (SCEP) that aims to support efforts of the Government of India (GOI) to manage large-scale integration of renewable energy (RE) into regional and national grids. An important component of GTG were the pilot projects intended to demonstrate promising technologies for RE integration. The evaluation was carried out by Panagora Group in February-August 2021, and uses a mixed-methods approach, combining extensive document review and key informant interviews with GTG stakeholders. The main conclusions were that several pilots were successfully completed; other pilots were not yet completed due to delays in implementation; there was insufficient emphasis on sustainability and replication; and overall GTG had only partially met its main objectives. However, GTG had successfully engaged the GOI at national and state levels on policy and regulatory issues; this was reflected in strong buy-in and support from the GOI, which gives USAID a solid foundation for follow￾on work with the GOI. Recommendations for future USAID/India programming include ensuring full engagement with state-level GOI agencies; ensuring that technology choices have support from all government and private-sector stakeholders; increasing the emphasis on the building the enabling environment for RE integration; allowing greater for flexibility and adaptation of activities in response to changing conditions; and ensuring robust, opportune monitoring and evaluation to inform learning and adaptive management. 3 EXECUTIVE SUMMARY Greening the Grid (GTG) is a five-year initiative under the U.S-India Strategic Clean Energy Partnership (SCEP) that aims to support efforts of the Government of India (GOI) to manage large-scale integration of renewable energy (RE). The theory of change (TOC) for this project is “to build the flexibility of India’s grid through new market opportunities for the private sector, for ancillary services, better forecasting, improved operating systems and equipment, and flexible energy services.” The bulk of GTG activities has focused on pilot projects implemented under the Renewable Integration and Sustainable Energy Initiative (RISE) that also integrates reforms through public-private partnerships at the state and regional levels and provides technical assistance (TA). GTG also includes an Interagency Agreement (IAA) between USAID and the U.S. Department of Energy (DOE) Laboratories to support power system planning reforms and targeted analysis for pilot programs. Two India addendums (buy-ins) to the cooperative agreements with the National Association of Regulatory Utility Commissioners (NARUC) and the U.S. Energy Association (USEA) support peer exchange for innovation by regulators, grid operators, and utilities. The final performance evaluation of the GTG program was conducted between February 25, 2021, and July 15, 2021. Given the logistical challenges posed by the COVID-19 pandemic, all data collection and analysis activities were conducted remotely. A three-person technical team was supported by two logisticians to conduct a comprehensive desk review of over 120 documents supplemented by 40-plus key informant interviews (KIIs) and focus group discussions (FGDs) as well as discussions with 4 implementing partners (IPs). In addition to validating the TOC and assessing the status and effectiveness of the individual pilots, the purpose of this performance evaluation was to generate learning inputs for implementing the new Country Development Cooperation Strategy (CDCS) and to inform similar mechanisms USAID might design in the future. The evaluation team conducted a cross-comparison of available documents against the pilots’ scope of work (SOW) and international best practices. The structured KIIs were developed in consultation with USAID/India—were used to compare responses from key informants (KIs) with the findings of the desk review. Given that most of the GTG activities focused on the pilots, the evaluation team necessarily spent a lot of time assessing the status and effectiveness of pilot activities. Also, the team examined the status of the cross- cutting and supporting activities (especially those aimed at providing regulatory and policy support to the pilots) as well as the role of U.S. Government (USG) partners in the design and implementation of GTG activities. The overall findings and conclusion were developed based on a triangulation of the compiled data. The resulting recommendations can be grouped as those that are: (A) pilot specific; (B) related to overall program initiatives; and (C) program focused. Overall findings and conclusions can be summarized as follows: Some pilot projects have been successfully completed. This is true of the pilots on Battery Energy Storage Systems (BESS) and Coal-Based Flexible Power Generation. The pilots have generated useful and specific learnings; however, given that the pilots were essentially designed as stand-alone initiatives to demonstrate the effectiveness of individual technologies, guidance on pathways to replicate, scale-up and sustain similar activities in other geographic locations in India is limited, often requiring different parameters of operation compared to those at the pilot site(s). It should be noted that the delays in implementation schedules have prevented meaningful replication and scaling up of successful pilots. For this reason, on balance, the GTG project has only partially met its overall objectives. The status of the pilot activities can be summarized as follows: 1. BESS in Transmission Utility – mainly met the original scope even though scalability and sustainability are not clearly addressed. There are incomplete details on Part A and Part B of the SOW. 2. BESS in Distribution Utility – while it is now being indicated that this activity was not supposed to be a separate pilot, it was presented as such to the evaluation team. This activity did not meet original scope objectives and activities have changed. Results were never clearly presented - scalability and sustainability cannot be assessed or documented. Changes in the SOW and failing to meet the original SOW objectives point to design flaws. 3. Coal flexibility – pilot did relatively well. 4. AGC – a lot of problems were noted, particularly for the AGC in PV implementation. At the time of the evaluation, the pilot was not complete even though there were no major technical challenges. Part B of the scope has never been completed. There was a design problem with the pilot. 5. DRPC – original scope was not met. Not a single KII could be scheduled, and documentation is non-existent. Again, this is a design problem. 6. Regional Platform for Reserve Sharing – mostly complete. The regulatory, policy and institutional support provided under GTG has been well received. Institutions such as the Forum of Regulators (FOR) and the Central Electric Regulatory Commission (CERC) have benefitted from the support provided by GTG, and this support is acknowledged by GOI 5 stakeholders. Several analytical reports have been delivered and the discussions facilitated by GTG have resulted in moving the agenda forward on the policy and regulatory front. More needs to be done by USAID as a follow up to GTG-RISE to address key issues such as compensation mechanisms and the final implementation of the National Open Access Registry (NOAR) to support RE integration. Clearly, these initiatives have long gestation periods - future USG and GOI programs need to provide necessary continuing support to build on the momentum provided by GTG. Several GTG (pilot) initiatives are still underway. Delays have resulted from several factors, including the need to re-design activities post contract-award, lack of coordination among complementary and supporting activities, insufficient coordination among stakeholders during project implementation, and logistical bottlenecks caused by the COVID-19 pandemic. In this respect, the final project evaluation was more of a mid-term evaluation and few recommendations could be developed to address future project design. Proper documentation is lacking. Even when project activities have been successfully completed, there is a gap in necessary documentation. This lack of data prevents a thorough evaluation through triangulation of data gleaned from the KIIs. Furthermore, that lack of complete documentation of the protocols used, technical specifications and drawings of the equipment procured, economic and financial analyses of the actions implemented, and other relevant information prevented an in-depth exploration of the potential for replicating and scaling up project initiatives. Implementing Partners were “siloed” in their activities. While they were responsible for delivering on their own pilots’/activities’ results/outcomes, they did not believe that they should be held accountable for the sustainability of GTG results/outcomes. Clear roadmaps, metrics, and benchmarks had not been established to link individual activities to overall program success and ultimate validation of the Theory of Change. The evaluation team concluded that the GTG project has not validated the project’s TOC. Project design and implementation efforts were not sufficiently inclusive. In particular, the pilot activities were initially designed without thorough consultations with stakeholders at the state level. At project inception, the planned activities had to be vetted with state level stakeholders (e.g., in Karnataka) – this was time consuming, often involved redesigning activities and significantly delayed project implementation. In the case of some pilots, these activities reflected the needs and ground-level realities of the pilot sites. However, given the vastly different needs, resource and skills availabilities, and infrastructure facilities among Indian states, these successes did not translate into appropriate “lessons learned” for activities to be replicated at other locations in India. The complementary and support activities (including those related to certain policy and regulatory support mechanisms) must be adjusted to reflect the location-specific parameters. This issue was not adequately addressed, resulting in program shortcomings. State level involvement in project design and implementation is critical – without this, pilot activities cannot be scaled and sustained. This coordination with state-level stakeholders was achieved in Karnataka and Gujarat – however, this may still not be sufficient to ensure scalability. Several KIs have noted the lack of involvement of state level stakeholders as an impediment to widespread replication and scalability of pilot initiatives. 6 The role of other GTG partners was quite limited, despite some successes. The evaluation team examined the activities of other GTG partners, including NREL and USEA. It is noteworthy that during the KIIs both NREL and USEA stated that they were unaware of any actions resulting from their initial work, because “Indian counterparts were not interested in the proposed frameworks”. The work done by NREL with BRPL is mentioned in the annexes to this report (as KII notes); there is limited information on this activity because additional supporting documents were unavailable. NREL activities were primarily study related and focused on technical support to the Indian teams. NREL had also offered to complete additional analyses, but this offer was not accepted by the GOI as there was a strong push to proceed with the pilot implementation. In several instances, reports and documents on subjects such as grid-wide integration were not shared opportunely with the evaluation team. Several reports (such as the flagship reports on national and state level grid integration - released in 2017 and produced by jointly by NREL, LBNL, POSOCO and USAID) developed by the GTG program were not made available to the evaluation team until after the desk review and field work were completed, and for this reason were not included in the analysis. They were reportedly critical documents in determining the selection of the pilots and the technologies to be used. Overall, the non-RISE partners’ contribution to capacity building, development of white papers (e.g., for CERC/POSOCO), conducting conferences, etc. resulted in “increased awareness” on renewable energy integration among India stakeholders. These activities have been counted here as partial successes. USAID has noted that the national and regional integration studies have transferred technical know-how and capacity to national state modeling teams. The RISE Secretariat was partially successful. The RISE secretariat successfully organized webinars, conferences, and capacity building sessions while also coordinating partner activities and communicating with the GOI. This set of activities led to increased awareness of relevant issues within the broad stakeholder community – this has been a success. GOI stakeholders praised the quality of communications provided by the secretariat. During project implementation, the USG partners were not “kept in the loop” and they felt that their inputs were sought only on an ad-hoc basis. In particular, there were no clear designation of stakeholders’ roles, resulting in a lot of confusion. Overall, the GTG program faced several major challenges that limited its success. These can be broadly divided as: (a) Design related - includes decisions that should have been made prior to commencing the projects by understanding the ground level requirements. Instead, projects were started hurriedly and only then was it realized that the project scope may not be adequate or that Limited Role of partners were not willing to participate in the pilot under given conditions. (b) Implementation related - primarily related to the unwillingness of bidders to participate in certain pilots due to the onerous regulations (US and Indian regulations had to be complied with) or slow approval processes. Implementation was also delayed by the Covid-19 pandemic. However, it would be unfair to attribute all delays to the pandemic. The pandemic exacerbated the existing delays. Put another way, four pilots (Pilots 1, 2, 4, and 5) have had challenges related to design. They are late, incomplete, and have different deliverables and outcomes compared to those contemplated in the original SOW. As 7 such, 2/3 (67%) of the original SOW of the Pilots has not been met in one way or another because of design issues. Covid related delays have compounded the problems arising from design flaws. The report provides further details on the findings and conclusions. These findings and conclusions have been used to generate a set of recommendations that is included in the report. Pilot projects should be completed. Clearly, we recommend that key pilot activities be completed as outlined in the original/modified SOW with a special emphasis on completing all documentation. This is an essential step toward knowledge sharing and the design of future initiatives based on lessons learned from GTG activities. Continued work on the enabling environment is needed. To the maximum extent possible, USAID and implementing partners should also work with the India stakeholders to ensure that the policies, regulations, and market mechanisms that have been developed under GTG be adopted so that an enabling environment is established. This environment can build the flexibility of India’s grid through new market opportunities for the private sector, for ancillary services, better forecasting, improved operating systems and equipment, and flexible energy services. In varying degrees, the completed and continuing activities under GTG have demonstrated the viability of these mechanisms within a pilot. Widespread adoption of these practices is possible only through continued action (beyond GTG) on several fronts. Particular attention should be paid to compensation mechanisms, NOAR implementation, evaluation of appropriate technologies for different locations, and cybersecurity for the grid. Future programming should emphasize stakeholder involvement. When considering future program design, special attention should be paid to including all stakeholders (especially those at the state level). USG partners should be better integrated into program design and implementation with clear channels of communication established among implementing partners. Greater private-sector participation should be encouraged by addressing private-sector concerns around cumbersome bidding conditions and opaque selection criteria. Future programming should be flexible and adaptable to changes in the operating environment. We recommend that future projects be designed for greater flexibility to modify specific activities based on a reading of ground-level realities during project implementation. The feedback loop for learning and refinement of activities should be tight and strong. Project activities should be linked to each other as part of an integrated whole with a focused agenda. Programming should be accompanied by strong monitoring, evaluation and learning frameworks. Appropriate benchmarks should be established to monitor, evaluate, and learn from progress at the systems level. Otherwise, we will continue to run the risk of delivering successful pilots that demonstrate specific technologies and methodologies without addressing the key issues around replication, scalability, and sustainability of successful project initiatives. 8 INTRODUCTION The United States Agency for International Development (USAID)/India contracted Panagora Group through the Collaborating, Learning and Adapting in India Mechanism (CLAIM) to conduct a final performance evaluation of the Greening the Grid (GTG) program. GTG is a five-year initiative of the United States–India Partnership to Advance Clean Energy Deployment. It aims to support the efforts of the Government of India (GOI) in managing large-scale integration of renewable energy (RE). The evaluation design and planning as well as the data collection and analysis were conducted remotely with limited support from the India-based team members. This Final Report includes a brief background on the GTG program and an overview of this evaluation that clarifies its purpose, as well as the evaluation questions (EQs) and sub-questions to be used in conducting the evaluation. It specifies the design and description of the methodology deployed by the evaluation team, including: • Descriptions of data collection methods and the methodology for analysis • Guiding principles under which the team operated • Limitations and potential biases of the methodology as well as ways in which the evaluation team attempted to mitigate the perceived limitations and biases ANNEXES PROVIDE: • Brief summaries of the five pilots the GTG program has implemented • A list of research documents that the evaluation team reviewed • The evaluation matrix listing data sources and methods for data collection and analysis • The data collection protocol • A list of key stakeholders who were interviewed. 9 OVERVIEW PROJECT BACKGROUND The GTG program aimed to enhance the efforts of Government of India’s to better manage large-scale integration of RE into the power grid. This goal was designed to be achieved through improved planning of RE generation, transmission, and system operation, as well as a series of innovation pilots to validate technologies and solutions that would support integration of RE into the grid. GTG innovation pilots1 were expected to involve key power-sector stakeholders, including state utilities, regulators, load dispatch centers, national power sector entities, and the Indian and external private sector. GTG combined multiple activities that interacted with each other: ● Renewable Integration and Sustainable Energy Initiative (RISE) initiative to validate technologies and solution to support RE integration through innovation pilots and technical assistance (TA). The RISE Task Order aims to support the design, implementation, and scaling of pilot initiatives for large-scale integration of RE into the power grid. It also aims to coordinate and consolidate the services that are provided under the Inter-Agency Agreement (IAA) with DOE and the buy-ins into the cooperative agreements with NARUC and USEA. Also, RISE provides monitoring, data collection, and reporting for the overall GTG program. ● An Interagency Agreement (IAA) between USAID and the U.S. Department of Energy (DOE) Laboratories to support power system planning reforms and targeted analysis for pilot programs for large RE parks and RE integration activities. ● Two India addendums (buy-ins) to the cooperative agreements with the National Association of Regulatory Utility Commissioners (NARUC) and the U.S. Energy Association (USEA) to support peer exchange for innovation by regulators and grid operators. EVALUATION PURPOSE As per the scope of work (SOW), the purpose of this performance evaluation is to provide monitoring, evaluation, and learning (MEL) services to generate learning inputs for implementing the new Country Development Cooperation Strategy. This evaluation will guide USAID/India in identifying lessons learned from GTG implementation and informing similar mechanisms USAID might design in the future. EVALUATION APPROACH The approach used by Panagora for the GTG performance evaluation is summarized in the following figure: 1 Annex 1 provides a summary of the GTG pilot initiatives. 10 EVALUATION QUESTIONS Through the findings derived from data collected over the course of this final performance evaluation, the evaluation team sought to provide conclusions and recommendations related to the following EQs and sub-questions. These questions were derived considering the technical and financial viability, sustainability, and overall scalability of the implemented pilot projects. The main EQs were developed in consultation with USAID while finalizing the evaluation SOW. In this report, we have presented a set of more granular sub-questions that were incorporated into the data collection instruments. TABLE 1 EVALUATION QUESTIONS AND SUB-QUESTIONS QUESTIONS SUB-QUESTIONS 1. To what extent has the GTG program achieved its objective of assisting the GOI in integrating large-scale, variable renewable energy (VRE) into the existing power grid? 1.1. Impacts on VRE integration. In terms of overall impacts, what has been the change in VRE integration into power grids compared to the baseline situation? 1.2. Contributions of pilots. What are the outcomes of the pilots in contributing to the overall objectives of the activity, and what are the demonstrable results on the selected utilities? 1.3. Impact on regulatory change. How effective has GTG been in supporting national and state regulatory bodies, in particular, the Forum of Regulators? What are GTG’s impacts on regulatory changes? 2. To what extent has the theory of change (TOC) for this project “to build the flexibility of India’s grid through new market opportunities for the private sector, for ancillary services, better forecasting, improved operating systems and equipment, and flexible energy 2.1. New market opportunities for the private sector. How effective has GTG been in identifying and demonstrating these opportunities? What specific GTG actions can you identify—how effective were these actions? 2.2. Ancillary and flexible energy services. How has GTG performed in this area? What specific program actions contributed to this objective? What could GTG have done better in this area? 2.3. Better forecasting. How and what did GTG contribute to better forecasting—what added benefits did the program provide to stakeholders? Who benefitted and how? Extensive desk review of available documents Cross comparison of documents against original pilots’ scope of work and international practices Set of structured KIIs for all pilot projects Cross comparison of KII responses against findings of desk review Findings, recommendations and conclusions 11 services” been validated? 2.4. Improved operating systems and equipment. What specific systems and equipment were improved under GTG and to what extent did this help provide flexibility in India’s grid? 2.5. Are there signs of measurable flexibility in India’s grid? What has contributed the most to this improved flexibility? 3. What possibilities and challenges are there for applying, replicating, and scaling up the GTG interventions—in particular, the pilot projects? Are they sustainable? 3.1. Is there evidence that the program outcomes are likely to grow, scale up and out, past the project period of implementation (sustainability)? 3.2. What are the challenges that pilots have faced and how can these be addressed during similar actions in the future? 3.3. What changes/improvements need to be made to make the program interventions more scalable and sustainable, and to achieve an enhanced development impact? 3.4. Have pilots contributed to establishing technical baselines that can be used to develop technical specifications and standards for equipment? 3.5. Have pilots allowed measurable impact on the existing market and suggested possible regulatory changes? 3.6. What recommendations can improve geographic selection, sub-sector identification, beneficiary private company selection, and resource allocation? 3.7. What additional support is required to make the program interventions —especially the pilots—sustainable? 4. How has the sector evolved in the five years since GTG was conceived and designed? Looking at this recent evolution and emerging technologies for VRE integration, what should be the focus of follow-on programming? 5. How far has GTG been successful in incorporating gender into the implementation of the overall project 4.1. Which market/technological/policy/regulatory issues should be the focus of future programming? 4.2. What should be the approach to pilot projects, integration studies, etc.? 4.3. How can this future work be made relevant for both India and the South Asia region? 4.4. Which components (pilots, studies, analyses, etc.) should future activities include so that a more uniform approach can be followed in project design and implementation by adjusting and linking program components? 5.1. Any lessons learned from the intervention regarding gender equality and the empowerment of women for future programming in the energy sector? 12 approach and the individual pilots/components? 13 METHODOLOGY The evaluation team deployed a mixed-methods approach, building on the initial document review. The approach included both quantitative and qualitative data collection methods to derive findings. Data were collected from multiple sources, and findings triangulated during the data analysis stage. The evaluation used four data collection methodologies: 1) document review, 2) analysis of GTG performance indicators and other quantitative data, 3) key informant interviews (KIIs), and 4) focus group discussions (FGDs). The evaluation team has reviewed more than 120 documents and the relevant findings, conclusions, and recommendations have been compiled as part of the desk review. Ten KIIs were completed with the implementing partners (IPs) along with 18 interviews and FGDs with GOI officials and 21 other KIIs and FGDs. Data analysis was parallel and sequential to identify emerging themes and trends for probing to strengthen findings as they emerged and to formulate conclusions as well as to test accuracy following the conclusion of initial data collection and analysis (see Annex 2). REMOTE DATA COLLECTION Given the limitations on travel during the COVID-19 pandemic, the team relied on remote data collection. Videoconferencing technology was used, and notes recorded for later analysis. The COVID￾19 pandemic prevented all face-to-face interactions. Also, several KIIs need to be rescheduled—often multiple times—to accommodate the uncertain and unanticipated workload changes of the targeted KIIs. Delays resulted in conducting the evaluation exercise, and the data-collection effort was adversely affected. Also, during several KIIs, the team was informed that the pandemic had slowed down or completely halted GTG activities in certain areas and on certain pilots, adding to existing delays. Several project activities were still ongoing while this final project evaluation was being conducted. DOCUMENT REVIEW AND PERFORMANCE INDICATOR ANALYSIS Document review involved assessing project-related literature to understand the context and underlying concept of the project, as well as to understand the roles and activities conducted by the different IPs. We reviewed the contracts/IAA/cooperative agreements; all project reporting including inception, quarterly, and annual progress reports; as well as work plans and MEL plans, evaluation and strategy documents, any background research documents on topics related to the project themes and context, and other technical documents produced by GTG, including assessments, pilot project designs, etc. Initial document review informed the development of data collection protocols for KIIs. In all, over 120 documents have been reviewed as part of this evaluation. KEY INFORMANT INTERVIEWS AND FOCUS GROUP DISCUSSIONS After the document review, the team leader, power grid integration specialist, and regulatory specialist conduct the KIIs with purposively selected samples of GTG program stakeholder groups. The team used a pre-existing data collection protocol (guide), while using a subject-driven iterative process in which 14 information was assembled transversely across successive interviews so that it could be aggregated and analyzed cohesively and consistently.i2 The evaluation team focused on: • USAID staff members, including staff members at the USAID/India Mission. • IP staff members, including GTG/RISE/Deloitte field-based and home-office staff members, NREL, USEA, and NARUC. • “Pilot Project” staff members, including companies contacted by GTG through road shows and other outreach stratified by companies contracting and non-contracting with GTG (non￾contracting companies will provide a counter-factual to understand why some interactions between GTG and companies did not come to fruition as contracts for pilots) • Partner country staff members, including relevant state and national government officials as well as staff members of state utilities The evaluation team developed a provisional list of specific respondents from each stakeholder group after the kickoff meeting with the full team. The list was shared with USAID/India for review, feedback, and eventual finalization. The purpose of the KIIs is to probe results of the document review and indicator analysis for more specific findings related to the evaluation questions. KII participants were purposively selected according to the likelihood of significant knowledge of GTG project activities, as well as convenience of access to the largest number of informants possible over the course of data collection within the limited time available to the evaluation team. To guide the KIIs, the team developed data collection protocols (interview guides) for KIIs, following initial unstructured interviews with USAID and staff of the GTG IPs. Annex 4 includes the protocols used during this evaluation. We have anonymized the notes from the KIIs and FGDs. Our initial analysis had identified a universe of 30 stakeholders (however, sometimes several people from the same organization interacted with GTG in different components/activities). Some KIIs required follow-up interviews to drill down on issues that emerged during data collection. FGDs were particularly useful in pilot projects. A list of stakeholder organizations is included in Annex 5. DATA ANALYSIS METHODS The team leader, power grid integration specialist, and regulatory specialist participated in a review of key documents and conducted KIIs with the identified groups. Under the supervision of the team leader, team members conducted ongoing basic analysis throughout document review to identify any emerging trends. These analyses were transmitted to the evaluation team as they became available3 and used to inform probing questions during subsequent KIIs. When analyses were completed, team members aggregated all data online and presented the data in a series of visualizations determined in coordination with the team leader. 2 King, Gary, Robert Keohane, and Sydney Verba, Designing Social Inquiry: Scientific Inference in Qualitative Research, Princeton University Press, 2016. 3 Annex 1 and Annex 3 summarize the results to date; review, data collection, and analysis are ongoing. 15 The evaluation team used methodological triangulation of data obtained initially during document review and analysis to develop parallel protocols with the same or similar questions across KIIs. Throughout KIIs overseen by the team leader, evaluation team members recorded data directly into audio recording software for subsequent transcription by the Panagora support team at Grant Thornton India. Also, the team members who conducted the interviews will transcribe key notes into Microsoft (MS) Word-based forms in real time, analyzing feedback daily to identify emerging trends to aggregate findings around common themes and generate further probing questions (see Annex 2: Evaluation Matrix). The team used three types of data analysis methods: • Triangulation: The evaluation team used methodological triangulation to cross-verify and cross￾validate findings that emerged from distinct data sources to identify correlations between findings related to the five evaluation questions. This method also enabled the evaluation team to strengthen potential linkages and accuracy of data in cases where results obtained through one method were less conclusive than another method. • Content Analysis: The evaluation team conducted an intensive review of KII transcripts and data to identify and highlight notable examples of GTG successes and challenges that contributed to or hindered progress against indicator targets identified through the document review and analysis. • Trend Analysis: Trend analysis enabled the evaluation team to further examine GTG progress toward targets, beyond the initial indicator analysis, over time to identify how specific exogenous and endogenous events may have contributed to the final outcomes. Data were analyzed throughout the course of this evaluation to identify initial findings and conclusions for a consultative presentation (out-briefing) with USAID/India following development of the interim evaluation report. Based on feedback during the presentation, analysis continued until submission of the draft final report. After the final report is submitted, all interview transcripts collected by the evaluation team will be made available to USAID in a format scrubbed of identifying text to protect respondent confidentiality. All audio recordings will be destroyed to protect respondent confidentiality. GUIDING PRINCIPLES The evaluation team operated under the following guiding principles: • Participation, to ensure that those affected by the project can voice their expectations, experience, learning points, and insights • Ownership, to ensure that USAID and other key stakeholders own the evaluation process • Teamwork, to ensure a diversity of approaches and seek consensus on the fundamental issues • Learning, to ensure that USAID, the evaluation team, and other stakeholders can identify and use the lessons learned and that the process contributes to clear project improvements over the remaining implementation period 16 LIMITATIONS AND POTENTIAL BIASES The evaluation methodology that was used has several potential biases and limitations that have implications for the types of findings and conclusions that can be drawn from this performance evaluation. These, and the steps the evaluation team took to mitigate them, include: • Positive response (“halo”) bias: Probing questions regarding finance issues and development outcomes may result in positive response bias, i.e., the tendency of respondents to subjectively focus on positive outcomes. The teams tried to mitigate this bias by probing for both successes and challenges to develop the most holistic picture possible of GTG program achievements as well as challenges relative to the evaluation questions. Responses were triangulated against data collected from the GTG activity documents, including contracts, progress reports, financial analyses, and other technical documents produced by the IP. • Selection bias: Selection bias is an inherent risk when implementers help to facilitate contact with members of some stakeholder groups. The team worked closely with USAID and GTG-RISE staff members and with project stakeholders to organize KIIs. However, the risk remains that GTG staff members selected the most active, responsive, or engaged individuals; the team may have heard only from key informants who reported positive experiences. To mitigate the risk of selection bias, before launching data collection, the team requested that the GTG-RISE staff members provide a universal list of stakeholders in advance of the KIIs. Subsequently, the team identified individuals from this list to contact for KIIs. • Sampling limitations: Due to restrictions on movement related to the COVID-19 pandemic, the team could not travel to field sites to conduct in-person observations. Also, due to time and resource constraints, the evaluation team could conduct KIIs only with a selected sample of each stakeholder group. While the team worked closely with the GTG staff members to identify a reasonably representative sample set of interviewees from each stakeholder group, this does not provide a comprehensive picture of GTG performance and results. This approach best served as a basis for probing around topics. • Subjective measurements: Qualitative approaches can result in performance analysis being dependent on the professional opinions and experience of the evaluation team. This may result in findings, conclusions, and recommendations being derived from the team’s subjective interpretations. The team attempted to mitigate this bias through systematic triangulation of findings across stakeholder groups and methods while drawing evidence-based conclusions and recommendations based on the data (rather than on their professional experiences). Also, where possible, the team sought out the professional opinions of relevant skilled personnel to collaborate and review findings and conclusions with the goal of improving their accuracy and soundness. 17 WORK PLAN TEAM COMPOSITION The evaluation team was comprised of five team members: Amit Bando, Senior Evaluation Specialist (Team Leader): Mr. Bando was responsible for overall implementation of the evaluation, including finalizing development of the data collection tools and ensuring that all expected tasks and deliverables are achieved on time and are of high quality. He oversaw design of the evaluation framework, including determining the methodology and organizing the schedule and meetings. He also led interviews and managed other data collection events, supervised and led data analysis with input from team members, led development of conclusions and recommendations based on findings derived from the data (using inputs from the other team members), and drafted the initial presentation of findings and final evaluation report. Velimir Lackovic, Power Grid Integration Specialist: Mr. Lackovic was responsible for providing technical contributions to the team for the latest RE integration and development practices. He was also involved in document review and indicator analysis, and preparation of KIIs and FGDs (as relevant). His technical contributions included cross-comparisons of the outputs and finding of the completed program against relevant standards, procedures, and practices. Mr. Lackovic provided independent findings, conclusions, and recommendations to the team leader, who incorporated them into the draft and final evaluation report. Antony Gerald, Regulatory Specialist: Mr. Gerald conducted a detailed review of the background documents, analyze the information, and provided written assessments of the quality, relevance, and importance of the data contained in the documents. Based on document review, he identified gaps and assisted the team leader in finalizing the data collection protocols. He supported the team’s efforts for the KIIs and FGDs. The regulatory specialist drafted sections of reports and provided comments on sections written by other team members. Also, he contributed to revising and rewriting sections of each report as warranted, based on feedback from USAID and other stakeholders. Rajesh Pamruwal and Tarini Gupta, Logisticians: Under the direction of the team leader, the logisticians were responsible for scheduling contacts with interviewees. The logistician maintained a schedule of all interviews by stakeholder group to guarantee the continuity of workflow throughout data collection (the schedule maintains the anonymity of contacts in line with USAID guidance on interviewee confidentiality). TABLE 2 TEAM COMPOSITION, LOE AND DELIVERABLES POSITION LOE DELIVERABLES Team Leader (TL) Amit Bando 64 • Develop inception report and work plan • Design evaluation methodology • Provide overall supervision of the evaluation team • Oversee data collection and analysis • Assume overall responsibility for draft and final evaluation report(s) 18 Power Grid Integration Specialist Velimir Lackovic 51 • Draft summaries of findings from document review as assigned by TL • Develop list of key informants for interviews • Provide review and input for finalizing data collection instruments as well as inputs for draft and final evaluation report(s) • Provide contextual briefings on topics related to interviews • Provide written summaries of findings from key informant interviews • Draft sections of consultative presentation as assigned by TL • Draft sections of evaluation report(s) as assigned by TL Regulatory Specialist Antony Gerald 54 • Draft summaries of findings from document review as assigned by TL • Develop list of key informants for interviews • Provide review and input for finalizing data collection instruments • Provide contextual briefings on topics related to interviews (oral) • Provide written summaries of findings from key informant interviews • Draft sections of consultative presentation as assigned by TL • Draft input for sections of evaluation report(s) as assigned by TL Logisticians Rajesh Pamruwal and Tarini Gupta 46 • Update and manage interview planner as assigned by TL • Provide written transcripts and digital recordings of key informant interviews • Provide weekly status report to TL on meetings scheduled and logistics arranged, and record management updates The evaluation team conducted this work between February 25, 2021, and July 15, 2021. The team leader and power grid integration specialist initiated the evaluation on February 25, 2021 with the launch of document review. Implementation milestones began with developing an inception report by March 19, 2021. This activity was followed by a series of remote kickoff meetings with the full evaluation team, the RISE team, and USAID/India from March through April. Data collection, including KIIs, took place April 5 through June 15, 2021, with a final out-brief to USAID/India on June 30, 2021.4 4 The overall schedule was impacted by numerous delays in schedule and cancellations that were related to the ongoing COVID-19 pandemic. This also resulted in Team LOE being used on additional logistics-related issues. 19 FINDINGS The findings and conclusions from the desk review of documents and the multiple KIIs can be grouped into those that (A) answer the evaluation questions (EQs), (B) evaluate the status of the Pilot Projects, and (C) address the status of the cross-cutting project activities and regulatory support as well as other supporting initiatives. This section summarizes each set of findings and conclusions. ANSWERS TO EVALUATION QUESTIONS EVALUATION QUESTION FINDINGS EQ1: To what extent has the GTG program achieved its objective of assisting the GOI in integrating large-scale, variable renewable energy (VRE) into the existing power grid? • The objectives cannot be considered fully met except for successful demonstration of certain pilots (isolated success5). Impact on the grid is yet to be determined. • Several activates are incomplete6 (few pilots, market, regulations, etc.). • All stakeholders have been sensitized; this has led to several ongoing discussions. • GTG provided positive impacts yet difficult to distinguish from impacts of other ongoing activities and programs.7 EQ2: To what extent has the theory of change (TOC) for this project “to build the flexibility of India’s grid through new market opportunities for the private sector, for ancillary services, better forecasting, improved operating systems and equipment, and flexible energy services” been validated? • While progress is seen on individual elements, the TOC is yet to be validated. • GTG provided support to the Central Electricity Regulatory Commission (CERC), Power System Operation Corporation (POSOCO) and Forum of Regulators (FOR). • Pilots show that better forecasting and flexible services can be achieved. • Draft ancillary services (AS) regulation published for public comments. Real￾time market (RTM) established. • Private-sector participation in most pilots, but new market opportunities need supporting regulations (e.g., compensatory mechanism). • Market creation and final regulations are still in process and need time for completion. EQ3: What possibilities and challenges are there for applying, replicating, and scaling up the GTG interventions—in particular, the pilot projects? Are they sustainable? • Good potential at pilot level but not proven for all. Single implementation does not lead to big change at the system level. Coal based flexible power generation pilot has been a success. • The parameters for “success” have not been clearly identified during project design and implementation. 8 • Pilot projects need to clearly demonstrate the added (incremental) value of the completed work and clearly separate from the added value of 5 This was noted during several KIIs; pilot projects were designed as stand-alone activities that demonstrated new U.S. technologies. Not enough thought was given to adapting to varying conditions in different parts of India. IPs and GOI KIs noted that overall system issues were not well integrated into project/pilot design. 6 This has been a key issue with several activities. Not all the delays are COVID-19 related. Many resulted from poor activity design resulting from adequate discussions with Indian counterparts during the initial phase of GTG. Many contracted activities had to be redesigned after lengthy discussions with Indian stakeholders, delaying start-up of several activities. 7 The overlap of GTG initiatives with other activities was noted by several GOI KIs. 8 The pilot implementors noted during several KIIs that they were tasked with implementing the specific pilot(s) and were not tasked with benchmarking, monitoring, and reporting on systems integration issues. 20 other programs, ongoing activities, and continuous development processes. • More focus is needed on strategic planning and technology selection9 . • Pilots need to clearly demonstrate financial sustainability; otherwise, their regulatory approval(s) may not be possible. • U.S.- based partners are not fully integrated in design and implementation; coordination and communications are inadequate.10 Same is true for Indian partners, especially those at the state level. • All stakeholders need to be involved and states needs to be targeted. • Lack of complete documentation—in many instances (e.g., battery energy storage system (BESS)—difficult to cross reference findings from KIIs. • Available documents do not provide specific data needed for a comprehensive evaluation of activities. • KIs are unable to provide documents to support their assertions. • Largely, Implementing Partners (IPs) have not considered “sustainability” as a requirement under their SOW. • No clear way forward on implementation of regulations—regulations are critical for scalability. EQ4: How has the sector evolved in the five years since GTG was conceived and designed? Looking at this recent evolution and emerging technologies for VRE integration, what should be the focus of follow-on programming? • Sector has evolved positively and rapidly. • GTG transferred proven technologies but did not introduce new ones. • All stakeholders have been sensitized—a lot of awareness creation and documentation. • Future focus on sustainability/scalability of existing technologies rather than implementing new ones. • Policy and regulatory issues need to be implemented before pilots can be scaled up. • Technology should be introduced at different state and regional grid interconnection points. • Technologies like hydrogen storage, pumped storage, and EV (Electric Vehicle) along with cybersecurity should be considered. EQ5: How has GTG helped to achieve the overarching objective of better managing large-scale integration of RE into the power grid? • “Mindset change” has been achieved—a behavioral change. • Information sharing/capacity building and consultation papers have helped discussions. 9 A major reason for delays in implementation (and resulting lack of time in replication of the pilot) had to do with the fact that a lot of time was spent on pilot design and getting partners on board. This points to a flaw in pilot design – a lack of proper mechanisms (e.g. compensation mechanisms) that would make it attractive for stakeholders to participate in the pilot activities). 10 This was noted during all KIIs with USG and U.S.-based IPs. They were rarely consulted and were often asked to step in after key decisions had been made; they were not in the communications loop. As such, they “parachuted in and out” with no involvement with GTG outside of their own specific tasks. Senior GOI KIs complemented the IPs for maintaining good communications with the GOI, keeping them informed on project progress and status. The lack of communication was felt at the activity implementation level(s). 21 • More work is needed at the state level. 11 • A lot of overlap with other initiatives and ongoing activities; needs to be avoided. • Pilots showed that control logics and algorithms along with modelling and simulation scenarios have been developed. EQ6: How far has GTG been successful in incorporating gender into implementation of the overall project approach and the individual pilots/components? • Gender charter established under South Asia Women in Energy (SAWIE) with10 founding members. • Good participation from women in the power sector at the webinars. • All stakeholder organizations seem to be giving importance to gender issues. • Female participation has increased at all levels. OVERVIEW OF FINDINGS AND CONCLUSIONS GTG had some limited success in supporting GOI in large-scale integration. GTG-RISE has clearly demonstrated that some pilots can be implemented at a larger scale – these include the work on coal flexibility. For other, there have been many challenges related to design and actual implementation. Some of the changes in the pilot scope were not properly addressed or captured - no clear documentation is in place. From that perspective the pilots had limited success. From the regulatory perspective, GTG-RISE was supposed to provide inputs, documented business cases, support decision making processes and help adopt the financial instruments and mechanisms that could be used in the future. This also includes market compensation mechanisms for ancillary services. The evaluation team cannot differentiate if any of the changes occurring in the Indian power sector over the last five years can be attributed to GTG-RISE alone. Instead, many activities are part of the ongoing POSOCO and GOI efforts. POSOCO was involved in this area even before the start of the GTG-RISE program. There has been good involvement with POSOCO on the multiple technical discussion papers and other regulatory activities - RTM market, draft AS regulation, draft report on the QCA/aggregator concept has been a success. Support to the FOR was well received especially on the SAMAST reports. On the regulatory component, the seeds have been sown through GTG-RISE and it remains to be seen how this evolves over a period. Also, the evaluation team could not determine if any of the financial reforms were completed over the program duration. From the perspective of the support to the states and capacity building, many workshops were delivered and are clearly recorded. So, significant awareness-raising can be attributed to GTG-RISE. Nevertheless, the evaluation team learnt from the KIIs that additional support at the state level is needed, failing which many of the actions are not fully implemented or sustainable. Also, instead of relying solely on the expertise of international consultants. better involvement of Indian academia and universities is needed. Many of the technologies used/introduced under the pilots are well known, but some are relatively new in Indian power sector. Implementation has shown that pilots for which local expertise is present (e.g. coal flexibility) did better than others as it was easier to provide confidence on the pilots’ 11 KIs from the GOI noted that state-level differences are significant; a successful pilot demonstration in one location cannot be viewed as a scalable activity in other locations. 22 appropriateness. Pilots that introduced relatively new technologies in the Indian power sector (AGC PV, BESS, etc.) faced with more challenges during regulatory approval and implementation. Therefore, in the future it may be advisable to solicit stronger inputs from local stakeholders and understand what is exactly needed on the ground rather than to select technologies for which there may not be sufficient interest in the community. The GOI may not be always the right stakeholder to decide on pilot selection - particularly if it is not presented with all relevant local needs and requirements. Consultations with generation companies, regulators, system operators, transmission and distribution companies (particularly at the state level) will yield better pilot design. These stakeholders should present their needs and program design efforts should address these problems. Stakeholder involvement in pilot selection was insufficient at the state level. Several KIs agree that there was insufficient involvement of involvement of stakeholders at state level. KIs are also concerned that project activities may not be sustainable and scalable if it is not adopted and approved at state level (the real strength and core of the system resides with state operators and relevant stakeholders). Given the federal structure of the Indian system, there are big differences between different states and each state has specific requirements and needs. Every state has its own regulations and policies - these differences need to be kept in mind while choosing technologies and pilots. A successful pilot in one state does not necessarily bode well for successful replication and sustainability in other state(s). This program could not be expected to address all state specific concerns - however, more effort should have been put developing a process to customize activities that cater to state-specific needs. Furthermore, pilots were selected from the NREL study. Pilot implementation considered technologies which are proven (hydro AGC, coal flexibility, and DRPC) and those that are relatively new (BESS in Transmission Systems, BESS in Distribution Systems, and AGC for PV). However, there may have been different levels of technology acceptance and awareness locally. Differences between the technology status caused different implementation challenges and due to that some of the pilot projects were either delayed or are still incomplete. Again, this is highly dependent on the pilot selection and design – in some instances, several obstacles and potential problems were not identified (e.g., no clear technical and financial criteria that could be used as benchmarks) leading to scope changes (e.g., BESS in Distribution Systems) and improper documentation. As such, many of the initial program objectives were not addressed (e.g., sustainability and scalability). Overall, the GTG program faced several major challenges that limited its success. These can be broadly divided as: (a) Design related - includes decisions that should have been made prior to commencing the projects by understanding the ground level requirements. Instead, projects were started hurriedly and only then was it realized that the project scope may not be adequate or that partners were not willing to participate in the pilot under given conditions. (b) Implementation related - primarily related to the unwillingness of bidders to participate in certain pilots due to the onerous regulations (US and Indian regulations had to be complied with) or slow approval processes. Implementation was also delayed by the Covid-19 pandemic. However, it would be unfair to attribute all delays to the pandemic. The pandemic exacerbated the existing delays. (c) Coordination related - several KIs stated that the program should not have “reinvented the wheel” – rather program activities should have been designed to supplement activities that were already 23 being implemented. KIs were not very specific about the other programs (DFID, GIZ, POSOCO, etc.) but mentioned that explicit complementarity between activities is one of the potential ways to improve future work. Also, KIs mentioned that there was inadequate support to state level stakeholders. Delays in implementation had multiple causes. Covid may have eventually made movement across the country difficult, but it remains unclear what has been done over the last four or five years in the case of pilots without major technical challenges (the evaluation team was informed by several KIs that there were no major technical challenges related to implementation of hydro AGC, as it is proven technology used around the world for years). This means that a lot of time was spent on discussion, pilot design, etc., and just a small amount of time was left for actual pilot implementation. Several pilots were designed in an ineffective manner, and the pilots suffered from faulty execution. The evaluation team was informed that the pilot on AGC PV is in the final stages of implementation and that equipment is at the site waiting for installation. However, this had not happened while the evaluation team tracked the progress of GTG-RISE. With this approach, the pilots cannot be judged against the key EQ3 questions related to scalability and sustainability. It remains unclear why the pilots were not started earlier excessive amounts of time has been spent on discussion and pilot design – very little time was left for their actual implementation. This clearly suggests that pilot design was an issue and that Covid-19 (which appeared significantly later) just contributed to the already existing delay(s). All of this is evidenced by very inconsistent and in some cases non-existing documentation (e.g., DRPC pilot), which leads the evaluation team to believe that the EQ questions, particularly those related to sustainability and scalability, cannot be properly addressed. STATUS OF PILOT PROJECTS The evaluation team has assessed the pilot projects and compared the achievements against initial objectives. Findings are summarized in Table 3 below. TABLE 3. SUMMARY OF THE PILOT FINDINGS PILOT FINDINGS PROGRESS 1. BESS in Transmission Utility - Narrow focus - Not adequately documented - No indicators of sustainability and scalability - No clear policy/market inputs - Considered as complete - No complete details on Part A and Part B of SOW12 12 Of the documents provided to the Team, none includes a “modelling study” – in addition, no study discusses cases that could be used in presentations at the CERC to support the process of decision making on ancillary service for the BESS. The evaluation team repeatedly requested access to network assessment studies or grid impact studies that could have demonstrated the impact of BESS on the transmission system (voltage recovery, frequency recovery, peak shifting applications, etc.). However, these have never been provided to the team - so it remains unclear how this support was provided to the CERC. Information on open-source modelling was not shared nor supported by relevant documentation. Also, several KIs stated that the work which is completed was done just to establish and use the BESS control algorithm (mechanism) that can be used for ancillary services, and they were not able to document how the system response improved after BESS connection. As such, there is a lack of documentation. 24 TABLE 3. SUMMARY OF THE PILOT FINDINGS PILOT FINDINGS PROGRESS 2. BESS in Distribution Utility - Information on testing, commissioning not available - Delhi Electricity Regulatory Commission (DERC) response is pending - Regulatory business case needs alignment with DERC targets - Clear framework (for design and implementation) that could be used for similar projects is missing13 - Difficult to scale up/sustain—only two feeders were considered - Considered as complete - Information was not made available (assessment of technologies, designing framework, deriving tariff for purchase, etc.). 3. Coal-based Flexible Power Generation - Major achievements in reducing technical minimums - Mindset change—success - Joint work with system operators is still needed - Absence of regulatory business case(s) for approvals from the respective Board of Directors and the appropriate Regulatory Commissions - Several strategic documents published - Recommendations for CERC were made - Financial initiatives (in particular, compensation mechanisms) are not in place—impediment to scaling up implementation - Considered as complete (Phase I and Phase II) - Regulation is in process 4. Automatic Generation Control (AGC) – Hydro and Solar - Photovoltaic (PV) AGC is still in the implementation stage (assessment against SOW is difficult). - No technical documents (drawings, specifications, studies, etc.) to support future implementation. - Not clear if pilot met initial objectives: relevant documents (technical reports and calculations, drawings and studies, etc.) were not made available for evaluation - AGC Hydro—further work on different turbine types may be needed - PV AGC is incomplete - Part B (compensation framework) of the SOW has not been implemented 13 Documents finalized after the cut-off date for document submission were not considered during the evaluation. 25 TABLE 3. SUMMARY OF THE PILOT FINDINGS PILOT FINDINGS PROGRESS 5. Dynamic Reactive Power Compensation (DRPC) for Large Solar Park Integration - Awarded in December 2020 and still not complete - Insufficient information to determine pilot’s achievements relative to the scope of work - Incomplete or missing documentation (drawings, studies, technical specifications, etc.)—prevented a complete evaluation - Items from original SOW need to be addressed (e.g., Power Quality) - No KII could be scheduled - Mainly incomplete (Parts A, B, and C) 6. Regional Platform for Reserve Sharing - Almost complete against the original SOW (Part A and Part B) - Assisted CERC in AS market regulation framework - RTM has been established - Only the physical delivery market is available, no financial market - No clear contributions from the pilot as Power System Operation Corporation (POSOCO) is doing ongoing work; no measurable effects to date - National Open Access Registry (NOAR) “go live” is planned for July 2021—not clear if this schedule will be met. - Lack of coordination at the state level, as each state has its own priorities - Mainly complete (Parts A and B) Detailed findings on each pilot project are noted below in this sub-section. Each subsection provides details on the respective pilot, and ends with a summary table of Strengths, Weaknesses, Opportunities and Threats (SWOT). 26 BESS IN TRANSMISSION UTILITY14 Following the tripping of the National Thermal Power Corporation (NTPC) Ramagundam power plant, the Final Report on Evaluation of Battery Energy Storage Systems15 (submitted in August 2020) describes reserve requirements in the case of the fault in the system for 2019 and 2020. The study concludes that frequency deviations can be reduced with BESS in service and quantifies the primary reserve requirements (from 1,412 MW to 927 MW); however, it does not comment on other key system quantities such as voltage support. The study summary16 does not comment on the initial pilot objectives, part A, which suggest that BESS at the Puducherry plant of Power Grid Corporation of India Limited (PGCIL) needs to address voltage/reactive power support, load following, peak shaving, renewable energy capacity firming, and RE time shift. A detailed project report on the BESS pilot17 provides details on proposed applications, stated above, but does not comment if any of these applications have been achieved (and to what extent). Part B of the original scope should have established the stacked value of BESS at the grid level, without trying to answer all questions regarding the “economic case” for storage. This exercise considers initial system assessment, capacity expansion optimization, and production cost optimization model with storage to reach the optimal storage capacity in the system. The report on economic valuation of grid￾connected BESS,18 submitted in February 2019, provides the scope of work but no further information is given, while the Summary Report on Evaluation of Battery Energy Storage System in Southern India,19 published in February 2021, briefly mentions that the Python tool was used for financial evaluation and concludes that “savings of approximately INR3,000 crore” (USD$ 40.2 M) can be generated annually because of the additional capacity being made available. The methodology used for these calculations, inputs, and calculation processes for the Python tools that are mentioned has not been made available. This suggests that it might be challenging to reproduce such analyses for similar projects in the future. Also, the “Battery Applications and Contacting Considerations20” presentation that was delivered by DNV-GL in New Delhi in February 2020 provides international cost ranges but does not mention how this relates to the BESS pilot at the Puducherry plant. Estimation of ancillary services for the southern region under varied scenarios of grid-connected BESS deployment21 that has been delivered to CERC in May 2020 discusses key findings, with particular focus 14 This was presented as a pilot to the evaluation team, with separate concept notes, implementation plans, etc. This was a separate pilot with clearly defined SoW (in PIP and Concept Note as well as the annual work plan and recurring integration monthly, quarterly and annual reports submitted by GTG-RISE. BESS in Distribution Utility had its own SOW (available in the concept note). So, either the technical scope was not conducted properly or the IP team has completely changed the work objectives/plans. Even if this pilot was supposed to be a set of studies, there is no clear evidence of the report and the methodologies that were used to conduct the studies. Instead, several documents were shared with the team such as a “Scalability paper”. This was shared with the team in April 2021 after the team insisted on receiving additional documents - again this document states the same SOW as the original concept note. It is not clear why after 4 years of project implementation, the same SOW is being referred to and used. If there was a change in the scope, then this should have been clearly mentioned in the documents that were provided to the team. Finally, this just supports the statement on the level of confusion and lack and lack of documentation that should be used as a basis for the future work. 15 Final Report on Evaluation of BESS, Deloitte, August 2020 16 Evaluation of BESS in Southern India – Study Summary 17 DPR on BESS pilot enhancements at Puducherry, Deloitte/GTG, February 2018 18 Greening the Grid – Renewable Integration and Sustainable energy (RISE) Initiative – Economic Valuation of Grid connected Battery Energy Storage Systems, February 2019 19 Evaluation of BESS in Southern India, Summary Report, USAID/MOP, February 2021 20 Battery Applications and Contacting Considerations, February 2020 21 Estimation of ancillary services for the southern region under varied scenarios of Grid-connected BESS deployment, Deloitte, May 2020) 27 on frequency and primary reserve support, while an additional section on value analysis provides two slides on the levelized annual fixed cost over the 15-year period. Even though the presentation does not present a methodology, it suggests is a 14 percent post-tax return on equity. Discussion with KIs revealed that an additional 100 kW PV system was combined with BESS, and it turned out as success; however, these claims have not been documented,22 indicating that it may not be straightforward to assess scalability of such activities. The same KI mentioned that completed implementation is sustainable without providing any documentary evidence that could support the claims.22 Similar assessments have also not been included in the BESS Pilot Review Board Meeting (USAID GTG-RISE Initiative) held in New Delhi in February 2019.23 A webinar on Deploying Cost-Effective Battery Energy Storage Systems into the Indian Grid24 held May 12, 2020, provides high-level BESS considerations without focusing on project implementation or conclusions. From the implementation perspective, KEC delivered its Inception Report25 in December 2018, presenting a high-level evaluation plan. This was followed by a DNV-GL presentation26 that provided a high-level discussion on BESS functions but did not comment how any of these functions will be implemented, used, and monitored at BESS at the Puducherry plant. Pilot implementation has been followed by development of a technical manual27 provided by KEC in April 2020. This is a concise document on BESS functions, particularly frequency regulation. However, this document refers to one function and does not describe all other BESS functions. Summary SWOT Analysis: BESS in Transmission and Distribution Utilities Strengths: The pilots clearly demonstrated that BESS connected to transmission and grids can be used to reduce primary reserves, stabilize frequency, reduce network losses, and potentially defer capital expenditure (CAPEX) investments. Completed financial analyses clearly demonstrate that such projects should be pursued in the future. Weaknesses: The pilot did not follow a clear implementation framework and did not provide sufficient details on additional ancillary services. No clear methodology was developed for future assessments of similar projects. 28 Basic technical design specifications for similar equipment were not developed. Testing and commissioning procedures have not been systematized—if developed, these could be improved and implemented by similar projects. 29 30 Opportunities: Explore and quantify other ancillary services that can be provided by BESS technology. Establish clear study procedures and methodologies that can be used for technical and financial evaluation. Develop 22 KII with GOI Stakeholder, April 16, 2021 23 BESS Pilot Review Board Meeting USAID GTG-RISE Initiative, New Delhi Date: February 21, 2019 24 Deploying Cost-Effective Battery Energy Storage Systems into the Indian Grid GTG Webinar #4 May 12, 2020 25 BESS Inception report, KEC, December 2018 26 Requirements for BESS applications for at a pilot site in Puducherry, DNV-GL, February 2019 27 BESS, Technical Manual, KEC, April 2020 28 Electricity Storage Valuation Framework, IRENA, March 2020 29 KII with Grantee, held May 12, 2021 30 In this case, a lack of available documents (delays in implementation) prevented a complete evaluation. 28 technical specifications and standardized drawings that should be made readily available for those pursuing similar projects in the future. Threats: Implementation could be further delayed, and the private sector may be hesitant to pursue similar projects in the future due to incomplete technical and financial evaluation methodologies as well as an uncertain regulatory framework. BESS IN DISTRIBUTION UTILITY System studies should be a basis for BESS development in distribution utilities. Such an approach is partially mentioned in the “Assessment of Battery Energy Storage System (BESS) along with battery sizing and evaluating effectiveness in distribution system”31,32, a one-page summary discussing impact deliverables and accolades (dispatch analysis and distribution system modelling, value stack analysis, etc.), but no additional details are provided. The document, Novel analytical framework,33 summarizes the collaboration between National Renewable Energy Laboratory (NREL) and BSES Rajdhani Power Limited (BRPL) by delivering the framework for two distribution feeders in the BRPL area for various scenarios of BESS and EV integration. This document sets a solid foundation, investigates a number of technical indices, and demonstrates that BESS in distribution utilities achieves minimal impact on losses reduction; selection of BESS system controls decides the value, purpose, and life of the asset; staging of the BESS deployments can be cost effective; and coupling BESS with EV leads to more system benefits. However, it is not clear if the conclusions of this study can be applied to any application or are specific only for the two feeders that were assessed. The detailed project report34 provided by BSES gives an analysis of 20 MW/40 MWh BESS at 33/11 kV Kilokari grid in Delhi. Even though the analyses presented the BESS sizing methodology, the impact on the distribution grid is only partially addressed. This is because the pilot has not been officially completed. The lack of impact on the grid is also summarized in the presentation35 delivered in July 2020. The document provides a comprehensive cost-benefit analysis through value stacking. The report suggests that “as per the findings, it may be concluded that annual levelized benefits from the project would be Rs. 63.56 lakh/MW. Therefore, an effective cost to BRPL, thus calculated, would be an annual fixed charge (AFC) of Rs. 67.44 lakh/MW, which is lower than the AFC of most existing stations in the power procurement portfolio of BRPL.” In this presentation, GTG did not use the framework for financial evaluation of the pilot that it had used in 2020. It is not clear which framework should be used for evaluating this type of project. 31 Assessment of BESS along with battery sizing and evaluating effectiveness in distribution system 32 KII with Pvt. Stakeholder, May 19, 2021 33 Preparing distribution utilities for utility-scale storage and electric vehicles, a novel analytical framework, July 2020 34 Implementation of Distribution Grid-Scale Battery Energy Storage, Detailed Project Report for 20MW/40MWh at 33/11 kV Kilokari Grid, Delhi 35 Preparing Distribution Utilities for Utility-scale Storage and Electric Vehicles 29 COAL-BASED FLEXIBLE POWER GENERATION (NATIONAL AND STATE) The pilot implementation plan submitted in August 201836 discussed Phase I (techno-economic feasibility of faster ramp rates and lower technical minimum) and Phase II (implementation of recommended measures by NTPC and Gujarat State Energy Corporation Limited (GSECL) for achieving flexibility of units) objectives, while the concept notes for NTPC37 and GSECL38 discuss possible areas of interventions, plant configurations (methods to improve control systems), and pilot costs and cost sharing options. The inception reports that were submitted in September39 and October40 2018 provide details of Ramagundam (Unit 2) and Jhajjar (Unit 1) along with examples of coal power plant cycling experiences from other countries (U.S., Germany, and United Kingdom). This is also followed by market compensation and regulation overview from the U.S., Germany, Canada, and Japan. Alternative approaches are also discussed. This approach is beneficial, as it allows cross-comparison of the international experiences against existing local practices and the potential for adopting best practices within the local context. The reports on the cost of cycling for the Indira Gandhi Super Thermal Power Project – Unit 141 and Ramagundam Unit 242, submitted in April 2019, and reports for GSECL Ukai Units 4&6, submitted in August 2019, discuss damage modelling and cost of cycling (top-down and bottom-up analyses), which are important components of the overall analyses. This has been followed by the publication of a fleet-wide strategy for NTPC coal-based plants43 in August 2019, which discusses strategies for wide-scale implementation. All of this has also been summarized in the roadmap44 prepared by the GOI, while the compensation mechanisms for flexible operation of coal thermal power plants45 were presented in July 2019. This presentation summarizes flexibilization costs, including CAPEX and operating expenses or expenditure (OPEX), as well as key recommendations, which include necessary compensation mechanisms to compensate for additional CAPEX and OPEX in identified units where unit load is reduced below 55 percent, AS market initiatives with flexible operations of coal plants, and review of long-term bilateral contracts (Power Purchase Agreements (PPAs)) at the national and state levels for addressing increased cost implications due to units under cycling. 36 Greening the Grid – Renewable Integration and Sustainable Energy (RISE) Initiative – Pilot Implementation Plan – Coal Based Flexible Power Generation Pilot 37 Greening the Grid – Renewable Integration and Sustainable Energy (RISE) Initiative – Concept Note NTPC – Coal Based Flexible Power Generation Pilot 38 Greening the Grid – Renewable Integration and Sustainable Energy (RISE) Initiative – Concept Note GSECL – Coal Based Flexible Power Generation Pilot 39 Greening the Grid – Renewable Integration and Sustainable Energy (RISE) Initiative – Inception Report – Coal Based Flexible Power Generation Pilot, NTPC, September 2018 40 Greening the Grid – Renewable Integration and Sustainable Energy (RISE) Initiative – Inception Report – Coal Based Flexible Power Generation Pilot GSECL, October 2018 41 Greening the Grid – Renewable Integration and Sustainable Energy (RISE) Initiative - Cost of cycling Report Indira Gandhi Super thermal Power Project – Unit 1, April 2019 42 Greening the Grid – Renewable Integration and Sustainable Energy (RISE) Initiative - Cost of cycling Report Ramagundam - Unit21, May 2019 43 Greening the Grid – Renewable Integration and Sustainable Energy (RISE) Initiative - Fleet wide Strategy for NTPC coal-based plants for flexible operations, August 2019 44 A Roadmap for Flexible Operation of Thermal, Gas and Hydro Power Stations to Facilitate Integration of Renewable Generation, January 2019. 45 Compensation for Flexible Operations of Coal TPPs Presentation to Central Electricity Regulatory Commission, New Delhi Date: Deloitte, July 30, 2019 (Tuesday, 12:00 pm – 1pm); 30 Summary SWOT Analysis: Coal Based Flexible Power Generation (National and States) Strengths: Pilot clearly demonstrated that coal flexible operation in India can be achieved and that has been demonstrated by conducting several tests on several coal generating units46. International experiences and procedures were collected, and they were adopted to local context. Several strategic documents that describe technical challenges and key recommendations have been published. Weaknesses: It is not clear if and to what extent the following have been accomplished: (i) Phase 2 of the pilot envisages collaborating, (ii) implementing a range of recommended interventions for changes to O&M procedures/standards and, (iii) required design modifications based on Indian grid conditions5. Basic technical specifications (including technical drawings, equipment specifications, datasheets, etc.) have not been defined and updated (along with implementation recommendations) – prevents replication and scaling up of pilot activities. Also, it is not clear if regulatory business case(s) have been developed for approvals from the respective Board of Directors and the appropriate Regulatory Commissions. In turn, this prevents assistance to the generation utilities in selecting implementing agencies and providing project management oversight that could be done by outlining pathways for devising suitable and necessary supporting regulations across India (through providing inputs to CERC/Forum of Regulators)5. Finally, further assessments of the flexible plant operation could be made by completing state-wide generation studies, including dynamic system response (frequency assessment). Absence of such analysis makes impact assessment on the overall system challenging. Opportunities: Establish compensation mechanisms for ancillary services related to units under consideration. Establish minimal O&M procedures and standards along with necessary technical considerations (functional specifications, drawings, etc.) for implementation. Understand impact on the system by completing generation modeling studies, including dynamic system response (frequency assessments). Threats: There are no major threats to this pilot, as it is in the final stages of implementation. However, CERC/Forum of Regulators still need to establish a clear framework for similar projects in the future.47 AUTOMATIC GENERATION CONTROL (AGC) For hydro and solar plants AGC is related to the overall plant integration into the grid and how the plant needs to participate in frequency regulation (primary, secondary). Depending on the plant’s given 46 KII with GOI stakeholder held April 21, 2021 47 KII with sub-contractor, held May 19, 2021. 31 role, various technical implementations are achieved.48, 49 Each approach comes with its own advantages and disadvantages and is affected by low-voltage ride through and active-reactive power requirements set in the grid code.50 These can only be understood through a set of dynamic studies that aim to understand frequency and voltage responses before and after implementing AGC functions. Typically, an initial assessment needs to be conducted before project implementation. Even though the pre-feasibility report51 on AGC at NTPC’s NP Kunta solar plant (published in September 2018) provides descriptions of the plant and communication network of the plant, little has been said about overall approach and suggestions on frequency/voltage stabilization. An updated report52 (July 2019) documents no major change. There is mention of an algorithm that will be used for AGC purposes but no mention of how that will be different and what benefits will be introduced for the plant and transmission grid. The roadmap53 for AGC that has been presented in August 2017, introduces the challenges of the pilot (India is a large synchronous area where it is not possible to test the whole AGC functionality).54 The importance of AGC for secondary control55 is fully understood—yet in 2017, the pilot was not at a stage to provide guidance on any actions since no significant results had been accomplished.56 The lack of progress on pilot implementation is also summarized at the pilot overview session (May 2018).57 Considering the above and the original scope of the pilot that asks for recording of system parameters and generator response from hydro, solar and wind plants for a duration of three to six months (Part A), it can be concluded that activities are being delayed and that clear records on the system performance are not available.58 This makes assessment of the pilot (and its success) challenging. Also, under the original scope, Part B was intended to support development of the regulatory framework for enabling AGC support from various types of generating units used in the country. This would inform CERC about necessary regulatory actions. Information on the status of the regulatory setup and the achievements of Part B of the pilot is unavailable. The technical scope for Part B should have included detailed modelling of the need and analysis of the economic output of the ancillary services using AGC system at all large-scale RE and conventional generation plants in India. To date, no 48 Active and Reactive Power Control of a PV Generator for Grid Code Compliance, Ana Cabrera-Tobar, Eduard Bullich￾Massagué, Mònica Aragüés-Peñalba and Oriol Gomis-Bellmunt 49 Debs A.S. (1988) Automatic Generation Control. In: Modern Power Systems Control and Operation. The Kluwer International Series in Engineering and Computer Science (Power Electronics and Power Systems). Springer, Boston, MA. https://doi.org/10.1007/978-1-4613-1073-0_6 50 Benchmarking of Power Control Strategies for Photovoltaic Systems under Unbalanced Conditions, Allan F. Cupertino, Lucas S. Xavierb, Erick M. S. Britob, Victor F. Mendesb, Heverton A. Pereirac. 51 Greening the Grid Pre-Feasibility Report – Pilot on Automatic Generation Control and Dynamic Reactive Control at NP Kunta Solar Park, Deloitte/GTG. 52 Greening the Grid Pre-Feasibility Report – Pilot on Automatic Generation Control and Dynamic Reactive Control at NP Kunta Solar Park, Deloitte/GTG, July 2019. 53 Roadmap for Automatic Generation Control (AGC) Pilot, August 2017. 54 Implementation in one control area is not enough to balance the whole synchronous area as the frequency does not return to the set point. Therefore, the pilot was designed to concentrate only on compensating the Partial Area Control Error of Karnataka, and the report describes the pilot implementation process. 55 Importance of AGC for secondary control and pilot modalities, August 2017. 56 KII with Grantee, held May 12, 2021 57 AGC for secondary control, Deloitte/GTG, May 2018 58 KII with GOI stakeholder, held April 16, 2021 32 cost- benefit analysis or financial assessments have been provided. It cannot be determined to what extent this pilot met the original objectives and if this set of activities it will be financially viable for large￾scale implementation. It should be noted that in India this technology was new – however, Automatic Generation Control (AGC) and particularly Dynamic Reactive Power Compensation (DRPC) are proven technologies that are being used all over the world for years. Even though the application of AGC in India could be limited, there were a number of experiences and practices that could be adopted, and benefits should have been clearly presented to the original pilot partners so that they did not feel that they had to back out. This should have been followed with adequate compensation mechanisms that would make the project worthwhile to the stakeholders. As designed/implemented, the stakeholders’ understanding may have been limited since they may not have had a clear understanding and financial incentive to participate in the pilot. This should have been known from the early start (during initial discussions) and pilots should have been designed in a way that such situations did not happen. Clearly DRPC is designed as a separate pilot, yet the evaluation team was not able to collect any supporting information and to schedule a single KII. Introducing entirely new technologies has its own challenges – it is not clear that any technical challenges existed. As far as technical implementation is concerned, it was clearly mentioned to the evaluation team by several KIs that there were no major technical challenges to complete AGC Hydro, and they have asked that support be provided for the additional turbine types as part of the future projects. For PV AGC, KIs mentioned that equipment was transported to the site and that it comes down to completing the installation. This was supposed to be finished in June 2020 but has not been done until the evaluation team tracked the progress. The team was interested in understanding how a completed AGC hydro implementation could address EQ3 related to sustainability and scalability and flexibility of the Indian system. Several suggestions that came out after discussion with KIs are that additional support may be needed for different turbine types. Even though there were no major technical challenges during AGC hydro implementation, further work may be needed so that involved stakeholders are fully capable of replicating similar work at a number of different sites. A single pilot implementation cannot make a significant change in the system and cannot make a major contribution unless it is scalable and sustainable. This is also related to compensation and financial initiatives that need to be adopted. USAID activities need to focus on scalable and sustainable implementation of the projects rather than to start the project and not take it to completion. Summary SWOT Analysis: Automatic Generation Control (AGC) for Hydro and Solar Plants Strengths: Pilot clearly demonstrated that AGC implementation for hydro and solar plants59 is possible and that existing plants can participate in AGC control. Weaknesses: Pilot did not follow clear implementation framework and did not provide sufficient details that would allow an understanding of what system parameters (voltage, frequency, reactive power, etc.) are affected 59 Activities at the solar plants are still on-going. 33 and to what extent. Simulations of the system dynamic performance were not done before and after AGC implementation; the changes could not be measured. 60 Also, there is no clear methodology that is to be followed for assessment of similar projects. Minimal technical design specifications for similar equipment, that may have to be deployed at other solar and hydro sites, were not developed. Testing and commissioning procedures have not been systematized. Opportunities: Upon completion of the pilot, records on the system response changes (frequency, voltage) could be developed to establish (i) clear study procedures (dynamic simulations which consider simulation of frequency changes by the outage of the largest plants or the loads, loss of the important interconnectors, etc.) and (ii) methodologies that can be used for technical and financial evaluation. The lack of readily available technical specifications and standardized drawings makes it difficult to pursue similar projects in the future. Also, complete financial analysis as specified in the original SOW will help fully assess the viability of this project. Threats: Implementation could be delayed, and the private sector may not be hesitant to pursue similar projects in the future due to incomplete technical and financial evaluation methodologies as well as an uncertain regulatory framework. DEVELOPMENT AND INTEGRATION OF DYNAMIC REACTIVE POWER CONTROL Successful integration of large solar parks into the grid depends on the adoption of an appropriate technical methodology.61 This methodology may include reactive power planning, static and dynamic system assessments, planning of control (logic) schemes and their adoption for large-scale system implementation. Several reactive power compensation technologies62 can be used, these include specific distinctive functions that can be used under given conditions. Reactive power analysis timeframes, assessment techniques, and reactive power sufficiency coordination63 need to be considered once these actions are planned. Even though some of these services have been included in the pilot implementation plans along with the pilot’s rationale, discussion is insufficient on cost-efficient reactive power compensation technologies.64 Similarly, the concept note document65 briefly discusses the pilot’s background (particularly related to Part A) and provides high-level cost breakdowns. However, it does not go into further details on the initial pilot requirements. A detailed project report66 submitted in March 2018 provides a brief 60 KII with GOI stakeholder, held April 16, 2021 61 Coordinated static and dynamic reactive power planning against power system voltage stability-related problems, Venkat Kumar Krishnan, Ames, Iowa, 2007 62 “Compensation Devices to Support Grid Integration of Variable Renewable Energy.” ESMAP Technical Guide, World Bank, Washington, DC. 63 (SAMS), NERC System Analysis and Modelling Subcommittee, Industry Webinar – Reactive Power Planning, 2017, North American Electric Reliability Corporation (NERC), USA 64 We were unable to conduct any KIIs related to this pilot – the IP concurred with the document findings. 65 Greening the Grid - Renewable Integration and Sustainable energy (RISE) Initiative – Concept Note – Dynamic Reactive Power Compensation (DRPC) 66 DPR – Pilot on Dynamic Compensation for Large Solar Park Integration, GTG/Deloitte 34 description of the voltage control at the interconnection point through solar inverter along with inverter control logic and plan for proposed harmonic measurements (scope Part C), as well as enhancements and retrofits in hardware and software along with the budget estimate. However, the report does not provide information on the achievements and cross-comparisons with the situation before implemented actions. The pre-feasibility report67 for DRPC at N P Kunta Solar Park that was submitted in April 2019 summarizes software upgrades needed to enhance and facilitate implementation of the DRPC mechanism at N P Kunta. The document refers to Supervisory Control and Data Acquisition (SCADA) data flow requirements and summarizes necessary system costs. Context setting68 for the pilot was presented in September 2017. This suggests that the pilot is broken into four independent stages: • Stage 1: System studies • Stage 2: Deployment, including inverter enhancement and STATic synchronous COMpensator (STATCOM) deployment • Stage 3: Evaluation STATCOM and inverter operation for reactive power compensation • Stage 4: Recommendation and scale-up However, the document does not provide information on pilot achievements to date. The document on reactive power management and voltage control in RE rich regime69 was delivered in September 2017. It discusses the overall approach on reactive power planning, dispatch and control techniques, and control devices. Case studies are provided presenting results of analyses, but these cannot be related to implementation at DRPC at N P Kunta Solar Park (one of the case studies mentions implementation for a wind farm). Part B of the initial scope asked for testing of inverter capability for providing DRPC that was supposed to involve carrying out a factory acceptance test (FAT) on a new inverter. The evaluation team could not collect information on FAT tests for the new inverters, and it remains unclear if this has been completed. Part C of the initial scope asks for power quality measurements at power of interconnection, but measurement records, used methodology, records of power quality instruments, measurement duration, and analyses of collected results were not found. 67 Pre-Feasibility Report (DPR) – Pilot on Dynamic Reactive Power Control at NP Kunta solar park, GTG/Deloitte 68 Pilot Background and Context Setting Location: Vidyut Bhawan, Jaipur, GTG/Deloitte, Date: September 13, 2017 69 Reactive Power Management and Voltage Control in RE rich Regime Location: Jaipur, GTG/Deloitte, Date: September 13, 2017. 35 Summary SWOT Analysis: Development and Integration of Dynamic Reactive Power Control Strengths: Owing to delays in implementation, project strengths cannot be completely evaluated as it is unclear if the proposed pilot addressed initial SOW and if so, to what extent70. Weaknesses: The pilot did not follow a clear implementation framework and may not provide demonstration of dynamic reactive power support from inverters (Part A). Also, testing of inverter capability for providing DRPC, which was supposed to carry out a FAT on a new inverter, is not explained in detail. Similar conclusions can be made for power quality measurements (Part C). Also, there is no clear DRPC methodology and equipment selection process that could be followed for assessment of similar projects. Technical design specifications for similar equipment were not developed while testing and commissioning procedures have not been systematized for use on similar projects. Opportunities: There is a need to quantify inverter capability to provide dynamic reactive power support during normal and dynamic system operation. It is important to understand if inverters can comply with network low-voltage ride through/high-voltage ride through requirements and if any additional reactive power compensation devices are needed to comply with the grid code regulations (leading and lagging power factor operation of the renewable energy source). No clear procedure has been followed starting with the detailed static and dynamic studies as suggested by International Renewable Energy Agency (IRENA), and the Institute of Electrical and Electronics Engineers recommendations and those that have been presented in initial pilot context. Technical specifications and standardized drawings are not readily available for those pursuing similar projects in the future. These shortcomings could be corrected. Threats: Lack of DRPC requirements may lead to unclear connection procedure for new inverter connected projects. Inverter connected generation needs to demonstrate inverter reactive power capabilities as per the grid code and if necessary, provide dynamic reactive power compensation devices. This is also highly related to the PPAs that are being signed between off-takers and developers, which define reactive power exchange at the interconnection point. Absence of such requirements may expose new developments to risk due to an uncertain regulatory framework. REGIONAL PLATFORM RESERVE SHARING Implementation of the pilot depends on the local context in India and on participation and support from relevant stakeholders but still should follow international practices and experiences. Europe provides a good example of the practices, such as electricity balancing,71 regulated by European Network of Transmission System Operators for Electricity. States can share the resources used by their 70 DRPC Pilot Implementation Plan, published in February 2020, clearly states that scope consists of three major components: Part A – Demonstration of DRPC; Part B – Testing of inverter capability; and Part C – Power quality assessment. The evaluation team does not have any documents that suggest that any of the above activities have been completed, which implies that these SOW components are not done. Also, the evaluation team was not able to schedule a single KII for this pilot. 71 https://www.entsoe.eu/network_codes/eb/ 36 transmission system operators. It is also about allowing new players such as those involved in demand response and renewables to take part in this market. The pilot implementation plan discusses pilot rationale, technical scope (Part A and Part B), benefits modelling, process of open access approval, system layers, and preliminary budget estimates. Information on the actual project implementation is not provided in the document. The summary report,72 published in December 2020, mentions that so far assistance has been provided to CERC to select market models and undertake stakeholder interaction and that the system integrator for implementing the NOAR has been appointed. The report also mentions that several activities have been completed, including: • Conducting a study on international examples of market-based procurement of ancillary services, co-optimization of energy and ancillary services (AS), and drawing lessons for the Indian context • Assisting the CERC in drafting AS regulations as well as an explanatory memorandum outlining market-based procurement, including frequency regulation, modelling, and illustrations of co￾optimization of energy and ancillary services, along with financial settlements in a day-ahead/real￾time horizon • Designing and illustrating a payment mechanism for resource providers, including opportunity costs for reserves under various scenarios • Assisting the CERC in undertaking stakeholder interactions These points from the summary report were mirrored in the NREL report73 published in August 2019 and RTM public announcement in June 2020.74 Details on the regulation or adoption of the international perspective to the local Indian context have not been provided. The same document mentions NOAR implementation stages and its benefits and concludes that a system integrator (M/s PwC (Pricewaterhouse Coopers), M/s CtrlS and M/s Wizertech) has been appointed. It is expected by the RISE team as well as PwC (the system integrator) that “NOAR Go-Live” will be achieved by July 2021.75 Even though this pilot is not fully implemented (it should have been completed by December 2020), significant work has been done in supporting CERC (Part A) in evaluating a range of interventions (market models) to enhance intra-day market operations effectively and efficiently in India and evolving a consensual way forward. The process that was used to demonstrate benefits (Python tool) has not been clearly presented or systematized for the evaluation team, but it is understood that the outcomes have been presented and discussed with CERC. Even though regulation is not publicly available and is still in process, it is expected to be completed soon and approved by CERC.76 However, the RTM launch has exceeded expectations in terms of trading volumes, sizable participation, and with encouraging results. The Draft AS regulation was published on May 29, 2021, for public comments. Support to POSOCO on NOAR has been delayed and it is expected that the implementation phase would be extended beyond the planned 12 months (July 2021). RISE Program Monitoring Support may 72 Greening the Grid – Renewable Integration and Sustainable energy (RISE) Initiative – Summary Report – Regional Platform for Reserve Sharing. 73 Opening Markets, Designing Windows, And Closing Gates India’S Power System Transition - Insights on Gate Closure, NREL, August 2019. However, if it did occur, it was after the cut/off date for reception of documents for this evaluation. 74 India’s launch of real-time electricity trading a huge step toward robust grid, efficient power market 75 KII with sub-contractor held May 13, 2021 76 KII with GOI stakeholder held April 22, 2021 37 end before the NOAR Go-Live is achieved.77 GTG-RISE was supposed to initiate, design and complete market/compensation mechanisms that would be adopted by the authorities and that could be used as a basis for scaling up the pilot implementation. However, this has not been done, which eventually makes pilot implementation difficult. Also, POSOCO is already leading the process in parallel, and it is not clear how USAID improved or eased the process. Even though it may be not realistic to achieve complete power market reforms in the given timeframe, it remains unclear how GTG-RISE contributed to the overall improvement of the existing ecosystem and how this contribution can be measured. The evaluation team was not provided with clear documentary evidence nor was it able to understand from KIIs what has been achieved beyond the work being done by GOI and POSOCO. Work that is being delivered by these programs need to be clearly separated and identified. Given the lack of a clear set the targets and expectations that can be met during program delivery, it seems that the points presented in the SOW are not completed (particularly if no clear documentation is available). The development of regulations and policies is a continuous and iterative process and may take time for consensus considering the Indian scenario. Financial market reforms are a major power sector reform area and needs deliberation. This is not identified as a limitation but as a finding. This can be a follow-up activity. Summary SWOT Analysis: Regional Platform Reserve Sharing Strengths: The pilot clearly demonstrated that intra-day market operation interventions can be adopted to the Indian local context by following and applying relevant international practices. Regulations can be adopted and used by all market participants. Also, it has been shown that NOAR can be implemented.76 Weaknesses: Pilot should have been completed on time by adopting relevant regulation (Part A) and implementing NOAR (Part B).78 79 Opportunities: Considering that adoption and use of regulations are a time-consuming and continuous process it is suggested that regulation application be monitored and updated as needed. NOAR may come with its own challenges that will be known after its continuous use. Its implementation needs to be monitored (including the benefits of such an approach) and adjusted. Threats: Since both parts of the pilot are still in progress, there is a risk that they will not be finished before program completion and that further tracking and evaluation of the pilot activities along with their advantages and disadvantages is not done. 77 Integrated quarterly performance report (Jan–Mar 2021) 78 KII with GOI stakeholder held April 27, 2021 79 POSOCO’s approval process was time consuming – this caused initial delays that were exacerbated by COVID-19 related delays. 38 STATUS OF CROSS-CUTTING, REGULATORY, AND SUPPORTING ACTIVITIES In addition to the pilots described above, several cross-cutting and supporting activities were implemented under GTG: FORUM OF REGULATORS (FOR). The FOR did significant work before GTG/RISE’s initiatives (other than SAMAST). After GTG-RISE started work, activities of various other parallel initiatives overlapped, so it was difficult to segregate the specific contributions of GTG/RISE initiatives. Nevertheless, there are some clear findings on activities under GTG. GTG-RISE supported the FOR in preparing Forecasting and Scheduling (F&S) Regulations/Deviation Settlement Mechanism (DSM) Regulations and a Detailed Project Report (DPR) for implementing the Scheduling, Accounting, Metering and Settlement of Transactions in Electricity (SAMAST) framework. SAMAST was implemented by GTG-RISE along with Idam Infrastructure Advisory (Pvt. Limited) to provide a robust, scalable, and transparent framework of scheduling, metering, accounting, and settlement of energy transactions at intrastate as well as interstate levels.80 This is an important requirement with manifold increase in VRE and resulting increases in intrastate, interstate, interregional. and transnational transactions. Therefore, the load dispatch centers need to be compatible with each other.81 With support from GTG/RISE the FOR conducted several activities, including preparing the DPR, conducting stakeholder workshops, approving the state-specific SAMAST scheme by the State Electricity Regulatory Commission, applying funding from Power Sector Development Fund, implementing the recommended IT infrastructure (hardware and software), and completing boundary metering.82 GTG-RISE also supported the FOR by facilitating discussions on developing a framework for Aggregators/Qualified Coordinating Agency (QCA) and drafting support of the QCA report. At the 16th meeting of FOR, participants deliberated technical committee issues related to QCA and model contract arrangements; at the 20th meeting, a few QCAs shared their experience of operationalizing forecasting and scheduling for renewable power projects in various states. Subsequently, a sub-group was formed with the mandate of examining 1) the feasibility of drafting a Model Tripartite Agreement between the QCA, State Load Dispatch Center (SLDC), and renewable energy generators and 2) the generic concept of Aggregator in the Power Sector. The need was established to introduce aggregator as a new player in the Indian Electricity Market. After a series of meetings, a draft report83 was published in June 2019. The following issues were addressed in the report: legal status of QCA and regulatory oversight, institutional structure, QCA interactions with 80 KII with sub-contractor held May 20, 2021 81 Report on Scheduling, Accounting, Metering, and Settlement of Transactions in Electricity (SAMAST), July 2016 82 1st meeting of reconstituted FOR technical committee – July 1, 2019 83 Report of the Sub-Group on Issues of Aggregators/Qualified Coordinating Agency (QCA) – June 2019 39 SLDC and RE generators, guidelines for model agreement, and regulating QCA and aggregators. Both technical committees accepted the draft report and recommended it for presentation before the FOR.84 Even though the FOR Technical Committee meeting presentation,85 held April 16, 2021, mentions that the QCA registration process was approved for six states (Andhra Pradesh, Gujarat, Karnataka, Maharashtra, Rajasthan, and Tamil Nadu), there is no clarity on 1) the approval process, 2) what was accepted, and 3) the status of the other states. This is also confirmed by the KII. NARUC PARTNERSHIP. NARUC along with the E3 team interacted with various stakeholders, such as CERC, Gujarat Electricity Regulatory Commission, Central Electrical Authority (CEA), POSOCO, SLDCs, DISCOMs, generators, and academicians, to identify concerns that could be addressed in the Regulatory and Market Guidelines report86 and the Regulatory Primer report.87 These reports were published by NARUC in January and February 2020. respectively. The reports were shared by the GOI and provided a U.S. perspective on priority areas for improving the forecasting, scheduling, and balancing of renewable energy in India (given that many emerging regulatory challenges in India have parallels in the U.S.). SUMMARY STATUS OF THE REGULATIONS States SAMAST DPR F&S Regulations DSM Regulations Region Proposal approved Under examination WIP*/Yet to Prepare Others Notified Draft Published WIP/Yet to Initiate Notified Draft Published WIP/ Yet to initiate North 4 1 - 2 4 - 3 7 - - West 1 1 1 2 4 - 1 3 - 2 South 4 1 - - 4 - 1 1 1 3 East 2 2 1 - 3 - 1 1 2 2 Northeast 7 - - - 5 1 2 2 - 5 Union Territories - - 7 - - - 7 - - 7 TOTAL 18 5 9 4 20 1 15 14 3 19 84 Report of Sub-Group on Framework Issues of Aggregators/ Qualified Coordinating Agency (QCA), 69th meeting of FOR, September 20, 2019 – Presentation 85 5th Meeting of Reconstituted FOR Technical Committee (Group-I and Group-II) April 2021– Presentation 86 Regulatory and market guidelines on key insights and considerations of priority areas for renewable integration in India – January 2020 87 Regulatory dimensions to renewable energy forecasting, scheduling and balancing in India – Regulatory Practices analysis and primer – February 2020 40 *WIP: Work in progress Regional Platform for Reserves Sharing pilot. The pilot included support to CERC, as part of its TA framework,88 provided in June 2019 to implement the National Open Access Registry (NOAR). The summary report89 on the Regional Platform pilot, published in December 2020, mentions that to date, assistance was provided to CERC to select market models and undertake stakeholder interaction, and that the system integrator for implementing NOAR has been appointed. The report also mentions several completed activities, including a study on international examples of market-based procurement of ancillary services, co-optimization of energy and ancillary services, and drawing learnings for Indian context; assisting the CERC in drafting the ancillary services regulations; an explanatory memorandum outlining market-based procurement, including frequency regulation, modelling and illustrations of co-optimization of energy and ancillary services along with financial settlements in day-ahead/real-time horizon; and designing and illustrating a payment mechanism for resource providers, including opportunity costs for reserves under various scenarios and assisting the CERC in undertaking stakeholder interactions. GTG also provided support on issues summarized in “Benefits modeling and Technical Assistance to CERC on Implementing Power Market reforms.” Assistance was also provided to the CERC in preparing an issue-wise summary of comments on draft regulations, preparing replies for consideration, and preparing final regulations and Statement of Reasons for the same. From the Security Constrained Economic Despatch (SCED) perspective, assistance was provided in analyzing operational aspects, understanding key issues in operation of SCED from data published by NLDC/Regional Load Despatch Centers (RLDCs), and assisting in determining cash in pool generated due to SCED operations. The benefits were illustrated through a detailed modelling exercise using a Python Model. In response to the consultation paper on SCED, the CERC issued suo-moto order to POSOCO for implementing SCED of Electricity for the Inter-State Generating Stations on a pilot basis. However, the methodology of sharing of benefits from the mechanism will be decided after the results of the pilot and the extent of savings are available. As per the summary report, benefits demonstration results have already been delivered for real-time market and Market Based Economic Dispatch (MBED). Further, the GTG team assisted in analyzing comments on draft regulations, preparing final regulations and an Explanatory Memorandum for RTM, preparing an issue-wise summary of comments received on the MBED discussion paper, analyzing the efficacy of the SCED pilot implemented by POSOCO, exploring various benefit-sharing options, and reviewing the Ancillary Services (AS) market. Draft regulations for the AS market were recently published for public comment. CERC is expected to analyze comments received on the discussion paper for Market Based Economic Dispatch of Electricity and preparation of draft regulations for the same.76 Also, support to POSOCO on NOAR is being 88 NOAR RFA Framework outline, Jun 14, 2019, GTG-Deloitte 89 Greening the Grid - Renewable Integration and Sustainable Energy (RISE) Initiative, Summary Report, Regional Platform for Reserve Sharing 41 delayed, but the system integrator has been appointed and it is expected that this RTM activity will be successfully completed. This program will enable implementation of NOAR for improved electricity trading in short time intervals, which will be in line with international markets. In June 2020, the RTM platform was launched90 and CERC has issued three papers on market re-design of RTM, AS, MBED, and day- ahead market (DAM). Hence it is evident that progress is happening on this front. On the cross-border trading (part of the original TOR) and future energy markets, there were a few discussions and presentations91 at the Asia Clean Energy Forum (ACEF) side event/webinars. No follow-up was conducted. Table 4 provides a summary of the regulatory engagement under GTG. TABLE 4: SUMMARY OF REGULATORY ENGAGEMENT FINDINGS • SAMAST DPR proposals are approved in 18 states • F&S Regulations are notified in 20 states and Deviation Settlement Mechanism (DSM) Regulations in 14 states • Multiple discussion papers on day-ahead and real-time markets submitted • NREL developed reports on India’s transitioning energy sector and facilitated dialog • Need to introduce aggregator/QCA established and draft report published • Regulatory and market guidelines report, and regulatory primer report published by NARUC • More than 25 meetings with FOR and participation of state regulators • Technical assistance provided successfully to implement and redesign the day-ahead, SCED, MBED, real- time, and ancillary markets • RTM launched and draft AS regulation published • Papers issued on market re-design of RTM, AS, MBED, and DAM • Discussions on cross-border trading through ACEF side events • No clarity of status on payment gateway for OA clearances, and establishment of clearinghouse • Block chain technology not considered feasible CONCLUSIONS • Good feedback on preparing SAMAST DPRs • SLDs have been sensitized. Handholding required at the state level for implementation • Framework is in place for regional balancing and reserve sharing mechanism but will take time • Delivery of ancillary services at the state level is not prioritized • An overlap of the work that FOR is doing (other than SAMAST) with other activities—difficult to isolate the incremental contribution of RISE initiatives • State grid codes need to be updated • No clear way forward on implementation of regulations – regulations are critical for scalability • Unable to identify incremental benefits from pilots, given overlaps with other existing initiatives • Development of regulations and policies—a continuous and iterative process—may take time for consensus National Renewable Energy Laboratory (NREL). Early on in the GTG project, NREL together with LBNL conducted a study, “Pathways to Integrate 175 Gigawatts of Renewable Energy into India’s 90 India’s launch of real-time electricity trading is a huge step toward robust grid, efficient power market, Draft announcement, June 3, 2020 91 Redesigning the Power markets: An enabler to RE Integration and multilateral trade presentation-ACEF 2020 event, June 16, 2020 42 Electric Grid, Vol. I—National Study” in close collaboration with the modeling team at POSOCO. USAID reports that this work resulted in the transfer of know-how to the POSOCO modeling team,92 as well as contributing to the selection and design of the GTG pilot projects. Also, to date NREL has completed modeling BRPL’s feeders with varying scenarios on battery energy storage and electric vehicles. These results have been presented to a stakeholder group at BRPL and jointly with BRPL to the Delhi Energy Regulatory Commission. NREL launched the report with USAID, MOP, and BRPL at a virtual event. Based on the modelling and simulation studies, two documents were released at a report launch event. The reports analyze the impact of new technologies, such as solar photovoltaics (PV), BESS, and EVs, on the distribution network. The first report, titled “Preparing distribution utilities for utility-scale storage and electric vehicles – A novel analytical framework,” has been developed by NREL, U.S., in collaboration with BRPL as part of the GTG program. The report presents a power distribution system impact analysis framework to evaluate the impact of PV, BESS, and EVs on utilities. The second document is a white paper on “Electric Vehicle Charging Infrastructure and Impacts on Distribution Network,” developed under the GTG-RISE initiative, in collaboration with BRPL. The paper analyzes key considerations for setting up EV charging infrastructure and presents an extensive review of the international experience and a prioritization framework for laying out charging infrastructure for a distribution utility.93 The two94 documents95 related to India’s transitioning energy sector were launched through a launch event and panel discussions based on the studies conducted. A DPR96 was submitted to BRPL in March 2020. The Solar Energy Corporation (SEC) of India submitted the DPR to DERC and RISE responded to DERC queries received. The report covers aspects such as design and BESS size methodology, BESS applications and use cases, cost-benefit analysis through value stacking, framework service linked agreements, and international case studies. The report thus provides a techno-commercial analysis and a business case. This pilot is considered complete as per the latest quarterly report. Regular meetings were held with GOI officials, updates on monthly progress were provided, and upcoming events were submitted to MOP, GOI. GTG Project Advisory Committee (PAC) meetings were held at regular intervals, the last one on March 2, 2021.97 Private/Sector Engagement. A charter and framework98 were established for Private-Sector Engagement (Industry Advisory Council (IAC)), which consisted of membership details, structure and voting rights, etc. IAC was established under US-India Strategic Partnership Forum (USISPF).99 A total 35 92 GTG. “Pathways to Integrate 175 Gigawatts of Renewable Energy into India’s Electric Grid, Vol. I—National Study”. GTG/LBNL. 2017. 93 KII with implementing partner held May 24, 2021 94 Preparing distribution utilities for utility-scale storage and electric vehicles – A novel analytical framework 95 Electric Vehicle Charging Infrastructure and Impacts on Distribution Network 96 Implementation of distribution grid scale battery energy storage- DPR for 20 MW/40MWh at 33/11 kV Kilokari grid, March 2020 97 Monthly Progress reports from March 2017 to November 2020 and PAC meeting minutes from July 2017 to March 2021 98 Original charter and framework_IAC – Final 99 KII with implementing partner held May 18, 2021 43 private firms have joined the IAC to date, and a few are on the pilot review boards. Various meetings and four webinars were organized. IAC was instrumental in Kreate Energy signing a Strategic Assignment with a U.S.-based wind energy consultancy company, Innovative Wind Energy, Inc.100 However, key documentation is lacking in several areas, including the governing council, online business￾to-business directory, working group formation, if any initial seed funding was contributed, and if any investments have happened between U.S.-India public/private companies. There are no documents showing if any joint ventures/memorandums of understanding/letters of intent have happened, and KIIs did not reveal answers to any of these questions. The quarterly report101 by GTG-RISE mentions a few potential loan guarantee opportunities and debt transactions with private-sector clients to enable the Development Credit Authority and multilateral banks to support scale-up of the grid integration pilots or similar initiatives for integrating large-scale VRE onto the grid. As per the latest quarterly report102 a short list of 20 potential opportunities was submitted to USAID and RISE is in the process of finalizing the deliverable as per USAID’s comments. South Asian Women in Energy (SAWIE) Forum. With the objective of strengthening gender equity, GTG-RISE proposed103 to establish SAWIE Forum to promote women’s participation and strengthen gender diversity and equity in the energy sector. Various discussions were initiated with women leaders to brainstorm SAWIE charter and activities. Specific work areas of this initiative included developing a charter and onboard members, establishing a gender-balanced steering committee, partnering with forums on women’s empowerment, developing a mentorship program for girls by engaging with academia, hosting SAWIE events, developing a sustainability plan for SAWIE beyond 2020, and ensuring a committed membership base of at least 50 SAWIE members (women leaders) by Oct 2020. A charter104 was established in March 2020 with 10 founding members105 along with a proposed work plan for 2020.106 The estimated timeline to complete this activity was September 2020, when a success story on SAWIE would be developed. A SAWIE whitepaper107 discusses the key themes and best practices that emerged from panel discussions for women who want to enter the industry and for companies striving to increase the gender diversity of their leadership teams. Other key achievements: Plaksha University expressed interest in a mentorship pillar, and Shell Foundation expressed interest in collaborating. • SAWIE hosted its first steering committee meeting on December 4, 2020. 100 GTG - Y4 - Key achievements-Final Paper 101 Integrated Quarterly Performance Report (October–December 2020), GTG-Deloitte 102 Integrated Quarterly Report (January–March 2021), GTG-Deloitte. 103 Scope of Work (IAC & SAWIE), January 13, 2020, GTG-Deloitte 104 Charter of South Asian Women in Energy (SAWIE)- A USISPF-IAC initiative, March 13, 2020 105 SAWIE- Founding Members List, March 13, 2020 106 SAWIE-Proposed work plan 2020 107 Best practices in gender mainstreaming – Bridging the gender diversity gap in the energy and industrial sector, July 2020 44 • The United Nations Industrial Development Organization invited SAWIE to share its experiences in the annual flagship Vienna Energy Forum virtual series. • The Energy and Resources Institute received a partnership interest to be knowledge partners at the World Sustainable Development Summit 2021 engagement. From the desk review and KIIs, we identified some issues in SAWIE organization and activities during GTG: • One gap found in the desk review was that the SAWIE membership base is not wide enough and engagements with academia are limited. • There was no specific documentation for power-sector governance. The information technology (IT) component is seen in most of the pilots with original equipment manufacturer and technology suppliers playing a role. However, from a Science, Technology, Innovation and Partnership perspective, documentation is insufficient, and no details are provided in any of the quarterly and annual reports. • The first annual report108 indicated the need to identify educational institutions to partner in the program. The USEA team will be conducting an Institutionalization Scoping Mission to identify an Indian-based training institution to partner with the GTG Project on system operator training and continue that training after GTG ends. Few internship opportunities from premier institutes were offered as per the second- year annual report109, No major activities were noted. • Most of the documentation is in place from an M&E/knowledge management perspective. However, the latest documents need to be updated (the quarterly reports, status of the contract deliverables with respect to the Task Order and indicators and targets, etc.). • Communication and Outreach activities comply with USAID norms. 108 Greening the grid – Integrated annual progress report (October 2016-October 2017) 109 Greening the grid – Integrated annual progress report (October 2017-October 2018) 45 CONCLUSIONS The GTG project evaluation exercise has established some overarching conclusions that can be grouped into those that are pilot specific and those related to the overall program.110 The pilot activities are: • At various stages of completion: o Only a few pilots have been completed and ready for final evaluation. At many points, the current evaluation was less of a final project evaluation and more of a mid-term evaluation of the project. o On several occasions, specific activities had to be redesigned after lengthy discussions with Indian stakeholders. As designed, the GTG initiatives were a result of consultations with several stakeholders. However, key Indian stakeholders viewed the proposed project activities as not completely suited to their needs and the ground realities in India. As such, activities had to be redesigned, resulting in significant delays in start-up and implementation111. o The COVID-19 related shutdowns have further exacerbated the problem with non￾completion of project activities. • Poorly documented: o Relevant project documents, including technical specifications (technical drawings, equipment specifications, datasheets, etc.), have not been defined and updated (along with implementation recommendations). o Technical design specifications for equipment were not developed, while testing and commissioning procedures have not been systematized for use on similar projects. o This hinders replication and scale-up of pilot activities. • Stand-alone activities: o They were designed to demonstrate the viability of specific technologies and protocols. Implementors were not tasked to ensure that individual pilot activities complement other pilot activities and/or cross-cutting activities. o Implementors were not responsible for replication and scale-up of successful initiatives. This would require a careful evaluation of state-level differences and appropriate modification of activities. 110 The earlier discussion in this section has provided the detailed findings and conclusions related to each pilot as well as the cross-cutting and regulatory/policy support activities. 111 Project activities needed to be designed in consultation with various stakeholders in India so that their requirements and needs on the ground could be fully identified and addressed. Instead of insisting on BESS or hydrogen that will most probably qualify as an emerging technology, many stakeholders suggested that energy storage could be better achieved via hydro pumped storage technology, since there is a potential to do it and there is already a base established. Also, the compensation mechanisms and financial initiatives needed to be implemented - so that generators would be incentivized to participate in the ancillary service market. 46 o To better manage the large-scale integration of RE into the Indian grid, much more work is needed at the state level to ensure that interventions are appropriately modified to reflect the needs of the specific location(s) and technology. Overall program initiatives. Activities related to regulatory and policy support have been well designed and often successfully implemented. However, these initiatives will need ongoing implementation. This is outside the control of the GTG project implementors and in the hands of relevant national- and state-level stakeholders in India. Delays have been considerable in formal approval and widespread implementation of the support activities. In other instances, such as the role of compensatory mechanisms, while the importance of such mechanisms and protocols is recognized, no progress has been made on design and implementation of critical initiatives. This is a shortcoming that needs to be addressed to replicate, scale up, and sustain GTG initiatives. Without this the GTG TOC cannot be evaluated. Cross-cutting activities. Those initiatives aimed at supporting the CERC and FOR have been successful. More needs to be done by the Indian counterparts before these initiatives can be sustained across other parts of India. Activities including those aimed at ensuring greater gender diversity in project activities were discussed and initiated but have a long way to go before they can be considered successful. Activities related to STIP have not been implemented. GTG coordinating role. A lack of regular communications on project initiatives resulted in a lack of coordination among implementing partners. This was especially true of USG implementing partners whose role could have been better integrated into GTG implementation to ensure that issues related to replication and scaling up of GTG activities were better addressed. High-level GOI counterparts complimented the GTG team for its open communications on project matters. It was the activity-level communication on project implementation details that was lacking. This situation further created activity silos. It should be noted that there was a lot of discussion and engagement on certain pilots (AGC, DRPC), that had initial design challenges - therefore a lot of interaction was needed to make some progress (this was accomplished). However, for other – non-pilot related - work the KIs indicated that additional stakeholders needed to be involved. This is particularly true for the stakeholders at state level, system operators, etc. (rather than just generator companies, etc.), since without their involvement any scalable and sustainable implementation of the pilots is not possible. State level stakeholders could include generation companies, local utilities, distribution companies, forum of regulators including state electricity regulatory commissions, regional commissions, load dispatchers, academia, research organizations and other private players in the state involved in implementation. This should have also been expanded to local universities, institutes, etc. Without wider stakeholder involvement, sustainable and scalable implementation of the achievements, particularly at the state level, may not be possible. Theory of change. Based on the findings, the theory of change for this project, “to build the flexibility of India’s grid through new market opportunities for the private sector, for ancillary services, better forecasting, improved operating systems and equipment, and flexible energy services,” cannot be validated. This due to the fact that many activities are still incomplete, and others are lacking critical documentation. 47 RECOMMENDATIONS The recommendations presented in this section include those that are: A. Pilot specific: actions that should be prioritized to ensure that GTG pilots are completed effectively and achieve the intended results B. Related to overall program initiatives: initiatives that can lead to replication, scaling up, and sustainability of key initiatives C. Program focused: initiatives that will inform future program design and implementation A. PILOT-SPECIFIC RECOMMENDATIONS: Evaluation of the overall program suggests that common recommendation can be made for similar pilot projects: • Project documents should be complete and comprehensive (GTG documents were typically generic and often missing) so that they can be easily shared and reviewed. • All project activities should be completed (e.g., AGC PV, BESS in distribution systems, DRPC) before starting any new work. This will allow for an understanding of the problems, gaps, and challenges as well as opportunities. • Every attempt should be made to replicate a completed pilot project multiple times. This will contribute to its scalability and sustainability and will help to establish it as a proven technology in multiple state-level scenarios. This replication may not be achieved within a single project. As such, future programming should address this important issue. • Pilots designed to demonstrate the viability of specific technologies and protocols should also ensure that individual pilot activities complement other pilot activities and/or cross-cutting activities. The following recommendations are specific to each pilot: 1. The BESS in Transmission Systems Pilot demonstrated that BESS connected to the transmission grid can be used to reduce primary reserves and stabilize system frequency; several improvements should be made to ensure the overall BESS integration process is more systematic: a. Establish a standardized evaluation process for BESS technologies and develop a framework to select and size similar projects to explore and quantify additional ancillary services that have been requested by the initial SOW. b. Develop a more detailed financial analysis to evaluate pre-defined tariff benefits for storage projects. 48 c. Given that a single project cannot make a difference at the system level, future projects should be designed to optimize grid-level performance. This can be achieved by replicating the same or similar projects on a number of different locations (would eventually lead to sustainable and scalable adoption of BESS technology in the transmission system). d. Some technical indicators of BESS system improvement could be:  Findings of the voltage/frequency and other key system parameters and cross comparison before and after BESS implementation. Pilot project could answer the level of improvement which could later be translated and implemented at the larger scale. e. Some financial parameters of BESS system improvement could be:  Deferral of investments  Lower reduction for spinning reserve (no need to use conventional generation anymore)  Reduced loading on the lines these transfers to lower technical losses, etc. 2. The above indicators could be used as a basis to establish technical criteria and show case improvements by cross comparing before and after situation. This has not been done - makes assessment difficult and business case may be difficult to present (and get approved by the system regulators). A clear framework needs to be developed so that it can be used by others who are trying to implement BESS in their state (part of the system). This framework would lead them through the process of demonstrating technical and financial benefits (business case)112. The BESS in Distribution Systems Pilot has not been completed as per the original SOW and has been revised to serve as an integration study (which is complete). In the future, we recommend that similar projects focus on long-term capacity planning rather than a single feeder implementation: a. Consider BESS in combination with new or existing renewable energy sources. This will allow for a cost-effective dispatch. b. Focus on useful nodes in the feeder and consider islanding services. c. Address consumer-side issues through analyses of demand response services. d. Support implementation by thorough benefit-cost analyses. 112 The GTG-RISE approach did not identify and/or quantify these indicators – as such there is no way to assess how successful a set of activities could be (there is no reference or benchmarking). 49 3. The Coal-Based Flexible Power Generation Pilot has achieved most of the initial objectives and provided support to all involved stakeholders. We recommend several activities that will need to be conducted in future projects. 4. The Automatic Generation Control Pilot cannot be considered as complete, to do. We recommend that: a. The pilot project follows a uniform and approved technical and financial methodology that throughout project implementation to establish a common platform for understanding project success (and could be easily understood by all project stakeholders). b. Future work beyond GTG focuses on the existing hydro plants with different technologies, primarily reaction turbines (Francis/Kaplan). This work will very likely need to be picked up in future projects. c. Scalability and sustainability of similar projects be emphasized by ensuring wider stakeholder participation to include generation companies, dispatchers and system regulators. All need to attend joint capacity building programs for a common understanding of the system operation. 5. The Dynamic Reactive Power Control Pilot has not been completed. We recommend that: a. All activities listed in the initial SOW be completed; in particular, the original pilot activities (points B and C) should be completed including power quality assessments. b. The project should address the reactive power compensation device selection process so that this process can be adopted on similar projects in the future. c. Implementation of activities using this equipment should be preceded by analyses from the system and strategic levels rather than as isolated pilot-implementation. The approach should be backed up with a set of system studies and would speed up development of similar projects in the future (as all stakeholders would have a clear path that needs to be followed). This will eventually ensure project sustainability and scalability. 6. The Regional Platform for Reserve Sharing Pilot has been mostly completed and we recommend that: a. NOAR is implemented on a priority basis. b. Regular follow up is scheduled with implementation agency. 50 c. The pilot provides evidence on the incremental achievements relative to other ongoing activities. Overall Recommendations on pilot projects I. Pilots should identify technical and financial parameters against which pilots’ outcomes and achievements could be compared: a. Technical parameters could include notable improvements in the system operation (decrease of outage time, improvement on system stability and flexibility, reduced technical losses, etc.). b. Financial parameters could include savings related to deferral on investments, financial benefits of decreased losses, etc. These parameters (benchmarks) should be established before project inception and project outcomes need to be tracked (monitored) against these parameters. Otherwise, projects activities may not clearly target these parameters/objectives. II. In addition, from the market development and regulatory support perspective, there could be several focused initiatives, with clearly established targets and objectives so that they could be monitored over time. These initiatives would provide incremental support over time so that the power can evolve in a gradual manner - this would avoid the need to redesign and restructure the whole power sector in few years’ time. Examples of focused initiatives could be: a. Development of the compensation mechanisms for ancillary services as well as accounting and settlement mechanisms and tools. If these are not properly structured and implemented, power sector trades will not occur (this has been the case at the state level). b. The generation companies (especially at the state level) should be made comfortable (e.g., through the establishment of compensation mechanisms and other incentives) so that there is more participation in pilot activities. III. Both the market-based mechanisms and regulations should evolve incrementally. In many cases it would be more beneficial to use “bottom up” approach to pilot design as opposed to the relatively “top down” approach used by GTG-RISE113. A “bottom-up” approach would include the following steps: a. Select states where the program will be implemented. 113 This approach has led to an overall lack of stakeholder buy-in to pilot activities. Under these circumstances, scaling and sustaining pilot activities is a challenge. 51 b. Identify problems, requirements, needs and future plans together with relevant stakeholders (generation companies, transmission and distribution operators, regulators, local academia, etc.) and understand what needs to be done to achieve the goals of the state. These goals could include (i) the need to increase penetration of renewables by 50% by 2030; and (ii) analyses of generation/consumption data to identify congestion in the feeders. c. Identify technologies that are of interest. These may not have to be new technologies such as BESS or Hydrogen – rather they could include pumped storage, upgrade of the systems at existing plants and control centers (e.g., SCADA). d. Define what needs to be done on market development and how this support is related to Item 3 above. Depending on the selected technologies, supporting market initiatives may be different. At this stage, the approach should clearly establish market rules and market mechanisms. e. Simplify the bid/tender process so that more organizations participate in the market. f. Establish a detailed plan addressing Items 3 and 4 implementation – establish benchmarks (for subsequent monitoring, evaluation and learning) and analyze the system requirements for mid to long term operational scenarios. Conduct risk assessments via alternate scenario analyses g. Proceed with pilot implementation and keep all stakeholders involved. B. RECOMMENDATIONS RELATED TO OVERALL PROGRAM INITIATIVES Successful pilot implementation is not sufficient to scale up, replicate, and sustain a program that assists the GOI in integrating large-scale, VRE into the existing power grid. Also, the TOC “to build the flexibility of India’s grid through new market opportunities for the private sector, for ancillary services, better forecasting, improved operating systems and equipment, and flexible energy services” cannot be validated through implementing stand-alone pilots. To this end, we recommend certain program-level initiatives that integrate and streamline efforts across pilot activities. These include: a. Finalizing and adopting necessary supporting regulations and policies in a timely way. b. Enhancing state level capacity to better manage grid-level integration and to ensure that interventions are appropriately modified to reflect the needs of the specific location(s) and technology. c. Designing compensation mechanisms during the planning of pilots. One of the most important conditions for pilot scalability is the need to finalize a regulatory compensation mechanism as soon as possible and expedite the implementation of various regulatory mechanisms with CERC and POSOCO. 52 d. Ensuring more widespread participation in the tendering process (especially for NOAR).114 e. Prioritizing implementation of the recommendations of SAMAST DPR. f. Aligning and integrating state-level initiatives with central initiatives to ensure effective system functionality in the long run. g. Integrating central and state interactions, activities, and regulations to be more effective; during program design and implementation, activities should not be stand-alone initiatives. h. Adequately addressing cybersecurity issues. i. Involving academia and civic society stakeholders particularly on cross cutting issues such as gender inclusion and STIM. Overall, projects (pilots) should: a. Be coordinated and should supplement each other. Instead of starting new activities with completely new objectives, it may be better to continue and build up on the conclusions of previously completed projects. This particularly refers to the financial viability of the projects as it is to be understood that they will not be sustainable and scalable without proper financial evidence and case studies. b. Focus on proven technologies and ensure that they are financially beneficial in the particular state or federal context. Otherwise, obtaining regulatory approval for wide application will be a true challenge. Only then will system regulators be able to finalize market and regulatory.115 c. Make sure that discussion papers on SCED, MBED, regional balancing and reserve sharing are completed, and the recommendations followed through till the end of the program and a plan is put in place to ensure that ongoing discussions are conducted. d. Recognize that the development of regulations and policies is a continuous and iterative process and may take time for consensus (considering the Indian scenario). 114 Repeatedly, at several KIIs, the team was told that the conditions of participation were onerous (GOI and USG protocols were involved), and the process was very time consuming – the rewards did not justify the costs of participation. 115 As has been noted in the findings and conclusions, currently, this is a key shortcoming under GTG. 53 C. RECOMMENDATIONS FOR FUTURE PROGRAMMING For future programming and project design, more focus needs to be put on strategic planning and technology selection. This means that priority should be given to proven technologies that are more appropriate to the specific energy and geographic context(s) (e.g., pumped storage, hydro). Future program design needs enhanced involvement of a wide group of stakeholders (e.g., system operators, regulatory bodies, etc.), particularly at the state level. This would help to bridge the gaps between different state agencies and thereby greatly improve the chances of sustainable adoption of successful pilot initiatives. In addition, selection of the program consultants and advisors needs to be further diversified and local universities and academia needs to be more involved during program design. Project design needs to incorporate rigorous baseline parameters, clear success indicators, as well as appropriate monitoring and evaluation metrics and approaches—otherwise quantifying and measuring program success are major challenges. If this is not done, it is also very difficult to segregate a pilot’s activities and achievements from those of other programs and initiatives—leads to questions on the rationale and benefits that are introduced by each of the programs. Success indicators need to be focused on sustainability and scalability of the technological solutions and need to use pre-defined metrices and indices. Projects need to be designed focusing on a particular problem that exists at the ground and needs to be defined in the consultation with the local stakeholders so that it fully addresses their needs. Instead for going for the latest technologies (e.g. hydrogen), more focus needs to be put on the existing proven technologies (e.g., hydro storage) that can provide considerable contribution to the overall grid flexibility and are at the same time proven and can be supported by the local expertise. Implementation of the latest technologies, that are still being tested, mainly in controlled conditions, bring a set of new challenges related to technical, financial and expert capacities that are needed for the implementation. If this is done, there may be learning curve which eventually delays projects and questions the whole approach including the funds spent. Furthermore, a bottom-up approach to project design should be adopted - this involves stakeholder participation and ground level discussions during program design. This is particularly true at the state level to ensure that stakeholder inputs are integrated into program design and subsequent implementation right from the project inception. A well-designed program based on this bottom-up approach would ensure stakeholder “buy-in” and better serve stakeholder needs. This recommended approach (the outline is enumerated in previous section of the report) is superior to a top-bottom approach in which the project design process (including the choice of appropriate technologies) is dominated by decision making from the top116. 116 Often, this process leads to project activities and choice of technologies that fail to address local stakeholder. Stakeholders feel pressured into activities that do not serve their needs and they do not have any” buy-in”, particularly when there is no accompanying financial initiative or compensation mechanism. This can lead to project delays and failures. 54 A programmatic approach needs to be adopted instead of focusing on a single, isolated pilot￾implementation that will not make any significant difference at the system level. The following are recommended for future programming: a. Improve pilot design at initial stage. A bottom-up approach (discussed in previous sections of this report) involving local stakeholders right from the design stage, will be more likely to yield sustainable outcomes. b. Consider long-term planning instead of isolated implementation. c. Target state-level stakeholders using a bottom-up approach (this approach has been elaborated earlier in this report) d. Involve all stakeholders (e.g., system operators and FOR), providing more coordination. e. Ensure sustainability and scalability by widely implementing existing technologies rather than repeatedly piloting new technologies. f. Harmonize tender requirements (national, international/regional and state level) so that more local companies can be involved. g. Develop exhaustive documentation to help regulatory commission staff members. h. Update relevant stet grid codes. Specifically, we recommend that for Coal Based Flexible Power Generation: a. The established process be scaled up and implemented on similar projects in the future this can be backed up with the financial analyses that has been completed under the pilot117. One of the key stakeholders that needs to be involved is forum of system operators. b. Assistance be provided to enhance existing plant operation simulators with flexible operation capabilities. A wider group of state operators will be trained and this will make project implementation highly sustainable. c. Future work be based on a more structured approach that is focused on the entire generation portfolio. 117 In several KIIs, participants noted that wider stakeholder group needs to be briefed on the pilot purpose, rationale, and objectives even before the pilot is started. 55 ANNEXES ANNEX 1: SUMMARY OF GTG PILOT PROJECTS IMPLEMENTED TO DATE BESS IN TRANSMISSION UTILITY This pilot project, identified by the RISE initiative, was designed to determine to what extent Indian transmission utilities can benefit from BESS installation and what preferred services could be used. Recognizing the advances made by PGCIL in commissioning the Puducherry facility, RISE held consultations with PGCIL in April 2017 to allow collaboration on this pilot. The goal was to leverage the investments being made by PGCIL and to enhance the coverage and functionality of the pilot with RISE support. PILOT DESCRIPTION: The pilot supports an initiative to test different battery technologies for various grid scale applications. Three different technologies were installed on a pilot basis: Li-ion, Advanced Lead Acid, and Flow batteries with a total of 1.25MW capacity. The pilot project is being implemented at Puducherry and all the BESS are connected to a 22KV distribution grid. Functionalities envision for this project include: - Frequency regulation - Energy time-shift - Capacity firming of VRE generators - Peak shaving/avoidance of capacity addition - Voltage support and black start - Spinning and non-spinning reserve - Congestion relief and deferral of T&D upgrades - Electric Vehicle Charging (auxiliary support) through BESS BESS IN DISTRIBUTION UTILITY This pilot project has objectives that are similar to the BESS pilot implemented by PGCIL. In this case, RISE is working with BRPL on a storage pilot project to help BRPL conduct a technical study for deploying BESS in BRPL’s distribution network. The study examines the value of BESS deployments with optimal sizing, in suitable locations on BRPL’s network. The analysis will feed into the development of the regulatory business case/framework and early investment opportunities for justifying BESS deployment and any needed viability gap funding. PILOT DESCRIPTION: This pilot involves distribution system modelling with a focus on the following aspects: • Assessment of Battery Energy Storage System effectiveness at the distribution level considering feeder load, line congestions, distribution transformer capacity/overloading, RoW, and network losses. • Modeling/simulation of BESS in distribution networks to assess the technical and financial issues. 56 • Scenario-based analysis for optimizing BESS size in the distribution network; undertaking scenario-based simulations to arrive at the most optimum size of BESS considering the load to support, load profile, battery types, number of cells in series, and determining battery capacity. • Value analysis of BESS under two scenarios 1) benefits to DISCOM and 2) benefits to consumer (regulatory business case). • A 20 MW/40 MWh BESS system has been evaluated for cost-benefit analysis. COAL-BASED FLEXIBLE POWER GENERATION (NATIONAL AND STATE) This pilot project allows the National Thermal Power Corporation (NTPC) to develop a framework for testing and implementing flexibility in its portfolio of coal-fired power plants. NTPC has offered one unit each at the Dadri (210 MW) and Simhadri (500 MW) stations to conduct the feasibility study. PILOT DESCRIPTION: The overall pilot objectives are: - To study and recommend the changes required in the power plant. - To estimate the cost of achieving flexibility including the initial capital investments required as well as the operations and maintenance expenses. - To support the development of a “business case” with associated commercial compensation mechanisms for flexibility–this will allow discussions with the Ministry of Power (MOP), CEA, and CERC. - To provide insights to the CERC on compensation for generation units to operate as flexible units. The pilot is being implemented in the following phases: - Phase 1: Techno economic assessment and business case for regulatory approval. - Phase II: (post NTPC Board approval): implementation process and monitoring. AUTOMATIC GENERATION CONTROL HYDRO AND SOLAR As outlined by the Central Electricity Regulatory Commission [CERC, USAID’s GTG-RISE initiative in collaboration with Karnataka Power Corporation Limited (KPCL), Karnataka Power Transmission Corporation Limited (KPTCL), Transmission Corporation of Andhra Pradesh (APTRANSCO), National Thermal Power Corporation (NTPC) and National Load Dispatch Center (NLDC), is implementing this pilot in the southern region as part of the roadmap to implement AGC across the country]. PILOT DESCRIPTION: Key activities under this pilot include: enhancement of existing control facilities at two hydro units (4x115 MW Varahi and 10 units x130.5MW Sharavathi, Karnataka) and two solar power plants (10 MW Shivanasamudra in Karnataka and 250 MW NP-Kunta in Andhra Pradesh). DYNAMIC REACTIVE POWER CONTROL (DRPC) FOR LARGE SOLAR PARK INTEGRATION The objective of the DRPC pilot is to demonstrate the viability of providing reactive power support to the grid from a large solar power project. The pilot will test the efficacy of existing power plant 57 equipment, as well determine cost-effective technology options in providing dynamic reactive compensation at the interconnection point. As an outcome, the pilot aims to outline the cost-benefits of enhanced solar integration into the overall grid through field-based deployment and subsequent modeling. PILOT DESCRIPTION: This pilot implementation plan is intended to further flesh out the DRPC pilot at NTPC’s 250 MW Ultra Mega Solar Power Plant at NP Kunta, Andhra Pradesh. GTG-RISE obtained NTPC’s approval to proceed with this pilot at its NP Kunta Ultra Mega Solar Park in May 2019. The pilot will enable: - Reactive power control at the solar park. - Test certain technical mandated parameters from the CEA regulation. - A comparative analysis of regulatory/market approaches followed internationally to compensate developers for participating beyond the obligatory reactive power requirement as per the relevant grid codes or equivalent technical interconnection standards. - Demonstrate learnings to other RE rich states for enabling large-scale RE integration at national level. REGIONAL PLATFORM FOR RESERVE SHARING This pilot will support CERC in the following two dimensions: - Provide technical assistance to CERC in implementation of an enhanced system for intra-day energy exchange enabling coordination amongst states. This would cover a modelling and assessment of benefits for stakeholders participating in various models of revamped intra￾day market operations. - Automate the process of granting open access and maintaining an online registry for NLDC as the nodal agency. The intended outcome of this pilot is to enable re-organization of the intra-day market with periodic collective auctions and gate closures along with substantial reduction in response time for open access approvals (thus enabling faster and shorter gate closures). It will lead to increased volumes of RE as well as balancing energy trade across states at short notices. Pilot Description: The pilot envisions the enabling of a mechanism for reserves sharing through implementing of the following tasks: - Demonstrating the benefits of a national power exchange (PX) based intra-day market (support to CERC). The scope under this task is to assist CERC in evaluating the market models in the intra-day timescales and evolve a consensual way forward on implementing the appropriate model/product in the national power market to effectively enlarge the balancing area from individual states to regions or one national market. - Automating the process of open-access approval and maintaining an online registry. The task is to establish an integrated IT-based system, accessible to all stakeholders (including open￾access participants, trade intermediaries and national/regional/state load dispatch centers 58 (LDCs’), regional power committees), with functionalities for maintaining a centralized repository of information related to open access, inter-state corridor availability, interim approvals and no objection certificates. Our data review and analyses suggest that additional information is needed to better understand the rationale, design methodologies implementation processes used as well as the results of implementing the pilot projects. The pilot projects are described in the documents summarized in Table 5. TABLE 5: ADDITIONAL INFORMATION REQUIRED ON PILOT PROJECTS – BASED ON DOCUMENT REVIEW PILOT DOCUMENT(S) ADDITIONAL INFORMATION REQUIRED Pilot 1 - Battery Energy Storage Systems (BESS) in Transmission Utility 1. Transmission system impact study (steady state, analyses, dynamic analyses, power quality analyses). This study is needed to understand how the BESS will affect performance of the transmission system in terms of voltage, frequency, and power fluctuation. How does the system respond to disturbances (three phase, line to ground faults, loss of the load, etc.), with and without the BESS? Are there any improvements? Does the BESS affect power quality in the area (harmonics, unbalance, flickers, system resonance points, etc.)? 2. Generation studies (generation adequacy) analyses. These analyses need to suggest how the BESS helps with the generation operational reserves (secondary, tertiary) and if there are any changes to the existing governor operating patterns due to this. Can typical reserves be reduced or replaced by the BESS? This should also answer the question about ancillary services and how BESS fits into that. 3. Technical specifications and standards Are any standards or technical specifications developed? This would allow application of the typical BESS solution on a wider scale. Without such documents, each project would be done with different designs which would make installation, training, operation and maintenance for the utilities more difficult. 4. FAT or SAT reports These tests would enable seeing and understanding if there is a universal testing procedure that would be applicable to future projects. This is related to the standardization process which is needed if BESS solutions are to be widely used. 5. Document explaining how BESS installation achieved all functions that it was designed for. Several documents provide a high-level overview of the BESS applications, such as frequency regulation, energy time-shift, capacity firming of VRE generators, peak shaving/avoidance of capacity addition, voltage support and black start, spinning and non-spinning reserve, congestion relief and deferral of T&D upgrades. However, there are no documents that quantify how any of these functions have been achieved and to what extent in this particular pilot. 6. Financial analysis Does the BESS installation help to reduce system operating costs? What are the costs before and after BESS installation? Pilot 2 - Battery Energy Storage 1. Distribution system impact study (steady state, analyses, dynamic analyses, power quality analyses). An understanding is needed of how the BESS will affect performance of the transmission system in terms of voltage, frequency, power fluctuation. How does the system respond to disturbances (three phase, line to ground faults, loss of the load, etc.), with and without the BESS? Are there any improvements? Does the BESS affect power quality in the area (harmonics, unbalance, flickers, system resonance points, etc.)? 59 TABLE 5: ADDITIONAL INFORMATION REQUIRED ON PILOT PROJECTS – BASED ON DOCUMENT REVIEW PILOT DOCUMENT(S) ADDITIONAL INFORMATION REQUIRED 2. Generation studies (generation adequacy) analyses. These analyses need to suggest how the BESS can help with the generation operational reserves (secondary, tertiary) and if there is any changes to the existing governor operating patterns due to this. Can typical reserves be reduced or replaced by the BESS? This should also answer question about ancillary services and how BESS fit into that. 3. Technical specifications and standards Are any standards or technical specifications developed? This would allow application of the typical BESS solution on a wider scale. Without such documents, each project would be done with different design which would make installation, training, operation and maintenance of the utilities more difficult. 4. FAT or site acceptance test (SAT) reports These tests would enable seeing and understanding if there is a universal testing procedure that would be applicable to the future projects as well. This is related to standardization process which is necessary if BESS solutions are to be widely used. 5. Document explaining how BESS installation achieved all functions that it was designed for. Several documents provide a high-level overview of the BESS applications, such as: frequency regulation, energy time-shift, capacity firming of VRE generators, peak shaving/avoidance of capacity. Along with voltage support and black start, spinning and non-spinning reserve, congestion relief and deferral of T&D upgrades. However, there are no documents that quantify how any of these functions has been achieved and to what extent in this particular pilot. 6. Financial analysis Does the BESS installation help to reduce system operating costs? What are the costs before and after BESS installation? Pilot 3 - Coal Based Flexible Power Generation (National and State) 1. Generation study This study needs to present how the operation of selected generation plants has changed and how it has impacted system performance. This needs to be assessed through participation in reserves, ancillary services, costs, etc. Has this affected any cost reduction? 2. Technical specifications and standards Are any standards or technical specifications developed? This would allow application of the implemented solution on a wider scale. Without such documents, each project would be done with different design which would make installation, training, operation and maintenance for the utilities more difficult. There is a requirement to standardize such activities. 3. Regulatory change Since this pilot affects operation at the national and state levels, has there been any changes in regulation policies? Are these changes documented and how this solution could be adopted on a wider scale? 4. Measurements and tests reports There are suggestions that measurements (e.g. damage modelling) were done but no information was provided about results implementing the solution. 5. Testing procedures It is important to know what testing procedures (load/ramp test) were used to enable solution implementation. These testing methods need to be developed and adopted as a result of this project. 60 TABLE 5: ADDITIONAL INFORMATION REQUIRED ON PILOT PROJECTS – BASED ON DOCUMENT REVIEW PILOT DOCUMENT(S) ADDITIONAL INFORMATION REQUIRED Pilot 4 - Automatic Generation Control (AGC) Hydro and Solar 1. Generation study How do selected generators participate in the system before and after implementation of the AGC? What is their participation in regulations (primary, secondary, tertiary)? What is their participation in ancillary services? Is there any evidence from conducting studies before and after implementation of such pilot? 2. Grid impact study It would be useful to understand how frequency of these units changes as a result of applied disturbance (faults, loss of load, etc.). Calculating RoFC before and after implementation of the pilot would be useful. Are there any improvements in frequency recovery? This is particularly important since it addresses renewable generation plants. 3. Standards and specifications It is not known if implemented hardware and software solutions are versatile enough to be used on a wider scale. This is particularly important since the pilot mentions that there is a plan to deploy such a solution on 55 inter-state generating stations. 4. PV plant operation It is mentioned that the power factor (PF) was changed manually but there are no clear records of any improvement on the PV plant operational procedures after implementing the pilot. 5. Compensation framework Has any compensation framework been developed after pilot implementation? 6. Regulatory framework Are there any changes to the regulatory framework? Does such plant operation of the plant violate any of the existing procedures and practices, particularly grid codes, power purchase agreements, etc.? 7. SCADA Is there evidence that the implemented SCADA system can be extended and upgraded to support wider scale application? Pilot 5 - Dynamic Reactive Power Control (DRPC) for Large Solar Park Integration 1. Grid impact study Reactive power planning (static and dynamic) can only be assessed after completion of power system studies that include steady state and more importantly dynamic performance. The aim of the studies is to understand if crucial network values (voltage, frequency) can recover after being exposed to disturbance. How does DRPC help with this and how does this affect large￾scale PV integration? 2. Technical standards and specifications Technical specifications and standards would allow wider application of such solution. Have any recommendations been made? 3. Power quality DRPC may affect power quality and it is suggested that power quality measurements be made to prove this. 4. Measurements and test What commissioning tests and measurements were done and how do they affect DRPC implementation? Is there a cross-comparison of the performance before and after implementation? 5. Operation and maintenance Have any recommendations been made for operation and maintenance, given that this this may be fairly new technology for the utility 61 TABLE 5: ADDITIONAL INFORMATION REQUIRED ON PILOT PROJECTS – BASED ON DOCUMENT REVIEW PILOT DOCUMENT(S) ADDITIONAL INFORMATION REQUIRED engineers and they may not have sufficient experience with this technology. 6. Financial mechanisms Are there any records on how DRPC affect reactive power charging mechanism at the point of connection? Typically, this is regulated between the independent power producer (in this case large scale PV) and the off-taker, but there is no records if there have been any changes in the compensation mechanisms and how any of the parties may benefit from it. 7. Inverter capability tests How does DRPC affect inverter reactive power capability? If this solution is to be widely adopted, it would be good to understand how it affects typical inverter operation parameters. 8. Regulatory change Have there been any regulatory changes due to change in the power factor improvement? Pilot 6 - Regional Platform for Reserve Sharing 1. Generation study How does reserves sharing affect generator operation in terms of reserve margins (are they reduced or increased)? Are any generators taken out of reserve, etc? Does this consider only generator constraints, or are transmission constraints included as well? 2. Communication (SCADA) How is information exchanged on a daily basis? Is the SCADA system robust enough to support this additional activity? Is the system designed in a way that it can be easily implemented on any other location? 3. Python based software Has this software been tested and verified? Have security risks been addressed? 4. Regulatory effect Have there been any changes to the existing regulation? If so, what? 5. Standards and technical specifications Are any standards or specifications for such solution adopted? This is important for scaling up the solution. 6. Cross comparison between existing arrangement and implemented platform The document needs to summarize advantages/disadvantages of each solution and to point out how implemented pilots benefits market participants. 7. Integration to external systems The document needs to summarize how communication and integration with external system can be achieved. 8. Financial analysis Is this project financially sound? What are the advantages (cost savings) from such a project? 62 ANNEX 2: SUMMARY OF RECOMMENDATIONS FOR PILOT PROJECTS TABLE 6: SUMMARY OF THE PILOT FINDINGS PILOT FINDINGS PILOT RECOMMENDATIONS 1. BESS in Transmission Utility - Narrow focus - Not adequately documented - No indicators of sustainability and scalability - No clear policy/market inputs a. Establish a standardized evaluation process for BESS technologies and develop a framework to select and size similar projects to explore and quantify additional ancillary services that have been requested by the initial SOW. b. Develop a more detailed financial analysis to evaluate pre-defined tariff benefits for storage projects. c. Given that a single project cannot make a difference at the system level, future projects should be designed to optimize grid-level performance. This can be achieved by replicating the same or similar projects on a number of different locations (would eventually lead to sustainable and scalable adoption of BESS technology in the transmission system). 2. BESS in Distribution Utility - Information on testing, commissioning not available - Delhi Electricity Regulatory Commission (DERC) response is pending - Regulatory business case needs alignment with DERC targets - Clear framework (for design and implementation) that could be used for similar projects is missing - Difficult to scale up/sustain—only two feeders were considered a. Consider BESS in combination with new or existing renewable energy sources. This will allow for a cost- effective dispatch. b. Focus on useful nodes in the feeder and consider islanding services. c. Address consumer-side issues through analyses of demand response services. d. Support implementation by thorough benefit-cost analyses. 3. Coal-based Flexible Power Generation - Major achievements in reducing technical minimums - Mindset change—success - Joint work with system operators is still needed - Absence of regulatory business case(s) for approvals from the respective Board of Directors and the appropriate Regulatory Commissions - Several strategic documents published - Recommendations for CERC were made - Financial initiatives (in particular, compensation mechanisms) are not in place—impediment to scaling up implementation a. The established process be scaled up and implemented on similar projects in the future. This can be backed up with the financial analyses that has been completed under the pilot . One of the key stakeholders that needs to be involved is forum of system operators. b. Assistance be provided to enhance existing plant operation simulators with flexible operation capabilities. A wider group of state operators will be trained and this will make project implementation highly sustainable. c. Future work be based on a more structured approach that is focused on the entire generation portfolio. 63 TABLE 6: SUMMARY OF THE PILOT FINDINGS PILOT FINDINGS PILOT RECOMMENDATIONS 4. Automatic Generation Control (AGC) – Hydro and Solar - Photovoltaic (PV) AGC is still in the implementation stage (assessment against SOW is difficult). - No technical documents (drawings, specifications, studies, etc.) to support future implementation. - Not clear if pilot met initial objectives: relevant documents (technical reports and calculations, drawings and studies, etc.) were not made available for evaluation - AGC Hydro—further work on different turbine types may be needed a. The pilot project follows a uniform and approved technical and financial methodology that throughout project implementation to establish a common platform for understanding project success (and could be easily understood by all project stakeholders). b. Future work focuses on the existing hydro plants with different technologies, primarily reaction turbines (Francis/Kaplan). c. Scalability and sustainability of similar projects be emphasized by ensuring wider stakeholder participation to include generation companies, dispatchers and system regulators. All need to attend joint capacity building programs for a common understanding of the system operation. 5. Dynamic Reactive Power Compensation (DRPC) for Large Solar Park Integration - Awarded in December 2020 and still not complete - Insufficient information to determine pilot’s achievements relative to the scope of work - Incomplete or missing documentation (drawings, studies, technical specifications, etc.)—prevented a complete evaluation - Items from original SOW need to be addressed (e.g., Power Quality) - No KII could be scheduled a. All activities listed in the initial SOW be completed; in particular, the original pilot activities (points B and C) should be completed including power quality assessments. b. The project should address the reactive power compensation device selection process so that this process can be adopted on similar projects in the future. c. Implementation of activities using this equipment should be preceded by analyses from the system and strategic levels rather than as isolated pilot- implementation. The approach should be backed up with a set of system studies and would speed up development of similar projects in the future (as all stakeholders would have a clear path that needs to be followed). This will eventually ensure project sustainability and scalability. 6. Regional Platform for Reserve Sharing - Almost complete against the original SOW (Part A and Part B) - Assisted CERC in AS market regulation framework - RTM has been established - Only the physical delivery market is available, no financial market - No clear contributions from the pilot as Power System Operation Corporation (POSOCO) is doing ongoing work; no measurable effects to date a. NOAR is implemented on a priority basis. b. Regular follow up is scheduled with implementation agency. c. The pilot provides evidence on the incremental achievements relative to other ongoing activities. d. To continue the NOAR initiative under the USAID SAREH program 64 TABLE 6: SUMMARY OF THE PILOT FINDINGS PILOT FINDINGS PILOT RECOMMENDATIONS - National Open Access Registry (NOAR) “go live” is planned for July 2021—not clear if this schedule will be met - Lack of coordination at the state level, as each state has its own priorities 65 ANNEX 3: DATA SOURCES, DATA COLLECTION AND ANALYTICAL METHODS TO BE USED TO ADDRESS EVALUATION QUESTIONS TABLE 7: SUMMARY EVALUATION MATRIX EVALUATION QUESTION DATA SOURCE DATA COLLECTION METHOD DATA ANALYSIS METHOD 1. To what extent has the GTG program achieved its objective of assisting the GOI in integrating large scale, VRE into the existing power grid? • Quarterly and annual reports • Evaluation and strategy documents • Document review and Indicator analysis • KIIs; all stakeholder groups • Triangulation • Content analysis • Trend analysis 2. To what extent has the theory of change (TOC) for this project “to build the flexibility of India’s grid through new market opportunities for the private sector, for ancillary services, better forecasting, improved operating systems and equipment, and flexible energy services” been validated? • Quarterly and annual reports • Evaluation and strategy documents • GTG pilot contracts • Document review and Indicator analysis • KIIs; all stakeholder groups • Triangulation • Content analysis • Trend analysis 3. What possibilities and challenges are there for applying, replicating, and scaling up the GTG interventions - in particular, the pilot projects? Are they sustainable? • Quarterly and annual reports • Evaluation and strategy documents • Document review and Indicator analysis • KIIs; all stakeholder groups • Triangulation • Content analysis • Trend analysis 4. How has the sector evolved in the five years since GTG was conceived and designed? Looking at this recent evolution and emerging technologies for VRE integration, what should be the focus of follow-on programming? • Quarterly and annual reports • Evaluation and strategy documents • Background research documents • Document review and Indicator analysis • KIIs; all stakeholder groups • Triangulation • Content analysis • Trend analysis 5. How far has GTG been successful in incorporating gender into implementation of the overall project approach and the individual pilots/components? • Quarterly and annual reports • Evaluation and strategy documents • Background research documents • Document review and indicator analysis • KIIs; all stakeholder groups • Triangulation • Content analysis • Trend analysis 66 ANNEX 4: RESEARCH BIBLIOGRAPHY As part of the desk study (review), the evaluation team reviewed several strategy-related documents. Findings are summarized below, grouped by key pilot activities. The focus on the pilot activities enabled development of the relevant evaluation questions used for the KIIs. The team reviewed other documents, particularly those related to quarterly and annual reporting of program progress, as well as those related to various contract modifications, to determine the extent to which the GTG program had met its objectives. The complete desk review is available as a separate document. PILOT 1 - BESS IN TRANSMISSION UTILITY DOCUMENT DESCRIPTION DATE CONCLUSIONS COMMENTS AND QUESTIONS Greening the Grid – Renewable Integration & Sustainable Energy (RISE) Initiative Concept Note – Proposed Pilot on Battery Energy Storage Systems Concept Note May 15, 2017 1. “The studies indicated that the country’s power system, with some enhancement in flexing capability of coal-based power plants, has the required technical flexibility in time frames of 15-minutes for balancing out this high quantity of VRE on the grid, with minimal curtailment.” 1. Has the generation study been done and has the least cost optimal model been implemented, e.g. Plexos model done by NREL? 2. What level of curtailment is achieved and how does this relate to the case without BESS? 2. The studies however pointed out that the value of fast responding, high￾quality energy sources for ancillary market operations could still be significant and needs to be assessed separately. 1. What studies? Are these studies available? 3. Storage could also avoid locational congestion and address power quality considerations, which can result in locational curtailment of VRE. 1. How has this been considered? 2. What power quality considerations were studied? 3. Has power quality (harmonics, flicker, etc.) been assessed? 4. The Battery Management System is currently set up to run two significant use cases outlined above; testing energy time-shift applications and studying response to frequency regulations signals. 1. Are testing reports available? 2. What are the lessons learned? 3. Have other applications been assessed, such as: - Frequency regulation - Energy time-shift - Capacity firming of VRE generators - Peak shaving / Avoidance of Capacity Addition - Voltage Support and Black start - Spinning and non-spinning reserve 67 PILOT 1 - BESS IN TRANSMISSION UTILITY DOCUMENT DESCRIPTION DATE CONCLUSIONS COMMENTS AND QUESTIONS - Congestion relief and deferral of T&D upgrades 4. Electric vehicle charging (auxiliary support) through BESS Pilot Implementation Plan-Battery Energy Storage System Pilot Implementation Plan July 19, 2018 5. Part A - Enhancement of an existing BESS facility at Puducherry: The overall scope of work includes developing and implementing control logics for different BESS applications along with required hardware/software retrofits to enable real-time monitoring of grid parameters and specified controlled operation of the BESS across the three battery technologies. 6. After successful deploying the software/hardware retrofits at the BESS in Puducherry, GTG-RISE will carry out data analysis with focus on parameterizing battery responses to external grid signals. 1. Are testing results available? 2. What gird parameters are monitored (e.g. voltage, frequency, etc.) and how do they compare with the results before BESS installation? 3. Has data analysis of BESS performance related to external grid signals been conducted? 7. Part B - Assessment of economic value of storage: The BESS pilot under Part B aims to conduct a national or regional (depending on data availability) assessment/modelling of the value of storage at the grid level with modeling efforts in the sub-15-minute timeframes to assess contribution to ancillary services, alleviation of transmission constraints, etc. The study scope also aims to include siting and sizing of grid connected battery energy storage systems in India for providing balancing support to the grid in various reserve regulation services. 1. Are results of these studies available? 2. How has BESS installation has affected ancillary services which it was supposed to impact? 3. What BESS sizing methodology was used and what studies were done to support such decisions? 68 PILOT 1 - BESS IN TRANSMISSION UTILITY DOCUMENT DESCRIPTION DATE CONCLUSIONS COMMENTS AND QUESTIONS 8. Advanced lead acid - 500kW/30 min (250kWh); Installation complete and commissioned Lithium ion; 500kW/30 min (250kWh); Installation complete and commissioned Flow; 250kW/4 hours (1 MWh); Commissioning by July 2018 1. Commissioning appears to have started in 2017 but was completed in 2018. Have there been any other activities in the meantime or is it just a result of a delay? 9. Data outputs from the enhanced BESS pilot facility (Part-A) would be used for parameterizing response under different operating scenarios and would be a valuable input to modelling studies for establishing the systemic value at the grid level. 1. Studies should have been done before the actual implementation. 2. What operating scenarios were considered? 10. It is proposed to appoint a technically qualified entity as the system integrator through the grants mechanism under the contract for enhancing the existing BESS Pilot facility at Puducherry covering the two tasks outlined above. 1. What is the benefit of this program if an additional technically qualified entity needs to be employed? 2. What would be the specific scope of the system integrator? 11. The scope of work under this component includes a modelling study to establish the stacked value of BESS at the grid level with a focus on harnessing grid services in sub-15- minute time scales. 1. Is this study available? 12. Assessment of economic value of storage: The specific focus is to identify and quantify the value that battery energy storage systems have in providing grid services, particularly in the sub-15 minutes time scales in today’s and 2022 forecasted grid operations. 1. What are these benefits? Are there any supporting documents? 69 PILOT 1 - BESS IN TRANSMISSION UTILITY DOCUMENT DESCRIPTION DATE CONCLUSIONS COMMENTS AND QUESTIONS 13. Assessment of Economic Value of Storage: NREL, which has conducted grid modelling studies at the national and regional level for the Indian power system will be a key part of the systemic value assessment to help in developing the scope of work for Part B, defining the framework for the analysis and in peer reviewing the outputs of the international firm that conducts the value assessment. 14. The scope of work will include a detailed modelling study on the need, quantity, and location of storage systems in India with specific focus on the sub-15 minutes time scale. The study would thus be an important input to identifying ancillary products aligned with storage systems. 15. The scoping of this study is yet to be undertaken as RISE continues to analyze the framework adopted in international instances of such value assessment at the system level. 1. This suggests that the study has not been completed. PILOT 2 - BESS IN DISTRIBUTION UTILITY DOCUMENT DESCRIPTION DATE CONCLUSIONS COMMENTS AND QUESTIONS Pilot Implementation Plan - BESS in Power Distribution Utility Pilot Pilot Implementation Pl March 16, 2020 16. The 33/11 kV sub-station at Kilokari has been identified for deploying of the BESS system. This substation is among the oldest stations of Delhi and the land is owned by BRPL. Multiple site surveys have been undertaken to design the placement and interconnection of the BESS at the substation assets. The table 1. Has the 20 MW/40 MWh BESS actually been implemented at the site? 70 PILOT 2 - BESS IN DISTRIBUTION UTILITY DOCUMENT DESCRIPTION DATE CONCLUSIONS COMMENTS AND QUESTIONS provides details of the site proposed for BESS installation. 17. Sizing BESS with evacuation design: To assist BRPL in optimally sizing the BESS based on BRPL’s ramping requirements for the future. 18. Assessing resources and technology 19. Analyzing project cost estimates and benefits: Deloitte propriety model D.VAST will be used to evaluate the costs of the BESS and the benefits accrued by ramping energy requirement, peak shifting, capacity deferral, transmission loss reduction and reduction of outages due to operation of BESS. 20. Designing of Framework BESS PPA with Service Level Agreement: Provide assistance to the Solar Energy Corporation of India and BRPL on finalizing the terms of the Power Purchase Agreement with Service Level Agreements based on international best practices. 21. Deriving the tariff for purchase of BESS Services 22. Preparing a regulatory business case to be submitted to DERC for kind approval 1. Has this been done? 2. Are these documents available? 1. D.VAST has been developed on a robust python-based open-source platform which easily integrates with standard, industry recognized software for power flow simulations such as PSSE and Open DSS. Values from power flow studies are integrated to assess benefits from CAPEX deferral and loss reduction to arrive at a final cost benefit assessment for BESS deployment. 1. If such analysis was done, what are the results and conclusions? 2. In addition to these data points, the existing power purchase agreements, upcoming/under-construction thermal projects, future RE generation growth, and RE generation profile would be used to project supply scenario to meet future demand projections. 1. There are no clear data on this. Has this been done and included in the analyses? Concept Note: Pilot on Battery Conc ept N Febr uary 24 - Benefits from ramping support - Benefits from peak shaving/ energy shifting application 1. Have listed benefits been considered? 71 PILOT 2 - BESS IN DISTRIBUTION UTILITY DOCUMENT DESCRIPTION DATE CONCLUSIONS COMMENTS AND QUESTIONS Energy Storage System (BESS) in Distribution Network - Benefits from transmission capacity upgrade deferral and loss reduction 3. Two main scenarios have been developed using a model developed in Python Programming Language. The two scenarios contribute understanding value of benefits from BESS for DISCOM and Consumers respectively. 1. Are results of these analyses available? 4. Key insights derived from the study are as follows: - Dispatch analysis and distribution system modelling - Value stack analysis – o CAPEX deferral, ramping support, peak shifting (Regulatory b-case) o DSM penalty reduction, capex deferral, peak shifting (Discom b-case) - Identification of locations for placement of BESS along with sizing estimates - Regulatory business case, petition, etc. 1. There are no results of these analyses. PILOT 3 - COAL BASED FLEXIBLE POWER GENERATION (NATIONAL AND STATE) DOCUMENT DESCRIPTION DATE CONCLUSIONS COMMENTS AND QUESTIONS 72 Minimum load/ramp test procedure for coal based thermal power plants Technical book April, 2020 1. The purpose of the minimum load/ramp test procedure is to support Coal-Based Thermal Power Plants (TPPs) across the country in preparing and conducting low load and cyclic operations to facilitate integration of renewable energy (RE) into the grid. Even though testing procedures are described there are no test results or descriptions of how selected power plants benefit from the implemented pilot. Transition towards flexible operations in India – Coal￾based flexible power generation pilot Summary report October, 2020 1. Damage modeling - Cycling-related damage for any component in a power plant can be estimated by direct damage modeling. This type of modeling combines physical measurements (e.g., pressure, temperature, strain, and heat flux) taken while the component is online, with state-of-the-art stress analyses and damage algorithms to produce a detailed estimate of the amount of damage suffered by the particular component. 1. It is not clear how this has been done. 2. The pilot’s Phase 2 (2019-2020) focused on designing low-load test run procedures and conducting pilot test runs for technical minimums at the target units. The objective was to identify the applicability of the test run procedures and witness the responsiveness of the plant equipment subject to the low loads on a sustained basis. This pilot phase also emphasized the need to create enabling frameworks through regulatory and policy interventions. 1. A description of the running procedures is available; however it is not clear how these have been implemented for the particular pilot. 3. Detailed recommendations were provided in unit wise reports submitted to NTPC and GSECL. The findings, analysis, and recommendations were also shared with these utilities in two (2) day knowledge dissemination workshops. Few of the recommendations have already been implemented/under implementation by NTPC and GSECL. 1. This information is not currently available. 73 PILOT 4 - AUTOMATIC GENERATION CONTROL (AGC) HYDRO AND SOLAR DOCUMENT DESCRIPTION DATE CONCLUSIONS COMMENTS AND QUESTIONS Pilot Implementation Plan - Automatic Generation Control Pilot Implementation plan August 4, 2018 1. Part A: Demonstration of AGC pilot on hydro and RE plants in Southern India - Key activities in this pilot include upgrade of software and communication infrastructure to enable AGC support from hydro, solar, and wind units for a period of three to six months with recording of system parameters and generator response. Software updates will be required at both the generation and system operator ends and adequate SCADA and communication links are required to be established. 1. What progress has been made on this activity? Has it been implemented? 2. Part B: Compensation framework for enabling AGC support and support in national rollout. Part B of the pilot will focus on analysis and development of a proposed compensation framework for enabling AGC support from various types of generating units in the country, which will inform CERC in framing necessary regulations. 1. Has this activity progressed? 2. Has regulation been set? 3. For this part of the scope, support from NREL will be leveraged in international experiences on pricing and compensation. 1. What experiences have been shared by NREL? 4. GTG-RISE has offered its services to POSOCO for assessing pre-feasibility of AGC deployment at about 55 inter-state generating stations, which form the target set for national roll-out of AGC. 1. There are no records that this has been started. 5. AGC at 10MW solar power plant, Shivanasamudra, Karnataka. Currently, the operators at the 10 MW Solar power plant at Shivanasamudra, Karnataka, are changing the inverter power factor settings manually at the inverter panels as per the requirement from the KLDC in Bangalore 1. What is the typical power factor and how does reactive power support changes with PF change? 6. The Grantee needs to provide the following facilities in accordance with the scope of work. - Communication between remote terminal unit RTU at KPTCL 66 kV Shiva substation and Solar plant RTU. - SCADA software update at Solar plant - Configure the inverters from manual mode to auto mode as per the inputs 1. Have any of these been completed? 2. Are records available? 74 from plant SCADA software to control inverter active power generation. 7. AGC at Varahi hydro power plant, Karnataka. Currently, at Varahi hydro power plant, operators are changing the generator output set points manually as per the information from LDC Bangalore over telecommunication. 1. What is the typical power factor and how does reactive power support changes with PF change? 8. The demonstration of AGC pilot at different location requires additional hardware (telemetry and communication systems) and software (control logics and human-machine interface) enhancement. The list of existing hardware/software and required hardware/software (estimated) to be deployed by the Grantee is provided in table based on recent field visits carried out by GTG-RISE team. 1. Any additional details on this? 9. Part B, Development of Compensation Framework for AGC support - The scope of work includes a detailed modelling study on the need and economic output from the ancillary services using the AGC system at all RE and conventional generation plants in India at large scale 1. There are no records that this has been done. There is no information on modelling procedure, measured benefits (savings), etc. 10. The data analysis outputs from enhanced AGC pilot facility would be used for parameterizing the attribution from AGC support to the grid with and conducting sensitivity studies while developing compensation framework for AGC service at large. 1. Details on compensation framework? Implementation of Automatic Generation Control at Hydro Power Plants in Southern India White paper September 4, 2019 11. This pilot communication network includes plans to enhance at plant side, two-way communication facilities to receive the AGC signals from SLDC and observe the response of hydro units as per the AGC signals. The pilot was initiated in the year 2018 and operated for three months from September 2019 to November 2019. The outcomes of the pilot will be analyzed to develop the framework guidelines for ancillary market mechanism focused on secondary reserves in India. 1. White paper has been published before the pilot is completed? 2. Have any results of sensitivity analyses been evaluated in terms of developing framework guidelines for ancillary market mechanisms? 12. White paper mentions dynamic simulation model. 1. Is this model available? What is the modelling scope? 13. White paper mentions plants of 1720 MW, 1700 MW, 1035 MW. 1. Where do these plants fit in into the discussed pilot? 75 PILOT 5 - DYNAMIC REACTIVE POWER CONTROL (DRPC) FOR LARGE SOLAR PARK INTEGRATION DOCUMENT DESCRIPTION DATE CONCLUSIONS COMMENTS AND QUESTIONS Pilot Implementation Plan - Dynamic Reactive Power Compensation (DRPC) Pilot Implementation Plan February 14, 2020 1. Assessment of dynamic reactive power support capability of the solar inverters for supporting RE interconnection point voltages during peak and off-peak generation from NTPC’s power block. 1. Was a dynamic study done? 2. Have any measurement or tests been done? 2. This evaluation will provide an independent field level assessment of technology options and the most optimal choice of reactive compensation with solar parks. 1. What options were considered (Static VAR Compensator, STATCOM, capacitor banks)? 3. The pilot implementation is expected to establish techno-commercial pathways for regulators and state transmission utilities/system operators to design the most cost-effective mechanism to address voltage stability issues due to large RE parks. 1. There is no indication that this was done. 4. The pilot will also include the power quality (PQ) assessment to measure the quality of output of NP Kunta solar power plant at Power Control Center (PCC) (33kV) and analyze and assess the impact of Power Quality (PQ) on grid to identify the need for PQ enhancement. 1. There is no indication that a power quality assessment (harmonic analysis, frequency scans, flicker analyses) was done. 5. Part A, DRPC; The pilot scope includes the demonstration of dynamic reactive power support from inverters for supporting RE interconnection point voltages during peak and off-peak generation at NP Kunta solar park through Aggregator software. It will also test the associated standards specified in “Technical Standard for Connectivity to the Grid” regulation 2019, issued by CEA in February 2019. 1. There is no indication that this has been done. Inverter reactive power capability has not been assessed, and no background information is available. Cross comparison against CEA regulation is not mentioned. 6. Part B: Testing of inverter capability for providing DRPC; The Part B involves carrying out factory acceptance test (FAT) on a new inverter (having similar technical specifications as already deployed at NP Kunta Solar Park for rating of 1 MW). 1. There is no indication that these tests were done or that test information is available. 7. Part C: The scope of Part C under DRPC is the Power Quality Assessment sub-pilot. It includes detailed measurement of power quality and its impact at PCC of the Solar Plant i.e., 33 kV bus, and the output of the three to five sampled inverters of the solar plant, which will be used in developing of impact mitigation strategies. 1. A power quality assessment and cross comparison against International Electrotechnical Commission (IEC) standards are not available. 76 8. The measurement will be conducted for a minimum period of 4 weeks to record the PCC point current harmonics, flicker, steady state voltage variation, voltage unbalance and DC current injection with the wind/solar farm in service and a detailed analysis of the power quality will be performed with reference to the technical standards, grid codes and IEC standards. 1. Results of these tests are not available. 9. Development of Pilot Case Study - July 2020 - Pilot Case Studies. 1. If completed, is this study available? PILOT 6 - REGIONAL PLATFORM FOR RESERVE SHARING DOCUMENT DESCRIPTION DATE CONCLUSIONS COMMENTS AND QUESTIONS Pilot Implementation Plan – Regional Platform for Reserves Sharing Project Implementation Plan September 30, 2018 1. The simulation will take into account the difference of day-ahead schedule and actual demand met by the states in the Western / Southern region (Andhra Pradesh, Telangana, Karnataka, Maharashtra, Madhya Pradesh, Chhattisgarh) and meet this difference through available un-requisitioned surplus (Inter-state + Intra-state) across all states dispatched on a merit order basis. 1. How is this integrated into the system in terms of communication, SCADA integration, data transfer, etc.? 2. Is day-ahead schedule and demand exchanged on a daily or weekly basis? 2. A python-based co-optimization algorithm is being used to simulate this and calculate benefits accrued for each time block in a day for an entire year. The proposed simulation will require actual 15-minute block-wise data for all the states for the past 12 months and compare the simulated system charges with actual system charges to arrive at benefits for each state. 1. Is this algorithm providing benefits at the state level or country level? What optimization procedure is used? 2. How is this algorithm, software implemented in the actual system? 3. The system charges will be calculated after taking into account clearing of un￾requisitioned surplus generation based on three different price options: - DAM price at Indian Energy Exchange. - Regulated prices as decided by regulatory authorities. - System Marginal Price (based on matching of demand and reserves for each slot) 1. These are pricing options but which of them is used? 4. These data were used in carrying out the simulation for a period of 12 months from April 1, 2016 to March 31, 2017. 1. What generator constraints were implemented? 77 2. What transmission constraints were considered? 5. The optimization exercise is expected to successfully establish that inter-state sharing of reserves, when dispatched on system marginal price, would lead to substantial reduction of power procurement cost of the DISCOMS. 1. Is this just assumed, or has it been actually achieved? 6. National Open Access Registry has been conceptualized as a one-stop solution for automating the short-term open access approval processes wherein applicants can apply for a transaction based on corridor availability as specified and the system would grant automatic recommendations for approval based on the same. 1. Is this just a concept? 7. A Python based optimization tool has been used to simulate and demonstrate these benefits for six states assumed to be participating in the PX based market in a closed mode. The simulation is carried out for each of the 96 time blocks for each day in 12 months (April 16-March 17). Data have been obtained from six states/ SLDCs - Telengana, Andhra Pradesh, Madhya Pradesh, Maharashtra, Karnataka, Chhattisgarh (three states in Western Region & 3 states in Southern Region). 1. Optimization results are not currently available. 8. GTG-RISE would also undertake an AS-IS study of existing IT landscape at NLDC / RLDC and determine various technology alternatives for implementing the National Open Access Registry. 1. What technology is currently being used and what technology is recommended based on the completed analyses? 9. A technology feasibility assessment on block chain has been concluded by GTG￾RISE. It is recommended that instead of designing the entire NOAR on blockchain, a pilot implementation would be the ideal way forward. 1. There is no clear information how block chain technologies fit into this pilot. 10. Integration with external systems. 1. There is no clear indication how integration with external systems has been done. 78 ANNEX 5: DRAFT DATA COLLECTION PROTOCOL AND LIST OF KEY STAKEHOLDERS INFORMED CONSENT – KEY INFORMANT INTERVIEWS Thank you for taking the time to meet with us today. My name is [NAME]. I am a researcher from a company named Panagora Group, which is based in the United States. Our team is speaking with people to conduct an evaluation of a project funded by USAID/India called Greening the Grid or GTG for short. We would like to conduct a brief discussion with you today to learn about your experience with this topic. Your responses, along with responses from other participants, will be compiled into findings for a report. The report will be publicly available once it is complete, but it will not include your name or other identifying information. Readers will not be able to identify the specific individuals we spoke to from any specific quotes or data in the report. It is important to understand that while we would like your help in this study, you do not have to participate if you do not want to, and you do not have to answer any questions if you feel uncomfortable doing so. The objective of this research is to improve the performance of projects like this one. Please note that we plan to record this interview. The recordings will be used to transliterate the interview so that we can review the content later. The recordings will not be shared with any third party. The interview is expected to take about 60 minutes. You may ask questions at any time during our discussion. If you have questions or concerns about the research after we leave today, you can contact me at [EMAIL] or [PHONE NUMBER]. Do you have any questions before we start? By saying “yes,” and participating in this study, you are indicating that you have heard this consent statement, had an opportunity to ask any questions about your participation, and voluntarily consent to participate. Will you participate in this interview? You may answer yes or no. ◻ Yes, I will participate. ◻ No, I will not participate. Are you okay with us recording the interview? You may answer yes or no. ◻ Yes, I am okay with recording the interview. ◻ No, I am not okay with recording the interview. 79 KII GUIDE – DONOR AGENCY STAFF MEMBER INTERVIEW DATE: INTERVIEWER: RESPONDENT NAME: RESPONDENT ORGANIZATION: RESPONDENT JOB TITLE: BG.1 Background ◻ EQ1: To what extent has the GTG program achieved its objective of assisting the Government of India (GOI) in integrating- large scale variable renewable energy (VRE) into the existing power grid? I. Impacts on VRE Integration. In terms of overall impacts, what has been the change in VRE integration into power grids compared to the baseline situation? II. Contributions of Pilots. What are the outcomes of the pilots in contributing to the overall objectives of the activity and what are the demonstrable results on the selected utilities? III. Impact on Regulatory Change. How effective has GTG been in supporting national and state regulatory bodies, and in particular the Forum of Regulators. What are GTG’s impacts on regulatory changes? ◻ EQ2: To what extent has the theory of change (TOC) for this project “to build the flexibility of India’s grid through new market opportunities for the private sector, for ancillary services, better forecasting, improved operating systems and equipment, and flexible energy services” been validated? I. New market opportunities for the Private Sector. How effective has GTG been in identifying and demonstrating these opportunities? What specific GTG actions can you identify? How effective were these actions? II. Ancillary and flexible energy services. How has GTG performed in this area? What specific program actions contributed to this objective? What could GTG have done better in this area? III. Better forecasting. How and what did GTG contribute to better forecasting? What added benefits did the program provide to stakeholders? Who benefitted and how? IV. Improved operating systems and equipment. What specific systems and equipment were improved under GTG and to what extent did this help provide flexibility in India’s grid? V. Are there signs of measurable flexibility in India’s grid? What has contributed the most to this improved flexibility? 80 ◻ EQ3: What possibilities and challenges are there for applying, replicating, and scaling up the GTG interventions, in particular, the pilot projects? Are they sustainable? I. Is there evidence that the program outcomes are likely to grow, scale up and out, past the project period of implementation (sustainability)? II. What are the challenges that pilots have faced and how can these be addressed during similar actions in the future? III. What changes/improvements need to be made to make the program interventions more scalable, sustainable, and to achieve an enhanced development impact? IV. Have pilots allowed measurable impact on the existing market and suggested possible regulatory changes? V. What are recommendations that can improve geographic selection, sub-sector identification, beneficiary private company selection and resource allocation? VI. What additional support would be required to make the program interventions, especially the pilots. sustainable? VII. Have pilots contributed to establishing technical baselines that can be used to develop technical specifications and standards for equipment? ◻ EQ4: How has the sector evolved in the five years since GTG was conceived and designed? Looking at this recent evolution and emerging technologies for VRE integration, what should be the focus of follow-on programming? I. Which market/technological/policy/regulatory issues should be the focus of future programming? II. What should be the approach with respect to pilot projects, integration studies, etc.? III. How can this future work be made relevant for both India and the South Asia region? IV. Which components (pilots, studies, analyses, etc.) should future activities include so that a more uniform approach can be followed towards project design and implementation by adjusting and linking program components? ◻ EQ5: How far has GTG been successful in incorporating gender into the implementation of the overall project approach and the individual pilots/components? I. Any lessons learned from the intervention regarding gender equality and the empowerment of women for future programming in energy sector? 81 KII GUIDE – IMPLEMENTING PARTNER STAFF MEMBER INTERVIEW DATE: INTERVIEWER: RESPONDENT NAME: RESPONDENT ORGANIZATION: RESPONDENT JOB TITLE: BG.1 Background ◻ EQ1: To what extent has the GTG program achieved its objective of assisting the Government of India (GOI) in integrating large-scale variable renewable energy (VRE) into the existing power grid? I. Impacts on VRE Integration. In terms of overall impacts, what has been the change in VRE integration into power grids compared to the baseline situation? II. Contributions of Pilots. What are the outcomes of the pilots in contributing to the overall objectives of the activity and what are the demonstrable results on the selected utilities? III. Impact on Regulatory Change. How effective has GTG been in supporting national and state regulatory bodies, and in particular the Forum of Regulators. What are GTG’s impacts on regulatory changes? ◻ EQ2: To what extent has the theory of change (TOC) for this project “to build the flexibility of India’s grid through new market opportunities for the private sector, for ancillary services, better forecasting, improved operating systems and equipment, and flexible energy services” been validated? I. New market opportunities for the Private Sector. How effective has GTG been in identifying and demonstrating these opportunities? What specific GTG actions can you identify – how effective were these actions? II. Ancillary and flexible energy services. How has GTG performed in this area? What specific program actions contributed to this objective? What could GTG have done better in this area? III. Better forecasting. How and what did GTG contribute to better forecasting – what added benefits did the program provide to stakeholders? Who benefitted and how? IV. Improved operating systems and equipment. What specific systems and equipment were improved under GTG and to what extent did this help provide flexibility in India’s grid? V. Are there signs of measurable flexibility in India’s grid – what has contributed the most to this improved flexibility? 82 ◻ EQ3: What possibilities and challenges are there for applying, replicating, and scaling up the GTG interventions, in particular, the pilot projects? Are they sustainable? I. Is there evidence that the program outcomes are likely to grow, scale up and out, past the project period of implementation (sustainability)? II. What are the challenges that pilots have faced and how can these be addressed during similar actions in the future? III. What changes/improvements need to be made to make the program interventions more scalable, sustainable, and to achieve an enhanced development impact? IV. Have pilots allowed measurable impact on the existing market and suggested possible regulatory changes? V. What are recommendations that can improve geographic selection, sub-sector identification, beneficiary private company selection and resource allocation? VI. What additional support would be required to make the program interventions, especially the pilots, sustainable? VII. Have pilots contributed to establishing technical baselines that can be used to develop technical specifications and standards for equipment? ◻ EQ4: How has the sector evolved in the five years since GTG was conceived and designed? Looking at this recent evolution and emerging technologies for VRE integration, what should be the focus of follow-on programming? I. Which market/technological/policy/regulatory issues should be the focus of future programming? II. What should be the approach with respect to pilot projects, integration studies, etc.? III. How can this future work be made relevant for both India and the South Asia region? IV. Which components (pilots, studies, analyses, etc.) should future activities include so that a more uniform approach can be followed towards project design and implementation by adjusting and linking program components? ◻ EQ5: How far has GTG been successful in incorporating gender into the implementation of the overall project approach and the individual pilots/components? I. Any lessons learned from the intervention with regard to gender equality and the empowerment of women for future programming in energy sector? 83 KII GUIDE – “GTG PILOT PROJECT” STAFF MEMBER INTERVIEW DATE: INTERVIEWER: RESPONDENT NAME: RESPONDENT ORGANIZATION: RESPONDENT JOB TITLE: BG.1 Background ◻ EQ1: To what extent has the GTG program achieved its objective of assisting the Government of India (GOI) in integrating large-scale variable renewable energy (VRE) into the existing power grid? I. Impacts on VRE Integration. In terms of overall impacts, what has been the change in VRE integration into power grids compared to the baseline situation? II. Contributions of Pilots. What are the outcomes of the pilots in contributing to the overall objectives of the activity and what are the demonstrable results on the selected utilities? III. Impact on Regulatory Change. How effective has GTG been in supporting national and state regulatory bodies, and in particular the Forum of Regulators? What are GTG’s impacts on regulatory changes? ◻ EQ2: To what extent has the theory of change (TOC) for this project “to build the flexibility of India’s grid through new market opportunities for the private sector, for ancillary services, better forecasting, improved operating systems and equipment, and flexible energy services” been validated? I. New market opportunities for the Private Sector. How effective has GTG been in identifying and demonstrating these opportunities? What specific GTG actions can you identify – how effective were these actions? II. Ancillary and flexible energy services. How has GTG performed in this area? What specific program actions contributed to this objective? What could GTG have done better in this area? III. Better forecasting. How and what did GTG contribute to better forecasting? What added benefits did the program provide to stakeholders? Who benefitted and how? IV. Improved operating systems and equipment. What specific systems and equipment were improved under GTG and to what extent did this help provide flexibility in India’s grid? V. Are there signs of measurable flexibility in India’s grid? What has contributed the most to this improved flexibility? 84 ◻ EQ3: What possibilities and challenges are there for applying, replicating, and scaling up the GTG interventions, in particular, the pilot projects? Are they sustainable? ◻ I. Is there evidence that the program outcomes are likely to grow, scale up and out, past the project period of implementation (sustainability)? II. What are the challenges that pilots have faced and how can these be addressed during similar actions in the future? III. What changes/improvements need to be made to make the program interventions more scalable, sustainable, and to achieve an enhanced development impact? IV. Have pilots allowed measurable impact on the existing market and suggested possible regulatory changes? V. What are recommendations that can improve geographic selection, sub-sector identification, beneficiary private company selection and resource allocation? VI. What additional support would be required to make the program interventions, especially the pilots, sustainable? VII. Have pilots contributed to establishing technical baselines that can be used to develop technical specifications and standards for equipment? ◻ EQ4: How has the sector evolved in the five years since GTG was conceived and designed? Looking at this recent evolution and emerging technologies for VRE integration, what should be the focus of follow-on programming? I. Which market/technological/policy/regulatory issues should be the focus of future programming? II. What should be the approach with respect to pilot projects, integration studies, etc.? III. How can this future work be made relevant for both India and the South Asia region? IV. Which components (pilots, studies, analyses, etc.) should future activities include so that a more uniform approach can be followed towards project design and implementation by adjusting and linking program components? ◻ EQ5: How far has GTG been successful in incorporating gender into the implementation of the overall project approach and the individual pilots/components? I. Any lessons learned from the intervention with regard to gender equality and the empowerment of women for future programming in energy sector? 85 KII GUIDE – PARTNER COUNTRY (INDIA) AGENCY STAFF MEMBER INTERVIEW DATE: INTERVIEWER: RESPONDENT NAME: RESPONDENT ORGANIZATION: RESPONDENT JOB TITLE: BG.1 Background ◻ EQ1: To what extent has the GTG program achieved its objective of assisting the Government of India (GOI) in integrating large-scale variable renewable energy (VRE) into the existing power grid? I. Impacts on VRE Integration. In terms of overall impacts, what has been the change in VRE integration into power grids compared to the baseline situation? II. Contributions of Pilots. What are the outcomes of the pilots in contributing to the overall objectives of the activity and what are the demonstrable results on the selected utilities? III. Impact on Regulatory Change. How effective has GTG been in supporting national and state regulatory bodies, and in particular the Forum of Regulators. What are GTG’s impacts on regulatory changes? ◻ EQ2: To what extent has the theory of change (TOC) for this project “to build the flexibility of India’s grid through new market opportunities for the private sector, for ancillary services, better forecasting, improved operating systems and equipment, and flexible energy services” been validated? I. New market opportunities for the Private Sector. How effective has GTG been in identifying and demonstrating these opportunities? What specific GTG actions can you identify – how effective were these actions? II. Ancillary and flexible energy services. How has GTG performed in this area? What specific program actions contributed to this objective? What could GTG have done better in this area? III. Better forecasting. How and what did GTG contribute to better forecasting? What added benefits did the program provide to stakeholders? Who benefitted and how? IV. Improved operating systems and equipment. What specific systems and equipment were improved under GTG and to what extent did this help provide flexibility in India’s grid? V. Are there signs of measurable flexibility in India’s grid? What has contributed the most to this improved flexibility? 86 ◻ EQ3: What possibilities and challenges are there for applying, replicating, and scaling up the GTG interventions, in particular, the pilot projects? Are they sustainable? I. Is there evidence that the program outcomes are likely to grow, scale up and out, past the project period of implementation (sustainability)? II. What are the challenges that pilots have faced and how can these be addressed during similar actions in the future? III. What changes/improvements need to be made to make the program interventions more scalable, sustainable, and to achieve an enhanced development impact? IV. Have pilots allowed measurable impact on the existing market and suggested possible regulatory changes? V. What are recommendations that can improve geographic selection, sub-sector identification, beneficiary private company selection and resource allocation? VI. What additional support would be required to make the program interventions – especially the pilots - sustainable? VII. Have pilots contributed to establishing technical baselines that can be used to develop technical specifications and standards for equipment? ◻ EQ4: How has the sector evolved in the five years since GTG was conceived and designed? Looking at this recent evolution and emerging technologies for VRE integration, what should be the focus of follow-on programming? I. Which market/technological/policy/regulatory issues should be the focus of future programming? II. What should be the approach with respect to pilot projects, integration studies, etc.? III. How can this future work be made relevant for both India and the South Asia region? IV. Which components (pilots, studies, analyses, etc.) should future activities include so that a more uniform approach can be followed towards project design and implementation by adjusting and linking program components? ◻ EQ5: How far has GTG been successful in incorporating gender into the implementation of the overall project approach and the individual pilots/components? I. Any lessons learned from the intervention with regard to gender equality and the empowerment of women for future programming in energy sector? 87 ANNEX 6: LIST OF KEY STAKEHOLDERS INTERVIEWED DONORS: USAID • USAID/India COR • Indo-Pacific Office Team Leader and Energy Team IMPLEMENTING PARTNERS: • Deloitte management team (including present and former Chiefs of Party), Home Office and Project Team • NREL • USEA • NARUC and their sub E3 PARTNER COUNTRY: GOI STAKEHOLDERS • Ministry of Power (MOP) • Power System Operation Corporation (POSOCO) • Power Grid Corporation of India Limited (PGCIL) • Central Electricity Regulatory Commission (CERC) • National Thermal Power Corp. (NTPC) • Solar Energy Corporation of India (SECI) GTG PILOTS: STATE GOVERNMENT REGULATORS AND SYSTEM OPERATORS • Karnataka Power Transmission Corporation Limited (KPTCL) • Southern Regional Load Dispatch Centre (SRLDC) • Andhra Pradesh State Load Dispatch Centre (APSDLC) • Southern Regional Power Committee (SRPC) • Forum of Regulators (FOR) • Delhi Electricity Regulatory Commission (DERC) GTG PILOTS: GENCOS/PRIVATE SECTOR • Gujarat State Energy Corporation Limited (GSECL) • Karnataka Power Corporation Limited (KPCL) • Power Grid personnel in Pudducherry • BSES Yamuna Power Limited (BYPL) • BSES Rajdhani Power Limited (BRPL) • NTPC Limited • Members of Industry Advisory Council (IAC) OTHERS: • Members of South Asia Women in Energy (SAWIE) 88