GEORGIA RURAL ENERGY PROGRAM (REP) EVALUATION FINAL REPORT 2 ACRONYMNS ABL Administrative Boundary Line BoG Bank of Georgia CDM Clean Development Mechanism CENN Caucasus Environmental NGO Network EBRD European Bank for Reconstruction and Development EE Energy Efficiency ESCO Independent Electricity Market Operator Kwh kilowatt hour IDP Internally Displaced Person IPP Independent Power Producer IRMP Integrated Resource Management Plan MHP Micro Hydropower Plant NATELI New Applied Technology, Efficiency and Lighting Initiative OFDA Office of Foreign Disaster Assistance (USAID) PMP Performance Monitoring Plan RE Renewable Energy REP Rural Energy Program SHP Small Hydropower Plant UNDP United Nations Development Program USAID United States Agency for International Development WA Watershed Assessment WEG World Experience for Georgia 3 the United States Agency for International Development. It was prepared by the 4 Contents I. INTRODUCTION 5 II. BACKGROUND 7 III. LESSONS LEARNED AND BEST PRACTICES 9 IV. ACTIVITY 1: INCREASED HYDROPOWER SUPPLY 11 V. ACTIVITY 2: INCREASE ACCESS TO RURAL ENERGY PROJECT FINANCING 14 VI. ACTIVITY 3: INCREASE CAPACITY IN ENERGY EFFICIENCY AND RENEWABLE ENERGY 18 VII. ACTIVITY 5: IMPROVE LOCAL COMMUNITY CAPACITY TO SUSTAINABLY UTILIZE AND PROTECT THE LOCAL ENERGY & NATURAL RESOURCE BASE 23 VIII. FINAL RECOMMENDATIONS 25 IX. APPENDICES 26 5 I. INTRODUCTION The purpose of this evaluation was to examine the accomplishments and impact of the Georgia Rural Energy Program (REP), which was implemented for USAID/Georgia by Winrock International under a $10.8 million Cooperative Agreement from September 28, 2005 – September 27, 2009. The REP project contained six components, explained in the next section. The program built upon the Georgia Energy Security Initiative (March 2003 – October 2007), with over-arching goals of: 1) increasing the supply of energy in rural areas; 2) improving management of local energy production; 3) building in-country capacity in rural energy and alternative energy applications; and 4) promoting sustainability and use of natural resources. The evaluators were asked by USAID/Georgia to: Analyze specific results of REP in its core component areas; Evaluate the impact of the program at the national and local levels; and Provide recommendations and key lessons learned for future programming. The evaluation team found that REP accomplished its objective of increasing energy supply in rural areas, particularly in the area of hydropower. REP demonstrated the economic viability of small, independently owned hydro plants and established a legal basis for their participation in Georgia’s electricity market. All of the operating small hydros rehabilitated by the project are profitable, meeting their financing commitments, and selling power to the grid. It is difficult to ascertain how transformative the project was on the sector, however, as other, larger hydro sites also became operational during the project period and all the extra capacity eliminated the country’s electricity shortages. Morever, the current export focus of Georgia’s electricity sector and the utilities’ desire to maintain grid integrity likely will favor larger hydro investments in the future, but REP improved the ability of smaller, private investors to compete for financing and access to hydro sites. Although REP implemented some pilot projects to investigate the feasibility of energy efficiency and alternative energy investments in Georgia, its activities in this area were too scattered and small-scale to have a catalytic impact. These projects benefitted the families/sites that received the technologies and contributed to greater knowledge of their use in Georgia, but the pilots were not part of an integrated approach to foster uptake and deployment. However, REP developed draft laws and policies to promote adoption of energy efficiency and renewable energy that will prove valuable when the government is ready to utilize them. 1.a Evaluation Team The evaluation team was led by Pamela Baldinger, USAID/EGAT/I&E, with assistance from Morning Washburn, EGAT/I&E. Aviva Kutnick of USAID/Georgia and Irakli Kaviladze, a local consultant, also were core team members. Valuable preparation assistance was provided by Cael Savage of USAID/Georgia, and Nick Okreshidze provided overall direction and invaluable insights. 1.b. Evaluation Methodology 6 The team utilized the following methodology to collect data for the evaluation: A) Review of REP program documents - the team reviewed reports and documents posted on the REP web site, the REP final report, and any interim progress reports it could obtain. B) Bank and Investor surveys - the team developed 1-page surveys for hydro investors and financial institutions, in order to obtain financial and other basic information before the external evaluators arrived in￾country. However, no responses were received, so the survey questions were incorporated into interviews. C) In-person and telephone interviews - the team conducted telephone interviews with former REP project staff now based in Dubai and Mongolia, a woodstove consultant based in Iowa, and the EBRD in London. Georgia-based interviews were conducted with the following institutions: Winrock/PA Consulting, Ministry of Energy, Energopro, ESCO, biodigester/stove manufacturer Bioenergy, UNDP, World Bank, hydro investors, Cartu Bank, and Bank of Georgia. D) Site visits - the team conducted site visits to the following small and micro-hydro plants: Misaktsieli SHP, Okami SHP, Lopota SHP, Kabali SHP, Kakhareti SHP, Nergeeti SHP, Gadamshi MHP. In addition, the team visited five biodigester sites, and interviewed five IDP households about energy efficient stoves in two villages close to Tskhinvali on the Administrative Boundary Line (ABL) with South Ossetia. The rest of this report details the evaluation team’s findings and recommendations. All PMP targets cited in the report come from REP’s Year 3 performance monitoring plan. 7 II. BACKGROUND By the time the Rural Energy Program (REP) began in 2005, Georgia had been free of Soviet occupation for nearly 15 years, but had suffered from political turmoil, financial deterioration and social upheaval for much of that period. The electricity sector was plagued by mismanagement and corruption. Electricity service became unreliable and prohibitively expensive for many citizens, some of whom resorted to theft. Social discontent and occasional violence over various energy-related issues characterized the late 1990s. Throughout the country, metal scavenged from factories and elsewhere (such as energy installations) was sold as scrap, further escalating the decline of infrastructure. When USAID/Georgia began planning for the REP program in 2004, the electricity sector had become more stable, but still suffered from production shortfalls and an unclear regulatory environment for private investors During the Soviet era, small-scale hydro systems were constructed throughout the country, particularly during the 1960s and 1970s. These systems fell into disrepair as the Soviets moved away from the distributed generation model in favor of larger, more centralized electricity production, characterized by the Enguri hydro plant. With Georgian independence, all small hydro facilities were privatized. However, new private owners often lacked the financial, and in some cases, the technical expertise, to put the facilities back on-line. The four years during which REP was implemented saw significant changes within Georgia’s energy sector, as well as significant national and international events that impacted the program. Increased investment in the sector brought new capacity online, to the extent that Georgia moved from suffering from electricity shortages to becoming a regional exporter of energy. The global financial crisis of 2008, as well as the 2008 Georgian-Russian military conflict led to the Russian occupation of South Ossetia and Abkhazia, also impacted the REP program. The financial crisis made Georgian financial institutions even more conservative than before, and helped drive prices for equipment rapidly upwards. The 2008 Georgia-Russia military conflict delayed project implementation and resulted in REP providing some humanitarian assistance not originally envisaged in the program. Rural Energy Program Components Initially conceived as the Rural Energy and Environmental Develop (REED) Project, the program was designed to “(a) increase supply of energy to rural areas..., (b) improve management of local production, (c) improve in-country capacity in rural energy and alternative energy applications; and (d) improve capacity to utilize and protect the local energy resource base.” This program was subsequently re-named the Rural Energy Program (REP). The Rural Energy Program had six components: 1. Hydropower development: Increase supply of energy in rural areas This component concentrated on rehabilitation of small and micro hydro systems in rural areas throughout the country. While REP included development of green field sites, this activity was predominately focused on rehabilitation of Soviet era, largely non-functional, hydro-electric facilities. 2. Financing: Increase access to rural energy project financing 8 While initially written to include financing for all types of rural energy renewable systems, this component evolved to focus solely on creating financing for the small hydro facilities. 3. Energy Efficiency and Renewable Energy: increase capacity in renewable energy and energy efficiency This component incorporated a number of pilot projects and some initiatives driven by USAID/Georgia. 4. Legal and Policy: Support institutional capacity and a legal, policy, and regulatory environment conducive to the promotion of rural access to clean, efficient energy resources This component was largely focused on assessments for renewable energy and energy efficiency development, construction code development to cover energy efficiency aspects during construction, and drafting of other energy legislation promoting renewable energy and energy efficiency in Georgia. Laws/policies specific to hydro development and energy efficiency and renewable energy were the focus. 5. Environment: improve integrated natural resource management and planning This component included projects ancillary to component one, focusing on mitigating environmental impact of hydro sites being supported by REP. 6. Outreach: implement public outreach in support of these objectives This component focused on generating media interest in energy efficiency and renewable energy, as well as development of project fact sheets and success stories. Grants for hydro rehabilitation totaled approximately $1 million, roughly 10% of the entire budget. The bulk of project funding and TA appeared to focus, with the agreement of USAID, on efforts to support development of commercially oriented, private-sector small hydro investments. Most of the work conducted in components 2, 5, and 6 was targeted toward attainment of the hydro goal rather than rural energy more broadly (this will be discussed more fully in the body of the report). Due to time and language constraints, the evaluation team evaluated work done under components 1, 2, 3, and 5, and some of the policy-related work done under component 4 as it related to hydro and efficiency/renewable energy development. 9 III. LESSONS LEARNED AND BEST PRACTICES Program-specific findings on the various REP activities may be found in sections IV-VII, and project￾specific recommendations in section VIII. As requested by the USAID/Georgia mission director during the evaluation team de-brief, the recommendations provided here deal with lessons learned from the REP evaluation with broad, programmatic impacts. These recommendations highlight issues that cut across multiple USAID programs, and are not specific to REP or the energy sector. Standardize requirements for submission and storage of project documents: Given increased emphasis on evaluations and evidence-based data in USAID decision making, USAID/Georgia should develop standard language regarding implementer requirements for submitting project data to USAID, and develop performance indicators related to this activity. For example, the mission might require an implementer to provide a CD-ROM containing all relevant project documents (i.e., reports and assessments; training materials; dates, places, contact info and evaluation forms from all training sessions; project management documentation; etc) organized in a pre-agreed manner halfway through project implementation, and again upon project completion. This information should be shared not just with the AOTR/COTR, but stored by the mission in a designated area along with key internal documents such as the project SOW, contract/agreement modifications, performance monitoring plans (PMPs), etc. In this way, external evaluators and other mission staff will be able to obtain crucial documents and contact information even after the project ends and implementing staff are no longer available. Consider conducting mid-term evaluations for projects four years or longer, especially if they involve multiple activities/components: a mid-term evaluation conducted by USAID subject-area experts can help mission staff determine if the various components of a project are on track and focused, recommend what changes (if any) might be useful, and help ascertain whether the implementer’s data quality is acceptable. Ensure program scopes-of-work, workplans, and PMPs clearly specify and prioritize key objectives and their intended impacts: A given program may contain many components, but should have a smaller number of priorities, with various activities developed to support those priorities. Funding and expectations of impact of supporting (or secondary) activities should be developed in accordance with their significance in meeting overall objectives. Ensure that activities focused on obtaining positive press exposure also have potential for significant impact: Activities that focus on public outreach, especially outreach tied to the introduction of new technologies, should be linked to programs that facilitate public access to those technologies. Typically, this would involve not just helping the public understand the potential benefits of the technologies, but also ensuring that there is a reliable supply of good quality products available and financial organizations willing to provide affordable consumer financing. Specify what data and analysis will be required from any pilot projects, especially those involving testing or consumer acceptance of new technologies: Implementers should be required to obtain 10 data on actual performance, socio-economic benefits and costs, and producer/consumer acceptance of any piloted technologies. It is not acceptable to provide estimated data based on feasibility studies. In addition, implementers should be required to provide information or recommendations on next steps if USAID or another entity wishes to take a project to scale. 11 IV. ACTIVITY 1: INCREASED HYDROPOWER SUPPLY Activity 1 provided technical assistance to help project partners establish new or rehabilitate existing small hydropower plants (SHPs) in Georgia. The program supported two different types of hydropower projects: 1) grid connected, privately owned and commercially-viable or near commercial (i.e. requiring no or limited grant support from the Rural Energy Program); and 2) off-grid locations with the potential to serve populations lacking access to electricity. The latter projects were largely humanitarian in nature, small-scale in size, owned by the communities they serve, and were greenfield projects. Nine SHP and three micro-hydro plants (MHP) projects were chosen for support from REP. The MHPs were all green￾field, off-grid sites. All of the SHP and MHP hydropower plants are run-of-river except for Misaktsieli, Okami, Pshaveli and Kakhareti, which receive water from irrigation canals. From its inception REP worked closely with the SHP owners to assist them in obtaining all necessary construction, environmental and other permits necessary to operate the plant. REP’s technical assistance included initial project designs, development of business plans to obtain financing, assistance with equipment and construction material procurement, construction oversight, assistance to negotiate power purchase agreements and advisory services as required. To ensure sustainability of the hydro projects and to increase technical and business management skills amongst hydro operators, REP designed and delivered training on technical energy issues such as technical design development, international procurement, construction management, and operations and maintenance. 12 The following hydro targets were listed in the PMP1 : • 10 SHPs rehabilitated • 5 micro-hydro projects (MHPs) constructed • 100 people trained in technical energy fields • 70 people trained in energy-related business management Evaluation Team Methodology The evaluation team designed a survey for small hydro owners (located in Appendix B), with the hope that it could be administered before they arrived. However, due to logistical constraints, the mission was not able to distribute the surveys ahead of time. So, those questions were incorporated into face-to-face interviews. The evaluation team conducted meetings with hydro owners in Tbilisi, and also visited five SHPs, and one MHP. In addition, the team met with staff from Winrock and its technical sub-contractor, PA Government Services (PA took the lead on the hydro technical assistance component of REP). Findings and Results At the time of the evaluation, REP had accomplished the following: • 7 SHPs had been rehabilitated, and one more (Dash Bash) received equipment from REP on the last day of the program) • 3 MHPs had been completed and were operational (although two were completed nearly a year after the project officially ended) • 2 SHPs (Nergeeti and Pshaveli) were awaiting additional funding from UNDP • 69 people had been trained in technical areas • 39 people had been trained in energy-related business management (Final Report, p.27) Of the plants that had been rehabilitated, one (Okami), was not operational because its water supply originated in South Ossetia and had been cut off in May 2010 by the Russians or South Ossetians. The Machakhela project was terminated due to breach of contract by the SHP owner. The control equipment REP helped procure for that project was transferred to Dash Bash, but no additional REP services were provided. Two community-owned projects, Nergeeti and Pshaveli, which were primarily funded by a grant from UNDP to Winrock, had not been completed due to cost overruns. UNDP intended to obtain additional funding for Nergeeti, but not Pshaveli. It is unclear whether the Pshaveli site will ever be completed. For these two SHP sites, USAID funds were used by Winrock to conduct feasibility studies and provide technical assistance related to project planning, engineering and procurement. In addition to the technical work to support the small hydros, Winrock conducted activities under Activity 4 (support institutional capacity and legal, policy, and regulatory environment conducive to promotion of rural access to clean, energy efficient resources) to support small, independent power producers (IPPs). REP participated in working groups with the government, regulator, and other industry stakeholders to revise the Electricity Market Rules. REP represented the interests of small, IPP hydros, 1 These targets were set in the Project Year 3 PMP, dated October 3 2007 and revised November 26 2007. 13 ensuring that they were permitted to sell electricity to the Independent Market Operator (ESCO) as balancing power. Initially, ESCO paid the small hydros a fairly high, fairly uniform rate throughout the year, which enabled them to run profitably. By summer 2008, however, Georgia was experiencing an electricity surplus, and the Market Rules were revised. Under the current system, ESCO pays seasonal prices for electricity, reflecting the system’s heavy reliance on hydropower. Small hydros now receive higher prices for around 7-8 months per year (as high as 9-11tetri/kwh2 ), and much lower prices (around 1-2 tetri/kwh3 ) the rest of the time, primarily in summer months. The evaluation team’s conclusions on this activity are: The rehabilitated hydros are operational and profitable With the exception of Okami (which could not operate after the water supply from South Ossetia was shut off following the 2008 Georgia￾Russia military conflict), all of the private SHP owners reported that their plants are operating well and are profitable, even after the revision of the Electricity Market Rules which resulted in low summer revenues. REP proved that IPP small hydro production is economically viable, and gave banks, ESCO, and the Ministry of Energy some familiarity and comfort in dealing with small investors. The impact of the rehabilitated hydros on Georgia’s energy supply is limited Although REP succeeded in adding approximately 10 MW of electricity to Georgia’s grid, Georgia moved from having energy shortages to energy surpluses by rehabilitating existing medium- and large-size hydropower plants over the course of the project. Georgia’s domestic energy market is no longer a driver for investment in medium and large size HPPs; potential export to Turkey (and thereby to EU) will drive any future investment. However, small hydropower plants might still be able to operate profitably in certain niches of the domestic market. If small hydros are able to gain access to the new high-voltage transmission line to Turkey currently being constructed, investment in such sites will be more attractive. However, the Government of Georgia would have to address certain legal regulatory issues to remove the barriers and facilitate power exports from new small hydro plants to the Turkish market. In terms of socio-economic impact, all of the communities surrounding the SHPs already had access to the grid; only the MHPs brought access to non-served communities. It is possible that the quantity, quality and reliability of electricity supply to communities near the SHPs improved under REP, but these potential benefits and their impacts were not measured by the project or the evaluation team. The greatest benefit to most of the communities was improved access to irrigation water, and employment opportunities at the hydro plants. The SHPs all sold power to ESCO, rather than to private customers. Only one SHP (Misaktsieli) had a direct contract with a private customer. One SHP (Lopota) reported that it was illegal for an IPP to sell power directly via contract, even though this activity is allowed under the Electricity Market Rules, and was originally envisioned as a potentially important source of income for the SHPs. Although ESCO and the Ministry of Energy accused the small hydro owners of being “lazy” on this score, the owners claimed they could not compete on price with Energopro (the utility), which assesses wheeling and other fees on customers using their distribution network but not purchasing power directly through them. Thus, virtually all of the small hydros sold all of their power to ESCO. 2 USD equivalent = 5.5 – 6.5 cents per kilowatt hour, March 2010 GEL-USD exchange rate, unadjusted. 3 USD equivalent = .5 −1.2 cents per kilowatt hour, March 2010 GEL-USD exchange rate, unadjusted. 14 The Gadamshi MHP only serves three households. According to the REP final report, the Gadamshi MHP is supposed to serve 17 households. However, the evaluation team’s site visit revealed that only three households are connected (there are only four or five inhabited houses in the entire village). The evaluation team cannot account for this discrepancy. REP assessments of the two community sites (Nergeeti and Pshaveli) did not realistically take costs and community capacity/willingness to pay into account During the course of the REP project, international commodity prices rose dramatically, pushing up project costs. Most of REP’s initial cost estimates, developed in 2006 and 2007, were too low and did not include contingencies. The privately owned site owners were able to absorb the increased costs by raising their equity or borrowing more money. The community-owned sites, however, were unable and unwilling to invest more resources (neither of the communities relied on the SHPs for their electricity, as they were already connected to the grid), and UNDP’s reliance on grants to fund construction costs left little flexibility to reassess/realign project costs. Although ultimately the onus rests with UNDP to undertake due diligence on its projects and partners, UNDP staff expressed unhappiness with Winrock’s performance and claimed that they “expected more” from USAID to resolve some of the issues at these two sites, since USAID had initially approached them about cooperation. UNDP indicated they would not provide additional funding to complete the Pshaveli site. V. ACTIVITY 2: INCREASE ACCESS TO RURAL ENERGY PROJECT FINANCING REP Activity #2 was designed to “Advance commercialization of the hydropower sub-sector and to promote the availability of financing, including micro-financing, for small-scale rural energy (RE) and energy efficiency (EE) investments in Georgia.” By the end of the project in September 2009, the scope of the financing activities under REP had constricted, and energy efficiency and renewable energy financing efforts were abandoned. Ultimately, the financing activities of REP concentrated on both the establishment, within the banking sector, of a viable grant and loan structure for small hydro producers as well as capacity building of bank personnel and small business owners. Additionally, an investor guide was created, on behalf of the Georgian Ministry of Energy, to encourage outside investment in the small hydro sector. The following finance-related targets were listed in the REP PMP: 5 projects funded through DCA or other financing mechanism 90 people trained in energy project financing Leverage $5 million in public and private dollars for energy infrastructure REP envisioned that funding for the small hydro projects would come from several sources, including: USAID grants USAID provided grant funding (ranging from around $15,000-$160,000 per project) to 10 small hydros.4 USAID grants were supposed to constitute no greater than 25% of the total cost of any project (except the greenfield micro-hydro sites). Bank loans REP initially envisioned that approximately 40% of project funding would come from bank loans. Expected sources of bank funding were the mission’s DCA with TBC Bank, and the loan 4 One of the projects, Machakhela, was later withdrawn and the equipment sent to another site, and the Pshaveli site, primarily financed by UNDP, has not been completed and will not be further supported by UNDP. 15 facility established between TBC and the European Bank for Reconstruction and Development (EBRD). The Bank of Georgia (BoG) and Cartu Bank were also expected to provide funding. Owner equity REP expected privately-owned small hydro owners to fund approximately 40% of their overall project costs. Carbon finance REP teamed with the World Bank early in the project to apply for carbon credits from the Clean Development Mechanism (CDM), a funding mechanism created under the Kyoto Protocol. Evaluation Team Methodology Efforts to evaluate the finance-related aspects of the REP program were carried out in multiple ways. The evaluation team developed a survey for banks (found in Appendix B), but completion of the surveys proved problematic. Turnover at banks in Georgia was extensive from the beginning of the REP program until its completion. Bank staff said they could not find anyone knowledgeable about or involved in the REP program, and that in addition, any information on energy-related projects which had received loans from the bank was difficult to find and evaluate. Eventually, non-face to face bank surveying efforts were abandoned. Once in-country, the evaluation team began face-to-face meetings with both banks and hydro owners. Face-to-face meetings with banks again proved difficult. Neither Winrock nor the USAID Georgia mission knew the most current energy staff at banks. Most banks claimed they had no staff knowledgeable on REP-related issues or who had written small hydro loans. Ultimately, only Cartu and Bank of Georgia were able to produce staff that worked or knew of work done on energy-related loans under the USAID programs. The Cartu staff that met with the evaluation team had not participated in the REP program and had only indirect knowledge of the Sulori hydro project (which was actually supported under another USAID program, GESI). The Bank of Georgia staff member had participated in the energy￾related loans secured under the REP program. Findings and Results: Hydro Financing REP’s financial assistance may have been critical in developing at least some of the projects which were ultimately completed. The Misaktsieli small hydro project owner, for instance, said that he felt his USAID grant was critical to his ability to obtain bank financing. The Lopota hydro owner invested in his site only when he learned about the availability of REP support. REP leveraged significant financial resources, although it fell short of the target set in the PMP. The REP program successfully stayed within the financing guidelines outlined in the original statement of work. Of the completed projects (i.e., not including Machakhela, Pshaveli or Nergeeti), bank or private loans constituted about 35% of total project funding, USAID grants 24%, and private equity around 41%. REP expectations and planning for use of donor funding (particularly from the EBRD and CDM/World Bank) was overly optimistic. Although REP was able to find alternative sources of funding for the REP small hydros, it was unable to stimulate catalytic sources of funding for the sector as a whole. Winrock initially planned to fund five projects through the TBC/EBRD loan mechanism, which failed to evolve within the time frame necessary for the project. The Bank of 16 Georgia, working within the EBRD mechanism, only funded three REP projects, despite evaluating an initial pool of over 20 projects. This was largely due to the misguided assumption that paperwork and processing for the EBRD loan mechanism would take no more than three months, when in fact, processing time for the mechanism was in excess of six or more months. EBRD staff informed the evaluation team that REP never should have assumed the EBRD loans could have been completed in three months. The owner of the Misaktsieli small hydro project claimed the wait time to procure a loan from the Bank of Georgia/EBRD was two years and that he eventually gave up and obtained financing from Cartu Bank. Similarly, the process of obtaining carbon credits through the CDM is notoriously slow and not assured; by the time of the evaluation one year after the close of REP, the World Bank’s submission with the REP projects had been rejected by the CDM on the grounds of additionality (i.e., that the projects were commercially viable without the credits), and the World Bank was deciding on whether to re-submit. REP should not have assumed or led potential project participants to believe that carbon financing could be obtained within the life of the REP program. Assessments of political and financial risks either were not conducted or were inadequate. The Okami hydro project fell into default as a result of larger political issues. The site, which is located close to the disputed territory of South Ossetia, suffered a loss of all water from its feeder irrigation canal when the border was closed between South Ossetia and Georgia in 2010. A dim possibility remains that a deal might be struck to purchase water directly from the Russian authorities, but the Bank of Georgia has suggested that this loan may have to be written off as a loss. The August 2008 conflict between Russia and Georgia was not predicted, thus no contingency planning for such political risks was conducted. In retrospect the project may have considered that water availability could be a future concern given the site’s proximity to South Ossetia. Findings and Results: Financial Training Finance-related training within REP predominantly focused on two groups, hydro owners and bank credit departments, though other groups also received training under this program. The impact of REP financial training could not be determined by the evaluation team due to the lack of consistency in numbers and information received from in-country meetings and Winrock documents. It is unclear, based on the final report and other Winrock documents, exactly who was trained and on what topic. The final report states that 23 people received Energy Financing Training. Thirty-five commercial bank representatives were listed in the Final Report as participants for the bankers’ training, but within Winrock’s own Bankers’ Training Report created in March 2008, only 14 attendees are listed. Thirteen of these came from eight different banks, with one of the participants being a non-bank employed third-party individual. Winrock’s final report also stated “When REP started, the program faced a lack of understanding of project finance and financing of energy projects by local banks,” and that as a result of the training, “all major (Georgian) banks established separate energy project financing units, and the country experienced an increase of 200% in the level of energy financing over the life of REP.” As far as the evaluation team could determine, by the end of the program, only one bank, the Bank of Georgia, had a separate energy financing unit, and that unit had been established before the REP 17 program began. Of the two banks which the evaluation team was able to speak to, neither said the training had resulted in changes in their operations. Given that most of the trainings consisted of a maximum of one day, this result is not surprising. Findings and Results: Investor Guide A number of activities were initiated within the framework of the REP program in response to the changing needs of the Government of Georgia and the changing needs of the country. The results of these activities are listed below. The REP program had an ability to quickly and effectively include new program components in response to changing political situations and country needs when it came to meeting the quickly changing needs and desires of Georgian Ministry Officials. Although not part of the initial goals of the finance portion of the REP evaluation, an investor’s guide was added to the list of activities of REP after the Ministry of Energy requested help in creating a positive policy environment to attract investors to small and medium hydro projects in Georgia. The Final Report states that two-page fact sheets were submitted to the Ministry of Energy for each of 30 sites. The information was then uploaded to the ministry website. Ministry staff indicated satisfaction with the guide and claimed it helped them to meet their goals. 18 VI. ACTIVITY 3: INCREASE CAPACITY IN ENERGY EFFICIENCY AND RENEWABLE ENERGY REP Activity 3 aimed to increase in-country capacity in renewable energy and energy efficiency. This was to be accomplished by introducing improved technology and practices; contributing to rural energy productivity; identifying and implementing EE/RE interventions in target communities; and improving quality and affordability of EE/RE technologies. Activities included policy work, feasibility studies, trainings, public outreach/community education, and installation of pilot technologies. REP Approach to EE/RE At the outset of the program, REP project subcontractor World Experience Georgia conducted several overarching assessments of energy efficiency and renewable energy potential in Georgia. These assessments included review of existing Georgian policies, technical analysis, and recommendations of areas of possible support. In addition, feasibility studies on geothermal energy, biomass (biodigesters), wood heating stoves, energy efficient construction materials, and solar water heating systems were commissioned. Based on the findings of the above reports, REP conducted participatory assessments in rural communities. Community members were asked to indicate their interest in and likelihood of adoption of various household-level technologies. Based on the feedback collected, REP identified biodigesters as a technology worth piloting (results from the pilots are presented in the next section). REP also provided a home for various EE/RE projects of interest to Winrock and/or USAID/Georgia. The primary activity in this category was support of the Energy Bus, a vehicle developed and partially funded by BP to demonstrate and provide information on energy efficiency and alternative energy technologies throughout the Caucasus. REP also conducted some weatherization pilots, although it is not clear upon what basis this decision was made. The following targets were listed in the PMP for Activity 3: • 30 EE activities implemented/technologies installed • 15 RE systems installed • At least 20 energy audits conducted • Each IPP hydro community trained in EE and RE techniques • Two businesses trained to provide TA in fabrication or delivery of EE or RE technologies • EE/RE financing mechanism(s) established Related work from Activity 4 (Support institutional capacity and legal, policy and regulatory environment conducive to promotion of rural access to clean, efficient energy resources) included training Georgian policymakers on RE and EE policies. Specifically, REP worked to develop draft RE and EE laws, and also provided technical assistance on amending the Construction Code. Evaluation Team Methodology 19 Activity 3 encompassed a large number of activities undertaken throughout Georgia. With agreement from the AOTR, the evaluation team focused its efforts within the energy efficiency track on activities associated with improved stoves. In this context, the evaluation team interviewed the US-based stove consultant and a Georgian stove manufacturer that received technical assistance from the project. Team members also traveled to two villages located near the border with South Ossetia, and interviewed five households that received improved stoves through the REP program. The evaluation team did not assess the four weatherization pilot projects conducted under REP (these involved replacing windows and doors, and installing efficient stoves and light bulbs), as the follow-on NATELI project provides a better platform from which to evaluate Winrock’s work in this area. For the renewable energy track, the evaluation team visited five of the seven pilot biodigester sites, and interviewed the households using the biodigesters. While the REP team did not evaluate the Energy Bus activity specifically, it did evaluate how this activity fit within REP’s overarching RE/EE strategy. Findings and Results REP met some, but not all of the targets laid out in the PMP. Based on the data provided in the Final Report and the evaluation team’s findings, REP accomplished the following: Implementation of 20 EE audits and 4 weatherization projects Installation of 7 biodigesters 181 public awareness meetings conducted. Technology-specific training was provided to owners of biodigester systems. The households that received improved stoves reported that they did not receive training beyond a written manual. Eight businesses were provided with technical assistance (TA) on improving their stove designs during a 3-day workshop. Georgian university professors also attended the trainings. Overall, the evaluation team found no clear strategy directing the energy efficiency and renewable energy activities undertaken under REP. Many technologies were considered and a few piloted in rural communities, but financial and technical resources were spread too thin for specific interventions to be sustainably taken up or adapted in rural communities. Lack of financial mechanisms to support market development also hindered REP’s ability to bring any interventions to scale. Although REP expended significant effort to improve the enabling environment for energy efficiency and renewable energy by educating policymakers and providing draft language they could utilize, the Government of Georgia’s unwillingness to adopt new EE/RE laws/policies during the project timeframe negatively impacted REP’s ability to develop a lasting impact on rural uptake of EE/RE technologies. However, REP pilots did demonstrate the technical (if not financial) viability of two types of biodigesters and some improved wood stoves. Findings and Results: Improved Stoves Winrock enlisted the services of a highly regarded US expert to assess Georgian wood burning heat stoves and conduct a 3-day workshop for stove manufacturers and professors from Tbilisi State University on how to improve stove efficiency. The REP project reports this as an energy efficiency certification activity; however Mark Bryden, the US consultant, told the evaluation team that the workshop time horizon was too short to certify the stoves in a meaningful way. Though testing conducted at the beginning and end of the workshop revealed that some manufacturers had significantly increased 20 the combustion efficiency of their stoves through design improvements, the testing was not rigorous enough to constitute a reliable certification program. The approximate cost of an efficient stove is 200– 250 lari, compared to 50 lari for a traditional (smaller) stove. The stove training was too ad hoc and not integrated into an overall program designed to promote demand and supply of improved stoves. Given the widespread use of wood-burning heating stoves throughout rural Georgia and the fact that many people pay for wood (200-550 lari per truck load), a commercially sustainable market for improved stoves might be feasible. However, the REP program did not incorporate any financing or market demand components to supplement the producer training, which was also too limited to have significant impact. Given USAID/Georgia’s desire to protect watersheds and natural resources, an integrated efficient stove/forest management program might be an option worth exploring in the future. In response to the August 2008 South Ossetia conflict, REP distributed 499 efficient wood stoves manufactured by companies trained by the project to conflict-affected families living near the Administrative Boundary Line (ABL). The evaluation team interviewed five of these households, but was unable to determine the fuel savings and socioeconomic impact of the stoves, as there was no baseline for comparison and all of the families received state-subsidized firewood. The energy savings provided on p.34 of the final report are estimated and were not measured before or after installation by Winrock or its consultant. While the stoves likely did register some efficiency gains, discussions with Winrock’s stove consultant and inspection of the stoves by the evaluation team revealed room for further efficiency and safety improvements. The evaluation team did not measure the efficiency of the installed stoves. The interviewed families provided the following insights about their new stoves: • The stoves, which were sized to heat the one-room houses built by UN agencies, were able to do so consistently. • Households used the stoves to bake bread, but the reduced size of the stove and its oven made it difficult to bake khatchapuri (traditional Georgian cheese bread) and other foods. • About half the households reported their efficient stoves used less wood than their old stove. In some cases, the positioning of the stove in the house (i.e., opposite the door) and condition of the chimney (i.e., not adequately sealed or angled) were negatively impacting fuel consumption. • Some IDP households buy wood once per season, hiring a car or truck, spending 300 to 550 lari per vehicle. The Government also supplies wood to IDPs, but many complained that the government did not supply enough wood for the entire heating season. Findings and Results: Renewable Energy The evaluation team surveyed five biodigester-using households in three villages. All of the biodigesters were installed on a pilot and demonstration basis, fully financed by a project grant (approximately $2,000/unit) with a community contribution of labor. At the close of project, REP verified that all units were working after two years of installation, but did not measure actual gas production or usage, nor socio-economic impact on the families. The final report contains energy savings estimates based on figures in the initial feasibility study. All of the households visited in the final evaluation reported that their biodigesters were both safe and easy to use. While none of the households were connected to a central gas line, all households had access 21 to gas cylinders prior to the REP pilot, so installation of the biodigester unit did not move households up the energy ladder. Rather, replacement of purchased cylinder gas with self-produced renewable energy enabled some households to reduce their energy expenditures. On average, before installation of the biodigesters, households purchased two gas cylinders per month for cooking, in some cases supplemented with wood. After installation of the biodigester, most households were able to produce enough gas to meet all their cooking needs (although wood was still used for heating), although some still kept a reserve supply of balloon gas/wood to supplement seasonal home food production, such as canning. Aside from production of gas, biodigesters yield liquid fertilizer as a by-product. REP did not attempt to measure the impact of this fertilizer production on household income. REP evaluation team conclusions on the biodigester pilot program follow. REP did not obtain adequate data on gas production or socioeconomic impact of the biodigester units Without such data, it is not possible to ascertain whether this pilot program has the potential to be scaled up, or should be one-off. Based on the evaluation team’s estimates, biodigesters are not economically feasible in Georgia without a significant subsidy An average family reported it spent approximately 44 - 50 lari per month (equivalent, 26 to 30 USD unadjusted) on cylinder gas for cooking prior to installing the biodigester. After installation of the biodigester system, expenditures on cylinder gas dropped by around 50% to 22 – 25 lari per month (equivalent to 13 -14 USD unadjusted), for annual savings of around USD150-200. Some families were also able to obtain increased crop production by using the biodigester fertilizer, but many were unable to reap this benefit (see below). Given the current steep cost of the biodigester units (approximately $2000), it would require both a significant subsidy as well as dramatic price reductions for biodigesters to attract individual families to purchase them. Given the unlikelihood of accomplishing either of these factors in the near term, the evaluation team does not believe USAID/Georgia should invest more funding in promotion of biodesters in its next funding cycle. Families had difficulty utilizing the liquid fertilizer, since their biodigesters were located some distance from their fields. Transport of the fertilizer is difficult and expensive (one family looked into hiring a pumping truck, but that would cost 250 lari (equivalent USD146 unadjusted), leaving many families to just let the sludge accumulate in their yards (and in some cases spill into the street). One farmer reported he dried his fertilizer and then transported it to his fields (which were not too far away). Another instilled pipes and let gravity take the fertilizer to his home garden, while a third shared excess fertilizer (at no cost) with neighbors. The first two innovations enabled the farmers to save small amounts of money on purchases of chemical fertilizer, but not enough to significantly offset the cost of the biodigester unit. Moreover, since the farmers that received the biodigesters were growing food for their own consumption rather than for sale, increased crop yields did not result in increased income. It is a significant shortcoming of the REP pilots that they seem neither to have predicted the problem of transporting the fertilizer, provided no help to the farmers on how they might adapt their systems to be better able to use the fertilizer, nor reported this issue in the final report. 22 REP work under the RE/EE component would have benefitted from a stronger Monitoring and Evaluation framework. The pilot projects would have benefitted from development of clear baseline studies that established energy production and consumption, costs, and other variables before the pilots were implemented. Comparison data could then have been obtained mid-way and at project completion to enable informed analysis of the pilot’s impact and desirability of scale-up. Findings and Results: Update of Construction Code and Draft EE/RE laws Between spring 2007 and summer 2008, REP sought to bring together stakeholders and ministry officials to improve the enabling environment for energy efficiency and renewable energy. Based on a request from the Ministry of Energy, REP provided assistance to draft new EE and RE laws, drawing on international best practices adapted to Georgian circumstances. REP also assisted the Ministry of Energy to review a draft revised Construction Code from the Ministry of Economic Development. However, the Prime Minister requested REP to stop working on these issues in the summer of 2008. The evaluation team heard various explanations for this course of events, but it is impossible to determine precisely what happened or evaluate the work that REP did prior to stopping its work in this area. However, the intention to support the Government of Georgia to develop its policy framework fit REP’s mandate well, and would have been the seminal achievement of the EE/RE work had the policies/laws been adopted. REP’s approach to the work seemed well-founded and in line with comparable USAID efforts in other countries. 23 VII. Activity 5: Improve Local Community Capacity to Sustainably Utilize and Protect the Local Energy & Natural Resource Base The objective of this activity was to improve capacity in Georgia to conduct integrated natural resource and management planning by providing assistance to the Ministry of Environment and Natural Resources. Specifically, REP sought to: 1. Build community capacity to manage natural resources and watersheds; and 2. Implement targeted watershed projects in selected communities. Two companies were competitively selected by Winrock to work on environmental activities: 1) CENN – Caucasus Environmental NGO Network and 2) Water Household of Georgia. However, the contract with Water Household of Georgia was suspended early in the program, as it was unable to conduct the planned watershed assessments. Winrock was responsible for implementation and monitoring of all environmental compliance activities conducted under REP. Winrock head office staff trained CENN how to conduct watershed assessments, the main purpose of which is to examine the conditions and problems existing in a given watershed and identify the potential for impairment due to human and environmental factors. Watershed assessments for the hydro sites supported by REP were prepared by teams of three individuals—a hydrologist, geologist, and forester. Winrock head office staff also trained CENN how to prepare Integrated Resource Management Plans (IRMPs) for communities near small hydro plants, to help raise community awareness on environmental and social impacts of hydro developments. The IRMP incorporates the results of a field study and watershed assessment. It serves as a roadmap for communities to mobilize human and natural resources to define problems, consider previous successes, evaluate local institutional capacities, prioritize opportunities and prepare a systematic and site-specific plan of action for the community to adopt and implement. IRMPs are not well-known in Georgia and REP was one of the first programs to seek to build capacity in this area. The participatory IRMP consists of the following activities: Energy Assessment – Winrock: 4-5 days per community Map development – CENN: 4 days per community Watershed assessment (WA) – CENN: 10 days per community WA analysis and identification of IRMP communities – Winrock & CENN: 10 days in total IRMP participatory workshops – Winrock & CENN Evaluation team methodology 24 The environmental consultant reviewed key project documents, including representative watershed assessments and IRMPs; interviewed the REP environmental component lead from Winrock and its subcontractor (CENN); and interviewed several small hydro owners and technical staff. Findings and Results According to the final report, REP accomplished the following activities to meet these objectives: Developed 9 watershed assessments for river systems on which REP’s rehabilitation projects were located Produced 7 integrated resource management plans (IRMP) for communities near small hydro plants supported by REP Conducted four demonstration projects (tree planting and slope reinforcement), with 20% in￾kind contributions from communities Conducted training for 85 forestry department mid-level staff on effective control and enforcement of forest license conditions and procedures Trained 60 community members in three small hydro communities on natural resource conservation Although REP supposedly developed multiple environmental documents for the hydro projects supported under the project, the evaluation team had difficulty obtaining the documents. Few of the documents were available on the project website, and virtually none of the owners nor technical staff of the SHP sites visited could locate their environmental documents or even remember if they existed. In a few cases they stated that the documents were stored elsewhere (at home or in another office). Toward the end of the evaluation period two IRMPs were produced for the Kakhareti and Nergeeti sites, conducted in June 2008 and September 2007, respectively. These IRMP documents were reviewed by the evaluation team’s local environmental expert. The evaluation team’s conclusions are: REP succeeded at building capacity within a local NGO (CENN) to conduct natural resource assessments, but its efforts to train government officials and communities were not designed to have lasting impact. The activities planned under this component were aimed primarily at ensuring the small hydros funded or rehabilitated by REP were conducted in an environmentally appropriate manner, and could serve as demonstration sites for integrated resource assessments. The activities and presumed funding levels clearly were not designed to instill broad-based capacity throughout Georgia to understand and implement such studies; most of the reports were not even posted on the project website for public access. Once REP support (financial and technical) to the communities ended, they were unable to continue to implement activities on their own. Despite the fact that community representatives were trained in project preparation and fundraising, they seem unready to undertake independent action without external funding and guidance. Currently, Georgian legislation does not require the creation of IRMP or watershed assessments. Consequently, 25 there is no monitoring of the implementation of these plans in place in Georgia. There is no organization to which the community could apply to or which would be officially responsible to react to their inquiries. None of the forestry officials trained by REP remain in their jobs The training on ‘Monitoring and Control of the Regular Implementation of the Forest Licenses’ was held in Bakuriani from September 26 to October 18, 2007 and was organized and implemented by Winrock Georgia in close cooperation with CENN. The training program was designed for four groups of the employees of Ministry of Environment and Natural Resources Protection Forestry Department. Training for each group consisted of a 7-day workshop. Heads of Forest Management, Forest Monitoring Departments, and Regional Forestry Districts attended the training course. However, the Forestry Department was subsequently re￾organized and employees transferred to other areas of the Ministry or fired. VIII. FINAL RECOMMENDATIONS Although the Rural Energy Program is over, there are several lessons learned from REP implementation that may be useful for other USAID/Georgia programs, particularly in the energy sector. USAID/Georgia should continue to build capacity to mitigate environmental impact of energy generation, especially for hydropower REP efforts to build community and national government understanding and capacity to plan and mitigate environmental impacts related to hydropower development were limited in scope and had little impact. As USAID continues to work in the hydropower sector, it should continue efforts to institutionalize Georgian capacity to conduct and implement IRMPs, as well as to incentivize environmental good behavior and enforcement of Georgian environmental laws. USAID/Georgia should carefully monitor implementer capacity building/training programs to maximize impact Many of the trainings conducted under the REP program were one-day programs that served more to introduce attendees to a topic rather than build true capacity (especially in the environmental management and financial areas). While this approach might be appropriate for some topics or in some settings, such “training” is too superficial to truly build institutional capacity. In addition, in some cases it appears that trainings were not offering much in the way of new or critical knowledge. Conducting participant needs assessments prior to developing the training courses might have resulted in greater impact. USAID/Georgia must ensure that implementers have sufficient budget and capacity to design and carry out longer, more hands-on trainings to truly build capacity that can affect an entire sector or community. Programs should also incorporate more rigorous training evaluation frameworks/components to gauge the impact of capacity building 26 activities. In addition, projects should seek to build local capacity to conduct future trainings once foreign experts leave/donor funding ends. Pilot projects demonstrating new energy technologies should be selected and evaluated on economic as well as technical criteria A thorough economic as well as technical assessment should be conducted before pilot programs are designed, in order to identify existing barriers and collect baseline data that can be used to design the pilots. In addition, all pilots should be evaluated for economic as well as technical feasibility, so that follow-on programs are designed for the most promising technologies with the highest potential for commercial uptake. The REP energy efficiency and biogas pilots did not include any actual socio-economic impact data, and the training provided to the stove manufacturers consisted of a one-off effort not coupled with any efforts to reduce market barriers to the improved technologies (i.e., links to consumer financing). Similarly, the Energy Bus promoted products for which there were no particular market incentives to encourage uptake. The impact and lessons learned from these various activities would have been greater if more foresight had been put into their selection and implementation strategy. Strengthen efforts to support new financial products and training REP did a good job leveraging the financial resources of other donors to further the impact of USAID funding, especially for the co-financing of hydro projects. However, the Georgian financial sector still lacks understanding of, and thus interest in, non-recourse project finance, a common way to finance large infrastructure projects in many countries. Georgian financial institutions also lack ability to evaluate energy efficiency or alternative energy projects. As USAID/Georgia is likely to receive climate change funding for mitigation, increased capacity in the financial sector to finance clean energy projects will be important. In addition, USAID/Georgia could investigate opportunities to mitigate perceived risk of clean energy projects and enhance consumer financing/credit for the purchase of smaller-scale energy efficiency and renewable energy projects (such as those promoted under the NATELI project). 31 32 B. Evaluation surveys and questionnaires USAID Rural Energy Project (REP) Survey Questions We have identified your bank as participating in USAID/Georgia’s Rural Energy Program (REP). The following survey is designed to help USAID evaluate the impact of REP and the quality of services that you received. All answers will be kept confidential. Please return your responses to mission fill in by tk. 1) Name of bank Name and title of person filling out this survey 2) How does your bank view the risk level of hydropower projects? Low Medium High 3) How many energy loans has your bank issued? # loans: # loans: %: # loans: %: How many hydropower loans were written before October 2009? What percentage of the energy loan portfolio was this? How many hydropower loans were written after October 2009? What percentage of the energy loan portfolio was this? 4) What is the average value of the energy and hydropower loans offered by your bank? What is the average term of the loans? Energy value: Energy term: Hydro value: Hydro term: 5) Has this amount or the term changed since October 2009? (If yes, please specify how either has changed) 6) How many defaults have occurred on energy loans? 7) How many defaults were hydropower projects? 8) Have you restructured hydropower project loans to avoid default? Yes No 9) What is the current interest rate for a loan to a small hydro project (less than 13MW)? What is the average interest rate for loans to larger hydro projects? 10) Does the bank anticipate providing future loans to hydro projects? Yes No 11) Has your bank received assistance from Winrock/the REP program? Yes No 33 12) Did your bank’s staff participate in REP training programs? Please circle/check the trainings attended by staff from your bank. Project Finance Evaluating Small Hydro Projects Renewable Energy Financing 13) How would you rate the assistance from REP? Very Useful Somewhat Useful Not useful USAID Rural Energy Project (REP) Survey Questions We have identified your company as participating in USAID/Georgia’s Rural Energy Program (REP). The following survey is designed to help USAID evaluate the impact of REP and the quality of services that you received. All answers will be kept confidential. Please return your responses to mission fill in by tk. 1) Name of site 2) Name of person filling out this survey and title Name: Title: 3) Did you secure bank financing for this project? If so, how much, and from what source? Bank financing amount: % of total investment: Source of financing: 4) Have you defaulted or had to re-schedule any loan payments? Yes - Defaulted Yes - Re-Scheduled Loan Payment No 5) In the Summer, Who do you sell your power to, what % of your power do they purchase, and how long are the agreements? Name % PPA term 6) In the Winter, Who do you sell your power to, what % of your power do they purchase, and how long are the agreements? Name % PPA term ESCO 7) Are the offtakers paying according to the terms of their contracts? Yes No 8) What type(s) of assistance did you receive from the Rural Energy Program (REP)? Please circle all that apply Grant funding (amount): Training Watershed assessment Integrated Resource Management Plan Other: (pls specify) 9) How would you rate the overall usefulness of the REP assistance? Extremely useful Somewhat useful Not useful 10) Would you have been able to obtain bank financing without REP assistance? Yes No 11) Have you invested in any other hydro/energy projects since receiving REP assistance? Yes No 12) If you answered yes to #11, did you receive financing? How much? Yes No Amount: 13) Was there any area for which you wish you could have received more REP Yes No 34 assistance? If so, which? 35 REP evaluation Questions for IDPs/Former IDPs regarding Improved Stoves Name of interviewee: ___________________________________________ Gender: male_____ female_______ Position in household (hh): __________________________ Type of hh structure: ______________________________ Size/composition of family: _________________________ Location of hh: ____________________________________ 1) How many months have you had your new stove? (check ONE) a. 0-3 b. 3-6 c. 6-12 d. Greater than 12 2) Would you have been able to obtain a new stove without this program? Yes (1) / No (2) a. If yes, where/how? i. Free from friend/family member ii. Bought at market/store iii. Bought from friend/family member iv. Build it myself v. Other (specify) ____________________ 3) Did you pay any money for your stove? Yes (1) / No (2) i. If yes, how much? (specify amount) ____________________ b. If yes, is this a reasonable amount for your budget? Yes (1) / No (2) 4) Did you receive training on how to properly operate your new stove? Yes / No (if yes, get details) 5) Are you happy with your improved stove? Yes (1) / No (2) a. If yes, why? (check all responses given) i. Heats house well ii. Easy to use iii. Works well for cooking iv. Produces less smoke v. Uses less fuel vi. Is attractive vii. Other (specify) ________________________ b. If no, why not? (check all responses given) viii. Does not heat house well ix. Difficult to use x. Doesn’t work well for cooking xi. Produces more smoke xii. Uses as much or more fuel 36 xiii. Is not attractive xiv. Other (specify) ________________________ A. INCREASED FUEL SAVINGS 6) Who in your household is responsible for buying/collecting fuel? a) female head of household b) male head of household c) male child d) female child e) other (specify__________________________________________________) 7) Have you saved fuel using your new stove? Yes (1) / No (2) / don’t know c. If yes, how do you know you have saved fuel? (check all responses given) i. I keep track of fuel I use ii. I am collecting fuel less often iii. I spend less money on fuel iv. Other (specify) ____________________ d. If yes, how much less fuel do you use with your new stove? i. 1-50% less fuel compared to my old stove ii. Between one half and 100 percent of the amount I needed for my old stove e. If no, do you use the same amount of fuel for your new stove as for your old stove, or do you use more fuel? i. Same amount of fuel ( skip to 10a) ii. More fuel f. If you use more fuel for your new stove, why? (check all responses given) i. New stove is less efficient/requires more fuel ii. I don’t know how to use my new stove well iii. I use my stove more often iv. New stove is bigger/provides more heat v. Don’t know vi. Other (specify) ________________________ 8) If you saved fuel with your new stove, do you save money every week by not having to buy as much fuel? Yes (1) / No (2) g. If yes, how much money do you save per week? (in local currency and USD) (check ONE) i. Up to US$3 i.a. Local currency equivalent ___5 lari____ 37 ii. US$3 to US$5 ii.a Local currency equivalent ____5-8 lari___ iii. US$5 to US$10 iii.a Local currency equivalent ____8 -18 lari___ iv. Greater than US$10 iv.a Local currency equivalent ____more than 18 lari___ h. If no, why aren’t you saving money as a result of needing less fuel? (check all responses given) i. Fuel has gone up in price ii. I need to buy more fuel for other reasons iii. I collect my fuel (do not purchase) iv. Other (specify) _____________________ 9) How many times per week did you buy/collect fuel for your old stove? (check ONE) a. v. 0-3 vi. 4-6 vii. More than 6 b. How many times per week do you buy/collect fuel for your new stove? (check ONE) viii. 0-3 ix. 4-6 x. Greater than 6 10) (if answer to 10b is less than 10a, ask #11—otherwise skip to #12) How much time do you save in one week by buying/collecting less fuel? (check ONE) xi. Less than 1 hour xii. One-half day xiii. Around one full day xiv. More than one day xv. Don’t know xvi. I don’t save time (try to ascertain why not) i. What do you do with the time you save? (Check all that apply) i. Rest/leisure ii. Income-generation activities iii. Spend time with family iv. Other (specify) B. FUEL CONSUMPTION 38 11) Indicate the type of fuel you use in your new stove 1 = Use frequently 2 = Use occasionally 3 = Use rarely 4 = Never use a. Firewood b. Charcoal c. Crop residues d. Straw e. Twigs f. Leaves g. Roots h. Dung i. Kerosene j. LPG k. Other: __________ 12) Do you have another heat or cooking source in your house—besides the new stove? What kind? C. REDUCED RISKS OF HOUSE FIRES AND BURNS 13) Have you or any of your children been burned by the new stove? Yes (1) / No (2) a. Did the person who was burned require medical help? Yes (1) / No (2) 14) Are you able to use your new stove safely? Yes (1) / No (2) a. If no, why not? i. Stove is not safe 39 ii. I didn’t receive adequate training iii. Other (specify) _________________________ 40 REP Evaluation Interview Questions for Biodigester owners & family Interviewer should fill out prior to interview: Interviewer: Biodigester location: Date: Name of interviewee: Number of people in household: Status in household: A. 1) Collect Biodigester basic info When was the biodigester installed: What type of biodigester was installed: What were the materials used: Did you pay for your biodigester? If YES: How much? i. ____ lari. ii. Cost Share—so paid for a portion of it. _____ % iii. In-kind labor: __yes, dug all of the holes in the ground for the biodigester installation_______ iv. Other: _______________ B. Biodigester use 2) Is the biodigester easy to use? 3) How many cattle (other animals) is the biodigester based on? (specify # and type of animal) 4) How is gas collected from the biodigester? 5) What do you do with the gas from your biodigester? 1) Cook food, tea/coffee for daily consumption 2) Heat house 3) Other: _________________ 6) How many hours of gas from your biodigester do you use each day? [Note: 1 hr = 1 hr of cooking = 1 hr of heating] 7) Does the biodigester produce enough gas to meet all of your requirements? Yes/no 41 C. Increased fuel savings 8) How much money do you estimate the biodigester saves you each month (savings = free gas + free fertilizer – maintenance costs) [check one] Less than 10 lari per month 10 – 20 lari per month 20-50 lari per month More than 50 lari per month 9) Is your house connected to a gas pipeline? If YES: 1) Do you still use this central gas supply, now that you have a biodigester? i. How many hours per day? ii. In which seasons? 2) Approximately how many hours of gas did you use each day before the biodigester was installed? i. How much did you pay for this gas (per month or another time period depending on respondent recall)? ii. Was gas affordable for you? If NO: a. What did you use for fuel before? 1. kerosene 2. wood and still do for supplemental cooking fuel and heating. 3. hay 4. peat 5. other: ________________ b. What types of activities/chores does the biodigester help you or your family with? Men’s activity/chores in the household? Women’s activity/chores in the household? Children’s activity/chores in the household? c. Does gas from the biodigester save you or your family time on these chores? [If respondent needs prompt, add: For example, did you previously spend a lot of time collecting and preparing a wood fire for cooking (heating water, washing clothes, etc)? Now that you can use a gas stove, does this save you any time? Please estimate how much time per day you save.] If YES: 42 a. How do you use the time you have saved? More time on: a. More farm work? b. Off-farm work? c. Leisure time? D. Secondary benefits/oucomes 10) Does your biodigester produce fertilizer? Yes / No If YES: What do you do with the fertilizer produced by your biodigester? E. Biodigester upkeep, maintenance, and use 11) How much time do you spend on daily upkeep of your biodigester adding fuel or removing fertilizer? 1) Less than 1 hr per day 2) More than 1 hr per day 3) Other estimate: __________ 12) Who in your households does daily chores to add fuel? 13) How much time do you spend on other maintenance, such as cleaning and repair? 1) Less than 5 hr per month 2) More than 5 hr per month 3) Other estimate: _________ 14) Who in your household does other maintenance work? 15) Do you use fresh or stored manure in the biodigester? 16) Do you use human waste in your biodigester? 17) Do you use any other organic materials in your biodigester? [Open ended, but examples are: crop residues; 2, human waste, food processing wastes, other?] 18) Do you use your biodigester year-round? Yes/no YES 1) Does it work better in certain seasons? 2) Do you make any seasonal adjustments? [Open ended, but if respondent needs a prompt, try: mixing hot water with manure? Insulation—with what? Other?] 19) Have you received any service from the company that produced your biodigester? If YES: a) What was the service the company provided? b) Are you satisfied with the service? c) Did you pay for this service? How much? 43 20) Did you participate in training on how to install and/or maintain your biodigester? 1) Do you think this training was sufficient? 2) Do you or other members of your household need any additional training? F. Secondary benefits/outcomes continued 21) Have there been any unexpected benefits from your biodigester? (yes/no) YES 1) ability to earn extra income (specify________________________) 2) house/farm is cleaner/more sanitary 3) other (specify_________________________________________) 22) Has your biodigester caused any conflict within the community? Yes/no 23) Have there been any accidents associated with the biodigester? 1) Burns 2) Explosions 3) Other (specify) 24) Have you taken any added safety precautions or added any safety features? If YES: Like what? 25) Are you satisfied with your biodigester system? Yes/no (elaborate) 26) Do you have any suggestions on how to improve your biodigester system? 27) Have you recommended a biodigester to your friends and neighbors? Yes/no 28) Did anyone else in your community purchase or construct a biodigester after seeing yours? Yes/no (If yes, who?) 29) How much money or hours of labor would you pay for a biodigester? Did not ask. 30) Have you stopped using your biodigestor for any reason? Why?