i GENERAL MANAGEMENT ASSISTANCE CONTRACT (GMAC) Contract No: 674-C-00-01-00051-00 MTI Subcontract No.: 0172-0406-PO-ME16 Final Evaluation Report - Deliverable 3 DATE July 2006 This report was produced for review by the United States Agency for International Development. It was prepared by Palmer Development Group, under Mega-Tec, Inc.’s prime contract and addresses USAID/South Africa’s Strategic Objective No. 6: Increased Access to Shelter and Environmentally Sound Municipal Services. The author’s views expressed in this publication do not necessarily reflect the views of the U.S. Agency for International Development or the United States Government i ii Please direct all queries regarding this report to: Mega-Tech/South Africa Bank Forum Building Lobby 1, Second Floor 337 Veale Street New Muckleneuk 0181 Pretoria RSA Tel. 012 452 0060 Fax 012 452 0070 Email megatech@intekom.co.za Or Mega-Tech, Inc. 180 South Washington Street, Suite 200 Falls Church, VA 22046 Tel. (703) 534-1629 Fax (703) 534-7208 Email info@mgtech-world.com ii iii Executive Summary Background The ICLEI Cities for Climate Protection (CCP) campaign is aimed at developing the capacity of local governments to achieve measurable reductions in greenhouse gas (GHG) emissions to improve air quality and enhance urban quality of life. The campaign is funded under the DEAT/USAID bilateral agreement. In South Africa, eleven municipalities currently participate in the initiative. The program has recently ended, necessitating this evaluative review. The campaign had three distinct phases: Phase 1 was aimed at allowing municipalities to develop projects which have strategic value or provide genuine benefit of cost saving. The phase placed a strong emphasis on training, capacity building and networking and provided a strong platform for the other two phases. The focus of phase 2 was to expand the program in terms of the number of participating cities and to assist participating cities to implement a range of larger projects and initiatives through partnerships with the cities. The main objective was to focus on the implementation of clean energy and transportation measures by: • Providing more targeted technical assistance to the cities on these tasks, such as in the development of PINS, assessing audit services and more • Actively leveraging the opportunities/resources identified in the project’s first phase The objectives of phase 3 centred on increasing awareness around air quality and projects were guided to focus on air quality monitoring and transport. Grants Awarded The total amount of grant funding awarded by USAID over the life span of CCP in South Africa totalled $2,533,869.00. Included in this figure are both the grants awarded to municipalities and any funds that were necessary for the administering of training sessions and workshops. Grants awarded for each phase to the various municipalities are as follows: Total Grants awarded to cities in each phase Project Grants Awarded( US $) City Phase 1 Phase 2 Phase 3 City of Cape Town 56, 000 35, 000 108, 500 eThekwini Metro Municipality 56, 000 36, 000 uMthlathuzi Local Municipality (Richard’s Bay) - - - Mangaung Local Municipality - - 81, 000 Saldanha Bay Local Municipality 52, 000 28,000 - Sol Plaatje Local Municipality 52, 000 - 2, 500 City of Ekhuruleni Metropolitan Municipality - 40, 000 25, 500 City of Johannesburg 56, 000 - - iii iv Metropolitan Municipality Buffalo City Local Municipality 52, 000 - - Tshwane Metro Municipality 56, 000 36,000 - Potchefstroom Local Municipality 52, 000 - 96, 200 Total Award (US $) 432, 000 194, 000 313, 000 Objectives of the evaluation The primary objective of the review is to undertake and evaluation of the ICLEI CCP campaign in South Africa to identify areas of success, weakness, accomplishments, and lessons learned within the program result areas, as well as to comment on its impact and effectiveness. A secondary objective of the evaluation is to review and comment on the quantitative data used to report on the program which included verifying the final number of households that benefited from the program, funds leveraged and data reported in the Global Climate and Clean Energy reports with prior performance reporting. Approach The evaluation used two main approaches to data collection: • Review of reports and other documents related to the program, produced by ICLEI and municipalities • Interview of municipal officials (and Councillors where available) and other key stakeholders(primarily ICLEI and DEAT) The aim of each was to collate quantified data related to the program, its inputs and outputs, and to gather more qualitative information and feedback from each of the interviewees on the program, its successes, failures, limitations and legacy for the future. The nature of interviewing meant that this approach had a strong emphasis on gaining experiential and anecdotal information on achievements at the municipal level, whilst the analysis of reports was the main method for extracting quantifiable project and program level data. Overall Impressions • In terms of establishing a local network, the project has been an undeniable success. It has opened up opportunities for municipalities to engage with each other around climate change, something which otherwise may not have taken place. • The CCP has also given local officials the opportunity to make a positive contribution to climate change and energy efficiency and this has stimulated creative and energetic responses to global issues at the local level. • As seed funding, the USAID and DEAT funds that were channelled through CCP have had a significant effect on local government environmental initiatives in the field of energy efficiency. It is clear that this funding has been a fundamental spark for many other projects and initiatives aimed and sustaining energy and addressing climate change. The spin-off effects have been noted by all those interviewed in this evaluation and there is a genuine appreciation amongst municipalities for the contribution that these funds have made to increasing awareness and stimulating other initiatives in the local municipality. The proven effects of the projects implemented and the knowledge gained and shared between municipalities have resulted in an impressive snowballing of iv v activity within municipalities which can be traced back to the influence of the ICLEI CCP program. • Workshops and technical training were generally well received and local officials have clearly benefited from these sessions, and in some cases so too have politicians. Lessons learned • The need for local support and capacity building cannot be underestimated. Municipalities have stressed the important value that they received from training and it is clear that in order to sustain work of this nature training and capacity building needs greater development. • The platforms provided for knowledge dissemination, experience sharing and fieldtrips have been inspiring. They have enlightened cities and exposed them to opportunities that are practical to apply and which can make a meaningful difference to both cost savings and environmental action. • The positive implications of strong and stable institutional structures and similarly the negative impact of weak institutional structures are important lessons to carry forward. • The presence of a champion in the municipality has been important to the success of these projects. It also noted that a good working relationship and understanding between the technical official and politician goes a long way in achieving wider success with these initiatives. • While individual drive and commitment makes a large impact on the success of initiatives it important that too much of the success does not rest on a few individuals. Sustainability depends in large part on creating wider awareness, obtaining greater buy-in and institutionalising the motivation to achieve results to become more energy efficient and responsive to climate change and its affects. • Communication and support are important for success, both within and between the municipalities as well as between the key role-players from other entities involved in the network of CCP. • National support and the provincial – local interface need to be advanced for local initiatives such as CCP to have a national impact. Recommendations for ICLEI • ILCEI needs to work on strengthening and stabilising their South African office, especially since it is the Africa Directorate. • Staff retention should be a focus and more significantly the retention of knowledge and passing on of skill should be given more attention. Institutional memory should also be addressed so as to ensure that the exit of human capital does not result in the exit of the knowledge base in the organisation. • There needs to be greater consideration given to understanding and overcoming internal obstacles facing municipalities such as local legislation e.g. procurement policy, which needs to be taken account of when implementing programs. This may indicate a need to improve channels of communication so as to engage with local officials on these issues prior to implementation. • Clear definitions and communication of the roles and responsibilities of ICLEI staff as well as other stakeholders are vital to the flow of the program. Municipal staff in the CCP cities needs to be informed as to who the role players are and v vi where responsibilities rest, as it will allow them to channel their communication to the appropriate people and may improve the efficiency and effectiveness of programs. Recommendations for DEAT • The fact that many municipalities were not aware of DEAT’s role in the program indicates a lack of communication from DEAT’s side. • It is clear that DEAT needs to play both a supportive and leadership role in climate change and energy efficiency initiatives in South Africa as municipalities look to national government for direction. Recent developments indicate that some headway has been made in this regard, with DEAT currently in the process of establishing a sub-directorate for municipal support. • DEAT is well placed to play a co-ordinating role of municipal activity and local government environmental initiatives. DEAT may do well to serve as the core of the local network and can play a key role in developing local skill and capacity by organising regular training and sharing workshops for local government in future. • Many of projects that have come out of the CCP have been very successful and it may be worthwhile exploring public-private partnerships to expand this kind of initiative to the private sector. Recommendations on Phases 1 and 2 • The flexibility of phase 1 and the fact that it allowed the municipalities to be innovative with their project design has been highly commended and a key part the success of this phase. It is recommended that this approach be continued in other initiatives of this kind as it is likely to stimulate great interest and promote the development of creative ideas for reducing emissions and saving energy. • While the value of networks has been identified as a key achievement of CCP in South Africa it is noted that these need to be more organised and focussed rather than too loose and informal. Informal relationships between local officials have been invaluable but there is a concern that this informality could weaken these networks if individuals leave the municipality. While it is understandable that personal relationships sustain these networks, it is preferable that the networks be institutionalised rather than rest on individual relationships. • The flow between phases 1 and 2 seems to have had good continuity but it is recommended that more needs to be done to monitor and evaluate each phase before progressing to the next, especially as phase 2 built on phase 1. • Verification and adequate assessment of the quality of the inventories has been a problem of note, one which has had repercussions later down the line. Accuracy of the inventories is important and it is imperative that inventories are verified and checked for reliability in the early phase. • The quality of technical skills to complete the inventories correctly needs to be identified early on and where there is a lack of skill, high-level support needs to be provided so that the quality of the inventories is ensured. • The 5 milestones process worked well because it provided direction and guidance for what should be achieved in the initiative. However, the monitoring and evaluation component was not adequately fulfilled and more needs to be done in future to adequately implement mechanisms for this. • There is a keen interest in some of the smaller cities to make use of HEAT software in future and ICLEI has indicated aspirations for this as well. However, vi vii it is clear that the greatest impediment to these goals is the lack of local support for municipalities. Thus it is recommended that before pursuing the expanded usage of HEAT software, local support and technical capacity is established to support municipalities in making effective use of this software. Recommendations on Phase 3 • There appeared to be a lack of continuity and flow from phase 2 to phase 3, most notable in the change of agenda and the relatively restrictive approach taken in phase 3 compared to the earlier phase. • Planning around phase 3 and the use of lessons learnt from the earlier phases may have helped improve the continuity between the phases. This links again with the importance of monitoring and evaluation between phases. • Phase 3’s restrictive focus on air quality and transport only, tight time frames and deadlines seemed to have had negative implications for the success of this phase. Many municipalities felt under enormous pressure to achieve unreasonable and unrealistic goals, which lead to a lower level of participation in projects for this phase. Thus it is recommended that reasonable time frames be adopted and that greater flexibility be given to inspire municipalities to devise innovative ideas for dealing with climate change and energy efficiency. vii viii Abstract This document is the preliminary evaluation report of the United States Agency for International Aid (USAID) and Department of Environmental Affairs and Tourism (DEAT) financed International Council for Local Environmental Initiatives (ICLEI) Cities for Climate Protection (CCP) campaign. viii ix Acknowledgements PDG would like to acknowledge and thank all representatives of ICLEI, DEAT and cities which participated in the CCP program for their enthusiastic responses to our questions and general interest shown in the evaluation process. ix x List of abbreviations and acronyms ICLEI – International Council for Local Environmental Initiatives USAID – United States Agency for International Development DEAT – Department of Environmental Affairs and Tourism CCP – Cities for Climate Protection GHG – Greenhouse Gas LPG – Liquid Petrolium Gas MoU – Memorandum of Understanding COP8 – Conference of Parties WSSD – World Summit on Sustainable Development x xi Contents Contract No: 674-C-00-01-00051-00 i Executive Summary iii Acknowledgements ix List of abbreviations and acronyms x 1 Introduction 1 1.1 Background 1 1.2 Problem Statement 1 1.3 Assumptions 1 2 Objectives of the evaluation 2 3 Evaluation methodology 3 3.1 Approach to Data Collection 3 3.1.1 Reports & Documents Reviewed 3 3.1.2 Stakeholder Interviews 4 3.2 Quantitative Data Collection & Verification 5 3.2.1 Identification of available quantitative data 5 3.2.2 Identification of Indicators 5 3.2.3 Verification of Data 6 3.3 Data Analysis 6 3.4 Limitations of Methodology 6 4 Evaluation findings 8 4.1 Overall Program Performance 8 4.1.1 Performance against broad goals and aspirations 8 4.1.2 Performance against Required Indicators 10 4.1.3 Performance against 5 milestones 12 4.2 Program performance by phase 12 4.2.1 Phase 1 12 4.2.2 Phase 2 17 4.2.3 Phase 3 19 4.3 Data quality and verification 23 4.3.1 Data collection 23 4.3.2 Reporting 23 4.3.3 Aggregation 23 4.3.4 Verification 23 4.4 Institutional issues 24 4.4.1 Program organisation and administration 24 4.4.2 Municipal implementation arrangements 26 xi xii 4.5 Networking and knowledge management 28 4.5.1 Conferences and training 28 4.5.2 Informal networking 28 4.5.3 Mentoring arrangements 28 5 Conclusion 28 5.1 Overall Impressions 28 5.2 Lessons learned 29 5.3 Recommendations 29 5.3.1 Recommendations for ICLEI 29 5.3.2 Recommendations for DEAT 30 5.3.3 Recommendations on Phases 1 and 2 30 5.3.4 Recommendations on Phase 3 31 Appendices 32 Appendix 1 32 Appendix 2 35 xii ICLEI CCP – Final Evaluation 1 1 Introduction 1.1 Background The ICLEI Cities for Climate Protection (CCP) campaign is aimed at developing the capacity of local governments to achieve measurable reductions in greenhouse gas (GHG) emissions to improve air quality and enhance urban quality of life. The campaign is funded under the DEAT/USAID bilateral agreement. In South Africa, eleven municipalities currently participate in the initiative. The program has recently ended, necessitating an evaluative review. 1.2 Problem Statement The CCP Program implemented in South Africa is part of a larger ICLEI CCP campaign funded by USAID and its partners (the South African partner being DEAT). As such, there are a set of founding goals, objectives and expectations from funders and from the global ICLEI community for this program. As this program is drawing to a close, an evaluation is thus required to identify whether the program has met expectations and to draw out the impacts, lessons learned and recommendations from all involved for how a program such as this could be improved in the future. 1.3 Assumptions • Data provided by the municipalities regarding this project is accurate and valid (i.e. there will be no attempt to verify municipal data through more rigorous examination of their records) • Verification of national level data compiled by ICLEI can be adequately carried out through an examination of municipal data used to compile national totals • A full set of progress reports and other reports related to the program is available and would be made available to the evaluation team during the course of the program. 1 ICLEI CCP – Final Evaluation 2 2 Objectives of the evaluation The primary objective of the project is to undertake and evaluation of the ICLEI CCP campaign in South Africa to identify areas of success, weakness, accomplishments, and lessons learned within the program result areas, as well as to comment on its impact and effectiveness. Specific objectives of the evaluation have been included in the contract for this work and are as follows: To carry out a review of the results, achievements and impact of the program, highlighting areas of achievement, in general as well as progress towards the following objectives: • Report on the extent to which ICLEI achieved the results and objectives as defined in the program descriptions for each phase. • Summarise briefly the activities supported by ICLEI catalyst funding by type and the impact of each of the activities on energy savings, GHG emission savings, municipal operations, and people-level impact. • Review the extent to which the ICLEI program improved environmental awareness and built momentum inside the municipality for implementing additional environmental activities, promoting environmental legislation, and involving South Africans in international discussions on environmental issues, especially related to climate change. • Evaluate the likelihood that any of the successes noted above will continue after the USAID/ DEAT support for ICLEI ends. • Review and comment on the quantitative data used to report on the program which included verifying the final number of households that benefited from the program, funds leveraged and data reported in the Global Climate and Clean Energy reports with prior performance reporting. 2 ICLEI CCP – Final Evaluation 3 3 Evaluation methodology 3.1 Approach to Data Collection The evaluation used two main approaches to data collection: • Review of reports and other documents related to the program, produced by ICLEI and municipalities • Interview of municipal officials (and Councillors where available) and other key stakeholders(primarily ICLEI and DEAT) The aim of each was to collate quantified data related to the program, its inputs and outputs, and to gather more qualitative information and feedback from each of the interviewees on the program, its successes, failures, limitations and legacy for the future. The nature of interviewing meant that this approach had a strong emphasis on gaining experiential and anecdotal information on achievements at the municipal level, whilst the analysis of reports was the main method for extracting quantifiable project and program level data. 3.1.1 Reports & Documents Reviewed The following documents were reviewed by the evaluation team: National level CCP related reports: • ICLEI CCP Campaign in South Africa Final Report, June 2006 • ICLEI Phase II Exit Report 2005 • CEI Report (USAID/ South Africa Office of Housing and Urban Environment) Period Oct 1 2005 – March 31 2006 • Strategic Objective 6 Report, USAID/South Africa Office of Housing and Urban Environment. 2006 (Rand value of new or improved urban services leveraged through USAID supported programs) • Strategic Objective 6 Report, USAID South Africa Office of Housing and Urban Environment 2006. Number of Households assisted to obtain new or improved urban services through USAID supported programs • GCC Report for FY 2005 (and accompanying spreadsheet of data for HHC for this period) • Quarterly Progress Reports by ICLEI to USAID/South Africa: 3rd Quarter 2004 to 1st Quarter 2006 ( 7 reports in total) • HEAT training materials Background Documents: • Program Description for Expanded Cities for Climate Protection Program: Attachment 4 Phase 2 Scope of Work • Program Description for Expanded Cities for Climate Protection Program: Attachment 5, Phase 3 Scope of Work • Cooperative Grant Agreement Program Description: South Africa Cities for Climate Protection Program 3 ICLEI CCP – Final Evaluation 4 • An End of Program Evaluation of the Cooperative Agreement between USAID/ EGAT and the International Council for Local Environmental Initiatives (ICLEI) (submitted to Bureau of Economic Growth, Agriculture and Trade, Office of Environment and Science and Policy, Global Climate Change Team Oct 2005) Selection of project reports from Municipalities: • Potchefstroom • eThekwini • Cape Town • Sol Plaajte • Mangaung • Ekurhuleni • Tshwane Note: The evaluation team experienced great difficulties in obtaining a full set of national level reports for the period of the program (i.e. since its inception in 2001 – to present). The partial set of quarterly progress reports, CEI and GCC reports noted above was given to the team by USAID at the start of the evaluation process. Subsequent requests were made to ICLEI South Africa, Roger Williams (ICLEI) and to USAID for the remaining reports in these series. It was made clear through email correspondence to each of the above that a thorough evaluation (particularly of quantitative data and its data quality) depended on a full set of reports being made available. This did not take place; as such the evaluation had to reply on the partial set noted above. The ability of the evaluation team to comment on quantitative data and its quality has thus been limited, as will be seen in section 4.3 below. 3.1.2 Stakeholder Interviews The following stakeholders were interviewed by the evaluation team. The majority of the interviews were carried out in person. Where it was necessary to carry out telephone interviews – this is noted. Table 1 Municipal Stakeholders Interviewed MUNICIPALITY PERSON INTERVIEWED City of Cape Town Metro Craig Haskins and Shirene Rosenberg eThekwini Metro Municipality Jessica Rich uMthlathuze Local Municipality (Richard’s Bay) Fred Phillips (telephone) Mangaung Local Municipality Tumelo Shakwane Saldanha Bay Local Municipality Ryan Groenewald and Piet Fabricious Sol Plaatje Local Municipality Robert Buchanan City of Ekhuruleni Metropolitan Municipality Deborah Ramalope City of Johannesburg Metropolitan Municipality Jane Eagles & Flora Mokgohloa Buffalo City Local Municipality Vuyani Dayimani (telephone) City Tshwane Metro Municipality Sam Mutswari (and Juan Mostert–telephone) Potchefstroom Local Municipality Mahesh Roopa and Councillor Stoltz 4 ICLEI CCP – Final Evaluation 5 Table 2 ICLEI and DEAT stakeholders interviewed ORGANISATION PERSON INTERVIEWED ICLEI – Johannesburg Annie Sugrue ICLEI – Johannesburg Luluma Matooane (former employee) ICLEI – Consultant Catherine Fedorsky ICLEI – Head Office (California) Roger Williams (telephone) DEAT Shirley Moroka Note: a preliminary interview (telephone) was carried out with Polokwane municipality prior to the start of the interview phase to establish their involvement in the program. The municipality pointed out that due to internal issues, they had not participated in the program in practice and felt they could add little to the evaluation process. A decision not to include Polokwane in subsequent elements of the evaluation process was made (in consultation with Mega-tech). Each interview was conducted following a pre-designed ‘interview instrument’ (see Appendix 1 for municipal interview instrument and Appendix 2 for ICLEI and DEAT stakeholder interview instrument). The interviews were intended to provide the municipalities with an opportunity to discuss their experiences of the program, the impacts it has had on the work of the municipality, any difficulties encountered, likely future sustainability of climate change work and so on. They were conducted as semi￾structured interviews to ensure that there was scope to allow the municipalities to provide information important to them, but within a framework of information required by the evaluation team. 3.2 Quantitative Data Collection & Verification 3.2.1 Identification of available quantitative data The evaluation team examined the reports available (see section 3.1.1 above) to identify the types of quantitative data available. This data was listed and used in later stages of data analysis. 3.2.2 Identification of Indicators The evaluation team identified a number of indicators against which the performance of the CCP program is required. These indicators are related to the three USAID related funding channels through which the program has received funding i.e.: • USAID Strategic Objective 6 (6.4) • USAID Environment and Science Policy (ESP), Global Climate Change Program • Presidential Initiative: Clean Energy Initiative (Healthy Homes & Communities – HHC program) It is believed that reporting to specific indicators was not required by DEAT (co-funders of the program). Information on the actual reporting against these objectives is provided in section 4.1 below. 5 ICLEI CCP – Final Evaluation 6 3.2.3 Verification of Data The purpose of verification is to identify the following: • Accuracy of quantitative data • Consistency between national level data and progress reporting – with data provided by the municipalities • Unpacking of aggregated data (reported at national level by ICLEI) to identify whether aggregation from municipal data has been done consistently and accurately by ICLEI • Examination of whether claims about the program by ICLEI at the national level are compatible with experiences at the municipal level. This evaluation carried out verification by comparing data presented by ICLEI in the national–level CCP reports against information provided by municipalities (in their reports and through the interviews). In doing so, an attempt was made to address each of the elements listed above. Note was also made of any comments on data quality1 provided by ICLEI in its reporting. It was felt that the above steps would allow the team to draw conclusions about the overall quality of the data available for the program. 3.3 Data Analysis Quantitative data was sorted and analyzed to identify the following: • Indicators for which information was reported • Performance of the program against indicators (using data available) Qualitative information (from interviews and reports) was used to supplement the quantitative data to identify: • Program performance: progress against defined objectives and deliverables (by phase) Qualitative data alone was used to identify broader issues, themes and comments related to program performance, achievements, barriers, suggestions for improvement and so on. 3.4 Limitations of Methodology The evaluation team did not find any problems in the collection of qualitative and experiential data for the program. Every municipality which has participated in the program gave generously of its time to participate in interviews. The interviews on the whole were very fruitful and have added great richness and depth to the information gathered on the history of the program since its inception. Other key stakeholders (ICLEI and USAID) also gave their time willingly and these interviews were also productive and added value to the process. Regarding ICLEI, Leluma Matooane was the CCP co-ordinator for the majority of the programme, but the lack of continuity of personnel in the later phases made it more difficult to gain a 1 The evaluation team examined the reports available (see section 3.1.1 above) to identify the extent of reporting against the above indicators and to attempt to verify the quality of data reported by ICLEI 6 ICLEI CCP – Final Evaluation 7 detailed overview of the program. More time was taken through the need to interview several ICLEI representatives in order to get all the qualitative information required and to understand fully how the program has operated in South Africa since 2001. By contrast, the evaluation team has been severely constrained by the lack of consistent and complete sets of quantitative information for the program. The team is confident in the completeness of the data for phase 3 of the program and less confident in data availability for phases 1& 2. The implications of having an incomplete set of data are outlined in more detail in section 4.3 below. 7 ICLEI CCP – Final Evaluation 8 4 Evaluation findings 4.1 Overall Program Performance 4.1.1 Performance against broad goals and aspirations The primary and broad purpose of this evaluation is to determine how the CCP program in South Africa has performed against its overall goals and to identify whether it has met the expectations of its primary funder, USAID. Broad goals and aspirations for the CCP Program in South Africa were articulated at the start of the program within the various program description documents prepared at the time. These are articulated in Box 1 below. Box 1: Overall Goals and Aspirations of the CCP program Overall Goals ™ Strengthen local commitment to reduce GHGs ™ Develop and disseminate tools that increase local capacity and enhance strategies for energy efficiency ™ Promote best practices to reduce energy use in buildings and transport ™ Enhance national and international ties through a collective voice for municipalities Overall CCP aspirations ™ CCP seeks to reduce the following emissions: • Carbon dioxide (CO2) emitted when fossil fuels are burned to produce electricity, power vehicles, heat buildings, in industrial uses • Methane (CH4) – a GHG emitted in urban areas by waste decomposing in landfills and waste sewerage treatment facilities • Conventional air pollutants such as nitrogen oxides, carbon monoxide and non￾methane volatile organic compounds that are precursors of ground-level ozone and smog as well as by-products of fossil fuel combustion. Comments on performance against goals • Strengthen local commitment to reduce GHGs All interviews have reinforced the fact that the CCP program has achieved this goal. It was wise to focus on the reduction of GHG because this can be achieved while implementing cost-saving initiatives and this presented a win-win scenario for municipalities. This has further provided an avenue to implement projects and achieve results and has stimulated an agenda around climate change, air quality and energy. Thus the demonstrable, tangible and measurable benefits and savings have stimulated greater awareness and interest in these areas and the visible impact of the initiative has demonstrated the benefit of local action to global problems. The escalation of municipal activity in the areas of climate change and energy efficiency since the inception of the CCP has been commendable and is mainly due to the targeted use of CCP funds as ‘seed’ funding which has ‘sparked’ other environment-focused activities within participating cities. Where this has been coupled with strong internal drive and local political support, great momentum has been 8 ICLEI CCP – Final Evaluation 9 achieved by some cities. This is most notable in the case of Potchefstroom which has become a world leader in local environmental activity on energy efficiency and GHG reduction. Many municipalities have gone on to produce broader sustainable energy strategies since the beginning of the program and the ICLEI - CCP program has helped put sustainable energy and climate change on the agenda. Its influence on the wider policy development in some cities has been significant, with many municipalities claiming that they would be doing far less work on climate change and energy efficiency if it were not for the opportunities provided by the CCP program. • Develop and disseminate tools that increase local capacity and enhance strategies for energy efficiency The 5 Milestone process has been a useful tool for the cities. It has guided activities in a systematic way and has provided strategic goals for achieving results. The software tools have had variable success. The initial inventory software was well received and applied by all the participating cities. However, the need for such software was thought to be less within the more capacitated metros where inventory activity was already initiated. Several metros have gone on to invest in very sophisticated air quality monitoring and data management systems which will reduce the usage of the CCP software. It should be noted that that these more detailed, data intensive software serve different purposes to ICLEI’s quantification software, and the CCP software is often use to compliment rather than substitute other software packages. The later development of the HEAT program has provided the municipalities with more difficulties, primarily due to the ongoing technical glitches within it. Several municipalities urgently want to update their initial inventories but have been instructed to wait until HEAT is finalised. ICLEI must ensure that this is done quickly and that despite the end of the program, cities have access to technical support on the application of this program when required. • Promote best practices to reduce energy use in buildings and transport This has been achieved and ICLEI has achieved its goals in rolling out proven technology. Activities such as the regulation of air-conditioning and retrofitting of lights have been particularly effective mostly because of their immediate and demonstrable impacts on energy efficiency. In terms of the LPG conversions, the effects and measurability seem less obvious and there are concerns around the small scale at which this has taken place, the availability of LPG in future and the lifespan of the vehicles. • Enhance national and international ties through a collective voice for municipalities The CCP teams in cities see themselves as part of the same ‘family’ and this networking relationship across the local sphere of government has proven highly successful. Intergovernmental ties and relationships between spheres of government were not well developed, mainly due to the lack of co-ordination from national government, in particular DEAT. The international exchanges and sharing at conferences has been inspirational and motivational for the SA cities and has given insight into what could be achieved. It has identified that the responsibilities for particular projects should be linked with the relevant operational department and not all be boxed into the environmental department, e.g. transport initiatives should be located in the transport department. Internal learning and exposure has been reciprocal, which has been every encouraging for cities. These conferences have also provided platforms for taking leadership on these issues, e.g. The Mayor of Potchefstroom represented local 9 ICLEI CCP – Final Evaluation 10 government at a major gathering of the global climate change community at the COP8. Comments on Performance against overall CCP aspirations • CCP seeks to reduce the following emissions: o Carbon dioxide (CO2) emitted when fossil fuels are burned to produce electricity, power vehicles, heat buildings, in industrial uses o Methane (CH4) – a GHG emitted in urban areas by waste decomposing in landfills and waste sewerage treatment facilities o Conventional air pollutants such as nitrogen oxides, carbon monoxide and non￾methane volatile organic compounds that are precursors of ground-level ozone and smog as well as by-products of fossil fuel combustion. At this stage only projected impacts can be commented on and the analysis has not been done by ICLEI to verify the overall impact of the CCP on emission reductions. The effectiveness of the reduction of emissions can be proven in some cities where they are sure of the reliability of the inventory data. However, in cities where they are not confident in the quality of their data, this will be hard to do without questioning the reliability hereof. 4.1.2 Performance against Required Indicators As part of a larger campaign (the Cities in Climate Protection Campaign run by ICLEI), this program receives much of its funding from USAID. Thus, it has an obligation to report on progress against specific indicators used by the various funding streams within USAID which contribute to the CCP program. These funding streams and the indicators they use to assess performance are outlined in Table 3 below. Table 3. Indicators for USAID funding streams Funding channel Indicator USAID Strategic Objective 6. Increased access to shelter and environmentally sound municipal services to historically disadvantaged populations ™ Rand value of new or improved urban services leveraged through USAID supported programs ™ Number of households assisted to obtain new or improved urban services through USAID supported programs USAID Strategic Objective 6.4 Improved capacity to apply sustainable participatory environmental management principles to local level urban development ™ Emissions of carbon dioxide equivalents avoided ™ Decreased carbon dioxide permitted per MW-h of energy produced ™ National/ sub national policy advances in the energy sector, industry and urban areas that contribute to the avoidance of greenhouse gas emissions ™ Increased capacity to address global climate change issues Clean Energy Initiative HHC program Twice yearly reporting ™ FY Funds obligated ($000) in Q1 and Q2 (or Q3 and Q4) for HHC activities ™ FY Funds Obligated ($000) for FY for HHC activities ™ Funding leveraged in Q1 and Q2 (or Q3 and Q4) ($000) for HHC activities ™ HHC/Clean fuels: Number of programs implemented to replace dirty fuels with cleaner fuels (half yearly) 10 ICLEI CCP – Final Evaluation 11 Funding channel Indicator ™ HHC/ Clean Fuels: Number of people benefiting from replacing ‘dirty fuels’ with cleaner fuels (half yearly) ™ HHC/Indoor Air: Homes adopting improved cooking/heating/ lighting techniques (not applicable) ™ HHC/Indoor Air: Number of people with reduced exposure to combustion pollutants indoor (not applicable) Global Climate Change ™ Project Type ™ Energy (MWh) saved ™ BTUs saved in thermal combustion ™ Fuel type saved or old fuel/ new fuel ™ Methane Capture (Y/N) ™ Nox Reduction (Y/N) ™ Particulate Reduction (Y/N) It should be noted that the presence of indicators such as these is positive in itself as they provide expanded guidance on areas of impact which the program should be achieving and some insight as to how this impact can be measured. They also create a discipline of reporting on quantitative data that can be used to assess performance if benchmarks for these indicators have been set at the start of the program. They are also useful for aggregating total global performance of these USAID funding programs and for comparative analysis of the contribution to global figures made by particular country programs. The evaluation team is not aware of any initial benchmarks for these indicators set by USAID or of the results for these indicators for other countries. Thus, the team cannot comment on the figures reported against these indicators within the scope of this evaluation. However, it may be useful for USAID to analyse the cumulative figures for some of these indicators reported by ICLEI for the timeframe of the program. Figures available are shown in Table 4 below. Table 4. Values for some key SO6, GCC and CEI indicators Indicator Value (total for project) SO6. Rand value of new or improved urban services leveraged through USAID supported programs R3.8m SO6. Number of households assisted to obtain new or improved urban services through USAID supported programs 1.8m people2 GCC: eCO2 tonnes avoided Phase 1: no data Phase 2: 7121 (projected) Phase 3: 73 (projected) GCC: Energy savings (mWh) Insufficient data to give program totals. Some information on project level savings CEI Funds obligated (by ICLEI) Phase 1: insufficient data 2 Note: there are some concerns around the utility, validity and precision of this figure. These are discussed in more detail in section 4.3 below. 11 ICLEI CCP – Final Evaluation 12 Indicator Value (total for project) Phase 2: $194000 (leveraging additional $100 000) Phase 3: $313700 (leveraging additional $179489 4.1.3 Performance against 5 milestones Each phase of the CCP program was structured using a set of 5 milestones used across the ICLEI global CCP campaign. These milestones are shown in Box 2 below. Box 2. Milestones of the CCP approach Milestone 1: Prepare an energy and emissions profile or inventory Milestone 2: Set an emissions reduction target Milestone 3: Identify emission reduction measures and develop local emissions reduction action plan (LAP) Milestone 4: Implement LAP measures to reduce GHG’s Milestone 5: Monitor and verify progress on implementation of measures and review where necessary The participating cities found the milestone approach a useful one, as it lent a logical structure and flow to their involvement, enabling them to establish a baseline of information on their GHG emissions and knowledge of their needs and practical capabilities upon which projects could then be established. The milestone process was applied well in phase 1, although it is noticeable that very few municipalities have reached the stage with their phase 1 projects of carrying out the monitoring and verification of projects which would provide an accurate assessment of the actual impacts of projects (as opposed to projected impacts calculated prior to implementation). Municipalities should be encouraged by ICLEI to strive to carry out the 5th milestone in the future despite the completion of the overall CCP program, to ensure that progress can be monitored and an ongoing picture of the impact of the projects on local emissions can be gradually verified. 4.2 Program performance by phase 4.2.1 Phase 1 The activities outlined for phase 1 prior to the start of the program and the list of expected deliverables is shown in Table 5 below. Table 5. Planned activities for Phase 1 of the CCP Program Activity 1. Project planning, participant and partner selection. Including: • Identifying interested municipal participants and appropriate technical partners (preference for South African firms) • Initiation of RFP process to select participant municipalities • Recruitment, hiring and initial training of field staff (preference for South African nationals) • Adaptation of CCP Greenhouse Gas Emissions Inventory software for use in South Africa • Establishment with DEAT and USAID of detailed work plan for project implementation 12 ICLEI CCP – Final Evaluation 13 By the end of the phase the in-country staff will have been hired and trained, local government participants secured and initial meetings to refine the projects scope will have been conducted. The GHG emission software will have been adapted to country conditions and user needs 2. Training and Capacity Building 6 workshops to be held to bring the participating local authorities together for the purpose of training, building capacity, identifying measures and implementation strategies, and facilitating appropriate partnerships. The workshops will assist the local governments and stakeholders in developing a comprehensive local action plan to reduce GHG emissions with actions that address priority municipal concerns and with implementation of initial measures 3. Grants to Cities To provide an incentive for the local governments to accomplish project tasks and other resources – mini grants in the range of R400 000 to 700 000 to be provided to each municipal participant. 4. Municipal Operations: Energy end use analysis and measure prioritization and identification of financing opportunities ICLEI and technical consultants will assist participating cities in quantifying energy end use in their municipal operations (i.e. buildings, vehicle fleets, facilities, water/sewerage treatment) and identifying optimal internal measures for emissions reduction. 5. City Peer Exchanges Capacity building in the participating cities can be further supported through facilitated exchanges with selected local governments in other regions. Specific Deliverables were also defined for this phase of the program as follows: Table 6. Deliverables for Phase 1 of the ICLEI CCP Program Deliverables ™ 6 training and capacity building workshops for the participating municipalities ™ Customization of the greenhouse gas emissions software for use by the South African local governments ™ Awarding and administration of grants to each participating local government ™ Technical assistance from the ICLEI staff and consultants provided to the staff teams established in the participating local governments ™ A baseline emissions inventory and forecast of GHG emissions completed for each municipality ™ City staff peer exchange between out of region CCP cities and the South African local government participants ™ GHG reduction measures identified by all local government participants ™ Implementation of measures by a minimum of three of the participating municipalities that result in quantifiable greenhouse gas emissions avoidance/reductions ™ Quantification and reporting of greenhouse has emission reductions achieved ™ Quarterly progress reports ™ Final report 13 ICLEI CCP – Final Evaluation 14 Comments on achievements of Phase 1 Phase 1 of the program was hailed by all of the municipalities as very successful. It provided a strong platform for the other 2 phases and was flexible, allowing municipalities to develop projects which had strategic value or provided genuine benefit or cost saving. The timeframes were realistic and were achievable for the municipalities. The strong emphasis on training, capacity building and networking created a momentum within the program and provided the municipal officials involved with a ‘sense of community’ which has been sustained through subsequent phases. The foundation of project work on a knowledge of emissions (i.e. the inventory) and plan of action was well structured and provided a logical ‘flow’ to the phase. It is not the purpose of this evaluation to re-state the information within the ICLEI final report on progress achieved within each phase of the program. However, short summaries will be provided for progress made on each activity within this phase so that more emphasis can be placed on commenting on impacts and implications of the work achieved. 1. Project Planning, participant and partner selection Eight South African municipalities were selected for participation in the Program, which was formalised through the signing of a formal Memoranda of Understanding with each. The municipalities included were: • Buffalo City • Cape Town • eThekwini • Johannesburg • Potchefstroom • Saldanha Bay • Sol Plaatje • Tshwane During this phase ICLEI identified staff within its South African set-up to run the program, liaise with the participants and organise events. Software was customised for use in later activities of the phase. 2. Training and Capacity Building Five training workshops were held throughout the phase involving all eight participants. These covered topics which ranged from broad induction and information sharing – to technical training in use of software and the preparation of an emissions inventory. Overall these workshops seem to have been very valuable to the participants on many levels. They helped to achieve the establishment of a network of local government officials and councillors involved in the program as well as act as an effective method of transferring information and building capacity. They were felt to be well organised and focused and all participants were positive about their value. In addition, ICLEI also involved itself in the preparation of a Cities Energy Strategy Conference (the 6th workshop!) and included many international speakers in this event. The value of the training events run by ICLEI during this phase is reflected in the attendance figures – with every municipality ensuring that it had some representation at each event. Other training and development activities included one-to-one assistance of municipalities with software and inventory preparation – which was either done by ICLEI staff or by consultants employed jointly with the municipalities. In general, 14 ICLEI CCP – Final Evaluation 15 municipalities were happy with the access they had to technical assistance. However, the need for technical expertise within cities was demonstrated during the compilation of the inventories and made many municipalities realise the necessity of having dedicated technical experts on their own staff for future work in this area. Comments by ICLEI in an exit report for Phases 1 and 2 stated that there was still a need at the end of Phase 1 for more awareness-raising to be carried out within municipalities to try to integrate energy and environmental auditing into IDP processes. There was also a need for further awareness raising campaigns to highlight the benefits (financial and carbon savings) of energy efficiency and possible business opportunities these provide. 3. Grants to Cities Grants were provided to the cities in four tranches, each of which was tied to specific requirements to be fulfilled by the city: Payment 1: provided upon signature of MoU (25%) Payment 2: on completion of emissions inventory, submission of city work plan and detailed grant spending (showing 25% matching or in-kind contribution by the city) (25%) Payment 3: on submission of ‘list of measures’ with approved local action plan of one of the listed measures, and proof of measure implementation (25%) Payment 4: on submission of proof of implementation of second measure It is testament to the enthusiastic uptake of this phase of the program that 7 of the 8 cities achieved all 4 payments. The one exception was Buffalo City which received only the initial payment following the signature of a MoU. No inventory was prepared and the funding seems to have been used for a project to retrofit a municipal building. All the others prepared an inventory and went onto identify GHG reduction measures for which projects were designed and implemented. A list of the projects implemented within each municipality is shown in Table 7 (below) although a lack of availability of information for phase 1 means that the data set is not complete or consistent for indicators across all municipalities. 4. Municipal Operations This element of the program involved assisting the municipalities to prepare GHG inventories for their municipal operations. Although the original expectation of the program was for the preparation of an inventory for both municipal operations and wider community, the latter was not compiled by any of the municipalities. However, all 8 municipalities did prepare an inventory of municipal operations. ICLEI, in its final report for the program, included summaries of several of these inventories. However, no mention was made of the ‘level of confidence’ municipalities had in the inventories finally prepared through the CCP process. Some municipalities (such as Potchefstroom) have complete confidence in the accuracy and reliability of the inventory they carried out. Others, such as Saldanha Bay and Johannesburg have little confidence in the data they compiled. The preparation of the GHG Inventories generally seemed to be difficult and restricted by several factors: poor record keeping in many cities; energy consumption calculated from financial records; data incompatible with the CCP software and an inability to confirm data included in the inventories. For all these reasons ICLEI should be cautious in the publication of inventory summaries and include, where required, notes on the limitations of the data presented. 15 ICLEI CCP – Final Evaluation 16 Table 7 Available data for projects in Phase 1 Cape Town Municipality Deliverable Grant size (total for City) $ Municipal Contribution Other funding leveraged Total Cost eCo2 avoided (annual) Annual Energy Savings kWh Annual Energy Cost savings Potchefstroom Methane Recovery 52000 ? ? R2.2m 7000 R 361,350 Lighting Retrofits ? R1.2m 290 33700 eThekwini Methane Recovery 56000 ? ? ? 472000 ? Solar water heating units ? Cape Town Parrow Energy Efficiency Project 56000 R213 000 ? ? ? ? Waste Minimisation Clubs ? ? ? ? Lighting Retrofits ? ? 175200 Joburg Retrofitting main admin block Civic Centre 56000 ? >R2m R4.1m 4646 4087500 R456 000 Streetlight Retrofits: Rivelea & Newtown ?? ? ? Pikitup CDM ? ? ? ? Water Supplyy Energy Efficiency ?? ? ? Sol Plaatje Energy Efficient Lighting 52000 R130 000 ? ? ? ? Moshoeshoe Eco Village ? Saldanha Municipal Transport Sector (LPG) 52000 R150 000 ? R 80,000 21 Other (trees) ? ? ? ? ? Tshwane Ga-Rankuwa Streetlight Retrofit 56000 R 253,200 ? R300 000 1187 ? ? Urban Greening 0 ? Buffalo City 52000 R 143,500 Totals $432,000.00 Incomplete 5. City Peer Exchanges Table 8 below identifies the number of municipalities that sent representatives to internal workshops, ICLEI conferences and international peer exchange activities. The international peer exchange part of the program was cited by many municipalities as very valuable. It was hailed as an effective method for increasing knowledge and capacity on climate change and energy efficiency within municipalities. It also had a role in inspiring and motivating many municipal officials and Councillors to take action when returning home. Providing local governments with an opportunity to communicate with international colleagues was exciting and informative and many longer term relationships between municipalities in South Africa and local governments overseas have been forged. For example, City of Potchefstroom has entered into a ‘twinning’ arrangement with Heidelberg. 16 ICLEI CCP – Final Evaluation 17 Table 8: Summary of attendance by municipalities at training and conference events Indicators Phase 1 Phase 3 Phase 1 Phase 2 Phase3 Phase 1 Phase 2* Phase 3* Tshwane √√ HEAT √ CES Mex √ Potchefstroom √√ HEAT √ CES and COP 8 Mex √ (Heidelberg & COP8) Cape Town √√ HEAT √ CES Mex √(Heidelberg) √ Joburg √√ HEAT √ CES and COP 8 Mex √ COP8 Saldanha Bay √√ HEAT CES √ (Heidelberg & SAM conference) Buffalo City √√ CES √ Sol Plaatje √√ HEAT √ CES Mex √(Heidelberg) eThekwini √√ HEAT √ CES Mex √ Mangaung Polokwane Ekurhuleni Richard's Bay √ HEAT √ Mex √ √ √ HEAT √ √ HEAT CES=City Energy Strategy COP8=Confe rence of Parties8 held in India Mex=ICLEI international conference held in Mexico Number of officials/councillors participating in formal peer exchange activities Phase 2 Workshop attandance and Training received Conference attandance ICLEI also facilitated opportunities for South African municipal officials and councillors to participate actively in international climate related events, such as COP 8 and the World Summit on Sustainable Development (WSSD). This ‘exposure’ helped to raise awareness of activity at the local level in South Africa and provided the participating municipalities with opportunities to make important international links and relationships. 4.2.2 Phase 2 Phase 2 of the CCP program started in 2004. The focus of phase 2 was to expand the program by three additional cities and to assist all cities to implement a range of larger and repetitive projects and initiatives through partnerships with the cities. This was felt to be a shift in approach from phase 1 which focused more on smaller projects which were used to demonstrate the potential of energy efficiency projects. The main objective was to focus on the implementation of clean energy and transportation measures by: • Providing more targeted technical assistance to the cities on these tasks, such as in the development of PINS, assessing audit services and more • Actively leveraging the opportunities/resources identified in the project’s first phase This would be done through the following activities: • Continued training and capacity building • Facilitating preparation of emissions inventories for the ‘new’ cities • Provision of grants for projects. Expansion of Program by three cities This phase aimed to add three new cities to the list of participants in South Africa. A process for identification and selection of cities was carried out and resulted in Mangaung, Ekurhuleni and Polokwane being earmarked to join the program. Of the three, Mangaung and Ekurhuleni have become successful additions to the program. Polokwane has delayed it entry to the program until it has succeeded in tackling some internal issues. 17 ICLEI CCP – Final Evaluation 18 Training and capacity building Three main activities were carried out during this phase: • Induction training for new cities. This was carried out via a 2 day workshop in late 2004 which focused on introducing cities to the program, its methodology and deliverables. • Support for cities to attend international ICLEI conference in Mexico (July 2004). The program enabled 17 people from South Africa representing 8 of the cities and the ICLEI South Africa office to attend this international conference. All cities who sent representatives commented on the usefulness of this event and the positive impact it had on their motivation, knowledge and network. The opportunity for both officials and Councillors from the same municipalities to attend was cited by many as very advantageous as it exposed councillors to the activities and attitudes of other cities throughout the world and enabled officials to cultivate support for their plans from Councillors upon return to South Africa. • Technical training in the use of the HEAT software. The purpose of this event was to introduce cities to the new software and to explain how it was being customised to suit South African conditions. This event was not as well attended – with only 11 people attending from over 13 cities, DEAT and the Energy Research Centre (ERC). Data Collection and Emissions Inventory Two of the new cities (Mangaung and Ekurhuleni) started work to prepare their emissions inventory during this phase. However, it is not clear whether these were completed within this phase. The ICLEI final report does claim that both completed inventories, but when the municipalities were asked, they both stated that inventories had not been completed although audits of some municipal buildings had been prepared. The issue here may be a misunderstanding in terminology (i.e. between the term inventory and audit) or error in reporting by ICLEI or knowledge of what has been achieved on the part of the municipal officials interviewed. Project Grants Allocated City grants were structured differently in phase 2 with cities having to apply for total grants ranging from 20,000 – 40,000 US$. The cities specified the amount of funding required and the final amount approved was dependent on the proposal the city submitted. Of the 6 city proposals submitted, 6 were successfully approved. The grant structure was as follows: Payment #1: 50% of total city grant upon signing a contract ICLEI. Payment #2: 50% of total grant upon submitting proof of completed implementation of the approved project (where ‘proof’ required either the submission of a formal progress report by the municipality or a tour of the project by an ICLEI representative). Not all municipalities applied for funding. Those who did not, claimed the funding timeframes were the primary reason for not applying. Some of the key indicators for the projects which were carried out are shown in Table 9 below. 18 ICLEI CCP – Final Evaluation 19 Table 9. Available data for projects carried out in Phase 2 Phase 2 Municipality Deliverable Status Grant size $US Municipal Contribution Other funding leveraged Total Cost eCo2 (tonnes) avoided (annual) Decreased Co2 emitted per MW-h energy produced Annual Energy Savings kWh Annual energy cost savings US$ Potchefstroom ? eThekwini Energy efficiency in municipal buildings In progress. Close to completion 36000 26000 ? 241 Cape Town 60 Solar water heaters in middle and low income households preparatory work done. No implementation yet 35000 20000 ? 77 87000 6147 Kuyasa solar water heating and energy retrofit project ? 6200 ? ? Joburg ?? Sol Plaatje Streetlight retrofit completed 19000 28000 R 120,553 53 42012 R 23,659 Saldanha Solar Thermal 28000 11000 ? 154 131400 ? Tshwane Energy Efficiency in municipal buildings completed 36000 33000 ? 133 220910 ? Balebogele Primary School Energy Retrofit Completed ? 8 ? ? Buffalo City Retrofitting of lights in municipal buildings completed ? ? ? Ekurhulei Energy Efficiency in Municipal Buildings Completed 40000 59000 R 249,120 308 328988 50664 Polokwane Mangaung uMhlathuze Totals 194000 177000 7174 ? Phase 2 projects were in general successfully implemented and the projects did seem to be a logical ‘follow-on’ from those carried out in phase 1 – being largely similar types of projects although larger in scale. 4.2.3 Phase 3 Phase 3 of the ICLEI CCP program started in 2005. The objectives of phase 3 centred on increasing awareness around air quality and projects were guided to focus on air quality monitoring and transport. The table below details the main objectives of this phase. Table 10. Objectives for Phase 3 of the CCP Program PHASE 3 OBJECTIVES ™ Increase awareness and disseminating best practices on how to reduce GHG and priority air pollutants emissions in response to the requirements of the AQMA through training and awareness workshops ™ Solicit requests for proposals and coordinate the award of grants to implement measures that would/ will reduce GHG priority air pollutant emissions in the transportation sector ™ Provide specialized technical assistance to eight municipalities interested in reducing emissions from the transportation sector by establishing inventories of fuel consumption from the transportation sector within their jurisdiction, assessing the emissions from these sources and evaluating measures to reduce these emissions using the HEAT model ™ Collaborate with USEPA in the provision of technical assistance to one or two municipalities to set up air quality monitoring systems. 19 ICLEI CCP – Final Evaluation 20 Aligned with these objectives is a set of activities for Phase 3, detailed in the table below. Table 11. Planned activities for Phase 3 of the CCP Program PHASE 3 ACTIVITIES Activity 1: Projects Selection with a focus on air quality and transport This to include: ™ Invite proposals for funding on air quality and transport projects for tier one and tier two funding projects and provide grants for approved projects. ™ Provide technical support for collecting new inventory data and compiling transportation GHG and priority air pollutants emissions inventories. ™ Local Action plans for t he transport sector to be developed Activity 2: Training and Capacity Building ICLEI to ™ Host a workshop a training workshop for municipalities focussed on the implications of the Air Quality Monitoring Act Activity 3: Grants to Cities ™ City proposals to be evaluated and funding were given as either Tier 1 or Tier 2 level grants to proposals accepted. Activity 4: City Peer Exchanges ™ Implement a mentoring component was included this phase, which involved the partnering of a suitably qualified Metro with a less experienced local municipality working in the area of air quality. Activity 5: At least 6 GHG emissions reduction measures begun implementation ™ Implementation of measures to begin and emissions reductions to be measured Comments on Phase 3 Performance against objectives Officials from the CCP cities attended a workshop held by USAID, ICLEI, DEAT and USEPA to increase awareness on the impact of the Air Quality Monitoring Act. Phase 3 proposals were very specifically focused on air quality and transport. While all the participating municipalities were invited to apply for phase 3 finding, only 7 of the cities submitted applications, namely: - Cape Town - Mangaung - Tshwane - Ekurhuleni - Potchefstroom - Sol Plaatje - Johannesburg The other 3 cities (Saldanha Bay, eThekwini and Richards Bay) elected not to do so. The main reasons identified include the tight time frame given for submitting proposals and the lack of internal capacity to commit to completing a funding proposal. Tshwane and Johannesburg’s proposals were not accepted due to concerns over the validity of the different fuel additives proposed by each. The other municipalities who applied were all successful in obtaining grants. 20 ICLEI CCP – Final Evaluation 21 Municipal officials were trained on the HEAT software and 8 of the cities produced an inventory for the municipal transport fleet using this software. ICLEI provided valuable technical support, through Catherine Fedorsky who assisted municipalities with their emissions inventories and provided some HEAT support. However, usage of HEAT software has been quite limited. Several municipalities mentioned that the lack of local support presented an obstacle to progress because the main HEAT support person (Margarita Parra) was located in the ICLEI Office in California. Many municipalities lack the internal technical skill to use HEAT and the absence of local support has presented a particular challenge. Only a limited number of HEAT manuals were distributed to municipalities and many felt that one workshop was not sufficient to prepare or equip them for the use of the HEAT software effectively. Some municipalities also faced technological challenges which have restricted usage of the online HEAT software, including poor quality of IT equipment and slow, unreliable internet access. While many municipalities have expressed a keen desire to make use of the software for future inventory compilations, they are uncertain as to how they will go about doing this in light of technological obstacles and the lack of access to support and training. The provision of air quality monitoring equipment and technical assistance to both Cape Town and Mangaung has been a successful one. The mentorship between these two cities has given Mangaung the opportunity to learn from Cape Town’s skill and experience in this area and it has allowed the municipality to continue and expand in this area of work. A good working relationship between these two cities has been established and is likely to be maintained. Performance against Planned Activities Activity 1. Projects selection with a focus on air quality and transport Projects for Phase 3 were focused specifically on air quality and transport. ICLEI provided technical support to help municipalities identify projects which would be suitably placed in either air quality or transport sectors and would bring about measurable GHG reductions. This phase also focused on collecting new and updating previous inventory data, with ICLEI providing some technical guidance in this regard. Some municipalities also took the opportunity to check and update their earlier inventories. In some instances errors were picked up and the cities have stressed that they will be correcting errors. Activity 2. Training and capacity building One training and capacity building workshop was held in phase 3, attended by USAID, DEAT, USEPA, ICLEI and staff from the CCP cities. New municipalities such as Mangaung benefited from this training in particular, as it offered them the opportunity engage with other stakeholders on issues related to air quality. Activity 3. Grants to cities Cities struggled to meet the tight deadlines for phase 3 funding proposals, which lead to this being extended by about a month. Only 5 city proposals were approved and only Cape Town and Mangaung received tier one grant funds. Many cities felt that the proposal deadlines as well as the time frames given for project implementation were unreasonable. Municipal procurement procedures caused delays 21 ICLEI CCP – Final Evaluation 22 and slowed down project implementation for many municipalities. It should be noted that the unrealistic time frames stem from USAID, rather than ICLEI. Thus, restrictive nature of phase 3 has been attributed to finite deadlines imposed by USAID, as the programme was coming to an end. ICLEI were aware of the unreasonable pressure that would be placed on cities, but ICLEI elected to work within the time constraints which were placed on the cities rather than decline funding which would help the cities to further their work on air quality and energy efficiency in transport. In hindsight, projects which involved procurement of services would not have pursued in phase 3. Government procurement processes are notoriously long, which inevitably slowed down the progress that was made on projects. Table 10 below shows the value of grants awarded and project carried out in phase 3. Table 10. Available data for projects carried out in Phase 3 There is also a perception that the rigidity of the phase 3 process limited the capacity of the municipalities to be creative and innovative in the types of projects which they proposed. Some cities mentioned that they would have appreciated more guidance in preparing the proposal. Activity 4. City peer exchanges Various options were evaluated but after careful consideration, City of Cape Town and Mangaung were elected as the mentor and the mentoring recipient respectively. Both cities received air quality monitoring equipment and training from ICLEI and USEPA. Both cities have expressed the value that this mentorship and equipment has brought to the municipality, indicating that this has helped raise awareness on the importance of air quality monitoring and assisted with improving their ability to monitor air quality. Mangaung in particular benefited from the opportunity to expand into an area which had previously been underdeveloped and used the opportunity to learn from Cape Town. The municipality has also used its equipment as an educational tool for school children by locating the equipment in the Bloemfontein Zoo and including an explanation of its use and functionality as part of the guided tours at the zoo. Although this partnering represents the only formal peer exchange for Phase 3, cities have generally expressed that the strength of the informal networks that have been established has been one of the big achievements of the CCP program. 22 ICLEI CCP – Final Evaluation 23 Activity 5. At least 6 GHG emissions reduction measures begun implementation Due to the time frame for phase 3 many municipalities struggled to implement their projects efficiently and effectively and some are still in the process of getting this underway, such as Sol Plaatje where the grant necessary to begin the LPG vehicle conversion has not yet been fully accessed. In other cities such as Mangaung, most of their budget has been spent but some responsibilities from phase 3 still need to be carried out, such as one remaining field visit with Cape Town. As a notation on the final ICLEI report, the eThekwini project which is part of the city’s ITP was included on the GHG reduction projects that have begun implementation even though it was not funded by ICLEI/USAID/DEAT. 4.3 Data quality and verification 4.3.1 Data collection The evaluation team experienced a certain degree of difficulty analysing quantitative data embedded in reports as it proved to be impossible to access a full set of these reports. The problem is most pronounced for phase 1 where very few reports were made available to the evaluation team. While there are also significant data gaps for phase 2, most reports for phase 3 were secured. Given the gaps in terms of reports received, it was not possible for the evaluation team to definitively comment on the quality of data collection under the program in all instances. However, the evaluation did not find anything to suggest that data collection and reporting under the project was not sound. Rather, the frustration facing the evaluators was the lack of good record keeping and access to historical data for this kind of after-the-fact analysis. 4.3.2 Reporting Municipalities were generally happy with the reporting requirements placed on them by ICLEI. Many felt that the relatively relaxed approach to reporting was an advantage as it did not place an undue burden on municipalities to prepare reports. However, there was the view that the reporting requirements could have been more structured without being too onerous. The rather unstructured reporting, along with the fact that reporting was not routine, but rather on the basis of project specificities, meant that the quantitative data embedded in the reports was not that easy to compare or to aggregate. Some municipalities felt that, had they been forced to report on a more regular basis and in a little more detail, pressure would have been exerted on them to keep their projects, including the inventory, up to date. 4.3.3 Aggregation Given the fact that the evaluation team was unable to secure a full set of municipal reports, it has not been possible to test whether all the data reported by ICLEI is an accurate aggregation of municipal reports. The absence of structured record keeping through the project on the part of ICLEI and USAID is an issue that needs to be addressed in this regard. Nevertheless, the evaluation team did not find anything to suggest that the aggregate data reported by ICLEI is not accurate. 4.3.4 Verification Where data and reports have been available for the evaluation team, attempts have been made to verify data reported in the Global Climate and Clean Energy reports. The verification of final number of households benefiting from the program is, however, a difficult issue. On the one hand, the reports suggest that all households within the 23 ICLEI CCP – Final Evaluation 24 participating municipalities have benefited from the program. At a conceptual level, this may be the case, but this does not really communicate an accurate picture of beneficiaries. The rationale for considering ‘total municipal population’ as the number of beneficiaries is related to the fact that some CCP funds have resulted in broad climate change policy development within municipalities. Whilst this has been the case, the extent of impact of the funds on policy making activity has varied from municipality to municipality. Large municipalities like Cape Town were adamant that the CCP funds could not be said to have a significant enough impact on policy activity to justify considering the total municipality population as beneficiaries. Smaller municipalities where CCP funding has had a direct and significant effect on placing climate change and energy efficiency on the municipal agenda could be more confident about making this assumption. Thus ICLEI should be a little cautious about the blanket use of total municipal population numbers as the estimate for beneficiaries of the program. At the other end of the spectrum, some municipalities indicated the specific individual beneficiaries from their projects (e.g. Balebogeng school project in Tswane, and the Galeshewe Eco Village and tree planting initiatives in Sol Plaatje). In these cases it is justifiable to use the community population directly benefiting from the project. 4.4 Institutional issues A range of institutional issues emerged and were explored during the course of the evaluation. Some of these issues were related to overall program organisation and administration, while others were more specifically related to actual implementation issues at the municipal level. 4.4.1 Program organisation and administration At the program level, a number of key issues emerged related to the organisation and administration of the program. These are outlined in some detail below. Roles, responsibilities and accountability During the interview phase, it emerged that there was some ambiguity around roles and responsibilities of the different stakeholders during various phases of the project. Some municipalities were unclear of the role of DEAT in the program and were under the impression that the Department’s role was fluid and viscous during the project. Generally, the roles and responsibilities of the ICLEI staff were well understood. However, staff change over in the ICLEI South Africa office was not always effectively communicated and some officials were uncertain as to the new roles and responsibilities played the consultants contracted for phases 2 and 3. Some municipalities mentioned that they were uncertain as to which consultant should be contacted for technical assistance and who was responsible for the project management role. More effective communication about these roles and responsibilities would have assisted in providing greater clarity for the municipalities. Institutional instability Some interviewees reported frustration around the fact that there sometimes seemed to be lack of continuity in the program. This was related to perceived institutional instability on the part of ICLEI from both a funding and staffing perspective. Funding Uncertainty around the ongoing funding and financial viability of ICLEI South Africa was a source of concern for Johannesburg who felt that the lack of adequate core funding for ICLEI put an unnecessary strain on the organisation and contributed to staff turnover which in turn impacted negatively on the ability to facilitate and support project 24 ICLEI CCP – Final Evaluation 25 work. While Leluma Mataoone co-ordinated much of the programme, his exit left a marked gap in the ICLEI SA office, which was not adequately filled and impacted negatively on continuity in the later stages of the program. Staff turnover Related to the funding issue highlighted above, funding uncertainty has contributed to high levels of staff turnover in ICLEI (South Africa) which impacted on continuity and the relationship between municipalities and ICLEI. Throughout the lifespan of CCP in South Africa, activities were overseen by the California office. This is mainly because the ICLEI Africa directorate is not a well established branch of ICLEI has not been able to operate autonomously. This situation is not ideal as it has created a fragmented network of stakeholders and the geographic distance between the two offices is likely to have made communication difficult. During the CCP program, the ICLEI leadership responsible for South Africa has changed hands in both the California and Johannesburg offices. This has negatively affected the flow of the program and it is uncertain as to how well the handover was managed, although ICLEI did receive an exit report from Leluma Matooane and held interviews with him prior to his departure. Staff turnover has had the greatest impact on institutional memory, which is somewhat lacking on both ICLEI offices and in some cases even municipalities identified the negative implications of the loss of institutional knowledge with the loss of human capital. Frequent staff turnover means that working relationships have to be redeveloped, often resulting in a lack of continuity. However, it appears that part of this staff turnover may have been in reaction to the end of secured funding via USAID/DEAT for the CCP Program. It should be noted that while Leluma’s exit may have left a gap in the ICLEI South Africa office, he went on to continue working in the climate change arena and the skills which he developed while working for ICLEI equipped him to broaden his horizons while still making a valuable contribution to climate change in South Africa, which he continues to do. Thus, ICLEI has made an important contribution to building capacity in the field of climate change and provided a platform for the development and transfer of skill within South Africa. Contact with municipalities Communication Some municipalities expressed a level of frustration in terms of the communication between themselves and the key stakeholders at times. This was largely due to the staff turnover issues highlighted above and the fact that DEAT’s role in the campaign was not entirely clear. Support Notwithstanding the issues highlighted above, most municipalities were relatively pleased with the support that they received from ICLEI during the course of the campaign particularly in terms of technical support, assistance with project selection and reporting. Support from ICLEI at the early stages was generally praised by municipalities. While technical support was received from consultants in phases 2 and 3, this was fairly limited because of the contractual nature of the support. Municipalities also created their own network of support partly due to good personal relationships that were established but also due the lack of formal support structures between them. It is encouraging to note that informal peer exchange has and in fact continues to take place but the sustainability of this may be at risk if it is too dependent on certain individuals and their placement in the municipality. Thus, mechanisms need 25 ICLEI CCP – Final Evaluation 26 to be put in place to sustain networks as well as organise and focus these better to ensure that when a new generation of officials enter, network arrangements continue. 4.4.2 Municipal implementation arrangements Institutional issues at the municipal level were also identified during the course of the evaluation. These are outlined below in some detail. Location of the program within municipalities The campaign was located in many different institutional locations, varying from municipality to municipality. This has important implications for how projects have been conducted, and how they are likely to play themselves out in the future from a sustainability point of view. For example, in Buffalo City the program is located strategically in the IDP office, having the advantage of being located in a strategic function relatively high up in the municipal departmental hierarchy. At the other end of the spectrum, in Saldanha Bay, the program was located in the Environmental Health function. When this function was shifted to the district municipality, the program was severely affected – this is discussed more fully below. In some instances air quality projects have been located within the health function while energy focused projects have been vested in environmental departments. In Tswane, partly as a result of the program, there are moves afoot to establish a dedicated department to cover sustainable energy projects, air quality projects and climate change projects, greatly contributing to the sustainability prospects of these functions. Municipal contributions and co-funding In many instances, CCP funding has only been part of the funding required to carry out projects stated in the ICLEI reports, although this is not always made clear. As discussed in section 4.3 of this report, the exact mix of contributions between ICLEI and other sources is not always clear, particularly in phase 1, as the reports detailing this have not been made available to the evaluation team. While ICLEI has indicated that its finance office has documented records on funds leveraged, the project team was unable to access this data. In all cases ICLEI has reported on the impact of projects (e.g. emissions avoided) noting the project as an ICLEI project and supplying total figures for the project when in fact some of the credit needs to be apportioned to other funding initiatives. While it is recognised that ICLEI funding acts as seed finance and is thus an important stimulus for many projects, this lack of qualification and clarification is an omission that can lead to a misunderstanding of the overall impacts of the campaign. Powers and functions In one instance, that of Saldanha Bay, the program has been impacted on by a shift in the municipal powers and functions which has resulted in the environmental health function shifting from the local municipality to the district municipality. This has resulted in the ICLEI project being left in something of an institutional vacuum. Role of dedicated champions In many of the municipalities it is clear that the role of a dedicated champion is a key plus factor contributing to the success of the program. The presence of a passionate, dedicated official is important. In some instances, such officials have left, resulting in the program stalling in the municipality. This issue is important for sustaining skills and knowledge within the institutions and mechanisms need to put in place to ensure that local institutional memory and historical knowledge is sustained. This can partly be improved better record keeping and staff changeover management but also comes to down to expanding knowledge 26 ICLEI CCP – Final Evaluation 27 sharing internally and embedding these environmental initiatives and awareness in the organisation. This may stimulate a wider web of interest and enthusiasm for projects directed at reducing emissions and saving energy. Municipal and overall program timing and funding cycles Some municipalities have had problems aligning their planning timeframes and budgets with those of the funders. This has resulted in some work being rushed in order to fit in with funders requirements and has had the effect of less than sustainable projects being developed. Funding cycles, particularly in the later phases of the program, also proved to be a deterrent to some municipalities who did not feel it was practically possible to plan and implement work within these phases and thus did not apply for grants. The relationship between officials and politicians Projects have tended to be more successful where there is political buy-in to projects on the part of municipal politicians. Where officials and politicians both see the value of projects, they are more likely to succeed and have the desired impacts. Municipalities, such as Potchefstroom, provide a clear illustration of what can be achieved through the combination of dedicated officials and interested, knowledgeable and enthusiastic Councillors and Mayors. With political buy-in and drive being essential to the success and sustainability of programs of this kind; some municipalities have addressed this by packaging their climate change strategies in a broader sustainable energy strategy or policy. This has generally proven successful for getting council on board to support the implementation the sustainable energy and climate change initiatives. Impact of the program on officials and politicians Participating officials and involved politicians are among those who have benefited most directly from the campaign. In most cases, the program has resulted in an increased awareness of energy saving programs and air quality issues among officials. Where cities have successfully managed to pair a political representative with a technical official at workshops and conferences it has had positive effects. The information flow from official to politician has increased their understanding of the importance of environmental and climate issues and the political support obtained has helped to speed up approval and implementation on certain projects. Range of bureaucratic issues faced Given the range of institutional locations of projects within municipalities and the fact that municipalities of greatly varying size and capacity were involved in the program, different municipalities’ experienced bureaucratic complexity in different ways. For example, smaller municipalities often felt it easier to secure political buy-in because it is easier for officials to make direct contact with politicians on a routine basis. Similarly, securing Council approval of projects tends to be easier in smaller municipalities than in large metros where Council agendas are more complex. In some instances municipalities found that procurement processes and procedures did not easily fit into the requirements of projects, creating administrative difficulties in terms of timeframes and funding. 27 ICLEI CCP – Final Evaluation 28 4.5 Networking and knowledge management 4.5.1 Conferences and training Most officials participating in the structured ICLEI workshops and conferences found these to be extremely useful in terms of knowledge sharing and networking. In many instances, these formal interactions had the spin-off of leading to less formal peer-to￾peer support. Formal networking was not restricted to conferences and workshops and a wide range of field trip exchanges was done which were well received by participants. There is a general sense that capacity and skills developed was achieved and this has spurred greater enthusiasm for a continuation of these training and workshop sessions. All municipalities have noted the benefits of these sessions as important for expanding their internal knowledge base and enlightening them as to the possibilities achievable for local government to impact on climate change and energy savings. 4.5.2 Informal networking Informal networking, often evolving from formal networking arrangements, is a feature that has emerged as a result of the campaign. In some instances, municipalities have indicated that they have established informal support networks with other participating municipalities to compensate for what they perceive to be gaps in formal support from ICLEI. This is largely a positive development that illustrates a commitment to sustainability of the projects outside of ICLEI support. There exists, however, the risk that this could lead to a lack of focus or that the focus of the networking could be narrowly driven by the interests of the existing network members. 4.5.3 Mentoring arrangements Aside from the informal mentoring discussed above, there was one instance where formal mentoring between Cape Town and Mangaung occurred at a project level. This interaction worked well, to the mutual benefit of both parties. Both cities have expressed a great appreciation for the opportunity to share and learn between them as well as to receive training from the air quality workshops attended. The cities plan on continuing their relationship and other small municipalities have looked to Mangaung for guidance on air quality monitoring, which is an encouraging outcome. 5 Conclusion 5.1 Overall Impressions o In terms of establishing a local network, the project has been an undeniable success. It has opened up opportunities for municipalities to engage with each other around climate change, something which otherwise may not have taken place. o The CCP has also given local officials the opportunity to make a positive contribution to climate change and energy efficiency and this has stimulated creative and energetic responses to global issues at the local level. o As seed funding, the USAID and DEAT funds that were channelled through CCP have had a significant effect on local government environmental initiatives in the field of energy efficiency. It is clear that this funding has been a fundamental spark for many other projects and initiatives aimed and sustaining energy and addressing climate change. The spin-off effects have been noted by all those interviewed in this evaluation and there is a genuine appreciation 28 ICLEI CCP – Final Evaluation 29 amongst municipalities for the contribution that these funds have made to increasing awareness and stimulating other initiatives in the local municipality. The proven effects of the projects implemented and the knowledge gained and shared between municipalities have resulted in an impressive snowballing of activity within municipalities which can be traced back to the influence of the ICLEI CCP program. o Workshops and technical training was generally well received and local officials have clearly benefited from these sessions, and in some cases so too have politicians. 5.2 Lessons learned o The need for local support and capacity building can not be underestimated. Municipalities have stressed the important value that they received from training and it is clear that in order sustain work of this nature training and capacity building needs greater development. o The platforms provided for knowledge dissemination, experience sharing and fieldtrips have been inspiring. They have enlightened cities and exposed them to opportunities that are practical to apply and which can make a meaningful difference to both cost savings and environmental action. o The positive implications of strong and stable institutional structures and similarly the negative impact of weak institutional structures are important lessons to carry forward. o The presence of a champion in the municipality has been important to the success of these projects. It also noted that a good working relationship and understanding between the technical official and politician goes a long way in achieving wider success with these initiatives. o While individual drive and commitment makes a large impact on the success of initiatives it important that too much of the success does not rest on a few individuals. Sustainability depends in large part on creating wider awareness, obtaining greater buy-in and institutionalising the motivation to achieve results to become more energy efficient and responsive to climate change and its affects. o Communication and support are important for success, both within and between the municipalities as well as between the key role-players from other entities involved in the network of CCP. o National support and the provincial – local interface need to be advanced for local initiatives such as CCP to have a national impact. 5.3 Recommendations 5.3.1 Recommendations for ICLEI o ILCEI needs to work on strengthening and stabilising their South African office, especially since it is the Africa Directorate. o Staff retention should be a focus and more significantly the retention of knowledge and passing on of skill should be given more attention. Institutional memory should also be addressed so as to ensure that the exit of human capital does not result in the exit of the knowledge base in the organisation. o There needs to be greater consideration given to understanding and overcoming internal obstacles facing municipalities such as local legislation e.g. 29 ICLEI CCP – Final Evaluation 30 procurement policy, which needs to be taken account of when implementing programs. This may indicate a need to improve channels of communication so as to engage with local officials on these issues prior to implementation. o Clear definitions and communication of the roles and responsibilities of ICLEI staff as well as other stakeholders are vital to the flow of the program. Municipal staff in the CCP cities needs to be informed as to who the role players are and where responsibilities rest, as it will allow them to channel their communication to the appropriate people and may improve the efficiency and effectiveness of programs. 5.3.2 Recommendations for DEAT o The fact that many municipalities were not aware of DEAT’s role in the program indicates a lack of communication from DEAT’s side. o It is clear that DEAT needs to play both a supportive and leadership role in climate change and energy efficiency initiatives in South Africa as municipalities look to national government for direction. Recent developments indicate that some headway has been made in this regard, with DEAT currently in the process of establishing a sub-directorate for municipal support. o DEAT is well placed to play a co-ordinating role of municipal activity and local government environmental initiatives. DEAT may do well to serve as the core of the local network and can play a key role in developing local skill and capacity by organising regular training and sharing workshops for local government in future. o Many of projects that have come out of the CCP have been very successful and it may be worthwhile exploring public-private partnerships to expand this kind of initiative to the private sector. 5.3.3 Recommendations on Phases 1 and 2 o The flexibility of phase 1 and the fact that it allowed the municipalities to be innovative with their project design has been highly commended and a key part the success of this phase. It is recommended that this approach be continued in other initiatives of this kind as it is likely to stimulate great interest and promote the development of creative ideas for reducing emissions and saving energy. o While the value of networks has been identified as a key achievement of CCP in South Africa it is noted that these need to be more organised and focussed rather than too loose and informal. Informal relationships between local officials have been invaluable but there is a concern that this informality could weaken these networks if individuals leave the municipality. While it is understandable that personal relationships sustain this networks, it is preferable that the networks be institutionalised rather than rest on individual relationships. o The flow between phases 1 and 2 seems to have had good continuity but it is recommended that more needs to be done to monitor and evaluate each phase before progressing to the next, especially as phase 2 built on phase 1. o Verification and adequate assessment of the quality of the inventories has been a problem of note, one which has had repercussions later down the line. Accuracy of the inventories is important and it is imperative that inventories are verified and checked for reliability in the early phase. o The quality of technical skills to complete the inventories correctly needs to be identified early on and where there is a lack of skill, high level support needs to be provided so that the quality of the inventories is ensured. 30 ICLEI CCP – Final Evaluation 31 o The 5 milestones process worked well because it provided direction and guidance for what should be achieved in the initiative. However, the monitoring and evaluation component was not adequately fulfilled and more needs to be done in future to adequately implement mechanisms for this. o There is a keen interest in some of the smaller cities to make use of HEAT software in future and ICLEI has indicated aspirations for this as well. However, it is clear that the greatest impediment to these goals is the lack of local support for municipalities. Thus it is recommended that before pursuing the expanded usage of HEAT software, local support and technical capacity is established to support municipalities in making effective use of this software. 5.3.4 Recommendations on Phase 3 o There appeared to be a lack of continuity and flow from phase 2 to phase 3, most notable in the change of agenda and the relatively restrictive approach taken in phase 3 compared to the earlier phase. o Planning around phase 3 and the use of lessons learnt from the earlier phases may have helped improve the continuity between the phases. This links again with the importance of monitoring and evaluation between phases. o Phase 3’s restrictive focus on air quality and transport only, tight time frames and deadlines seemed to have had negative implications for the success of this phase. Many municipalities felt under enormous pressure to achieve unreasonable and unrealistic goals, which lead to a lower level of participation in projects for this phase. Thus it is recommended that reasonable time frames be adopted and that greater flexibility be given to inspire municipalities to devise innovative ideas for dealing with climate change and energy efficiency. As a final thought, the evaluation team felt that there was so much positive feeling about the CCP program within the cities involved, that a selection of quotations from officials interviewed would be of interest to USAID and other interested in the achievements of the program… 31 Key Quotes - what the cities had to say o “Phase 1 was brilliant! It allowed us to be flexible and creative.” Craig Haskins, City of Cape Town. o “Mentoring proved to be invaluable and the workshops exposed us to new ideas.” Tumelo Shakwane, Mangaung. o “CCP was a great kick-start program which had many positive spin-offs for the municipality.” Bobby Buchanan, Sol Plaatje. o “CCP as acted as the catalyst for putting climate change and energy efficiency on the political agenda.” Mahesh Roopa, Potchefstroom. o “The ICLEI-CCP network was very useful and it evolved into a solid family which met to share experiences and knowledge”. Sam Mutswari, City of Tshwane. o “It was a great advantage that the CCP program allowed municipalities to be part of the national and international program.” Jane Eagles, City of Johannesburg. o “It is only thanks to the work done with ICLEI on CCP that awareness of climate change issues has increased.” Jessica Rich, eThekwini Municipality. ICLEI CCP – Final Evaluation 32 Appendices Appendix 1 Interview Instrument: Municipalities Introduction General introduction of the interviewer, Introduce PDG as the evaluator, Outline the main aims of the interview session. Overall Perceptions: Successes and Failures Semi-structured discussion Structure the discussion around the 5 Milestones of the program • Emissions inventory and forecast • Establish an emissions goal • Develop and obtain approval for a Local Action Plan • Implement policies and measures • Verify and monitor results Key questions Most successful elements: what worked best and why? Least successful elements: what where the obstacles? Progress Semi-structured discussion Focus on the specific achievements against each objective/deliverable of the phase Phase 1 Discussion pointers Reflect on the following activities when discussing this phase: • Project planning, participant and partner selection, • Training and capacity building, • Grants to cities, • Municipal operations: Energy end-use analysis, identification of opportunities, • City peer exchanges. Discuss the achievements of this phase of the program, Discuss any shortcomings, failures related to this phase of the program, Discuss the role of DEAT and USAID and joint administrators, Discuss the role of ICLEI, 32 ICLEI CCP – Final Evaluation 33 Discuss the value of the technical support and tools provided (HEAT software customisation, technical support for the inventories). Phase 2 Discussion pointers Phase 2 extended the program to an additional three cities and focused on measuring implementation of identified activities. Reflect on the following activities when discussing this phase: • Measure implementation, • Technical assistance from ICLEI, • Expansion to include additional cities, • Linkages with other programs/initiatives. Discuss the achievements of this phase of the program Discuss any shortcomings, failures related to this phase of the program Phase 3 Discussion pointers Phase three expands ICLEI’s CCP program to include three tasks: • Increasing awareness and disseminating best practice on how to reduce GHG and priority air pollutants in response to the Air Quality Management Act (AQMA), • Providing technical assistance to targeted municipalities interested in reducing emissions from the transportation sector, • Awarding of grants to cities to implement GHG and priority air pollutant emissions. Discuss the achievements of this phase of the program Discuss any shortcomings, failures related to this phase of the program Note: Ask for supporting information or reports on each of the phases Data collection and reporting Have the data reporting mechanisms worked? Are they easy to comply with? Have comprehensive reports been prepared? Overall Impacts Internal Discuss training and capacity building in the municipality as a result of the program, To what extent has institutionalisation of the program occurred within the municipality Discuss the extent to which the program has changed the municipality’s institutions External Project Impacts • What specific projects have been implemented? • Who has benefited directly from the projects? 33 ICLEI CCP – Final Evaluation 34 Sustainability Can the municipality sustain the momentum without donor and external technical support? • Sustainability of emissions inventories • Sustainability of projects Replication Potential and Knowledge Sharing What have you learnt that could be useful to other municipalities? How has information been shared and disseminated? • to the public, • within the municipality, • to the funders and • across municipalities Do you think this program ought to be rolled out to other cities? 34 ICLEI CCP – Final Evaluation 35 Appendix 2 Interview Instrument: USAID, DEAT, and ICLEI Introduction General introduction of the interviewer, Introduce PDG as the evaluator, Outline the main aims of the interview session. Overall Perceptions: Successes and Failures Semi-structured discussion Structure the discussion around the 5 Milestones of the program • Emissions inventory and forecast • Establish an emissions goal • Develop and obtain approval for a Local Action Plan • Implement policies and measures • Verify and monitor results Key questions Most successful elements: what worked best and why? Least successful elements: what where the obstacles? Progress Semi-structured discussion Focus on the specific achievements against each objective/deliverable of each phase outlined below. Phase 1 Discussion pointers Reflect on the following activities when discussing this phase: • Project planning, participant and partner selection, • Training and capacity building, • Grants to cities, • Municipal operations: Energy end-use analysis, identification of opportunities, • City peer exchanges. Discuss the achievements of this phase of the program, Discuss any shortcomings, failures related to this phase of the program, Discuss the role of DEAT and USAID and joint administrators, Discuss the role of ICLEI, Discuss the value of the technical support and tools provided (HEAT software customisation, technical support for the inventories). 35 ICLEI CCP – Final Evaluation 36 Phase 2 Discussion pointers Phase 2 extended the program to an additional three cities and focused on measuring implementation of identified activities. Reflect on the following activities when discussing this phase: • Measure implementation, • Technical assistance from ICLEI, • Expansion to include additional cities, • Linkages with other programs/initiatives. Discuss the achievements of this phase of the program Discuss any shortcomings, failures related to this phase of the program Phase 3 Discussion pointers Phase three expands ICLEI’s CCP program to include three tasks: • Increasing awareness and disseminating best practice on how to reduce GHG and priority air pollutants in response to the Air Quality Management Act (AQMA), • Providing technical assistance to targeted municipalities interested in reducing emissions from the transportation sector, • Awarding of grants to cities to implement GHG and priority air pollutant emissions. Discuss the achievements of this phase of the program Discuss any shortcomings, failures related to this phase of the program Note: Ask for supporting information or reports on each of the phases Data collection and reporting Have the data reporting mechanisms worked? Have comprehensive reports been prepared? What confidence is there in the data reported? Has the data been useful in overall program management? Overall Impacts Internal Discuss training and capacity building in the participant cities as a result of the program, To what extent has institutionalisation of the program occurred within the cities, Discuss the extent to which the program has changed the municipality’s institutions. External Project Impacts • What are the main project achievements on the ground? 36 ICLEI CCP – Final Evaluation 37 • Who has benefited directly from the projects? Sustainability Can the municipality sustain the momentum without donor and external technical support? • Sustainability of emissions inventories • Sustainability of projects Replication Potential and Knowledge Sharing How has information been shared and disseminated? • to the public, • within the municipality, • to the funders and • across municipalities Do you think this program ought to be rolled out to more SA cities? How well has the peer-to-peer knowledge sharing worked? Institutional Issues What have the institutional arrangements been? How well have the institutional arrangements worked? Discuss issues around institutional memory, staff turnover etc and the impact that these issues have had on the program How efficient have the management structures been (including financial management)? 37