IUWASH PLUS MIDTERM EVALUATION REPORT USAID Contract No. AID-497-C-16-00006 Indonesia Monitoring & Evaluation Support Project November 2019 DISCLAMER. This report is made possible by the generous support of the American people through the United States Agency for International Development (USAID). The contents are the responsibility of the author and do not necessarily reflect the views of USAID or the United States Government. i | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV TABLE OF CONTENTS LIST OF FIGURE....................................................................................................................................................III LIST OF TABLE......................................................................................................................................................IV ACRONYMS AND ABBREVIATIONS .............................................................................................................V EXECUTIVE SUMMARY...................................................................................................................................VIII SUMMARY CONCLUSIONS ...................................................................................................................IX SUMMARY RECOMMENDATIONS .......................................................................................................X FINDINGS AND CONCLUSIONS .........................................................................................................X 1. EVALUATION PURPOSE AND EVALUATION QUESTIONS ............................................................. 1 1.1. PURPOSE AND OBJECTIVES...................................................................................................... 1 1.2. EVALUATION QUESTIONS........................................................................................................ 1 2. PROJECT BACKGROUND........................................................................................................................ 2 2.1 SCOPE OF THE IUWASH PLUS PROJECT............................................................................. 2 3. EVALUATION METHODS AND LIMITATIONS................................................................................. 3 3.1. LIMITATIONS .................................................................................................................................. 3 4. EVALUATION QUESTION 1: REACHING THE POOR (B40)........................................................ 4 4.1 DESCRIPTION OF PROJECT ACTIVITY ................................................................................. 4 4.2 FINDINGS PROJECT ACHIEVEMENTS: INTERMEDIATE-LEVEL RESULTS .................. 6 4.3 FINDINGS: SYSTEM STRENGTHENING...............................................................................11 4.4 CONCLUSIONS............................................................................................................................21 4.5 RECOMMENDATIONS...............................................................................................................23 5. EVALUATION QUESTION 2: SUSTAINABILITY CHECKLIST .....................................................26 5.1. DESCRIPTION OF PROJECT ACTIVITY ...............................................................................26 5.2. PROJECT ACHIEVEMENTS........................................................................................................28 5.3. FINDINGS.......................................................................................................................................29 5.4. CONCLUSIONS............................................................................................................................34 5.5. RECOMMENDATIONS...............................................................................................................35 6. EVALUATION QUESTION 3: SANITATION MARKETING STRATEGY...................................38 6.1. DESCRIPTION OF PROJECT ACTIVITY ...............................................................................38 6.2. PROJECT ACHIEVEMENTS........................................................................................................39 6.3. FINDINGS.......................................................................................................................................40 6.4. CONCLUSIONS............................................................................................................................49 6.5. RECOMMENDATIONS...............................................................................................................50 7. EVALUATION QUESTION 4: NATIONAL/ LG POLICY ON B40 AND SDG 6.1 AND SDG 6.2 53 7.1. DESCRIPTION OF PROJECT ACTIVITY ...............................................................................53 7.2. FINDINGS.......................................................................................................................................56 7.3. CONCLUSIONS............................................................................................................................60 7.4. RECOMMENDATIONS...............................................................................................................61 8. GENDER........................................................................................................................................................62 8.1. FINDINGS.......................................................................................................................................62 8.2. CONCLUSIONS............................................................................................................................63 8.3. RECOMMENDATIONS...............................................................................................................63 ANNEX I: SCOPE OF WORK MIDTERM EVALUATION OF IUWASH PLUS PROJECT...............64 ANNEX 2: SOURCES OF INFORMATION..................................................................................................74 USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | ii ANNEX 3: EVALUATION METHODS ..........................................................................................................84 ANNEX 4: TOOLS ..............................................................................................................................................87 ANNEX 5: IUWASH PLUS RESULTS FRAMEWORK..............................................................................128 ANNEX 6: SUMMARY OF COMPONENT I ACTIVITIES ......................................................................129 ANNEX 7: OVERVIEW OF COMMUNITY MOBILIZATION AND STBM (12 STEP PROCESS)130 ANNEX 8: PROJECT METHODS TO MEASURE ACCESS TO WASH SERVICES AND B40 INDICATORS......................................................................................................................................................131 ANNEX 9: HOW ACCESS TO IMPROVED WATER AND SANITATION IS MEASURED .........132 ANNEX 10: DETAILS OF PROJECT ACHIEVEMENTS ...........................................................................134 ANNEX 11: IUWASH PLUS HOTSPOT MAPPING AND COMMUNITY PARTICIPATORY PLANNING..........................................................................................................................................................142 ANNEX12: CYCLE AND ACCOMPLISHMENTS OF PARTICIPATORY M&E.................................145 ANNEX 13: WASH ECOSYSTEM.................................................................................................................146 ANNEX 14: B40 VERIFICATION OF MWATER SURVEY COSTS ......................................................147 ANNEX 15: SUMMARY OF MTE KII SITES IUWASH PLUS PROJECT PY3 B40 TARGETS FOR WATER AND SANITATION AND IDENTIFIED HIGH LEVEL CITY B40 FOCUS/ IF ANY........148 ANNEX 16: SAMPLE SUSTAINABILITY CHECKLIST .............................................................................149 ANNEX 17: SURVEY RESPONSES REGARDING INFRASTRUCTURE NOT FUNCTIONING AS DESIGNED...........................................................................................................................................................151 ANNEX 18: ANALYSIS OF MAIN ACTIVITIES ACROSS SEVEN SUSTAINABILITY CHECKLIST152 ANNEX 19: TOILET SYSTEM.........................................................................................................................157 ANNEX 20: IUWASH PLUS MTE REVIEW OF 27 POLICIES SUPPORTED BY THE PROJECT THROUGH PY2 .................................................................................................................................................159 ANNEX 21: URBAN DOMESTIC WASTEWATER MANAGEMENT .................................................171 ANNEX 22: MEMBERS OF THE MIDTERM EVALUATION REVIEW TEAM....................................172 iii | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV LIST OF FIGURE FIGURE 1: ACCESS TO WATER AND SANITATION SERVICE...................................................... 8 FIGURE 2: COMPARISON OF COMPLETION OF STAKEHOLDER MEETINGS IN HOTSPOTS WITH B40 SANITATION ACCESS (HOTSPOT) AND B40 DIRECT WATER ACCESS (REGION)......................................................................................................................................17 FIGURE 3: THREE MAIN SECTIONS OF SUSTAINABILITY CHECKLIST...................................27 FIGURE 4: WHAT MAKES THE SUSTAINABILITY CHECKLIST TOOL USEFUL?....................34 FIGURE 5: SANITATION MARKETING IN MARKET-BASED APPROACH................................41 FIGURE 6: PROJECT ACTIVITIES FOR POOL OF FUNDS ..............................................................54 FIGURE 7: PROJECT ACTIVITIES FOR MICROFINANCE-SUPPORTING POLICIES ...............54 FIGURE 8: REGULATION APPROACH TO ADDRESSING B40 AND ATTAINMENT OF SDG 6.1 AND 6.2 .........................................................................................................................................59 FIGURE 9: CUMULATIVE RESULTS FOR B40 WATER SERVICES BY SITE (PY3 PROJECTED RESULTS)......................................................................................................................................................135 FIGURE 10: ACCESS TO INDIRECT WATER SERVICES................................................................136 FIGURE 11: PROJECTED NUMBER OF B40 INDIVIDUALS GAINING ACCESS TO WATER, BY TYPE OF SERVICES CONNECTION.............................................................................................137 FIGURE 12: ACCESS TO SANITATION BY REGION, PY3 PROJECTED..................................139 FIGURE 13: INDIVIDUALS GAINING ACCESS TO BASIC AND SHARED SANITATION, BY TYPE AND HOTSPOT/NON-HOTSPOT...........................................................................................140 FIGURE 14: PERCENTAGE OF B40 BY SANITATION SYSTEM TYPE .......................................141 USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | iv LIST OF TABLE TABLE 1: HOTSPOT COMMUNITY PARTICIPATORY PLANNING/ STAKEHOLDER PROCESS STATUS (AS OF APRIL 2019) ...............................................................................................19 TABLE 2: SUSTAINABILITY CHECKLISTS COMPLETED IN PY1 AND PY2..............................28 TABLE 3: SUMMARY OF MAIN ACTIVITY CONTENT ACROSS SEVEN SUSTAINABILITY CHECKLISTS..................................................................................................................................................30 TABLE 4: HOW OFTEN DO STAKEHOLDERS MEET AS A GROUP TO DISCUSS AND COLLABORATE DURING.........................................................................................................................31 TABLE 5: SANITATION MARKETING ACTIVITIES FOR EACH AREA.......................................39 TABLE 6: BEHAVIOR CHANGE AND OTHER FORMATIVE RESEARCH REPORTS FOR THE SANITATION MARKETING STRATEGY.....................................................................................41 TABLE 7: STATUS OF FECAL SLUDGE MANAGEMENT/WASTE WATER TREATMENT IN SANITATION MARKETING PILOT SITES ............................................................................................47 TABLE 8: NATIONAL AND LOCAL GOVERNMENT POLICIES, FRAMEWORKS AND REGULATIONS DEVELOPED WITH IUWASH PLUS SUPPORT...................................................56 TABLE 9: CRITERIA SELECTION FOR MTE SITE VISITS..................................................................85 TABLE 10: WATER AND SANITATION SERVICE TYPES SUPPORTED BY THE PROJECT .........................................................................................................................................................................132 TABLE 11: PY3 AND CUMULATIVE RESULTS FOR WATER SERVICE ACCESS OF APRIL 2019................................................................................................................................................................134 TABLE 12: STATUS OF BASIC AND SHARED SANITATION SERVICES..................................138 TABLE 13: SAMPLE OF TWO B40 VERIFICATION SURVEY’S COSTS CONDUCTED IN 2019................................................................................................................................................................147 TABLE 14: OVERALL BUDGET AND COST FOR B40 SURVEYS................................................147 v | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV ACRONYMS AND ABBREVIATIONS ADS Automated Directive System AMPL Air Minum Penyehatan Lingkungan / Drinking Water Environmental Sanitation AMEP Activity Monitoring and Evaluation Plan APBD Anggaran Pendapatan dan Belanja Daerah/ Local Government Budget APBN Anggaran Pendapatan dan Belanja Negara/ National Government Budget APPSANI Asosiasi Pengelola dan Pemberdayaan Sanitasi Indonesia/ Association of Indonesian Sanitation Entrepreneurs B2B Business-to-Business B40 Bottom 40 percent of the population in terms of wealth BAIK Baytul Ikhtiyar Cooperative BC Behavior Change Bappeda Badan Perencanaan dan Pembangunan Daerah/Regional Development Planning Agency Bappenas Badan Perencanaan dan Pembangunan Nasional/National Development Planning Agency BDS Business Development Services BMI Benteng Micro Indonesia Sharia Cooperative BPD Bank Pembangunan Daerah/ Regional Development Bank BPPSPAM Badan Peningkatan Penyelenggaraan Sistem Penyediaan Air Minum/Agency for Improving the Implementation of Water Supply System BPR Bank Perkreditan Rakyat/Rural Bank BPRS Bank Perkreditan Rakyat Syariah/Syaria Rural Bank BPSDM Badan Pengembangan Sumber Daya Manusia/Human Resources Development Agency BUMDes Badan Usaha Milik Desa/ Village-owned Enterprises CBO Community Based Organization CEM Citizen Engagement Mechanisms CLTS Community-led Total Sanitation COP Chief of Party CSR Corporate Social Responsibility DAI Development Alternatives, Inc DAK Dana Alokasi Khusus/ Special Allocation Budget DEC Development Exchange Clearinghouse DED Detailed Engineering Design DHS Demographic and Health Survey Dinas Perkim Dinas Perumahan dan Permukiman/Provincial Office of Housing and Settlements Dinkes Dinas Kesehatan/Provincial Health Office DISPERUM KPP Dinas Perumahan, Kawasan Permukiman, dan Pertanahan/Provincial Office of Housing, Settlement, and Land Ditjen Cipta Karya Direktorat Jenderal Cipta Karya/Directorate General of Settlements DitSPAM Direktorat Pengembangan Air Minum/ Directorat of Water Supply Development DPRD Dewan Perwakilan Rakyat Daerah/Regional Parliament EQs Evaluation Questions USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | vi FGD Focus Group Discussion FORKAMI Forum Peduli Kualitas Air, Limbah, dan Lingkungan/Water, Waste and Environment Quality Forum FSM Fecal Sludge Management FY Fiscal Year GOI Government of Indonesia HH Household HR1 High-level Result 1 HR2 High-level Result 2 IDHS Indonesia Demographic Health Survey IPAL Instalasi Pengolahan Air Limbah/ Wastewater Treatment Plant (WWTP) IPLT Instalasi Pengolahan Lumpur Tinja/ Septage Treatment Plant (STP) IUWASH PLUS Indonesia Urban Water Sanitation and Hygiene/ Penyehatan Lingkungan untuk Semua JMP Joint Monitoring Program KEMKES Kementerian Kesehatan/Ministry of Health KI Key Informant KII Key Informant Interview KKMA Kajian Kerentanan Mata Air/ Vulnerability assessment and action plan KOTAKU Kota Tanpa Kumuh/ City without Slums KPK Komisi Pemberantasan Korupsi/ Corruption Eradication Commission KPP Kelompok Pengguna dan Pemanfaat/Community User Group KSM Kelompok Swadaya Masyarakat/Community Based Organizations (CBO) L2T2 Layanan Lumpur Tinja Terjadwal/ Scheduled Desludging Service LG Local Government LOP Life of Project LOE Level of Effort LSIC Local Sustainability and Innovation Component (LSIC) M&E Monitoring and Evaluation MBR Masyarakat Berpenghasilan Rendah/Low Income Community MCH Mother and Child Health MEGA M&E GIS Assistance MESP Monitoring & Evaluation Support Project MFI Microfinance Institution MIS Management Information System MM Master Meter MOF Ministry of Finance MOH Ministry of Health MOHA Ministry of Home Affairs MOU Memorandum of Understanding MPWH Ministry of Public Works and Housing MSC Most Significant Change MSI Management Systems International MTE Mid-Term Evaluation NGOs Non-Governmental Organization NUWAS Framework National Urban Water Supply Framework OJK Otoritas Jasa Keuangan/ Financial Services Authority O & M Operations and Maintenance PCA Principal Components Analysis vii | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV PDAM Perusahaan Daerah Air Minum/ Indonesian Regional Water Utility Company Perbup Peraturan Bupati/ Regent Regulation Perda Peraturan Daerah/ Regional Regulation Perkada Peraturan Kepala Daerah/ Head of Regional Regulation Perkim Perumahan dan Pemukiman/ Housing and Settlement Areas PKK Pembinaan Kesejahteraan Keluarga/ Family Welfare Movement PMP Performance Monitoring Plan POKJA Kelompok Kerja/Working Group PPLP Pengembangan Penyehatan Lingkungan Permukiman/ Environmental Sanitation Development PU Pekerjaan Umum/Public Work Office PUPR Pekerjaan Umum dan Perumahan Rakyat/ Public Work and Housing Office PY1 Project Year 1 PY1/Q1 Project Year 1/1st Quarter PY2 Project Year 2 QA Quality Assurance QPR Quarterly Progress Report RFP Request for Proposal RPAM Rencana Pengamanan Air Minum/ Water Safety Plan Rusunawa Rumah Susun Sederhana Sewa/Low-Cost Rented Apartment SC Sustainability Checklist SGD Sustainable Development Goal SOPs Standard of Procedures SOW Scope of Work SPAM Sistem Penyediaan Air Minum/ Water Supply System SPALD-S Sistem Pengolahan Air Limbah Domestik Setempat/ Onsite Domestic Wastewater Management System SPALD-T Sistem Pengolahan Air Limbah Domestik Terpusat/ Offsite Domestic Wastewater Management System SSEI South Sulawesi and Eastern Indonesia STBM Sanitasi Total Berbasis Masyarakat/Community Based Total Sanitation TA Technical Assistance TNP2K Tim Nasional Percepatan Penanggulangan Kemiskinan/The National Team for the Acceleration of Poverty Reduction TOC Theory of Change TPM Team Planning Meeting UPTD PALD Unit Pelaksana Teknis Daerah Pengelolaan Air Limbah/Regional Technical Implementing Unit of Domestic Wastewater Management UPTD PPA Unit Pelaksana Teknis Daerah khusus Perlindungan Perempuan dan Anak/ Regional Technical Implementing Unit for Women and Child USAID U.S. Agency for International Development USG United States Government WASH Water, Sanitation and Hygiene WASH Infrasturcture Water Supply or Sanitation Systems such as Septic Tanks, IPALs, Master Waste Meters, ect WSUP Water and Sanitation for the Urban Poor USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | viii EXECUTIVE SUMMARY EVALUATION PURPOSE AND EVALUATION QUESTIONS The midterm evaluation (MTE) of the Indonesia Water, Sanitation and Hygiene PLUS 'Penyehatan Lingkungan Untuk Semua' (IUWASH PLUS) Project aims to provide USAID, the Government of Indonesia (GOI) and other WASH stakeholders with an objective review of the project’s performance to date. The purpose of the MTE is to determine: (i) if any course correction is required; (ii) how the project is contributing to the achievement of the Sustainable Development Goals (SDGs) and the GOI’s universal access targets (100-0-100); and (iii) what factors are needed to sustain and scale approaches beyond the project period. The evaluation was conducted with 35 months of the project remaining and focused on answering four questions related to the project’s performance specific to Component 1: 1. To what extent is the IUWASH PLUS approach reaching the B40 (the bottom 40 percent of the population in terms of wealth) with WASH services? 2. Do the sustainability checklists provide utility to the project and serve as an effective tool to measure the results of direct activities? 3. To what extent are IUWASH PLUS sanitation marketing approaches effective in building public-private interest in this market-based approach? 4. How has IUWASH PLUS influenced national- and city-level policy in support of targeting the B40 and achievement of United Nation Sustainable Development Goals (SDGs) SDGs 6.1 and 6.2? PROJECT BACKGROUND The five-year project started in June 2016; with a contract extension, it is scheduled to be completed December 21, 2021. The original total contract was for $39.6 million and the current contract is valued at $48,357,584. In collaboration with the GOI, local governments (LGs), the private sector, local community-based organizations (CBOs), service utilities and local communities, the project has two high-level results: • An increase of 1,100,000 people in urban areas with access to improved water supply service quality, of which at least 500,000 are from the poorest 40% of the population (also referred to as the “Bottom 40%” or “B40”), vulnerable groups or Indonesia’s eastern provinces; and • An increase of 500,000 people in urban areas with access to safely managed sanitation. In addition, the project has two high-level indicators and 23 component-level indicators. Under Component 1, the key indicator of interest for this MTE given its focus on the poor is: • The number of B40 people gaining access to basic or shared sanitation services, with a target of 250,000 people. To achieve these results, the project is working with 32 LGs1 in eight provinces across Indonesia in urban areas, and supporting activities in the following four component areas: 1. Improving household WASH services; 2. Strengthening city WASH institutional performance; 1 It was applied during the MTE data collection period which was March-April 2019. ix | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV 3. Strengthening the WASH financing environment; and 4. Advancing national WASH advocacy, coordination and communication. The project’s four components are highly interrelated, and each must all perform well if the project is to achieve its higher-level results. • To increase demand for improved drinking water and sanitation, the project is working directly with communities, with a focus on increasing community WASH knowledge and strengthening the ability of communities to advocate for and manage communal water and sanitation services, particularly in areas with a high B40 populations. • To address the supply of improved services, the project is working with LGs and regional utility water providers (Perusahaan Daerah Air Minum, or PDAMs) to increase their capacity and pro-poor targeting, advocate for increased GOI funding for services for the B40 and improve the enabling environment through the development or revision of WASH policies and regulations. • To support achievement of its sanitation targets, the project’s sanitation marketing strategy is piloting efforts to support a private sector approach to providing improved sanitation products and services for low-income populations, particularly for “safely managed sanitation.” • As sustainability is a critical factor affecting the WASH sector, the project has developed “Sustainability Checklists” to support the management and implementation of sustainable initiatives that the project is involved in directly supporting. At the time of the MTE, several aspects of the project appear on track and are performing well while other aspects have not yet achieved significant results or have been slow to develop. The evaluation’s summary conclusions and recommendations follow. SUMMARY CONCLUSIONS • The project has done an excellent job in assisting LGs in identifying urban areas with a high concentration of poor households -- “hotspots.” This has raised awareness among LGs as to the importance of serving B40 populations. • To date, it has been IUWASH PLUS’ indirect work with PDAMs that has resulted in the majority of B40 households’ receiving improved water services, and these households mainly reside outside of hotspot areas.2 The majority of those having received improved access to basic and shared sanitation are also mainly in non-hotspot areas. • The project’s approach to reaching the poor appears to be sound in terms of the challenges being addressed; however, as of the time of this MTE, IUWASH PLUS results are inconclusive in terms of the effectiveness of the approach in significantly increasing the availability of WASH services for B40 urban populations. Several key aspects of the program, such as working to increase LG financial allocations for serving B40 populations, are still in their early stages of implementation and will take further time to reach fruition and demonstrate their level of effectiveness. • A lack of available financing appears to be the most significant constraint to expanding WASH services for B40 populations. This is the case for GOI financing of water infrastructure and is also the case for the provision of sanitation services at the community and household levels. 2 IUWASH PLUS indirect services are water connection services provided directly by Indonesian regional water utility companies (PDAMs) who have received technical assistance from IUWASH PLUS. USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | x • The use of Sustainability Checklists (SCs) has directly and positively supported the development of sustainable WASH infrastructure, including by strengthening the capacity of CBOs. SCs have been an effective tool to document and measure IUWASH PLUS direct activities. • The project’s sanitation marketing strategy has not achieved significant progress in developing private sector initiatives to provide increased sanitation services to B40 populations. The strategy lacks adequate analysis of the business models that would attract the private sector and customers. It also has an insufficient focus on the development of financing approaches suitable for poor households. • IUWASH PLUS has engaged with GOI agencies and other stakeholders to influence national policy supporting SDGs in several important ways. The efforts have helped to inform policy development through activities such as workshops, study visits, and twinning agreements. SUMMARY RECOMMENDATIONS • It is recommended that the project increase its focus on expanding the availability of finance through the following actions: 1) continue work to improve PDAM financial and operational efficiency; 2) continue to advocate for greater public resources to provide WASH services to B40 populations, including the increased use of subsidies; 3) increase engagement with the private sector on the provision of sanitation products and services, as the potential market is potentially large; and 4) continue to support policy and regulatory changes, such as support for the current OJK initiative, so as to increase the availability of financing mechanisms that can be utilized by the B40. • There are a few aspects of IUWASH that are particularly critical to its longer-term success, and which to date are still in their early stages of implementation, and thus somewhat untested. These include the ability to strengthen PDAM efficiency and effectiveness in ways that enable the PDAMs to serve an increased number of B40, and the ability to influence LGs in ways that make increased resources available to serve B40 populations. These aspects of the program should be considered for rigorous testing under a focused learning agenda. • SCs should continue to be used as they have proven effective. The use of the checklists will benefit from improved guidance and more consistent IUWASH PLUS training and support. • IUWASH PLUS should work to revise and reinvigorate its sanitation marketing strategy. An initial priority should be to hire additional staff to provide the effort increased attention. IUWASH PLUS should also redouble efforts to develop viable financing models (appropriate for B40 populations) and engage more directly with private sector companies to develop a commercial market for low-income sanitation products and services. More detailed findings, conclusions and recommendations are provided in the sections that follow for each of the MTE’s four evaluation questions, as well as in the body of the main report. FINDINGS AND CONCLUSIONS The approach used by the project to reach B40 populations includes several key elements, such as the Community Mobilization/Community-Based Total Sanitation (STBM) approach, ”hotspot” identification and targeting, B40 verification, and sanitation marketing. A significant component of IUWASH PLUS is its innovative approach to targeting B40 households in “hotspots.” The project is focusing implementation of the Community Mobilization/Urban STBM approach in hotspot areas, of which there are 102 among the 32 cities and districts. Evaluation Question 1: To what extent is the IUWASH PLUS approach reaching the B40 with WASH services? xi | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV Community Mobilization/Urban STBM is one IUWASH PLUS’ main approaches to target B40 populations. This approach is adapted from the Community-led Total Sanitation (CLTS) and GOI rural STBM approach to urban settings. A feature unique of the approach is that it allows for outside subsidies, “smart subsidies” and government assistance to address identified WASH needs. This is in contrast with existing GOI STBM and CLTS approaches that have traditionally had a “no subsidy approach.” These differences are considered critical for this approach to be effective in urban settings. While many WASH projects have a mandate to serve the poor, few have taken the steps that IUWASH PLUS has undertaken through its use of data and geospatial analysis to identify and target the poor. The project is innovative in that it is working to validate the socioeconomic status of project beneficiaries to ensure that it is reaching poor households. This is done through its use of the USAID Demographic and Health Survey household wealth index survey method, incorporated into a mobile phone platform (mWater). This approach has the potential to provide the project, and the WASH sector in general, with valuable information on methods for measuring whether the urban poor are being served. Conclusions regarding the extent to which IUWASH PLUS is reaching the B40 population include: • The IUWASH PLUS approach is increasing LG capacity, but this has not yet necessarily translated to increased services for the poor. Based on the data and analysis at the time of the MTE, the project is noticeably behind meeting its B40 population targets – although this is perhaps to be expected at this point in the implementation process. • The IUWASH hotspot mapping tool, designed to help identify and target those areas with high B40 population density, has proven to be powerful in building awareness and advocating LGs to acknowledge that these community pockets require special attention and assistance from their annual planning and budgeting process. • IUWASH PLUS reports that it has been successful in facilitating access to financing for 26 projects3 during its second year through national and local government budgets. While IUWASH PLUS staff indicated that many of the projects were already budgeted by the LGs prior to IUWASH PLUS’ involvement, IUWASH PLUS is assisting with implementation. It is important to note that IUWASH PLUS’ work to increase LG budgetary resources for providing WASH services to poor communities is expected to require more time to realize its full effectiveness. • While the project is supporting linkages to financing schemes for poor households to connect to water services, or to build sanitation facilities on their own, the availability and/or terms of these schemes appear inadequate to meet the needs and affordability of low￾income populations. RECOMMENDATIONS • Increase emphasis on developing finance and financing products to provide WASH services to B40 populations. This should include helping PDAMs operate more effectively, and exploring ways that service extension to low-income areas can be expanded; advocating for increased budget allocations and subsidies to be provided by the GOI; and increasing work to develop financial products appropriate to B40 populations, including continued coordination with the GOI’s Financial Services Authority (OJK) on its sanitation financing reform initiative. 3 As of the October 2018 of the 26 systems included 22 where construction is still in process. USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | xii • Reassess Life of Project water and sanitation B40 targets at the end of year 3 to determine whether the B40 targets need to be adjusted. • Conduct B40 data analysis for households directly served by IUWASH PLUS as data becomes available. This may help the IUWASH PLUS team to monitor performance more frequently and institute course corrections as needed. FINDINGS AND CONCLUSIONS The Sustainability Checklist (SC) is a tool created by IUWASH PLUS containing a list of activities that need to take place before, during, and after construction of WASH infrastructure, including activities that IUWASH PLUS considers critical to sustainability. For each SC activity, the SC allows teams to indicate which collaborator each WASH activity partners with for activity completion. There are seven versions of the SC form: three for water and four for sanitation infrastructure, with each infrastructure type having a different activity list. IUWASH PLUS reports describe two purposes for the SCs. First, SCs document the contribution of collaborating parties in building WASH infrastructure, including LGs, WASH entrepreneurs, CBOs and IUWASH PLUS. When activities have been completed, beneficiaries that gain access to water or sanitation can be counted in reporting as having received “direct support” for the project’s two high￾level result indicators for drinking water and safely managed sanitation. A second purpose of the SCs is to identify areas “that are critical to sustainability that are not adequately supported and where the project could effectively contribute in terms of identifying and filling such gaps.” In the first two years of the project, IUWASH PLUS completed 64 SCs, corresponding to 103,600 individual WASH connections (36 percent in sanitation) that were directly supported by IUWASH PLUS. A review of SCs completed during the project’s first two years confirms that IUWASH PLUS directly supported activities leading to more sustainable WASH infrastructure through activities such as triggering behavior change activities and CBO capacity building. This was confirmed in KII interviews with WASH partners, i.e., collaborators. All IUWASH PLUS and 81 percent of collaborator survey respondents indicated that SCs had been useful for collaboration as a reminder of the elements needed at each stage, as well as a reminder of collaborators’ commitments. Some key informants stated that the SCs had helped to provide joint accountability among collaborators. KII and survey data indicated that written guidance on how to complete the SCs was inconsistently disseminated, and that some LG collaborating partners found the forms confusing. Overall, however, the SC process has exposed LGs to important implementation aspects that must be considered in different sanitation approaches to increase the sustainability of such systems. RECOMMENDATIONS • Clarify the purpose of SCs and align design, implementation and “marketing” about the SC tool with its purpose. Refine and disseminate written guidance and training materials for the SCs based on a review and provide continual support to staff and partners. • Review, revise and standardize the various SCs, providing clarity on issues described in the findings, such as the incorporation of activities that ensure the quality of construction, clarity on how to verify that activities are completed, and a mechanism for tracking multiple infrastructure documented on the same checklist. • Review the process to increase the involvement of partners in the use of SCs. As part of this effort, explore if there is an interest by Indonesian agencies to adopt the tool. Evaluation Question 2: Do the sustainability checklists provide utility to the project and serve as an effective tool to measure result of direct activities? xiii | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV FINDINGS AND CONCLUSIONS The project’s sanitation marketing strategy is working to create a commercial marketplace for the provision of sanitation products and services for B40 populations. This includes working to make an increased number of financing options available, training entrepreneurs to provide products and services and building demand for services through community-focused behavior change messaging. However, the strategy was late in its implementation, with only three pilots being implemented between August 2018 and December 2018, and with the sanitation strategy document not finalized until February 2019. This will likely impede the project’s ability to meet its sanitation targets, as evidence indicates it can take five to seven years to establish a viable sanitation market. The strategy’s behavior change approach is based on extensive formative research. The research primarily focused on B40 households and was lacking in research on those involved in manufacturing or selling sanitation products. The research was also developed when the project’s original high-level targets included a safely managed sanitation target for B40 populations, which has since been removed. This has, in part, contributed to the lack of a clear, well-understood behavior-change approach and associated messaging within the strategy, which is critical as the market for these two populations are different (B40 and non-poor), as are the sanitation services (basic and shared facilities, and the safely managed delivery system). Financing. A significant constraint to the sanitation strategy’s implementation is the issue of affordability in terms of whether B40 households can make improvements to their sanitation facilities. There has not been adequate analysis of the business models that would attract the private sector and customers for serving poor households, or on mechanisms to finance such approaches. While the latter is being addressed in part through IUWASH’s coordination with the Financial Services Authority (Otoritas Jasa Keuangan, or OJK) in setting up a government sanitation loan program, this effort is still in process. While IUWASH has undertaken work to address finance options for B40 households, including addressing micro-finance regulations at the GOI policy level, the overall availability of financing schemes is insufficient to meet the needs of poor households for safely managed sanitation facilities. Entrepreneurs and the private sector. The strategy has not resulted in a sufficiently developed or supported private sector to meet current or potential increased consumer demand. Few entrepreneurs have been trained and even fewer retained.4 Poor retention of trained entrepreneurs appears to be in part due to entrepreneurs not having a viable market for the business of providing services to low-income households. This, which in part results from inadequate financing schemes, contributes to an inability of entrepreneurs to sustain business among poor consumers. The project has also worked to address the regulatory framework for fecal sludge management (FSM)/wastewater treatment and the capacity of desludging services and facilities. These have worked to support the creation of safely managed sanitation systems. However, other challenges 4 Forty-one entrepreneurs were trained during PY2 (IUWASH PLUS APR 02 QPR 09 Report) from a total of 100 sanitation entrepreneurs targeted by the project. Within the pilot project area, five entrepreneurs had been trained to date, but only two were identified as still being active in the pilot sites. Evaluation Question 3: To what extent are IUWASH PLUS sanitation marketing approaches effective in building public-private interest in this market-based approach? USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | xiv impede targeted sites from increasing safely managed sanitation to B40 households through a market-based approach. For example, reoccurring desludging costs were reported to be a disincentive for poor households to build watertight facilities, and desludging methods for hard-to￾reach densely populated urban poor continue to be lacking. From a policy perspective, the lack of enforcement of regulations for safely managed septic tanks, particularly for existing septic tanks, could also be serving as a disincentive for all households to pursue water-tight septic tanks. RECOMMENDATIONS • Accelerate activity by forming a sanitation business development service that coordinates closely with other stakeholders. IUWASH PLUS should be more proactive in approaching sanitation enterprises with a clear value proposition and business case, and in providing high￾quality market information. • IUWASH PLUS should increase engagement with established commercial sanitation companies to try and gain insights and identify opportunities on how to better commercially address the large potential low-income market for sanitation products and services. • Prioritize the identification of financing providers and mechanisms for B40 populations, while continuing to improve microfinance institution loan schemes and undertaking similar efforts with the GOI, the banking industry and other lending institutions, including creating flexible payment options to finance sanitation improvements. Consider partnerships with organizations and companies that specialize in this sector. • Redesign the sanitation marketing approach to link entrepreneurs more strategically with public service providers and government coordination forums, such as the LG Coordination Office (Pokja). Assist with the development of sanitation business plans, market analysis and capacity building that extends beyond training -- specific to development of sanitary products and services. FINDINGS AND CONCLUSIONS IUWASH PLUS has influenced national policy supporting SDGs in several important ways. It has conducted numerous activities to help inform policy development, including studies, workshops, study visits, and twinning agreements. In the process, the project has engaged with GOI agencies and other stakeholders to identify key policy or regulation gaps that could facilitate greater access to water and sanitation. To address these gaps, the project has worked with stakeholders to develop WASH policies and regulations and supported their implementation. Among five national-level policies identified by USAID,5 policy development was in process at the time of this MTE. The project also successfully supported the development of the policy framework for the DKI Jakarta Grand Design for drinking water and domestic wastewater (2018-22). Among city-level regulations, the project completed 47 local regulations as of March 2019, exceeding the life-of-project target of 35 local regulations. Among the 27 LG regulations supported in the first 5 Water safety plan, FSM framework and implementation, urban STBM, microfinance, pool of funds. Evaluation Question 4: How has IUWASH PLUS influenced national and city-level policy in support of targeting B40 and achievement of SDGs 6.1 and 6.2? xv | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV two years of the project, all but one directly supports the achievement of the SDGs. Of the remaining regulations, 10 relate to water supply, 16 to sanitation, and 11 address the B40 directly. The project has also directly supported the national-level universal access (100-0-100) policy that targets poor communities living in slum areas. IUWASH PLUS has indirectly supported national policies targeting WASH access for poor communities, such as the GOI’s Urban STBM. While the Urban STBM policy does not explicitly target the poor, it indirectly addresses the needs of poor households, given that the poor disproportionately lack access to water and sanitation in Indonesia. In addition, IUWASH PLUS has supported policies that benefit the poor directly, such as policies supporting household-level financing mechanisms for WASH service infrastructure, as well as the water safety plan concept. City-level policy. The project has influenced LG policy supporting the SDGs by engaging with WASH working groups, such as the LG Coordination Forums (Pokjas). IUWASH PLUS has prioritized its LG policy support work by identifying key WASH regulations that each LG needs, and conducting an inventory of WASH regulations for each partner LG. The project also assessed existing regulations to identify potential improvements. Finally, the project has supported the drafting or revising of new WASH regulations, as well as the implementation process. With respect to supporting policies targeting the B40, IUWASH PLUS has supported LG regulations that have established tariffs, service fees, financing guidelines or other schemes with special consideration for low-income households. RECOMMENDATIONS • Advocate for the inclusion of a clear policy on line ministry support for equitable access to WASH with respect to low-income communities. • Continue work to strengthen LG partners’ commitment and requirements to serve the poor, specifically the B40 in MOUs and annual partnership agreements. • The project should explore undertaking an integrated planning process with the GOI on how to better provide WASH services to low-income populations. This process should discuss the attainment of the project’s B40 targets, and the GOI’s goals, such as 100-0-100 and the SDGs, and how these targets will not be met unless stakeholders increase and improve their efforts to serve the poor. • Continue to collaborate with the GOI and LGs on developing a strategy for implementation and enforcement of existing and newly adopted regulations. Place a priority on policies that will contribute to increasing the demand for improved water quality and safely managed sanitation, such as policies to improve WASH affordability through service provider payment plans and other finance-related regulatory actions. 1 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV 1. EVALUATION PURPOSE AND EVALUATION QUESTIONS 1.1. PURPOSE AND OBJECTIVES The purpose of this midterm evaluation (MTE) is to: 1. Validate existing monitoring tools utilized over the first phase of the project, to determine if they require course correction in the remaining years of the project; 2. Assess the contribution by the Indonesia Urban Water, Sanitation and Hygiene ‘Penyehatan Lingkungan Untuk Semua’ 6 (IUWASH PLUS) to achievement of the Government of Indonesia’s (GOI’s) target of universal access (100-0-1007) by 2019 and Sustainable Development Goal (SDG) 6, “Water and Sanitation for All by 2030”; and 3. Identify key factors necessary to sustain and scale up IUWASH PLUS approaches beyond the project period. This MTE analyzes the IUWASH PLUS project’s performance to date and generates recommendations on improvements needed for the project to meet its intended purpose. The MTE conclusions and recommendations are to be used by USAID/Indonesia’s Environment Office and DAI to strengthen project implementation. The findings will enable all project stakeholders, including the GOI, to understand IUWASH PLUS strengths, and areas where technical methods could be improved and strategic approaches could refocus. This evaluation therefore provides information on project performance, program methods, approaches and sustainability specific to Component 1 of the project. Furthermore, this evaluation is intended to inform USAID’s global learning agenda on urban water, sanitation and hygiene (WASH). 1.2. EVALUATION QUESTIONS 1. IUWASH PLUS Targeting and Approaches. To what extent is the IUWASH PLUS approach reaching the B40 with WASH services? 2. Sustainability. Do the sustainability checklists provide utility to the project and serve as an effective tool to measure result of direct activities? 3. Sanitation Marketing Approaches. To what extent are IUWASH PLUS sanitation marketing approaches effective in building public-private interest in this market-based approach? 4. Alignment with GOI Policy. How has IUWASH PLUS influenced national and city-level policy in support of targeting the poorest 40 percent of the population (B40) and the achievement of SDGs 6.1-6.2? Lines of inquiry for each evaluation question can be found in Annex 1. 6 Penyehatan Lingkungan Untuk Semua is Indonesian for “Environmental Sanitation for All.’ 7 100 percent access to safe drinking water; 0 slums (eradication); and 100 percent access to access to improved sanitation. USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 2 2. PROJECT BACKGROUND The USAID Indonesia Urban Water, Sanitation and Hygiene ‘Penyehatan Lingkungan Untuk Semua’ (IUWASH PLUS) project is a five-and-a-half-year $39.6 million activity to assist the GOI in increasing access to drinking water supply and basic, shared and safely managed sanitation services, as well as improving key hygiene behaviors among urban poor and vulnerable populations. To inform future USAID investments in water, sanitation and hygiene (WASH), USAID/Indonesia is commissioning the Monitoring & Evaluation Support Project (MESP) to perform a midterm performance evaluation of IUWASH PLUS. 2.1 SCOPE OF THE IUWASH PLUS PROJECT IUWASH PLUS was originally a five-year activity (June 2016 – June 2021) to help the GOI increase access to improved water and sanitation services, and improve key hygiene behaviors, especially among the poorest and most vulnerable. The project received a six-month cost extension at the time of the midterm evaluation (MTE), resulting in a new project end date of December 21, 2021, and a total budget of $48,357,584. IUWASH PLUS, implemented by DAI Global LLC, works with GOI (national) and local government (LG) agencies, the private sector, non-governmental organizations (NGOs), service utilities and communities to achieve the following “high-level” results: • An increase of 1,100,000 people in urban areas with access to improved water supply service quality, of which at least 500,000 are from the poorest 40% of the population (also referred to as the “Bottom 40%” or “B40”), vulnerable groups or Indonesia’s eastern provinces; and • An increase of 500,000 people in urban areas with access to safely managed sanitation. In addition to these high-level results, the project has 23 component-level result indicators. Under Component 1, the key indicator of interest for this MTE is: Number of people from the bottom 40 percent by wealth gaining access to basic or shared sanitation services, with a target of 250,000 people. During the past three years, IUWASH PLUS has worked with 32 LGs in eight provinces across Indonesia (North Sumatra, West Java, Central Java, East Java, South Sulawesi, Maluku, North Maluku and Papua) and the special regions of DKI Jakarta8 and Tangerang District, supporting activities in four component areas: 1. Improving household WASH services; 2. Strengthening city WASH institutional performance; 3. Strengthening the WASH financing environment; and 4. Advancing national WASH advocacy, coordination and communication. In addition, IUWASH PLUS aims to cultivate local expertise through shared investments under the Local Sustainability and Innovation Component (LSIC). Each evaluation question (EQ) section contains a “Description of Activities” heading preceding a summary of the Component 1 project activities related to each EQ. 8 Daerah Khusus Ibukota Jakarta, or Jakarta Capital Special Region. 3 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV 3. EVALUATION METHODS AND LIMITATIONS For a detailed description of the three primary phases of the evaluation, please refer to Annex 3. APPROACH TO ANSWERING EVALUATION QUESTIONS Appendix D in Annex 1 summarizes key information for the analytical approach for addressing each evaluation question, including the data source, methods, potential key informants sampling plan and data analysis methods. The following section provides a description of the framework for answering each evaluation question. SITE SELECTION Fieldwork included site visits and direct observations at 11 sites in selected districts in West Java (Kota Bogor, Kabupaten Tangerang, Kota Depok), DKI Jakarta, Central Java (Kota Magelang, Kota Surakarta, Kota Salatiga), East Java (Kabupaten Lumajang, Kota Probolinggo) and South Sulawesi (Kota Makassar, Kabupaten Barru). Each site is a city or district where IUWASH PLUS provides technical assistance to address the four evaluation questions. Sites were selected based on location of project activities related to evaluation questions and logistical considerations. The team also visited Jakarta specific to EQ2, as the rusunawa, a type of sanitation system, operates only in Jakarta and the project does not have a memorandum of understanding (MOU) with the city government. The following inclusion criteria were considered in selecting the sites: • Location in a province (i.e., regional cluster of sites) where IUWASH PLUS sanitation marketing activities have occurred; • Coverage of seven types of water and sanitation systems (water systems include: master meter, communal SPAM9 and water grants; sanitation systems include: individual, communal, small-scale city sewerage and rusunawa); • Opportunity to observe Sustainability Checklist (SC) signing process (original intention); and • Logistical considerations. 3.1. LIMITATIONS Several limitations were present while conducting the MTE. The evaluation questions were fairly narrow in scope, but the overall project scope is large and its ecological approach means other project components have both a direct and indirect relationship with the results in the focus of this project, which is Component 1. To that end, given time and resource constraints, the evaluation was unable to consider all inter-related factors from other project components into its findings. The MTE also acknowledges that, given the depth and scope of information contained in IUWASH PLUS reports, some data and information may have been missed, despite best attempts to triangulate findings. Another limitation was the availability of updated monitoring data to assess progress toward the achievement of the B40 target results. To address this limitation, the project used results for PY3 (see EQ1). However, we recognize that the methods used could contain validity and accuracy issues that will need to be validated against final PY3 results when submitted to USAID. That said, the methods used are in line with PY2 methods and make the best use of available information at the 9 Sistem Penyediaan Air Minum, Water Supply System. USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 4 time of the evaluation (April 2019). In addition, IUWASH PLUS staff reviewed and approved them as appropriate. While the MTE conducted site visits in five regions, time constraints prevented visits to North Sumatra or any communities in the region of South Sulawesi and Eastern Indonesia (SSEI). Therefore, the findings from LG KIIs may not be applicable to these areas. Project M&E data were looked at from all areas, and regional staff who oversee work in Eastern Indonesia were interviewed. Finally, while the MTE considered gender-related issues to some degree, it limited its gender focus to information collected from KIIs and review of project documents, as relevant and available to respective EQs. The team was not able, as originally planned, to disaggregate responses among KIs by gender per evaluation question, because there was overlapped Kis across evaluation question. 4. EVALUATION QUESTION 1: REACHING THE POOR (B40) 4.1 DESCRIPTION OF PROJECT ACTIVITY The project’s overall focus is to improve access to WASH services for households in urban areas, with a specific emphasis on poor and vulnerable populations. The project is working to achieve two main results for the poor: increased access to safely managed drinking water services (HR1b) and increased access to shared or basic sanitation services (C1a).10 The project has defined “the poor” as households that are from the poorest 40 percent of the population (also referred to as the “bottom 40 percent” or “B40”). EQ1 focuses on Component 1 activities that have led to obtaining drinking water and sanitation services for B40 populations among the 32 targeted cities/districts among five regions. The project’s approach has several key elements among its Component 1 activities to achieve the B40 water and sanitation results. A brief summary of the key elements and how they align with the overall project WASH Ecosystem framework follows. WASH Ecosystem framework. The overarching project framework is its Urban WASH Ecosystem11 (see Annex 13). This framework recognizes that activities under one component are “highly interrelated” with the other project components to achieve WASH access for all households.12 A significant focus within the framework includes the provision of technical assistance to LGs involved in the delivery, planning or financing13 of WASH services. Combined, the project’s component activities work to create a more effective enabling environment from which WASH 10 These have been modified since the original contract was signed, most recently in April 2017. 11 IUWASH PLUS APR PY2-QPR09 This approach is reported by the project to also be in alignment with USAID’s “Sustainable Service Delivery in an Increasingly Urbanized World” policy (2013); see https://2012-2017.usaid.gov/news￾information/press-releases/oct-31-2013-urban-policy-launched-prepare-population-growth-cities 12 The other components are: 2. Strengthening city WASH institutional performance; 3. Strengthening the WASH financing environment; and 4. Advancing national WASH advocacy, coordination and communication. 13 For example, Pokja AMPL Kelompok Kerja Air Minum dan Penyehatan Lingkungan/working groups for water supply and sanitation. LEAD EVALUATION QUESTION 1: To what extent is the IUWASH PLUS approach reaching the B40 with WASH services? 5 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV services can improve and expand to address the needs of all households, particularly poor households. Community Mobilization/Urban STBM. A key element of the project’s B40 approach is its community mobilization and Urban STBM approach, in collaboration with the Ministry of Health (MOH) and LG health offices. This approach is modeled after the community-led total sanitation (CLTS) participatory methodology that the GOI adapted and adopted nationwide in rural areas in 2008, referred to as STBM. Similar to CLTS, a significant component of the IUWASH PLUS project’s approach is involved in working directly with communities to assist them in identifying their WASH needs and mobilizing demand for new services. However, there are differences in the Urban STBM approach compared with the Rural STBM approach, as well as the traditional CLTS method. For example, and to be consistent with broader GOI policy, Urban STBM requires that households access improved sanitation, as well as safely managed sanitation, and not just stop the practice of open defecation. The CLTS and GOI STBM rural approaches are also non-subsidy models.14 Under these approaches’ households, including poor households, are expected to address their own sanitation and water15 needs without outside financial assistance. Recognizing that B40 households in urban areas will likely require additional resources to address their water and sanitation needs, IUWASH PLUS’ Urban STBM approach encompasses more in-depth work with communities to link them to potential funding sources, such as LG sources and corporate social responsibility (CSR), along with activities under other components in the WASH ecosystem discussed above. Thus, the IUWASH PLUS approach is very much a systems approach. The IUWASH PLUS project’s approach recognizes that B40 households in urban areas are likely to require outside resources to address their water and sanitation needs. To this end, beyond working to link households to self-financing schemes (i.e., microfinance institutions, or MFIs), IUWASH PLUS has also advocated for the inclusion of LG “smart subsidies” and government financing for households into the Urban STBM approach. Finally, the project includes a sanitation marketing component that is also linked and working to increase access to sanitation services (see EQ3). It is important to note that, while the project has been implementing STBM in urban areas, the GOI Urban STBM Guide, and associated policies, have yet to be completed or adopted at the time of the MTE. IUWASH PLUS has been very involved in development of the draft guide (quarterly reports). However, the draft guide was not available to the MTE. As such, it was not possible to fully assess how the project is implementing the approach against GOI Urban STBM standards, or to fully identify the differences from the Rural STBM model. The IUWASH PLUS 12-step community engagement approach, includes “traditional” STBM methods, in addition to others, such as linking communities to local stakeholders who fund WASH services, it also includes a significant participatory M&E component (see below; also see Annex 7 for an overview of the Community Mobilization and the STBM process). Targeting Hotspots. A second key element of the project’s approach in reaching B40 is targeting Community Mobilization/STBM activities to communities with high concentrations of poor households, which the project has defined as “hotspots.” While the project is working in all 32 14 GOI Ministry of Health National Strategy for Community-Based Total Sanitation (CBTS/STBM), 2008. 15 Note CLTS is specifically a sanitation model, however STBM addresses safe drinking water. USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 6 districts and cities (“sites”) to increase access to water and sanitation services, community mobilization and Urban STBM activities are mainly being conducted in, or “targeted” to, 102 hotspots. The hotspots were identified as a result of the project planning process during PY1 (see further details below), with three to five hotspots in each site. The idea is that the hot spots “… will serve as models for connecting the marginalized and underserved to institutional WASH providers,”16 with the objective that the LG will replicate and implement the same approach in non￾hotspot areas on their own. (See Annex 7 for details on the steps of the community mobilization and participatory monitoring process.) Table 10 in Annex 9 describes the types of water and sanitation infrastructure or service connections that households are gaining access through the project’s activities, as well as how they obtain access to these services, for example, through technical assistance, project grants, LG funding, CSR, LSIC, self-financing, the sanitation market, MFI/loans, and individual grants, etc. Project Services Implementation: For water, improved services are identified as either having either “direct” or “indirect” attribution to the project.17 For example, each year a percentage of all new PDAM18 water connections among the 32 sites is indirectly attributed to the project as a result of the project’s technical assistance (TA) efforts to PDAMs. Direct attributions are a result of households gaining access to improved water sources through the project’s direct interventions, such as forming a KSM (kelompok swadaya masyarakat, or community-based organization) to support the implementation of a master water meter, or through leveraged government or CSR or donor financing, LSIC funds, etc. For sanitation, new services are being captured by hotspot and non￾hotspot areas, and funding mechanisms are similar to direct water interventions. 4.2 FINDINGS PROJECT ACHIEVEMENTS: INTERMEDIATE-LEVEL RESULTS Project Water and Sanitation, and B40 Target MTE Performance Data Analysis Methods. As the MTE was conducted seven months into PY3, it was important to look at more recent data to assess overall progress toward its B40 targets. To this end, the MTE worked to obtain the most recent project data. Measuring progress toward targets during the project period generates several challenges in the project design. These challenges include: • PDAM data are provided only once a year and have to be audited to confirm the name and addresses of those connected, to enable B40 surveys to be conducted; • Data from non-PDAM water services, as well as all sanitation services, rely on project partners, who do not always report the data to IUWASH PLUS at the point that service infrastructure or service connection are completed; and • B40 wealth survey data for HR1b and C1a are reported to USAID annually only, with surveys conducted on a rolling basis “when there is significant or no more progress made in a city/district (especially for non-hotspot areas) or late in the year in August in preparation for the annual report” (IUWASH PLUS staff). 16 IUWASH PLUS APR PY2 – QPR09. 17 IUWASH PLUS AMEP Vs 2.0 April 2018, page 9. 18 Perusahaan Daerah Air Minum/Municipal Drinking Water Company. 7 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV As PY3 data were still in the process of being collected at the time of the MTE, projections were based on availability of new data provided to the MTE through April 2019 for PY3 (seven months), including known and planned sanitation service projects, non-audited PDAM data, and more recently collected B40 survey data in eight hotspots for sanitation. To project the status of PY3 B40 results, the project used both completed known B40 survey data and projections.19 This uses past data as a predictor for PY3, which may not be accurate, but does provide one method to assess the current status, given the lack of more current data. Tables and figures from this analysis can be found in Annex 10. Project B40 Verification Tracking Method. To track whether the project reaches B40 populations, IUWASH PLUS conducts surveys in both hotspot and non-hotspot communities to verify the wealth status of household water and sanitation beneficiaries. The project uses the Demographic and Health Surveys (DHS) household wealth index method supported by USAID20 to verify poverty status. The wealth index questions21 have been incorporated into the mobile phone platform mWater—an open-access system specifically designed for water projects that the project selected from among other mobile-phone methods. B40 verification process surveys use two sampling approaches:22 (i) purposeful “census” surveys of all houses that are “tightly clustered” and have benefited from services, for example, from a master meter or an IPAL (these surveys are to be conducted on a “rolling basis” after households receive services); and (ii) statistically representative sample surveys among households that are “widely dispersed”, such as beneficiaries receiving water services from a PDAM through indirect attribution. (The sample surveys are to be conducted only twice during the project period.) According to AMEP, beneficiaries will only be surveyed once over the life of the project, with sampling surveying to be done just prior to the MTE and prior to the project closing. The project has no other method for verifying or capturing B40 data of served households at this time. Overall Achievements in WASH Access. Project data show that, as of PY2 (September 2018), the project was falling significantly short in achieving its life-of-project B40 targets for basic and shared sanitation services. Most of these services were obtained in one year during PY2, as PY1 was focused on planning and securing MOUs with LGs and other partners. Figure1 shows the cumulative result of B40 individuals gaining access to these services as of September 2018 (PY2), with only 7 percent progress toward the target for access to shared and basic sanitation services and 11.4 percent progress toward improved water service access for B40 populations. 19 To determine the status of PY3 B40 results, the project used a combination of both completed B40 connections/survey data (as of April 2019), e.g., direct projects where B40 surveys have been completed and projections. For example, to determine the percentage of B40 for water with PDAM connections for PY3, the team looked at non-audited PDAM (see Methods section for definition) totals for both direct and indirect connections to cities/districts provided to the project by PDAM, then applied PY2 percentage of poor to PY3 household data. 20 http://dhsprogram.com/topics/wealth-index/ 21 The “B40 survey” also incorporates other questions, e.g., house condition and whether or not the household participates in any social protection programs and current access to water and sanitation using JMP definitions. Thus, this method provides another source of data to which the project could assess and validate access to water and sanitation services. 22 IUWASH AMEP Versions 2.0, Annex A and B, pages 73-76. USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 8 Please refer to Annex 10 for tables and figures referenced in this section. WATER ACCESS Overall and B40 Target Results. Based on projections for new water services for PY3 through April 2019, the project is behind in meetings its life-of-project target (55.57 percent off target for all persons). The progress rate indicates a decline in service connection achievements from PY2 compared with projected PY3, but an increase in B40 access. In PY2, 236,035 people obtained access to safely managed drinking water, 11.4 percent of whom were B40, while the projected number for PY3 is 211,584 persons, 23.6 percent of whom are projected to be B40. Table 11 in Annex 10 shows cumulative results as of April 2019 based on actual PY2 reported results and projected (or estimated) results expected for PY3 against PY3 targets for access to improved water services. The estimated results are that a total of close to 107,000 poor (B40) persons in 21,389 households will have obtained access to water services by the end of PY3. Based on these projections, the project is only 7.98 percent short of the estimated target for PY3, but 78.61 percent off from its overall target for the project: 500,000 B40 individuals having new water access, with 35 months remaining in the project including the recent six-month contract extension. (See Table 11 in Annex 10 for complete PY2 and PY3 projected aggregate data for water access) Most of the achievements (74 percent) are attributed to indirect connections to PDAMs based on available data. Overall greatest achievements in direct water interventions are seen in six sites (Kab Deli Serdang, Bogor, Magelang, Kab Sidoarjo and Kab Bantaeng), with Central Java having more B40 coverage among cities/districts than other regions. However, 60 percent of the cities/districts have no direct water interventions reaching B40 individuals, based on the current available data (Figure 9 in Annex 10). 57,010 17,520 24,915 0 100,000 200,000 300,000 400,000 500,000 600,000 Access to improved water service quality (HR1b) Access to shared and basic sanitation (C1a) Access to safely managed sanitation (HR2) B40 Access to Water and Sanitatio Services (Cumulative as of September 2018) PY2 Cumulative Result Life of Project Target FIGURE 1: ACCESS TO WATER AND SANITATION SERVICE 9 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV B40 Water Access Target Attainment Variance by City/District. Among the areas where KIIs were conducted,23 cities/districts are either achieving, or close to achieving, their B40 targets. The exceptions to this are Surakarta and Surabaya.24 KIs in Surakarta indicated that their challenge is loss of PDAM customers due to population decreases, as well as households building their own wells and disconnecting from PDAM. This affects the ability to bring in revenue and expand services. They also noted that they lacked new water sources to draw from. While the project has no MOU with Surabaya, overall water coverage there is reported by PDAM to be at 98 percent and expected to rise to 100 percent in 2019. PDAMs also have policies/tariffs in support of the poor and are working with IUWASH PLUS to implement master meters. All of the Pokja KIs at the visited sites discussed the project’s efforts to improve PDAM operations as significant in their ability to provide increased water services overall. Based on analysis of KIIs and comparison of B40 results, the sites visited that are close to achieving, or have achieved, their B40 targets, either have: (i) some form of water tariff scheme or discounts for poor households, or other GOI grants that target some poor households; and/or (ii) they are fairly functional, based on the PDAM Index. Of the three sites that look to be furthest from their targets, Surakarta and Barru have the lowest PDAM index scores among the sites visited (42 and 25.5, respectfully); Surakarta, as per KIs, has no current pro-poor funding or tariff schemes in place, and Barru only established one in 2018.25 The other, Surabaya, has overall high coverage rates, a relatively high PDAM index score (67.5, fourth-highest among the 11 sites visited) and pro-poor policies, and is working with IUWASH PLUS to establish master meters for poor households. It appears that Surabaya’s gap-to-target may in part be due to the project’s B40 data not being up to date.26 (See Annex 15 for a summary of all KI sites visited, B40 target status and high-level review of their approach to water services to B40 populations). B40 Water Access Target Attainment by Direct Versus Indirect Services. Direct interventions were about 34 percent more effective in reaching the poor, but they do so on a much smaller scale. The data indicate that the majority of indirect connections are being made to non-B40 households. (See additional information and Figure 10 in Annex 10) B40 Attainment by Type of Water Intervention. Overall, connections related to direct IUWASH intervention (PDAM Water Hibah grants, community-based water supply systems and community water meters) constitute 25.85 percent of all B40 new water service connections. Figure 11 in Annex 10 shows total new B40 water services projected for PY3 by type of service connection. Most direct interventions are from connections to PDAMs resulting from individual household Water Hibah grants. B40 figures are expected to rise, in particular for master meters27 for PY3, as B40 surveys have yet to be conducted. 23 Project data analysis had yet to be conducted prior to selecting site visit locations; while no bias was known in selecting sites, the evaluation team did miss areas that were further off their PY3 targets. 24 This may due, in part, to the lack of a MOU with Surabaya and no B40 surveys conducted in these areas. 25 IUWASH PLUS PY2 Report Annex page 34. 26 PDAM director estimates that 10,000 households do not have access to water and 40 percent (4,500 HH) of them are poor. 27 MTE site visits included visits to two completed project supported master meters that have yet to be included in project M&E data or have B40 surveys completed. USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 10 SANITATION ACCESS Overall and B40 Targets Results. Based on projections through PY3,28 the project is also behind in meeting its overall revised project targets for basic and shared sanitation, with 93,740 people projected to have new access to basic or shared sanitation as of April 2019, but only 45,941 are projected to be B40 (49.01 percent). This represents 18.38 percent of the life-of-project target and 11.37 percent of the PY3 target with 35 months remaining in the project (Table 12 in Annex 10). B40 Attainment by Hotspots and Non-Hotspots. Unlike for water services, access appears to be increasing for sanitation services from PY2 to PY3. However, most of the increase in access is seen in non-hotspot areas and only a little over half of them are identified as being B40. Overall, based on available known and projected data to date, 96.2 percent of all new basic and shared sanitation access are in non-hotspot areas. The percentage of B40 is anticipated to change for PY3, according to IUWASH PLUS staff, as the project is still in the process of conducting B40 surveys for the year.29 In addition, staff reported that they expect that their community mobilization/Urban STBM activities in hotspot areas, which are still underway (see Findings Section), will result in the construction of more sanitation facilities in the years ahead, particularly in hotspots, as the stakeholder engagement step of these processes become more complete. There is some evidence that this may be accurate. Based on the eight cities/districts that have reported B40 households from B40 verification surveys30 with new sanitation access, five have completed all cycles of the community mobilization/Urban STBM process, including the culminating stakeholder meeting. These five hotspots account for 71.6 percent of all new sanitation access among the hotspots. Another site, Kota Bogor, is the location of the initial sanitation marketing strategy pilot. It is not known if the B40 households that obtained access here resulted from the marketing strategy. (See further discussion below in Findings regarding the hotspot targeting approach.) Looking at projected results and targets for PY3 by region, only East Java will have achieved its B40 results for PY3 based on current projections, exceeding its target by about 318 households (about 1,590 individuals). However, only 125 households will have gained access in hotspot areas. The other remaining regions are significantly behind in their PY3 targets based on currently available data (Figure 12 in Annex 10). As with water, these figures will need to be updated for PY3 when B40 surveys are completed. B40 Attainment by Type of Sanitation Facility. Most households gaining access to shared and basic sanitation are a result of individual household sanitation facilities (55.9 percent), followed by communal systems (IPALs, 37.9 percent; Figure 13 in Annex 10). In hotspot areas, IPAL communal systems31 are reaching significantly more B40 persons than in non-hot spots areas. It is not fully 28 Projections include actual B40 surveys conducted in some hotspots as of April 2019 (eight surveys completed in hotspots) and projection of B40 PY2 survey percentage onto PY3 projected results for non-hotspots. For non-hotspots, if a percentage for PY2 was not available for a specific Kota/Kab site, the regional B40 was used. In almost all cases, percentages from the same type of system were used if available. 29 As of April 2019, it appears that only 10 B40 surveys have been conducted among hotspot areas’ sanitation systems constructed in both PY2 and PY3. 30 B40 surveys are still on-going and not completed for all sites for services that have been completed by partners. 31 IPALs are “offsite domestic wastewater treatment systems” categorized as SPALDTs (see Annex 21 for project figure on types of urban domestic wastewater management systems). It’s not known how many IPALs this represents, as data 11 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV known why more of these have not been built in hotspot areas. Given that hotspots are to be areas of high concentrations of poor, the cost to build an IPAL, compared with an individual septic tank per household, and the lack of fiscal resources, is one of the key reasons. Most funds leveraged by the project in PY2 have been for IPAL systems. However, the data are not provided by hotspot or non-hotspot (APY2 Annex 34). A review of the available percentage of B40’s by type of system and hotspot versus non-hotspot areas found that sanitation systems in hotspot areas generally have a higher percentage of poor than non-hotspot areas (Figure 13 in Annex 10). Similar to the project’s water “direct interventions,” this supports the notion that the project is targeting the right areas with the right interventions to reach B40 populations. However, resource gaps or other issues (see more below in Findings under project approach) appear to be inhibiting the implementation of more interventions in hotspots. 4.3 FINDINGS: SYSTEM STRENGTHENING EXTENT THAT PROJECT APPROACHES CONTRIBUTE TO REACHING B40 Section 4.2 focuses on project performance related to the degree that B40 targets are being met, as well as what service interventions are reaching B40 most effectively. This section focuses on the extent to which the project’s approach (the approach elements are described in Section 4.1) is effective in reaching B40 populations with WASH services. • The IUWASH PLUS approach is increasing LG capacity, but has not yet necessarily led to increased services for the poor. As discussed, the project’s urban WASH Ecosystem approach assumes that efforts to improve LG institutions’ capacity and efforts to create a more effective enabling environment will contribute to increased access for all households, including B40 individuals being served. However, evidence suggests that, according to KIIs among Pokja AMPL and individual LG KIs interviewed, IUWASH PLUS has supported the creation of an improved enabling environment in a wide variety of ways, including the following specifically mentioned by KIs:32 • Support and institutional planning capacity building for Pokja AMPLs. • Support for new or revised LG policies and regulations, including new tariff schemes (see EQ4) and regulations that support safely managed sanitation, e.g., requirements for watertight septic tanks (see EQ3). • Building technical capacity of wastewater treatment facilities (UPTDs33). • Building technical capacity of water utilities/suppliers (PDAM and public works). • Building technical capacity related to: construction of various sanitation models (communal/IPAL and household septic tanks); behavior change/STBM urban community mobilization processes and community-based organizations (KSMs) to enable them to indicates HHs only. IPALs are reported to range from 11 to 110 HH per IPAL, with one IUWASH WASH facilitator’s reported numbers indicating 53 HHs per IPAL in his area. Note the number provided during this KII were not in alignment with reported M&E data for that community/region. 32 This list categories where there were identified by at least 2 to 3 KIs, findings were triangulated against project reports and with interviews with staff. In the field. 33 Unit Pelaksana Teknis Daerah, regional technical implementing unit. USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 12 implement and support community water and sanitation systems, e.g., master meters and IPALs. There is information indicating that some capacity building and TA efforts are working to increase services to the poor. For example, among the most significant areas where tariff schemes have been supported, some have reduced fees or developed longer payment periods for poor households, for example UPTD desludging in Makassar. Another promising example is the project’s work with communities and LGs to promote communal systems (e.g., IPALs for sanitation services and master meters for water services with PDAMs). Here the project appears to have been instrumental in working to identify CSR and LG funding for those systems. Strategy for WASH financing requirements to ensure that the poor are served There was also considerable variance among KIs as to whether LGs or PDAMs should target the poor (see quote) (See Annex 15 for a summary of MTE KII sites and high-level comparison of their B40 attainment targets against KIIs’ indication of whether or not they have an explicit pro-poor B40 approach). It is acknowledged by staff that it is a “sensitive” matter in Indonesia to delineate the “poor” in such agreements. However, not including more specific requirements may inhibit the IUWASH PLUS project’s ability to meet its B40 targets. Project activities and LG budgeting processes could be better aligned, and financing for water and sanitation services for B40 is insufficient. Key components of the WASH Ecosystem are focused on household and sector financing for WASH services. For the community mobilization/ Urban STBM and hotspot approach to result in new services being implemented, they must also be in line with LG and GOI budgeting processes. All Pokja AMPL KIIs indicated that IUWASH PLUS has assisted greatly in the planning and preparation of budgets for specific intervention projects being implemented by the LG. However, especially in its initial years of operation, there was often misalignment between the project’s community mobilization planning processes in hotspots and the LG budgeting/Musrembang processes for WASH service improvements. The issue of timing appears to have been recognized by some of the regions, for example: “The participatory monitoring activities in all cities have reached stage 5 out of a total of 6 stages. This is a significant moment since the 6th stage will allow community members to propose the monitoring results in the development planning meeting at sub-district level so as to be budgeted in the 2020 APBD. When the Receiving GIS TA and capacity building has helped PDAM to efficiently map out our customers and expedite the service to others because we now know where the needs are. The ability to identify needs has gone from weeks to days due to GIS maps. This has also helped to expedite allocation of money to poor HHs. (Probolingo PDAM Pokja KI.) We felt assistance was extraordinary especially in wastewater management (capacity building and other skills). We didn’t have much background in this before. (and) In our agency, IUWASH PLUS has completed institutional mentoring in wastewater training, operations and maintenance. (and) training in safe septic tanks (water tight = national standard). Before that we had no idea what a safe septic tank was before IUWASH PLUS. (Makassar UPDT staff KIs.) There is an argument that “if you do (serve or target) everyone you will be serving (the health) of everyone (including the poor), so no you don’t have to target or you won’t serve everyone. (Housing and Settlement Pokja AMPL KI) 13 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV APBD is not able to finance the proposed monitoring results, they can propose to receive CSR funds.” (Regional Progress, North Sumatra, QRT 10 Report, 2019, page 66.). Beyond initial misalignment in budget processes, information was lacking to determine how hotspots are determining what it will cost to meet their identified needs, or if they are doing this at all. While the MTE team received data on the status of the Community Mobilization/Urban STBM processes in each of the hotspots, no significant data were provided that systematically captured the findings from these processes to provide an overall picture of the financial resources required to meet the identified needs from the participatory M&E processes in the hotspots. Project reports, for example, the most recent Quarterly Report 10, provide summative information on each region’s progress and plans for WASH services, but very few delineate activities or plans in hotspots vs. non-hotspots. IUWASH PLUS reports, including quarterly reports, contained mostly anecdotal budget information for various efforts, or a list of LG budget figures or summative information. Other than looking to LG resources, hotspots could also look to special allocation grants, namely, DAK and Hibah, as resources for funding. However, these are “relatively modest in relation to LG financing” and “cover a fraction of the eligible population” only (World Bank, 2017, p. 58 and 60). Furthermore, Pokja AMPL KIs indicated that not all LGs have access to these resources, given the complex requirements to obtain them, or have not prioritized them as funding sources to pursue from one year to the next. In addition, a review of 2018 LG budget allocations among the 31 targeted areas (not including Jakarta) indicated that 45 percent had seen budget decreases in relative terms for the WASH sector from the prior year (2017) (IUWASH PLUS Annual Progress Report 2, Annex 33). This suggests that IUWASH PLUS has not been successful in mobilizing LG commitments to increase investment in WASH services in some areas. On the positive side, IUWASH PLUS reports that it has been successful in facilitating access to financing for 26 projects34 during PY2 through national (APBN) and local government (APBD) budgets (IUWASH PLUS APR PY2-QR09 report, Exhibit 30, pages 49 to 52). However, IUWASH PLUS staff indicated that many of the projects supported by the project during PY2 were already budgeted by the LG; with IUWASH PLUS assisting in implementation of the already planned systems, including socialization of community members and training (see also EQ2). Of these 26 systems, the majority were still under construction in September 2018. Unfortunately, the PY2 annual report included no information to indicate which if any of these systems are being implemented in hotspots to assess the association of these systems with the project’s hotspot and Community Mobilization/Urban STBM approaches. In addition, B40 surveys had yet to be conducted for most to determine the degree that these served the poor. For needs to be met, current LG, including PDAM, resources are likely to be insufficient, according to KIIs (Pokja AMPL KII notes) and staff. To meet the gaps in government funding, both staff and KIs indicated that they are looking to IUWASH PLUS to help facilitate linkages to CSR funds, or they will have to make these linkages on their own. However, it is unlikely that CSR and other resources (international donors) will be sufficient to meet potentially significant budget needs in the targeted areas during the life of the project (see analysis below specific to sanitation). 34 As of the October 2018 of the 26 systems included 22 where construction is still in process. USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 14 Barriers to funding and connecting B40 to sanitation services. Based on the current B40 target for basic and shared sanitation facilities, the fiscal budget required is significant. For example, if the data projected for PY3 are correct, more than 40,00035 B40 households will need to gain access to a basic or shared sanitation facility over the next two years to meet this IUWASH PLUS target (see Box 1). This does not take into consideration the household contribution or MFI support or activities under the project’s supported sanitation marketing strategy. However, as Pokja AMPL and LGs, as well as hotspot KIs, noted, these loans are limited, and it can take more than a year for households to borrow a sufficient amount from MFIs to build a septic tank (see also EQ3). While households could build facilities that do not include septic tanks to end open defecation practices, these types of facilities are not supported by broader GOI urban sanitation policy. Furthermore, in many urban communities in Indonesia the poor often live along rivers/canals, where basic pit latrines cannot be built due to high water tables, posing a risk of potential leaching of waste into nearby rivers. Finally, IUWASH PLUS formative research and literature for Indonesia indicate that households aspire to obtain septic tank facilities from the outset, rather than working up the “sanitation ladder” over time.36 BOX 1. TOILET FACILITY COST ESTIMATES The MTE used available estimated costs to construct one household toilet from various sources in US dollars giving ranges from $104 to $207 (MTE KII household estimated ranges) to $119 (World Bank, 2017; cited average cost for typical household toilet in 2011) and a high of $214-$357 (IUWASH PLUS budget figure for septic tank toilets) to assess the total approximate budget needed to construct sanitation services to meet remaining project targets. Based on the estimated costs, the total cost of 40,000 toilets would range from $4.16 million (based on a KI low-end estimate of $104 per basic toilet) to $37 million for (based on high end budget estimate of IUWASH PLUS septic tank toilet budget of $928). Communal IPALs cost more per household and are not included in these estimates. Another barrier in hotspots (which are often in densely populated areas) is the frequent lack of space to build IPALs. In particular, KIs pointed out that there had been a misalignment with GOI policies, where these centralized communal sanitation systems had to comprise at least 50 households prior to construction, which was prohibitive given space limitations for such a large system. The project worked to support regulations enabling the construction of smaller systems (SPALDS and SPALDT) to allow for fewer household connections—a promising policy change (WASH facilitators and regional project staff KIIs). Of the two IPALs visited during site visits, both served a mix of poor and non-poor households. According to one local leader KI interviewed, this was critical in having the IPAL constructed (in terms of affordability), but also a challenge, as some non-poor households were reluctant to be part of the system because they preferred to have their own septic tank. While IPALs serve more households, the process to have them constructed takes longer than individual household septic tanks, and is much more intensive, requiring advocacy, education and coordination efforts at both the community and LG institutional levels. One of the IPALs visited was reported by staff to have taken more than one year to construct following the 35 A total of 41,050 HH sanitation facilities were reported to be built under IUWASH, per the Final Evaluation Report (April 2016); (23,500 improved sanitation facilities and 17,500 in Central Java as a result of co-financing with PNPM Mandiri) over the life of the five-year project. 36 Hirai, M, Kelsey, A and Mattson, K, et. al (2012) “Determinants of toilet ownership among rural households in six eastern districts of Indonesia.” Journal of Water, Sanitation and Hygiene for Development, (8), 3. 15 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV completion of Phase 1 of the Sustainability Checklist (see EQ2). The project’s efforts to support the construction of these facilities, including the establishment and training of local KSMs to manage and maintain the facilities, was identified by KIIs and Pokja AMPL and community beneficiaries as being instrumental to these facilities’ successful implementation. Barriers to funding and connecting B40 to water services. One reason why more direct water interventions, including communal systems such as master meters, are not being implemented, according to Pokja AMPL, including PDAM KIs, is that they require targeted budgets to implement services, such as central or LG funds, Water Hibah grants or DAK37 assistance, and these funds are limited as noted above. The project collects a significant amount of information on TA to water service providers, namely PDAM, and their direct interventions (quarterly and annual reports). However, based on information available to the MTE, it does not appear to capture: (i) data on planned water projects; (ii) the estimated number of B40 persons who would be served among those projects; and (iii) the budget required to address those needs or a plan to do so. Such information would provide a better indication of whether or not the approaches being deployed to reach B40 targets are attainable by the end of the project. Among the reasons for the low number of B40 individuals being served through indirect PDAM connections and other direct interventions, as identified by MTE Pokja AMPL/PDAM KIs, are the following: • Most PDAM KIs and some GOI KIs indicated that they do not focus on serving the poor. Many KIs indicated that they serve all households and do not consider or do not target B40 populations or are doing so on a limited basis with limited funding schemes. • Insufficient, or a high annual variance, in pro-poor subsidies, e.g., Water Hibah or DAK funds,38 to enable the poor to connect to existing PDAM systems, e.g., master water meters, or to develop other systems, e.g., through public works. • Underperforming PDAMs, which results in a lack of resources to expand networks to households, inhibiting the ability to provide financing schemes to poor households. • The poor live in areas where the main water lines of PDAM do not exist or cannot be expanded due to costs or physical construction challenges. • The poor live in “illegal housing” that the PDAM cannot connect to by law. • Overall lack of availability of viable raw water sources located near poor households or insufficient resources on the part of PDAM or public works to construct systems that would bring water from available resources to poor households. • Reports from some KIs that poor households are not interested in connecting to PDAM for a variety of reasons, including concerns over connection costs, quality and reliability. KIs from the Pokja indicated that IUWASH PLUS efforts have addressed some of these barriers in their TA, outreach and advocacy efforts. For example, they have worked to support the creation of Community Based Organizations (KSMs) and have linked KSMs to support the construction of master water meters. They have also used IUWASH PLUS project funds through LSIC (Component 37 Dana Alokasi Khusus, or special allocation fund. 38 LGs and PDAMs need to have met performance standards before they are eligible for DAK assistance, with which Pokja/PDAM KIs say IUWASH PLUS is assisting them. USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 16 2 activities) to support, via subcontracts, the implementation of master meters, especially in Surabaya, where this partnership (according to community members and PDAM director) has proven particularly successful (see Box 2).39 Several Pokja KIs, including PDAM participants, indicated that IUWASH PLUS’ role in working with poor communities to convince them to connect to PDAM has also been instrumental in encouraging households to connect, as most are initially reluctant to do so. These efforts (e.g., TA, LSIC and community mobilization) have enabled poor households, sometimes in illegal areas, to connect to PDAM main lines, and appear to be a particularly promising approach for reaching poor households with WASH services. BOX 2. MASTER METER (MM) KI CASE EXAMPLE In Surabaya, household master meter beneficiaries living in “illegal housing” indicated that the project was instrumental in their decision to connect to a MM. They noted that they had heard similar offers before, without the option of a grant, and were not interested. Instead, they had been relying on wells (non-potable) and water vendors (for drinking water). They boil PDAM drinking water to make it safe to drink. They reported they were now spending about IDR 26,000 ($1.82) a week on water per household for all water needs, whereas before they had been spending IDR 51,000 ($3.57) a week per household. They also reported significant electricity cost savings due to reduced usage of pumps they had relied on for non-potable water. The PDAM managing director of Surabaya noted that the project is “Very useful to address the poor, as there are 10,000 households who do not have access to water. Partly because of lack of legality, but also lack of capability to pay, so PDAM provides a discount for connections. …Throughout the collaboration with IUWASH PLUS, PDAM have provided 100 MM of which 70 percent PDAM responsible and fund the individual connections through PPP (public, private partnership) and the other 30 percent is facilitated by IUWASH PLUS.” The Community Mobilization/Urban STBM and hotspot approaches are effectively triggering and mobilizing the poor, but more time is needed to assess the approach and improvements are still needed to ensure activities lead to increased service access for the poor. As discussed, the project’s approach is that the Community Mobilization/Urban STBM processes will also contribute to, along with the other project’s activities40 and available financing, increased service access. This will come in part as a result of LGs listening to and responding to community needs with dedicated LG resources following the cumulating stakeholder meeting, as well as linkages to other household financing schemes.41 IUWASH PLUS project staff indicated that the stakeholder meetings provide particularly important opportunities to link communities with potential funders, including CSR.42 They noted that as these meetings had only recently been completed, they expect that they will more than likely bear fruit as time goes on, and as more 39 Local Sustainability and Innovation Component (LSIC) “a tool to provide targeted technical and capacity building to finance or co-finance activities to strengthen community, the private sector and government WASH Service provision.” Schemes have been used or will be used to support implementation of master meters in Surabaya and small shared communal septic tanks in Magelang. (IUASH PLUS Work Plan PY 2 and PY 3), As this was a Component 2 activity the MTE did not evaluate LSIC activities specifically. 40 Including activities under other components discussed previously as well as sanitation marketing efforts. 41 This is the sixth stage of the participator M&E cycle. The project indicates a possibility of multiple stakeholder meetings. 42 One example discussed by the IUWASH PLUS CoP included verbalized commitment from a bank to support construction of toilets in the Sumatra at a Stakeholder meeting. The MTE was only able to attend 1 Stakeholder meeting in which budgets and financing of potential projects were not discussed at that meeting. 17 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV meetings are held. As of April 2019, the majority of 102 hotspots had completed all the steps of the process. In total, 63 hotspots (61.7 percent) have held stakeholder meetings, with all held in PY3 between the months of February and April 2019 (30 percent were held in April) (Table 1) Total projected sanitation connections for PY343 appear to have a correlation with hotspots that have completed stakeholder meetings, though data are limited and also do not include the timing of stakeholder meetings and implementation of new services (Figure 244). As water access data are not captured by hotspots in the data provided to the MTE, it is not possible to assess linkages of increased water access to these activities. However, Figure 2 provides a summary of new water access that is not attributed to PDAM indirectly by region. During the two MTE site visits to hotspots, the MTE team saw that both had obtained increased access to water and/or sanitation services (see Box 3). Stakeholder meetings for both of these had occurred in February 2019. Community members in the second KII hotspot were very interested in improving their sanitation facilities but lacked affordable options and/or workable financing schemes. In addition, they also thought that smaller, shared septic tanks for up to five households would be a better option for them, both to reduce the cost to households, as well as address the space limitations of larger IPLT systems. 43 Based on project provided sanitation data as of April 2019 used to prepare figures under Project Achievements. 44 This data is derived from Table 1 and data used for Figure 9 and Figure 10 in Annex 10. FIGURE 2: COMPARISON OF COMPLETION OF STAKEHOLDER MEETINGS IN HOTSPOTS WITH B40 SANITATION ACCESS (HOTSPOT) AND B40 DIRECT WATER ACCESS (REGION) USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 18 BOX 3. HOTSPOT MTE SITE VISITS Semanggi, a hotspot in Surakarta, has conducted all stages of the STBM process and implemented an IPAL (SPALD-T). However, the relationship between the planning processes and the construction of the IPAL is not known.45 A B40 survey for Surakarta indicates that 65.22 percent of people at that site were B40 individuals.46 A master water meter was also reported to have been constructed in this community, but a B40 survey had not been completed for that intervention. Kayu Bongkok, a hotspot in Tangerang, has also conducted all 12 stages of the process and self￾reported new toilets had been built. However, as this site had received support from IUWASH, it was not clear what facilities IUWASH PLUS supported.47 In-depth discussions with community members at this KII, including kader, a sanitarian and a sanitation entrepreneur, indicated a high level of awareness about the importance of sanitation, with all indicating open defecation no longer occurred. However, KIs indicated that most houses had toilets or septic tanks that were not watertight and lacked sufficient funds to build watertight facilities. Those able to build new facilities had done so as a result of qualifying for LG grants for poor households. The participants talked in depth about the inadequacy of the MFI terms (either loan payments were seen as too high or it could take up to five years for one group loan scheme to provide enough money) to build a watertight septic tank (see also EQ3). Based on the data available and KIIs, the Community Mobilization/Urban STBM process appears to be effective in its ability to increase awareness of the need for improved sanitation facilities in particular. Among project beneficiaries that the MTE team met, it was apparent that they were being triggered beyond simply stopping the practice of open defecation toward acknowledging that safe watertight septic tanks were the desired and needed type of sanitation facility. Based on qualitative information, there appear to be multiple key factors that determine whether these processes result in the poor being reached with WASH services, specifically basic and shared sanitation in hotspots: 48 (i) linking households to financial resources (discussed above); and (ii) of the completion of stakeholder meetings of the Community Mobilization/Urban STBM process. If communities are to be empowered to advocate with their LGs to assist them to address their WASH needs, these planning and information-sharing mechanisms are needed. Finally, the Community Mobilization/Urban STBM process also includes assisting communities with the development of an action plan (Step 3 of Community Mobilization). Unfortunately, the MTE team had limited information on what the action plan is expected to contain, or how it is developed. That said, it was clear that improvements can be made in this activity. First, the timing of this step may be too soon in the overall Community Mobilization/Urban STBM process to include participatory M&E data, which happens later in the process, to inform the development of the plan for specific water and sanitation service needs. Second, the plans may lack sufficient specificity to be actionable. For example, in one of the hotspots visited, community members’ KIs were very clear about the need for improved watertight septic tanks. However, they did not have a clearly defined strategy on how 45 Some IUWASH PLUS staff noted that some projects were already underway/planned in some communities and were not necessarily linked to the community mobilization processes. 46 B40 survey data were provided to the MTE only by city or district, so it is not possible to link B40 data to a specific hotspot community. 47 Eleven individual sanitation facilities have been built in Tangerang hotspots; 54.55 percent of them serve B40 populations, but the data do not indicate if they are from this hotspot. However, all of Tangerang’s hotspots have completed the entire process. 48 Note new water access data was not available to the MTE by hot spots, only down to the kota/kab level. 19 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV the identified needs were going to be addressed, including how to finance or pay for the services needed. While this may be an anomaly, project reports also did not provide information to indicate that such detailed action plans were being developed. Unfortunately, as noted in the limitations, limited time was spent in hotspots talking with actual beneficiaries to explore this issue further. TABLE 1: HOTSPOT COMMUNITY PARTICIPATORY PLANNING/ STAKEHOLDER PROCESS STATUS (AS OF APRIL 2019)49 REGION PLANNED/ TARGET PHASE 1 COMPLETED* PHASE II STAKEHOLDER MEETING HELD % COMPLETING STAKEHOLDE R MEETING TARGET North Sumatra 19 6 11 58% WJDT 10 0 3 30% Central 21 4 21 100% East Java 19 3 6 32% SSEI 33 15 22 67% Total 102 28 63 61.8% *Of those not completed, all but Activity 6 (establishing citizen engagement mechanisms) have been completed, with only 27 of 102 hotspots (26%) having established a citizen engagement mechanism. Urban STBM is a new model for Indonesia and better staging of its implementation could have brought about improved results. STBM, similar to the WASH Ecosystem approach, may need a longer gestation period before it bears fruit. As a new model, the project may have needed more time to develop, and roll the process out, and/or test it in a few hotspots prior to implementing it in all 102 hotspots. In addition, there may be issues inherent to the application of the rural STBM model to the urban setting, where WASH issues are more complex and expensive to address (CLTS, 2016).50 Given these factors, the project may also have taken on too many hotspots, or could have phased the staging of them over a longer period of time. As there still are no GOI guidelines or methodology for the project’s Urban STBM, no blueprint was available to assess the project’s rollout. In addition, as noted above, few data were provided outside of summative information in PY2 and quarterly reports to assess when the project implemented each of the steps in the hotspots. In PY2, 95 hotspots were reported to have initiated the process, including the collection of baseline data, with only five starting the process of participatory M&E. Another factor that maybe inhibited the project’s ability to effectively serve B40 with WASH services is that its project resources may be spread too thinly with respect to the project’s sanitation work. For example, the project is focused on basic and shared sanitation for B40 households in hotspots using a new model (Urban STBM), while simultaneously supporting the 49 Dates for individual hotspot activities outside of the date the stakeholder meeting were held were not included in the data available to the MTE. 50 Lessons learned from workshop findings on “Using a CLTS Approach and Tools in Peri-Urban and Urban Environments” (June 2016) https://www.communityledtotalsanitation.org/blog/can-clts-work-urban-areas USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 20 development of a safely managed sanitation system and sanitation marketing strategy, all of which are significant undertakings on their own. The sanitation approach could be better articulated with respect to how it is working, to increase access to services for B40. In addition, how the project’s sanitation approaches relate to one another, specifically to increasing access to B40 populations, is not always clear, and/or not well communicated by the project, and possibly not well understood by all Pokjas/LGs or the GOI (see below and also EQ3). Complicating this further is that, while safely managed sanitation does not have a B40 project target, safely managed sanitation is the goal of the GOI when it comes to sanitation, and has received significant emphasis by the project, particularly under Component 2.51 The project’s overall approach to improving sanitation access is sound and based on best practice in the sector, as well as working on innovative approaches, namely Urban STBM. It is also seen that the project is working to improve the sanitation system within the targeted communities (see below regarding the Sanitation Index). However, more could be done to show how the individual project components relate to one another in the system and, more specifically, how they are working to address the needs of B40, where, and if, relevant. There appear to be considerable variance among Pokja’s KIIs in their focus on sanitation as a “system”. Two looked at had a more in-depth expressed knowledge and reported to have taken steps to develop a “white paper or master plan” on sanitation (Tangerang) and a “City Sanitation Strategy” (Surakarta), with both of these indicating that IUWASH PLUS had provided input on their plans, or assisted in developing them. Few, however, articulated how the various IUWASH PLUS sanitation components were working together to serve the poor. When the poor were discussed in relationship to sanitation, it was mostly related to tariffs for UPDT and associated regulations (which may or may not be focused on the poor, see EQ4), STBM or KOTAKU. Few mentioned sanitation marketing, even where it was being implemented, although this is understandable given its late implementation and the fact that it has only been rolled out in two of the three pilot areas. None of this is to say that the project is deficient, but simply that more could be done to show how the various aspects work together, and how the project is working to see that the poor are served by the various interventions/approaches. Given that the sanitation “system” in many ways is in its infancy in Indonesia, it is understandable that more work is needed to help articulate the system, and how IUWASH PLUS activities and components fit into it. As noted by the projects Sanitation Index,52 of the 32 targeted areas only 10 have an Index over 50 (the highest is 72.5 in Bekasi), with most Indexes ranging between a low of 12.0 and a high of 46.5). Thus, there is considerable room for improvement and IUWASH PLUS is supporting LGs to improve their Indexes. Finally, it is recognized that the project has taken on a great deal in sanitation (see EQ3), and that working on so many aspects all at once may be impeding its ability to achieve key B40 sanitation results, which were significantly off target at the time of the MTE. 51 It is noted again that the MTE did not focus on an evaluation of this component. 52 IUWASHPLUS APR PY2-QPR09 draft for review_22October2018, Annex 7 Sanitation Index, (Total possible score is 100) page 143 (Uncer Component 2). 21 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV 4.4 CONCLUSIONS Based on PY3 projected results as of April 2019, the project is not meeting its life-of-project B40 targets for either water or sanitation: • HR1a: 106,945 B40 individuals are projected to have obtained improved water quality, only 21.39 percent of the established target of 500,000 people; and • Cla: 45,941 B40 individuals are projected to have to have obtained basic or shared sanitation services, just 18.38 percent of the established target of 250,000 people. Neither the project’s direct interventions for water nor its activities directed toward hotspots have resulted in a significant number of B40 individuals being served to date. This may be partly due to the limited number of B40 surveys that have been completed, or insufficient M&E data on completed interventions that have yet to be reported to IUWASH PLUS from project partners at the time of the MTE. It may also mean that the targets for B40 were set too high in light of the available LG resources. It could also be that the project’s approach was not appropriately staged in collaboration with LG partners, given the number of new, and sometimes untested, approaches it was undertaking to address the needs of the poor. A variety of factors among the project’s approach have worked to serve B40 populations, with other factors identified as not working as effectively as intended, or as well as they could have. These are briefly summarized below: WATER • Overall, the main providers of water services in the targeted areas that the project is relying on (mostly PDAM) do not target the poor and the number of financing schemes is insufficient to enable the poor to connect to PDAM networks. • The poor often live in illegal housing that PDAMs cannot serve. While master meters are one option to connect B40 households in these areas to PDAMs, this requires both available financing and a willingness on the part of the poor to connect to PDAM infrastructure. Many were reported to be reluctant to do this, especially if subsidies are not available. • IUWASH PLUS has played a key role in convincing poor communities to connect to PDAMs when subsidies for the poor are available. The use of LSIC funds to support the Master Meter process has demonstrated that the IUWASH PLUS community engagement and financing approach is effective in supporting the development of new water connections in difficult-to￾reach communities. • Insufficient capacity of PDAMs to expand their networks, combined with overall low operational capacity, means that many PDAMs do not have the ability to extend subsidies to the poor. The project is working to strengthen PDAM capacity in a variety of ways. Under its ecosystem approach, improving the capacity of a water utility can be expected to take time, but IUWASH PLUS has a process to measure institutional capacity progress, and the success of these efforts should become clearer over the coming year or so. SANITATION • How the project’s sanitation approaches relate to one another, specifically to increasing access to B40 populations, is not always clear and/or not well communicated by the project, and possibly not well understood by all Pokja’s/LGs. • There appears to be significant fiscal (LG) and financing (MFI, loans, etc.) gaps that may prevent the project from realizing its B40 sanitation targets before the end of the project. This is due to both what appears to be insufficient LG resources for poor household water and sanitation infrastructure service support, and insufficient and limited financing schemes, e.g., MFIs, to meet USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 22 the needs of the poor among the targeted hotspot areas specific to sanitation interventions. The latter in part is associated with delays in the implementation of the sanitation marketing strategy (see EQ3). • It appears that there was an initial misalignment between IUWASH and LG budgeting processes, but over time the alignment between IUWASH PLUS efforts and local budget cycles is not expected to be an issue. With most who have conducted the initial stakeholder meeting doing so in the 2nd and 3rd quarters of PY3, the 2020 GOI budget cycle is almost completed. The project’s position is that it may be too early to determine if the community mobilization approach is effective in contributing to increased access for B40 populations, because activities are still underway. However, given the current status, it is clear that more emphasis needs to be placed both on securing resources to meet the significant needs, regardless of the fiscal formula or combination of resources to meet the needs identified in hotspots (e.g., such as Water Hibah grants, LG resources, DAK, CRS, household self-funding and/or MFIs, etc.). • The B40 targets for basic and shared sanitation in urban settings, where high concentrations of the poor live (hotspots), may be unrealistic for the remaining project timeframe. Sanitation issues in urban settings are more complex and also often more expensive to address. This, combined with a lack of sufficient poor-inclusive financing to support the construction of facilities for poor communities and insufficient viable financing options, e.g., MFIs, for poor households to construct their own facilities (see also EQ3) and a new Urban STBM approach in 102 hotspots, may be too large a target for LGs to take on in such a short time. • Urban STBM is still being tested in urban areas globally, and the design and guidelines were still being developed in Indonesia at that point of the MTE. While the model has been adopted by the GOI as its urban WASH approach, it is still in its infancy. More time will be needed to further test and refine this approach, but useful learns learned should emerge from this process. TARGETING B40 • The project appears to lack monitoring tools to enable the measurement or analysis of progress toward its B40 targets at any given time, or at other regular intervals, e.g., a quarterly or semi￾quarterly basis. Having to wait until the end of the year to measure progress toward B40 targets is inadequate as a program monitoring tool. Additional approaches need to be considered, or more frequent B40 surveys need to be undertaken, especially for sanitation, to measure progress toward targets. Failing this, there needs to be recognition that this is a pilot effort to determine if B40 targeting will result in more B40 being served and use end-of-project verification results to test the overall hypothesis, rather than use B40 verification as a means to monitor project result attainment. • USAID required the project to develop a targeting strategy specific to serving B40 households. The approach it used appears sound and is seen as helpful to LGs. To verify if the project’s beneficiaries among the sites selected are serving the B40 population, the project implemented a verification process. Past projects funded by USAID and other donors have often not taken, or been required to take, this extra step of validating whether served populations are indeed poor. Instead, they have used proxies such as primary location-based data to determine poverty. This project’s efforts have provided an opportunity to test targeting methods to determine whether the poor are actually being served by its interventions. This is an important issue for the WASH sector, particularly in urban areas. • To date, it appears that the number of B40 individuals is smaller than the targeting approach envisioned. However, as discussed, data may be insufficient to make this determination. Furthermore: o The delayed fruition of activities (e.g., stakeholder connections with LGs) may have been incompatible with project targets within the project’s timeline from the outset. The time 23 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV required for project start-up and activity adoption may also mean that progress is likely to accelerate as the project matures. The degree to which this happens will provide a useful gauge as to the success of IUWASH PLUS’ approach. o For some project interventions, such as IPAL, the systems are constructed in areas that have both poor and non-poor residents, as identified by the project; both poor and non￾poor households and the broader environment benefit from the services in these areas. This is not a negative and may lead to more sustainable WASH services and systems over the longer term, as well as improved public health outcomes for the broader community. 4.5 RECOMMENDATIONS TARGETING THE POOR WITH SERVICES • It is recommended that after final PY3 data are compiled, including B40 survey data, USAID/IUWASH PLUS determine whether or not the LOP B40 targets need to be adjusted, particularly for direct water (B40) and overall sanitation B40 targets, in particular for hotspots, for the remaining project period. • In collaboration with the hotspots, IUWASH PLUS should assist communities to prepare an analysis and more specific plan of action than currently exists for each hotspot to capture the identified needs for improved water and sanitation, the budget needed and potential funding sources to address identified needs. They should create a short document that captures all the information to be used, as a tool for communities to present needs to potential funders, including LGs and CSRs. They should ensure that the document is consistently used across all hotspots to enable the aggregation of data by the project. This method will serve to support replication of the process to other communities. The project should prioritize TA toward these activities in hotspots over the next six months to one year. • Explore ways that the Pokja and LG agencies involved in WASH services could be more strategic in addressing the poor’s WASH needs beyond targeting them with special projects. Specifically, work to include hotspots and LG responsibilities related to addressing needs in hotspots over the remaining project period in the MOUs/annual plans. • It is recommended that the project work to increase its partners’ overall commitment to serving the poor to assist in the attainment of the IUWASH PLUS project’s B40 targets. The project could explore undertaking a review/planning process on poverty and WASH with the GOI, its partner LG Pokja and Bappedas. This process should discuss not only the attainment of the project’s B40 targets, but also the GOI’s goals, such as 100-0-100 and the SDGs, and how these targets will not be met unless stakeholders improve their efforts to reach the poor. This is particularly timely, given that Indonesia officially adopted the SDG targets in 2019. o Use these processes to identify a specific set of actions that can be implemented to address identified barriers in the short term (LOP) and for longer-term consideration. • Report back to project partners at the Pokja to share PY3 project data, e.g., from this report with LG and GOI partners, including data that show where the project is not meeting its B40 targets. Reemphasize IUWASH PLUS’ focus on and approach for targeting B40. Following review, revisit the question of what the shared interests are between stakeholders and the project with respect to serving the poor. Isolate factors identified in the MTE that appear to impede providing B40 persons with WASH services to validate them and to identify other factors as needed. Discuss options for addressing identified factors over the remaining project period. Consider leverage that could be taken by the project that future TA be dependent upon how partners perform with respect to targeting B40 and make this explicit in annual MOU agreements. USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 24 • Consider providing incentives or requirements to agencies that benefit from IUWASH PLUS technical assistance. For example, if a PDAM is becoming more profitable due to a reduction in non-revenue water or overall improved performance as a result IUWASH PLUS activities, consider requiring that the PDAM dedicate a percentage of its resources to serving more B40 households through a reduction in tariffs or reduced connection prices, etc. For agencies that are demonstrating that they reach poor households with improved services, consider acknowledging those efforts in formal, public ways. WATER • Advocate for PDAMs to take on more pro-poor policies, e.g., pro-poor tariffs and or pro-rated water connections based on a household’s poverty level. Consider piloting this in the project’s hotspot areas where a PDAM is a service provider, using mWater as the poverty verification tool. • Consider providing knowledge-transfer opportunities where PDAMs that have successfully implemented pro-poor solutions, and have visions to serve the poor, can share their approaches with other PDAMs. • Consider support of policies and behavior change campaigns to discourage the digging of wells by households and incentivize connections to PDAMs. As digging wells is a considerable problem nationwide that is threatening Indonesia’s ground water, this should be incorporated into the project’s “Be a Good Neighbor” campaign and aligned with the country’s climate change activities. • Consider modifying water targets at the end of PY3 based on the final results and after the above recommendations have been considered. SANITATION • There should be a renewed effort to implement the sanitation marketing approach. To date, there has been insufficient attention to the strategy’s implementation. It is recommended that IUWASH PLUS increase staff capacity in this area, revisit the strategy of how to engage with the private sector, and explore partnerships with established commercial firms providing sanitation products and services. In addition, IUWASH PLUS should continue to coordinate closely with OKJ on its sanitation financing reform efforts, and be prepared to help implement the new policy when it reaches approval (or prior to this through pilot testing). o Significantly increase project resources to support greater MFI and other financing scheme options that are available to, and realistic for, poor households (see EQ3). o Consider partnering with international experts in the MFI sector to better inform the project’s MFI TA activities. o Increase the connection between the sanitation marketing strategy (and/or its components, e.g., MFIs, products) and the hotspot stakeholder process to address sanitation gaps identified in hotspot areas. For example, invite banks/MFIs and entrepreneurs to stakeholder meetings where they can hear the presentation of community needs (see also EQ3 recommendations). • In concert with increased fecal sludge management (FSM) services through UPTDs (supported under the project’s WASH ecosystem activities), consider accelerating adoption of regulations that require safely managed septic tanks that work to push demand for safely managed sanitation, particularly basic affordable household septic tanks (see EQ4). In addition, work to support efforts to improve enforcement of adopted regulations for septic tanks to be watertight, and discuss with the Pokja both incentives for meeting those requirements (and perhaps fines for those that are not, for example, large businesses or high-income households), with 25 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV fine payments going toward a pool of funds that poor households could use to build safely managed septic tanks. MONITORING AND EVALUATION • The project needs to identify other methods or increase the frequency of B40 data collection and analysis, to be able to more regularly measure progress toward its B40 targets. This could include training project partners or other stakeholders on how to use the mWater platform to collect data. This could be used to determine the validity of the method, and whether it could be taken to scale and, if so, the resources that would be required. Also consider expanding data collection to not only collect data when a project is completed, but also at other key points. For example, for some projects it may be possible to conduct a survey earlier, such as at the point an IPAL in a community is designated for construction for a specific number of households, rather than completion. To verify final results, the wealth index questions could be incorporated into the endline survey. o Look at recently released 2017 DHS data to see if they provide any relevant proxies to assess whether selected project sites have high concentrations of poverty. o Broaden the analysis of data to further assess who is being served by wealth quintile, not just limiting analysis to B40 and non-B40, to gain a greater understanding of beneficiary demographics. • Look at access not only by water and sanitation, but also by factors such as done by the MTE, by type of intervention, as well as collaborators, to further assess the approaches that are more effective at targeting the poor, and where they would be most effective. Also consider how to decrease the cost per household tied to the particular type of service infrastructure. • Explore opportunities to replicate the use of mobile-phone platform data collection methods, for example, mWater, with project partners, for not only collection of poverty data, but also other WASH data needs. Look for opportunities to use other project partners, such as health services and public health officials, to collect household poverty data. o Start to share and discuss the B40 verification process with the Pokja, and explore if it is feasible to take to scale, as well as what challenges, barriers and opportunities may exist after the IUWASH PLUS project ends. • The project has a wealth of data on B40 and interventions. Consider partnering with a university or other research institute to conduct a research project to better understand and inform future interventions about which project factors and activities are more likely to lead to increased access to poor households. While it may not be feasible to modify variables at this junction, work to ensure that future USAID programming uses consistent data variables across a project to improve the ability to analyze project data. USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 26 5. EVALUATION QUESTION 2: SUSTAINABILITY CHECKLIST 5.1. DESCRIPTION OF PROJECT ACTIVITY BACKGROUND AND RATIONALE FOR SUSTAINABILITY CHECKLIST Too often, donor-funded WASH projects have supported the construction of WASH infrastructure, only to have such WASH systems cease to function shortly after completion. The most common reasons relate to operations and maintenance (O&M),53 or a lack of consistent funding for activities. In the case of communal WASH infrastructure, additional challenges relate to managing functions on behalf of beneficiaries, often by community organizations called KSMs (non-governmental organizations) or KPPs54 (users’ groups). In response to this issue, the IUWASH PLUS team developed Sustainability Checklists (SCs) to ensure that activities that the project considered critical for WASH infrastructure sustainability would be identified and tracked as implementation moved forward. These checklists will be used to establish how new water supply and sanitation systems will be developed and, most importantly, identify important gaps in implementation. The need for such checklists is based on the understanding that […] there are often important areas that are critical to sustainability that are not adequately supported and where the project could effectively contribute in terms of identifying and filling such gaps, directly or through the engagement of other agencies (governmental, donor or private sector).55 Due to the multi-sectoral nature of WASH, the construction and sustainability of WASH infrastructure requires substantial collaboration among various LG offices, KMS organizations,56 operators, sanitation entrepreneurs and implementing partners. In addition, the completion of SCs allows the project to report on high-level results indicators for drinking water services and safely managed sanitation (HR1 and HR2): […] if all critical activities [itemized in sustainability checklists] are conducted, including with USAID IUWASH PLUS direct implementation of one or more such activities, they may be counted as “direct support” and the beneficiaries included under HR1 or 2.57 IUWASH PLUS uses SCs to facilitate all WASH connections that the project supports through direct activities.58 Thus, the tool both ensures that elements supporting infrastructure sustainability are in place and that they document achievements in—and IUWASH PLUS contributions to— 53 IUWASH Final Evaluation Report, page 49. 54Kelompok Pengguna dan Pemanfaat, Community Users’ Group. 55 IUWASH PLUS Quarterly Progress Report (QPR) 2, Nov-Dec 2016. 56 KMS organizations are often involved in triggering and behavioral change activities and, when they are beneficiaries, WASH infrastructure management, operations and maintenance. 57 IUWASH PLUS PY1/Q2, page 14. 58 Connections to the water utility (PDAM) do not utilize SCs, except if facilitated with water grants. LEAD EVALUATION QUESTION 2: Do the sustainability checklists provide utility to the project and serve as an effective tool to measure result of direct activities? 27 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV WASH access. The following sections describe the activities that IUWASH PLUS conducted to implement the SC system. Sustainability Checklist design. IUWASH PLUS reported developing the first set of SCs in late 2016,59 reporting six versions of the form corresponding to collaborations on different WASH infrastructure: four for sanitation systems (individual, communal, small-scale sewerage, rusunawa) and two for water systems (master meter, communal SPAM). The project later designed an additional checklist for PDAM connections facilitated by water grants. A full sample SC is provided in Annex 16. The current version of the SC should be filled in twice: first, in the early stage of the collaboration, to document the agreement on responsibilities for each partner; and then it is filled again to verify the completion of the support from each partner.60 According to IUWASH PLUS, one form is prepared “per program, per year and per source of funding.”61 “Per program” refers to an agreement with a specific funder (usually a LG agency) to collaborate on a water or sanitation system of a particular type (e.g., 300 individual sanitation systems). The “per year” stipulation defines an annual period that initiates in August. The form also documents in its first page the location (city or district), and the number of beneficiary households and individual connections that the system corresponds to. Figure 3 shows a sample SC containing three sections. The Main Activity section lists infrastructure-specific activities grouped into five functional areas: community preparation, planning, construction, O&M, and institutional. The other two sections are Check Agreement on Responsibilities and Check Realization, which both allow for the documentation of a date and an explanation. KIIs indicated that the former is completed when stakeholders (such as LG office representatives, hereinafter called “collaborators”) agree on roles and responsibilities for each activity, and document agreement details for each main implementer. Once the WASH infrastructure is built, a second check confirms that the activities took place as planned, including that the infrastructure was built. IUWASH PLUS M&E and GIS assistants (MEGA) conduct a final verification, since they support reporting on IUWASH PLUS direct assistance indicators that are managed using SCs. FIGURE 3: THREE MAIN SECTIONS OF SUSTAINABILITY CHECKLIST As IUWASH PLUS has gained experience with SCs, the form content has changed, although the national office team, not regional offices, develops each new version. For instance, when project staff 59 IUWASH PLUS PY1/Q2. 60 USAID IUWASH PLUS SC PowerPoint presentation, Q#2 Process Sustainability Checklist_03082019. 61 Ibid. USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 28 found that it was often difficult to obtain signatures from LG collaborators, the project’s leadership changed the policy to not require signatures. More recently, the national team reported that the current guidance is to implement the checklist using mWater. IMPLEMENTATION OF SUSTAINABILITY CHECKLISTS Development and rollout. According to KIIs, the national staff developed the initial designs for SCs and introduced them to regional offices during routine staff activity monitoring meetings. After this, joint teams of national and regional specialists discussed the concept and design with relevant collaborators, and incorporated their input into a revised design. This revised design was then disseminated to IUWASH PLUS specialists. IUWASH PLUS specialists were directly trained on the SC process during routine monthly regional office meetings. Training included topics such as guidance on the type of partner activities that would warrant SCs and how to fill in the tool. Training was first introduced in West, Central and East Java. 5.2. PROJECT ACHIEVEMENTS IUWASH PLUS reported the completion of 64 SCs in PY1 and PY2. Of these, 40 percent were in West Java, 25 percent in Central Java, 21 percent in East Java, 10 percent in SSEI and 4 percent in North Sumatra. Of the SCs completed in PY1 and PY2, 59 percent used the mWater application, mostly in Central and East Java. Table 2 shows the number of individual SCs completed corresponding to these WASH infrastructure types. TABLE 2: SUSTAINABILITY CHECKLISTS COMPLETED IN PY1 AND PY2 Infrastructure Type Number (%) of WASH Infrastructure Completed Overall Number (%) of Individuals Benefiting from WASH Connections Through Direct Support from IUWASH PLUS62 Individual sanitation 33 (52.0%) 28,380 (27.4 %) Communal sanitation63 19 (30.0%) 7,610 (7.3%) Sewerage 1 (1.5%) 1,585 (1.5%) CB water supply 2 (3.0%) 62,420 (60.3%) Water grant 9 (14.0%) 3,605 (3.5%) Total 64 103,600 62 Connections are at the household level, and we used the IUWASH PLUS assumption of five members per household for these estimates. Data source: APR1 and APR2. 63 Including communal IPALs. 29 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV 5.3. FINDINGS SUSTAINABILITY CHECKLIST DESIGN Assessment of Sustainability Checklist design. As IUWASH PLUS has gained experience from implementation, the tool design, content and implementation guidance have evolved. According to national-level staff, the most important part of the SC is the main activities section detailed under each functional section. Table 3 integrates the content of the activities that IUWASH PLUS considers critical to sustainability from among the seven SCs. Inconsistency of main activities across SCs. A comparison of the main activity content across the seven SCs indicates some inconsistency with respect to inclusion of activities. While no international guidelines exist for activities that are necessary to develop safely managed water and sanitation infrastructure, activities deemed important for some infrastructure may also be valuable for other types. Annex 18 lays out main activities by SC; a red question mark designates some activities that might be considered for inclusion in individual SCs, based on comparison with other infrastructure. For instance, under the Community Preparation section, the master meter and communal water supply SCs include the activity Assist the KSM to develop the articles of association and bylaws, while this activity is absent under the communal sanitation SC. KSMs also support communal systems. Similarly, under the Construction section, Procurement for construction material and contractor is a main activity in three of seven SCs, whereas Supervision [of] construction process is present in only two SCs. Both of these activities likely will be needed for other infrastructure. As another example, under the Institutional section, the communal sanitation system SC has no activity that involves desludging, whereas the remaining three sanitation SCs do: Operating (routine) desludging system present, in functioning sludge treatment plant. In the same section, the sewerage SC includes the activity System for regular effluent quality monitoring in place, while no SCs of the other sanitation systems do, even though increasing effluent standard regulations is being implemented among the LGs targeted by the project, e.g., Ministry of Public Works and Housing (MPWH) effluent minimum quality standards64 are available for other sanitation systems, including septic tanks. The activities all relate directly or indirectly to infrastructure sustainability. 64 AusAID and World Bank. Indonesia Country Study. East Asia Pacific Region Urban Sanitation Review. September 2013. Page 5. USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 30 In some cases, the absence of relevant parallel activities in certain SCs may be justified, but in others the inclusion or exclusion of such activities could be reconsidered, especially if they have been useful for one SC. For instance, if the quality of some types of WASH infrastructure is more reliable than others (e.g., due to well-documented LG regulation and compliance applied to a pre￾approved, government-preferred supplier), then SC collaborators could decide that quality assurance (QA) at the pre-construction stage may be less important for that infrastructure type. Insufficient or inconsistent inclusion of activities ensuring quality and sustainability. Another observation regarding the main activity content is the apparent lack of systematic inclusion of quality assurance (QA) activities, or activity details that are vital to extending the lifespan of infrastructure projects. For instance, while detailed engineering design (DED) is a checklist item, additional steps or details are needed to ensure that the WASH infrastructure construction meets Public Works and Public Housing (PUPR) guidelines. For instance, the checklist does not indicate that a site visit was conducted to create the design, that an engineer verified that the DED meets quality standards, that the mason or contractor has the skills to correctly interpret the DED, or whether any inspections took place during, or at the end of, the construction process to assess that it adheres to the DED and meets code standards. IUWASH PLUS KIs have reported that QA activities do take place, and that they are usually undertaken by the party financing the infrastructure, often a LG agency. In addition, although the project does not directly fund construction, IUWASH PLUS KIs explained that the project also informally monitors the quality of the infrastructure that the project supports. In fact, during the review of completed SCs, the evaluation team did note an instance in which a supervision activity was written in a SC explanation column. Thus, given that activities ensuring the quality of construction also increase the sustainability of WASH infrastructure, adding key QA activities to the SC would help to systematically document and coordinate related activities. Coordinating such efforts could potentially reduce duplication. The construction section for all infrastructure SC forms does include a system testing activity, although it is unclear what this step includes. If SCs continue to be used after the IUWASH PLUS project ends, household-funded sanitation systems may especially benefit from a systematically documented QA activity step post-construction to ensure that households have guidance on when and where to find O&M service providers. Currently, there is a post-construction O&M step for KSMs and households to be trained in O&M, which may provide such training content. However, because households have fewer human resources to draw from than KSMs or KPPs, at least a written maintenance monitoring plan, if not a more robust QA approach, may be important. Individual systems represent half the sanitation systems that IUWASH PLUS directly supported to completion in the first two years of the project. TABLE 3: SUMMARY OF MAIN ACTIVITY CONTENT ACROSS SEVEN SUSTAINABILITY CHECKLISTS 31 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV Lack of clarity for some activities to be considered completed. In analyzing Table 3, most activities appear to be clearly defined such that there should be little ambiguity as to whether they are completed or not. However, there appear to be a couple of exceptions. In some cases, the activity is described at a high level one (e.g., strengthen KSM), so that it is unclear when this would be completed and what would be considered as “completed.” In other cases, activities are supposed to be conducted on a periodic basis indefinitely, such as routine maintenance and hygiene promotion. Thus, by definition, they could never be completely checked off. Such activities directly relate to infrastructure sustainability, and would not fit under the “per program, per year and per source of funding” framework. In other areas of the form, such as institutional activities, staff in some IUWASH PLUS KIs in East Java and Makassar indicated that some activities65 could be removed from SCs and addressed at an institutional level rather than per a specific project. SUSTAINABILITY CHECKLIST IMPLEMENTATION PROCESS KIIs indicated significant variability in how SCs are implemented, which IUWASH PLUS KIs explained is due to the diverse regional contexts, differing levels of collaborator commitment, and increased caution among some LG staff to document commitments (reportedly related to increasingly publicized anti￾corruption efforts, i.e., KPK, making LG staff hesitant to sign official documents). Thus, as described in the Evaluation Methods section, two online surveys were launched to provide a larger sample size of respondents to help further elucidate how SCs have been implemented. Insufficient guidance on completing SCs. IUWASH PLUS leadership confirmed reports from IUWASH PLUS KIs that no job aids, manuals, SOPs or other written guidance have been provided to support the implementation of SCs. Some IUWASH PLUS KIs indicated that they could benefit from additional guidance on how to complete SCs, so this question was investigated further in the online survey. Most (36, or 90 percent) IUWASH PLUS staff surveyed reported receiving guidance (verbal or written) on how to complete the SC. Among IUWASH PLUS staff that did receive guidance, eight (22 percent)—half of whom were specialists—did not think the training was sufficient to enable them to administer the tool. Interestingly, 16 (40 percent) of IUWASH PLUS survey respondents indicated that they received written instructions or job aids, even though IUWASH PLUS leadership stated that no written guidance has been provided to staff on how to complete the SC. Given this unexpected finding, guidance may be available that has been developed independently, or this result could be an indication of courtesy bias. 65 For example, on the Individual Sanitation Systems SC, preparation of prospecting databases for the development of desludging system and operating (routing) desludging system present, in functioning sludge treatment plant. TABLE 4: HOW OFTEN DO STAKEHOLDERS MEET AS A GROUP TO DISCUSS AND COLLABORATE DURING USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 32 Among collaborator survey respondents, 14 (64 percent) reported receiving verbal or written guidance on how to complete the SC. This appears to be consistent with comments from some IUWASH PLUS staff survey respondents and KIs, which noted that the SC form is confusing for some LG collaborators. In fact, among recommendations that IUWASH PLUS staff survey respondents provided, the most cited theme related to dissemination of SCs. Several respondents said SCs should be disseminated at the beginning of projects, with specific mention of dissemination to the leadership level, so that the commitment is at the agency level, and dissemination to IUWASH PLUS facilitators and supervisors. Context in which SCs have been completed. According to national staff, IUWASH PLUS staff normally discuss SCs during routine collaborator meetings (e.g., to discuss the governance or sustainability index). However, if a SC collaborator is unavailable, IUWASH PLUS will meet with him or her separately. Table 3 shows how survey respondents confirmed the broad variation in meeting frequency. There is also variation as to which IUWASH staff members complete the SC. A review of completed SCs indicated that one-quarter (27 percent) of forms completed in PY1 and PY2 documented the name of an IUWASH PLUS facilitator at the end of the form, who is presumed to be a main coordinator of the process, while the rest documented a specialist or the WASH associate in DKI Jakarta. At the same time, among collaborators who provided recommendations for improving the SC form or process, the most common response (four of 11) was for more frequent or regular meetings, or more M&E through the SC. Documentation of roles and responsibilities. Review of the 44 SCs completed at visited sites revealed that main implementers (i.e., collaborators) documented on SCs were largely limited to IUWASH PLUS and LGs (often just one agency). Additional collaborators were rarely included. The SC review also pointed to discrepancies between the way staff said SCs should to be completed and the way they are completed. For example, the explanation column under Column 3 of the sustainability checklist (Figure 3) is supposed to detail the roles and responsibilities that collaborators agree to. When IUWASH PLUS survey respondents were asked how they fill out this column, 30 percent indicated Roles or activities of implementers, 15 percent said IUWASH PLUS activities, and another 33 percent responded Activities—with no additional specificity. The IUWASH PLUS role described in the review sample included triggering/behavior change activities and KSM capacity building, including setting and collecting fees for master meters, communal SPAM and communal septic tanks. All reviewed SCs listed IUWASH PLUS activities. However, only 22 (50 percent) of the SCs completed in PY1 and PY2 contained any description of responsibilities of any LG collaborator, the project’s main partner. The survey also asked collaborators if they received a written copy of the completed tool, with the rationale that the tool is supposed to be a reminder of the agreed-on division of labor. About three￾quarters of collaborators (73 percent) noted that they received a copy of the SC after roles and responsibilities were agreed on. Notably, among the 11 collaborator survey respondents who provided recommendations, two suggested that copies of the SC form be distributed after agreement on roles and responsibilities is reached. Verification of realization of main activities. Column 4 of the SC (Figure 3) is supposed to verify the accomplishment of agreed-on activities. Contrary to what one might have expected, the review of completed SCs indicated that 32 (73 percent) of SCs had at least one activity that was not documented as completed, suggesting that either they should not have been signed off as completed, or documentation of verified activities was poor. A related challenge is the guidance that one SC form be filled “per program, per year and per source of funding.” Some survey responses indicated that the SC tool is not designed to monitor multiple 33 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV WASH infrastructure in the same form. One may speculate that there simply is not enough space or a clear method for documenting progress of more than one infrastructure project on one SC. In addition, to understand how this instruction may have been interpreted and implemented, the survey asked respondents how many SCs would be completed if 50 master meters/communal sanitation/individual sanitation systems were in place in a city (multiple choice). About one-third (38 percent) said they would fill out one SC, while 30 percent said 50 SCs would need to be completed. The other third (32 percent) provided a written response found to be consistent with the requirement that one SC be completed per program and per funding source (i.e., the most correct response per IUWASH PLUS guidance). In fact, a single three-page SC form for individual systems in Sukoharjo documented that the checklist corresponded to 3,526 household connections, including 1,987 B40 households. Overall, it appears that the guidance may have been inconsistently implemented. Finalization of SC (sign-off). IUWASH PLUS leadership explained that, given challenges in obtaining signatures from LG offices, as previously described, current IUWASH PLUS guidance no longer requires signatures for completing SCs (and national office guidance has shifted toward mWater implementation), although the project accommodates requests by some collaborators to continue using the hard copy process using signatures. Given the lack of requirement for signatures, documentation of IUWASH PLUS staff and collaborators’ names using the SC to develop WASH infrastructure would seem desirable for documentation purposes. However, among the SCs completed in PY1 and PY2, seven (16 percent) had no signatures or names of IUWASH PLUS staff, and 40 (91 percent) contained no names or signatures of collaborators. In some cases, signatures were accompanied by the stamp of the agency funding the infrastructure. In fact, the lack of agency point-of-contact names affected this evaluation. KIIs for EQ2 were set up with agencies, not individuals, since SCs do not systematically document names of collaborating agency representatives. Thus, the MTE team discussed SCs in general with KIs from the Pokja AMPL or Pokja Sanitasi who reported having completed at least one. The team found that some KIs were not aware of the specific SCs that evaluators showed them; in at least two sites, KIs reported not being aware of SCs in general. Collaborator KIs at one of the two sites confirmed that they worked closely with IUWASH PLUS staff, but none had ever seen the paper or mWater version of the SC. At the same time, a completed SC corresponded to the infrastructure at the site visited. Sustainability of SC-facilitated WASH infrastructure. Most collaborator survey respondents (73 percent) reported that WASH infrastructure they worked with established a tariff or fee collection system for O&M for household users. Among those, 92 percent said that fees were still being collected. When asked if WASH infrastructure supported by the SC process was still functioning as designed at the time of the survey (April 2019), 80 percent of IUWASH PLUS staff and 73 percent of collaborators said yes. Annex 17 shows survey responses regarding infrastructure that was not functioning as designed. Perceived utility of sustainability checklists. One of the first survey questions asked respondents to rate the usefulness of SCs on a Likert scale. An overwhelming majority felt they were Useful (95 percent of IUWASH PLUS staff and 100 percent of collaborators), and more than half considered them Very useful (59 percent and 58 percent, respectively). Figure 4 shows their multiple-choice responses when asked about what makes the SC process useful. Overall, the evaluation team observed that the SC process appears to have provided LG staff with a greater awareness of aspects in preparation, planning and the follow-up of sanitation systems construction that is necessary to increase the sustainability of such systems. USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 34 One of the last survey questions asked respondents to describe the degree to which they felt that SCs have been useful for supporting more sustainable WASH infrastructure. All IUWASH PLUS staff and 81 percent of collaborators responded Useful or Very useful. When asked why, emergent themes in each respondent group were similar: IUWASH PLUS STAFF COLLABORATORS Division of labor and responsibilities Clarity (of activity details and stakeholder responsibilities) Encourages commitment Stakeholder commitment Monitor activities done Track and control activities done/not done Understand steps/stages that need to be taken (ensure done) / common understanding 5.4. CONCLUSIONS “EFFECTIVENESS” OF THE SUSTAINABILITY CHECKLIST TOOL • All KIs with knowledge of SCs confirmed IUWASH PLUS’ contribution as documented in the SCs. The evaluation team concludes that the SC has successfully served as a tool for IUWASH PLUS to document its contribution to more sustainable WASH infrastructure. Thus, it has unquestionably provided utility to the project, and served as an effective tool to document and measure IUWASH PLUS’ direct activities. • IUWASH PLUS’ documented role in WASH infrastructure project collaborations has indeed focused on activities that contribute to sustainable WASH infrastructure, as assessed in the SC review (i.e., triggering and behavior change activities, and KSM capacity building). Both activities contribute to more sustainable WASH systems. • The SC appears to have effectively served as a planning and monitoring tool for many WASH infrastructure collaborations, and it may have had a role in increasing joint accountability at sites that filled it out more completely, a process led by varying LG offices at each site. However, FIGURE 4: WHAT MAKES THE SUSTAINABILITY CHECKLIST TOOL USEFUL? 35 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV most of the infrastructure projects using SCs are projects already planned and likely budgeted by LGs. Thus, by design, buy-in by at least one LG agency should be expected. • Form design and completion (i.e., filling in the form): o Given the findings on the content of SC form design and how the forms have been filled out, an additional form design review may be beneficial to reassess the content of SC main activities fields before the next round of SCs are due to be filled out. The form design could incorporate more activities that would enhance sustainability of infrastructure, in part by cross-pollinating learning that has led to elements that some, but not all, SCs have already incorporated. • Content vs. process: o Usage of the tool has varied significantly, both in how the form is completed and the collaboration process. o If the most important part of the tool is its main activities field (form content and completion), and half of the SCs completed by PY3 do not describe LG roles and responsibilities, then it appears that the process around SCs may have successfully led to WASH infrastructure construction, including support from IUWASH PLUS staff. The MTE team cautions against ascribing credit to the tool alone, which risks losing the opportunity to capture the process and learn from it. • The SC has the potential to have much greater impact if the tool and process could guide collaboration on sustainable WASH infrastructure independently of IUWASH PLUS. To achieve this, the SC would need to better reflect key elements of the process that make it work. The challenge is that documentation of the process is absent, but would help provide a better understanding of what makes a successful process work. 5.5. RECOMMENDATIONS We recommend that IUWASH PLUS decide on the primary purposes of SCs, and align form design, guidance on completion, implementation and expectations of acceptable completion (i.e., supervision standards) to the purposes that the project defines. The project could commit to several potential purposes for SCs. For instance, if the tool’s purpose is to document IUWASH PLUS’ direct contribution to sustainable infrastructure, the current tool accomplishes this goal, but may not be documenting all IUWASH PLUS contributions. However, if the tool’s purpose is also to provide clarity on division of labor or promote accountability or commitment, which appears to be inconsistent with how it has been implemented thus far, then: (i) forms without names of collaborators representing collaborating LG agencies and IUWASH PLUS should not be accepted; (ii) specific collaborator commitments for all parties should be documented; (iii) community/ beneficiary members should be involved so that they are clearly aware of their role in sustaining the infrastructure over the long term; (iv) users should make greater effort to ensure that other collaborators, such as entrepreneurs and KMS organizations, are included in the process; and (v) any collaborative efforts that do not use mWater should ensure that all collaborators have a copy of the SCs soon after roles and responsibilities are agreed upon. Another consideration is whether the tool is a potential legacy product that IUWASH PLUS might prefer to leave behind, i.e., whether stakeholders feel the tool could provide value in collaborations without IUWASH PLUS support. If the project adopts this objective, then IUWASH PLUS could consider transitioning of the SC stewardship to LGs while IUWASH PLUS is still supporting the project to help ensure a smooth transition. USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 36 Finally, another purpose the tool could serve is to shape GOI (and other partners’) policy, guidelines, and activity implementation so that the gaps that IUWASH PLUS fills now do not persist in the sector. To summarize, once IUWASH PLUS is internally clear on the project’s goal for SCs, the evaluation team recommends that the project: • Develop written learning materials and guidance for use of the SC and provide continual opportunities for learning and refreshers for appropriate staff and partners. • Review, revise and standardize the SC, with an emphasis on incorporating what has been learned from field practice. • Review the process to increase the involvement of partners in the use of the SC. As part of this effort, conduct discussions and outreach to determine if there would be an interest or benefit for adoption by Indonesian agencies. We also propose the following specific recommendations for using SCs: • Provide forums for continual awareness raising (“socialization”) of SCs to the GOI/Tim Teknis, both IUWASH PLUS staff, collaborators and LG office supervisors/leadership to obtain and keep their buy-in. Topics should include the purpose, background, form content, process, results, model examples and exercises, and discussions. • Provide written guidelines (SOP) for completing the tool, as well as the implementation process. • Identify champion IUWASH PLUS staff and LG collaborators who can share their experiences with other LGs on the details of how to implement SCs (content, as well as process). • Incorporate internal mechanism to audit SC to ensure they are being completed according to IUWASH PLUS guidelines. CONTENT • Consider how to simplify and make the tool easy for LG collaborators to understand and use. • Design: − Consider designing an addendum or sub-forms to the tool to easily monitor progress when one checklist contains multiple water or sanitation system projects, such as for the individual sanitation system SC and possibly other versions of the form. − Consider use of MOUs, or one form up front in the SC for institutional agreements or other items, e.g., pulling from some of the components in Section 5 that may apply to multiple projects but that may not be needed to be recompleted for each new SC. − Re-assess and standardize content of main activities section of the tool. ▪ Activities ensuring the quality of construction in the pre-construction, construction, and post-construction phases should be included as main activities. ▪ Activities considered critical for the design for one type of infrastructure should be assessed for relevancy for other infrastructure. − Describe the main activities with verification in mind. Consider how collaborators will verify how each main activity has taken place (e.g., back-up documentation, etc.) and to what degree of verification is necessary for each activity and reword accordingly. For example, preventive maintenance inspection activities can be described within a specific timeframe, such that they are either completed or not. • Completion of the form (i.e., filling it in) 37 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV − Systematically document dates (initiation, verification, milestones), collaborator names, as well as agreed-upon roles and responsibilities. PROCESS • Include other stakeholders besides LG offices in the SC process, or at least conduct meetings open to other stakeholders. Beneficiaries (e.g., KPP, households for individual sanitation systems) and other stakeholders may have more investment in seeing infrastructure projects to completion and, at the very least, their presence in key meetings could increase accountability. In turn, increased stakeholder understanding regarding the efforts needed to build WASH infrastructure could also encourage beneficiaries to play a greater role in maintaining infrastructure seriously. − In some cases, beneficiaries could be trained and tasked to help supervise part of the construction process, e.g., such as assessment of materials used for construction, or the presence of baffle walls, etc. • Disseminate SCs to all collaborators in initial meeting to discuss: − technical guidelines for filling in SCs; − the role of each collaborator (agency/committee/entrepreneur/KSM); and − the schedule for checking progress and joint-filling of the SC. • Consider scheduling regular monitoring meetings,66 with more frequent check-ins for infrastructure requiring longer time to implement. One option would be to schedule such meetings directly before or after Pokja meetings, when many collaborators are already gathered. • Options to consider: − Systematically track SCs that were initiated and not completed on schedule, or that did not function as planned, to better assess both the process, as well as the identification of issues, that may have inhibited sustainability. − Check in with beneficiaries and conduct a site visit of the completed infrastructure at discrete times (quarterly or semi-annually) after the SC/project has been completed, to assess whether or not the project has been sustained, and to capture any issues that need to be addressed. This could exist under addendums (above) in a follow-up section that captures when completed, by whom and status (operational Yes/No) and any other issues, and plan to address these. − Collaborate with LG agencies that check the quality of WASH infrastructure construction to harmonize the inspection approach and potentially coordinate efforts. • Consider developing a sustainability plan for the tool’s use with stakeholders so that it can be replicated and continued after the IUWASH PLUS project is completed 66This was specifically recommended by some LG collaborator survey respondents. USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 38 6. EVALUATION QUESTION 3: SANITATION MARKETING STRATEGY 6.1. DESCRIPTION OF PROJECT ACTIVITY The sanitation marketing strategy is one of many activities under Component 1. The IUWASH PLUS marketing strategy is working to align best practices in sanitation marketing, USAID guidance (e.g., Scaling Market-Based Sanitation Desk Review, 2018) and Indonesia’s Community-Led Total Sanitation Program (Program Sanitasi Total Berbasis Masyarakat, or STBM) for urban areas. The IUWASH PLUS sanitation marketing strategy began implementation in 2018, with pilot projects in three areas during the following months: • Bogor City (West Java Province), in August 2018; • Lumajang District (East Java Province), in December 2018; and • Magelang City (Central Java Province), in December 2018. The sanitation marketing strategy has three focus areas (products, services and financing) and includes five steps (or phases), beginning with site selection in coordination with the LG and extending to the provision of TA to stakeholders (i.e., entrepreneurs, service operators and financing). These occur in close collaboration with the behavior change (BC) strategy under Component 1. Key components of the strategy assessed by the MTE team, including BC, are briefly described below: 1. Behavior change activities. Specific to the strategy, BC activities work to raise awareness, and create interest and demand for healthy sanitation at the household level, including watertight septic tanks, to achieve safely managed sanitation targets under SDG 6.2. These activities take place in all of the project’s targeted communities, and are directed to both households that do not have toilets (i.e., those practicing open defecation) and those that already have toilets. The BC messages focus on: (i) safely managed sanitation, with segmented messaging targeted to those who do not have toilets and those who have septic tanks that are not watertight; and (ii) regular desludging. The project has developed BC messages, such as “Super Mom” and “Being a Good Neighbor,” which are designed to bring about changes in household behavior— specifically, moving households away from using non-safe toilets (or open defecation) and toward safely managed sanitation. 2. Household financing. The project works to link households with two primary sources of funding to construct toilets: a) Microfinance institutions (MFIs): This scheme is dedicated for B40 individuals who have the capacity to pay installments; and b) Smart subsidies program (using the special allocation fund, or DAK), which is dedicated to helping B40 individuals who are not capable of paying installments. 3. Sanitation products. The project designed a watertight septic tank (single-ring and double￾ring) and combined this with a pipe and gooseneck squatting plate as a safely managed toilet facility available for consumers to buy. LEAD EVALUATION QUESTION 3: To what extent are IUWASH PLUS sanitation marketing approaches effective in building public-private interest in this market-based approach? 39 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV 4. Sanitation entrepreneurs. The projects links other actors in sanitation marketing, such as households, MFIs and LGs, to sanitation entrepreneurs/enterprises that design, build and install sanitation facilities, or can provide desludging services to the targeted communities. 5. Technical assistance (TA)/capacity building. The project increases the capacity of LGs, sanitation entrepreneurs/enterprises and MFIs to provide more effective services to the community. 6.2. PROJECT ACHIEVEMENTS The sanitation marketing strategy activities initiated at the community level in PY3/Q1. The following are some key achievements and their current status relating to the strategy in the three pilot areas, identified through reports and KIIs, as of April 2019. TABLE 5: SANITATION MARKETING ACTIVITIES FOR EACH AREA 67 Village-owned business entity. Phase Bogor City Lumajang District Magelang City Start Date Started with a stakeholder workshop with LG officials, MFIs, sanitation entrepreneurs and a desludging operator service on August 9, 2018. Started with a meeting with LG on December 13, 2018 and followed up with a workshop with LG, MFIs, BUMDes, UPT PALD in March 2019. Started with an MoU between the Bank Magelang and the Housing and Settlement Agency on December 14, 2018. Strategic Planning The action plan is developed in a workshop and includes action steps to: (i) adopt the sanitation marketing strategy; (ii) map the sanitation market; (iii) establish linkages among workshop participants; (iv) develop a marketing plan for each key actor; and (v) integrate new sanitation market partners. Workshop status: Completed Workshop status: Completed Workshop status: Not yet completed Financing/ (Microfinance Institutions) Status: • MoU signed with the BAIK cooperative to provide sanitation loans. • Sanitation67 loans provided to 7 HHs Status: •MoU with two financial institutions still in progress (Fastabiqul Khoirot Coop, BUMDes) Status: •MoU signed with Bank Magelang to provide sanitation loans. • 1 customer approved for sanitation loan. • 2 group loan applications being reviewed (using 1 collateral for each KSM). Sanitation Product Introduced Septic tank with upflow filter (single ring and double ring) Sanitation Enterprises One enterprise (bio tank producer) is participating to strengthen supply side USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 40 6.3. FINDINGS The team found that the comprehensive sanitation marketing strategy documented does not reflect what IUWASH PLUSH is doing, because full implementation is not being undertaken by IUWASH PLUS. The pilot projects in the three areas have not been comprehensively executed and have not allocated all the available capabilities. The behavior change campaign as the driver of hygiene behavior has not been effectively applied within the sanitation marketing activity, due to the lack of a common understanding of BC messages and BC materials among the IUWASH PLUS team. IUWASH PLUS also does not have an effective approach to engage the private sector, due to a lack of staff with private sector engagement expertise. This has led to a lack of progress in working with sanitation entrepreneurs/enterprises. The household financing component has also not yet been fully developed as a comprehensive solution for funding sanitation for poor households, due to a lack of collateral to secure financing. However, IUWASH PLUS has an effective approach to building public sector interest, especially for developing local regulations, and building capacity for technical sludge management and wastewater treatment. For example, the project has applied this approach to support LG development of Perdas/Perkadas related to wastewater management and building UPTD PAL capacity. The project has taken a socio-ecological approach68 to addressing sanitation in its marketing strategy. This is in line with the STBM strategy, Indonesia’s 100-0-100 strategy and the GOI’s goal of achieving SDG 6.2 by 2030. A review of recent literature and best practices also indicates that successful sanitation marketing approaches include behavior change/creating demand, addressing affordability/financing and strengthening supply capability. Figure 5 outlines what IUWASH PLUS has included in its approach and the relationship between the various components. 68 McLeroy, et al. 1988 (cited under History), which appeared in Health Education Quarterly as “An Ecological Perspective on Health Promotion Programs,” further defined the social ecological model for health promotion to depict inter-related systems at the intrapersonal, interpersonal, organizational, community and policy levels, illustrated as concentric circles. TA/Capacity Building • 2 sanitation entrepreneurs trained, only 1 still active. • L2T2 socialization. • Facilitation of development of Management Information System (MIS) for desludging service. •No sanitation entrepreneur trained yet. • 3 sanitation entrepreneurs trained, only 1 still active. Regulations PERDA Domestic Wastewater adopted (signing process in the legislature). PERBUP for UPTD & PERDA for Domestic Wastewater Management (signed). Mayoral regulation for UPTD PAL and Domestic Wastewater Management drafted. 41 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV FIGURE 5: SANITATION MARKETING IN MARKET-BASED APPROACH Overall, the approach should be seamless if it is to work effectively to build and meet consumer demand. To achieve this, the following key elements are essential: behavior change promotion; household financing (i.e., financial resources and financing opportunities to enable access to safely managed sanitation); private sector actors interested in providing sanitation products and services; wastewater and desludging operators; and government regulations. The following sections outline findings on these components. BEHAVIOR CHANGE IUWASH PLUS conducted BC formative research and other formative research activities to inform its sanitation marketing strategy, as outlined in Table 6. TABLE 6: BEHAVIOR CHANGE AND OTHER FORMATIVE RESEARCH REPORTS FOR THE SANITATION MARKETING STRATEGY What Report Completed Purpose • Study Protocol BC Formative Research Report • March 2017 • To improve understanding of WASH conditions and provide a foundation for development of a behavior change strategy. • BC Strategy • April 2018 • To provide guidance on how to effectively promote WASH in urban areas among the targeted communities; specifically, to increase demand for and access to water and sanitation facilities. • BC Formative Research Report • May 2018 • To better understand WASH conditions for urban B40 households; to explore barriers and motivations around sanitation; and to identify BC entry points and communication channels. • Microfinance and Grant Beneficiary Report • April 2018 • To profile the current WASH microfinance market in the IUWASH PLUS project areas. Note: The market mapping and business inventory are contained in the Sanitation Marketing Strategy. USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 42 The BC strategy for sanitation is based on two theories of change. IUWASH PLUS developed two theories of change (TOC) for sanitation outlined in the BC strategy. One TOC relates to upgrading to safely managed sanitation service (in PY2, 24,915 people of a life-of-project target of 500,000 people, or 4.98 percent, gained access to safely managed sanitation service). The second TOC focuses on timely safe desludging, once again targeting people who have accessed safely managed sanitation services. Based on the available data, the percentage of survey households that met all safely managed criteria was only 1.5 percent.69 Inconsistent BC messaging. The IUWASH PLUS project created campaign materials to inform its overall project’s BC strategy. However, IUWASH PLUS KIIs indicated that the messages and BC activities have not been consistently applied within the sanitation marketing activity. Five of the six IUWASH PLUS KIs (BC/marketing specialists and WASH facilitators) mentioned that they were aware of the BC campaign “Being a Good Neighbor,” but did not fully understand the intention, background and purpose of this campaign. One WASH facilitator was not even aware of this BC campaign. Furthermore, from the perspective of IUWASH PLUS, a successful BC campaign would depend on the WASH facilitator in the field. However, according to interviews, some WASH facilitators were not equipped with BC skills and knowledge, which was reflected in their inconsistent understanding of the BC campaign. In addition, KIs reported that the absence of printed BC campaign communication material regarding “Being a Good Neighbor” made it difficult to convey the messages to the parties who assist in the BC campaign dissemination process, especially the sanitarians and cadres70 who work directly with communities. If sanitarians and cadres do not have a good understanding of the BC campaign, it will be difficult for them to deliver the key BC messages at the household level. BC strategy targeted to the household level. Up to now, the project BC message has primarily targeted BC at the individual household level, whereas the socio-ecological approach requires BC beyond just the household level, for example, at the community, government, market and financing levels. Furthermore, some key messages did not seem appropriate for the household level. For example, some materials included information on how to construct safe watertight septic tanks, which is perhaps beyond what is needed by households and instead more relevant to entrepreneurs. Insufficient focus on entrepreneurs in BC formative research. The formative research was extensive, particularly with respect to B40 populations. However, for the BC strategy to be effective, the private market side of the strategy—specifically products and entrepreneurs (businesses)—will likely also need to work with other populations to become viable businesses, given the current availability (or lack thereof) of MFIs and grants for B40. The formative research appears to be lacking for these populations. 69 USAID Indonesia IUWASH PLUS Final Report: Behavior Change Formative Research. 70 The health kader in Indonesia is a community health volunteer selected by the village head and village committee who may weigh children under 5 and pregnant women, provide health promotion and nutrition counseling, and support village midwives in maternal health services. The kader lives and works in the village that he or she serves and receives a small monthly incentive. 43 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV HOUSEHOLD FINANCING Efforts to address the lack of collateral. IUWASH PLUS is fully cognizant that a lack of collateral is a key barrier for poor households to access financing from banks, even for MFI loans, and the project has engaged with this problem on several levels. The MTE team found that at the national level IUWASH PLUS is exploring a promising initiative in collaboration with the Financial Services Authority (Otoritas Jasa Keuangan, or OJK) to set up a government sanitation loan program. IUWASH PLUS implemented a focus group discussion with several MFIs, and a plan for a pilot project between OJK and IUWASH PLUS in Sumatra is currently underway. This initiative should make it easier for poor households to access credit from MFIs. However, despite findings from the WASH Microfinance and Grant Beneficiary Study that microfinance is the preferred financing solution, other financing initiatives are also proving to be successful. For details on these alternative financing initiatives see the section on Sanitation Enterpreneurs/Enterprises Findings below. The project has also worked with MFIs to develop cooperative group loans. This financing model does not require collateral because installments are the joint responsibility of the loan group. From the KIIs, the MTE team found that this loan model nonetheless is still encountering challenges because of its stringent requirements, which include the following: • Borrowers must belong to a group consisting of 10 members who all participate in weekly meetings and make weekly installments for one year; • The amount of the loan that can be borrowed after one year is insufficient to build a safely managed watertight septic tank (toilet bowl and septic tank, not including the superstructure); and • Sanitation loans can be given only to members who have belonged for more than one year, because of the large size of the sanitation loan needed to build a toilet with a watertight septic tank. Lack of focus on affordability by LGs, MFIs and communities. The team also found that LGs, MFIs and communities do not appear to prioritize tackling the problem of funding and financing the construction of safely managed toilets. While various efforts (i.e., grants, microfinance, etc.) are currently underway, these are often small projects, or implemented incrementally in some communities. As noted in EQ1, without a significant influx of money, it may well be decades before Indonesia achieves SDG 6.2, as the country has the second-highest absolute number of people practicing open defecation and considerably more not using safely managed toilets. According to IUWASH PLUS formative research, only 13 percent of households have toilets with proper containment. To address this gap, the project’s socio-ecological model addresses many challenges that inhibit achieving Indonesia’s goal of 100-0-100 and the SDGs. However, what appears to be lacking is behavior change at the institutional levels. Banks and other financial institutions, LGs and the GOI also need to change their practices or “behaviors” to enable more favorable terms for households wanting to obtain financing, as well as an influx, or reprioritizing, of funding to meet this demand. This behavior change also requires that existing policies are enforced, for example, through a regulation on safely managed septic tanks. While the project is working at these institutional levels, more BC and/or advocacy is warranted. In addition to the above, a WASH Microfinance Toolkit was developed to attract MFIs to the sanitation market by understanding demand (promotion and marketing), as well as supply (masons, contractors, desludging operators, etc.). It also includes technical guidelines and tools to assist in the development of financial products for WASH. While this toolkit has not been completed, it could be useful in promoting microfinance once finalized. USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 44 SANITATION ENTREPRENEURS/ENTERPRISES Few entrepreneurs were trained and even fewer have been retained. The MTE team found that IUWASH PLUS efforts to engage sanitation entrepreneurs/enterprises have been inadequate. Forty-one entrepreneurs were trained during PY2 (IUWASH PLUS APR 02 QPR 09 Report) from a total of 100 sanitation entrepreneurs targeted by the project. Within the pilot project area, five entrepreneurs had been trained to date, but only two were identified as still being active in the pilot sites (Table 5). Based on KIIs, the team found several key issues that discouraged the recruitment and retention of sanitation entrepreneurship: • Key project staff dealing with sanitation entrepreneur training and recruitment did not have a business background. This role is primarily the responsibility of BC/marketing specialists, whose background is primarily focused on behavior change rather than on business skills. Consequently, the ability to provide business technical assistance is limited. • The selection of sanitation entrepreneurs was not based on their key qualifications but depended instead on the request made to the villages and suggested by the LG health office. They are often unaware of the key qualifications needed for a sanitation business entrepreneur. Therefore, some cases indicated that sanitation entrepreneur candidates lacked a background or interest in the field of constructing toilets or sanitation systems. • Post-training follow-up activities were inadequate to provide guidance or mentoring to recruited entrepreneurs. Existing post-training activity was limited to connecting entrepreneurs with MFIs and sanitarians, but this effort was not sufficient to encourage entrepreneurs to become viable sanitation businesses. • The project focuses primarily on B40 populations in its strategy, and this market (poor households) is not, or is not seen to be, sufficient on its own to entice business entrepreneurs, as they view this market as too limited. • Available microfinance funds for B40 households are insufficient to build a complete safely managed toilet within the one-year loan timeframe, while grant money is available only to a limited number of households each year. In Tangerang (Kayu Bongkok village, not a pilot site), one entrepreneur noted that he was able to build 25 toilets in the previous year, with only five from MFIs. The others were all funded by grants. • MFI KIs also reported that households would rather use a neighbor or family member to build their latrine than an entrepreneur, partly because they were less expensive but also because they would rather give their money to someone they know. Offering this choice means that many households end up with non-safely managed toilets. • The MTE team found that trainees unable to survive being sanitation entrepreneurs were lacking in business experience and were trained to work solely on building latrines. Meanwhile, sanitation entrepreneurs who survive are those who added latrine construction to an existing entrepreneurial business portfolio. This does not appear to recognize that creating sanitation entrepreneurs involves more intensive development than the current training it provides to existing construction workers. For example, in Makassar, the UPDT KIs indicated that it is challenging to recruit Indonesians to work in the sanitation field, as culturally it is not seen as an acceptable field. Other research, such as that by the World Bank and Water and Sanitation for the Urban Poor (WSUP), shows that more successful efforts in the recruitment and retention of sanitation businesses require more intensive programming to establish sustainable businesses. Global research and previous work in Indonesia show that stand-alone sanitation businesses are very unlikely to succeed. Sanitation products are most often successful when offered from 45 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV diverse service providers such as masons who can also build other facilities, hardware stores, sales teams, or MFIs that sell a wide range of products.71 • Physical challenges, such as a lack of space in high-density neighborhoods for a septic tank toilet, high water tables, etc., were also identified as challenges in urban areas, and entrepreneurs lacked the skills/knowledge to address these challenges. An example of a sanitation enterprise engaged in Bogor. The involvement of sanitation enterprises has started, in one of the pilot-project areas (Bogor City), to ensure the availability of a standard prefabricated watertight septic tank. Other toilet supply materials are widely available in the subdistrict areas (gooseneck squatting plate, pipes, cement, sand and brick). However, specific 71 See WASHPaLS global review for more details. There are some other opportunities for sanitation entrepreneurs that can be utilized by the IUWASH PLUS project. The MTE team found one sanitation entrepreneur that provides a one-stop service sanitation business named Terima Bersih (CV Naga Kencana), which operates in the East Java region, particularly in the Surabaya, Sidoarjo, Probolingo, Pasuruan, Nganjuk, Pamekasan, Malang, and Jombang areas, some of which are also IUWASH PLUS pilot project areas. Terima Bersih has a marketing network in collaboration with local health offices through cadres and sanitarians, and also has its own sales teams. For training and healthy toilet construction, it collaborates with the Association of Indonesian Sanitation Entrepreneurs (APPSANI). It also has access to sanitation credit through Bank BPD Jombang, Bank Mandiri and Bank BRI, and is in the process of offering sanitation financing with PT. Harta Insan Karimah in West Java. In addition to providing the construction of healthy sanitation for the community for cash payments, Terima Bersih also provides payment on credit. According to Terima Bersih’s experience, this model has some benefits, based on testimonials from their customers: … “I feel reluctant to deal with MFIs and feel more comfortable when dealing directly with Terima Bersih”… … “It is faster and more straightforward, as the whole process is handled by Terima Bersih (approval, construction process, and installment collection)”… … “Terima Bersih builds healthy sanitation with watertight septic tank and, if there is a problem with the toilet function, I can complain directly and the problem be immediately addressed by Terima Bersih”… But Terima Bersih also faces some challenges if communities pay on credit: ● To provide continuous community credit, a funding source is needed. Therefore, though Terima Bersih already has access to financial institutions, it still needs other sources of financing because all of these financial institutions use collateral or back-to-back money guarantees provided by Terima Bersih. ● Collection is a problem. Though this has been minimized by using a group approach through community leaders or cadres, sometimes defaults do occur. USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 46 products, including septic-tank models, appear to be lacking in the market to address some of the challenges faced, such as space limitations and high water tables. Pokja AMPL membership is inadequate to develop a comprehensive solution to sanitation challenges. The MTE team found that the Pokja and LGs lack the perspective and knowledge needed for a comprehensive solution to the sanitation problems they face. While membership of the Pokja is dictated by regulations, the private sector—whose perspective would be essential to developing a comprehensive solution—was missing. One Pokja in Probolinggo (not among the pilot-project sites) included sanitarians as members, and it was clear that their involvement focused efforts on sanitation marketing—more so than other Pokja interviewed. Furthermore, the sanitarians in Pokja Probolinggo acted as entrepreneurs or offered microfinance and other schemes to households where they worked. BUSINESS MODELS The MTE team found that IUWASH PLUS noted that there were six business models for household sanitation products involving sanitation entrepreneurs, cadres, masons and sanitation enterprises to build toilets and septic tanks for households/customers that did not rely on MFIs. Of these six types of business models, described in IUWASH PLUS Annual Progress Report 2 and Quarterly Progress Report 9, October 2018, there was very little evidence of their use in the field. As a result, evaluation of the business models was not undertaken. Meanwhile, in Bogor City, Magelang City (pilot site) and Tangerang District (not a pilot site), the MTE team found business models for the development of sanitation facilities through MFIs. The MFIs (the BAIK cooperative in Bogor City, Bank Magelang and the BMI Tangerang cooperative) provide Sanitation enterprises always strive to develop their products and services and could be a valuable source of innovation for the IUWASH PLUS project. Sanitation companies such as American Standard Indonesia develop latrine products for poor households in areas with limited water supply. One such product is called SATO toilet systems, which the company claims can save up to 80 percent in water use (Annex 19). This product has already been ordered by water.org to be used in its projects, and several other NGOs are also interested in adopting it, despite the fact that it is still in its assessment period. American Standard also indicated that IUWASH PLUS had not actively approached the private sector to look for innovations in the latest product developments, and that its approach was more directed toward government. Sanitation enterprises also develop products and services targeted at low-income communities, especially in swamp areas, areas with high water tables and densely populated areas that do not allow the construction of watertight septic tanks. One such product and service, called Portable Toilet Systems, is being tested in the Palembang area, in collaboration with the LG (Bappeda). Portable Toilet Systems is a complete system using hygiene portable toilets (Annex 19), waste collecting at the RW level using sealed containers, and scheduled desludging for processing in wastewater treatment. To avoid burdening the community, the portable toilet units are provided free of charge and the community is only charged service fees for the waste collection and desludging process, which also takes into account the margin for members to be able to continue with their businesses. Sanitation enterprises developed this system and service because it had been so difficult to sell other products that used septic tanks, given the limited land, swamp areas, high water tables and the weak purchasing power of people who were unable to build a complete system of toilets using septic tanks. This is in line with the difficulties faced by IUWASH PLUS in developing products that fit the areas mentioned above. 47 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV sanitation loans to the community. The construction of sanitation facilities can be undertaken by the community with the loan funds, which means that the MFI only provides the loan, or the sanitation facilities are built using microfinance provided through the sanitation entrepreneur. The construction of sanitation facilities by MFIs is actually preferred by the BAIK cooperative, because this helps to ensure that loans are fully used for the manufacture of sanitation facilities (toilets and watertight septic tanks) and also guarantees their quality, in particular the manufacture of watertight septic tanks, because these are built by well-trained sanitation entrepreneur partners. However, the BAIK cooperative is still experiencing problems in finding a sanitation entrepreneur that can help with the construction of these healthy toilets. Another business model that is more comprehensive, such as the one carried out by Terima Bersih, which combines collaboration with the LG through sanitarians with MFIs and sanitation entrepreneurs, is also not yet among the business models offered by IUWASH PLUS. FECAL SLUDGE MANAGEMENT (FSM)/WASTEWATER TREATMENT Based on the IUWASH PLUS project indicator (HR-2) and SDG 6.2, safely managed sanitation is the final desired result from the sanitation marketing strategy. A key component of safely managed sanitation is fecal sludge management (FSM)/wastewater treatment. Table 7 captures the current situation of FSM in the three pilot project areas. TABLE 7: STATUS OF FECAL SLUDGE MANAGEMENT/WASTE WATER TREATMENT IN SANITATION MARKETING PILOT SITES Pilot Site Regulations in Place or Missing, Including Tariffs Desludging Services/ Operators in Place (Public or Private) UPTD in Place (Public) • Bogor City • PERDA Domestic Wastewater adopted (signing process in the legislature) • 3 trucks (public) • Installed capacity 26 m³/day (will increase to 45-100 m³/day in anticipation of the L2T2 program) • Filled capacity +/- 8 m³/day • Lumajang District • PERBUP for UPTD & PERDA for Domestic Wastewater Management (signed) • 1 truck (public) • 1 kedoteng / 3-wheel motorcycle (private) • 1 truck (private) • Installed capacity 25 m³/day • Filled capacity +/- 1.5 m³/day • Magelang City • Prepared / assist Mayoral Regulation for UPTD PAL and Domestic Wastewater Management (drafted) • 2 trucks (public) • 2 trucks (private) • Installed capacity 30 m³/day • Filled capacity +/- 2 m³/day To achieve safely managed sanitation, the MTE team found that a variety of challenges remain in the pilot areas’ FSM systems. For example: • Recurring cost of desludging can be a disincentive. People who are approached to build toilets with watertight septic tanks reported that, not only did they have problems with the construction, but they were also reluctant to build watertight septic tanks knowing that they would have recurring future expenses to cover the desludging. USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 48 o Makassar City (not a sanitation marketing pilot project area) is being supported by an initiative from UPTD PAL to provide a special desludging payment scheme to B40 communities with three options, depending on households’ financial ability: pay in installments for two years, pay at a 50 percent discount, or receive free of charge. • Inadequate enforcement of regulations requiring safely managed septic tanks. While new regulations are being established to require safely managed septic tanks, enforcement is minimal, particularly for existing septic tanks. • Desludging trucks cannot access densely populated neighborhoods. The density of settlements in some urban areas constitutes a challenge for existing desludging trucks to access. Makassar City has tried to overcome this problem by using three-wheeled vehicles (kedoteng), but the two that the UPTD owns have been nonfunctional for over a year, reportedly due to the UPTD being unable to source the necessary spare parts. • IPLTs are operating below capacity. In urban areas, only a few households have watertight septic tanks (based on formative research, only 13 percent of households in urban communities have a toilet with proper containment), and as a result few households need desludging. Consequently, many existing IPLTs operate below full capacity. This also impacts one of the desludging operators in Jabodetabek, named PT. Jayalindo, which reports that there are still too few households that have watertight septic tanks such that PT. Jayalindo does not receive daily orders. • Wastewater operator staffing challenge. UPTD PAL operators reported significant challenges in recruiting staff, as most view the waste management sector as embarrassing, with low pay and stigma attached to it, resulting in an insufficient supply and retention of human resources. STAFFING/RESOURCES/TIMING The project did not appear to have sufficient dedicated—or perhaps suitable—staffing resources at the time needed to develop the sanitation marketing strategy, or it has delayed putting these resources to work in implementing the sanitation marketing strategy activities. This was partly evidenced by the delay in the strategy’s development; the initial draft was not created until April 2018 (PY2/Q3) and not finalized until February 2019 (PY3/Q2). In addition, the hiring of one key position, the business-enabling specialist, did not occur until April 2019. This is a pivotal position, as it addresses gaps in creating policies to support financing mechanisms that can be taken to scale (e.g., microcredit for households), as well as strengthening the business and economic component of the project’s sanitation marketing strategy, with skills not typically found in WASH programming staff. IUWASH PLUS indicates that it has been a challenge filling this position, partly because it was difficult to find candidates with both sanitation and marketing experience, and particularly because of the salary level offered. Delays were also reported to be related to IUWASH PLUS focusing on its formative research for behavior change, which was not completed until May 2018 (PY2/Q3). A review of Component 1 job descriptions at both the national and regional levels for positions that play a role in sanitation, and sanitation marketing in particular, found that the descriptions were balanced in their inclusion of marketing, promotion and other components to support sanitation marketing, but did not include any of the expected qualification or skills, such as business development and marketing. KIs indicated that the project recruited for sanitation skills first and business skills second. In addition, the key position descriptions, in particular the regional behavior change marketing specialist (five positions) description, have a broad range of duties and tasks. The MTE team found that, while existing staff interviewed were skilled and experienced in WASH in Indonesia, with many having already worked on IUWASH PLUS and other USAID-funded projects, few had specific experience with the private sector. 49 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV 6.4. CONCLUSIONS BEHAVIOR CHANGE While “Being a Good Neighbor” appears to be a good message for the BC campaign and is understandable to the community, it has not been effectively implemented. This is associated with the inconsistent understanding of the key messages and the overall quality in delivering the key messages, which in turn is linked to differences in the skills of BC/marketing specialists and WASH facilitators responsible for delivering the key messages. In addition, the stock of printed promotional material is insufficient, the material is too technical for the targeted communities, and dissemination has been limited to households. Finally, a BC strategy relating to the project’s sanitation marketing plan targeted to the private sector, such as to banks, MFIs and sanitation entrepreneurs/enterprises, is also lacking. HOUSEHOLD FINANCING One positive ongoing step is to involve OJK in the sanitation loan program at the national level. There are also efforts to create linkages between sanitation entrepreneurs, LGs and MFIs to promote sanitation loan programs to address financing needs among households to build healthy toilets. However, significant gaps remain in the financing and loan schemes available, including the loan period, to make them more attractive and viable for poor households. The available loan schemes are too few to address the significant sanitation needs, not only in the pilot areas but also in the targeted areas and in Indonesia overall. SANITATION ENTREPRENEURS/ENTERPRISES So far, IUWASH PLUS does not appear to have the right skillset among its staff to create a compelling business case and support package that would attract more entrepreneurs into sanitation. As a result, the recruitment and post-training processes have not been properly implemented. The planning process at the LG level for the sanitation sector has also not been oriented toward a market-based solution involving the private sector. Sanitation solutions are limited or lacking for entrepreneurs to address the physical constraints for desludging and latrine construction that are often prevalent in the target areas. It will be easier to develop existing sanitation entrepreneurs, especially those that have access to LG and financial institutions, such as Terima Bersih, to speed up the process, instead of creating new ones. Furthermore, existing sanitation entrepreneurs are already established in the same areas as IUWASH PLUS pilot projects, and their business model as a one-stop service could be replicated in these pilot projects. Sanitation enterprises, with their own strategies and resources, are always trying to develop the market by creating new products and services that meet the needs of the community. It is possible that what they offer addresses the obstacles that prevent achieving safely managed sanitation, but that IUWASH PLUS has yet to pay them sufficient attention or attempt to collaborate with them. BUSINESS MODELS IUWASH PLUS has not yet fully included alternative sanitation financing into its business models and has so far failed to evaluate or improve them. Instead, the MTE team found that more MFIs are needed for the provision of access to healthy sanitation facilities (toilets and watertight septic tanks) for B40 populations. In addition, the relationship between MFIs and sanitation entrepreneurs has not USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 50 been fully established, as evidenced by the BAIK cooperative’s difficulty in finding sanitation entrepreneurs to fulfill the demand for healthy sanitation construction for its members. FECAL SLUDGE MANAGEMENT (FSM)/WASTEWATER TREATMENT FSM is an area in which IUWASH PLUS is doing well by assisting in passing Perda/Perkada relating to wastewater management and strengthening the capacity of UPTD PAL, although human resources are still limited due to the stigma regarding their work. Another challenge is the limited number of households that desludge, because the community has so few watertight septic toilets. As a result, prospective beneficiaries are reluctant to pay the additional cost of desludging, while residential conditions are congested, making it difficult for desludging trucks to access. Finally, enforcement of the regulations relating to watertight septic tanks is lacking, particularly for existing septic tanks. STAFFING/RESOURCES/TIMING IUWASH PLUS does not have staff with the business and marketing expertise that the program requires. The MTE team notes that this combination of expertise is unlikely to be found among traditional WASH personnel. As such, the project may have been better served by focusing on hiring staff with primarily business and marketing skills to inform the design and implementation of the strategy, with existing IUWASH staff providing orientation and TA in sanitation. IUWASH PLUS has recently hired a new position, that of business-enabling specialist, for this initiative (April 2019). However, this is a headquarters-based position, with the strategy being implemented at the community level. 6.5. RECOMMENDATIONS The sanitation marketing project start was delayed and began with three pilot areas, although in several other areas, the MTE team found that it also began to be implemented. Thus, due to the limited remaining project time, quick-win actions are needed to accelerate results, as follows: BEHAVIOR CHANGE COMMUNICATION • Behavior Change Communication (BCC) should be understood as the driver to effectively reach the target audiences, not only households and communities but also the GOI and the private sector. The IUWASH PLUS team needs to have a common understanding of the BCC strategy, and not only leave it to BC specialists and field facilitators. • Establish better, more robust and uniform implementation in conveying the behavior change (BC) campaign message of “Being a Good Neighbor” in all 32 project areas, especially to the internal IUWASH PLUS staff. This should be done by conducting training with adequate communication materials, such as posters, flyers, print ads, etc., before these are disseminated to LGs, sanitarians and cadres, or at the household level. It would be useless to have excellent BC materials but lack execution in the field due to low levels of knowledge among staff. In addition, the BC campaign should focus on building private sector interest in engaging in the sanitation sector. • The formative research audience segmentation should expand to include subgroups other than B40 communities, particularly regarding safely managed septic tanks. The MTE team recommends that the new business-enabling specialist conduct limited in-depth focus group discussions with moderate-income households to learn more about this market segment. If sanitation entrepreneurs can remain viable by expanding their market, they will continue to be available to address the needs of B40 households. At this time, entrepreneurs are not being retained, partly because they perceive the B40 market to have limited viability. 51 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV HOUSEHOLD FINANCING • At the national level, IUWASH PLUS should continue working with OJK to make sanitation loans into a government program, such as public housing loans for the poor with lower interest rates, and other special terms and conditions. Once this becomes a government-adopted or supported program, it will also become easier for MFIs and local banks to create schemes at the local level that can be taken to scale to meet the significant sanitation loan demand. • IUWASH PLUS needs to assist and collaborate with more MFIs and link them with LGs and entrepreneurs to cover more people who need healthy sanitation. The MTE team also recommends that the project work more with MFIs that have already been developed by other programs (for example, water.org) and already have sanitation credit schemes. • IUWASH PLUS needs to work with its partner MFIs to assess and revise the terms of loans for poor households, especially for group loan models without collateral. However, the length of time it requires to approve a sanitation loan for the first time needs to be shortened, while the loan term also needs to be extended to reduce installment payments. To overcome people’s reluctance to join loan groups, the project should consider developing a savings and loan cooperative model or turn an existing arisan group into a rotating savings and credit association (ROSCA), to shorten the one-year membership requirements. Then people would be able to obtain sanitation loans without collateral and pay in installments. • The MTE team suggests that IUWASH PLUS consider developing innovative ways to overcome the constraints posed by the need for collateral. One option would be to use a sum of money or other collateral owned by sanitation entrepreneurs as collateral for household loans to fund healthy toilets built by sanitation entrepreneurs (back-to-back guarantee mechanism). • IUWASH PLUS needs to identify other type of financing sources beyond MFIs that have worked in the sanitation business and restructure the Microfinance Toolkit to fit the wider audience. SANITATION ENTREPRENEURS/ENTERPRISES • IUWASH PLUS should be more proactive in approaching sanitation enterprises with a clear value proposition and business case, and also provide high-quality market information and market opportunities. Sanitation entrepreneurs/enterprises should be able to see commercial opportunities that would allow them to increase customers and sales in the sanitation market. • IUWASH PLUS should accelerate its activities by forming a business development service (BDS) to act as a center for sanitation business activities that is integrated with other stakeholders, such as LGs, MFIs, sanitarians, cadres and masons, including the conducting of sanitation entrepreneur training to meet the needs of healthy sanitation for the B40 community. The BDS should be appointed from a sanitation entrepreneur/enterprise that has networks and experience related in the sanitation market, to accelerate the process and continue to operate beyond the project period. • IUWASH PLUS should maintain the continuity of sanitation entrepreneurs by improving the recruitment process through the sanitation business inventory process. Entrepreneur candidates should be recruited from those who have a construction background, such as contractors, masons, building-material business owners, and existing sanitation entrepreneurs who have an interest in expanding their business. • IUWASH PLUS should improve the post-training activities/mentoring for sanitation entrepreneurs. Although there have been efforts to connect entrepreneurs with MFIs and sanitarians/cadres in the BC campaign/triggering process, the project can facilitate more strategic and systematic opportunities. For instance, sanitation entrepreneurs could be invited to join the BC campaign and trigger community events to facilitate connecting with households that are interested in building healthy sanitation facilities, as well as linking them to financial resources. USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 52 • IUWASH PLUS should work to support the replication of the Probolinggo Pokja model, which includes entrepreneurial sanitarians who understand the private sector. It should also expand the involvement of the private sector in all Pokja so that the private sector can become a more integral player in identifying market-based solutions toward resolving the sanitation challenges in project target areas. BUSINESS MODELS • IUWASH PLUS should include components of MFIs and sanitation entrepreneurs/ enterprises into its business models. Both belong to the private sector and can serve to accelerate the process of achieving managed sanitation by procuring healthy toilets for the B40 community. • IUWASH PLUS should undertake a pilot project to create a business model as a one-stop service that serves healthy sanitation needs from upstream to downstream, and makes the sanitation business an attractive and sustainable one. FECAL SLUDGE MANAGEMENT (FSM)/WASTEWATER TREATMENT • IUWASH PLUS should propose the enforcement of regulations requiring safely managed watertight septic tanks by applying sanctions where necessary. Without enforcement of the regulations for watertight septic tanks it will be difficult to achieve targets for safely managed sanitation. • IUWASH PLUS should advocate that UPTD PAL provide a mechanism for reducing the cost of desludging through installments, discounts or making it free of charge for the B40 community. • IUWASH PLUS should explore and expand design options with desludging providers, such as the three-wheeled vehicles with suction devices and sludge collectors that can access densely populated urban area. It should also work to ensure that materials to build and repair such systems are locally available. • IUWASH PLUS should suggest training, remuneration packages and rotating staff mechanisms among other UPTDs by LGs to ensure the availability and retention of human resources in UPTD PALs, as well as peer learning. STAFFING • IUWASH PLUS should contract experts with business development skills in the private sector to form a BDS to stimulate pro-B40 sanitation business activities in IUWASH targeted areas, and identify existing potential sanitation businesses that are affordable and accessible by B40. • IUWASH PLUS should review job descriptions and qualifications of key staff involved in the sanitation marketing strategy to ensure that adequate staff resources with the specific skills needed are in place among the three pilot projects. 53 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV 7. EVALUATION QUESTION 4: NATIONAL/ LG POLICY ON B40 AND SDG 6.1 AND SDG 6.2 7.1. DESCRIPTION OF PROJECT ACTIVITY NATIONAL-LEVEL ACTIVITIES TO INFLUENCE POLICY IN SUPPORT OF TARGETING OF B40 AND ACHIEVEMENT OF SDGS 6.1-6.2 All IUWASH PLUS policy work directly or indirectly supports the achievement of Sustainable Development Goals 6.1 and 6.2,72 or population access to safely managed drinking water and sanitation services. As such, IUWASH PLUS policy and regulatory support at the national level cuts across project focus areas, including household access to WASH, institutional strengthening, and WASH financing. At the national level, policy action has focused on four areas: (i) financing; (ii) water access/security (iii) safely managed sanitation; and (iv) the overarching national policy framework of Urban STBM. Financing is critical because it helps lubricate the WASH market system gears (Figure 6). For instance, to achieve the SDGs in Indonesia, public water operators (PDAM) will need to expand their services or improve infrastructure to meet the needs of those currently lacking access to safely managed water services. Current financing options to improve infrastructure are often out of reach. Conventional loans tend to come with high interest rates and shorter repayment periods because incomes of PDAM are partly generated from regular customers rather than businesses, and many PDAM are not financially solvent.73 IUWASH PLUS has facilitated and identified alternative sources of financing to complement and leverage government funding. To this end, it has also supported policies and regulations associated with new financing mechanisms. As discussed in EQ1, financing must ramp up for the project to achieve its targets, as well as for Indonesia overall to achieve SDG 6.1 and 6.2 and its 100-0-100 policy targets. A brief summary follows of the policy work IUWASH PLUS has been engaged in at the national level: POLICIES/REGULATIONS RELATED TO NEW FINANCING MECHANISMS Pool of Funds. This mechanism for financing WASH capital expenditures (i.e., infrastructure) would facilitate loans with lower interest rates and longer repayment periods. The fund, called a Water 72 As defined by the WHO/UNICEF Joint Monitoring Program, SDG 6.1 refers to safely managed drinking water, i.e., drinking water located on premises, available when needed and free from contamination. SDG 6.2 refers to safely managed sanitation and handwashing facilities, i.e., use of an improved sanitation facility that is not shared with other households and where excreta is safely disposed in situ or transported and treated off-site. Improved sanitation facilities include: flush/pour flush to piped sewer, septic tank or pit latrine; composting toilet or pit latrine with slab. SDG 6.2 also addresses the availability of handwashing facilities with soap and water (defined as basic facilities). 73Fifteen of 31 cities/districts reported in APR2 had not achieved full cost recovery. LEAD EVALUATION QUESTION 4: How has IUWASH PLUS influenced national and city-level policy in support of targeting B40 and achievement of SDGs 6.1 and 6.2? USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 54 Revolving Fund, would be capitalized by the GOI and located within provincial development banks. Figure 6 describes selected activities that the project has reported in its approach to influence policy. Regulatory Framework for Business-to￾Business (B2B) procurements issued by the MPWH. Focus of this work has been on creating a regulatory framework for B2B arrangements between the private sector and the PDAM. IUWASH PLUS has conducted research to identify policy and regulatory gaps, and provided technical assistance to DitPSPAM and the Ministry of Public Works and Housing (MPWH) toward the development of a regulation. IUWASH PLUS has also worked to facilitate collaboration between the private sector, including CSRs, and PDAM to link the provision of private funds to improve water supply. Policies/Regulations Related to Financing Mechanisms for Households. IUWASH PLUS has also worked on household-level financing for WASH, with a focus on research, identification and support of financing schemes to help households procure water and sanitation infrastructure. This is particularly relevant to B40 populations, as commercial banks typically focus on productive not consumptive loans, and terms for these loans are often out of the reach of lower-income and poor households. Figure 7 shows some activities that IUWASH PLUS has reported in its approach to influence policy. Water Safety Plans/ Rencana Pengamanan Air Minum (RPAM). Project support to the MPWH, the MOH and the Ministry of Home Affairs (MOHA) in their water safety plan concept, framework and manual. While the declining availability of raw water for domestic use is a critical issue for all Indonesians, it threatens WASH access for the poor and vulnerable disproportionately,74 in part due to their greater dependence on raw water. Key RPAM components include a focus on public health and consideration of water access during natural disasters. The project also worked with Bappenas and the MPWH to develop the concept of a spring vulnerability assessment and action plan (KKMA) 75 that focuses on springs used by PDAM as water sources for their services. IUWASH PLUS works 74 APR1 page 49. 75 Kajian Kerentanan Mata Air. FIGURE 6: PROJECT ACTIVITIES FOR POOL OF FUNDS FIGURE 7: PROJECT ACTIVITIES FOR MICROFINANCE-SUPPORTING POLICIES 55 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV with LGs in 13 locations to support the implementation of KKMA. Fecal Sludge Management (FSM). FSM is a critical component of a safely managed sanitation system. KIs have considered IUWASH PLUS a key player in the development of an FSM framework and overall FSM strategy in Indonesia. This work has involved: research and engagement with, and the study of, other countries’ systems; development of technical guidelines (e.g., for IPLT plants); spatial-based maps of septic tank service areas; trainings/training materials (e.g., for desludging service); and policy/regulation development at the LG level. (See Annex 20 for additional information on local regulations.) Urban STBM. As discussed in EQ1, while STBM was adopted nationwide in 2005,76 IUWASH PLUS has supported the adaptation of this rural model to the urban context of Indonesia. Through a series of meetings and discussions, IUWASH PLUS worked with the MOH to include the Urban STBM context and safely managed sanitation (SDG 6.2) into the revised STBM policy. This review process is reportedly almost complete and is awaiting finalization of the indicator for STBM Pillar 3 (drinking water and food management). This model is also being applied to the IUWASH PLUS project in its hotspot areas. DKI Jakarta Grand Design. In addition to the above policy work, IUWASH PLUS supported the development of the policy framework for DKI Jakarta’s Grand Design for Drinking Water and Domestic Wastewater for 2018-22.77 This consists of a mapping of planned, as well as existing, WASH projects in Jakarta that LG agencies implement, including those supported by external technical assistance. It includes an interactive tool that can identify gaps in coverage to support coordination, collaboration and avoid overlap of efforts. As such, it is intended to increase the effectiveness of planning and implementation of SDG activities, and is supposed to be continually updated. Local Government (City- and District-Level) Activities to Influence Policy in Support of Targeting of B40 and Achievement of SDGs 6.1-6.2. IUWASH PLUS has supported LG policy as part of its strategy to strengthen the enabling environment for increased WASH access under the project’s mandate to bolster LG institutional performance. Typically, both the regional office governance specialist and the local IUWASH PLUS facilitator provide policy and regulation support and technical assistance to the LG. National IUWASH staff based in Jakarta, including the national governance coordinator, support them. The following project activities have been reported by IUWASH PLUS in support of LG policy and regulations for WASH: • Advocated to LGs to accelerate the provision of regulations/policies on water supply and sanitation management. • Established a list of key WASH regulations that LGs should have in place: o Eleven regulations for each LG (two for water supply and six for wastewater management, and three related to gender-responsive planning and budgeting; community engagement mechanisms; and KOTAKU78); and 76 World Bank. 2017. Improving Service Levels and Impact on the Poor: A Diagnostic of Water Supply, Sanitation, Hygiene and Poverty in Indonesia. WASH Poverty Diagnostic. World Bank, Washington, DC. 77 Citation. 78 Kota Tanpa Kumuh, City without Slums. USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 56 o Eight additional program-specific regulations for specific locations. • Conducted an inventory of existing WASH-related regulations for each of the 32 partner LGs and identified priority regulations to advance. • Compared existing regulations with existing policies on WASH and identified improvements needed, if any. • Facilitated critical new or revised WASH regulations by assisting local stakeholders in drafting or revising critical WASH regulations to improve services. • Supported adoption and implementation of new WASH regulations. ACHIEVEMENTS While IUWASH PLUS continues to support a number of national-level activities, as described in the previous section, to date it has reported on completing one national policy framework—the Grand Design for DKI Jakarta. At the LG level, IUWASH PLUS reported on the completion of 47 local regulations supporting WASH services by March 2019 (Table 8), and also reported that it continues to support the development of 31 additional local regulations, as mentioned in QPR10 (December 2018). TABLE 8: NATIONAL AND LOCAL GOVERNMENT POLICIES, FRAMEWORKS AND REGULATIONS DEVELOPED WITH IUWASH PLUS SUPPORT LOP target PY2 target PY1 PY2 PY3 up to 3/19 C4e Number of national-level WASH policies, regulations, guidelines or frameworks developed, strengthened or adopted. 5 1 1 C2e Number of new and/or revised laws, policies, regulations or agreements in place that promote access to improved WASH services. 35 24 3 24 20 7.2. FINDINGS APPROACH TO INFLUENCING NATIONAL-LEVEL POLICY79 Achieving the SDGs. At the national level, the project identifies how policy and regulation work needs to be embedded into its strategy for each of its four components, and it regularly monitors efforts on each front. Review of project reports indicates that the overall approach for identifying WASH policy development has included: 1. Implementation of studies, workshops, study visits, twinning and other efforts for learning and providing information for decision-making; 2. Identification of key gaps (e.g., in WASH sector market development, in policies or regulations); 79 There does not appear to be a WASH policy framework that states what policies are needed at the national level. 57 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV 3. From gap analysis, development of potential solutions, and the policies or regulations needed to address identified gaps; 4. Support to relevant government agencies by jointly developing policies and regulations; and 5. Policy implementation support. Targeting B40 (indirectly). With respect to influencing national policy targeting B40, IUWASH PLUS’ approach has been largely indirect. The examples provided in the previous section (microfinance for household access to WASH, RPAM and Urban STBM) are not explicitly designed to exclusively target the poor; rather, they are expected to disproportionately benefit the poor. This is in line with the indirect approach of the GOI STBM and universal access policies. The former includes the elimination of open defecation as one of its five pillars, while the latter aims to eliminate all slum areas, which are visible evidence of urban poverty (the “100-0-100” targets are 100 percent access to safe drinking water, slum eradication, and 100 percent access to improved sanitation by 2019). While 100-0-100 targets slums, there is no “clearly stated poor-inclusive mission underlying the 2019 universal [WASH] access target”80 for water supply. As a result, B40 households likely have experienced, and will continue to have, delayed access to piped water81 relative to households in higher income levels. The gap in access between the B40 and the top three quintiles (T60) for urban areas was measured at 14 percentage points for access to improved water (i.e., 84 percent vs 70 percent), and at 19 percentage points for access to sanitation in urban areas in 2015,82 as well as safely managed sanitation. While 100-0-100 does not provide clarity on targeting B40 communities beyond a focus on eradicating slum areas, the GOI and various LGs do implement programs that target poor households,83 as described in EQ1. Low-income households could save as much as 5 to 12 percent of their income by paying tariffs instead of buying vendor-supplied water.84 A 2017 World Bank report suggests that, in some cases, LGs may not fully understand regulations on tariff-setting (122/2015), so that subsidized tariffs have been set too low and exacerbate cost-recovery efforts:85 The rationale behind the tariff calculation is generally not well understood by local governing bodies, creating an incentive to push tariffs down rather than allow for cost recovery. In some cities water tariffs for the poor (“social tariffs”) are set so artificially low ($0.03/m³) that even with cross-subsidies from higher-paying customers, PDAM are forced to sell water to the poor at a loss. This ends up disadvantaging the poor because PDAM are discouraged from serving predominantly poor areas. 80 World Bank. 2017. Improving Service Levels and Impact on the Poor: A Diagnostic of Water Supply, Sanitation, Hygiene and Poverty in Indonesia. WASH Poverty Diagnostic. World Bank, Washington, DC. Page 64. 81 Ibid. 82 National Social Economic Survey 2015 data cited in World Bank. 2017. Improving Service Levels and Impact on the Poor: A Diagnostic of Water Supply, Sanitation, Hygiene and Poverty in Indonesia. WASH Poverty Diagnostic. World Bank, Washington, DC 83 Programs targeting the poor mentioned by KIs include: identification of 20 poor households for a sanitation stimulus with support from a community working group and eligibility for Water Hibah grants based on household power voltage and assets. 84 World Bank. 2006. Enabling Water Utilities to Serve the Urban Poor. Washington, DC: World Bank Group. 85 World Bank 2017. USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 58 As 2017 DHS data86 indicate that 48 percent of urban households source their drinking water from bottled or refilled water, there could be ample opportunity for improved cost recovery. In short, while IUWASH PLUS has worked with the GOI to support national WASH policies that indirectly target the B40 population, efforts may have been hindered by a lack of clearly stated GOI policy and strategy supporting equity of WASH access and, possibly, an incomplete understanding of policy implementation at the LG level. APPROACH TO INFLUENCING LG REGULATIONS Working with the WASH working group (Pokja AMPL). At the city and district levels, IUWASH PLUS has generally taken a flexible approach to working with LG partners, since considerable variation exists among the 32 cities/districts with respect to their level of WASH system and governance structure development. One KI described how IUWASH PLUS took an organic approach in its collaboration with LGs, starting with support in the form of information, education and communication on safe water and waste management for a riverside community. From this effort, the project raised awareness among LG staff about the need for regulation to promote sanitation. To support regulation, IUWASH PLUS reconstituted the WASH working group (Pokja AMPL/Pokja Sanitasi), which helped drive forward legislation on domestic wastewater management. From this collaboration, Pokja members learned how both regulations and IUWASH PLUS could support their WASH efforts; regulations “made it easier when we submitted budget requests to central government.” At the same time, as noted in the “Description of Project Activity” section, IUWASH PLUS identified the previously described list of basic WASH regulations necessary for all cities/districts, which suggests that the project also has a methodical approach for identifying priority regulations to support. In addition, IUWASH PLUS designed the sanitation and governance indices to include a regulation component, and thus the presence of key regulations is assessed for each city/district annually as part of a scorecard for LG partners. This multi-pronged approach may help explain the rapid increase in regulations developed and adopted over the first half of the project. Concurrently, the multiple water and sanitation activities that have taken place in each city/district have led to a recognition of the need for additional regulations, or changes to existing ones. For instance, an IUWASH PLUS KI explained that in the past, there were difficulties in installing SPAL DS because the old regulation was for 50 households. As such, IUWASH PLUS supported revisions in the regulations that allowed the construction of these systems for two to 50 households, making it a more viable option in urban settings. Regulations supported in PYI and PY2. IUWASH PLUS shared 27 local regulations with the MTE that the project helped to develop over its first two years. All but one of these address the SDGs directly (the outlier is a guideline that relates to gender-responsive planning and budgeting); 10 of these relate to access to water supply and 16 to sanitation. None appears to set minimum standards of quality (i.e., for drinking water quality or wastewater treatment or management). 86 National Population and Family Planning Board (BKKBN), Statistics Indonesia (BPS), Ministry of Health (Kemenkes), and ICF. 2018. Indonesia Demographic and Health Survey 2017. Jakarta, Indonesia: BKKBN, BPS, Kemenkes, and ICF. 59 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV Eleven regulations address B40 directly or indirectly, and most that do directly relate to the establishment of tariffs or other schemes that include “pro-poor” financing or programming. Figure 8 summarizes how regulations support the attainment of SDGs and targeting B40. (See Annex 20 for a detailed analysis table of the 27 regulations by city/district, as well as EQ1 for additional information related to work targeting B40.) FIGURE 8: REGULATION APPROACH TO ADDRESSING B40 AND ATTAINMENT OF SDG 6.1 AND 6.2 HOW LG REGULATIONS ADDRESS B40 HOW LG REGULATIONS ADDRESS ATTAINMENT OF SDGs ● Sets PDAM tariff based on customer classification, including rates for defined social groups and B4087 households. ● Establishes that financing domestic wastewater management for B40 can be sourced from APBD. ● Establishes that SPALD service fees be based on the principles of affordability, fairness, service quality, cost recovery, transparency and accountability. ● Establishes tariffs and guidelines for tariffs for specific social groups and low-income households for L2T2, drinking water, SPALD. ● Regulates total sanitation implementation through behavioral change and hygiene promotion, i.e., STBM, an approach to increase universal access to improved water and sanitation, to which the poor disproportionally lack access (WHO/UNICEF JMP, 2015). ● Provides assistance for the development of residential environmental sanitation via funding or development for B40 households. ● Establishes that individual and communal SPALD-S financing in low-income areas come from the regional budget or other sources. ● Establishes citywide wastewater treatment, at the level of settlements and low-income communities. SDG 6.1 Access to water ● Develops regional drinking water infrastructure. ● Sets tariffs to support access to drinking water. ● Establishes equity fund for PDAMs to ensure LG support to PDAM by APBD budget allocation. SDG 6.2 Access to sanitation ● Defines customer groups and associated tariffs to support safely managed sanitation. ● Establishes regional technical implementing unit (UPT or UPTD) for wastewater management with structure, functional positions and job descriptions. ● Establishes a comprehensive domestic wastewater management system, including rights, obligations and enforcement. ● Establishes STBM, increasing access to basic sanitation, through behavior change and hygiene promotion. ● Establishes desludging operator agreement in the context of implementing the L2T2 program. GOI AND LG AWARENESS OF IUWASH PLUS POLICY WORK AND THE SDGS Policy support recognized by GOI KIs. When GOI KIs were asked about agency involvement with IUWASH PLUS, some mentioned IUWASH PLUS’ support in drafting the water safety plan to support SDGs and its work with Urban STBM. When asked more specifically about how IUWASH 87 Different terminology was used for regulations, such as “low-income.” USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 60 PLUS has influenced national and local policy and practice supporting B40 and SDGs, one agency mentioned L2T2 and FSM, while others did not mention any policies. Lack of LG awareness about the project’s support to national WASH policies. Similarly, LG KIs were generally not very aware of the scope of the project’s policy work at the national level, but KIs who did recognize that work, mentioned knowledge of the project’s support to Urban STBM and the water safety plan. One IUWASH PLUS KI agreed with the general lack of interest, opining that, “With respect to SDGs, the LG does not care about [them].” That said, one LG KI felt there was a high level of commitment among city governments for water and sanitation, though this had not trickled down to the kecamatan and kelurahan levels. The KI explained that, while no regulations contain specific requirements about reaching B40, scattered programs do target that population, including water and sanitation grants and KOTAKU. Another LG KI believed that more regulations are not needed, and that work needs to focus on implementing or enforcing those that are in place. GOI AGENCY AND LG MANDATE AND ROLES TOWARD ACHIEVING 100-0-100 KIs and document review confirm that the MPWH has direct responsibility for the 100-0-100 program, but this responsibility is split across different levels of government.88 The 100-0-100 program defines specific urban and rural service levels to be met by the end of 2019 for universal access. At the local level, the KI response regarding the LGs’ agency mandate and role toward 100-0-100 was not clear. At a couple of interviews with Pokja AMPLs, the MTE team was told that “all programs [Dinas] are working to achieve this.” It appeared that Perkim, KOTAKU and sanitation (at one site) are local agencies that have a more specific mandate, although no respondents volunteered clear answers to the question. (See also the EQ1 content related to 100-0-100.) 7.3. CONCLUSIONS IUWASH PLUS has influenced national policy in support of achieving SDGs by focusing policy support on key areas that are underdeveloped (e.g., FSM, Urban STBM) or friction points (e.g., financing for households and PDAM capital expenditure). They have indirectly influenced targeting B40 communities for WASH access, though policy work on household-level financing and RPAM largely benefit the poor. The project has also played a significant role in working with LGs on establishing regulations that support the attainment of the SDGs, with most efforts targeted to sanitation, the area where IUWASH PLUS most lags behind in targets (see EQ1). With respect to WASH regulations that are directed at increasing B40 access, the most common have been regulations to establish water or sanitation customer tariff groups that include subsidized rates for poor households, and comprehensive wastewater management policies that include guidance on financing services for low￾income communities. IUWASH PLUS has influenced the development of regulations supporting B40 access to WASH through direct support. Efforts to address city-scale wastewater treatment facilities and other water facilities are also seen to address the poor, because they are to serve all households 88 Ryu SK. Dancing to Different Tunes: Indonesia's Federalism in the Water Sector. Panel paper. American Society for Public Administration 2019 Conference. https://www.aspanet.org/conference2019/panels/presentations/RyuShinKue.pdf 61 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV in the city/district (see EQ1). However, evidence in the literature suggests that this does not necessarily mean the poor will be served without inclusion of specific pro-poor policies (World Bank, 2013 and 2017) (see also EQ1). 7.4. RECOMMENDATIONS HOW CAN IUWASH PLUS BE MORE STRATEGIC IN ITS POLICY EFFORTS? USAID AND IUWASH PLUS • Consider a review of the policy effort to: o Assess how current national and LG policies/regulations were identified for TA, and the extent to which adopted policies/regulations have advanced project objectives, as previously expected. o Find out why the LOP goal set for 35 regulations has so far produced fewer than 11, many of which have already been superseded at mid-project. o Given that IUWASH PLUS has nearly met its target for LG regulation support, decide on which regulations should be prioritized for project support at each LG for the remainder of the project, for the most effective use of resources. IUWASH PLUS • Continue to engage with the GOI (i.e., Bappenas, the MPWH) on development of GOI policy that will follow 100-0-100, since the policy target year is 2019. o Advocate for inclusion of a clear policy on line ministry support for equitable access to WASH, particularly with respect to low-income communities. o Support development and adoption of explicit strategies at the LG level for targeting the B40 population to achieve universal access. o Support the GoI in transitioning from Universal Access (100-0-100) to SDG. • Prioritize support to GOI and LG strategic planning. Regulations function in the context of overall plans for water and sanitation, taking into consideration resources available and associated constraints, as well as the needs of B40 populations. Thus, the MTE team suggests the following actions: o Complete water safety plans for cities/districts that do not yet have them in place, as well as national-level policy support. A recent paper89 investigating the reasons behind only marginal increases and large regional variations in access to piped water found that a structural cause behind the inability to expand PDAM service coverage was the lack of water supply provision, i.e., lack of water to push through the pipe distribution system. Whether this is due to water treatment facilities operating at full capacity or to a scarcity of raw water, Pokja AMPL need a roadmap to guide the management of resources. 89 Ryu SK. Dancing to Different Tunes: Indonesia's Federalism in the Water Sector Panel paper. American Society for Public Administration 2019 Conference. https://www.aspanet.org/conference2019/panels/presentations/RyuShinKue.pdf USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 62 o Support the development of regulations balancing PDAM and deep well exploration (a KI noted that the number of PDAM subscribers has decreased despite regulations against digging deep wells). o Continue to support development of master sanitation planning, such as the SSK. • Collaborate with the GOI and LGs on implementation and enforcement of existing and newly adopted regulations, including: o Enforcing regulations to ensure that septic tank design and construction are watertight; o Setting tariffs that are appropriate for achieving cost recovery; and o Implementation at select learning sites, including how to develop this approach through systematic adaptive learning. • Consider proposing support to the GOI and LGs to study and collaborate on problem-solving friction points, operating within the GOI’s NUWAS framework.90 Friction points that could be targeted: o Policy addressing the lack of bargaining leverage for cities/districts whose raw water source is not on their land. o PDAM incentives issues. o PDAM are not incentivized to work with B40 because of a perception that this would risk their financial standing (i.e., cost recovery). o PDAM are incentivized to run down assets (i.e., infrastructure), as capital expenditure is funded and owned by central government, while O&M is transferred to the PDAM.91 o PDAM may set tariffs too low92 due to political pressures, while cost recovery target tariff levels for B40 (4 percent) are not evidence-based. For potential PDAM IUWASH PLUS￾supported sites with this issue, consider a feasibility for study in which tariffs are restructured with project support (raised for B40) to assess how this affects cost recovery. 8. GENDER An assessment of how the IUWASH PLUS is addressing gender issues within the WASH context was considered, as feasible, within each evaluation question. The MTE also captured the gender of KIs interviewed. Overall, 38 percent of KIs interviewed were female. The team was not able, as originally planned, to disaggregate responses among KIs by gender per evaluation question, because there was overlapped KIs across evaluation question. It was also felt that looking at the results from a gender perspective for this MTE would not be helpful to answering the evaluation questions. 8.1. FINDINGS Overall, the project has dedicated resources to encourage and train women to be more involved in WASH planning activities. Detailed data are provided on the number of training sessions conducted and resources spent on such training. According to annual and quarterly reports and among Pokja KIs and other KIs, the IUWASH PLUS project has involved women in communicating its intervention at the local level, such as during the STBM triggering process through PKK93 or women’s groups and through cadres who are primarily women. This was the area that KIs cited most for women’s 90 National Urban Water Supply framework for all water supply, created with support from the World Bank. 91 UNICEF, Equity in public financing of water, sanitation and hygiene (WASH) Indonesia, pages 32-33. 92 World Bank 2017, page 69. 93 Pembinaan Kesejahteraan Keluarga, family welfare movement. 63 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV involvement. The project also recognizes that women have an important role in decision-making at the household level and thus it has targeted women as receivers of behavior change messages. However, culturally, men appear to retain the primarily decision-making role for large expenditure, such as latrine construction (MFI KI interview). That said, about 50 percent of participant involved in the IUWASH PLUS behavior change informative research indicated that decisions for large purchases were made jointly (IUWASH formative research report, page 80). Women are well represented among IUWASH PLUS staff, in both leadership roles and technical positions, which provides a positive role model for LGs. Outside the project’s focus on including and empowering women’s involvement in WASH, at least one KI mentioned that while IUWASH PLUS’ role has been significant, so has the GOI, as it requires that 30 percent of meetings and groups [participants] comprise women (Tangerang Pokja). Furthermore, the B40 survey does capture the gender of the respondent (husband, wife or other) and their status in the household (head of household or not). The majority of information collected to date related to gender captures the number of women trained and/or participating in activities. 8.2. CONCLUSIONS The project’s major work in this area, as mentioned, is still underway and a review of work related to gender is to be conducted in PY3. Up to now, the project seems to have focused its M&E efforts related to gender on tracking its inputs into improving women’s role in WASH at the LG level, e.g., number of women trained and/or participating in activities. To date, measuring outcomes as a result of those efforts does not appear to be taking place. However, there is an indication such efforts will take place in PY3, for example, capturing lessons learned as part of the project’s Most Significant Change (MSC) process. 8.3. RECOMMENDATIONS The project would benefit from more analysis of the data it collects to assess and measure activity outcomes. This is also relevant to the work on gender and involvement of women in WASH. Capturing the number of training sessions and participation of women in project activities is one measure of engagement, but the larger question should be: What has happened as a result of engaging women in WASH planning efforts? While the MSC effort could provide valuable information, the project could consider other ways to measure the impact of its efforts on WASH gender programming. For example, the project could engage a graduate student to conduct an evaluation or analysis of some of its data from a random sample of sites, against some measure(s) of interest, to measure the impact of its efforts. USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 64 ANNEX I: SCOPE OF WORK MIDTERM EVALUATION OF IUWASH PLUS PROJECT INTRODUCTION The USAID Indonesia Urban Water, Sanitation and Hygiene 'Penyehatan Lingkungan Untuk Semua' (IUWASH PLUS) project is a five-year, $39.6 million initiative designed to assist the Government of Indonesia in increasing access to drinking water supply and safely managed sanitation services as well as improving key hygiene behaviors among urban poor and vulnerable populations. Implemented by DAI Global LLC (under USAID contract AID-497-TO-16-00003), IUWASH PLUS works with governmental and donor agencies, the private sector, NGOs, communities and others to achieve the following "high level" results; (i) Increased access to improved water service quality to at least one million urban residents, of which 500,000 are from the poorest 40% of the population; and (ii) Increased access to safely managed sanitation services for at least 500,000 urban residents. USAID/Indonesia has requested the Monitoring Evaluation and Support Program (MESP) to conduct Midterm Evaluation of IUWASH Plus to assess and analyze the project achievements, impacts, and progress towards project goals; to examine existing monitoring tools and marketing approaches; and to examine the extent to which the project contribute to GOI and SDG goals. In addition, the evaluation will discuss about lessons learned and inform USAID’s global learning agenda on urban Water, Sanitation ACTIVITY DESCRIPTION IUWASH PLUS focuses on strengthening the overall urban WASH Ecosystem. The concept of a WASH Ecosystem recognizes that all sector actors— from Mayors to utilities to health center staff to private sector septage haulers— have an important role to play and that the linkages between them must be supported and reinforced. This concept stands in contrast to most traditional approaches to sector development that prioritize the construction of new facilities while ignoring the systemic weaknesses that undermine sustainability. To ensure that improvements in access to WASH services are sustained, IUWASH PLUS focuses on strengthening service delivery systems so they can more effectively reach the poorest and most vulnerable segments of the population. In order to achieve this at scale, IUWASH PLUS undertakes activities through four interrelated components, including; (i) Improving household WASH services; (ii) Strengthening city WASH institutional performance; (iii) Strengthening the WASH financing environment; and; (iv)Advancing national WASH advocacy, coordination and communication. In support of the above, IUWASH PLUS also manages a Local Sustainability and Innovation activity that provides an opportunity to push the boundaries of WASH programming in Indonesia through the engagement of diverse stakeholders in open innovation approaches that build local capacity. With an established technology sector, a burgeoning maker movement, and a multitude of academic institutions, Indonesia is fertile ground for such home-grown solutions. IUWASH PLUS works with 32 local governments across Indonesia in the populous provinces of North Sumatra, West Java, Central Java, East Java, South Sulawesi, as well as the eastern provinces of Maluku, North Maluku, and Papua, as well as DKI Jakarta and Tangerang Districts. 65 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV BACKGROUND AND DEVELOPMENT HYPOTHESIS To ensure that improvements in access to WASH services are sustainable, IUWASH PLUS is guided by development hypothesis that focuses on strengthening the systems to deliver these services as a whole, which includes reducing barriers to access to the poorest and most vulnerable and strengthening the underlying enabling environment. The project is divided into four components – household WASH services and products (Component 1), city/district WASH institutions (Component 2), national WASH financing (Component 3), and national WASH advocacy and coordination (Component 4). INFORMATION SOURCES USAID suggests the following materials for the IUWASH PLUS Midterm Evaluation desk review: 1. IUWASH PLUS Contract and technical program description from the Project Agreement 2. IUWASH PLUS AMEP 3. IUWASH PLUS Quarterly and Annual Progress Reports 4. IUWASH PLUS Work Plans 5. RIG Audit Report 6. Other relevant project document and reports from GoI and partners, including local and national government, water utility, local NGOs and donor organizations 7. Other IUWASH PLUS technical materials, e.g., case studies, factsheets, infographics, profiles, program briefs, technical briefs, et al. EVALUATION PURPOSE, AUDIENCE AND INTENDED USES Purpose The purposes of this midterm evaluation are to: (1) validate of existing monitoring tools utilized over the first phase of the project to determine if they require course correction in the remaining years of the project; (2) assess of IUWASH PLUS’ contribution to achievement of GOI’s target of universal access (100-0-100) by 2019 and Sustainable Development Goal (SDG) 6 -- Water and Sanitation for All by 2030; and (3) identify key factors necessary to sustain and scale-up IUWASH PLUS approaches beyond the project period. Audience and Intended Uses This midterm evaluation is being undertaken to analyze the IUWASH PLUS project performance to date and obtain recommendations on improvements needed for the project to meet its intended purpose. The evaluation conclusions and recommendations are to be used by USAID/Indonesia’s the USAID/Indonesia ENV Office and the implementing partner to strengthen project implementation. Findings will enable all project stakeholders, including GOI, to understand IUWASH PLUS strengths as well as areas where technical, methods could be improved and strategic approaches refocused. This evaluation will therefore serve to provide information on project performance, program methods, approaches, and sustainability. Furthermore, this evaluation is intended to inform USAID’s global learning agenda on urban Water, Sanitation and Hygiene. This midterm evaluation will be a performance evaluation as defined in the USAID Evaluation Policy (Appendix A). All evaluation materials will be posted to the USAID Development Exchange Clearinghouse (DEC). USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 66 EVALUATION QUESTIONS To guide this evaluation, USAID has identified key questions (below) and lines of inquiry in Appendix B 1. IUWASH Plus Targeting and Approaches. To what extent is the IUWASH PLUS approach reaching the B40 with WASH services? 2. Sustainability. Are the sustainability checklists an effective tool to measure indirect attribution of results? If not, how might IUWASH PLUS improve appropriate attribution results? 3. Sanitation Marketing Approaches. To what extent are IUWASH PLUS sanitation marketing approaches effective in building multi-sectoral interest in this market-based approach? 4. Alignment with GoI Policy. How far has IUWASH PLUS influenced national and city level policy and practice in order to put Indonesia on track to target B40 and to achieve SDGs 6.1- 6.2? GENDER CONSIDERATION In accordance with USAID’s Automated Directive System (ADS) 201 point 7, the research design for this evaluation will consider gender-specific of IUWASH PLUS. The evaluation team will explore gender aspects of the activity per the questions and data sources in Appendix C. DATA COLLECTION AND ANALYSIS METHODS The evaluation team will propose a comprehensive Evaluation Design and Work Plan to address the evaluation questions, including setting criteria for site visits and field data collection. The objective will be to maximize the evaluation team’s ability to develop evidence-findings, conclusions, and recommendations that address the purpose and objectives of this evaluation. Within this Design, the evaluation team will propose the best and most rigorous methods for data collection appropriate to address the evaluation questions. The evaluation team will conduct a Team Preparation Meeting during the first week in country to meet with USAID, discuss the evaluation SOW, and prepare detailed data collection and analysis methods. Appendix D provides an illustrative “Illustrative Analytical Framework for the Evaluation” matrix, including a range of potential data collection methods that may be suitable for each evaluation question. This matrix will be further developed by the evaluation team in the Evaluation Design. DELIVERABLES The evaluation team will be responsible for the following deliverables. Specific due dates will be proposed in the Evaluation Design, following the TPM. Deliverable Estimated Due Date 1. Evaluation Design and Work Plan draft, including detailed research methodology, drafts of data collection instruments, sampling plan, and implementation plan & schedule 1. One week after completing the TPM in Jakarta and associated meetings with USAID and the IUWASH PLUS implementing partner 2. Final Evaluation Design and Work Plan 2. One week after receiving USAID comments 3. Draft Midterm Evaluation Report 3. Within three weeks after completion of fieldwork 67 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV 4. Oral presentation(s) to USAID of key findings and any preliminary conclusions and recommendations. 4. Within four weeks after completion of fieldwork 5. Final Midterm Evaluation Report 5. Two weeks after receiving last USAID comments on the Draft Evaluation Report and associated materials REPORTING AND DISSEMINATION The format of the evaluation report should follow USAID guidelines set forth in the USAID Evaluation Report Template (http://usaidlearninglab.org/library/evaluation-report-template) and the How-To Note on Preparing Evaluation Reports (http://usaidlearninglab.org/library/how-note￾preparing-evaluation-reports). Evaluation team members will be provided with the USAID’s mandatory statement of the evaluation standards they are expected to meet (see Appendix A) TEAM COMPOSITION The suggested composition of the IUWASH Plus Indonesia midterm evaluation team would be four people: • Team Leader, international • Market Development Specialist, International/Indonesian • Behaviour Change Specialist, Indonesian • WASH Monitoring and Evaluation Specialist, Indonesian • Research Assistant, Indonesian Each Team member will have writing responsibilities for the Evaluation Report draft and final version, per assignments by the Team Leader. All team members should have the following qualifications: • Strong knowledge of Indonesia and the Indonesian WASH sector. • Expertise in program evaluations, including qualitative & quantitative evaluation practices. • Knowledge of USAID programming practices. • For the Team Leader, prior successful experience in leading evaluation or research teams. • Excellent writing and inter-personal communication skills. USAID PARTICIPATION Regular communication between the evaluation team and the designated USAID Activity Manager will be essential to the successful execution of the IUWASH PLUS midterm evaluation. The evaluation team will keep USAID apprised of changes and developments that necessitate any significant decision-making or modification of the approved evaluation design. Possible USAID participation in the data collection phase of the evaluation will be discussed in the TPM, prior to the start of fieldwork. USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 68 SCHEDULING AND LOGISTICS USAID Indonesia has requested MESP to finalize the SOW and implement this midterm evaluation of IUWASH Plus, including handling all logistics. The chart below presents an estimated timetable for this task. Estimated IUWASH PLUS Midterm Evaluation Timeline Task/ Deliverable February 2019 March April May June SOW and Budget Approval and recruitment Evaluation Preparation and TPM Evaluation Design & Work Plan Desk Review Fieldwork In-Country USAID Debrief(s) Analysis & Report Writing Draft Evaluation Report & Other Deliverables Final Eval Report & Deliverables SCHEDULING AND LOGISTICS MESP will submit to USAID an estimated budget for this midterm evaluation, once the SOW is agreed. 69 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV APPENDIX A: USAID EVALUATION POLICY APPENDIX I USAID EVALUATION POLICY, APPENDIX 1 CRITERIA TO ENSURE THE QUALITY OF THE EVALUATION REPORT • The evaluation report should represent a thoughtful, well-researched and well organized effort to objectively evaluate what worked in the project, what did not and why. • Evaluation reports shall address all evaluation questions included in the scope of work. • The evaluation report should include the scope of work as an annex. All modifications to the scope of work, whether in technical requirements, evaluation questions, evaluation team composition, methodology or timeline need to be agreed upon in writing by the technical officer. • Evaluation methodology shall be explained in detail and all tools used in conducting the evaluation such as questionnaires, checklists, and discussion guides will be included in an Annex in the final report. • Evaluation findings will assess outcomes and impact on males and females. • Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.). • Evaluation findings should be presented as analyzed facts, evidence and data and not based on anecdotes, hearsay or the compilation of people’s opinions. Findings should be specific, concise and supported by strong quantitative or qualitative evidence. • Sources of information need to be properly identified and listed in an annex. • Recommendations need to be supported by a specific set of findings. • Recommendations should be action-oriented, practical, and specific, with defined responsibility for the action. USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 70 APPENDIX B: EVALUATION QUESTIONS AND LINES OF INQUIRY Evaluation Question Lines of Inquiry 1. To what extent is the IUWASH PLUS approach reaching the B40 with WASH services? ● Currently, IUWASH PLUS uses TNP2K definition for targeting B40, however the GOI has multiple standards of poor and vulnerable across different sectors. Which GOI definition should the project adopt, if any, to improve the uptake of project data, analysis and recommendations by GOI? ● Given that IUWASH PLUS utilizes multiple approaches to reach B40 population, what should be the balance of level of effort between its indirect approach (e.g., technical assistance to water utilities and sanitation operators) and direct approaches (e.g., hotspot, master meter, communal septic tanks, microfinance) in order to reach the most B40 in the most sustainable manner? (Efficiency) ● Compare various approaches ability to reach B40 (hotspot, master meter, communal septic tanks, microfinance, etc)? ● Validate B40 protocol and tool ● Compare protocol to local government protocol (single criteria: HH electricity connection capacity) ● Document cost of B40 targeting and verification 2. Are the sustainability checklists an effective tool to measure indirect attribution of results? If not, how might IUWASH PLUS improve appropriate attribution results? ● Audit use of the tool ● Validate sustainability checklists theory of change ● Comment on external perceptions of the approach 3. To what extent are IUWASH PLUS sanitation marketing approaches effective in building multi￾sectoral interest in this market-based approach? ● What is still needed for targeted city and district governments to understand and utilize the concept of sanitation marketing? ● What additional activities should IUWASH PLUS undertake in order increase domestic resource mobilization and private sector investment to sustain and scale-up this market-based approach? 4. How far has IUWASH PLUS influenced national and city level policy and practice in order to put Indonesia on track to target B40 and to achieve SDGs 6.1-6.2? ● N/A 71 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV APPENDIX C: ILLUSTRATIVE GENDER ASPECTS OF EVALUATION QUESTIONS Evaluation Question Possible Gender Disaggregation Required 1. To what extent is the IUWASH PLUS approach reaching the B40 with WASH services? • Gender disaggregated perception data from different stakeholder groups related to B40 approaches (direct and indirect approach) 2. Are the sustainability checklists an effective tool to measure indirect attribution of results? If not, how might IUWASH PLUS improve appropriate attribution results? • Gender disaggregated perception data from different stakeholder groups related to sustainability of IUWASH PLUS 2. To what extent are IUWASH PLUS sanitation marketing approaches effective in building multi￾sectoral interest in this market-based approach? • Gender disaggregated perception data from different stakeholder groups related to the effectiveness of IUWASH PLUS sanitation marketing approaches 3. How far has IUWASH PLUS influenced national and city level policy and practice in order to put Indonesia on track to target B40 and to achieve SDGs 6.1-6.2? • Gender disaggregated perception data from different stakeholder groups related to the contribution of IUWASH PLUS to the GoI policy USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 72 APPENDIX D: ILLUSTRATIVE ANALYTICAL FRAMEWORK FOR THE EVALUATION - GETTING TO ANSWERS Evaluation Question Data Source Data Collection Methods Sampling or Selection Plan Data Analysis Methods 1. To what extent is the IUWASH PLUS approach reaching the B40 with WASH services? • M&E plan • Annual and Quarterly Report • RIG Audit Report and other relevant documents from GoI and partner (including local & national government, water utility, local NGOs and donor organizations) • IUWASH PLUS technical report • Primary data (focus group, in depth interview with key stakeholders) • Desk review • Key informant interview (structured interview/semi structured interview/FGD) • Purposive sampling • Quantitative and qualitative analysis 2. Are the sustainability checklists an effective tool to measure indirect attribution of results? If not, how might IUWASH PLUS improve appropriate attribution results? • M&E plan • Annual and Quarterly Report • RIG Audit Report and other relevant documents from GoI and partner (including local & national government, water utility, local NGOs and donor organizations) • IUWASH PLUS technical report • Primary data (focus group, in depth interview with key stakeholders) • Desk review • Key informant interview (structured interview/semi structured interview/FGD) • Purposive sampling • Quantitative and qualitative analysis 73 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV Evaluation Question Data Source Data Collection Methods Sampling or Selection Plan Data Analysis Methods 3. To what extent are IUWASH PLUS sanitation marketing approaches effective in building multi￾sectoral interest in this market-based approach? • Annual and Quarterly Report • GoI and partner documents (including local & national government, water utility, local NGOs and donor organizations) • IUWASH PLUS technical report • Primary data (focus group, in depth interview with key stakeholders) • Desk review • Key informant interview (structured interview/semi structured interview/FGD) • Purposive sampling • Quantitative and qualitative analysis 4. How far has IUWASH PLUS influenced national and city level policy and practice in order to put Indonesia on track to target B40 and to achieve SDGs 6.1-6.2? • Annual and Quarterly Report • GoI and partner documents (including local & national government, water utility, local NGOs and donor organizations) • IUWASH PLUS technical report • Primary data (focus group, in depth interview with key stakeholders) • Desk review • Key informant interview (structured interview/semi structured interview/FGD) • Purposive sampling • Quantitative and qualitative analysis USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 74 ANNEX 2: SOURCES OF INFORMATION LIST OF KEY INFORMANT INTERVIEWS AND INTERVIEW SCHEDULE DATE NAME OF KEY PERSON POSITION / INSTITUTION Mar 13, 2019 Ir. Desrah, MT Section Head of Technical Support / Ministry of Public Works and Housing (KPUPR) Usman Madikarim Advisor / Ministry of Public Works and Housing (KPUPR) Eliza Bhakti Amelia, ST, M.S.E Subsection Head Policy Recommendation Development Facilitation / Ministry of Public Works and Housing (KPUPR) Ir. Dodi Krispatmadi M. Ev. E Director of Environmental Sanitation Development / Ministry of Health Dr. Imran Agus Nurali Sp.KO Director of Environmental Health / Ministry of Health Indah Ministry of Health Tri Dewi Virgiyanti, ST. MEM. Director of Urban Areas, Housing and Settlement / National Planning Agency (Bappenas) Aisyah Staff or Urban Areas, Housing and Settlement / National Planning Agency (Bappenas) Mar 14, 2019 Alifah Sri Lestari Deputy Chief of Party for Programs/ USAID IUWAH PLUS Mar 15, 2019 Kurniawati W. Directorate of Drinking Water Supply System Development / Ministry of Public Works and Housing (KPUPR) Susi MDS Simanjuntak Tyas Puri Rohmawati Tiasti Wening Ade Syaiful Mar 19, 2019 Febriant Abby BC/Social Mobilization Specialist / USAID IUWASH PLUS Benny Djumhana Water and Sanitation Financing Advisor Component 3 / USAID IUWASH PLUS Allen Hollanbach Director of Environment Asia Region - DAI Ika Fransisca BC and Marketing Advisor Component 1 / USAID IUWASH PLUS Mar 20, 2019 Ir. Hj. Erna Hernawati, MM. MBA Head of Bappeda Kota Bogor Tetty Permatasari Head of Section – Environmental Pollution Control and Conservation / Environmental Office Kota Bogor Inolasari Head of Sub-directorate – Housing and Settlement Planning / Bappeda Kota Bogor Rina Apsari Argarini Head of Section – Water and Water Waste / Office Public Works and Housing Kota Bogor Dedi Mulyadi Staff / PDAM Kota Bogor Febby L. Staff / Bappeda Kota Bogor Hastuty Handayani Staff / Bappeda Kota Bogor Nurisnindah WASH Facilitator – Kota Bogor / USAID IUWASH PLUS Iwan Setiawan WASH Coordinator / Koperasi Baytul Ikhtiar, Kota Bogor 75 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV Titin Prasetyawati Coordinator / Koperasi Baytul Ikhtiar, Kota Bogor Euis Customer / Koperasi Baytul Ikhtiar, Kota Bogor Mar 20, 2019 Cecep S. Policy Analyst / Bappeda Kabupaten Tangerang Erwin Mawandy Head of Department / Bappeda Kabupaten Tangerang Imam Sutopo Member of Pokja AMPL / Kabupaten Tangerang M. Heru H. Head of Section – Settlement Quality Improvement / Office of Housing, Settlements, and Cemetery Kabupaten Tangerang Sri Indriastuti Head of Health Section / Health Office Kabupaten Tangerang Ardiyah S. Head of R & D Unit / PDAM TKR Kabupaten Tangerang Oka Dwi A. Staff of R & D Unit / PDAM TKR Kabupaten Tangerang Abd. Goni Sofjan UPT PALD / Kabupaten Tangerang Efi Indarti Functional Staff / Bappeda Kabupaten Tangerang Hilda In Staff / Bappeda Kabupaten Tangerang dr. Manik Head of Public Health / Health Office Kabupaten Tangerang Sc. Harum Monitoring & Evaluation Team / Kayu Bongkok Village, Kabupaten Tangerang Jubaedah Elah Mulyani Wanah KSM UMS / Kayu Bongkok Village, Kabupaten Tangerang Zainal Arifini Sanitarian / Posyandu Melati V – Kayu Bongkok Village, Kabupaten Tangerang Uki Suharma Head of Sanitation Entrepreneur Group / Kayu Bongkok Village, Kabupaten Tangerang Samroh Monitoring & Evaluation Team / Kayu Bongkok Village, Kabupaten Tangerang Enung R. Monitoring & Evaluation / Sarakan Village, Kabupaten Tangerang Mairah Samrotul Jaria Siti Jaenabun Ratu Imas Julfaidah Suhaemi WASH Facilitator – Kabupaten Tangerang / USAID IUWASH PLUS Mar 21, 2019 Nuraeni Development Division / Regional Secretariat dr. Tiur F. Pohan Head of Section – Work and Sport Health Environment / Health Office, Kota Depok Rr. Aninta S. Functional Staff / Bappeda Kota Depok Wahyu Hidayat Accelerated Sanitation Development Program – Clean Water / Office of Housing Settlement, Kota Depok Indra Kusuma Accelerated Sanitation Development Program – Waste Water / Office of Housing Settlement, Kota Depok Burhanudin Head of Technical Implementation Unit Septage Treatment Plant / Office of Housing Settlement, Kota Depok Sujadi Operational Director / PDAM Kota Depok Nurhasan Staff / PDAM Kota Depok Mulyadi Staff / PDAM Kota Depok Dhika Prabu Staff / PDAM Kota Depok USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 76 Akhmad Mukhsin WASH Facilitator – Kota Depok / USAID IUWASH PLUS Mar 22, 2019 Dias Manda Engineer / Rototama Berlianplast Brendy Febrian Sales Executive / Rototama Berlianplast Mar 26, 2019 Muchlis Head of Division V – Waste Water / Perumda Air Minum, Kota Surakarta Bayu Tunggul P. Head of Section – Information and Documentation Management Control / Perumda Air Minum, Kota Surakarta Luluk S. Head of Section – Drinking Water and Waste Water / Office of Public Works and Spatial Planning, Kota Surakarta Saryanto Head of Division / Office of Housing Settlement, Kota Surakarta Gunawan AP Head of Division – Infrastructure and Regional Development / Bappeda Kota Surakarta Suratna Secretary / Bappeda Kota Surakarta Arif Dwi W. Head of Section – Environmental Health / Health Office, Kota Surakarta Ratna Tri Utami Institutional Assistant / KOTAKU Indah Warniati Environment Office Dwi Anggraheni Behavior Change Specialist / USAID IUWASH PLUS, Central Java Regional Office Iwan Triteny S Head of Division – Economy and Infrastructure / Bappeda Kota Surakarta Dewi Prasetyanti Staff / Bappeda Kota Surakarta Joko Soeparno Head of Bappeda / Bappeda Kota Surakarta Yulis Hindarwati Head of Section – Women Empowerment / Women Empowerment and Child Protection Office, Kota Surakarta Handini Rahayu Head of Department / Housing Office, Kota Surakarta Sri Rahayu P. Staff / Bappeda Kota Surakarta Sri Purwanti Functional Staff / Bappeda Kota Surakarta Wisnu Hidayat Head of Section – Environment and Building Arrangement / Public Works and Housing Office, Kota Surakarta Ir. Hery Nurhianto, MM President Director / PD BPR Bank Magelang Taufik Hidayat, SE Director / PD BPR Bank Magelang Ary Hartanto Head of Subsection – Credit / PD BPR Bank Magelang Sudino Tri Y. Jefry Budiman Regional Manager / USAID IUWASH PLUS, Central Java Regional Office Agung Kurniawan WASH Facilitator – Kota Surakarta / USAID IUWASH PLUS Mar 27, 2019 Umi Nadiroh Head of Section – Settlement and Environmental Health / Housing and Settlement Office, Kota Magelang Pujiati Head of Section – Environmental Health / Health Office, Kota Magelang Eka Ariani Women Empowerment and Child Protection Office, Kota Magelang Resi Iswara Staff – Supervision and Control of Space Utilization / Public Works and Spatial Planning Office, Kota Magelang Raymundus Dachi Environmental Health / Public Works and Housing Office – Cipta Karya, Kota Magelang Ragil Budi R. Planning Staff / PDAM Kota Magelang 77 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV Parjiyo Head of Subsection – Planning / PDAM Kota Magelang Sukadi KOTAKU Agus Pratanto Health Office / Kota Magelang Widoyoko Head of Sub-division – Settlement / Bappeda Kota Magelang Sobron Environment Office, Kota Magelang Abdul Muhtar WASH Facilitator Kota Magelang / USAID IUWASH PLUS Mar 28, 2019 Hendriana Ywangtini Head of Section – Housing Planning / Housing and Settlements Office, Kota Salatiga Nofi Eko W. Head of Section – Facility of Residential Area / Housing and Settlements Office, Kota Salatiga Ilham Sulistyana Head of Technical Division / PDAM Kota Salatiga Kaerudin WASH Facilitator Kota Salatiga / USAID IUWASH PLUS Juniris Head of Section – Settlement / Housing and Settlements, Kota Salatiga Mar 26, 2019 Aris P. Office of Housing and Settlements, Kabupaten Lumajang Tulus Triwibowo PDAM Kabupaten Lumajang Sukirno KPSPAM Association Kabupaten Lumajang Arie Kusumayanti Environment and Community Participation / Environment Office, Kabupaten Lumajang Tutik Sudyandini DPMD, Kabupaten Lumajang Irianto Environment Office, Kabupaten Lumajang Budi Purwanto Health Office, Kabupaten Lumajang Ahmad Robich Public Works and Spatial Planning Office, Kabupaten Gandhi Maharto Lumajang Retno Wulan A. Bappeda Kabupaten Lumajang Endah M. Public Works and Spatial Planning Office, Kabupaten Lumajang Sony Suharsono WASH Facilitator Kabupaten Lumajang / USAID IUWASH PLUS Mar 27, 2019 Sri Indriyani Head of Drinking Water Development / Public Works and Housing Office, Kota Probolinggo Bambang Ariyanto Sanitarian / Puskesmas Wonoasih, Kota Probolinggo Akhmad Fauzan Health Office, Kota Probolinggo Erang BT Clean Water Section / PDAM Kota Probolinggo Arief Planning Section / PDAM Kota Probolinggo Holek Staff / Public Works and Housing Office, Kota Probolinggo Sulistyo Triantono P. Sanitarian / Puskesmas Ketapang, Kota Probolinggo Darul Tamam Sanitarian / Puskesmas Jati, Kota Probolinggo Mujiati Sanitarian / Puskemas Kanigasan, Kota Probolinggo Mamik Sri Sanitarian / Puskesmas Sukabumi, Kota Probolinggo Hermaliana Head of Sub-division – Housing and Settlement / Bappeda Kota Probolinggo Soby E. Chayaningati Head of Sub-division – Area Development / Bappeda Kota Probolinggo Galuh Prakofi Staff – Area Development / Bappeda Kota Probolinggo Firman Soelijanto WASH Facilitator Kota Probolinggo / USAID IUWASH PLUS Mar 28, 2019 Mujiaman President Director / PDAM Kota Surabaya USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 78 Robi Yayasan Investasi Sosial Indonesia (YISI), Kota Surabaya Catur Head of KSM / Penjernihan Dalam, Kota Surabaya Apri Program Manager / Yayasan Investasi Sosial Indonesia (YISI), Kota Surabaya Badik Facilitator / Yayasan Investasi Sosial Indonesia (YISI), Kota Surabaya Laksmi Cahyaniwati Regional Manager / East Java Regional Office, USAID IUWASH PLUS Rudhy Finansyah UWS Specialist / East Java Regional Office, USAID IUWASH PLUS Dwi Wasis Wash Finance Specialist / East Java Regional Office, USAID IUWASH PLUS Imam Suhadi Governance Specialist/ East Java Regional Office, USAID IUWASH PLUS Ristina Aprillia Urban Sanitation Specialist/ East Java Regional Office, USAID IUWASH PLUS Ratih Dewi BC/Marketing Specialist/ East Java Regional Office, USAID IUWASH PLUS Aris Rizqiawan ME/GIS Associate/ East Java Regional Office, USAID IUWASH PLUS Apr 1, 2019 Umar S. Secretary / Bappeda Kabupaten Barru Syamsubaird S. Head of Division / Bappeda Kabupaten Barru H. Amiruddin Planning / PDAM Kabupaten Barru Marawiah Education Office, Kabupaten Barru Suardi A. Head of Sub-division – Area Development / Bappeda Kabupaten Barru Baso Arta Sub-division Housing & Transportation / Bappeda Kabupaten Barru Muh. Amir Head of Division – Socio-cultural and Human Resources / Bappeda Kabupaten Barru Megawati Pamsimas, Kabupaten Barru A. Pananrang Health Office, Kabupaten Barru Mirwan Head of Division – R & D / Bappeda Kabupaten Barru H. Minu Kalibu Badan Amil Zakat Nasional, Kabupaten Barru A. Ratnawaty Service Secretrary / Housing and Settlements Office, Kabupaten Barru Heriani Syamsu Staff / Health Office Kabupaten Barru Hasbi Arsyad Environment Office Kabupaten Barru Nurhayati Aziz Village Chief / Kelurahan Sumpang Binangae, Kabupaten Barru Haslinah WASH Facilitator – Kabupaten Barru / USAID IUWASH PLUS Apr 1, 2019 Jamal PDAM Kota Makassar Kerlinus Bumbungan UPT PALD Kota Makassar M. Arif UPT PALD Kota Makassar Devi Primavera A. PDAM Kota Makassar Imbong Muryanto Bappeda Kota Makassar 79 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV Sulha Kuba Health Office, Kota Makassar Rahmi I. Sain Public Works Office, Kota Makassar Sri Herdiyanti WASH Facilitator – Kota Makassar / USAID IUWASH PLUS Apr 2, 2019 Rieneke Rolos Regional Manager / SSEI Regional Office, USAID IUWASH PLUS Fauzia Melaniawati Urban Water Supply Specialist / SSEI Regional Office, USAID IUWASH PLUS Selviana Hehanusa Urban Sanitation Specialist / SSEI Regional Office, USAID IUWASH PLUS Johannes Valentino BC/Marketing Specialist / SSEI Regional Office, USAID IUWASH PLUS Vicky Mundiahi Municipal Finance/Microfinance Specialist / SSEI Regional Office, USAID IUWASH PLUS Nurlinah Ahmad M & E/GIS Assistant / SSEI Regional Office, USAID IUWASH PLUS Kerlinus Bumbungan Head of UPTD PAL, Kota Makassar Syukro UPTD PAL, Kota Makassar M. Arif Apr 9, 2019 Wahyu Prabowo Head of UPRS Muara Baru, DKI Jakarta Bonar S. Head of Administration Subsection/ UPRS Muara Baru, DKI Jakarta Agung Head of the Facility and Infrastructure Unit/ UPRS Muara Baru, DKI Jakarta Doddy Suparta DKI Coordinator/ USAID IUWASH PLUS Apr 22, 2019 Sofyan Iskandar Institutional Development Advisor/ USAID IUWASH PLUS Bagus Prambodo Governance Specialist/ USAID IUWASH PLUS Budi Raharjo National Coordinator Governance/ USAID IUWASH PLUS Kartika Hermawati Governance/ USAID IUWASH PLUS Apr 23, 2019 Puji Astutik Head of KPP, Kelurahan Sukun RW 02, Kota Malang Susiyantono Member of KPP, Kelurahan Sukun RW 02, Kota Malang Yetty Kusrini Head of RT, Kelurahan Sukun RW 02, Kota Malang Dwi Cahyono Head of RW, Kelurahan Sukun RW 02, Kota Malang Apr 25, 2019 M. Anis Januar Head of Infrastructure and Regional Development Division/ Research and Development Agency, Kota Malang Indhira Dwi Nanda Staff of Infrastructure and Regional Development Division/ Research and Development Agency, Kota Malang April 29, 2019 Ika Fransisca Behavior Change and Marketing Advisor/ USAID IUWASH PLUS Benny Djumhana Water/ Sanitation Financing Advisor/ USAID IUWASH PLUS May 9, 2019 Alifah Sri Lestari Deputy Chief of Party for Programs/ USAID IUWASH PLUS Bill Parente Chief of Party/ USAID IUWASH PLUS USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 80 LIST OF STEERING COMMITTEE (TIM TEKNIS) GOI Day/Date Venue Name Position Institution Wed, 13-Mar Kantor BPPSPAM, Ruang Rapat, Jl. Wijaya I No. 68, Jakarta Selatan Ir. Bambang Sudiatmo, Dipl. SE Head of Supporting Agency for Drinking Water System Development (BPPSPAM) Ministry of Public Works and Public Housing (KPUPR) Wed, 13-Mar Kementerian PUPR, Gedung Cipta Karya, Meeting Room 7th floor, Jl. Pattimura No. 20, Kebayoran Baru, Jakarta Selatan Ir. Dodi Krispratmadi M. Ev. E. Director of Environmental Sanitation Development Ministry of Public Works and Public Housing (KPUPR) Wed, 13-Mar BAPPENAS, Gedung Madiun, 4th floor, Ruang Direktur Perkotaan, Perumahan & Pemukiman, Jl. Taman Suropati, Jakarta Pusat Tri Dewi Virgiyanti, ST. MEM. Director of Urban Areas, Housing and Settlement Ministry of National Development Planning (BAPPENAS) Wed, 13-Mar Kementerian Kesehatan, Gedung Adyatma, 7th floor/ #721, Jl. HR. Rasuna Said Kav 4-9, Jakarta Selatan Dr. Imran Agus Nurali Sp.KO Director of Environmental Health Ministry of Health (KemKes) Fri, 15-Mar Kementerian PUPR, Gedung Cipta Karya, 8th floor, Jl. Pattimura No. 20, Kebayoran Baru, Jakarta Selatan Ir. Agus Ahyar MSc* Dir. Of Drinking Water Supply System Development Ministry of Public Works and Public Housing (KPUPR) *Ir. Agus Ahyar was not able to attend due to last minute need to travel out of Jakarta. The evaluation team met with sub-directors/other staff (complete list of all KIs and meetings will be included in the final report. 81 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV DOCUMENTS REVIEWED AAN Associates. "Indonesia National Sanitation (Sanitasi Total Berbasis Masyarakat) Programme." 2017. Blondal, Jon R, Ian Hawkesworth and Hyun-Deok Choi. “Budgeting in Indonesia.” OECD Journal on Budgeting (Volume 2009/2). DAI. USAID Indonesia Urban Water Sanitation and Hygiene Final Report March 2011 – June 2016, 2016. DAI Global, LLC. USAID Indonesia Urban Water, Sanitation and Hygiene Penyehatan Lingkungan Untuk Semua (USAID IUWASH PLUS) Behavior Change Strategy, 2018. DAI Global, LLC. USAID Indonesia Urban Water, Sanitation and Hygiene Penyehatan Lingkungan Untuk Semua (USAID IUWASH PLUS) Final Report Behavior Change Formative Research, 2018. DAI Global LLC. Indonesia Urban Water Sanitation and Hygiene Penyehatan Lingkungan untuk Semua (IUWASH PLUS) Quarterly Progress Report 1 June – September 2016, 2016. DAI Global LLC. Indonesia Urban Water Sanitation and Hygiene Penyehatan Lingkungan untuk Semua (IUWASH PLUS) Quarterly Progress Report 2 October – December 2016, 2017. DAI Global LLC. Indonesia Urban Water Sanitation and Hygiene Penyehatan Lingkungan untuk Semua (IUWASH PLUS) Quarterly Progress Report 3 January – March 2017, 2017. DAI Global LLC. Indonesia Urban Water Sanitation and Hygiene Penyehatan Lingkungan untuk Semua (IUWASH PLUS) Quarterly Progress Report 4 April – June 2017, 2017. DAI Global LLC. Indonesia Urban Water Sanitation and Hygiene Penyehatan Lingkungan untuk Semua (IUWASH PLUS) Quarterly Progress Report 6 October – December 2017, 2018. DAI Global LLC. Indonesia Urban Water Sanitation and Hygiene Penyehatan Lingkungan untuk Semua (IUWASH PLUS) Quarterly Progress Report 7 January – March 2018, 2018. DAI Global LLC. Indonesia Urban Water Sanitation and Hygiene Penyehatan Lingkungan untuk Semua (IUWASH PLUS) Quarterly Progress Report 8 April – June 2018, 2018. DAI Global LLC. Indonesia Urban Water Sanitation and Hygiene Penyehatan Lingkungan untuk Semua (IUWASH PLUS) Quarterly Progress Report 1 June – September 2016, 2016. DAI Global LLC. Indonesia Urban Water Sanitation and Hygiene Penyehatan Lingkungan untuk Semua (IUWASH PLUS) Annual Progress Report 2 October 2017 – September 2018 Quarterly Progress Report 9 July – September 2018, 2018. DAI Global LLC. Indonesia Urban Water Sanitation and Hygiene Penyehatan Lingkungan untuk Semua (IUWASH PLUS) Quarterly Progress Report 10 October – December 2018, 2019. DAI Global, LLC. USAID Indonesia Urban Water, Sanitation and Hygiene Penyehatan Lingkungan untuk Semua (IUWASH PLUS) "SANITATION MARKETING STRATEGY", 2018. USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 82 DAI Global, LLC. USAID Indonesia Urban Water, Sanitation and Hygiene Penyehatan Lingkungan untuk Semua (IUWASH PLUS) "SANITATION MARKETING STRATEGY: Increasing Access to Safely Managed Sanitation Services in the Context of Urban Indonesia", 2019. DAI Global, LLC. USAID Indonesia Urban Water, Sanitation and Hygiene Penyehatan Lingkungan untuk Semua (USAID IUWASH PLUS) Study Protocol: Behavior Change Formative Research, 2017. DAI Global, LLC. Study: WASH Microfinance and Grant Beneficiary. 2018. “Entrepreneurship in Sanitation, Indonesia.” Enterprise in WASH Policy Brief 2. 2016 "East Asia Pacific Region Urban Sanitation Review Indonesia Country Study." 2013. "Equity in Public Financing of Water, Sanitation and Hygiene (WASH) Indonesia." 2016. Financing WASH: how to increase funds for the sector while reducing inequities. Position paper for the Sanitation and Water for All Finance Ministers Meeting. IRC, Water.org, Ministry of Foreign Affairs of the Netherlands, and Simavi. 2017 Hadipuro, Wijanto. "Indonesia's Water Supply Regulatory Framework: Between Commercialisation and Public Service?" Water Alternatives (2010): 475-491. Hawkins, Peter, Isabel Blackett, and Chris Heymans. Poor-Inclusive Urban Sanitation: An Overview. Washington, DC: International Bank for Reconstruction and Development/The World Bank, 2013. International Bank for Reconstruction and Development. Project Paper of Proposed Additional Loan in the Amouunt of US$300 Million to the republic of Indonesia and Restructuting for a Second Additional Financing for the Third Water Supply and Sanitaiton for Low Income Communities/Community Based Water Supply Project (PAMSIMAS III). The World Bank. 2015. IUWASH Team. Improving Lifestyle and Health A Guide to Urban Sanitation Promotion, 2015. IUWASH PLUS Completed sustainability checklists from PY1 and PY2, all IUWASH PLUS regions. Ministry of Health Decree No. 852/MENKES/SK/IX/2008. National Strategy of Community-Led Total Sanitation. Murigi, P., Stevens, L., Mwanzia, P., and Pasteur, K. Lesson Learnt in Urban Community-Led Total Sanitation from Nakuru, Kenya. 38th WEDC International Conference. Water, Sanitation and Hygiene Services Beyond 2015: Improving Access and Sustainability. Briefing Paper 2174. Loughborough University, UK. 2015 Myers, Jamie. “Urban Community-Led Total Sanitation: A Potential Way Forward for Co-Producing Sanitation Services.” Waterlines Vol. 35 No. 4, 2016: 388-396 Progress on drinking water, sanitation and hygiene: 2017 update and SDG baseline. Geneva: World Health Organization (WHO) and the United Nations Children's Fund (UNICEF), 2017. Ryu, Shin Kue. Dancing to different tunes: Indonesia’s Federalism in the Water Sector. Idaho State University. Sanitation Needs Assessment 2015 – 2019. Urban Sanitation Development Program, 2014. 83 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV Thomas, Evan et al., eds. Innovations in WASH Impact Measures Water and Sanitation Measurement Technologies and Practices to Inform the Sustainable Development Goals. Washington, DC: International Bank for Reconstruction and Development/The World Bank, 2018. USAID. Scaling Market Based Sanitation: Desk review on market-based rural sanitaiton development program. Washington, DC: USAID Water, Sanitation, and Hygiene Partnerships and Learning for Sustainability (WASHPaLS) Project, 2018. USAID/Indonesia. USAID Water and Development Country Plan for Indonesia. Venkataramanan, Vidya, Crocker, Jonny, Karon, Andrew, and Bartram, Jamie. Community-Led Total Sanitation: A Mixed-Methods Systematic Review of Evidence and Its Quality. Environmental Health Perspective, Vol. 126, No. 2, 2018: 026001-1 – 026001-17. Water Supply and Sanitation in Indonesia Turning Finance into Service for the Future. International Bank for Reconstruction and Development/The World Bank, 2015. World Bank. Improving Service Levels and Impact on the Poor: A Diagnostic of Water Supply, Sanitation, Hygiene, and Poverty in Indonesia. Washington, DC: WASH Poverty Diagnostic/World Bank, 2017. USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 84 ANNEX 3: EVALUATION METHODS The evaluation comprised three primary phases, as outlined below. PHASE 1: PLANNING This phase included development of the evaluation design, workplan and initial review of project documents. The evaluation design initiated during team planning meetings (TPMs), with an initial meeting on March 12. Participants included the implementing partner DAI, USAID/Indonesia and USAID’s WASH technical advisor to review the project and refine evaluation questions. All evaluation members, relevant IUWASH PLUS staff and representatives of USAID’s Program and Technical offices participated. The IUWASH PLUS evaluation team continued with TPMs through the week and presented the evaluation design, proposed site selection and workplan to USAID on March 18 at the offices of Management Systems International (MSI). During the TPM week, the team also met with key national-level GOI stakeholders to apprise them of the evaluation and to gain further understanding of the context of the GOI steering committee (the Tim Teknis, or “technical team”) and the roles each of these agencies have in the IUWASH PLUS project (see Annex 22 for Tim Teknis members). These meetings also included collection of initial information specific to each evaluation question to both inform the evaluation design, and selection of communities to visit, as well as to gather national level insight into the evaluation questions. PHASE II: INFORMATION GATHERING DATA COLLECTION METHODS DOCUMENT REVIEW: The evaluation team reviewed IUWASH PLUS documents, spreadsheets and presentations; GOI and partner documents; and other documents related to WASH and/or Indonesia (e.g., USAID’s Scaling Market-Based Sanitation desk review and the World Health Organization’s “Progress on Drinking Water, Sanitation and Hygiene: 2017 Update and SDG Baselines”). KEY INFORMANT INTERVIEWS (KIIS): Data were collected from semi-structured KIIs with multiple stakeholders including USAID; IUWASH PLUS (DAI) staff at the national and regional levels and those based in cities and districts; GOI officials from the Ministry of Public Works and Housing (MPWH) and the Ministry of Health (MOH); local government (LG) agency officials; sanitation entrepreneurs; and KMS representatives. Interviews with team members who did not speak Indonesian were interpreted simultaneously and consecutively. Annex 2 contains a list of sources consulted, including documents and a list of KIIs. SURVEY: Online surveys were conducted in Indonesian through Sogosurvey.com targeting stakeholders that had collaborated together to prepare, construct and maintain WASH infrastructure through sustainability checklists (SCs). The purpose of the survey was to learn about their experience and perspectives working with SCs. Two survey versions were used; one for IUWASH PLUS staff (facilitators and specialists), and the other for all other SC collaborators, who were mostly LG officials. Respondents had a week to respond. The survey was sent to 50 project staff and 56 SC collaborators. Forty IUWASH PLUS staff (an 80 percent response rate) with experience completing sustainability checklists completed the survey; 85 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV 31.7 percent of them were women. Two-thirds (67.5 percent were WASH facilitators, with an additional 32 percent evenly distributed among urban water, urban sanitation and behavior change and marketing specialists, in addition to a construction specialist. Collaborators completed only 25 surveys (a 44.6 percent response rate), including 22 by collaborators who were personally involved with completing sustainability checklists; some had inherited the work from colleagues. The 22 respondents were mostly from local government: 36 percent were from the local housing and settlement office (Perkim), 27 percent were from the regional water utility company (PDAM), 13.6 percent from the public works and public housing office (PUPR),94 13.6 percent were from the health office; and 9 percent were from the regional technical implementing unit (UPTD/UPT). More than half (63.6 percent) were men. TABLE 9: CRITERIA SELECTION FOR MTE SITE VISITS Region Site Sustainability Checklist Process Observation Sanitation Marketing WASH System Employing Sustainability Index West Java and DKI Jakarta Kota Bogor X Oldest Water grant Individual Communal Kabupaten Tangerang Individual Communal Jakarta Sewerage Rusunawa Kota Depok X Water grant Individual East Java Kabupaten Lumajang X Medium Water grant Individual Communal Kota Probolingo Individual Communal Central Java Kota Magelang X Newest Kota Surakarta Individual Communal Sewerage Kota Salatiga CB water supply Individual Communal SSEI Region Kota Makassar Individual Communal Kab Barru * *For Kab Barru, the team worked to focus on the urban approach and the institutional capacity strengthening and regulations of PDAMs (the water utility) and domestic waste efforts (related to EQs 1 and 4). 94 Pekerjaan Umum dan Perumahan Rakyat USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 86 PHASE III: DRAWING CONCLUSIONS DATA ANALYSIS METHODS Overall, where relevant and possible, the evaluation will triangulate findings across data types, sources and respondent groups. Qualitative data from KIIs were cleaned, tagged by respondent type and, for questions that cut across multiple KI groups, organized into Excel spreadsheets to facilitate comparison of responses across groups. For quantitative analysis, descriptive statistical methods were applied to compare IUWASH PLUS progress toward key indicator targets relevant to the evaluation questions. 87 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV ANNEX 4: TOOLS THE LIST OF INTERVIEW QUESTIONS EQ1 – Targeting B.40. Key Informant Interview Guide Date: Kota/ Kabupaten: (list and circle): Province: Evaluation Team Member Name Role Organization/Agency: □ □ □ □ □ □ Person(s) Interviewed (Please be sure to complete sign-in form as well) Name(s) Title Gender (M/F) Evaluation Team Participant(s): Name Role USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 88 EQ1 IUWASH Plus Targeting and Approaches. To what extent is the IUWASH PLUS approach reaching the B40 with WASH services? 1 Can you describe your agency/organizations involvement and role with the IUWASH PLUS project? What has your role been? Dapatkah Bapak/Ibu menjelaskan peran dan keterlibatan dinas ini dengan proyek IUWASH PLUS? Apa saja tugas anda? 2 Please describe how your agency/organization target the poor? Mohon jelaskan bagaimana dinas ini/pemerintah daerah men-targetkan program2nya kepada masyarakat miskin? Provide discount for connection. Very useful to address the poor, as there are 10,000 households who do not have access to water. Partly because of the lack of legality, but also lack of capability to pay, so PDAM provide discount for connection. PDAM expected to reach 100% access through PDAM in 2019 >> As part of this evaluation we are looking to assess to what extent the IUWASH PLUS approach is reaching the B40 (defined as the bottom 40% of the population in terms of wealth). Sebagai bagian dari evaluasi ini, kami diminta untuk mengkaji bagaimana pendekatan yang dilakukan oleh IUWASH PLUS kepada kelompok B40 (atau yang dikelompokkan sebagai kelompok 40% pada piramida terbawah kemiskinan). 2a. Do you think the IUWASH PLUS project is effectively reaching the poor (B40 populations) with WASH services? Menurut Bapak/Ibu apakah IUWASH PLUS sudah cukup efektif dalam menjalankan programnya yang terkait dengan pelayanan air minum, sanitasi dan higyne nya  Yes (Ya)  No (Tidak) 2b. If YES, what makes you say this? If no, why not? Bila YA, mengapa? Bila TIDAK, mengapa? Mohon jelaskan. 2c. What interventions are you are aware of that the project has undertaken to reach the poor? Intervensi dan pendekatan apa saja yang Bapak/Ibu ketahui dari proyek IUWASH PLUS yang menjangkau kelompok miskin kota? (Probe after they respond: consider both direct and indirect interventions, e.g. hot spots, master meter, septic tanks, microfinance, focus on PDAM, etc.)? (Probing: pertimbangkan intervensi langsung maupun tidak langsung; misalnya terkait hotspot, master meter, septik tank, pendanaan mikro maupun PDAM) 2d. Do you think that the IUWASH PLUS interventions are the right interventions to reach the poor? Menurut Bapak/Ibu apakah intervensi yang digunakan IUWASH PLUS sudah tepat dalam menjangkau kelompok masyarakat miskin?  Yes (Ya)  No (Tidak) 89 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV 2e. Why/Why not? Mengapa/mengapa tidak? 3. Are you aware of the IUWASH PLUS’ approach for identifying the poor (B40) population using TNP2K? Apakah Bapak/Ibu tahu tentang pendekatan yang dilakukan oleh IUWASH PLUS dalam mengidentifikasi kelompok miskin kota dengan menggunakan data dari TNP2K?  Yes (Ya)  No (Tidak) 3a If YES, what do you think of this as a method for identifying and targeting the poor? Bila YA, menurut Bapak/Ibu ini adalah pendekatan yang tepat dalam mengidentifikasi dan mengarahkan program pada kelompok masyarakat miskin? 4. What methods does your agency and/or the local government use to identify and target the urban poor? Probe: Are these methods applied to water, sanitation & hygiene services/interventions? Metode pendekatan apa yang digunakan oleh dinas dan atau pemerintah kota/kabupaten dalam mentargetkan program kerjanya pada kelompok miskin kota? Probe: apakah metode ini juga digunakan untuk program pelayanan air minum, sanitasi dan hygiene (WATSAN)? 4a. Do you think the various methods used by the GOI and local government and IUWASH PLUS to identify and target the poor for WASH services are in alignment? Probe: Do they all have the same vision? Are these “targeting” approaches the best way to ensure that the poor receive improved access to water and/or sanitation. Are there other approaches that would be more effective? Menurut anda, apalah menurut Bapak/Ibu metode yang digunakan oleh baik pemerintah maupun IUWASH PLUS dalam mengidentifikasi kelompok miskin kota untuk pelayanan WATSAN sudah saling melengkapi? Probe: Apakah mereka mempunyai visi yang sama? Apakah pendekatan berbagai pendekatan ini sudah cukup untuk menjamin peningkatan pelayanan WATSAN. Apakah ada pendekatan lain yang lebih efektif? 5. In verifying whether or not IUWASH PLUS is effectively targeting the urban poor, they use mWater Mobile Application using a wealth index (definition) questions to determine level of poverty. Are you aware of this application and the way the data is being collected? Yes (Ya) No (Tidak) If YES, how effective you think this method is? Probe: Have you been involved in the development of the tool and/or the assessment process? Dalam mem-verifikasi efektifitas dari program IUWASH PLUS, mereka menggunakan aplikasi mWATER yang menggunakan index kesejahteraan untuk menyusun pertanyaan2 yang menetapkan tingkat kemiskinan. Apakah Bapak/Ibu tahu tentang aplikasi ini dan cara pengumpulan datanya? USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 90 Bila YA, seberapa efektif metode ini Probe: apakah Bapak/Ibu terlibat dalam penyusunan alat ukur ini? 6. In general, is the IUWASH PLUS project sufficiently addressing the needs of women when considering access to water, sanitation and hygiene in its interventions (direct or indirect)? Secara umum, apakah IUWASH PLUS project dalam menjalankan programnya sudah secara tepat memperhitungkan kebutuhan dan akses para wanita terkait WATSAN (langsung maupun tidak langsung)? 7. Is there anything else with respect the IUWASH PLUS Project, specific to reaching the poor that you wish to share with us? Apakah ada aspek lain terkait IUWASH PLUS project yang ingin Bapak/Ibu sampaikan, khususnya terkait dengan upaya menjangkau kelompok masyarakat miskin? 91 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV EQ2 – Sustainability Checklist Key Informant Interview Guide Date: ___/____/2019 Kota/Kabupaten: (list and circle) ___________________________________ Province: ____________ Evaluation Team Member Name Role Organization/Agency □ KSM □ Local Government □ Operator □ Cooperative □ WASH/Sanitation Engineer □ IUWASH Plus Type of system the interviewee(s) have used the Sustainability Check-list for (Circle): Water: Master Metre Communal SPAM Water Grants Sanitation: Individual Communal Small Scale or City Sewage High Rise Public Housing Rusunawa N/A Person(s) Interviewed (Please be sure to complete sign-in form as well) Name(s) Title Gender M / F M / F M / F M / F Have a copy of the completed sustainability checklist to show 1. Can you describe your agency/organizations involvement and role with the IUWASH PLUS project? What was your role? Dapatkah Bapak/Ibu menjelaskan peran dan keterlibatan dinas/organisasi/pemerintah daerah ini dengan proyek IUWASH PLUS? Apa saja tugas Bapak/Ibu? USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 92 EQ2 Do the sustainability checklists provide utility to the project and serve as an effective tool to measure direct attribution of results? 1 Date that sustainability checklist was signed 1. First: __ /___/_____ 2. Second ___/___/_____ Tanggal pada saat sustainability checklist ditanda tangani 1. Pertama: __ /___/_____ 2. Kedau ___/___/_____ 2 Do you remember signing this checklist? Apakah Bapak/Ibu ingat saat menandatangani checklist ini?  Yes (Ya)  No (Tidak) 2a If Yes, What was your role in and the process in completing the checklist? Jika Ya, Apakah peran Bapak/Ibu dalam proses melengkapi dan menyetujui checklist tersebut? 3 Do you think it is/was a useful tool? Why or why not? Apakah menurut Bapak/Ibu checklist tersebut adalah alat bantu yang berguna? Bolehkan dijelaskan alasannya? Probe: Does the tool include all the necessary components/steps related to sustainability of water and sanitation interventions (is anything missing from your perspective)? Probe: Apakah alat ukur ini sudah mencakup semua komponen / langkah yang diperlukan terkait dengan pengelolaan air dan sanitasi berkelanjutan (dari pandangan Bapak/Ibu apakah masih ada yang kurang dan perlu dilengkapi)?  Yes (Ya)  No (Tidak) 4 Have you continued to use the checklist? Apakah Bapak/Ibu untuk seterusnya akan tetap menggunakan sustainability checklist ini?  Yes (Ya)  No (Tidak) 4a If yes, how? Jika iya, Mohon penjelasannya? 5 To what extent have women been involved in the activities corresponding to the checklist, i.e. the planning and implementation of this water or sanitation system? Sejauh mana perempuan dilibatkan dalam kegiatan yang berhubungan dengan penggunaan checklist ini, apakah sudah dimulai dari tahap perencanaan misalnya atau hanya pada tahap implementasi sistem air atau sanitasi saja? How? Bagaimana?  Not at all (Tidak sama sekali)  A little (Sedikit) 93 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV Planning – Terkait perencanaannya Implementation – Terkait Implementasinya  Somewhat (Cukup berperan)  To a great extent (Sangat terlibat) 6 Do you think the checklist contributed to better results for the project you were involved in? Apakah menurut Bapak/Ibu checklist ini memberikan kontribusi dalam memperoleh hasil yang lebih baik untuk proyek dimana bapak/ibu terlibat didalamnya?  Yes (Ya)  No (Tidak) 6a If yes how? If no, why not Mohon penjelasannya! 7 What are the main challenges to ensuring that this WASH system is functional/sustained over the long term to ensure that it can continue to provide access to (drinking water / safely managed toilets). Apa tantangan utama untuk memastikan bahwa sistem WASH ini berfungsi dan berkelanjutan dalam jangka panjang serta memastikan bahwa sistem tersebut dapat terus menyediakan akses kepada masyarakat terkait air minum /toilet yang dikelola dengan baik dan aman – safely managed toilet). Does the sustainability checklist assist in tracking the right components that lead to sustainable WASH programs? If Not what’s missing? Apakah sustainability checklist membantu Bapak/Ibu dalam mendapatkan pendekatan yang tepat sehingga dapat mengarah pada program WASH berkelanjutan? Jika Tidak, apa yang masih kurang dan perlu dibenahi? 8 How does the checklist work? How does the checklist lead the work you do? Bagaimana cara kerja checklist tersebut? Apakah checklist tersebut membuat Bapak/Ibu melakukannya? 9 Do you think that the checklists contribute to making water and sanitation infrastructure projects to be more sustainable? Menurut Bapak/Ibu apakah sustainability checklist ini dapat turut berkontribusi dalam memastikan bahwa proyek infrastruktur air dan sanitasi akan berkelanjutan?  Yes (Ya)  No (Tidak) 10 Probe: Why/Why not? What if anything would you change about the tool or process to improve it? USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 94 Probe: Mengapa / Mengapa tidak? Bila ada yang perlu diperbaiki, apa saja yang akan Bapak/Ibu ubah dari checklist ini? 11 Anything else you think we should know about the sustainability checklist with respect to how the project can increase access to [water and sanitation system] among the poor? Adakah hal lain yang menurut Bapak/Ibu harus kami ketahui tentang sustainability checklist ini sehingga proyek ini dapat meningkatkan keterjangkauan dan akses kelompok masyarakat miskin baik pada air maupun sanitasi? 95 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV EQ3 – Sanitation Marketing Key Informant Interview Guide Date: ____/____/2019 Kota/Kabupaten: (list and circle) __________________________ Province: ______________ Evaluation Team Member Name Role Organization/Agency (Circle): Local Government Sanitation Entrepreneur Micro Finance Sanitarian / Health Officer IUWASH PLUS team Other: ……………………………. Persons being interviewed: Name(s) Title Gender M / F M / F M / F M / F This evaluation is intended to assess IUWASH PLUS Sanitation Marketing Strategy, how it has been implemented to this point and what, if any, changes need to be made to assist the project in meeting its targets over the remaining project period and to inform future sanitation marketing strategies. Evaluasi ini dimaksudkan untuk menilai Strategi Pemasaran Sanitasi dari IUWASH PLUS yang telah diterapkan selama ini, dan apakah (jika ada) diperlukan perubahan dalam upaya membantu mencapai target selama periode proyek yang tersisa dan untuk memberikan masukan mengenai strategi pemasaran sanitasi yang akan datang. USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 96 EQ3 To what extent are IUWASH PLUS sanitation marketing approaches effective in building public-private interest in this market-based approach? (EQ3 Sejauh mana pendekatan pemasaran sanitasi dari IUWASH PLUS efektif dalam membangun minat kemitraan pemerintah dan swasta dalam pendekatan berbasis pasar ini?) 1. Can you describe your agency/organizations/business involvement and role with the IUWASH PLUS project? What is your role? Dapatkah Bapak/Ibu menjelaskan peran dan keterlibatan dinas/organisasi ini dengan proyek IUWASH PLUS? Apa saja tugas Bapak/Ibu? Probe: what is your background as it relates to marketing in general, social marketing, sanitation, etc. Probing: apakah Bapak/Ibu memiliki latar belakang yang terkait dengan marketing secara umum maupun marketing sosial, sanitasi, dll. 2. How would you describe “sanitation marketing” what does it include? Apa pemahaman Bapak/Ibu tentang Marketing Sanitasi itu? Kegiatan apa saja yang termasuk di dalamnya? Probe: marketing tools and approaches to strengthen demand and supply for sanitation product & service Probing: penggunaan teknik dan pendekatan marketing untuk menciptakan permintaan serta memperkuat pemenuhan kebutuhanakan pada barang dan jasa untuk produk sanitasi. 3. What do you think the role of women in sanitation marketing is? Menurut Bapak/Ibu, seperti apakah peran wanita dalam Marketing Sanitasi? Given this, how are women being engaged in sanitation marketing? Mengingat tersebut di atas, bagaimana perempuan terlibat dalam marketing sanitasi? Probe: role in marketing activities, designing and selling toilets, financing sanitation Probing: Peran dalam kegiatan pemasaran, menjual toilet, membiayai komponen sanitasi, mengadvokasi orang lain, mempengaruhi suami, dll. 4. What support has been provided to you by IUWASH PLUS specific to Sanitation Marketing Support Apakah saja yang telah diberikan kepada Bapak/Ibu oleh IUWASH PLUS khususnya terkait Marketing Sanitasi? Probe: Support for capacity building, advocacy, partnership program, etc. Probing: Misalnya, bantuan terkait peningkatkan Kemampuan, advokasi, program kerjasama, dll. 5. Has IUWASH PLUS helped to strengthen sanitation marketing? If yes, What’s been helpful? If not, why not? What support is needed? Apakah IUWASH PLUS membantu Bapak/Ibu dalam memperkuat marketing sanitasi? Jika Ya, apa saja yang hal yang membantu? Jika tidak, mengapa demikian? Support apa yang menurut Bapak/Ibu perlukan belum diberikan dalam hal ini? Probe: What’s the most helpful, can be scaled up beyond project period Probing: Apakah hal yang paling membantu, dan masih bisa di tingkatkan/diaplikasikan setelah proyek selesai.  Yes (Ya)  No (Tidak 97 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV 6. To what extent has the IUWASH PLUS worked with local government programs / policies to accelerate the sanitation market? Sejauh mana IUWASH PLUS bekerjasama dengan program pemerintah daerah untuk mempercepat pemasaran sanitasi ini? Probe: Smart subsidies, availability of financing, and establishment and enforcement of regulations and standards in the sanitation sector Probing: Subsidi pintar, ketersediaan pembiayaan, dan juga regulasi serta aturan hukum dan standardisasi di sektor sanitasi 7. Are there other local government or national initiatives that have also worked to strengthen sanitation marketing? If yes, please describe. Adakah inisiatif lainnya dari pemerintah daerah atau nasional untuk memperkuat marketing sanitasi? Jika ya, mohon Bapak/Ibu bisa jelaskan. Probe: Are they in line with safely managed sanitation, focus area, target, etc. Probing: Apakah inisiatif ini sejalan dengan program safely managed sanitation (sanitasi yang dikelola dengan aman), area fokus, target, dll.  Yes (Ya)  No (Tidak 8. What makes you interested in entering sanitation market? Apakah yang membuat Bapak/Ibu tertarik untuk terlibat dalam ke pasar sanitasi ini? Probe: social responsibility, profit, government programs, etc. Probing: tanggung jawab social, keuntungan, program pemerintah, dll. 9. To what extent does IUWASH PLUS support entrepreneurs in sanitation market Sejauh mana IUWASH PLUS mendukung Bapak/Ibu dalam pemasaran sanitasi ini? CIRCLE RELEVANT QUESTION/LINGKARI jAWABAN YANG TEPAT  to create demand (Probe: What is your opinion of Good neighbor and Super Mom campaign, promotion channel, triggering, etc.) OR/atau Dalam menciptakan permintaan pasar (probing: Apa pendapat bapak/ibu tentang kampanye Good Neigbor/Tetangga yang Baik dan Super Mom, promosi, pemicuan, dll.)  fulfill demand (Probe: Do the product / service suitable with market, support capacity building, competitive price, etc.)? Dalam memenuhi permintaan (Probing: Apakah produk / servis sudah sesuai dengan permintaan pasar, bantuan peningkatan kemampuan, harga yang kompetitif, dll.) 10. [MFI] Are you aware of specific banking products that support increased access to support sanitation market? If Yes, which ones are you aware of? Apakah Bapak/Ibu mengetahui produk perbankan tertentu yang mendukung peningkatan akses untuk pasar sanitasi? Jika Ya, yang mana? Mohon dijelaskan! Probe: Does it refer to internal policy, government regulation, interest rate? Probing: Apakah berkaitan dengan kebijakan internal, peraturan pemerintah, suku bunga?  Yes (Ya)  No (Tidak USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 98 If No, what are some barriers? Jika Tidak, apa saja kendala yang dihadapi? Probe: lack of fund, policy, risk, etc. Probing: Ketersediaan/kurangnya permodalan, kebijakan, resiko, dll. 11. Is there anything else you want to share with us about the IUWASH PLUS sanitation marketing strategy or needs or issues as it relates to increasing access to safely managed sanitation in Indonesia specific to the private market approach? Apakah ada hal lain yang ingin Bapak/Ibu sampaikan kepada kami tentang strategi pemasaran sanitasi IUWASH PLUS atau kebutuhan /masalah lain yang berkaitan dengan peningkatan akses ke safely managed sanitation – sanitasi yang dikelola dengan baik di Indonesia khusus untuk pendekatan pasar swasta? 99 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV EQ4 – Alignment with GoI and City Policy (B40 and SDG 6.1 and 6.2 Guide) Date: ___/____/___ Evaluation Team Member Name Role Organization/Agency □ □ □ □ □ □ Province (Circle): City/Kota/Kabupate (Circle): Person(s) Interviewed (Please be sure to complete sign-in form as well) Name(s) Title Gender (M/F) Evaluation Team Participant(s): Name Role USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 100 We’re interested in knowing if the IUWASH PLUS has influenced any national and city level policies in order to put Indonesia on track to target B40 populations (defined as the bottom 40% of the population in terms of wealth) towards the achievement of SDG 6.1 and 6.2 (universal access to water and sanitation): Kami ingin mengetahui bagaimana proyek IUWASH PLUS menyumbang pada perumusan dari berbagai kebijakan nasional maupun daerah dalam upayanya untuk membantu Indonesia mencapai targetnya menjangkau kelompok miskin kota atau B40 (diartikan sebagai kelompok 40% terbawah dalam tingkat kesejahteraan/keniskinan) sehingga mampu mencapai target SDG 6.1 dan 6.2 (target pemenuhan akses air dan sanitasi). 1 Can you describe your agency/organizations involvement and role with the IUWASH PLUS project? What has your role been? Dapatkah Bapak/Ibu menjelaskan peran dan keterlibatan dinas ini dengan proyek IUWASH PLUS? Apa saja tugas anda? EQ4 How has IUWASH PLUS influenced national and city level policy in support of targeting B40 and achievement of SDGs 6.1-6.2? 2. What policies have your agency/organization developed in collaboration with the IUWASH PLUS Project? (See if related policy documents they may have developed) for your local government/community target area? ● Have any of those policies focused on targeting the B40 Apa saja kebijakan atau peraturan daerah yang telah diterbitkan dengan dukungan atau kolaborasi dengan IUWASH PLUS? (Mengacu pada dokumen Peraturan Daerah yang ada). • Apakah Peraturan/Kebijakan ini mengacu pada kelompok miskin atau B40 3. Has your agency created any new, or revised any, policies with IUWASH PLUS assistance that directly or indirectly address supporting attainment towards achievement of SDG 6.1 and 6.2 (i.e. Universal access 100-0-100) at either the local or national level? Apakah dinas/pemerintah daerah menyiapkan peraturan baru atau merevisi peraturan yang sudah ada bersama dengan IUWASH PLUS yang secara langsung maupun tidak langsung mendukung upaya pencapaian target SDG 6.1 dan SDG 6.2 Probe: At the national level IUWASH PLUS has worked on or is working on* national WASH policies (e.g. water safety plan, FSM framework and implementation, urban STBM, sanitation marketing, microfinance* and pool of funds*) are you aware of these? Probing: Pada tingkat nasional, IUWASH PLUS telah membantu beberapa perumusan kebijakan terkait Air dan Sanitasi dan terus mendorong pembuatan kebijakan baru (misalnya: rencana pengamanan air, manajemen air 101 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV tinja/IPAL, STBM perkotaan, pemasaran sanitasi, keuangan mikro dan pendanaan lainnya) apakah bapak/ibu mengetahui hal terkait perumusan kebijakan ini? 3a If Yes (created and/or aware), discuss the policies, and how they support attainment towards SDG 6.1 and 6.2? Bila YA, mohon sebutkan kebijakan/peraturan yang mana saja yang mendukung pencapaian SDG 6.1 dan SDG 6.2? Probe: How do they help? Are they being applied? Probing: bagaimana kegunaannya? Apakah kebijakan ini telah dijalankan? 4. To achieve SDG 6.1 and 6.2, B40 populations will need to be targeted. Given this, are there any policy level gaps that the project should be focusing on in collaboration with national or local government? Agar mampu mencapai target SDG 6.1 dan SDG 6.2 maka program perlu diarahkan pada kelompok masyarakat miskin (B40). Melihat pada hal ini, menurut Bapak/Ibu apakah ada hal yang belum dan harus diperhatikan oleh IUWASH dalam kerjasamanya dengan pemerintah daerah maupun pusat? 5. There are various policies to identify and target B40 populations in Indonesia. Is your organization/agency mandated or guided by any particular policy related to B40 “the poor”? Ada berbagai kebijakan dalam mengidentifikasi dan menyasar pada kelompok penduduk miskin di Indonesia. Apakah ada pedoman/panduan tertentu bagi dinas atau pemerintah daerah dalam menentukkan sasaran program pada kelompok B40 ini?  Yes (Ya)  No (Tidak) 5a. If Yes, what are the policies? Bila Ya, apa pedoman/panduan yang digunakan? 5b. Has your agency created any or revised any policies and associated methods with IUWASH PLUS assistance that directly or indirectly work to identify and/or target B40 populations for WASH services? Apakah dinas yang Bapak/Ibu pimpin atau pemerintah daerah bekerjasama dengan IUWASH PLUS telah menyusun atau merevisi kebijakan terkait dengan pedoman/panduan untuk mengidentifikasi target B40, baik secara langsung maupun tidak langsung?  Yes (Ya)  No (Tidak) 5c If Yes can you please share with us what those policies/methods are and how they have helped to identify or target B40 populations? If No are there any modifications/improvements that are needed to improve policies to support improved identification and/or targeting of B40 population that the IUWASH project could assist with during the second half of the project period? Bila YA, mohon untuk menginformasikan kepada kami kebijakan/pedoman dalam mengidentifikasi kelompok B40 tersebut. Bila TIDAK, apakah ada modifikasi atau perubahan kebijakan yang telah diidentifikasi yang perlu dilakukan untuk mempertajam upaya mentargetkan kelompok B40, yang mana IUWASH PLUS dapat membantu khususnya pada paruh akhir programnya? USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 102 6 Does your agency have a mandate/policy/guidelines to increase water or sanitation services specifically to the poor? If yes, please describe. If no, do you think there is a need for such policies? Apakah dinas Bapak/Ibu memiliki kewajiban untuk meningkatkan akses air maupun pelayanan sanitasi pada kelompok masyarakat miskin? Bila Ya, mohon dijelaskan Bila Tidak, menurut Bapak/Ibu apakah perlu ada kebijakan ini?  Yes (Ya)  No (Tidak) 7. Indonesia has established the 100-0-100 policy (100% coverage of safely managed water supply and sanitation and elimination of urban slum areas) are you familiar with it? Indonesia telah menetapkan target kebijakan 100-0-100 (target 100% untuk pengamanan air dan sanitasi serta eliminasi daerah kumuh perkotaan) apakah bapak/ibu mengetahui hal ini?  Yes (Ya)  No (Tidak) 7a. Does your agency have a mandate or role to contribute to attainment of the 100-0-100 policy? Apakah dinas Bapak/Ibu juga berperan dalam upaya untuk berkontribusi pada kebijakan pemerintah Indonesia untuk pencapaian 100-0-100 ini?  Yes (Ya)  No (Tidak) 7b. If Yes, please describe the mandate or role: Bila YA, mohon dijelaskan apa saja peran dan tugasnya 8 Is there anything else with respect to national or local policy specific to B40 and the achievement of the SDG 6.1 and 6.2 and the IUWASH PLUS Project that you wish to share with us? Apakah ada hal-hal lain terkait kebijakan nasional atau daerah terkait kelompok masyarakat miskin (B40) dan juga upaya pencapaian target SDG 6.1 dan SDG 6.2 dalam kaitannya dengan IUWASH PLUS project yang ingin disampaikan kepada kami? 103 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV IUWASH PLUS Staff Key Informant Interview Guide Date: ___/____/2019 Kota/Kabupaten: (list) ________________________________ District/Province: ____________________ Agency: ________________________________________ Role Person(s) Interviewed (Please be sure to complete sign-in form as well) Name(s) Title Gender M / F M / F M / F M / F EQ1 To what extent is the IUWASH PLUS approach reaching the B40 with WASH services? Seberapa jauh pendekatan yang dilakukan oleh IUWASH PLUS dalam mengupayakan pelayanan dan keterjangkauan air minum dan sanitasi mampu menjangkau kelompok masyarakat miskin (B40)? 1 Regarding the use of TNP2K for targeting the B40 population, what was the original rationale? Terkait dengan penggunaan data TNP2K dalam menjangkau target populasi B40, apa yang menjadi dasar penggunaan data ini? 2 How did IUWASH PLUS identify the types of methods for increasing access to water and sanitation? For instance, hotspot, master meter, communal septic tanks, microfinance, focus on PDAM. Bagaimana IUWASH PLUS mengidentifikasi berbagai metodologi untuk meningkatkan akses air bersih dan sanitasi? Misalnya, terkait penentuan hotspot, master meter, communal septic tank, micro finance maupun PDAM? 3 What is the cost of targeting and verifying the B40? Berapa biaya yang dikeluarkan untuk melakukan targeting dan verifikasi B40? How long does the verification survey take? Berapa lama waktu yang digunakan untuk melakukan survey/pendataan verifikasi? USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 104 4 To what extent do you feel your GOI and LG partners are committed to reaching the poor with WASH services? Sejauh mana Bapak/Ibu menyadari akan komitmen dari pemerintah Indonesia dan juga pemerintah daerah untuk mencapai target untuk menjangkau masyarakat miskin dalam memberikan pelayanan air dan sanitasi serta hygiene?  Not at all (Tidak sama sekali)  A little (Sedikit)  Somewhat (Cukup)  To a great extent (Sangat baik) 4a Why do you feel this way? Mengapa demikian? EQ2 Do the sustainability checklists provide utility to the project and serve as an effective tool to measure direct attribution of results? Apakah sustainability checklist mampu memberikan kegunaan pada project dan berfungsi sebagai alat ukur keberhasilan pencapaiannya? 5 [Facilitator] Please describe the step-by-step process by which you implement the sustainability tool. Tolong jelaskan langkah-langkah pelaksanaan sustainability checklist. How long does it take? Berapa lama waktu yang diperlukan? How do you identify partners that are interested in collaborating on a WASH system? Bagaimana caranya mengidentifikasi para mitra yang bersedia berkolaborasi untuk mengembangkan system air dan sanitasi? 6 How well do you feel the sustainability tool is working? Menurut Bapak/Ibu sustainability checklist ini berfungsi sesuai peruntukannya?  Not at all (Tidak sama sekali)  A little (Sedikit)  Somewhat (Cukup)  To a great extent (Baik sekali) 6a Why do you feel this way? Mengapa demikian? 7 What is the most difficult aspect of implementing this tool? Apa bagian yang paling sulit dalam menggunakan alat ukur sustainability checklist ini? 8 What is your advice on how to improve the tool or its implementation for best results? Apakah ada saran untuk menyempurnakannya untuk memberikan hasil yang lebih baik? EQ3 To what extent are IUWASH PLUS sanitation marketing approaches effective in building public-private interest in this market-based approach? Seberapa jauh pendekatan pemasaran sanitasi (Maeketing SAnitasi) secara efektif membangun minat public dan korporasi (public private) pada pendekatan berbasis pasar ini? 9 [specialist] What does sanitation marketing mean to you? Apa pengertian pemasaran marketing bagi Bapak/Ibu? 10 Please describe all the different ways that IUWASH PLUS supports sanitation marketing development. 105 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV Mohon dijelaskan berbagai terobosan baru yang digunakan oleh IUWASH Plus dalam mendukung pengembangan pemasaran sanitasi ini? 11 To what extent do your current sanitation activities reflect your strategy? Sejauhmana kegiatan sanitasi Bapak/Ibu sudah mampu merefleksikan strategi program yang dijalankan? 12 What sanitation marketing activities do you believe IUWASH PLUS should focus on most in the next six months? Menurut bapak/ibu, dalam 6 bulan kedepan ini, focus kegiatan pemasaran sanitasi IUWASH PLUS harus diarahkan kemana? EQ4 How has IUWASH PLUS influenced national and city level policy in support of targeting B40 and achievement of SDGs 6.1- 6.2? Bagaimana IUWASH PLUS mampu mendorong tersusunnya kebijakan di tingkat nasional maupun daerah untuk menyasar kelompok B40 dan mendukung pencapaian SDG 6.1 dan 6.2? 13 How has IUWASH PLUS supported or promoted national and city policies to achieve SDGs 6.1 and 6.2? Bagaimana IUWASH PLUS mendukung atau mempromosikan kebijakan nasional dan daerah untuk mencapai target SDG 6.1 dan SDG 6.2? USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 106 ONLINE SURVEY TOOLS For IUWASH PLUS Staff 1. Nama: ____________________________________________________________________ ____________________________________________________________________ 2. Nomor telepon: ____________________________________________________________________ ____________________________________________________________________ 3. Alamat email: ____________________________________________________________________ ____________________________________________________________________ * 4. Posisi Bapak/Ibu: (Pilih satu opsi) Fasilitator WASH (WASH facilitator) Specialis Sanitasi (Sanitation Specialist) Spesialis Air Perkotaan (Urban Water Specialist) Spesialis Air (Water Specialist) Name: _______________________ Phone Number: _______________________ Email: _______________________ Your Position:  WASH Facilitator  Sanitation Specialist  Urban Water Specialist  Water Specialist  Behavior Change & Marketing Specialist  M&E Specialist  Other ____________ 107 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV Specialist Perubahan Perilaku & Pemasaran (Behaviour Change & Marketing Specalist) Specialis M & E (M & E Specialist) Lainnya __________ * 5. Jenis kelamin: (Pilih satu opsi) Laki-laki Perempuan Pertanyaan-pertanyaan berikut menanyakan Sustainability Checklist secara spesifik, dimana Bapak/Ibu terlibat dalam pengisiannya, baik untuk tahap I (perjanjian peran dan tanggung jawab) dan/atau tahap II (realisasi). * 6. Apakah Bapak/Ibu terlibat dalam menyelesaikan Sustainability Checklist? (Pilih satu opsi) Ya Tidak * 7. Seberapa bergunakah Sustainability Checklist menurut Bapak/Ibu? (Pilih satu opsi) Your Gender:  M  F The following questions are about specific Sustainability Checklists YOU have personally been involved in completing either for phase I (agreement of role and responsibilities) and/or phase II (Realization) : 1. Have you personally been involved in completing any Sustainability checklists?  Yes  No 2. How useful do you feel sustainability checklists are? USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 108 Sangat berguna Berguna Cukup berguna Tidak berguna sama sekali * 8. Apa yang membuat Sustainability Checklist atau prosesnya bermanfaat? Checklist membantu tim dalam berkolaborasi sebagai pengingat elemen yang diperlukan di setiap tahapan program Checklist membantu mengingatkan semua pihak yang terlibat terkait dengan komitmenmasing-masing Checklist membantu meningkatkan akuntabilitas dari pihak-pihak terkait Melakukan diskusi informasi dengan pihak lain yang juga terlibat terkait dengan proses Sustainability Checklist Lainnya ______________  Very useful  Useful  Quite useful  Not useful at all 3. What makes the sustainability checklist tool or process useful?  The checklist helps the team in collaboration, as a reminder of elements needed at each program stage  The checklist helps to serve as a reminder of collaborators’ commitments  The checklist helps to keep other collaborators accountable  It is helpful in information discussion with other collaborators involved in the sustainability checklist process  Other: ______________________________ 109 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV * 9. Siapa yang biasanya melengkapi bagian Pelaksana Utama? Gambar 1: Sustainability Checklist untuk Sanitasi IUWASH PLUS Specialist IUWASH PLUS WASH Facilitator Instansi terkait Lainnya ______________ 4. Who usually completes the Pelaksana Utama (Main Implementer) column?  IUWASH PLUS specialist  IUWASH PLUS WASH facilitator  Relevant agencies  Other (describe): ___________________________________ USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 110 * 10. Menurut pengalaman Bapak/Ibu, seberapa seringkah para pemangku kepentingan terlibat dalam pengisian Sustainability Checklist serta melakukan pertemuan secara berkelompok untuk membahas dan berkolaborasi selama pengisian Sustainability Checklist tersebut? (Pilih satu opsi) Tidak pernah Satu kali Dua kali Tidak tentu Tidak tahu Lainnya (jelaskan) __________ * 11. Apakah Sustainability Checklist dibuat untuk setiap proyek insfrastruktur air dan/atau sanitasi (misalnya, bila ada 50 master meter/sanitasi komunal/sanitasi individual di satu kota apakah akan dibuatkan 50 SC atau hanya 1 SC)? (Pilih satu opsi) Ya, satu SC untuk 50 master meter/sanitasi komunal/sanitasi individual di satu kota Ya, satu SC untuk 1 master meter/sanitasi komunal/sanitasi individual di satu kota Tidak pasti Lainnya __________ 5. Based on your experience, how often do stakeholders that are involved in completing a sustainability checklist for a specific water or sanitation system meet as a group to discuss and collaborate on that project?  Do not meet  Once  Twice  It varies  Don’t know  Other (describe): __________________________________ 6. Is Sustainability checklist completed for each water or sanitation infrastructure project (for example, if there were 50 master meters/communal sanitation/individual sanitation in a city, will there be 50 SCs or just 1 SC)? (select one)  Yes, one SC for 50 master meter/communal sanitation/individual sanitation in one city  Yes one SC for 1 master meter/communal sanitation/individual sanitation in one city  It varies  Others ___________ 111 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV * 12. Apa yang Bapak/Ibu isi pada bagian kolom 3 keterangan? Gambar 2: Contoh formulir SC ________________________________________________________________ ____ ________________________________________________________________ ____ 7. How do you fill out the Column 3 Keterangan (Description)? USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 112 * 13. Bagaimana Sustainability Checklist difinalisasi pada penyelesaian proyek? Ditandatangani oleh IUWASH PLUS Ditandatangani oleh semua pemangku kepentingan dalam sebuah pertemuan Ditandatangani oleh masing-masing pemangku kepentingan secara terpisah (tidak dalam satu pertemuan) Dengan stempel/cap Tidak tahu Lainnya (jelaskan) ______________ 14. Terkait dengan jawaban di atas, apabila finalisasi dilakukan dengan stempel/cap, stempel/cap dari institusi mana? ____________________________________________________________________ ____________________________________________________________________ * 15. Berapa lama proses yang dibutuhkan untuk melengkapi Sustainability Checklist mulai dari diskusi awal hingga penandatanganan? (Pilih satu opsi) Satu bulan Dua bulan 8. How is the sustainability checklist finalized at the end of project?  Signed by IUWASH  Signed by other stakeholders during a meeting  Signed by other stakeholders obtained individually (not in a meeting)  Seal/stamp  Don’t know  Other (describe): ___________________________________ With regard to the answer above, if finalized with a seal/stamp, which organization’s stamp/seal? 9a. How long is the process to complete Sustainability Checklist starting from initial discussion until signing? (Select one)  One month  Two months  Three months  Longer than 3 months  Other ________________ 113 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV Tiga bulan Lebih dari 3 bulan Lainnya __________ * 16. Sustainability Checklist ini digunakan untuk kegiatan berbagai jenis sistem air dan/atau sanitasi apa saja? Master meter SPAM Komunal Water Grant (Hibah Air) Sistem Sanitasi Perorangan Sistem Sanitasi Komunal Sistem Pembuangan Air Limbah (Skala Kecil atau Skala Kota) Sistem Sanitasi Rusunawa Sanitation Grant (Hibah Sanitasi) Lainnya ______________ * 17. Apakah sistem tersebut di atas menetapkan sistem tarif/iuran bagi pengguna rumah tangga untuk biaya operasional dan pemeliharaan? (Pilih satu opsi) 9b. This Sustainability Checklist is used for what type of water and/or sanitation system projects?  Master meter  Communal SPAM (Water Supply System)  Water Grant  Individual sanitation system  Communal sanitation system  (Small scale or city wide) Sewerage System  Rusunawa sanitation system  Sanitation Grant  Others______________ 9c. Did the above system establish a tariff/fee collection system for operation and maintenance for household users? USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 114 Ya Tidak Tidak tahu Tidak berlaku * 18. Jika ya, apakah biaya/iuran rumah tangga saat ini masih ditagihkan? (Pilih satu opsi) Ya Tidak Tidak tahu Tidak berlaku * 19. Apakah sistem pengelolaan air dan/atau sanitasi berfungsi sebagaimana yang direncanakan sampai saat ini (per April 2019)? (Pilih satu opsi) Ya Tidak Tidak tahu Lainnya (contoh: untuk air, ada kebocoran sistem tapi air masih tetap mengalir) __________  Yes  No  Don’t know  N/A 9d. If Yes, are those household fees/tariffs still being collected?  Yes  No  Don’t know  N/A 9e. Is this water or sanitation system working as designed until this time (April 2019)? (select one)  Yes  No  Don’t know  Other (e.g. for water, there is leak but water is still flowing) : __________________ 115 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV 20. Bila tidak berfungsi, jelaskan mengapa? ____________________________________________________________________ ____________________________________________________________________ * 21. Menurut Bapak/Ibu, apa langkah selanjutnya dari pengisian checklist ini? Penyusunan rencana kerja Memastikan bahwa para pihak yang terlibat akan menindaklanjuti kesepakatan dan komitmen mereka masing-masing Infrastruktur dapat dibangun lebih cepat karena adanya Sustainability Checklist Tidak ada tindak lanjut Lainnya ______________ Pertanyaan selanjutnya mengenai bagaimana proyek ini memberikan orientasi Sustainability Checklist pada staf IUWASH PLUS dan pemangku kepentingan, termasuk manfaat keseluruhan dari Sustainability Checklist pada proyek ini dan efektivitas pembuatan sistem infrastruktur air minum dan sanitasi yang berkelanjutan. 9f. If Not functional, why not? (Describe) 9g. In your opinion, what are the next steps after completing this checklist?  Develop work plan  Ensuring other collaborators involved will follow through on the agreement and their commitments  The infrastructure was built more quickly thanks to the sustainability checklist  Nothing  Other: _________________________ The next questions are about how the project provides Sustainability Checklist orientation to IUWASH PLUS staff and stakeholders, as well as the checklist’s overall utility to the project and the effectiveness of creation of sustainable water and sanitation infrastructure systems USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 116 * 22. Apakah Bapak/Ibu mendapatkan pelatihan/orientasi terkait Sustainability Checklist? (Pilih satu opsi) Ya Tidak 23. Jika ya, apakah Bapak/Ibu merasa pelatihan/orientasi tersebut cukup memadai untuk menerapkan Sustainability Checklist secara konsisten? (Pilih satu opsi) Ya Tidak * 24. Apakah Bapak/Ibu telah mendapatkan instruksi tertulis atau alat bantu (misal SOP atau dokumen panduan) untuk mengerjakan Sustainability Checklist? (Pilih satu opsi) Ya Tidak * 25. Mohon beri nilai untuk tingkat pemahaman Bapak/Ibu dalam cara mengisi Sustainability Checklist. (Pilih satu opsi) Sangat mengerti Mengerti 10. Did you receive any training/orientation on how to complete the sustainability checklist?  Yes  No a. If Yes, Do you feel the training/orientation was sufficient to enable you to administer the checklist?  Yes  No 11. Have you received any written instructions or job aids (e.g. SOPs or “How-To” document(s)) on how to complete the Sustainability Checklists?  Yes  No 12. Please rate your level of understanding on how to complete the SC checklists  Fully understand  Understand  Limited understanding  No understanding 117 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV Kurang mengerti Tidak mengerti 26. Apa Bapak/Ibu mempunyai usulan untuk membuat Sustainability Checklist lebih bermanfaat dan efektif? ____________________________________________________________________ ____________________________________________________________________ * 27. Menurut Bapak/Ibu, seberapa besar manfaat Sustainability Checklist ini dalam mendukung pelaksanaan proyek-proyek infrastruktur air dan sanitasi secara berkelanjutan? (Pilih satu opsi) Sangat bermanfaat Bermanfaat Tidak bermanfaat Sama sekali tidak bermanfaat 28. Alasan Bapak/Ibu memberi nilai tersebut: ____________________________________________________________________ ____________________________________________________________________ * 29. Menurut Bapak/Ibu, seberapa besarkah peran Sustainability Checklist dalam memperjelas tugas & tanggung jawab masing-masing 13. What additional information/training, if any, do you think is needed to make the Sustainability Checklists more useful and effective? 14. How useful do you think the SC have been to support building of more sustainable water and sanitation infrastructure projects?  Very useful  Useful  Not useful  Not useful at all Any comments about why you rated it this way: 15. How useful do you think has the SC been at defining more concretely the roles and responsibilities of stakeholders involved in water and sanitation infrastructure projects? (Select one) USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 118 stakeholder yang terlibat dalam proyek air dan sanitasi ini? (Pilih satu opsi) Sangat bermanfaat Bermanfaat Tidak bermanfaat Sama sekali tidak bermanfaat 30. Alasan Bapak/Ibu memberi nilai tersebut: ____________________________________________________________________ ____________________________________________________________________ 31. Bila ada masukan, perubahan apa yang ingin diusulkan untuk membuat Sustainability Checklist ini lebih bermanfaat dan efisien untuk Proyek IUWASH PLUS? ____________________________________________________________________ ____________________________________________________________________  Very useful  Useful  Not useful  Not useful at all Any comments about why you rated it this way: 16. What, if anything, would you recommend be changed about the Sustainability Checklists themselves and/or to the process of using them to make them more useful and more efficient to the IUWASH PLUS Project? 119 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV For IUWASH PLUS Collaborators 1. Nama: ____________________________________________________________________ ____________________________________________________________________ Name: _______________________ 2. Nomor telepon: ____________________________________________________________________ ____________________________________________________________________ Phone Number: _______________________ 3. Alamat email: ____________________________________________________________________ ____________________________________________________________________ Email: _______________________ * 4. Institusi/lembaga pemerintah: (Pilih satu opsi) Bappeda Dinas PUPR Dinas Perkim Dinas Kesehatan UPTD/UPT Lembaga pemerintah daerah lainnya Government agency or Institution:  BAPPEDA  Dinas PUPR  Perkim  Dinas Kesehatan  UPTD/UPT  Other local government  PDAM  Wusan  KSM/KPP  Other: ___________________ USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 120 PDAM Wusan/sanitarian KSM/KPP/grantee Lainnya __________ * 5. Jabatan: ____________________________________________________________________ ____________________________________________________________________ Your Position: _______________________________________ * 6. Jenis kelamin (Pilih satu opsi) Perempuan Laki-laki Your Gender:  M  F * 7. Apakah Bapak/Ibu terlibat dalam pengisian Sustainability Checklist? (Pilih satu opsi) Ya Tidak (langsung ke bagian akhir, pertanyaan no 24) 1. Have you personally been involved in completing any Sustainability Check Lists?  Yes  No (Please Skip to the end) 8. Menurut Bapak/Ibu, seberapa bermanfaatkah Sustainability Checklist tersebut? (Pilih satu opsi) 2. How useful do you feel the sustainability checklists are?  Very useful  Useful  Not useful 121 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV Sangat bermanfaat Bermanfaat Tidak bermanfaat Sama sekali tidak bermanfaat (jika Bapak/Ibu memilih jawaban ini, silakan langsung ke pertanyaan no 10)  Not useful at all (If you select this answer, please skip to question no. 10) 9. Apa yang membuat Sustainability Checklist atau prosesnya bermanfaat? Checklist membantu tim dalam berkolaborasi sebagai pengingat elemen yang diperlukan di setiap tahapan program Checklist membantu mengingatkan semua pihak yang terlibat terkait dengan komitmen masing-masing Checklist membantu meningkatkan akuntabilitas dari pihak-pihak terkait Melakukan diskusi informal dengan pihak lain yang juga terlibat terkait dengan proses Sustainability Checklist Lainnya ______________ 3. In your opinion, why made the sustainability checklist tool or process useful?  The checklist helps the team in collaborating as a reminder of elements needed at each program stage  The checklist helps to serve as a reminder to collaborators involved of their own commitments  The checklist helps improve accountability of relevant collaborators  In assisting to have informal discussions among collaborators involved in the Sustainability Checklist  Others: ______________________________ 10. Dalam pengalaman Bapak/Ibu, seberapa seringkah para pemangku kepentingan terlibat dalam pengisian Sustainability Checklist serta melakukan pertemuan secara 4. Based on your experience, how often are stakeholders involved in completing a sustainability checklist and meet as a group to discuss and collaborate during the filling out USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 122 berkelompok untuk membahas dan berkolaborasi selama pengisian Sustainability Checklist tersebut? (Pilih satu opsi) Tidak pernah bertemu Satu kali Dua kali Tidak tentu Tidak tahu Lainnya (jelaskan) __________ process of the Sustainability Checklist? (Select one)  Do not meet  Once  Twice  It varies  Don’t know  Other (describe): _______________ Pertanyaan berikut berlaku untuk Sustainability Checklist paling awal ("tahap realisasi" terselesaikan). Seingat Bapak/Ibu, Bapak/Ibu pribadi dalam pengisiannya (jika belum pernah mengisi Sustainability Checklist silakan langsung ke pertanyaan 24): 5. The following questions apply to the earliest (completed “Realization Phase”) checklist you recall that you were personally involved in (If you have not completed a SC please skip to question 23): 11. Berapa lama proses yang dibutuhkan untuk melengkapi Sustainability Checklist mulai dari diskusi awal hingga penandatanganan? (Pilih satu opsi) Satu bulan Dua bulan Tiga bulan Lebih dari 3 bulan Lainnya __________ a. How long does it take to complete a Sustainability Checklist starting from initial discussion to signing off? (Select one)  One month  Two months  Three months  More than 3 months  Others ___________ 123 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV 12. Sustainability Checklist ini digunakan untuk kegiatan berbagai jenis sistem air dan/atau sanitasi apa saja: Master meter SPAM komunal Water grant (hibah air) Sistem sanitasi perorangan Sistem sanitasi komunal Sistem pembuangan air limbah (skala kecil atau skala kota) Sistem sanitasi Rusunawa Sanitation grant (hibah sanitasi) Lainnya ______________ b. What type of water or sanitation system was this sustainability checklists used for:  Master Meter  SPAM Communal  Water Grant (Hibah)  Individual Sanitation System  Communal Sanitation System  (Small Scale or City Wide) Sewerage System  Rusunawa Sanitation System  Sanitation grant  Others ___________ 13. Apakah sistem tersebut diatas menetapkan sistem tarif/iuran bagi pengguna rumah tangga untuk biaya operasional dan pemeliharaan? (Pilih satu opsi) Ya Tidak Tidak tahu Tidak berlaku c. Did the above system establish a tariff/fee collection system for operation and maintenance for household users? (Select one)  Yes  No  Don’t know  N/A USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 124 14. Jika ya, apakah biaya/iuran rumah tangga tersebut saat ini masih dijalankan? (Pilih satu opsi) Ya Tidak Tidak tahu Tidak berlaku d. If Yes, are those household fees/tariffs still being collected?  Yes  No  Don’t know  N/A 15. Apakah sistem pengelolaan air dan/atau sanitasi berfungsi sebagaimana yang direncanakan sampai saat ini (per April 2019)? Ya Tidak Tidak tahu Lainnya (contoh: untuk air, ada kebocoran sistem tetapi air masih tetap mengalir) ______________ e. Is this water or sanitation system working as designed until now (per April 2019)?  Yes  No  Don’t know  Other (e.g.: for water, there is leak but water is still flowing) : __________________ 16. Bila tidak berfungsi, jelaskan mengapa? ____________________________________________________________________ ____________________________________________________________________ f. If Not functional, why not? (Describe) 17. Menurut pendapat Bapak/Ibu, apa langkah selanjutnya dari pengisian checklist ini? g. In your opinion, what is the next step after having filled this checklist?  Developing work plan 125 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV Penyusunan rencana kerja Memastikan bahwa para pihak yang terlibat akan menindaklanjuti kesepakatan dan komitmen mereka masing-masing Infrastruktur dapat dibangun lebih cepat karena adanya Sustainability Checklist Tidak ada tindak lanjut Lainnya ______________  Other collaborators were more likely to follow through on their commitments  The infrastructure was built more quickly than without the sustainability checklist  Nothing  Other: _________________________ 18. Apakah Bapak/Ibu mendapatkan pendampingan (misalnya pelatihan, penjelasan tertulis maupun tidak tertulis) terkait dengan bagaimana memanfaatkan Sustainability Checklist? (Pilih satu opsi) Tidak tahu Tidak Ya, mohon jelaskan bentuk pendampingannya __________ 6. Did you receive any guidance (eg. training, written or verbal instructions) on how to use the Sustainability checklist?  Don’t know  No  Yes, please describe type of guidance: _____________________- 19. Menurut Bapak/Ibu, seberapa besar manfaat Sustainability Checklist ini dalam mendukung pelaksanaan proyek-proyek infrastruktur air dan sanitasi secara berkelanjutan? Sangat bermanfaat Bermanfaat Tidak bermanfaat 7. How useful how beneficial has the SC been in supporting the implementation of water and sanitation infrastructure projects sustainably?  Very useful  Useful  Not useful  Not useful at all  Other (please specify) ____________________ USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 126 Sama sekali tidak bermanfaat Other (Please specify) ______________ 20. Terkait dengan jawaban di atas, mohon berikan penjelasannya. ____________________________________________________________________ ____________________________________________________________________ Any comments about why you rated it this way: 21. Menurut Bapak/Ibu, seberapa besarkah peran Sustainability Checklist dalam memperjelas tugas & tanggung jawab masing-masing stakeholder yang terlibat dalam proyek air dan sanitasi ini? Sangat bermanfaat Bermanfaat Tidak bermanfaat Sama sekali tidak bermanfaat 8. How useful do you think the SC been at defining more concretely the roles and responsibilities of stakeholders involved in water and sanitation infrastructure projects?  Very useful  Useful  Not useful  Not useful at all 22. Terkait dengan jawaban di atas, mohon berikan penjelasannya ____________________________________________________________________ ____________________________________________________________________ Any comments about why you rated it this way: 127 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV 23. Apakah Bapak/Ibu menerima salinan Sustainability Checklist setelah bersama-sama menyetujui tugas dan tanggung jawab masing-masing? (Pilih satu opsi) Ya Tidak 9. Did you receive a copy of the sustainability checklist after roles and responsibilities were agreed upon?  Yes  No 24. Bila ada masukan, perubahan apa yang ingin diusulkan untuk membuat SC ini lebih bermanfaat dan efisien untuk Proyek IUWASH PLUS? ____________________________________________________________________ ____________________________________________________________________ 10. What, what changes would you like to see to make this SC more useful and efficient to the IUWASH PLUS Project? Terima kasih untuk masukan Bapak/Ibu. Thank you for your support USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 128 ANNEX 5: IUWASH PLUS RESULTS FRAMEWORK 129 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV ANNEX 6: SUMMARY OF COMPONENT I ACTIVITIES BOX 4. SUMMARY OF COMPONENT 1 ACTIVITIES • Formative research to inform project’s behavior change (BC) and sanitation marketing strategies. (See EQ3.) • Collaboration with LG and institutional partners to increase access to sustainable WASH services (See EQ2.) • Support for household microfinance schemes for WASH services. • Community mobilization and urban community-based total sanitation and hygiene (STBM95) and participatory monitoring activities. • Capacity building/technical assistance. • Support toward development of the urban STBM approach and associated national policy and guidelines. • B40 poverty verification (monitoring and evaluation). 95 This acronym derives from the Indonesian phrase Sanitasi Total Berbasis Masyarakat. STBM is a national approach led by the MOH, started in 2008 under Decree No. 852/2008. The decree includes five pillars; one is related to household drinking water and two are related to sanitation (stopping open defecation and household sewer management). STBM builds on the community-led total sanitation (CLTS) model, a bottom-up demand-creation approach that empowers communities to end open defecation (OD) with improved sanitation and hygiene practices through “triggering” and post￾triggering monitoring activities. It is a non-subsidy approach. Developed by Kamal Kar in the context of rural Bangladesh, many countries have developed CLTS. Because it has been primarily a rural model, the MOH revised the decree to expand its implementation in urban and peri-urban areas in January 2014. IUWASH PLUS has been assisting in the efforts to revise the national framework for urban areas, as well as supporting its expansion into urban areas through its project activities. USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 130 ANNEX 7: OVERVIEW OF COMMUNITY MOBILIZATION AND STBM (12 STEP PROCESS) BOX 5. OVERVIEW OF COMMUNITY MOBILIZATION AND STBM (12 STEP PROCESS) Community mobilization focuses on assessing and identifying individual community WASH conditions, including “triggering” using the CLTS method. The method also includes needs assessment of the enabling environment (stakeholder analysis), and the development of mechanisms and action plans to address the identified needs. It includes six steps. The culminating step is the establishment of a citizen engagement mechanism (CEM). Participatory monitoring and evaluation (M&E) includes six steps. It focuses on preparing community members to conduct M&E activities related to their assessment and action plan developed under community mobilization, including the analysis of their findings. A key cumulating component of this cycle is the stakeholder meeting(s), where community members present their findings and action plans to LG leaders and organizations. 131 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV ANNEX 8: PROJECT METHODS TO MEASURE ACCESS TO WASH SERVICES AND B40 INDICATORS IUWASH PLUS uses three methods to measure and monitor access to WASH services 1. The first utilizes data collected from partners, including PDAM, to assess how many persons have obtained improved services. This method is also used to track indirect attribution (for PDAM services) and direct services. 2. The second method includes surveying households (as reported on through the above method) to “verify” among those who have obtained access, how many of the households are identified to be B40 (see B40 Verification section below for more on this method). Methods 1 and 2 are used to report on the project indicators, as outlined in Box 6. 3. The third method is part of the Community Mobilization/STBM’s Participatory Monitoring process. This method is conducted by community members in hotspots on an ongoing basis to monitor household progress towards improved access to water and sanitation. This data includes initial baseline data collected by the community of every household’s water and sanitation service type(s) and monitoring data on those services that is collected over time. This data is used to develop the hotspot’s community action plans including presentation of needs to stakeholders in the Stakeholder Meeting(s) as described in Box 5 above. BOX 6. IUWASH PLUS WATER AND SANITATION B40 INDICATORS HR1a and HR1b: Number of people receiving improved service quality from an existing basis of safely managed drinking water services as a result of U.S. Government (USG) assistance. HR1a measures indirect attribution and HR1b measures direct attribution and B40 targeted households (HH). HR1a Target: 1,000,000; HRb1 Target: B40 and Eastern Indonesia (EI): 500,000 C1a: Number of people from the bottom 40 percent of wealth gaining access to shared or basic sanitation services as a result of USG assistance. C1a Target: 500,000 USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 132 ANNEX 9: HOW ACCESS TO IMPROVED WATER AND SANITATION IS MEASURED The project monitors the water and sanitation services that household access, tracking results disaggregated by service type (direct/indirect), household type (B40 status), and location (hotspot/non-hotspot)96. For water, improved services are identified as either having either “direct” or “indirect” attribution to the project.97 For example, each year a percentage of all new PDAM98 water connections among the 32 sites are indirectly attributed to the project as a result of project’s technical assistance (TA) efforts. Direct attributions are a result of households gaining access to improved water sources through the projects direct interventions, such as forming a KSM (kelompok swadaya masyarakat, or community-based organization) to support the implementation of a master water meter. For sanitation, new services are being captured by hotspot and non-hotspot areas. Table 10 describes the types of water and sanitation infrastructure or service connections households are gaining access through the project’s activities, as well as how they obtain access to these services. TABLE 10: WATER AND SANITATION SERVICE TYPES SUPPORTED BY THE PROJECT WATER SERVICES (DIRECT OR INDIRECT) SANITATION SERVICES (IN HOTSPOTS AND NON-HOTPOTS) TYPE OF SERVICES HOW ACCESS IS OBTAINED TYPE OF FACILITY HOW ACCESS IS OBTAINED Individual HH connections to PDAM (indirect) Resulting from project￾provided TA in all 32 sites Individual HH toilets/system (may include several houses sharing a septic tank) IUWASH tracks and supports two types of sanitation facilities: • Basic/shared sanitation facilities99 • Safely managed* Households build sanitation facilities with their own resources following STBM or Behavior Change or Sanitation Marketing processes or receive support to build via MFI/loans, grants or CSR they are linked to through the project. Individual HH connections to PDAM (direct) Resulting from Water Hibah grants Communal sanitation systems, e.g., IPALs 100 LG and other stakeholders. CSRs contribute resources to build or 96 The types of services household’s access are not included in the Activity Monitoring and Evaluation Plan (AMEP). Two exceptions to this are reported: 1) indirect versus direct water connections to PDAM and 2) basic/shared sanitation facilities. 97 IUWASH PLUS AMEP Vs 2.0 April 2018 page 9 98 Perusahaan Daerah Air Minum/Municipal Drinking Water Company 99 Under the new Joint Monitoring Program (JMP), Sustainable Development Goals (SDGs) defines basic household sanitation facilities as facilities at households that are not shared with other households; with households sharing an improved facility between two more households considered as having a “limited” level of services. The IUWASH PLUS project does not disaggregate household facilities between individual or shared facilities in its AMEP. (WHO/UNICEF) 100 Instalasi Pengolahan Air Limbah/Wastewater Treatment Plant (WWTP). 133 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV WATER SERVICES (DIRECT OR INDIRECT) SANITATION SERVICES (IN HOTSPOTS AND NON-HOTPOTS) Master meters connected to a PDAM supply for a group of HHs (direct) Resulting from water grants or CSRs and support from IUWASH PLUS project LSIC funds. Community (small-scale or citywide sewerage systems) facilitate access to new or expanded sanitation facilities in the community. LSIC PLUS project LSIC funds also to support some communal systems. Communal-based water supply (SPAM Communal101), e.g., a well or gravity-fed piped system not connected to PDAM (direct) Resulting from LG (public works) expansion, community water managed systems (HIPPAMs102) and other grants/CSR Rusunawas (low-income housing complex wastewater treatment system) In general, the project-supported rusunawas have already been constructed. All are in Jakarta, where no hotspot activities are conducted. The project is providing TA to improve their operational functionality, including to HHs to ensure that they use their toilets properly. 101 Sistem Penyediaan Air Minum, drinking water supply system. 102 Himpunan Penduduk Pemakai Air Minum, consumer council for water. USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 134 ANNEX 10: DETAILS OF PROJECT ACHIEVEMENTS WATER ACCESS Overall and B40 Targets Results. Based on projections for new water services for PY3 through April 2019, the project is behind in meetings its life-of-project target (55.57 percent off target for all persons). The progress rate indicates a decline in service connection achievements from PY2 compared to projected PY3, but an increase in B40 access. In PY2, 236,035 people obtained access to safely managed drinking water, 11.4 percent of whom were B40, while the projected number for PY3 is 211,584 persons, 23.6 percent of whom are projected to be B40. Table 10 shows cumulative results as of April 2019 based on actual PY2 reported results and projected (or estimated) results expected for PY3 against PY3 targets for access to improved water services. The estimated results are that a total of close to 107,000 poor (B40) persons in 21,389 households will have obtained access to water services by the end of PY3. Based on these projections, the project is only 7.98 percent short of the estimated target for PY3, but 78.61 percent off from their overall target for the project: 500,000 B40 individuals having new water access, with 35 months remaining in the project including the recent six-month contract extension. (See Table 11 for a table on complete PY2 and PY3 projected aggregate data for water access). TABLE 11: PY3 AND CUMULATIVE RESULTS FOR WATER SERVICE ACCESS OF APRIL 2019 Status of New Water Service Quality Access (as of April 2019) All Individuals (HR1a) B40 Individuals (HR1b) Target Projected PY3 Result % of Target Achieved Target Projected PY3 Result % of Target Achieved PY3 AMEP Workplan Estimated Targets (2019) 536,095 447,679 82.87% 116,215 106,945 92.02% Five Year Life-of-Project Target (2021) 1,000,000 44.43% 500,000 21.39% Figure 9 shows the cumulative HR1b results projected as of April 2019 by city/district against PY3 targets (right Y axis) for B40 persons, including results for both direct and indirect interventions. These data show significant differences among areas, with 43 percent achieving and more than 90 percent close to achieving PY3 targets. Most of these have exceeded PY3 targets by a significant margin (from 113 percent to 421 percent), with the majority in West and Central Java. Most of the achievements (74percent) are attributed to indirect connections to PDAM based on available data. Overall greatest achievements in direct water interventions are seen in six sites (Kab Deli Serdang, Bogor, Magelang, Kab Sidoarjo and Kab Bantaeng), with Central Java having more B40 coverage among cities/districts compared to other regions. However, 60 percent of the cities/districts have no direct water interventions reaching B40 individuals, based on the current available data. 135 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV FIGURE 9: CUMULATIVE RESULTS FOR B40 WATER SERVICES BY SITE (PY3 PROJECTED RESULTS) ● Denotes locations of MTE KIIs B40 Water Access Target Attainment Variance by City/District. Among the areas where KIIs were conducted,103 cities/districts are either achieving, or close to achieving, their B40 targets. The exceptions to this are Surakarta and Surabaya.104 KIs in Surakarta indicated that their challenge is loss of PDAM customers due to population decreases, as well as households building their own wells and disconnecting from PDAM. This affects the ability to bring in revenue and expand services. They also noted that they lack new water sources to draw from. While the project has no MOU with Surabaya, overall water coverage there is reported by PDAM to be at 98 percent and expected to go to 100 percent in 2019. They also have policies/tariffs in support of the poor and are working with IUWASH PLUS to implement master meters. All of the Pokja KIs at visited sites discussed the project’s efforts to improve PDAM operations as significant in their ability to provide increased water services overall. Based on analysis of KIIs and comparison of B40 results, the sites visited that are close to achieving or have achieved their B40 targets, either have 1) some form of water tariff scheme or discounts for poor households or other GOI grants that target some poor households and/or 2) they are fairly functional, based on the PDAM Index. Of the three that look to be furthest from their targets, Surakarta and Barru have the lowest PDAM index scores among the sites visited (42 and 25.5 respectfully); Surakarta, per KIs, has 103 Project data analysis had yet to be conducted prior to selecting site visit locations; while no bias was known in selecting sites, the evaluation team did miss areas that were further off their PY3 targets. 104 This may due, in part, to the lack of a MOU with Surabaya and no B40 surveys conducted in these areas. USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 136 no current pro-poor funding or tariff schemes in place and Barru had only established one in 2018.105 The other, Surabaya, has overall high coverage rates, a relatively high PDAM index score (67.5, fourth-highest among the 11 sites visited) and pro-poor policies, and is working with IUWASH PLUS to establish master meters for poor households. It appears that Surabaya’s gap to target may in part be due to the project’s B40 data not being up to date.106 B40 Water Access Target Attainment by Direct Versus Indirect Services. Figure 10 shows the cumulative number of persons with indirect access (PDAM) to water by B40 status through projected PY3. Direct interventions were approximately 34 percent more effective in reaching the poor, but they do so on a much smaller scale. The data indicate that the majority of indirect connections are being made to non-B40 households. Based on B40 surveys: • Project sites had an indirect connection average of 29.08 percent B40 in PY2, with a range of 1 percent (Sidoarjo in East Java) to a high of 74 percent (Barru in South Sulawesi), not considering East Indonesia, where all are designated as 100 percent B40. • Direct PY2 water connections had an average of 38.60 percent B40, with a range of 3 percent (Depok in West Java) to a high of 61 percent (Magelang in Central Java). B40 Attainment by Type of Water Intervention. Overall, connections related to direct IUWASH intervention (PDAM Water Hibah grants, community-based water supply systems and community water meters) constitute only 25.85 percent of all B40 new water service connections in Figure 11 shows total new B40 water services projected for PY3 by type of service connection. Most 105 IUWASH PLUS PY2 Report Annex p. 34 106 PDAM director estimates that 10,000 households do not have access to water and 40 percent (4,500 HH) of them are poor. 0 10,000 20,000 30,000 40,000 50,000 60,000 Medan Pematangsiantar Tebing Tinggi Kb Bogor Bekasi Kb Tangerang Surakarta Kb Magelang KbSukoharjo Kb Sidoarjo Probolinggo Kb Lumajang Makassar Kb Bantaeng Ternate K dan Kb Jayapura North Sumatra West Java Central Java East Java South Sulawesi & Eastern Indonesia Number of B40 persons gaining access to basic drinking water service from Indirect water connections supported by IUWASH PLUS PY1, PY2 and projected PY3 Indirect Non-B40 (individuals) Indirect B40 (Individuals) FIGURE 10: ACCESS TO INDIRECT WATER SERVICES 137 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV direct interventions are from connections to PDAM resulting from individual household Water Hibah grants. B40 figures are expected to rise, in particular for master meters107 for PY3, as B40 surveys have yet to be conducted. SANITATION ACCESS Overall and B40 Targets Results. The project has revised its sanitation targets since it began. Originally, the high-level results target (HR2) was for B40 individuals representing 50 percent of those gaining safely managed access. Following a technical site visit to Indonesia from USAID/Washington in 2017, the B40 focus for this target was removed. The former target that 250,000 people would gain access to shared and basic sanitation services (C1-1) shifted; now the target is that all 250,000 would be B40 (C1a) following the site visit in 2017. Table 12 provides a summary of the status of basic and shared sanitation services as of April 2019 based on PY2 results and known and projected results for PY3. Based on projections through PY3,108 the project is also 107 MTE site visits included visits to two completed project supported master meters that have yet to be included in project M&E data or have B40 surveys completed. 108 Projections include actual B40 surveys conducted in some hotspots as of April 2019 (eight surveys completed in hotspots) and projection of B40 PY2 survey percentage onto PY3 projected results for non-hotspots. For non-hotspots, if a percentage for PY2 was not available for a specific Kota/Kab site, the regional B40 was used. In almost all cases, percentages from the same type of system were used if available. PDAM Indirect (Individual connections ) Water hibah (Direct individual connections ) Community￾based water supply (Direct) Master meter (Direct) Total persons 79528.83259 26302.5402 1114.730942 0 106946.1037 0 20000 40000 60000 80000 100000 120000 Number of new water connections Projected number of B40 gaining access to water as a result of IUWASH Plus assistance (HR1b) PY2 and Projected PY3 Life of project target = 500 000 (21.39% of target) PY2 and PY3 target = 216,215 (48.46% of target) FIGURE 11: PROJECTED NUMBER OF B40 INDIVIDUALS GAINING ACCESS TO WATER, BY TYPE OF SERVICES CONNECTION USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 138 behind in meeting its overall revised project targets for sanitation, with 93,740 people projected to have new access to basic or shared sanitation as of April 2019, but only 45,941 are projected to be B40 (49.01 percent). This represents 18.38 percent of the life-of-project target and 11.37 percent of the PY3 target with 35 months remaining in the project. TABLE 12: STATUS OF BASIC AND SHARED SANITATION SERVICES IUWASH PLUS Status of access to basic and shared sanitation services, All and B40 (HH/persons) PY2 PY3 Projected PY2+PY3 All B40 % B40 All B40 % B40 All B40 % B40 Non-Hotspot Households 6,969 3,464 49.71% 11,065 5,589 50.51% 18,034 9,053 50.20% Hotspot Households 546 40 7.33% 168 95 56.55% 714 135 18.91% Total Households 7,515 3,504 46.63% 11,233 5,684 50.60% 18,748 9,188 49.01% Total Number of People with Shared and Basic Sanitation 37,575 17,521 46.63% 56,165 28,420 50.60% 93,740 45,941 49.01% Cla.: Percentage of Life-of-Project Target B40 with New Basic/Shared Sanitation (250,000) 7.01% 11.37% 18.38% PY3 Target N/A target changed 77,440 95,000 % of Target 36.70% 48.36% B40 Attainment by Hotspots and Non-Hotspots. Unlike for water services, access appears to be increasing for sanitation services from PY2 to PY3. However, most of the increase in access is seen in non-hotspot areas and only a little over half of them are identified to be B40. Overall, based on available known and projected data to date, 96.2 percent of all new basic and shared sanitation access are in non-hotspot areas. The percentage of B40 is anticipated to change for PY3, according to IUWASH PLUS staff, as the project is still in the process of conducting B40 surveys for the year.109 In addition, staff reported that they expect that their community mobilization/urban STBM activities in hotspot areas, which are still underway (see Findings Section), will result in the construction of more sanitation facilities in the years ahead, particularly in hotspots as more complete the stakeholder engagement step of these processes. There is some evidence that this may be accurate. Based on the eight cities/districts that have reported B40 households from B40 verification surveys110 with new sanitation access, five have completed all cycles of the community mobilization/urban STBM process, including the culminating stakeholder meeting. These five hotspots 109 As of April 2019, it appears that only 10 B40 surveys have been conducted among hotspot areas’ sanitation systems constructed in both PY2 and PY3. 110 B40 surveys are still on-going and not completed for all sites for services that have been completed by partners. 139 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV account for 71.6 percent of all new sanitation access among the hotspots. Another site, Kota Bogor, is the location of the initial sanitation marketing strategy pilot. It is not known if the B40 households that obtained access here resulted from the marketing strategy. (See further discussion below in Findings regarding the hotspot targeting approach). Looking at projected results and targets for PY3 by region, only East Java will have achieved its B40 results for PY3 based on current projections, exceeding their target by approximately 318 households (about 1,590 individuals). However, only 125 households will have gained access in hotspot areas. The other remaining regions are significantly behind in their PY3 targets based on currently available data. As with water, these figures will need to be updated for PY3 when B40 surveys are completed. 4753 3807 1510 16474 1400 70 40 210 125 30 11955 16990 18315 15150 15030 0 2000 4000 6000 8000 10000 12000 14000 16000 18000 20000 0 2000 4000 6000 8000 10000 12000 14000 16000 18000 NORTH SUMATRA WEST JAVA/DKI/ KAB. TANGERANG CENTRAL JAVA EAST JAVA SOUTH SULAWESI/ EASTERN INDONESIA C1a Number of people from B40 and Eastern Indonesia PY3 Projected Access to Basic and Shared Sanitation by Region against PY3 Target (Result N=77,440) B40 Non Hot Spot Persons Total B40 Hot Spot total Total B40 B40 Persons Target PY3 by Region FIGURE 12: ACCESS TO SANITATION BY REGION, PY3 PROJECTED USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 140 B40 Attainment by Type of Sanitation facility. Most households gaining access to shared and basic sanitation are a result of individual household sanitation facilities (55.9 percent), followed by communal systems (IPALs, 37.9 percent). In hotspot areas, IPAL communal systems111 are reaching significantly more B40 persons than in non-hot spots areas. It is not fully known why more of these have not been built in hotspot areas. Given that hotspots are to be areas of high concentrations of poor, the cost to build an IPAL, compared to an individual septic tank per household, and the lack of fiscal resources is one of the key reasons. The majority of funds leveraged by the project in PY2 have been for IPAL systems, however the data is not provided by hotspot or non-hotspot (APY2 Annex 34). 111 IPALs are “offsite domestic wastewater treatment systems” categorized as SPALDTs (see Annex 21 for project figure on types of urban domestic wastewater management systems). It’s not known how many IPALs this represents, as data indicates HHs only. IPALs are reported to range from 11 to 110 HH per IPAL, with one IUWASH WASH facilitator’s reported numbers indicating approximately 53 HHs per IPAL in his area. Note the number provided during this KII were not in alignment with reported M&E data for that community/region. FIGURE 13: INDIVIDUALS GAINING ACCESS TO BASIC AND SHARED SANITATION, BY TYPE AND HOTSPOT/NON-HOTSPOT 141 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV A review of the available percentage of B40’s by type of system and hotspot versus non-hotspot areas found that sanitation systems in hotspot areas generally have higher percentage of poor than in non-hotspot areas (Figure 13). Like the project’s water “direct interventions”, this supports the notion that the project is targeting the right areas with the right interventions to reach B40 populations. However, resource gaps or other issues (see more below in Findings under project approach) are inhibiting the implementation of more interventions in hotspots. 0.79 0.3333 0.62 0.4782 0.4319 0.2171 0 0.2 0.4 0.6 0.8 1 Individual system Shared Septic Communal system (IPAL) Sewerage Approximate percentage of B40 by sanitation system type comparing hotspot to non-hotspots Hot Spots Not-Hot Spots FIGURE 14: PERCENTAGE OF B40 BY SANITATION SYSTEM TYPE USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 142 ANNEX 11: IUWASH PLUS HOTSPOT MAPPING AND COMMUNITY PARTICIPATORY PLANNING 143 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 144 145 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV ANNEX12: CYCLE AND ACCOMPLISHMENTS OF PARTICIPATORY M&E USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 146 ANNEX 13: WASH ECOSYSTEM 147 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV ANNEX 14: B40 VERIFICATION OF MWATER SURVEY COSTS B40 Verification MWater Survey Costs (April 2019) TABLE 13: SAMPLE OF TWO B40 VERIFICATION SURVEY’S COSTS CONDUCTED IN 2019 City/ District Total Sample Actual Cost Cost for per respondent USD (cost per respondent) Enumerator Training Survey Total Training and survey Survey alone Kabupaten Sukoharjo 553 4,975,000 18,450,000 23,425,000 42,360 $2.99 $2.35 Kota Makassar 343 2,332,000 13,140,000 15,472,000 45,108 $3.19 $2.70 TABLE 14: OVERALL BUDGET AND COST FOR B40 SURVEYS Budget Area IDR USD Description Survey Planning/Implementation 605,811,276 $ 42,777.00 Method development, trail, enumerator training, data collection, cleaning and data analysis IUWAS PLUS M&E 1212285081 $ 85,603.00 8 Staff (3.6 FTE) IUWASH PLUS staff Operation costs (travel) 666,387,538 $ 47,054.00 Travel to support implementation the survey Equipment 141,392,000 $ 9,984.00 18 phones (80% of computers for B40) Total 2,625,875,895 $ 185,418.00 Total surveys (220 pilot, 12,426 B40 water, 11,342 B40 sanitation) 23,988 Cost per survey $0.13 USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 148 ANNEX 15: SUMMARY OF MTE KII SITES IUWASH PLUS PROJECT PY3 B40 TARGETS FOR WATER AND SANITATION AND IDENTIFIED HIGH LEVEL CITY B40 FOCUS/ IF ANY KI site Visit B40 Water PY3 Targets Achieved? KI Explicit Pro-Poor Water B40 approaches identified? Y/N* (Other B40 approaches identified) Regional B40 Sanitation PY3 Targets Achieved? & Site Non￾Hot Spot/Hot Spot B40 Persons PY3 Results KIs Explicit LG Focus on B40 Poor Sanitation efforts identified? Mixed, Y/N (Other B40 approaches identified) West Java No Bogor No No (DAK only/Pipes don’t reach poor communities) 807/10 Mixed (“Environment covers everyone.” Focus on one community by river. Environmental risk assessment identifies high risk areas, usually poor.) Depok Yes No (Serves poor with Water Hibah/PUPR water grant) 167/0 Yes (Uses data from Bappeda/targets B40 communities for IPALs with LG and DAK financing) Tangerang Yes No (No current funds targeted to poor. Except small community-based water supply project for 100 HHs) 547/30 Mixed (New focus due to IUWASH PLUS/others One KI did not know IUWASH PLUS had focus on B40. Nothing specific in current regulations) Central Java No Surakata No* Yes (Has explicitly five-year city strategic plan in red zone, LG agencies aligning their work to target poor) 440/150 Yes (Red Zones and SSK (city sanitation strategy. Focus on IPLTs. Thinks CSR best way to serve B40, not only APDB LG funding Magelang Yes Yes (Broad pro-poor connection assistance and use of DAK funds) 384/60 Mixed (Socialization by sanitarians and kaders to collect data) Salitiga No Yes (Broad pro-poor financing connection assistance, pro-poor tariffs) 349/0 Mixed (In slum areas, which there are few declared slums in city) East Java Yes Surabaya No* Yes (Broad pro-poor Connection assistance/Master Water meter plan, 98% PDAM coverage) N/A N/A Probolinggo Yes No (Use of Hibah, DAK and DAU) 5042/0 Yes (STBM triggering, Musrenbang will propose B40 target, built 86 toiles with various schemes in 2018 (MFI/Loan “pool”) Lumanjang No No (Water Hibah for low cost connections for some HHs) 1400/0 Yes (DAK and DAU for B40. Collecting information by PW to identify poor) SSEI No Makassar Yes No (Working on MOU between MFIs and PDAM to extent PDAM pipelines to poor HHs) 1400/0 Yes (Working with IUWASH B40 “pilot project” in 4 villages [hot pots] to identify direct and indirect resources to meet B40 needs) Barru No N/A N/A N/A Total N = 6 Y = 5 Yes = 3 No = 6 Mixed = 5 Yes 4 149 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV ANNEX 16: SAMPLE SUSTAINABILITY CHECKLIST USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 150 151 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV ANNEX 17: SURVEY RESPONSES REGARDING INFRASTRUCTURE NOT FUNCTIONING AS DESIGNED Is this water or sanitation system working as designed until this time (April 2019)? IUWASH PLUS staff Collaborators Yes 30 (75.0%) 16 (72.7%) No 1 (2.5%) Don’t know or No response 5 (22.7%) Other (e.g.: for water, there is leak but water is still flowing) 9 (22.5%) 1 (4.5%) Total 40 22 Other IUWASH PLUS staff • There are some WWTPs that are not functioning properly. • Some are functioning and some are not functioning, for example Communal Sanitation Systems are damaged, not optimal, there are no managers, no maintenance fees etc. • Not fully in line with expectations because there is some function and some are not functioning • Still valid. But it is less than the maximum use of the facilities. Especially for sanitation facilities, for example, the number of beneficiaries is less than planned. • Some have been running and some are still not running. • Not yet used for sanitation because the project has not been completed. • For WWTP facilities it is still functioning. • Strived to follow the agreed system, but if there are obstacles, anticipated by carrying out discussions and joint solutions • For sanitation, there is a problem with the inlet pipe so it still needs repairs and maintenance training for KPP If Not Functional, Why Not? (Describe) IUWASH PLUS staff response • Because there are still 1-2 tasks from the division of labor that have not been resolved by stakeholders • Especially for the North Sumatra region sanitation, it is still in the process of getting regular desludging • PUPR and Dinkes (health office) did not follow the technical guideline • The location of work makes it difficult to conduct it in accordance with the plan • There is damage to the communal IPAL system while the dues collected from the community would not cover operations and maintenance costs of IPAL • There are usually technical constraints or misperceptions in the processing group, and the challenges of local institutions that still need to be reinforced Collaborator response • The wastewater flowing through the main pipe often time gets clogged due to the lack of awareness by the community in cleaning their grease traps periodically. USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 152 ANNEX 18: ANALYSIS OF MAIN ACTIVITIES ACROSS SEVEN SUSTAINABILITY CHECKLIST The following tables show how main activities across the seven sustainability checklists do not appear to be consistent for activities that should be parallel. The first column summarises the grouping logic for activities that appear to be parallel across sustainability checklists. Cells with a red question mark”?” indicate sustainability checklists with what appear to be missing activities as a result of comparison across checklists. 153 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 154 155 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 156 157 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV ANNEX 19: TOILET SYSTEM SATO TOILET SYSTEM USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 158 PORTABLE TOILET SYSTEM 159 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV ANNEX 20: IUWASH PLUS MTE REVIEW OF 27 POLICIES SUPPORTED BY THE PROJECT THROUGH PY2 City/ District Type of Laws/ Policies/ Regulations/ Agreements No. of Policy Category Subject of Policy Date of Policy Signing Content/ Objective Evaluation Question 4 How has IUWASH PLUS influenced national and city level policy in support of targeting B40 and achievement of SDGs 6.1-6.2 (Line of inquiry - Review of policies that the IUWASH PLUS project had a role in creating with stakeholders that have contributed to national and LG policy related to B40 and SDG 6.1 and 6.2 activities?) Related with GOI WASH Policy (Stated on LG Policy) Do these Policies Address the Poor? (Yes or No) If yes how? Does Policy Support obtainment of SDG 6.1 or 6.2? (directly or indirectly) (Yes or No) If yes, which SDG(s) and how? A. North Sumatra Medan City Agreement 1 Partnership Agreement on Water Supply System (SPAM/ Sistem Penyediaan Air Minum) of Medan City, Binjai City and Deli Serdang District Region in North Sumatera Province 25-Jan-18 Acceleration of implementation of Regional SPAM Drinking Water Systems No Yes SDG 6.1 Development of drinking water infrastructure to improve the quality of the environment and the degree of public health Presidential Regulation of the Republic of Indonesia Number 185 year of 2014 concerning Acceleration of Provision of Water and Sanitation USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 160 Deli Serdang District Regent Regulation 2 UPTD Regent Regulation No.8 Year 2018 on Establishment of Regional Technical Implementing Unit (UPTD/ Unit Pelaksanan Teknis Daerah) in Housing and Settlement Office of Deli Serdang District 20-Feb￾18 Establishment of Regional Technical Implementing Unit (UPTD/ Unit Pelaksanan Teknis Daerah) with organization structure, functional positions and job descriptions No Yes SDG 6.2 Improve community services towards a healthy environment and achieve the goals of the SDGs in the sanitation sector Minister of Public Works and Public Housing Regulation No. 04 / PRT / M / 2017 concerning the Implementation of Domestic Waste Management Systems Pematang Siantar City Mayor Regulation 3 UPTD Mayor Regulation No.13 Year 2018 on Establishment of Regional Technical Implementing Unit (UPTD/ Unit Pelaksanan Teknis Daerah) of Domestic Wastewater in Housing and Settlement Office of Pematangsiantar City 28-May￾18 Establishment of Regional Technical Implementing Unit (UPTD/ Unit Pelaksanan Teknis Daerah) with organization structure, functional positions and job descriptions No Yes SDG 6.2 Improve community services towards a healthy environment and achieve the goals of the SDGs in the sanitation sector Minister of Public Works and Public Housing Regulation No. 04 / PRT / M / 2017 concerning the Implementation of Domestic Waste Management Systems B. West Java/ DKI Jakarta/ Kab. Tangerang Bogor District Regent Regulation 4 Gender Regent Regulation No.78 Year 2017 on Gender-Responsive Planning and Budgeting Guideline 29-Dec￾17 Not relevant with B40 and SDG 6.1 - 6.2 No No 161 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV Bekasi City Regional Regulation 5 Wastewater Management Regional Regulation No.5 Year 2018 on Domestic Wastewater Management in Bekasi City 9-Feb-18 To get a comprehensive domestic wastewater management system, including the rights, obligations , enforcement, and sanction Yes Article 60 Subsection 3: Financing domestic wastewater management for the community Low income can be sourced from APBD and / or other sources legitimate. Yes SDG 6.2 Domestic waste water management to support safely managed sanitation Government Regulation Number 82 year of 2001 concerning Management of Water Quality and Pollution Control Water Tangerang District Regent Regulation 6 Wastewater Management Regent Regulation No.80 Year 2017 on Domestic Wastewater Management 6-Nov-17 To get a comprehensive domestic wastewater management system, including the rights, obligations , and enforcement Yes Article 32 Subsection 2 SPALD service retribution are based on the principles of affordability, fairness, service quality, cost recovery, transparency and accountability Yes SDG 6.2 Domestic waste water management to support safely managed sanitation Minister of Public Works and Public Housing Regulation No. 04 / PRT / M / 2017 concerning the Implementation of Domestic Waste Management Systems Minister of Health Regulation Number 3 year of 2014 concerning STBM (Sanitasi Total Berbasis Masyarakat) USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 162 Tangerang District Regent Regulation 7 STBM Regent Regulation No.22 Year 2018 on Community Total Sanitation Program (STBM/Sanitasi Total Berbasis Masyarakat) 11-Apr￾18 Not enough data. Only scan of page 1 and 12 (missing page 2- 11) Not enough data. Only scan of page 1 and 12 (missing page 2-11) Yes SDG 6.2 Strengthening clean and healthy behavior, preventing environmenta l-based diseases, increasing access basic sanitation Not enough data. Only scan of page 1 and 12 (missing page 2-11) DKI Jakarta Governor Regulation 8 Wastewater Management Governor Regulation No.1 Year 2018 on Septage Management 25-Jan-18 To get a comprehensive domestic wastewater management system, including the rights, obligations , and enforcement No Yes SDG 6.2 Domestic waste water management to support safely managed sanitation Government Regulation Number 82 year of 2001 concerning Management of Water Quality and Pollution Control Water Minister of Public Works and Public Housing Regulation No. 04 / PRT / M / 2017 concerning the Implementation of Domestic Waste Management Systems C. Central Java Surakarta City Mayor Regulation 9 Retribution and Tariff (Wastewater) Mayor Regulation No.5 Year 2018 on Determination of Scheduled Desludging Service Tariff and Water Utility (PDAM) Customer Classification 5-Feb-18 Determination of customer groups and tariffs for each group for L2T2 Yes Appendix Rates for social groups and subsidized households Yes SDG 6.2 Desludging management to support safely managed sanitation 163 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV Surakarta City Mayor Regulation 10 Retribution and Tariff (Wastewater) Mayor Regulation No.6 Year 2018 on Tariff Determination of Offsite Domestic Wastewater Management System and Water Utility (PDAM) Customer Classification of PDAM Kota Surakarta 5-Feb-18 Determination of customer groups and tariffs for each group for SPALD Yes Appendix Rates for social groups and subsidized households Yes SDG 6.2 Desludging management to support safely managed sanitation Surakarta City Agreement 11 Partnership Agreement of PDAM Kota Surakarta and CV Budhi Mulya on Scheduled Desludging Service No. 019.5/ 813.3/PAM dan No. 26/07/BM 26-Jun-18 Desludging operator agreement along with type of work, fees, method of payment and rights and obligations in the context of implementing the L2T2 program No Yes SDG 6.2 Support the target of the Surakarta city government in the framework of sustainable development through proper sanitation by 2030 Surakarta City Agreement 12 Partnership Agreement of PDAM Kota Surakarta and CV. Agung Jaya on Scheduled Desludging Service No. 019.5/ 813.2/PAM dan No. 01/Agung/07/2018 26-Jun-18 Desludging operator agreement along with type of work, fees, method of payment and rights and obligations in the context of implementing the L2T2 program No Yes SDG 6.2 Support the target of the Surakarta city government in the framework of sustainable development through proper sanitation by 2030 USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 164 Surakarta City Agreement 13 Partnership Agreement of PDAM Kota Surakarta and CV. Cahaya on Scheduled Desludging Service No. 019.5/ 813.1/PAM dan CHY.1.0506.2606.2018 26-Jun-18 Desludging operator agreement along with type of work, fees, method of payment and rights and obligations in the context of implementing the L2T2 program No Yes SDG 6.2 Support the target of the Surakarta city government in the framework of sustainable development through proper sanitation by 2030 Surakarta City Agreement 14 Partnership Agreement of PDAM Kota Surakarta and CV. Daffa Jaya on Scheduled Desludging Service No. 019.5/ 891/PAM dan No. 01/Daffa/07/2018 9-Jul-18 Desludging operator agreement along with type of work, fees, method of payment and rights and obligations in the context of implementing the L2T2 program No Yes SDG 6.2 Support the target of the Surakarta city government in the framework of sustainable development through proper sanitation by 2030 165 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV Salatiga City Mayor Regulation 15 UPTD Mayor Regulation No.10 Year 2018 on Establishment of Regional Technical Implementing Unit (UPTD/ Unit Pelaksanan Teknis Daerah) of Domestic Wastewater Plant in Public Work and Housing Office 15-Jan-18 Establishment of Regional Technical Implementing Unit (UPTD/ Unit Pelaksanan Teknis Daerah) with organization structure, functional positions and job descriptions No Yes SDG 6.2 Improve community services towards a healthy environment and achieve the goals of the SDGs in the sanitation sector Salatiga City Mayor Regulation 16 PDAM Tariff Mayor Regulation No.18 Year 2018 on Guideline of Calculation, Determination and Payment of PDAM Water Tariff 14-Mar￾18 Guideline to determine the amount of the drinking water tariff based on customer classification Yes There is a subsidy rate to determine low rates for certain groups Yes SDG 6.1 Determinatio n of tariffs to support the creation of access to healthy drinking water for the community Minister of Home Affairs Regulation Number 71 year of 2016 concerning Calculation and Determination of Drinking Water Tariff Salatiga City Mayor Decree 17 PDAM Tariff Mayor Decree No. 500/ 146/2018 on Drinking Water Tariff of PDAM Salatiga City 2018 22-Mar￾18 Determine the amount of the drinking water tariff based on customer classification Yes Appendix Rates for social groups and low income households Yes SDG 6.1 Determinatio n of tariffs to support the creation of access to healthy drinking water for the community Minister of Home Affairs Regulation Number 71 year of 2016 concerning Calculation and Determination of Drinking Water Tariff D. East Java USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 166 Sidoarjo District Regent Regulation 18 UPTD Regent Regulation No.91 Year 2017 on Technical Implementing Unit (UPT/ Unit Pelaksanan Teknis) in Office of LG Sidoarjo District 21-Dec￾17 Establishment of Regional Technical Implementing Unit (UPT/ Unit Pelaksanan Teknis) with organization structure, functional positions and job descriptions of Local Government No Not specific for B40. Only establishment of UPT No Not specific for SDG. Only establishment of UPT Probolinggo City Mayor Regulation 19 STBM Mayor Regulation No.102 Year 2018 on Community Total Sanitation Program (STBM/Sanitasi Total Berbasis Masyarakat) in Probolinggo City 1-Aug-18 Regulate total sanitation implementation through sanitation and behavior changes and hygiene promotion Yes Not specifically, but the main target for OD is actually the B40 community Yes SDG 6.2 Strengthening clean and healthy behavior, preventing environmenta l-based diseases, increasing access basic sanitation Government Regulation Number 66 year of 2014 concerning Environmental Health 167 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV Malang City Regional Regulation 20 Wastewater Management Regional Regulation No.2 Year 2017 on Domestic Wastewater Management in Malang City 6-Feb-17 To get a comprehensive domestic wastewater management system, including the rights, obligations , and enforcement Yes Article 23 Subsection 4 Providing assistance for the development of residential environmenta l sanitation in the form of funding or development to poor people Yes SDG 6.2 Domestic waste water management to support safely managed sanitation Regulation of the Minister of Environment Number 112 year of 2003 concerning Domestic Waste Water Quality Standards Minister of Public Works Regulation Number 16 / PRT / M / 2008 concerning National Policy and Strategy for Domestic Settlement Water Management System Development Lumajang District Regent Regulation 21 UPTD Regent Regulation No. 79 Year 2017 on Establishment of Regional Technical Implementing Unit (UPTD/ Unit Pelaksanan Teknis Daerah) of Domestic Wastewater Management in Public Work and Housing Office in Lumajang District 27-Dec￾17 Establishment of Regional Technical Implementing Unit (UPTD/ Unit Pelaksanan Teknis Daerah) with organization structure, functional positions and job descriptions No Yes SDG 6.2 Improve community services towards a healthy environment and achieve the goals of the SDGs in the sanitation sector Minister of Public Works and Public Housing Regulation No. 04 / PRT / M / 2017 concerning the Implementation of Domestic Waste Management Systems USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 168 Lumajang District Regional Regulation 22 Wastewater Management Regional Regulation No.6 Year 2018 on Domestic Wastewater Management in Lumajang District 18-Apr￾18 To get a comprehensive domestic wastewater management system, including the rights, obligations , and enforcement Yes Article 66 Subsection 3 Individual and communal SPALD-S financing in low-income areas comes from the Regional Budget and / or other legitimate sources in accordance with statutory regulation Yes SDG 6.2 Domestic waste water management to support safely managed sanitation Presidential Regulation Number 185 year of 2014 concerning Acceleration of Provision of Water and Sanitation Minister of Health Regulation Number 3 year of 2014 concerning STBM (Sanitasi Total Berbasis Masyarakat) Minister of Public Works Regulation Number 16 / PRT / M / 2008 concerning National Policy and Strategy for Domestic Settlement Water Management System Development E. South Sulawesi/ Eastern Indonesia Bantaeng District Regional Regulation 23 PDAM Equity Participation Regional Regulation No.7 Year 2017 on Equity Fund of LG Bantaeng District to PDAM 31-Jul-17 Ensuring local government’s support to PDAM by APBD budget allocation No Yes SDG 6.1 Equity participation to improve PDAM performance in creating universal access that will impact SDG 6.1 169 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV Bantaeng District Regent Regulation 24 PDAM Tariff Regent Regulation No. 26 Year 2017 on Drinking Water & Non Drinking Water Tariff Adjustment in Water Utility (PDAM) 31-Jul-17 Determine the amount of the drinking water tariff based on customer classification Yes Appendix Rates for social groups and low income households Yes SDG 6.1 Determinatio n of tariffs to support the creation of access to healthy drinking water for the community Minister of Home Affairs Regulation Number 71 year of 2016 concerning Calculation and Determination of Drinking Water Tariff Barru District Regional Regulation 25 PDAM Equity Participation Regional Regulation No.1 Year 2018 on Equity Fund in PDAM Tirta Waesai Barru District 25-Jun-18 Ensuring local government’s support to PDAM by APBD budget allocation No Yes SDG 6.1 Equity participation to improve PDAM performance in creating universal access that will impact SDG 6.1 Ternate City Regional Regulation 26 Wastewater Management Regional Regulation No.6 Year 2018 on Domestic Wastewater Management in Ternate City 13-Aug￾18 To get a comprehensive domestic wastewater management system, including the rights, obligations , and enforcement Yes Article 58 Carry out city-scale wastewater treatment, settlement scale and certain scale for low￾income communities Yes SDG 6.2 Domestic waste water management to support safely managed sanitation USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 170 Ternate City Mayor Regulation 27 UPTD Mayor Regulation No.16 Year 2018 on Establishment of Regional Technical Implementing Unit (UPTD/ Unit Pelaksanan Teknis Daerah) of Domestic Wastewater Management in Public Work and Housing Office 17-Sep￾18 Establishment of Regional Technical Implementing Unit (UPTD/ Unit Pelaksanan Teknis Daerah) with organization structure, functional positions and job descriptions No Yes SDG 6.2 Improve community services towards a healthy environment and achieve the goals of the SDGs in the sanitation sector 171 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV ANNEX 21: URBAN DOMESTIC WASTEWATER MANAGEMENT USAID.GOV IUWASH PLUS MIDTERM EVALUATION REPORT | 172 ANNEX 22: MEMBERS OF THE MIDTERM EVALUATION REVIEW TEAM TEAM COMPOSITION Kay Mattson, Team Leader. Kay has over 25 years of experience in international public health, program planning and management in low-income housing, human service, and health administration/policy analysis. She has an MPH in International Health, an MSW in Planning, Administration and Management and a B.A. in Sociology. Her international work focuses on providing technical assistance, and conducting assessments and evaluations on water, sanitation and hygiene (WASH) and public health projects/systems as an independent consultant. She has also served as an instructor for George Washington University Milken Institute School of Public Health. In addition to the US, she has worked in 11 developing countries with over a dozen international organizations. Ari Prasinto Jati, Sanitation Market Development Specialist. Mr. Jati is a Sanitation Marketing Consultant with a background in Environmental Biology’s Science and Master Degree in Marketing from Gadjah Mada University, Yogyakarta Indonesia. He has a solid experience in marketing for 15 years. Especially in the field of sanitation marketing, he has some experience working in multinational sanitation product’s company and consultant background in Water and Sanitation Program (WSP) World Bank and grantees program from MCA-Indonesia (Millennium Challenge Account - Indonesia). Nona Sri. K Pooroe Utomo, Behaviour Change Specialist. Nona has more than 20 years of experienced in managing behavior change communication intervention on health, education, water & sanitation, gender and social change endeavor in humanitarian field. She has participated in several high-level project evaluations including the IUWASH MTE, an assessment of the effectiveness of a Canada-Indonesia youth exchange program, and a 2018 effort to capture lessons learned from the MCAI project. Yashin Lin, Evaluation Specialist. Yashin is a program evaluation specialist with 18 years working in global health technical assistance projects. She has worked with various methods to measure, evaluate, and improve programs, with special interest in health systems and sustainability. Yashin has conducted performance evaluations in Zambia, Vietnam, and Angola, and she holds a Master's degree in Public Health in epidemiology from the University of Washington. MESP Logistical Support. Technical support and direction for the evaluation will be provided by MSEP Chief of Party Kusumastuti Soepardjo and Technical Director David Callihan. Retno Sri Handini of the MESP team will provide general oversight and research in support of the evaluation. Scheduling, coordination, and logistics will be managed by MESP staff (Felisita Lethe evaluation research assistant), in coordination with DAI implementing partners and USAID. MSI headquarters in Washington will provide additional technical support, as required. LEVEL OF EFFORT The level of effort (LOE) for the IUWASH PLUS evaluation team ranges from 40 days for research assistants to 60-65 days for international evaluation specialists and the team lead. The following table summarizes the estimated LOE for the evaluation, by team position and task. 173 | IUWASH PLUS MIDTERM EVALUATION REPORT USAID.GOV Position Management/ Planning Doc Review Initial Briefing - Meetings Methodology Development Data Collection Analysis Drafting Final Report Presentation/ Workshop Total Days Team Leader 5 6 1 7 14 10 10 9 3 65 WASH M&E Specialist 5 6 1 7 14 10 9 6 2 60 Behavior Change Specialist 1 4 1 5 14 5 4 4 2 40 Marketing Sanitation Specialist 1 4 1 5 14 5 4 4 2 40 Research Assistant 1 4 1 5 14 5 4 4 2 40