This publication was produced for review by the United States Agency for International Development (USAID). It was prepared by Three Stones International. The opinions and contents of this report are the sole responsibility of Three Stones International and do not necessarily reflect the views of the United States Agency for International development or U.S. Government. EVALUATION REPORT RWANDA RURAL SANITATION / ISUKU IWACU ACTIVITY FINAL EVALUATION FINAL REPORT FEBRURARY 2021 PREPARED BY THREE STONES INTERNATIONAL 26 November 2020 RURAL SANITATION ACTIVITY FINAL EVALUATION For SNV USA, ISUKU IWACU Project 2 Table of Contents Acronyms and Definitions..................................................................................................................... 3 Executive Summary ............................................................................................................................... 4 Introduction .......................................................................................................................................... 8 Isuku Iwacu Rural Sanitation Activity .................................................................................................... 8 IR1: Increased demand for sanitation products and services....................................................................... 8 IR2: Improved supply and availability of sanitation products and services in the private sector.................. 9 IR3: Improved governance for sustained access to sanitation and hygiene products and services............... 9 Isuku Iwacu Theory of Change......................................................................................................................... 9 Evaluation Purpose and Objectives..................................................................................................... 10 Methodology....................................................................................................................................... 10 Demographic Characteristics............................................................................................................... 11 Survey Participants....................................................................................................................................... 11 Qualitative Interview Respondents............................................................................................................... 12 Findings............................................................................................................................................... 14 Relevance ..................................................................................................................................................... 15 Effectiveness................................................................................................................................................. 19 Efficiency ...................................................................................................................................................... 26 Impact .......................................................................................................................................................... 27 Sustainability and Scalability ......................................................................................................................... 29 Sustainable Outcomes .............................................................................................................................. 32 Conclusions & Recommendations....................................................................................................... 33 Tables Table 1: Water Source Use by Intervention....................................................................................................... 11 Table 2: Focus Group Discussion Respondent Characteristics............................................................................ 12 Table 3: Key Informant Interview Details........................................................................................................... 13 Table 4: Results per Indicators Measured in Baseline and Final Evaluation........................................................ 14 Table 5: Adoption of Hygienic Latrine Behaviors ............................................................................................... 27 Figures Figure 1: Theory of Change Assumptions............................................................................................................ 9 Figure 2: Demographic Characteristics- Type of House Construction................................................................. 11 Figure 3: Source of Income based on Intervention Type.................................................................................... 11 Figure 4: Key Influencers for Building/ Improving Latrines................................................................................. 20 Figure 6: Observed Latrine Hygiene................................................................................................................... 21 Figure 7: Type of Loan Used to Finance Latrine ................................................................................................. 24 Figure 8: Types of Handwashing Facility by Intervention ................................................................................... 28 3 ACRONYMS AND DEFINITIONS AMIR Association of Microfinance Institutions in Rwanda BCC Behavior change communications CHC Community Health Club CLTS Community Led Total Sanitation DSC District Sanitation Centers DWA District Wide Approach EVD Ebola Virus Disease FGD Focus Group Discussion GBV Gender- Based Violence GoR Government of Rwanda HWWS Hand Washing with Soap IDIs In- Depth Interviews IR Immediate Results JADF Joint Action Development Forum KII Key Informant Interview LCSO Local Civil Society Organization MEL Monitoring, Evaluation, and Learning MFI Microfinancing Institution MHM Menstrual Hygiene Management MINALOC Ministry of Local Government MININFRA Ministry of Infrastructure MIS Management Information System MOH Ministry of Health NECDP National Early Childhood Development Program NGO Non-Governmental Organization NISR National Institute of Statistics Rwanda NSP ODF National Sanitation Policy Open Defecation Free PPE Person Protective Equipment SACCO Savings and Credit Cooperative Organizations - A government initiative aimed at providing rural Rwandans with loans to improve their earnings and enhance their livelihoods. SILC Savings and Internal Lending Communities SNV Stichting Nederlandse Vrijwilligers (Netherlands Development Organization) SRH Sexual and Reproductive Health TBS Total Basic Sanitation TSI Three Stones International UNICEF United Nations Children’s Fund USAID United States Agency for International Development USG United States Government VSLA Village Savings and Loan Association – A community group whose members voluntarily save small amounts of money, which are used to make small interest￾based loans to the group’s members. WASAC Water and Sanitation Cooperation WASH Water, Sanitation and Hygiene WATSAN Water and Sanitation Sector Rwanda ZOD Zero Open Defecation 4 EXECUTIVE SUMMARY Introduction The Isuku Iwacu Activity (also known as Rwanda Rural Sanitation Activity) is a four-year rural sanitation project, funded by the United States Agency for International Development (USAID). The Activity is executed by a consortium of Non-Governmental Organizations (NGOs) headed by SNV USA and includes SNV Rwanda, World Vision International and Water for People. The consortium implemented activities in 8 districts to help a targeted 500,000 people gain access to basic household sanitation and hygiene. The overall goal of the Isuku Iwacu Activity is to increase local ownership and capacity to deliver sustainable, high quality sanitation and hygiene services designed to tackle childhood stunting. Evaluation Purpose and Objectives The purpose of this endline evaluation was to understand whether the intended objectives of Isuku Iwacu have been achieved, by comparing end-point core indicator results with the Activity’s 2016 baseline, as set out in the Monitoring, Evaluation, and Learning (MEL) plan. The evaluation also explored enabling and disabling factors influencing the outcomes of the Activity from which recommendations were drawn to inform the design of future interventions. The scope of the evaluation encompasses the implementation of the Activity from its start date in September 2016 to closure of the Activity in November 2020. The geographical scope of the evaluation included six districts selected from each province in which Isuku Iwacu works in order to represent the eight target districts as follows: Nyabihu, Ruhango, Nyanza, Nyarugenge, Ngoma, and Rwamagana. The specific evaluation objectives included: 1. Establishment of an understanding of how the strategy, implementation, and results of Isuku Iwacu activities in Rwanda have contributed to the overall goal of the project. 2. Highlighting the strengths and weaknesses of different activities of Isuku Iwacu, focusing on their effectiveness, efficiency, relevance, impact, and sustainability. 3. Identification of enabling and disabling factors that affected the implementation of Isuku Iwacu activities. 4. Identification of lessons learned and provision of recommendations for future projects and for learning in relation to WASH project programming. Based on the evaluation terms of reference, the evaluation scope covered five key domains for investigation relating to the Isuku Iwacu Activity implemented from September 2016 to November 2020, including relevance, effectiveness, efficiency, impact and sustainability. Methodology The evaluation employed a mixed methods approach, with a focus on generating comprehensive quantitative and qualitative data. Key Findings1 Evaluation Criteria Key Findings Relevance The extent to which Isuku Iwacu Activity’s objectives and its interventions have been consistent and aligned to the GoR priorities and USAID strategic priorities. • The Activity was designed to address governance, household financing of sanitation, supply, and behaviors, all of which global learnings have illustrated to be essential to sustain sanitation and hygiene improvements. • Isuku Iwacu’s design to use market-based approaches to create demand for basic sanitation, through improved access to products and services and stronger governance, is relevant to addressing weak supply chains, limited transport options and limited technical expertise. This has advanced progress towards sustainable development targets, while spurring local economic activity and communal action. • In addition to creating a ‘ways of working’ document, Isuku Iwacu employed the original Activity design in late 2018 and engaged Local Civil Society Organizations (LCSOs) to accelerate community 1 Figures based on evaluation survey conducted in October 2020, thus represents a snapshot of the situation at the time of the evaluation and do not represent total cumulative figures achieved by the Isuku Iwacu Activity. 5 Evaluation Criteria Key Findings engagement and enhance sanitation marketing approaches. These were deemed to be relevant actions in response to the mid- term evaluation recommendations. • The ODF Training methodology itself is relevant to the context of Rwanda as it builds upon the key cultural principal of communal support for development – umuganda – and accountability -imihigo - to improve sanitation. • The creation of District Sanitation Centers (DSCs) to introduce technologies and services is clearly aligned with the NSP and the GoR’s approach to improving the sanitation market. Effectiveness Whether the project interventions were effective within the benefiting communities, and which objectives were achieved and to what extent have they contributed to the strategic objectives. • Isuku Iwacu effectively increased household ownership of a basic latrine from 55% at the baseline to 67% in the final evaluation2 . • Facilitating the development of the sanitation market, coupled with the implementation of the ODF training and targeted subsidies, and sanitation and hygiene behavior change promotion through CHCs, social media, and community events effectively provided a package that addresses the needs of all people in a community, including the most vulnerable. • 93% of households from ODF predeclared and 100% of households from ODF trained communities were observed to have a functioning latrine. While not all functioning latrines met the standards for basic sanitation, this clearly illustrates the effectiveness of the ODF Training Approach to trigger whole communities towards improved basic sanitation, and stimulate households to invest their own resources to ensure that they have their own latrine. • Across Isuku Iwacu support, the hygiene of latrines improved in comparison to the baseline. The final evaluation survey results show 76% of household latrines having no visible signs of feces, compared with only 59% observed during the baseline. Latrines in communities that received the ODF training were notably more hygienic than those in communities that didn’t receive the training. Efficiency Whether project activities were perceived to be cost￾efficient, were achieved on time and were implemented in the most efficient way. • Use of a fee-for-service basis for engagement of service providers (LCSOs and AMIR) not only accelerated achievement of results, but also drove transparency – making results, as well as barriers, visible in a way that enabled LCSOs to learn and adapt their way of working to be more effective • The uptake of loans within the short period of time before COVID-19 lockdown measures, and the continued issuing of loans during lockdown, were noted to be impressively efficient. In addition to having effective mechanisms for matching qualified households with sanitation improvement needs with the loan, AMIR and the SACCOs were financially motivated to ‘sell’ the loan products. • In a short amount of time the LCSOs were efficient in the level of outreach they were able to achieve, linking BCC messages to improve sanitation to the promotion of the loan product. This was an effective link to trigger household demand to improve their latrine with supply of a product to finance the improvement. Impact Changes over time in demand for sanitation products and services; Improved Household Infrastructure • The final evaluation shows a 12- percentage point increase in household ownership of an improved latrine, from 55% in August 2017 to 67% in October 2020, this is a notable achievement. 2 Total number of households surveyed in final evaluation survey was 494 out of the 126,080 households accessing basic sanitation as a result of the Isuku Iwacu Activity 6 Evaluation Criteria Key Findings improvement in supply and availability of sanitation products and services in the private sectors; and improvement in governance for sustained access to sanitation and hygiene products and services. Hygiene • An 11- percentage point increase in household handwashing facilities between the Activity’s baseline and this endline is noted, which could also be in response to the national COVID-19 prevention measures. • There is a significant difference between ownership of tippy taps by households that benefited from specific Isuku Iwacu interventions (52% voucher and 46% ODF Training recipients owned tippy taps) than with those households that did not participate in ODF training (9% owned tippy taps). Considering Isuku Iwacu’s emphasis in promoting tippy taps, this difference in coverage, suggests attribution to Isuku Iwacu’s efforts. Gender and Equity • Key approaches that the evaluation recognized and believed contribute to gender and equity in sanitation and hygiene improvements include: Deliberate engagement of both women and men in community training and outreach activities, as well as business engagement and trainings; and the requirement of the agreement from both adult partners in a household, before a SACCO agrees to issue a loan of loan product. Access and availability of sanitation products • While access to, and the cost of materials was reported by FGD participants to be a key constraint to building establishing basic sanitation, there is evidence that households are purchasing materials to upgrade their sanitation. Diversification of their product line and linkages with Kigali wholesalers has allowed some DSC managers to respond to consumer demand as sale needs change. Sustainability The extent to which program outcomes are likely to be sustained based on changes in local ownership, capacity or perception of the intervention. Institutionalize systems for basic sanitation • Isuku Iwacu’s deliberate use of key GoR strategies and priorities namely the District Sanitation Centers, the National Social Behavior Change Communications Strategy for Integrated Early Childhood Development, Nutrition and WASH, and local market-based solutions in their Activity design optimizes GoR buy-in and ownership of approaches. Isuku Iwacu also prioritized the engagement and capacity building of district and local leadership to promote and monitor household sanitation and hygiene coverage, mainly through the ODF process. Market based solutions • There is strong national and district ownership of the District Sanitation Centers. All stakeholders indicate the importance of supply chain linkages with wholesalers from the national level and from DSCs to other sellers. Isuku Iwacu concerted significant effort to strengthening the supply chain, linking wholesalers with DSCs and engaging companies with masons to support latrine building in communities. • Providing financing options to access desired product and services to improve sanitation is essential for Rwanda to reach universal access to basic sanitation and hygiene services. Loan products offer an exciting financing option for households that may not have immediate cash for purchases, but have agency and assets. Basic household sanitation • Household investment in their own improved latrine is a key indicator of sustainability, and 32% of households in ODF pre-declared communities that were surveyed had invested their own finances in improving latrines, with another 50% indicating that they plan to make further upgrades. The majority of those households that had invested their 7 Evaluation Criteria Key Findings own finances report to be in Ubudehe category 3 (53%), followed by Ubudehe category 2 (40%) and Ubudehe category 1 (7%). • Only 10% of households were observed to have no cleaning supplies near their latrine and over 60% reported cleaning their latrines on a daily basis. When households value something enough to invest in it, intend to make further improvements, and have systems to maintain – these are all positive signs for sustainable use of the infrastructure. Voucher System • Isuku Iwacu’s market-based approach contributed to increased awareness and helped to strengthen the supply chain, setting the stage for future interventions. If a similar private-sector driven voucher system is to be implemented in the future, improvements on customer service is recommended. Recommendations for future programming 1. Advocate for complementary approaches to appropriately target market- based solutions – targeted sustainable subsidy system , formal and self- financing options (financial institution and community￾based loans) and sanitation market 2. Continue to ensure access to sanitation for vulnerable households, complementing any social protection and financing mechanisms with appropriate behavior change promotion to optimize sustainable use and maintenance of resulting basic sanitation. 3. Support supply chain strengthening- ensure that District Sanitation Centers are linked to national markets and community level markets 4. Refine the ODF process name and focus to one that reflects the Government of Rwanda’s vision of achieving universal basic sanitation by 2024 and universal access to safely managed sanitation by 2030. 5. Consideration should also be made for further engaging with the existing GoR structures to continue monitoring and sustaining household sanitation and hygiene behaviors. Strengthening and aligning CHW responsibility and activity on WASH promotion with CBEHPP will foster sustainability and integration within an accountable government system. 6. Continue to complement district level Activity implementation with advocacy efforts for national level engagement and capacity development, in order to facilitate taking successful sanitation market models to scale. 8 INTRODUCTION The Isuku Iwacu Activity (also known as Rwanda Rural Sanitation Activity) is a four-year rural sanitation project, funded by the United States Agency for International Development (USAID). The Activity is executed by a consortium of Non-Governmental Organizations (NGOs) headed by SNV USA and includes SNV Rwanda, World Vision International and Water for People. The consortium implemented activities in 8 districts to help a targeted 500,000 people gain access to basic household sanitation and hygiene by increasing government and private sector ownership and capacity to deliver sustainable, high-quality sanitation and hygiene services designed to tackle childhood stunting. Three Stones was contracted to conduct a final evaluation to assess the impact of the Rural Sanitation Activity – Isuku Iwacu – to understand whether the intended objectives have been achieved, by comparing end-point core indicator results with the Activity’s 2016 baseline, and as set out in the Monitoring, Evaluation, and Learning (MEL) plan. The final evaluation would like to acknowledge the fact that Isuku Iwacu’s last three quarters of implementation were complicated by the emergence and severity of the global COVID-19 pandemic. Lockdown restrictions and redirected efforts to support prevention measures, delayed, and in some cases cancelled, planned activities. For a market-based activity, the economic ramifications of COVID-19 also impact household ability to self-finance sanitation improvements and commit to loans. ISUKU IWACU RURAL SANITATION ACTIVITY The Isuku Iwacu Activity was designed to contribute to improved access and promote safe and consistent use of household sanitation and hygiene facilities. In 2017, Isuku Iwacu conducted a baseline as well as formative research to understand current WASH coverage, as well as barriers preventing and factors enabling households in improving their sanitation. The main barrier identified was a lack of means and resources to build and maintain basic sanitation and adequate handwashing facilities. Knowledge of these key WASH behaviors and their benefits was high, but lack of funds, access to supply, and awareness of latrine technology resulted in low levels of practice.3 The overall goal of the Isuku Iwacu Activity was to increase local ownership and capacity to deliver sustainable, high quality sanitation and hygiene services designed to tackle childhood stunting. To accomplish this goal, Isuku Iwacu sought to improve access to and encourage correct, consistent use of household sanitation and hygiene facilities. The strategies used to achieve the Strategic Objective were: (i) Directly supporting and contributing to the Government of Rwanda’s (GoR) led efforts to improve access to sanitation in Rwanda, and (ii) Promoting districtwide, private sector-driven household sanitation and hygiene interventions and on district- and national-level capacity development.4 . Three intermediate results (IRs) contributed directly to achieving the SO: IR1: Increased demand for sanitation products and services Isuku Iwacu increased community and household demand through a three-pronged approach. 1. Implementation of the Open Defecation Free (ODF) approach, a modification of Community Led Total Sanitation (CLTS) drives community demand to ensure that all households have their own latrine through participatory activities that highlight the danger if even a minority continues to defecate in the open everyone is at risk. The activities trigger community desire for collective change and encourage sanitation innovation. 2. Isuku Iwacu also used behavior change communications (BCC) to drive awareness not only of the importance of sanitation and hygiene, but also of product and service options for sanitation and hygiene improvements. BCC messages were linked to the delivery of the ODF approach to provide viable options to households who are propelled into action to improve their sanitation. 3. Isuku Iwacu targeted all wealth quintiles for sanitation and hygiene improvements. Recognizing that Ubudehe 1 and 2 households struggle to afford and access products and services, Isuku Iwacu provided a voucher system that linked households with companies that constructed improved latrines 3 USAID-funded Isuku Iwacu. Rwanda Rural Sanitation Activity, Baseline Survey Report. Rwanda. 2018 4 Globalwaters.org, Rwanda Rural Sanitation Activity (Isuku Iwacu) 9 for the households. This not only improved a household’s sanitation, but it provided basic sanitation models for neighboring households and showcased company sanitation services within communities. IR2: Improved supply and availability of sanitation products and services in the private sector Isuku Iwacu worked with the private sector to introduce and promote affordable sanitation and hygiene products and services that address local challenges and preferences. District Sanitation Centers (DSCs) were established to market WASH products and services, showcase sanitation technologies, and to facilitate the training of professionals in WASH service delivery. Isuku Iwacu also collaborated with the Association of Microfinance Institutions in Rwanda (AMIR) and local Savings and Credit Cooperative Organizations (SACCOs) to establish and offer microfinancing products to households for sanitation and hygiene improvements. IR3: Improved governance for sustained access to sanitation and hygiene products and services To promote sustained access to improved sanitation and hygiene, Isuku Iwacu recognized that national, district and local government ownership around programs and results monitoring is essential. Isuku Iwacu worked with government at all levels to strengthen capacity to manage government sanitation and hygiene priorities, monitor progress, and take appropriate actions in response to monitoring results. Isuku Iwacu Theory of Change If the Isuku Iwacu Activity takes an integrated approach to increasing demand for improved sanitation and hygiene for all quintiles, And if the Activity facilitates the adoption and use of systems to monitor, regulate, finance supply, and create profitable sanitation businesses, Then demand will increase, and targeted Rwandans will be motivated and capacitated to move up the sanitation ladder. This will be a major step toward the Government of Rwanda’s health and sanitation goals. The theory of change underlying Isuku Iwacu’s Activity suggests that an increase uptake of basic household sanitation will be achieved when demand, supply and enabling environment are in alignment, as shown in Figure 1. The validity of this theory of change depends on several assumptions. Private sector entities must exist and be motivated to participate in the Activity by seeing the benefits to their businesses. Local government must have the capacity to engage in oversight, monitoring, and decision making. Households must have the means to engage in market systems. Figure 1: Theory of Change Assumptions Demand: The household’s need or desire to own an improved sanitation or hygiene product or experience a service. Supply: Includes design, production, sales, and installation of sanitation and hygiene products, desludging services, and financing arrangements. Enabling Environment: The role of the political economy, institutional and regulatory framework, infrastructure, and governance in supporting the private sector to start up and run sanitation and hygiene businesses and enabling households to access those businesses. 10 EVALUATION PURPOSE AND OBJECTIVES The purpose of this endline evaluation was to understand whether the intended objectives have been achieved, by comparing end-point core indicator results with the Activity’s 2016 baseline, as set out in the Monitoring, Evaluation, and Learning (MEL) plan. The evaluation also explored enabling and disabling factors influencing the outcomes of the Activity from which recommendations were drawn to inform the design of future interventions. The scope of the evaluation encompasses the implementation of the Activity from its start date in September 2016 to closure of the Activity in November 2020. The geographical scope of the evaluation included six districts selected from each province in which Isuku Iwacu works in order to represent the eight target districts as follows: Nyabihu, Ruhango, Nyanza, Nyarugenge, Ngoma, and Rwamagana. The specific evaluation objectives included: 1. Establishment of an understanding of how the strategy, implementation, and results of Isuku Iwacu activities in Rwanda have contributed to the overall goal of the project. 2. Highlighting the strengths and weaknesses of different activities of Isuku Iwacu, focusing on their effectiveness efficiency, relevance, impact, and sustainability. 3. Identification of enabling and disabling factors that affected the implementation of Isuku Iwacu activities. 4. Identification of lessons learned and provision of recommendations for future projects and for learning in relation to WASH project programming. The detailed evaluation questions, organized by the five key domains, approach to exploring the questions, and the analysis plan can be found in Annex B. The evaluation also assessed the uptake and influence that the mid-term evaluation recommendations made. The mid- term evaluation considered implementation conducted between September 2016 through Fiscal Year 2019, Quarter 2, providing approximately one year of Activity implementation to execute recommendations. METHODOLOGY To answer the evaluation questions Three Stones developed a mixed methods approach, in a representative sample of the target districts, to facilitate a robust analysis and triangulation of findings across the domains of the evaluation. Data collection generated findings from the following approaches: • A comprehensive desk review that explored both sanitation sector specific documents (national policies, strategies, and research), as well as internal program documents to understand the Activity’s strategies and implementation mechanisms within current Rwandan and global context. • 494 household surveys and sanitation and hygiene observations • 30 district key informant interviews • 9 national key informant interviews • 18 community focus group discussions Quantitative analysis of the survey data was analyzed using R, for cross tabulation and descriptive analysis. Qualitative data transcripts were analyzed thematically against the qualitative evaluation questions (deductive analysis). Key themes were also identified during reading, coding (inductive analysis-grounded approach), and quantified to determine level of importance. Transcripts were reviewed and data was coded in an excel spreadsheet according to themes found within each of the evaluation question areas. The detailed methodology, providing data collection, tools, site and participant selection, and quantitative and qualitative analysis approaches, is presented in Annex C. 11 DEMOGRAPHIC CHARACTERISTICS Survey Participants The household (HH) survey captured summary descriptive statistics on a total of 494 households. Basic demographic data, for both the total population as well as disaggregated by gender for Isuku Iwacu beneficiaries is presented below in the following tables and graphs. The average age of the total survey respondents was 51.2 years, with 37% of household’s being female- headed and the average number of children under five per household was 1. The average household size was 4.9 persons per household. Little variation was seen in these indicators across the Activity’s different intervention type or comparing by gender. The vast majority of respondents across all disaggregates owned their own home (96%) with just 4% living in rented accommodation. Houses were primarily made of stone with mud or cement (33%), concrete or brick (32%) or bamboo or straw with mud (25%). Over half of respondents had a fibrous cement roof (55%). There was little variation disaggregated by intervention or gender. Almost three quarters of respondents (72%) reported either having no formal education or not finishing primary school. Only 10% of respondents had completed secondary education or higher. In the majority of households, the main source of income was agriculture (38%). However, there was a significant spread in sources of income, especially between intervention types as seen in Figure 4. Without significant variation across disaggregation, the majority of households were found to be using an improved source of water, with almost three quarters (72%) relying on piped water into yard, a public tap, or a protected public well, as shown in Figure 5. 58% of the households spent less than 30 minutes to collect water, including the trip to and from the water source. Only 13% of respondents required more than an hour to collect water. Table 1: Water Source Use by Intervention Household Water Source % Access (n=494) % Rural access according to 2016 EICV5 Piped into yard /plot 11% 2.3% Figure 2: Demographic Characteristics- Type of House Construction Figure 3: Source of Income based on Intervention Type 12 Household Water Source % Access (n=494) % Rural access according to 2016 EICV5 Public tap 44% 33.9% Protected public well 17% 1.1% Total improved source 72% 37.3% Open public well 15% 0.6% Spring 22% 52.8% River/stream/pond/lake 18% 5.2% Rainwater harvesting 7% 0.1% While 41% reported saving money on a weekly basis, 41% also reported rarely or never saving household money. The highest rates of agriculture as a source of income and highest rates of never saving household money were found amongst voucher recipients. Over 65%of households reported belonging to a savings group, with the vast majority belonging to a VSLA and a further 35% belonging to a SILC. Slightly over half of respondents reported having a bank account (54%), with the vast majority of these respondents having a SACCO account (73%). Despite the surprisingly high rate of households belonging to a savings group or having access to a bank, only 9% of these respondents reported ever taking a loan. Qualitative Interview Respondents Focus group discussions were conducted for voucher, self- financing and non- recipient beneficiaries as described in Annex (Data Collection Procedure section). In- depth interviews were conducted with loan recipients instead of FGDs, as this group made up a small proportion of the overall beneficiary groups. Table 2 details the characteristics of FGD respondents and Table 3 the KIIs. Table 2: Focus Group Discussion Respondent Characteristics Characteristic Focus Group Discussions Female Male Total Total % Total % Total % Category Vouchers 25 74% 9 26% 34 31% Self- Financing 19 53% 17 47% 36 32% Non- Recipient 19 54% 16 46% 35 32% Loan Recipient 1 17% 5 83% 6 5% Total 64 58% 47 42% 111 100% Age 20-29 5 8% 2 4% 7 6% 30-39 16 25% 13 28% 29 26% 40-49 13 20% 12 26% 25 23% 50-59 14 22% 8 17% 22 20% 60-70 13 20% 9 19% 22 20% >70 3 5% 3 6% 6 5% Total 64 58% 47 42% 111 100% 13 Table 3: Key Informant Interview Details Location Key Informant Interview National Executive Director and Senior Program Manager, AMIR USAID Chief of Party, Isuku Iwacu Chief of WASH, UNICEF Country Director, Water for People Behavior Change Communication Advisor, World Vision Head of the Environmental Health Department, Ministry of Health WASH Division Manager, Ministry of Infrastructure District Local Civil Society Organizations District Sanitation Centers Local Government including Executive Secretaries and District JADF coordinators Community SACCOs Loan Recipients 14 FINDINGS The sections below address the five key evaluation domains, including the relevant evaluative questions, key findings from the desk review, qualitative and quantitative data, and discussion, providing further elaboration of details from the evaluation activities. The final evaluation assesses the contribution of the three intermediate results towards achieving the Activity’s Strategic Objective, across the five evaluation domains of relevance, effectiveness, efficiency, impact and sustainability. Intermediate results include: IR1: Increased demand for sanitation products and services; IR2: Improved supply and availability of sanitation products and services in the private sector; and IR3: Improved governance for sustained access to sanitation and hygiene products and services. The evaluation broke IR2 into two areas for ease of assessment: 1) Supply strengthening to reinforce demand generation and increase availability and access to sanitation and hygiene products and services; and 2) Financing to enable all wealth quintiles in Rwanda to afford basic household sanitation and hygiene. Basic sanitation is defined as a an improved, hygienic and private latrine. In Rwanda, the superstructure, including the roof and door are critical to be categorized as an improved latrine. Key components of an improved latrine include: 1. Pit: big/ large enough to last 10 years and safe enough so it does not collapse; 2. Slab: hygienic and removes waste from human contact and flies; and 3. Superstructure: provides privacy and protection from rain5 . The slab can be made from mud, wooden planks, Sato pan, concrete slab, or plastic squat as long as it durable, cleanable and solid. The overall evaluation findings are that the Isuku Iwacu Activity was successful in creating demand for basic household sanitation and introducing market-based solutions to address the demand. In the period from September 2016 to November 2020, a four-year period, Isuku Iwacu facilitated 561,037people to gain access to basic sanitation facilities, increasing coverage from 55% to 67%, comparing the baseline to endline coverage. By November 2020, 219 communities had been monitored by a collaborative team of LCSOs and local authority and deemed to be predeclared as ODF. Measured against the key Activity indicators, Isuku Iwacu reached most critical targets. Where targets were not achieved, the final evaluation explores reasons and proposes solutions in the relevant sections. Key results are noted in Table 4. Table 4: Results per Indicators Measured in Baseline and Final Evaluation # Indicator Target Activity Achievement* Baseline Endline Survey Results6 1 Number of villages verified as open-defecation free as the result of USG assistance 137 219 n/a % of HHs having a latrine – improved and unimproved ODF declared– 95% ODF trained – 100% 2 Number of people gaining access to basic sanitation facilities 500,000 561,0377 55% of HH had a basic latrine 67% of HH have a basic sanitation facility % of HHs with basic sanitation ODF declared – 54% ODF trained – 79% 5 Republic of Rwanda, Ministry of Health, UNICEF. A Latrine that Meets our Household Needs. Instruction for Latrine Construction. July 2018. 6 Percentages based on the evaluation survey conducted in October 2020, thus represents a snapshot of the situation at the time of the evaluation and does not represent total cumulative figures achieved by the Isuku Iwacu Activity. 7 Previously (Up to June 2020), this indicator was calculated using the National average size of HH 4.3. Isuku Iwacu currently count members of HH in each of reported household. 15 # Indicator Target Activity Achievement* Baseline Endline Survey Results6 3 Number of households with clean latrines including covers 250,000 259,504 39% 40% of all households had a basic sanitation facility that was hygienic 4 Percent of HHs who access financial products to pay for basic sanitation 22% 0.072% n/a 430 *Achievements as reported October 31,2020 in Project Indicator Matrix FY20 Relevance The extent to which Isuku Iwacu Activity’s objectives and its interventions have been consistent and aligned to the GoR priorities and USAID strategic priorities. As outlined in the background section, the lack of access to improved, hygienic latrines is clearly a problem with vast health, nutrition and development consequences in Rwanda. Access to sanitation services and products, which households can use to improve latrines and hygiene infrastructure, is limited in Rwanda, providing few options for households, at out of reach prices. Weak supply chains, limited transport options, and the dearth of technical expertise multiply barriers for rural households to improve their sanitation. Isuku Iwacu’s design to use market-based approaches to create demand for basic sanitation through improved access to products and services and stronger governance is one that makes it possible to address these issues and advance progress towards sustainable development targets, while spurring local economic activity and communal action. As such, the Activity approach is assessed as relevant. This approach uses a design that requires a shift from orthodox direct implementation of services to one in which the Activity served to facilitate the creation of ownership and strengthening of systems for more effective sanitation service delivery. This shift requires time, resources, government investment, and significant adaptive management. The final evaluation finds that while there was a slow start to the Activity, it is because Isuku Iwacu Activity did a profound amount of learning, exploration, testing and subsequent modification of approaches during the last four years. Throughout the Activity life, Isuku Iwacu developed approaches and learnings that are relevant to strengthening the sanitation sector in Rwanda. According to both SNV USA and USAID leadership, the Activity was intended to be an adaptive learning process, testing approaches to both private sector engagement as well as fostering local ownership of sanitation improvements. It was acknowledged that the targets established for the Activity did not reflect this intention of piloting approaches to strengthen the market and as such, the Activity often appeared to be underperforming. Because the Activity contract was target driven, the time and space for testing and learning from different models was not available, and thus was not relevant for the intention of the Activity. “The way USAID set targets were not necessarily the right approach for this project because other USAID projects are more linear. This was not true for Isuku Iwacu. This was a learning process with need to intense advocacy in the early years.” USAID Alignment with USAID Strategic Priorities Isuku Iwacu’s strategic objective to improve access to and encourage correct consistent use of household sanitation and hygiene facilities in order to decrease childhood stunting, is relevant to USAID’s 2015-2020 Development Objective – Health and nutritional status of Rwandan’s improved. Isuku Iwacu fits within USAID Rwanda’s Community Health and Improved Nutrition (CHAIN) Project, along with 16 other activities in Rwanda aimed at increasing the use of health services and products, increase adoption of healthy behaviors, and “The activity was committed to explore both the focus on local ownership and private sector engagement. But different needs were noted on both sides that required some learning as we went forward. One huge success, is that USAID saw this as a pilot project and were open to change, test and adapt as we went forward. “ SNV USA 16 improve food security and nutrition.8 While there is significant global evidence linking improved sanitation and hygiene to improved nutritional outcomes, the relative impact of improved sanitation and hygiene in respect to other health, nutrition, and WASH programming is unclear. Considering the novel approach and opportunity to explore the resulting synergies and effects this collaboration presents, the final evaluation suggests conducting a retrospective evaluation of Ex- Post Impact Evaluation of the CHAIN Project to strengthen learning about best practices for integration of activities and impact that such synergies can have on improving health and nutrition outcomes. The Activity was designed to address governance, household financing of sanitation, supply, and behaviors, all of which global learnings have illustrated to be essential to sustain sanitation and hygiene improvements. • Demand creation to drive community and household desire for products and services and practice of key improved sanitation and hygiene behaviors; • Supply strengthening to reinforce demand generation and increase availability and access to sanitation and hygiene products and services • Financing to enable all wealth quintiles in Rwanda to afford basic household sanitation and hygiene • Strengthening of the enabling environment so that improvement approaches can be scaled up and sustained As mentioned above, the design required collaboration and coordination with other USAID CHAIN Activities, namely Gikuriro, which initially was responsible for creating demand for sanitation. This integrated approach is complex and requires a strong understanding of the environment in which it is being implemented as well as clear guidance on coordination between partners; again, systems that require time, resources, and adaptive management. The mid- term evaluation found that the complexities of the synergistic Activities made it ‘unclear how well this collaboration worked over time.’ In addition to creating a ‘ways of working’ document, Isuku Iwacu also decided to return to the original Activity design in late 2018 and engage Local Civil Society Organizations (LCSOs) to accelerate community engagement (Refer to Efficiency for further analysis) and enhance sanitation marketing approaches. These were deemed to be relevant actions in response to the mid￾term evaluation recommendations. Alignment with Government of Rwanda Priorities The Activity design is clearly aligned with GoR priorities. Isuku Iwacu was designed as the 2016 National Sanitation Policy (NSP) was being drafted, and as such mirrors the same goals and strategies. Because the NSP introduced sanitation marketing as a new strategy to strengthen availability and access to products and services, Isuku Iwacu was able to advocate for national buy-in of the Activity, especially with the Ministry of Infrastructure (MININFRA). The final evaluation finds that Isuku Iwacu appropriately prioritized strategies laid out in the NSP in its design, and collaborated with district authorities, MININFRA to implement them. Establishing MININFRA as the line Ministry for the Isuku Iwacu Activity in project design and set-up was appropriate. However, although the Activity successfully collaborated with the Ministry of Health (MOH) in several respects (participation in MOH Environmental Health Technical Working Group (TWG) and the Health Promotion TWG, as well as coordinating BCC messages through the Rwanda Health Communication Center (RHCC)/Rwanda Biomedical Center (RBC)), the Activity would have benefited from a more formal working relationship with MOH to leverage their authority to work with District Health Services to strengthen household sanitation and hygiene. The Final evaluation recommends that design of future programming incorporate a dotted line of Government collaboration to the MOH in addition to the MININFRA through a formal process. In the second year of the Activity, Isuku Iwacu introduced the Open Defecation Free (ODF) / Zero Open Defecation (ZOD) Training, as an approach to engage district and local authority in supporting communities to map, identify, address, and monitor progress towards addressing household sanitation needs. This approach is based on Community Led Total Sanitation (CLTS), a behavior change intervention aimed at triggering communal action to ensure that all households have access to sanitation and achieving Open Defecation Free status. Considering that Rwanda’s EICV5 findings indicate that the country is close to achieving ODF status, with only 3.8% without access to some sort of sanitation, the focus of the name of the training is misleading. Isuku Iwacu made efforts to adapt the approach to prioritize engagement of local authorities to lead community sanitation improvements and to target reaching higher levels of sanitation service for both households that do not have latrines, as well as those houses that require improving their level of sanitation service. 8 USAID Community Health and Improved Nutrition Project (2017) 17 The ODF Training methodology is relevant to the context of Rwanda as it builds upon the key cultural principle of communal support for development – Umuganda – and accountability -Imihigo - to improve sanitation. Though the approach is relevant and has driven uptake of household sanitation, the acceptance of the approach by GoR leadership has been limited because national monitoring shows near full access to basic sanitation by all. For future implementation of this intervention the Evaluation recommends focusing efforts on increasing district and local authority understanding and capacity to promote, implement and monitor for basic household sanitation. Increasing Demand Isuku Iwacu’s Behavior Change Communication (BCC) Strategy uses key strategies and channels as defined in the National Social Behavior Change Communications Strategy for Integrated Early Childhood Development, Nutrition and WASH, highlighting Community Health Clubs (CHCs) as centers for creating a ‘culture of health’ promoting improved sanitation and hygiene practices. Isuku Iwacu’s design initially targeted three key behaviors targeting household head engagement in market-based actions for improving and maintaining latrines, a relevant target for a market-based solutions Activity. The in-house BCC Strategy later prioritized behaviors that targeted hygiene, focusing on latrine cleaning and maintenance, improved hygiene habits, and safe latrine usage. The focus on hygiene, rather than on just focusing on having a latrine is relevant in order to sustain and optimize the impact of the use of basic latrines. Availability and Access to Sanitation Products and Services Isuku Iwacu’s aim to improve the supply and availability of sanitation products and services required significant research, trial, and error, in the landscape of a private sector that was not committed to investing their own resources into strengthening the sanitation market without the promise of ‘lucrative business opportunities.’ District Sanitation Centers. The creation of District Sanitation Centers (DSCs) to introduce technologies and services is clearly aligned with the NSP and the GoR’s approach to improving the sanitation market. The approach to establishing and managing the DSCs was not defined in the NSP and required significant time for Isuku Iwacu to establish, resulting in an operationalization plan half way through the Activity in June 2018, with the last DSC renovation completed during quarter 2 of 2020. The slow upstart to the DSCs means that there is still learning to be done about how the DSCs function, how they facilitate increasing community access to products and services, and their relevance and sustainability. The final evaluation acknowledges the undeveloped sanitation market in Rwanda, which creates significant barriers to reaching rural areas, and mainly impacts poorer households. The focus on district level sanitation markets was relevant for households with easy access to district centers and access to finances, whether through loans or savings. However, the way the current DSC model is functioning – primarily at the district level with limited reach to sector and cell levels – needs to be modified to meet the needs of rural communities, leaving gaps in access for rural, poor households, which reported that products and services were either unavailable to access or beyond their financial means. Based on global evidence and market experience in Rwanda, the market size, let alone demand, does not warrant dedicated sanitation providers at sector or cell level. However, there is a critical need to open up the supply chain between the DSCs and existing cell level traders. As such, during the last quarter of implementation, Isuku Iwacu successfully linked 92 entrepreneurs to DSCs to sell WASH products and services. Plans to further link DSCs with outlets at the sector / cell level, were interrupted in FY20Q2 because of COVID￾19; however, DSCs report some self-generated innovations to connect communities with DSC services, which are further explored in the Effectiveness section. Considering the resource and time limitation of this evaluation, and the time to access the connections made, the final evaluation did not seek to further Targeted BCC Behaviors • Heads of households provide resources to construct improved household latrines for their houses • Heads of households save money every month to purchase sanitation products and services. • Heads of households secure loans for sanitation products and services as needed. • Cleaning and maintenance for hygienic latrines, especially with a focus on limited water use • Handwashing with soap at critical times • Safe usage of a hygienic latrine 18 investigate these linkages. It is recommended that any new market-based sanitation programs start with a market analysis that includes the DSC links with local entrepreneurs. Financing through Loan Product. The Isuku Iwacu baseline, formative research, and the final evaluation all reinforce that finances is the top barrier preventing households from investing in their sanitation. Introducing financing products to Rwanda is highly relevant. Isuku Iwacu initially focused solely on building local stakeholder ownership while developing a truly private sector focused approach, requiring buy-in from formal banks. Lengthy consultations with formal financial institutions revealed that banks had unrealistic expectations of higher interest rates on loans compared to community savings groups, such as SACCOs. The amount of time it required to establish stakeholder buy-in and a cost-efficient model for SACCO engagement was underestimated, both of which were foundational to project success. While this learning caused implementation delays, it shifted Isuku Iwacu’s approach to one that best served the Rwandan context. SACCOs were finally able to roll out the loan product in 2020. FGDs with households who were both members and non-members of SACCOs revealed that most households do not feel that the sanitation loan product is accessible. The loan requirements – SACCO membership, provision of collateral, and full repayment – means that this approach is relevant for households with agency and assets, which are generally Ubudehe 3 and 4. Other savings and loan options (SILC, VSLA) seemed to present more relevant options for increasing rural household access to financing for sanitation. Members of SACCOs and those meeting the requirements for the loan, discuss the ease in which they accessed the product, highlighting limited accessibility. "One day I went to the SACCO to look at the money, I heard the news that they provide a loan to build a toilet. Because I am a member of SACCO, I immediately asked the local manager about that loan, and he explained to me what was needed to give it to me. It was not difficult for me to get the loan because they asked me for a house certificate as collateral and they asked me to bring someone to sign for me, and they immediately gave me money. I did not face any obstacles." Male, 42, Ruhango New Technologies. The use of technologies was decided by engaging the communities and creating designs that were fit to the community’s specific needs. To date the norm in Rwanda is to bury a full latrine and move to an empty spot to build a new latrine. As Rwanda’s population rapidly grows and available land becomes scarce, this solution loses its relevance. Fecal sludge management (FSM) is part of a long-term solution and increasing demand for higher quality services is critical. Isuku Iwacu’s Fecal Sludge Management Assessment, conducted in mid- 20189 indicated that only 10% of households in Rwanda were emptying their latrine pits / septic tanks. Across the seven districts assessed, availability of safe, mechanical pit emptiers was virtually non￾existent, with three of the seven districts having manual emptiers that reported varying degrees of capability to safely manage emptying, transport, and disposal. Safely and sustainably containing waste on-site is likely a more realistic option, as recommended by the mid-term evaluation, considering the nascent development of even urban off-site solutions. During the last year of implementation, Isuku Iwacu explored a couple of technological latrine solutions that prioritized the safe storage, treatment and subsequent safe use of the human waste as compost. Two latrines using Tiger Worm technology were installed in each district in late February 2020 for a total of 16 latrines. Considering the resource and time limitations of this evaluation, the limited time that these new technologies were in place, and the technical expertise required to assess, the final evaluation was limited to conducting phone interviews with 4 of the 16 households, represented in four districts (Kayonza, Nyanza, Ruhango and Rwamagana). Six months after the installation of the Tiger Worm latrines, 3 out of 4 housesinterviewed reported that their latrines were still functioning and that they were satisfied with their use of them; indicating that they find that the Tiger Worm latrines are less malodorous and have fewer flies than regular pit latrines. Three out of four of the houses sampled reported challenges with water and the Tiger Worm technology, mainly access to sufficient levels of water for flushing to adequately dilute the ammonia from the urine. The one latrine that was not functioning had been destroyed by heavy rains that compromised the water seal, thus 9 Accessed from https://www.dropbox.com/sh/pjvbhomwzzmopap/AAAtYkaX_mvzO9jWTasXh45ua?dl=0&preview=Rwanda+- +Isuku+Iwacu+presentation+by+Pius.pptx on 18 November 2020. 19 killing the worms. The household reported that they chose to revert to using a pit latrine, because Tiger Worm technology wouldn’t work in their environment. Three of the four households reported that they felt that they had limited access to technical services in case of problems with the technology. While no additional households in the community have been influenced to use their own resources to build a Tiger Worm latrine, the interviewed households report that other households have indicated their interest in the technology. If priced at an accessible level and marketed well with testimonials from households that are satisfied with their latrines, the Tiger Worm technology may be in high demand. However, considering the short time period that these households have been functioning and potential hazard that heavy rains, common in Rwanda, may have on the worms, the final evaluation recommends conducting a follow-up technical assessment at least two years after installation to look at use of the by-products, long-term effectiveness and sustainability. Effectiveness The extent to which project interventions were effective within the benefiting communities, which objectives were achieved, and to what extent have they contributed to the strategic objectives. The final evaluation finds that facilitating the development of the sanitation market, coupled with the implementation of the ODF training and targeted subsidies, and sanitation and hygiene behavior change promotion through CHCs, social media, and community events effectively provided a package that addresses the needs of all people in a community, including the most vulnerable – effectively increasing household ownership of an improved latrine from 55% at the baseline to 67% at endline. During the life of the activity 561,037 people gained household access to basic sanitation in the Isuku Iwacu target districts. Of all surveyed households, 41% privately own a basic sanitation facility and reported that they had constructed (33%) or renovated (8%) their latrine during the life of Isuku Iwacu’s Activity. As households that were given support through the Voucher System were targeted based on having no or unimproved sanitation, this approach had higher rates of latrines built (75%) or renovated (25%) since the beginning of the Activity. These rates are lower for other approaches, but contribute to raising the overall coverage of basic household sanitation coverage to 67%, 12% points higher than the coverage of basic latrines in the 2016 baseline. As implementation approaches overlap in communities and complement each other, this limited the ability to assess the effectiveness of discrete approaches. While this created complexities in evaluating individual approaches, the final evaluation finds this wholistic approach to be essential to achieving results and proposes that it will contribute to better sustainability of results. In the sections below, the final evaluation attempts to dissect the advantages and disadvantages, as well as, enablers and barrier to each approach. Intermediate Result 1: Increased demand for sanitation products and services Behavior Change Communication. The creation of demand for sanitation through BCC utilized a number of different partners and channels to reach communities and households with key behavior change messages throughout the life of the Activity. • Gikuriro initially took the lead on creating demand through CHCs and the delivery of the Community Based Environmental Health Promotion Program (CBEHPP) through the CHCs. This work was often complemented by the use of radio, drama, community events and mass media campaigns to raise awareness. • World Vision managed the BCC strategy development for Isuku Iwacu and facilitated the design of key BCC tools including community dramas, a series of radio soap opera episodes, and BCC messages through CHC, local leaders, and CHW delivery. Key messages delivered through these efforts focused on connecting consumers (households) with suppliers (sanitation products and services) that would help them upgrade and improve their sanitation and hygiene infrastructure and subsequent behaviors. • LCSOs were hired in late 2018, after external consultants were engaged to inform the design and support to the sanitation market, to accelerate the implementation of BCC and marketing efforts, assuming both support roles to CHCs as well as facilitator roles of marketing and monitoring. 20 While there were consistent efforts by all parties to clarify roles and coordinate efforts, including the development of the Isuku Iwacu Activity Collaboration Plan with Gikuriro, FGD participants were never clear around who was doing what in their communities. Monitoring reports were not able to clarify each program’s distinct contributions. For an Activity design that was intended to capitalize on each other’s work for greater, more effective, and efficient outcomes, this shouldn’t matter. However, for programs that want to demonstrate their effectiveness, this is a challenge. In late 2018, Isuku Iwacu decided to return to the original intent of the Activity design and engaged LCSOs to accelerate demand creation efforts and to more effectively link potential consumers with relevant sanitation markets. This not only simplified working arrangements, but the LCSOs contract with Isuku Iwacu was target driven, incentivizing LCSOs to effectively target households with appropriate support. LCSOs reported that considering the short timeline of their role, SNV USA committed significant resources and training to build the LCSO capacity; LCSO members articulated comprehensive understanding of relevant sanitation and hygiene monitoring indicators and criteria including key requirements for a household’s sanitation and hygiene to be categorized as basic, as defined by global Joint Monitoring Program standards, as well as standards required by GoR. The final evaluation notes that the division of responsibility for the delivery of behavior change messages through specific channels cannot be clearly attributed to any one intervention. In this light, the final evaluation focused on the different BCC approaches and the relative influence they had in creating demand for basic sanitation and hygiene. There was a broad spread in survey participant responses around why they built / renovated their current latrine. CHCs (18%), local leaders (21%), the Isuku Iwacu voucher (12%), and building a new home (19%) were the top reasons reported in the survey. Local Influencers. FGD participants reported local leaders or CHCs as key influencers in their decision to make sanitation improvements. The active engagement of local and district leaders appears to have provided a level of authority to the behavior change. As such, the final evaluation recognizes the creation of a social environment where there was an expectation that each household should have a basic sanitation facility and that those that did not build a latrine would be penalized, as a key enabler. Local leaders indicate their ability to continue promoting and monitoring progress in the uptake of basic sanitation facilities because of the trainings that Isuku Iwacu delivered. Prioritizing the role of institutionalized government roles and systems – local authority and CHCs – in promoting key sanitation and hygiene behaviors provides an effective and sustainable channel for continued promotion and monitoring of household sanitation and hygiene. The Gikuriro Activity facilitated CHCs and implementation of the CBEHPP also played a key role in increasing household awareness about the benefits of improved sanitation and hygiene. It is clear that community members are participating in and have verbalized being motivated to ‘take action’ by CHC and CBEHPP 18 21 3 6 12 2 2 19 CHC Local leader Marketing from store or DSC Isuku Iwacu Isuku Iwacu voucher Neighbor Accessed sufficient finances Built new home “It is our duty and responsibility to make sure things last by monitoring especially such good actions done by the project. The training given by Isuku Iwacu to the local leaders and people was key to sustaining the knowledge people have on the importance of sanitation and hygiene.” District Representative ‘In the beginning of the project we had a list of all people who do not have latrines and those who had unimproved ones. Every month we would go with Isuku Iwacu staff to check for the ones who made changes by carrying out household inspections and mark them on the list.’ LCSO Figure 4: Key Influencers for Building/ Improving Latrines 21 activities. While CHCs and CBEHPP are not sufficient to drive and technically support latrine and handwashing facility improvements, the final evaluation findings suggest that with strengthened CHC capacity and quality CBEHPP implementation, they can complement other activities that can enable (vouchers, loans, private sector) and drive sanitation and hygiene improvements (ODF training) by serving a role in nudging community members to sustain the practice of targeted behaviors. ODF / ZOD Training Approach. In Rwanda, positive culture and norms on the use of latrines point to effective government-led demand creation. Isuku Iwacu’s ODF training capitalizes upon this strategy, using local leaders to implement a series of activities to trigger community action to ensure that all households have a latrine. Final evaluation survey results show that 93% of households from ODF predeclared communities and 100% of households from ODF trained communities, were observed to have a functioning latrine. While not all met the standards of improved or basic latrines, this clearly illustrates the effectiveness of the ODF Training approach to trigger whole communities to engage in improvements and stimulate households to invest their own resources to ensure that they have their own latrine. At the time of the final evaluation 67% of households surveyed were observed to have a basic latrine (79% in ODF trained households and 54% in ODF pre-declared households). Under the Isuku Iwacu Activity, Achievement of ODF was declared following GOR guidelines, i.e.: upon whether 100% of households in a community have a basic sanitation facility; stability of the slab and facility was not assessed. There is an inherent trade-off between slab durability and longevity versus affordability of latrine slabs. Frequently, surveyed households reported that they did not have the capacity – access to products, access to funds to purchase needed materials, knowledge on how to build a durable, hygienic slab as reasons for the condition of their latrines at the time of the survey. FGD findings indicate that over time, and with inappropriate use against the program’s guidance (i.e.: using the latrine to as a private place to shower as per Activity monitoring feedback), the durability of wooden slabs diminished. Therefore, future implementation must continue communication campaigns stressing the importance of proper use of latrines with wooden slabs to ensure longevity and durability. FGDs provide further evidence that community members felt triggered into taking action to improve their household latrines as well as contributing community resources and communal work efforts through amasibo and umuganda to supporting less able households by the ODF trainings and the directives from the local leaders. These efforts, combined with promotion through the CHCs and other BCC activities, also effectively improved maintenance and hygiene of household latrines. Across Isuku Iwacu support, the hygiene of latrines improved in comparison to the baseline. The final evaluation survey results show 76% of household latrines with no visible signs of feces, compared with only 59% observed during the baseline. Latrines in communities that received the ODF training were notably more hygienic than those in communities that did not receive the training. The final evaluation finds that the ODF training, led and monitored by local leaders, is a low cost, effective approach for mobilizing households and communities into collective action to address their sanitation and hygiene. To elevate the approach to achieve higher levels of sanitation service ODF trained communities should * The 2016 baseline did not measure coverage of latrines that were covered and without signs of feces. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% Baseline All intervention types ODF Trained Voucher No ODF Training Clean and covered No feces or urine Figure 5: Observed Latrine Hygiene 22 be linked with sanitation markets at the onset of the training. As households cite the high cost of transporting products as one of the key barriers preventing their access of private sector products and services, linkages with markets should include mechanisms for facilitating and/or subsidizing the cost of transport. LCSO interviews indicate that they were actively linking households that had unimproved latrines to private sector latrine supply and service options. Creating this linkage at the same time the community is triggered to take action, could prove more efficient. Sanitation Marketing. Strengthening community demand for improved sanitation and access to private sector products and services requires greater awareness of availability and understanding of benefits of different technologies. While there is significant feedback from participants from all FGDs around hygiene and sanitation behavior promotion, the evaluation notes that there is little evidence of community awareness of private sector marketing of products and services. Throughout the life of the Activity, there were efforts to promote and link products and services with communities through outreach events such as mini exhibits, sanitation fairs, and market events. As a sanitation market facilitator, Isuku Iwacu encouraged the presence of services, including voucher companies and other private sector companies, at these events to connect communities to needed services. Unfortunately, organizing events requires logistics and resources which resulted in limited reach of the events, leaving a barrier for vulnerable, more remote communities. Review of the BCC Strategy influenced Isuku Iwacu to include a behavioral focus on personal sanitation and hygiene behaviors - promoting the achievement of ODF, handwashing with soap, and the benefits of moving up the sanitation ladder. While there are important behaviors to target for sustained use of improved sanitation and hygiene practices, they must be linked to the sanitation marketing messages that motivate and enable households to invest and access in needed sanitation products and services. FGD discussions indicate that many community members were motivated to improve their sanitation but were still only accessing the basic product options that were located in local outlets. The introduction of the LCSOs introduced an important bridge, connecting market knowledge and access to products and services with communities and households. This role, of a bridge between the consumers and the suppliers, is one that the CHCs could never assume because of their inherent positioning in the community. The final evaluation survey revealed that the construction of a new home influenced 19% of respondents to build or improve their sanitation. Review of the BCC messages and marketing approach indicates that this large consumer segment is not specifically targeted. This unexplored segment offers a marketing opportunity that should be explored in future sanitation marketing interventions. Intermediate Result 2: Improved supply and availability of sanitation products and services in the private sector Market-based solutions offer a promising approach to sustainable household and community sanitation and hygiene improvements. However, it takes time and considerable resources to understand and engage with the private sector. The GoR is committed to home-grown and market-based solutions, but sanitation marketing in Rwanda is new and the market in general in Rwanda has limitations across all sectors that need to be factored into the development of sanitation market-based solutions. Because most of Isuku Iwacu’s market-based approaches had a slow start to implementation, the final evaluation recommends a mini follow-up evaluation to assess the sustainability and any growth of the approaches in the absence of the Activity. District Sanitation Centers. The District Sanitation Centers are core to Isuku Iwacu’s strategy to increase availability and access to sanitation products and services. The 2018 District Sanitation Centers’ Approach and Illustrative Operationalization Plan defines the centers as having three key functions: 1) business outlet sales of quality sanitation and hygiene products and services 2) training for service providers, and 3) an exposition site to feature new technologies and services. DSC representatives report that the existence of the centers has been effective at stimulating interest in sanitation and engaging private sector. Isuku Iwacu initiated a series of trainings to increase their sanitation business capacity, specifically around sanitation technologies and accounting skills. The DSC managers require more training on business and marketing skills to be effective in widening their reach to the larger community. “Bringing the private sector in is a good way to work towards sustainability of access to basic sanitation. The DSCs, however, are not quite independent yet. They haven’t had long enough to become profitable stand-alone enterprises. But they are heading in that direction and this is promising.” USAID 23 However, all stakeholders – LCSOs, community members, district authorities, and DSC representatives - indicated that one district level market for sanitation and hygiene is insufficient to effectively expand services to reach remote rural communities. Future sanitation marketing work needs to explore additional effective ways of linking DSCs with local outlets to reduce barriers to community members. The evaluation recognizes that plans to expand DSC services through to community level outlets beyond the 92 entrepreneurs was disrupted by the COVID-19 pandemic prevention measures, but this remains a limiting factor to the effectiveness of the DSCs. There have been a couple of promising actions that provide options for future market development activities: 1. DSCs report that they were independently exploring the use of community members to market DSC products at the community level to enable remote access to market options. While this practice has not been standardized, the final evaluation recommends studying this approach and exploring other mechanisms for expanding DSC services into remote communities. 2. LCSOs have become a bridge between consumers and suppliers, linking households that they have identified as having unimproved sanitation with market options. To further enable this role, marketing materials (product and service brochures, improved latrine building / upgrading guide) and systems for transporting products between the consumers and suppliers could be developed. "We have set up people to represent us in each sector, and we often take our products to different markets in the district. This way we can reach more people without products. The number of people who come to the markets are very high, at least before COVID-19 they were. And they are able to access the sanitation products." DSC Representative Sanitation Loan Products. The delayed start to the introduction to the SACCO sanitation loan and the unpropitious emergence of COVID-19, means that the final evaluation has little evidence to assess the effectiveness of the loan. Between the introduction of the loan in February 2020 and the end of the Activity, 430 loans, serving 373 households were granted loans, reaching only 0.06% of households, as opposed to the targeted 22%. However, all FGD participants who obtained a loan reported satisfaction with and appreciation for the loan product and process for acquiring. In a short amount of time the LCSOs did a tremendous amount of outreach work linking BCC messages to improve sanitation with promotion about the loan product option. This was an effective link to trigger household demand to improve their latrine with supply of a product to finance the improvement. Because the loan product has only been available for nine months, eight of which were confounded by social distancing limitations caused by COVID-19, continued marketing of the loan product by service providers will be essential to ensure that the population is aware and interested in them. According to Isuku Iwacu management, the programs partner SACCOs are continuing to market the loan product and have fully integrated it into their product line since the close of the program. If this marketing does not organically evolve through the onus of the service providers, the final evaluation recommends continued USAID programmatic support to ensure that the loan product remains available. "We heard the information about the importance of sanitation and hygiene and about the loans provided by SACCO from our village leaders. I was happy to hear that I could get a loan from SACCO and build a latrine and bathroom at the same time. That was good news. When I requested a loan from SACCO it did not even take three days to receive. I didn’t face any challenges in the process because I am a member and own my home." Male, 63, Rwamagana 24 The final evaluation found no evidence that the sanitation loan products were reaching the U-1 or U- 2 segments. In fact, U-1 and U-2 households indicated that they felt the loans were out of their reach because of the strict criteria: membership to the SACCO and some type of collateral. Instead, U-1 and U-2 indicated that community savings and loans groups such as VSLA, SILC and AMASIBO were much more feasible to access revolving fund loans for household sanitation improvements. Only 27% of survey participants responded that they had used a loan to finance their current latrine; 49% of these loans were from community savings group. The mid-term evaluation’s recommendation to formally explore options for promoting community savings and loans as a financing option for U-1 and U-2 households does not seem to be formally pursued in the last year and half of the program, as efforts were focused on getting the SACCO loan product off the ground. Despite this, the final evaluation finds anecdotal evidence that households are already accessing finances through revolving loan systems and recommends further exploration of this option to provide more vulnerable households with financing options. Intermediate Result 3: Improved governance for sustained access to sanitation and hygiene products and services The Isuku Iwacu Activity by design was primarily implemented at the district level, focused on increasing district and local leader capacity to deliver and monitor sanitation marketing and demand creation approaches. While there was some collaboration at the national level, mainly around the NSP, and the Low-Cost Technologies Guide, Isuku Iwacu dedicated its focus to working at the district level to establish systems and implement activities. As such the Activity’s accountability was at the district level, as was the weight of its influence. To effectively advocate for scalable change, the final evaluation believes that national leadership needs to be brought actively on board, strengthening national systems to remove district-level barriers, as well as facilitate adaptation of policies, guidelines, rules and regulations that evolve out of district level innovations. Building Capacity for Governance Isuku Iwacu worked with a myriad of different actors at the national, district, and local levels to strengthen sanitation markets and create demand. These stakeholders include national working groups, district and local government leaders, LCSOs, construction companies, DSCs, local outlets, and financial institutions. Each stakeholder has its own unique role in facilitating market strengthening, requiring unique capacity building for each. Isuku Iwacu assumed an essential role in bringing these partners together at the district level and facilitating the creation of shared vision and responsibility for building and implementing a market that will enable households to improve their own sanitation. 49% 29% 16% 3% 3% Savings group MFI/SACCO Bank Cooperative Friend/relative Figure 6: Type of Loan Used to Finance Latrine 25 Isuku Iwacu played an active role at the national level. Through project document review, it is noted that the Activity participated in several national level advocacy efforts and provided support to the National Government through varied activities. Isuku Iwacu utilized the Joint Action Development Forum (JADF), a multi-stakeholder platform aimed at facilitating community participation in the decentralized and participatory governance processes that are aimed at promoting development. As a key forum to bring these different stakeholders together to engage in the different stages of the sanitation market, JADF enabled a district level forum for collaborating around the different stages of planning and implementation of market-based solutions to be promoted in communities. Facilitating the introduction of SACCO loans through this channel is one reason that there was quick uptake in the loans. To ensure that all stakeholders understand the role of the sanitation market in development and have the capacity and motivation to play their role, continued district and local stakeholders capacity building is essential. LCSOs and district and local leaders acknowledged that district and local authorities’ ability to prioritize sanitation and hygiene promotion and monitoring was sometimes limited because of competing government priorities and competing systems. An example of this is with the national Human Security monitoring systems, which aligns with the National Strategy for Transition 1 (NST 1) – prioritizing universal access to basic sanitation by 2024 and universal access to safely managed sanitation by 203010. In Rwanda there are a number of ministries that are monitoring sanitation, namely the MOH, MININFRA, and Ministry of Local Government (MINALOC). As a result, district and local leadership who were motivated to monitor ODF communities in coordination with the Isuku Iwacu Activity reported that they would sometimes simply use the existing Human Security systems because of their existing accountability to the Ministry of Local Government. District Wide Approach (DWA) assessments/Management Information System (MIS) for WASH Isuku Iwacu supported the Government of Rwanda through MININFRA in the operationalization of WASH MIS in seven Districts: Nyarugenge (City of Kigali), Nyanza, Ruhango (Southern Province), Nyabihu (Western Province), Rwamagana, Kayonza and Ngoma (Eastern Province). The purpose of MININFRA’s DWA was to harmonize indicators across all 30 districts of Rwanda, collect data to be uploaded in the MIS, and support districts in establishing their WASH investment plans. Data was collected from different settings including: (i) All households in all cells and sectors; (ii) Schools; (iii) Health facilities; (iv) Public Institutions; (v) Public Places and (vi) Water supply systems. As part of its support, the program held consultative meetings with MININFRA at DWA inception and organized district-level workshops, gathering local government officials, representatives from MININFRA and WASAC, and district-based implementing partners. Workshops raised awareness of DWA among the WASH district stakeholders, and clarified the roles and responsibilities of districts and their development partners to support the DWA execution. The program also participated in a series of meetings organized by the WATSAN working group for the development of MININFRA’s MIS to review and validate the MIS inception and interim reports, discuss sector indicators to be monitored by the MIS ensuring alignment with Sustainable Development Goals (SDGs)s, and reviewed the MIS final report which was validated. After its completion, Isuku Iwacu presented its findings, implications, and gave recommendations on how to proceed. 10 Rwanda’s goal is to join SDG, i.e.: by 2030 universal access to safely managed sanitation The Role of the Activity at National level Isuku Iwacu was an active member in several national level advocacy efforts:  WATSAN sector group  MOH Environmental Health Technical Working Group  Health Promotion at the Rwanda Health Communication Center (RHCC)/Rwanda Biomedical Center (RBC) Isuku Iwacu also supported the National Government through:  The translation and dissemination of the Sanitation and Water Supply Policies and Strategies  Support in the District Wide Approach  Completion of the Low-Cost WASH technologies study in collaboration with MININFRA  Collaboration with National Early Childhood Development Program (NECDP) in construction of 645 household-level ECD child-friendly latrines.  Participation in working groups and workshops  Technical review and validation of reports 26 Efficiency For the scope of this evaluation, efficiency will be defined as whether project activities were perceived to be cost-efficient, were achieved on time, and were implemented in the most efficient way. Fee-for-Service Contracting. The engagement of LCSOs at the end of 2018 not only accelerated implementation, but led to better quality implementation of ODF trainings, awareness building, capacity building of government, and appropriate follow-up to targeted households. Isuku Iwacu’s decision to contract the LCSOs on a fee-for-service basis, paying for services based on achievement of households gaining access to improved latrines, was able to incentivize LCSOs to deliver measurable outcome results in a very limited amount of time. Deciding to engage LCSOs on a fee-for-service basis for not only accelerated achievement of results, but also drove transparency – making results, as well as barriers, visible in a way that enabled LCSOs to learn and adapt their way of working to be more effective. LCSOs were selected and engaged based on their local knowledge and close connections to the community. By paying for results, Isuku Iwacu allowed the LCSOs to focus on problem solving and use their local knowledge to modify programs as needed to get the desired results rather than focusing on only satisfying USAID reporting requirements. Engagement of the LCSOs through a fee-for-service contract was an efficient move that enabled the LCSOs to achievement of household sanitation and hygiene targets. AMIR was similarly contracted on a fee-for-service basis to trial the sanitation loan product. All relevant parties bemoaned the fact that the Activity failed to reach their targets with the loans because of the delayed start, owing mainly to the need to better understand viable mechanisms for providing finance options for sanitation improvements. However, the uptake of loans within the short period of time before COVID-19 closed down Rwanda and then the continued issuing of loans during lockdown, were noted to be impressively efficient. In addition to having effective mechanisms for matching qualified households with sanitation improvement needs with the loan, AMIR and the SACCOs were financially motivated to ‘sell’ the loan products. Consultant Support and Delayed LCSO Engagement. While the Government of Rwanda is committed to driving development through market-based solutions, Rwanda has a nascent sanitation market. Formative research, which indicated a dearth of private sector engagement capacity in Rwanda, influenced Isuku Iwacu’s decision to shift from the original plan to engage LCSOs at the initiation of the Activity and instead engage external consultants to build capacity and systems for strengthening the market. Once a foundation for the sanitation market was established, LCSOs were brought in to link households to markets and accelerate community engagement. CHAIN Partnership. USAID Rwanda’s Community Health and Improved Nutrition (CHAIN) Project was intended to facilitate the integration of USAID investments for better health and nutrition outcomes. It is an ambitious approach with complexities in designing a program that addresses the myriad of multi-sectoral factors that influence health and nutrition outcomes. CHAIN collaboration was key to the successes of several activities as reported by the Activity: “If we had opted to work with LCSOs from the beginning, the budget would have been used differently. But we agreed to initially bring in consultants to support formative research and the development of the private sector strategies. Once LCSOs were engaged, they were also able to increase the results and number of people gaining basic sanitation which we were not able to achieve at the beginning.” USAID 27 • The provision of 645 child-friendly ECD latrines was a result of collaboration with the National Early Childhood Development Program (NECDP) • CHAIN facilitated learning between its members in private sector promotion of sanitation and hygiene in communities through CHAIN WASH taskforce meetings • Harmonization of social behavior change (SBC) messages through the SBC task force • Organization of Ebola preparedness training by Isuku Iwacu team presented to CHAIN members and given by MOH • Knowledge sharing and education on DSCs, contributing to awareness raising of DSCs at the community level • Linking BCC interventions through joint organizing of or contribution to awareness raising events such as community football games Under CHAIN, Isuku Iwacu was designed to complement Gikuriro, matching demand creation with market-based solutions; which on paper appears to have clear areas of responsibility. However, although demonstrating collaborative efforts (as detailed in the text box), the two activities struggled to draw clear lines around implementation despite regular consultations to define work, which caused confusion both at the management level, as well as in the community, and thus are deemed to not be an efficient approach to collaboration. Impact Measurement of changes over time in demand for sanitation products and services; improvement in supply and availability of sanitation products and services in the private sectors; and improvement in governance for sustained access to sanitation and hygiene products and services. Isuku Iwacu’s strategic objective was to encourage correct and consistent use of household sanitation and hygiene facilities in order to decrease childhood stunting. While the impact of Isuku Iwacu’s Activity on decreasing childhood stunting are not possible to assess, there are measurable changes in household ownership and use of sanitation and hygiene facilities. Improved Household Infrastructure. Because Isuku Iwacu invested a significant amount of time and energy to design and testing of approaches that were based on the understanding of both consumer preferences and motivations and the supply capacity of the market in Rwanda, achievements of results had a slow start. However, by end of the Activity, 183 communities had been declared ODF and over 500,000 people had gained access to improved sanitation as a result of exposure to the program. The Survey shows a 12-percentage point increase in household ownership of an improved latrine, from 55% in August 2017 to 67% in October 2020, this is a notable achievement for an Activity that was described as a ‘series of pilots to innovate and test approaches for building market-based solutions.’ Hygiene. Based on Activity reporting, Isuku Iwacu exceeded its target for # of HHs with clean, covered latrines, and in almost all cases there is an increase in household adoption of hygienic latrine behaviors, including maintenance of improved latrines and having a handwashing facility with both soap and water. Table 5: Adoption of Hygienic Latrine Behaviors Hygiene Criteria Baseline11 Endline12 Hygiene of latrine Clean and covered 39% 41% No feces or urine 59% 76% No cover 20% 35% 11 Total number of households surveyed at baseline: 2,227 12 Total number of households surveyed at endline: 494 Isuku Iwacu and Gikuriro Collaborative Efforts  Coordinated work planning  Review of collective monitoring data  Information sharing on technical implementation  Dissemination of sanitation policy  Contribution to CBEHPP where ODF process has been added to the document and endorsed by MOH  Review of latrine prototypes used in voucher system  Provision of iron sheets via WATSAN campaign  Complementary training programs of masons 28 Hygiene Criteria Baseline11 Endline12 Handwashing Presence of handwashing station 34% 58% Availability of water and soap 79% 68% Handwashing station with soap and water 27% 39% Latrine Cleaning Availability of latrine cleaning agents 33% 76% Given the importance of handwashing for COVID-19 prevention, diarrheal disease prevention, and improving nutrition outcomes, hygiene – specifically handwashing with soap at critical times – is a critical behavioral component of the Activity’s Strategic Objective. Hygiene was not the key focus of Isuku Iwacu, but is intrinsically connected to sanitation (demand creation) and water (increased quantity required) and featured in BCC messaging, as well as DSC product supply and marketing. An 11-percentage point increase in household handwashing facilities between the Activity’s baseline and this endline is noted, however cannot attribute whether this is a result of Isuku Iwacu’s Activity or GoR’s promotion of handwashing with soap to prevent the spread of COVID-19 transmission. However, there is a significant difference noted in Figure 9 between ownership of tippy taps by households that benefited from specific Isuku Iwacu interventions (52% voucher and 46% ODF Training recipients owned tippy taps) than with those households that did not participate in ODF training (9% recipients with no ODF training owned tippy taps). Considering Isuku Iwacu’s emphasis in promoting tippy taps, this difference in coverage, suggests attribution to Isuku Iwacu’s efforts. Figure 7: Types of Handwashing Facility by Intervention Gender and Equity. Isuku Iwacu’s design and implementation strategies articulate the importance of inclusive decision-making around household sanitation and hygiene improvements. Key approaches that the evaluation recognized and believe contribute to gender and equity in sanitation and hygiene improvements include: • Deliberate engagement of both women and men in community training and outreach activities, as well as business engagement and trainings; • Requirement of the agreement from both adult partners in a household before a SACCO agrees to issue a loan or loan product. FGDs illustrated strong understanding of benefits and requirements of sanitation and hygiene improvements from both women and men, as well as articulated engagements of men and women in communal processes for improving household latrines. 9% 46% 52% 36% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% No ODF Training ODF Declared Voucher Total Tippy Tap Bucket Tap Wash Basin Other "Gender was considered in every single activity… as it is our government rules that gender should be considered in all sectors. Isuku Iwacu was the same either in training in the employers and in DSC representatives. Even myself, I consider gender in employing people who help me in this DSC, employing both men and women.” DSC Representative 29 The final evaluation was not specifically designed to target households of people living with disabilities and the final evaluation has little evidence at the household level to assess whether or not sanitation marketing efforts sought to make facilities inclusive or accessible. In 2019, Temaco, a business that Isuku Iwacu was collaborating with to facilitate the sanitation market, devised, constructed, and distributed 550 latrine seats for people that had mobility limitations. Unfortunately, these recipients were not captured through surveys or FGDs, so there is no assessment of their effect. DSCs reported beginning to diversify the types of products and services being sold at the center to sustain their businesses, including sale of menstrual hygiene products. Access and availability of sanitation products and services. While access to, and the cost of materials was reported by FGD participants to be a key constraint to building an improved latrine, there is evidence that households are purchasing materials to improve their sanitation. Except for community members who live near district centers, there was little evidence that rural households were directly accessing the DCSs. DSC representatives reported having between 60 to 300 clients a month who primarily purchased soap, handwashing equipment and Satopans. While the representatives reported seeing a decrease in overall sales during the past eight months, they indicated that there was an increase in sales in hygiene supplies, specifically handwashing facilities, in response to COVID-19 prevention measures. Diversification of their product line and linkages with Kigali wholesalers has allowed some DSC managers to respond to consumer demand as sale needs change. Sustainability and Scalability The extent to which program outcomes are likely to be sustained based on changes in local ownership, capacity or perception of the intervention. Both USAID and SNV USA deemed the Isuku Iwacu Activity to be a series of pilots to test the effectiveness, efficiency, and appropriateness of different market-based solutions to increase household ownership and use of improved sanitation and hygiene. For these approaches to contribute to GoR’s target of universal coverage of basic sanitation and hygiene by 2024, it is critical that systems for implementation are sustainable, and that resulting infrastructure is durable and valued to optimize sustainability. It is also critical to consider the scalability of these approaches, whether through institutionalized uptake of activities or through intentional, donor-funded replication to address the unmet need for all of Rwanda. Institutionalized systems for improving sanitation. There has never been a successful at-scale rural sanitation program that did not have high-level political leadership. Isuku Iwacu’s deliberate use of key GoR strategies and priorities, namely the District Sanitation Centers, the National Social Behavior Change Communications Strategy for Integrated Early Childhood Development, Nutrition and WASH, and local market-based solutions in their Activity design optimizes GoR buy-in and ownership of approaches. Where approaches were introduced to Rwanda like the ODF training, there is less formal engagement from leadership, which took greater effort to foster. Isuku Iwacu also prioritized the engagement and capacity building of district and local leadership to promote and monitor household sanitation and hygiene coverage, mainly through the ODF process. The institutionalization of sanitation and hygiene promotion through existing GoR systems and human resources is a sustainable approach to demand creation. Building capacity of CHC groups and local leaders to promote, mobilize, and monitor household sanitation and hygiene will ensure that even with the completion of Isuku Iwacu, the knowledge and motivation remains within the community. Local leaders communicate their intention to continue promoting and monitoring sanitation. “Local government and authorities are capable of sustaining the Isuku Iwacu Activities achievements at the end of the activity at 100%. We will keep visiting and monitoring the community to check that the latrines that Isuku Iwacu developed are still functioning and not destroyed. We will keep training community on sanitation and hygiene, and we will ask them to help those who haven't changed to make the change too. There are no gaps or barriers that remain.” District Executive Secretary “People did not have money due to COVID-19 lockdown. So, we had to learn a new strategy of bringing the most needed COVID-19 prevention materials, like hand washing stations sanitizers and all kinds of soaps, to the market. This at least helped us to reduce the risk of loss in our business. The strategy here was to bring to the market the most needed sanitation materials and equipment." DSC Representative 30 The Ministry of Health (MoH) sees universal improved household latrine coverage as one of its core objectives and is committed to supporting existing CHCs and local leaders towards this goal. The MoH has of yet been able to scale the CHC model throughout the country, but has depended upon partner support in targeted project areas. Instead, the MoH has focused its investment towards ensuring that the MoH cadre of CHWs are trained and equipped to promote and monitor sanitation and hygiene. Ensuring that the CHW service delivery package includes appropriate sanitation and hygiene BCC messages, as well as guidance to link local leaders and communities with sanitation markets, will drive the institutionalization and sustainability of the CHWs role in supporting communities and local leaders in advancing their level of sanitation and hygiene service. Strengthening and aligning CHW responsibility and activity on WASH promotion with CBEHPP will foster sustainability and integration within an accountable government system. Because the MoH is rolling out the CHW service delivery packages throughout Rwanda, this approach is immediately scalable. Sustainability of a private sector focused voucher system. In a country where 16% and 30% of HHs are categorized as belonging to Ubudehe 1 and 213 and most rural households surveyed indicated that they did not have the collateral to qualify for a loan or were concerned that they could not repay the loan, having social protection mechanisms in place to assist vulnerable households to improve their sanitation is critical. In the sanitation sector, while subsidies are often considered to create dependency and result in unsustainable sanitation, when implemented in tandem with other parallel efforts, they can be effective. The Isuku Iwacu voucher system contributed to increased awareness around the importance of sanitation and hygiene in rural communities and delivered a targeted subsidy to a reported 11,371 vulnerable households. Many latrine voucher recipients indicated that the support they received, integrated with behavior change (BCC) messaging, prompted and enabled the household to take additional actions to improve the sanitation and hygiene in their homes (i.e.: improved household water handling, established handwashing facilities, created hygienic spaces for food preparation and utensil storage, and purchased supplies to maintain their newly constructed latrines). The voucher system combined with BCC messaging even stimulated some households to access loans from their community savings and loans group or invest their own finances to make further household hygiene improvements. For some FGD participants, the voucher system also stimulated interest from households that did not receive a voucher latrine and provided models for households to copy, which was an Isuku Iwacu design intent. Thus, ongoing sanitation and hygiene promotion will be essential to ensure that the latrine is maintained and valued by the household, thus becoming a sustainable aspect of what is considered the household. “I used to have latrine which was far from my home and to use it during the night was a challenge. Isuku Iwacu built an improved latrine for me, which is near to my home now. After this, I was inspired to make other changes as a result of that support, like building a hand washing station with soap in it. We also made kitchen garden, and drink clean water!” Female, 58, Rwamagana. In addition to raising awareness in rural communities on the importance of sanitation and hygiene, another component of the voucher system was to garner further local leader ownership around sanitation in their communities by having them engaged in the process. When applied appropriately, the final evaluation found that the use of the Ubudehe system effectively strengthened transparency of the subsidy and acceptance by the community. Village leaders were responsible for identifying and selecting voucher recipients, based on the criteria of belonging to U-1 or U-2 and not owning an improved latrine. Survey results show initially, some households were miscategorized and classified as the wrong Ubudehe category. As a result, evaluation survey results show that only 25% and 47% of voucher recipients were from the U-1 and U-2 category, respectively, with 28% belonging U-3 – resulting in the provision of a subsidy for houses financially able to improve their household sanitation. After mis categorizations were corrected, Isuku Iwacu also made corrective actions whereby all remaining selected U-3 households that had not yet been given vouchers were replaced with U-1 and U-2 households. Another design intention of Isuku Iwacu voucher system was to feature services provided by local companies and link them to potential customers – thereby contributing to increased demand of sanitation supplies and services. The voucher companies were instructed by Isuku Iwacu to offer three design options to their customers/Voucher recipients; however, this choice was usually not provided with most FGD respondents 13 Ubudehe is a national government-led poverty identification system 31 indicating that they were instead told what they would get and what contribution they had to make in order to receive the service. When a voucher process allows households to make choices in products and services, markets are driven to diversify their options. The final evaluation finds that the Isuku Iwacu Voucher construction companies did not effectively offer choices that would facilitate the growth or diversification of the sanitation private sector. Secondly, as part of the community contribution (Amasibo) to voucher latrine construction, some households provided wooden slabs. FGD findings indicate that over time, and with inappropriate use against the program’s guidance (i.e.: using the latrine to as a private place to shower), the structural integrity was diminished and some latrines built through the Voucher System were not functioning properly as a result. While Isuku Iwacu established a process for reporting concerns, gaps in awareness were noted by community members participating in FGDs. Therefore, future implementation of the Voucher System must focus on a communications plan which utilizes various channels including mass and print media, local influencers, and community testimonials, to ensure that voucher recipients are aware of: (1) the consumer feedback system for reporting issues with latrine construction quality and (2) proper use and maintenance of latrines with wooden slabs. Given the pilot nature of this activity, there was no evidence available at the time of completing this evaluation to confirm if it generated additional business for companies. With improvements in customer relations and communications, it is possible for similar voucher systems to contribute building the sanitation sector. Early in the design, Isuku Iwacu realized that the latrine prototypes being offered through the Voucher System had a significantly greater value (RWF 85,000) than the design had originally intended (RWF 20,000). Isuku Iwacu pursued an approach to engage the private sector, i.e.: construction companies, which inevitably resulted in the higher cost, but targeted the building of latrines that had a high level of service. There are other examples of non-private sector-focused targeted subsidies in the country that provide lower valued options that also achieve improvements in household sanitation, at a quarter of the price (RWF15,000 to 35,000). Voucher system approaches implemented in Rwanda have used varying degrees of community contributions (like Amasibo14) and differing approaches to implementation (e.g.: some subsidized materials, others voucher to be redeemed through local outlet, etc.). In all cases there is a trade-off between building the private sector and providing low-cost options. To best advise government on how to provide support to vulnerable households while stimulating the sanitation market in a sustainable way, the final evaluation recommends a comprehensive nation-wide review of all approaches applied since 2016 for subsidizing household sanitation. With a comprehensive evaluation, the government can use evidence to evolve its market-based approach to improving sanitation into one that is aligned with the current stage of the WASH private sector in Rwanda and one that can be financed and delivered by the government to all districts, sectors and villages in the most sustainable and forward-thinking manner possible. Market-Based Solutions. There is strong national and district ownership of the District Sanitation Centers. All stakeholders indicate the importance of supply chain linkages with wholesalers from the national level and from DSCs to other sellers. Isuku Iwacu concerted significant effort to strengthening the supply chain, linking wholesalers with DSCs and engaging companies with masons to support latrine building in communities As addressed in the Effectiveness section, providing financing options to access desired product and services to improve sanitation is essential for Rwanda to reach universal access to basic sanitation and hygiene services. Loan products offer an exciting financing option for households that may not have immediate cash for purchases, but have agency and assets. However, for some the loan conditions appear to have made this an unappealing or even inaccessible option and thus Isuku Iwacu’s targeted subsidies – the Voucher System - provides an effective option to responsibly support sanitation investment for vulnerable households. Voucher systems need to be carefully administered in relation to microfinancing options to prevent market distortion that would dissuade able houses from choosing to self-finance. This can be seen in discussions with local leaders and community members who focus on the need for further subsidy support to improve sanitation rather than on self-financing mechanisms. Because the Voucher System 14 Groups of houses within a community, umuganda that complete collective-community labor efforts Isuku Iwacu has linked our DSC to industries that manufacture sanitation products. We take products directly from industries at lower prices, and we sell them at lower prices compared to anywhere else sanitation products can be found. I also give my products to other sellers on debt and they pay me later when they do not have money to pay me immediately.” DSC Representative 32 was introduced first and provided a value of assistance that is beyond the means of most rural households in Rwanda, this potentially created an expectation of assistance that demotivated households from self-financing. The two approaches can and should be implemented together, but timing is key in order to foster a sanitation market that is sustainable. Future efforts should focus on absorbing demand with microfinancing options before introducing the Voucher System to support households that were unable to self-finance. Governance and Enabling Environment. Capacity building and investment for national to district government, private sector and local NGOs is critical. Increasing the number of NGOs working in WASH may be important for continued sustainability and support. Private sector traders exist but may be unaware of the potential of sanitation markets. Isuku Iwacu concerted significant effort to strengthening the supply chain, linking wholesalers with DSCs and engaging companies with masons to support latrine building in communities. The link that will threaten the sustainability of the DSCs is the weak connection with communities and households. Further investing in approaches to connect the DSCs with existing local level businesses will continue to aid in broadening their scope of work into providing sanitation services. Sustainable Outcomes Improved household latrines. Investing in an improved latrine is a key indicator of sustainability, and 32% of households in ODF predeclared communities that were surveyed had invested their own finances in improving latrines, with another 50% indicating that they plan to make further upgrades. The majority of those households that had invested their own finances report to be in Ubudehe category 3 (53%), followed by Ubudehe category 2 (40%) and Ubudehe category 1 (7%). Only 10% of households were observed to have no cleaning supplies near their latrine, and over 60% reported cleaning their latrines on a daily basis. When households value something enough to invest in it, intend to make further improvements, and have systems to maintain – these are all positive signs for sustainable use of the infrastructure. The targeted Voucher System is an important tool to support vulnerable households to improve their sanitation to a level that does not require continuous rebuilding because the level of quality that the household can reach, limits the durability of the latrine. As such, the voucher system can provide finances that will allow vulnerable households to access products and services to make the latrine more durable. Latrines built through the Isuku Iwacu Voucher System resulted in 98% having a fibrous cement or galvanized steel roof and 92% having a door; these are critical for maintaining integrity of the structure during the heavy rains that Rwanda regularly faces. Unfortunately, at the time of this evaluation, 34% of Voucher latrines surveyed were observed to have a slab that was not solid. As part of the community contribution (Amasibo) to voucher latrine construction, some households provided wooden slabs. Over time, and with inappropriate use against the program’s guidance (i.e.: using the latrine to as a private place to shower), the structural integrity was diminished. Future implementation of the Voucher System must ensure durability of the slab through construction quality control mechanisms as well as reiterate and continually reinforce consistent messaging on the proper use of the latrines with wooden slabs in order to promote sustainability of latrines. Open Defecation Free Communities. Achievement of ODF was declared following GOR-issued guidelines, i.e.: upon 100% of households in a community having an improved latrine – without a focus on the durability of the slab or proper use and maintenance of the latrine. The evaluation estimates that latrines, which had been verified as improved, but did not have durable slabs or were improperly used (e.g. used as bathing areas), likely reverted to unimproved status between the time of verification and the evaluation. Future implementation will need to continually reiterate the GOR issued guidelines on use, maintenance, and replacement of wooden slabs. There is an inherent trade-off between slab durability and longevity versus affordability of latrine slabs. Therefore, future implementation must continue communication campaigns stressing the importance of proper use of latrines with wooden slabs to ensure longevity and durability. 33 CONCLUSIONS & RECOMMENDATIONS Basic household sanitation is critical to preventing human exposure to pathogens that cause diarrheal disease, chronic gut infections, and worms, all which compromise nutrition. However, the level of exposure at the household is not just dependent on the level of hygiene of the household toilet, but is affected by the level of contamination of the community overall. Therefore area-wide sanitation is critical for public health. Achieving area-wide basic sanitation coverage depends on both the public and private sectors to collaborate – private sector, government, and communities. Isuku Iwacu started the process of moving away from direct implementation of services to applying systems strengthening approaches that target key functions of the enabling environment to enable markets to support sanitation service delivery. Overall, Isuku Iwacu piloted many approaches and adapted over the course of the 4-year project to achieve its target results. A tremendous amount of learning was achieved, which is captured in this evaluation and support recommendations below Advocate for complementary approaches to appropriately target market-based solutions – targeted sustainable subsidy system , self-financing options (loans), and sanitation market Market-based solutions offer a promising approach to sustainable household and community sanitation and hygiene improvements. However, it takes time and considerable resources to understand and engage with the private sector. The GoR is committed to home-grown and market-based solutions, but sanitation marketing in Rwanda is new and the market in general in Rwanda has limitations across all sectors that need to be factored into the development of sanitation market-based solutions. Because most of Isuku Iwacu’s market-based approaches had a slow start to implementation, the final evaluation recommends a mini follow-up evaluation to assess the sustainability and any growth of the approaches in the absence of the Activity. Also, considerations should be made for further examining the sequencing of targeted subsidies with market-based options to avoid creating market distortion that may disincentivize households from investing their own resources in advancing their household level of sanitation. Voucher System The voucher system design was relevant per the need of vulnerable populations in Rwanda. However, ongoing sanitation and hygiene promotion will be essential to ensure that the latrine is maintained and valued by the household, thus becoming a sustainable aspect of what is considered the household. The average cost of RWF 85,000 for the latrine voucher construction limits the opportunity for bringing the approach to scale in Rwanda. UNICEF Rwanda’s voucher support intervention, which provided vulnerable households with vouchers, valued at RWF 20,000, that could be redeemed at local outlets for sanitation and hygiene products, has resulted in increasing coverage of basic sanitation. Because sustainability of this approach has not been evaluated, it is challenging to compare these two similar market-based approaches. As such the evaluation recommends a comprehensive nation-wide review of all approaches applied since 2016 for subsidizing household sanitation. With a comprehensive evaluation, evidence can be offered to evolve an approach into one that can be financed and delivered by the government to all districts, sectors, and villages. The Activity decision to deliver the construction of three standard designs of latrine through the voucher system, and the companies’ apparent lack of consultation with households around the desired design, fails to harness the power that vouchers can have to strengthen markets and stimulate further demand for products and services. When a voucher process allows households to make choices in products and services, markets are driven to diversify their options. The voice of the consumer needs to be heard and choices offered to facilitate the growth or diversification of the sanitation private sector. Loans The loan product has only been available for nine months, eight of which were confounded by national COVID￾19 prevention measures limiting movement and limited access to services. Continued marketing of the loan product by service providers will be essential to ensure that the population is aware of, and interested in them. If this marketing does not organically evolve through the onus of the service providers, it is recommended that USAID programmatic support is continued to ensure that the loan product remains available. All FGD participants who obtained a loan reported satisfaction with and appreciation for the loan product and process for acquiring. In a short amount of time the LCSOs did a tremendous amount of outreach work linking BCC messages to improve sanitation with the promotion of the loan product option. This was an effective link to trigger household demand to upgrade their latrine with supply of a product to finance the improvement. 34 New Technologies During the last year of implementation, Isuku Iwacu explored a couple of technological latrine solutions that prioritized the safe storage, treatment, and subsequent safe use of the human waste as compost. Two latrines using Tiger Worm Technology were installed in each district in late February 2020, totaling 16 Tiger Worm Latrines. However, considering the short time period that these households have been functioning and potential hazard that heavy rains, common in Rwanda, may have on the worms, the final evaluation recommends conducting a follow-up technical assessment at least two years after installation to look at use of the by-products, long-term effectiveness and sustainability. Ensure access to sanitation for vulnerable houses The proposed support to vulnerable households, aimed at supporting vulnerable households with unimproved latrines to obtain basic sanitation, has an established and effective system to identify who the subsidy is available for. Future subsidy support must also have an established process for applying the system and providing the subsidy. A communication strategy and robust monitoring system for such activities would help to inform local leaders, community, and providers around the process and rationale for the process, as well as hold them accountable. The mid-term evaluation’s recommendation to formally explore options for promoting community savings and loans as a financing option for U-1 and U-2 households does not seem to have been formally pursued in the last year and half of the program, as efforts were focused on getting the SACCO loan product off the ground. Despite this, the final evaluation finds anecdotal evidence that households are already accessing finances through revolving loan systems and recommends further exploration of this option to provide more vulnerable households with financing options. Prioritizing efforts to decentralize DSC functions further into sector and cell level markets and closer to households will improve access to vulnerable households. Considering that one of the key main functions of the DSCs is to train service providers, they should be equipped to facilitate existing cell-level traders to carry more sanitation products, closer to the home. This approach relies on government/partner demand creation – through BCC efforts or the ODF training, although also mobilizes businesses to promote as well. More formative research with the private sector at all levels is needed. Support supply chain strengthening – Ensure that DSCs are linked to national and community level markets Based on global evidence and market experience in Rwanda, the market size, let alone demand, does not warrant dedicated sanitation providers at sector or cell level. However, there is a critical need to open up the supply chain between the DSCs and existing cell level traders. Future sanitation marketing work needs to further explore effective ways of linking DSCs with local outlets to reduce barriers to community members. Plans to continue linking DSCs with outlets at the sector/cell level, beyond the 92 entrepreneurs, were interrupted due to COVID-19, however DSCs report some self-generated innovations to connect communities with DSC services. There have been a couple of promising actions that provide options for future market development activities. 1. DSCs report that they were independently exploring the use of community members to market DSC products at the community level to enable remote access to market options. While this practice has not been standardized, the final evaluation recommends studying this approach and exploring other mechanisms for expanding DSC services into remote communities. 2. LCSOs have become a bridge between consumers and suppliers, linking households that they have identified as having unimproved sanitation with market options. To further enable this role, marketing materials (product and service brochures, improved latrine building / upgrading guide) and systems for transporting products between the consumers and suppliers could be developed. Refine the ODF process. In order to meet the financial means of Rwandans and facilitate Rwanda in meeting its goals of universal access to basic sanitation by 2024, future implementation will need to continually reiterate the GOR issued guidelines on use, maintenance, and replacement of wooden slabs. Recurrent campaigns on the importance of proper use, maintenance, and replacement of latrines with wooden slabs is critical. Linking targeted communities to sanitation markets and financing options will enable households to target higher levels of sanitation service when they are triggered by the training process to improve sanitation. The ODF training, led and monitored by local leaders, is a low cost, effective approach for mobilizing households and communities into collective action to address their sanitation and hygiene. To elevate the approach to achieve higher levels of sanitation services, ODF trained communities should be linked with sanitation markets 35 at the onset of the training. As households cite the high cost of transporting products as one of the key barriers preventing their access of private sector products and services, linkages with markets should include mechanisms for facilitating and/or subsidizing the cost of transport. Quality Control and Community-Level Presence Any future sanitation marketing interventions should prioritize the establishment of control systems to guarantee the delivery of sanitation and hygiene products and services that are sustainable and enabling households and communities to achieve higher levels of sanitation and hygiene services. In the nascent stages of the sanitation market in Rwanda and introduction of new sanitation technologies, it is essential that any market facilitators have a strong presence in the community in order to empower clients to demand the level of quality that is expected and required. The evaluation recommends that market facilitators should support the private sector to establish feedback mechanisms and a communications plan that would utilize various channels including mass and print media, local influencers, and community testimonials. The communication plan should ensure that voucher recipients are aware of: (1) the formal consumer feedback system for reporting issues with latrine construction quality and (2) proper use and maintenance of latrines with wooden slabs. Refine National Sanitation Monitoring Systems LCSOs, district and local leaders acknowledged that district and local authorities’ ability to prioritize sanitation and hygiene promotion and monitoring was sometimes limited because of competing government priorities and competing systems. An example of this is with the national Human Security monitoring systems and the National Strategy for Transition 1 (NST1). The NST 1 prioritizes basic sanitation by 2024 and safely managed by 2030. In Rwanda there are a number of ministries that are monitoring sanitation, namely the Ministry of Health, Ministry of Infrastructure, and Ministry of Local Government. As a result, district and local leadership who were motivated to monitor ODF communities in coordination with the Isuku Iwacu Activity reported that they sometimes simply used the existing Human Security systems because of their existing accountability to Ministry of Local Government. Triggering communities through the ODF training to upgrade their sanitation was not enough to successfully progress households to universal basic sanitation coverage. Local leader engagement in the process effectively provided a level of authority to the approach, and their regular monitoring efforts held households accountable to ensuring their house had a toilet. Ultimately standards for latrines and monitoring systems need to target higher levels of service in order to achieve sustainable results. This will require continued inter-ministerial and stakeholder coordination and consultation to agree on refined indicators and standards that will meet national standards and long-term goals. Consideration should also be made for further engaging with the existing GoR structures to continue monitoring and sustaining household sanitation and hygiene behaviors. Strengthening and aligning CHW responsibility and activity on WASH promotion with CBEHPP will foster sustainability and integration within an accountable government system. Focus advocacy efforts on National Level Engagement and Capacity Development To effectively advocate for scalable change, national leadership needs to be brought actively on board, to strengthen the national systems and remove district-level barriers, as well as facilitate adaptation of policies, guidelines, rules and regulations that evolve out of district level innovations. This will ensure that innovations tested at the district level will be more effectively embraced and create critical linkages for strengthening national systems to remove district-level barriers. For example, Isuku Iwacu’s project was designed as a pilot effort. The ODF approach, through efforts of both the Isuku Iwacu Activity and the Gikuriro Program, was new to Rwanda and implemented along with village and sector leaders, SEDO in charge of the Human Security monitoring and reporting. The ODF approach was incorporated into the CBEHPP and to be adopted and scaled up by the Districts and implementers. Future projects should take the knowledge gained in this process and base implementation on the lessons learned. Working directly with both district and national government will facilitate adaptation of approaches policies, guidelines, and regulations that evolve out of district level innovations. To ensure that all stakeholders understand the role of the sanitation market in development and have the capacity and motivation to play their role, continued district and local stakeholder capacity building is essential. 36 Future market-based solutions activities should invest in further capacity building in district leadership (from Mayor to community leaders) to build a strong understanding of sanitation markets, including financing options that can decrease community dependency on direct assistance. Capacity development efforts must also strengthen understanding of improved sanitation and promote harmonized systems monitoring, in order to build ownership within these critical and influential decision-makers.