i EVALUATION OF THE USAID/UGANDA SANITATION FOR HEALTH ACTIVITY (USHA) February 4, 2022 This publication was produced at the request of United States Agency for International Development (USAID). It was prepared independently by the Uganda Learning Activity (ULA). ii EVALUATION OF THE USAID/UGANDA SANITATION FOR HEALTH ACTIVITY (USHA)) Submission date: February 4, 2022 Submitted by: Chris Degnan, Chief of Party QED Group, LLC Embassy Plaza 1st Floor, Plot 1188-1190 Ggaba Road Tel: +256 778 983 568 Email: CDegnan@qedgroupllc.com Submitted to: Martin Muwaga COR Uganda Learning Activity Contract/Agreement Number: 72061719C00003 Cover page picture: School toilet constructed by USHA showing handwashing steps. Taken from the USHA Annual Report 2019. DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. i Table of Contents TABLE OF CONTENTS......................................................................................................................................I ACRONYMS ......................................................................................................................................................III EXECUTIVE SUMMARY................................................................................................................................IV 1. INTRODUCTION........................................................................................................................................... 9 1.1 Performance of the WASH Sector in Uganda ...............................................................9 1.2 Activity Background and Objectives...........................................................................10 1.3 Evaluation Purpose ......................................................................................................10 1.4 Evaluation Scope .........................................................................................................11 1.5 Key Evaluation Questions............................................................................................12 2. EVALUATION APPROACH AND METHODOLOGY.......................................................................... 12 2.1 Sampling ......................................................................................................................12 2.2 Data Collection Methods.............................................................................................13 2.3 Data Analysis...............................................................................................................14 2.4 Methodological Limitations.........................................................................................14 3. DISCUSSION OF FINDINGS .................................................................................................................... 14 3.1 Status of Open Defecation (OD) in USHA Districts...................................................14 3.2 Sanitation and Hygiene Implementation Approaches Identified .................................16 3.2.1 Approaches Identified in Other Countries.................................................................16 3.2.2 Approaches Identified in Uganda ..............................................................................17 3.2.3 Approaches Implemented by USHA .........................................................................19 3.3 Approaches for WASH in Schools..............................................................................21 3.3.1 Approaches for WASH Friendly Schools in Uganda ................................................21 3.3.2 WASH Friendly Schools Approaches by USHA.......................................................22 3.4 Innovative Approaches for Water Supply....................................................................23 3.4.1 Innovative Water Supply Approaches in Other Countries ........................................24 3.4.2 Innovative Water Service Delivery models in Uganda..............................................24 3.4.3 USHA’s Approaches to Water Supply to Rural Communities..................................27 3.5 Opportunities for WASH Systems Strengthening .......................................................28 3.5.1 Governance and Leadership.......................................................................................28 3.5.2 Information Systems..................................................................................................29 3.5.3 Civil Society Participation .........................................................................................30 3.5.4 Supply Chains............................................................................................................30 3.5.5 Financing and Accountability....................................................................................32 3.5.6 Infrastructure Operations...........................................................................................32 3.5.7 Management and Regulation .....................................................................................32 3.6 Sanitation Financing ....................................................................................................33 3.6.1 USHA’s Sanitation Financing Interventions .............................................................33 KIIs revealed USHA carried out the financing activities shown in Table 6.......................33 3.6.2 Barriers to accessing sanitation financing..................................................................34 The evaluation’s findings from the literature review and KIIs indicate there are number of barriers to sanitation financing and that the barriers are different for big businesses, MSMEs, and households. The findings are summarized in Table 7 below........................34 3.7 Gender Integration Within WASH in Uganda.............................................................34 3.7.1 Gender Gaps in WASH..............................................................................................35 3.7.2 Approaches that Address Gender-related Gaps.........................................................35 3.8 Enabling Factors for USHA’s Sanitation Approaches.................................................35 3.8 Challenges for Implementing Sanitation and Hygiene, and Rural Water Supply Approaches..................................................................................................................37 3.8.1 Challenges for Sanitation and Hygiene Approaches .................................................37 ii 3.8.2 Challenges for Rural Water Supply ...........................................................................38 3.9 Cost of Implementing Sanitation and Hygiene, and Rural Water Supply Approaches..................................................................................................................38 3.9.1 Cost of Implementing Sanitation and Hygiene Approaches......................................38 3.9.2 Cost of Implementing Approaches for Rural Water Supply......................................40 3.10 Opportunities for USAID’s Future Programming .....................................................40 3.10.1 Opportunities in Sanitation Programming ...............................................................40 3.10. 2 Opportunities in Rural Water Supply Programming ..............................................41 4.0 EVALUATION TAKE AWAYS............................................................................................................... 42 4.1 Take Aways from the Evaluation of Sanitation and WASH Friendly Schools Approaches..................................................................................................................42 4.2 Take Aways from USHA that Inform Future Gender Integration...............................43 5.0 CONCLUSIONS .......................................................................................................................................... 43 6.0 RECOMMENDATIONS FOR USAID FUTURE PROGRAMMING.................................................... 44 7.0 ANNEXES..................................................................................................................................................... 46 Annex 1: Detailed Evaluation Questions...........................................................................46 Annex 2: District Performance in the WASH Sector ........................................................47 Annex 3: Rationale for Selecting Districts, Countries, and Organizations........................48 Annex 4: Districts Selected................................................................................................50 Annex 5: List of Key Informants.......................................................................................51 Annex 6: Focus Group Discussions Conducted.................................................................52 Annex 7: USHA Evaluation SOW.....................................................................................53 Annex 8: References..........................................................................................................58 List of Tables Table 1: Performance of Selected WASH Indicators in Rural Communities............................9 Table 2: Implementation Approaches in Selected Countries...................................................16 Table 3: Water Supply Service Delivery Models....................................................................24 Table 4: Phases in the MWE O&M Roadmap 2030................................................................27 Table 5: Constraints to and Actors in the Sanitation Supply Chain Based on the Literature Review and Interviews ............................................................................................31 Table 6: Sanitation Financing by USHA .................................................................................33 Table 7: Barriers to Accessing Finance ...................................................................................34 Table 8: Unit Costs for Sanitation Service Levels...................................................................39 Table 9: Costs of WASH in Schools .......................................................................................39 Table 10: Water Service Unit Costs.........................................................................................40 List of Figures Figure 1: Map of Uganda Showing USHA Districts Covered by the Evaluation ...................11 Figure 2: Percentage of OD Households in USHA Districts (2018-2019)..............................15 Figure 3: USHA Parameters for a WASH Friendly School ....................................................22 Figure 4: Water Service Chain in Uganda ...............................................................................30 iii Acronyms ASP Area Service Provider CBMS Community-Based Management System CHAST Child Hygiene and Sanitation Training CLTS Community-Led Total Sanitation CSO Civil Society Organization DEFAST Decentralized Fecal Sludge Treatment DHI District Health Inspector DWSCC District Water & Sanitation Coordination Committee FGD Focus Group Discussion FIs/MFIs Financial Institutions/Microfinance Institutions GIZ German Agency for International Cooperation HEWASA Health through Water and Sanitation HIC Household Improvement Campaigns HPM Hand Pump Mechanic IRC International Water and Sanitation Centre KI Key Informant KII Key Informant Interview MBSIA Market-Based Sanitation Improvement Approach FIs/MFIs Financial Institutions/Micro Finance Institutions MHM Menstrual Hygiene Management MOES Ministry of Education and Sports MOH Ministry of Health MWE Ministry of Water and Environment NDP III Third National Development Plan 2020/21-2024/25 NGO Nongovernmental Organization OD Open Defecation ODF Open Defecation Free O&M Operation and Maintenance PHASE Participatory Hygiene and Sanitation Education PHAST Participatory Hygiene and Sanitation Transformation PMA Professional Management Arrangement SBCC Social and Behavior Change Communication SDG Sustainable Development Goal SNV SNV Netherlands Development Organisation SOW Statement of Work UN United Nations UNICEF United Nations Children’s Fund USAID United States Agency for International Development USHA Uganda Sanitation for Health Activity UWASNET Uganda Water & Sanitation Network WASH Water, Sanitation and Hygiene WHO World Health Organization WUC Water User Committee iv EXECUTIVE SUMMARY This report presents the findings of the Uganda Learning Activity’s evaluation of the USAID/Uganda Sanitation for Health Activity (USHA). The evaluation was conducted between October and November 2021 and was commissioned by the USAID/Uganda Mission. The evaluation aimed to document how the approaches implemented by USHA and other Water, Sanitation and Hygiene (WASH) programs have contributed to stopping open defecation and increasing basic sanitation services. The evaluation also aimed to identify what has worked and what has not worked in USHA and other programs and make actionable and relevant recommendations that will inform future USAID-funded WASH programs. Approach and Methodology The methodology was qualitative in nature. Quantitative data was used to corroborate qualitative findings. Data collection methods included desk research, key informant interviews, and focus group discussions. The evaluation also included an analysis of USHA progress data. The evaluation covered USHA’s implementation to date, that is, from January 2018 to September 2021. Data collection took place in Northern Uganda, Central East Uganda, and Central West Uganda, covering three districts per region. Approaches implemented by other organizations and countries were identified through desk research and national level key informant interviews Evaluation Findings The evaluation identified three sanitation approaches implemented in Uganda, in other countries, and by USHA. These approaches are Community-Led Total Sanitation, Market￾Based Sanitation Improvement, and WASH Friendly Schools. Findings from the field and desk research indicate that Community-Led Total Sanitation (CLTS) is preferred by the Government of Uganda because it does not use subsidies as an incentive, and it focuses on behavior change to lead to real and sustainable improvements. The Ministry of Health (MOH) and UNICEF’s Roadmap for Elimination of Open Defecation and Acceleration of Basic Sanitation in Uganda (MOH ODF Roadmap, 2019) describes CLTS as being highly successful in rural settings. However, contrary to the sustainable improvements claimed, Ficek and Novotny (2018) note that CLTS achieves Open Defecation Free (ODF) status by means of low quality and non-durable sanitation facilities that suggest improvements are not sustainable. Findings from national and district key informants indicated that in order to sustain ODF status, there has been a general shift in Uganda from traditional CLTS to a blend of CLTS and other approaches. It was noted that most regions in Uganda have transitioned from traditional CLTS to including other approaches that enhance CLTS’ efficacy. This is corroborated by the MOH Roadmap, 2019, which states that CLTS is deemed the most effective approach, but there are other approaches such as Household Improvement Campaigns (HIC), Participatory Hygiene and Sanitation Transformation (PHAST), and Child Hygiene and Sanitation Training (CHAST), that can complement CLTS. Although CLTS is the preferred approach in Uganda for elimination of open defecation, findings from the literature and field interviews indicated that Community-Led Total Sanitation Plus (CLTS+) is the most widely used approach. CLTS+ is a modification of the CLTS approach and aims to improve the quality of sanitation facilities and reduce relapse of communities to Open Defecation (OD) after attaining ODF status. Findings from interviews v indicated that combining approaches like PHAST, enforcement, HIC, and elements of sanitation marketing, is necessary to overcome the challenges of CLTS. CLTS+ is considered an improvement from CLTS because it impacts several factors, such as the mindset of people, including leaders, and income levels that affect the outcomes of sanitation promotion. The findings from the selected districts showed USHA implemented a CLTS+ sanitation promotion approach in the Northern region. USHA leverages the strength of triggering realizations in communities and integrates elements of sanitation marketing, in an effort to improve the quality of latrines so that they meet the basic sanitation status. As a result, there have been great improvements in the status of sanitation and hygiene in the selected districts, especially in the construction of basic sanitation facilities. Contrary to traditional CLTS, the quality of household sanitation facilities is prioritized in the CLTS+ approach and this led to USHA overcoming the challenge of quality and sustainability associated with the traditional CLTS approach. The Government of Uganda’s (GOU’s) Integrated Sanitation and Hygiene Financing Strategy (2017) identifies sanitation marketing as another approach. Key informants at the national level cited the Financial Inclusion Improves Sanitation and Health (FINISH) Diamond approach, as promoted by the Caritas Fort Portal Diocese Health through Water and Sanitation (HEWASA) project, as an example of a sanitation marketing approach used in Uganda. The MOH in Uganda also recognizes sanitation marketing as an approach to scale up the supply and demand for improved sanitation and hygiene facilities1 . This evaluation established that in the Central East and Central West, USHA focused on the Market-Based Sanitation Improvement Approach (MBSIA) approach that aims to develop sanitation markets and bridge the gap between demand and supply. The USHA MBSIA approach had a strong focus on demand creation through community triggering, awareness creation, market promotion, and use of sales catalysts and sanitation promoters. The noted benefits of the MBSIA approach over traditional CLTS and CLTS+ include allowing households to improve their toilets by themselves and product sales at the household level allowing people to know and understand product choices and how best they can buy sanitation products. Households are at liberty to select the mason of their choice and the building negotiations are concluded between the two parties without involvement of the project. WASH Friendly Schools are found in and outside Uganda. The approach involves a number of interventions that can be divided into “hardware” and “software” types. Hardware approaches include construction of school WASH infrastructure with facilities such as incinerators for Menstrual Hygiene Management, and construction of group hand washing facilities. Promising software approaches seen in Uganda include school toilet competitions and the Three-Star Approach. School toilet competitions instill a sense of responsibility in stakeholders, so they help ensure WASH improvements and their sustainability. The Three-Star Approach is premised on the assumption that once children learn the preferred good behaviors, they become hygiene change agents in their homes and communities. The evaluation identified five approaches for water supply implemented internationally, in Uganda, and by USHA. These approaches are the Community-based Management Model (CBMS), Direct Local Government Provision, Public Utility Provision, Supported Self-supply, and Private Sector Provision. In Uganda, the Ministry of Water and Environment is 1 Government of Uganda, Ministry of Health. National Sanitation and Hygiene Guidelines, 2017. vi transitioning to CBMS+ (CBMS with professionalized management arrangements) through a stepwise process. Findings from the literature and field interviews guided the evaluation to identify opportunities to address WASH gender gaps. These opportunities include social mobilization that includes conducting community and household outreaches through CLTS and sanitation marketing. During social mobilization, the emphasis should be on equal engagement of men and women. Awareness creation, such as through radio talk shows, community dialogues, radio spots, and campaigns, can also encourage people to appreciate the roles of men and women in WASH interventions. Opportunities for Future USAID Sanitation and Rural Water Supply Programming i. Findings indicated that USHA worked mainly with communities and the government, with only minimal involvement with financial institutions like banks and microfinance institutions (MFIs). Facilitating and coordinating communities, businesses, financiers, and the government to work together as the four major elements of sanitation marketing will make the approach work better because of the interdependence of these elements. ii. Extensive awareness campaigns and outreaches were identified by this evaluation as an important aspect in the marketing and promotion of sanitation products to rural households. From interactions with districts, it was apparent that the districts will not sustain the sensitization or follow-up activities due to the limited WASH budget. This means there is an opportunity to design market driven sanitation marketing promotion campaigns instead of USHA subsidizing the districts. iii. There were concerns from USHA implementing grantees that some local leaders do not encourage communities to embrace sanitation and hygiene but rather want to use enforcement to get people to participate in sanitation and hygiene activities. This means there is an opportunity to better engage local leaders, through discussion forums, so they are able to better influence and encourage their communities to take decisions that promote accelerated attainment of ODF. iv. Evaluation findings indicated that the sanitation service chain is not fully developed yet, especially in rural and peri-urban areas and in the treatment and reuse of fecal sludge. Therefore, opportunities exist in strengthening the sanitation service chain through increasing the number of waste disposal, containment, and treatment sites. v. All districts raised the challenge of limited funding for WASH activities. There is an opportunity to engage MOH to persuade it to allocate a sufficient WASH budget to district WASH departments. The current WASH budget from MOH is given to the District Environmental Health Departments as part of the Primary Health Care grant of which 30 percent is used for prevention activities such as immunization, sanitation, and hygiene. vi. The WASH budget is controlled and managed by health facilities and is not easily accessed by District Health Inspectors. There is an opportunity to train districts on how to develop operational sustainability plans for interventions implemented by external projects. vii. Sanitation marketing is an emerging intervention area requiring support and leadership from the government for it to thrive. There is an opportunity to support the government as it finalizes and implements guidelines and strategies for district local governments to adopt. vii viii. Findings indicated that line ministries have parallel reporting lines and use different information systems for data capture, analysis, and reporting. It was established that UNICEF plans to support three line ministries (Ministry of Water and Environment [MWE], Ministry of Health [MOH], and Ministry of Education and Sports [MOES]), to have an integrated information system aligned to the UN Sustainable Development Goals (SDGs). This plan is not implemented yet. In the meantime, there is an opportunity to strengthen information systems (for example through monitoring and evaluation processes and training district personnel in data collection, analysis, storage, reporting, and use, especially for sanitation activities), from the national to district level will enhance the quality of the data reported by the ministries. The absence of documented statistics about OD in the districts covered and the country at large also points to a gap in identifying sanitation needs. ix. The evaluation did not obtain any information regarding interventions in health facilities from any the districts covered. It was not established why no respondents acknowledged the presence of USHA interventions in health facilities. Rotary International does work in selected health facilities, especially where water is not to a point of care. USHA reached only four health care facilities in the North because of the FY 2021 budget shortfall and programmatic realignment. There is an opportunity for USAID to prioritize WASH programs that focus beyond households to include institutional settings, i.e., more schools, health care facilities, and markets. x. USHA focused mainly on ODF and basic sanitation, but the opportunity to attain SDG targets also lies in households attaining a safely managed sanitation service level. This looks beyond ODF to the containment, treatment, and safe disposal of human waste. xi. The national Micro Planning Handbook for WASH in Public Primary and Secondary Schools in Uganda (2019) noted that water in primary schools stood at 58 percent. This implies that there is a significant gap in water supply in schools that needs to be addressed. Conclusions i. Based on the findings from interviews, the literature, and progress reports from USHA, the evaluation concludes that the identified sanitation approaches (CLTS, CLTS+, market-based sanitation approaches, and MBSIA) are all context specific and their success in attainment of ODF status therefore depends on the context where each approach works well. However, the MBSIA and other market-based approaches have an edge over other approaches since they go beyond attainment of ODF to the sustainability of ODF and moving towards basic sanitation. MBSIA and market-based approaches reduce the possibility of a community relapsing to OD, because they involve developing the sanitation and hygiene supply chain and leveraging the private sector. ii. Information obtained from district respondents and focus group discussions in the Northern region indicate that since USHA started implementing in the region, there has been noticeable improvement in the status of sanitation. The evaluation was informed that this was because USHA blended CLTS with other approaches to design CLTS+, which was a good innovation. CLTS+ is a promising approach to achieve ODF status and will gradually lead communities to moving to basic sanitation. iii. Based on interviews with national level KIs and a review of ministerial documentation, this evaluation concludes that rural water supply in Uganda is undergoing reform. However, it was viii concluded that CBMS+ is a promising approach towards professionalizing the water supply service. CBMS+ will involve strengthening districts, involving the private sector, and providing a conducive environment for the rural water authorities to improve Recommendations for USAID future programming i. Based on the results of the USHA market-based approach to attaining ODF status and eventual moving to basic sanitation, this evaluation recommends USAID scales up and replicates the approach in sub-counties and districts not originally covered by USHA. ii. USAID should commission a comprehensive cost analysis study to get an in-depth understanding of the cost of specific approaches for sanitation and water supply. None of the USHA grantees or districts contacted by the evaluation team had readily available documented costs of WASH activities. iii. Although information from USHA indicated that thinly spread interventions built more capacity at district local governments than concentrated interventions, this evaluation recommends that for future programming, WASH interventions are concentrated in one district as opposed to covering many districts and selecting only a few sub-counties. The current approach used by USHA gave inconsistent results across a district with some areas being ODF while still having OD in other areas. Phased, concentrated interventions would leave a more consistent impact across a district than thinly spread interventions. iv. WASH programs should be comprehensive enough to focus beyond households and selected institutions like schools and health care facilities, to also include other public places such as markets. v. Information systems should be strengthened through monitoring and evaluation processes and building the capacity of districts in data collection, analysis, storage, use, and reporting, especially for sanitation activities vi. Marketing campaigns and promotion should be continued to nurture positive attitudes, values, and mindsets towards sanitation and hygiene. This is a fundamental element towards achieving the SDG targets. It also aligns with the mission of the MOH ODF Roadmap, 2019, that is to inculcate a mindset that cherishes environmental health through appropriate disposal of human excreta throughout the country. vii. Future USAID programs need to focus on holistic interventions with greater attention on activities that build the practical skills of teachers that lead to improvements in hygiene practices, such as making liquid soap, making bags for Menstrual Hygiene Management (MHM) supplies, and making sanitary pads. 9 1. INTRODUCTION This report presents the findings of the performance evaluation of the USAID/Uganda Sanitation for Health Activity (USHA). The evaluation was commissioned by USAID/Uganda and implemented by Uganda Learning Activity in October and November 2021. The evaluation aimed to document how the approaches implemented by USHA and other Water, Sanitation and Hygiene (WASH) programs have contributed to stopping Open Defecation (OD) and improving basic sanitation services. The evaluation also aimed to identify approaches that have worked for USHA and other programs, and make actionable recommendations to inform future USAID-funded WASH programs. 1.1 Performance of the WASH Sector in Uganda Uganda subscribes to the UN Sustainable Development Goals (SDGs), including SDG 6 that highlights the desire to achieve elimination of OD and have universal access to safe water and sanitation by 2030. The water, sanitation, and hygiene sector in Uganda is guided by the goals and strategies outlined in the Government of Uganda’s Vision 2040 document and other government targets. For example, reaching the target of 100 percent of the population having access to safe piped water by 2040 is a priority. The government’s Third National Development Plan 2020/21-2024/25 (NDP III) has a target of 45 percent of households having an improved toilet by 2025. The NDP III also envisions 85 percent of rural and 100 percent of urban households having access to safe water supply by 2025. The Ministry of Health (MOH) and UNICEF’s Roadmap for Elimination of Open Defecation and Acceleration of Basic Sanitation in Uganda (MOH ODF Roadmap, 2019) has the major goal of attaining Open Defecation Free (ODF) status and basic sanitation throughout Uganda by 2025. Uganda faces an enormous challenge in ensuring universal access to safely managed sanitation and hygiene services in public places and institutions at the community level, despite the institutional, legal, and policy frameworks needed for improved hygiene and sanitation promotion already being in place.2 Table 1 below presents the performance of the water and sanitation sector on key indicators in FY 2019/2020. Table 1: Performance of Selected WASH Indicators in Rural Communities Indicator 2019/20 Percentage of population practicing open defecation 22% Percentage of population using an improved sanitation facility not shared with other households 18% Percentage of villages with a source of safe water supply 68% Improved latrine coverage 19% Source: Ministry of Water and Environment. Water and Environment Sector Performance Report 2020 and NDP III 2 Government of Uganda, Ministry of Water and Environment. Water and Environment Sector Performance Report 2020, September 2020 10 1.2 Activity Background and Objectives USHA is a five-year activity implemented by Tetra Tech ARD, Inc. in collaboration with SNV Netherlands Development Organisation (SNV), BRAC, and Sanitation Solutions Group. USHA also collaborates with Rotary Uganda through a memorandum of understanding between Rotary International (RI) and USAID/Uganda. Rotary Uganda is part of USAID’s Global Development Alliance in Uganda. USHA’s overarching objective is to contribute to reducing the incidence of diarrhea and other diseases related to poor sanitation. USHA works to stop OD and to increase sustainable access to basic and safely managed sanitation, as defined by the WHO/UNICEF Joint Monitoring Programme for water supply, sanitation and hygiene. USHA aims to increase household access to sanitation and water services; expand key hygiene behaviors at home, school, and health facilities; and to strengthen district water and sanitation governance for sustainable services in 20 districts (seven in Northern Uganda, seven in Central East, and six in Central West). Under USHA, WASH interventions are carried out at the community level (working with households) and the institutional level (working with schools and health facilities). USHA Design and Development Hypothesis USHA was designed within the framework of USAID/Uganda's Country Development Cooperation Strategy 2016-2021, and contributes to DO2, Demographic Drivers Affected, and DO3, Strengthened Systems. The Activity was designed to address chronic failures and gaps in the water and sanitation sector that have led to poor sustainability of services. USHA’s development hypothesis, as presented in the Activity’s Monitoring, Evaluation and Learning Plan, summarizes the overarching design and interconnectedness of interventions: If individuals, communities, and institutions have increased access to and ability to finance demand-responsive water and sanitation products and services supplied by the private-sector; AND IF an evidence-based behavior change strategy is used to expand the demand for improved sanitation and the adoption of key hygiene behaviors in households and public institutions; AND IF a well-coordinated enabling environment at sub-county, district, and national levels is strengthened to facilitate and monitor WASH products and services; THEN access to improved WASH services in Uganda will increase and be sustained, eventually in the lower wealth quintiles, leading to improved health outcomes and education conditions. 1.3 Evaluation Purpose In line with the Statement of Work (SOW), the evaluation had four objectives: i. To generate insights into how the approaches used by USHA have contributed to stopping open defecation and sustaining this cessation, and to increasing basic sanitation services in districts. ii. To comparatively analyze how other WASH programs in similar contexts in and out of Uganda have contributed to stopping open defecation and sustaining this cessation, and to increasing basic sanitation services. iii. To understand the efficiency, success, and cost of WASH intervention models. iv. To understand the enablers and constraints that affect scalability of WASH intervention models. 11 1.4 Evaluation Scope i. Programmatic Scope: The evaluation focused on how the approaches used by USHA contributed to cessation of OD and to basic sanitation services. Key Informant Interviews (KIIs) and Focus Group Discussions (FGDs) were conducted to generate insights into how other WASH interventions in similar contexts in and out of Uganda contributed to stopping OD and sustaining this cessation, and to basic sanitation services. In addition, desk research explored how other developing countries in and outside Africa have implemented WASH interventions in an effort to attain ODF status in their communities. ii. Temporal Scope: The evaluation covered USHA’s implementation period to date, January 2018 to September 2021. The desk research was limited to sources published in the last ten years, this was considered sufficient to provide a wider perspective of the topic under review. iii. Geographic Scope: USHA operates in 20 districts spread across three regions: Northern , Central East Uganda, and Central West Uganda. The evaluation’s geographical scope was aligned with USHA’s operation areas and covered three districts in every region. (See Annex 4 for the list of selected districts.) The selection criteria were guided by the literature available from USHA and for regional representation of selection of districts. The team purposely selected three districts from each of the three regions covered by USHA. the selection of districts was based on whether USHA had implemented sanitation and hygiene interventions in both communities and in schools. The selection criteria were further guided by data in the national Water and Environment Sector Performance Report, 2020. Three districts of the nine selected (Omoro, Kaliro and Kyotera) were visited for the KIIs and FGDs while virtual KIIs (through Google Meet and phone calls) were conducted with the remaining six districts. Figure 1: Map of Uganda Showing USHA Districts Covered by the Evaluation 12 The desk research covered literature from Uganda, Kenya, Ghana, and the Philippines. For Uganda the research focused on three major non-USAID WASH implementers in Uganda: GIZ, Water for People, and the International Water and Sanitation Centre (IRC). Literature from two developing countries in East and West Africa (Kenya and Ghana) and one developing country in Asia (the Philippines) were also reviewed to get insights on other WASH implementation approaches and results. 1.5 Key Evaluation Questions The SOW had four Key Evaluation Questions with multiple sub-questions. The sub-questions are presented in Annex 1. 1. What are the promising implementation approaches to obtain open defecation free status and move towards basic sanitation for communities in developing countries with a similar context? 2. What are the innovative approaches to improve access to water for rural communities? 3. What opportunities exist for WASH systems strengthening? (Cover governance and leadership, information systems, supply chain, civil society participation, financing and accountability, infrastructure operations, and management and regulation) 4. What are the opportunities and lessons for gender integration within WASH programming in Uganda? 2. EVALUATION APPROACH AND METHODOLOGY To answer the questions in the Statement of Work (SOW), the evaluation reviewed the approaches used by USHA and determined how effective they had been in attaining Open Defecation Free (ODF) status and moving towards basic sanitation. The evaluation included the review of USHA progress data, synthesis of findings obtained through field interviews, and a desk review of relevant literature. The approach to this evaluation was learning oriented. USHA regional teams were involved in the mobilization of the district personnel identified by the evaluation selection criteria, as well as the masons and sanitation promoters involved in the activity. The regional teams also coordinated with local leaders to mobilize Focus Group Discussion (FGD) community participants. The evaluation methodology was qualitative in nature. Quantitative data was analyzed to corroborate qualitative findings. The evaluation assessed the status of Water, Sanitation, and Hygiene (WASH) in selected USHA districts of operation and was forward looking to identify potential opportunities in the sector for future programming. The opportunities were identified from the WASH approaches implemented by USHA and other actors. 2.1 Sampling The selection criteria were guided by the literature available from USHA and for regional representation of selection of districts. The team purposely selected three districts from each of the three regions covered by USHA. As USHA has not implemented activities in schools in some districts (due to the restrictions enforced in response to COVID-19) in some districts, the selection of districts was based on whether USHA had implemented sanitation and hygiene interventions in both communities and in schools. The selection criteria were further 13 guided by data in the national Water and Environment Sector Performance Report, 2020, including the district’s performance in terms of access to and functionality of water supply, sanitation coverage, and ODF coverage. The performance of the USHA districts of operation is presented in Annex 2. The desk research focused on three organizations that have implemented or are implementing related programs in Uganda: GIZ, Water for People, and International Water and Sanitation Centre (IRC). This selection was limited by the time available and was agreed with USAID during the evaluation inception stage. Two developing countries in Africa (Kenya and Ghana) and one developing country in Asia (the Philippines) were purposely selected. The rationales for selection of each district, organization, and country are presented in Annex 3. 2.2 Data Collection Methods Desk Research: The team’s internal desk research reviewed relevant USAID literature and USHA progress reports and other documentation on water and sanitation interventions implemented by USHA. The evaluation assessed USHA’s progress toward expected Life of Project targets to determine which results are on track and to corroborate findings from the field. The evaluation used the established targets for assessing progress achieved toward the set results. The desk research of secondary literature focused on studies, reports, and journal articles published online, that had covered WASH approaches implemented outside Uganda. The online search included searching PubMed (pubmed.ncbi.nlm.nih.gov) using variations of keywords to refine the search. The PubMed search yielded more than 2,000 results. Gray literature tools, such as Bing and Google, were used to systematically identify gray literature using various combinations of keywords and search settings. For each result, the team reviewed the abstract and then, for selected publications, the full text. Key Informant Interviews (KIIs): The team conducted 39 in-depth interviews with various key stakeholders to gain an understanding of USHA’s contribution and the successes, challenges, and potential opportunities in the implementation of its WASH interventions. Key Informants (KIs) were identified during the inception stage through review of relevant literature and were selected based on their role in the WASH sector. District KIs were mainly selected from district Environmental Health and Education Departments. KIIs were also conducted with the USHA team in USAID’s regional and Kampala offices, representatives from relevant line ministries, district personnel, institutional partners, and private sector actors. See Annex 5 for the list of all KIs interviewed. Focus Group Discussions (FGDs): Ten FGDs with masons, sanitation promoters, and USHA community beneficiaries were conducted in the three districts that were visited. Initially nine FGDs were planned, three in each district. However, the team was intrigued by two neighboring villages supported by USHA in Omoro district. One village had ODF status but the other still had cases of OD. The team held FGDs in both villages to understand the contextual dynamics. FGDs conducted are presented in Annex 6. 14 2.3 Data Analysis The data analysis to generate answers to the specific evaluation questions in the SOW entailed several approaches: i. Content analysis of the data gathered from KIIs and FGDs. ii. Triangulation of data from different sources to answer each evaluation question. iii. Review of USHA progress report data. iv. Analysis of findings across countries and organizations to identify common best practices, innovative approaches, challenges, and lessons learned. v. Review of secondary data to establish or confirm contextual information. 2.4 Methodological Limitations i. Limited physical visits. Due to COVID-19 restrictions, the team only visited three of the nine selected districts. It would have been preferable to visit all nine of the districts to get a wider range of opinions. ii. Limited numbers of KIIs. Due to the limited time allocated to data collection, the team could not interview several KIs from districts that were interviewed virtually. iii. Limited data analysis. The time allocated to triangulation and data analysis of data was not sufficient for the scope of the evaluation. 3. DISCUSSION OF FINDINGS This section presents the evaluation’s findings with respect to the key evaluation questions in the SOW. The findings are based on information from discussions with key stakeholders from the nine sampled districts and stakeholders at national and organizational levels including the private sector, and an extensive literature review. Question 1: What are the promising implementation approaches to obtain open defecation free status and move towards basic sanitation for communities in developing countries with a similar context? 3.1 Status of Open Defecation (OD) in USHA Districts Open defecation (OD) refers to defecating in the open, such as in fields, forest, bushes, open bodies of water, on beaches, in other open spaces, or disposing of feces with solid waste. The WHO/UNICEF Joint Monitoring Program Progress Report (2020) estimates that 5 percent of the population in Uganda practices OD (6 percent in rural areas and 2 percent in urban areas). Available statistics from district baseline surveys indicated that in 2018 OD was highest in the Northern region, followed by Central East (CE) and was the least in Central West (CW) as indicated in Figure 2 below. 15 Figure 2: Percentage of OD Households in USHA Districts (2018-2019) Source: District-wide Quantitative Household Baseline Survey Report Extract for the Central East (CE), Central West (CW) & Northern Cluster – 2018-2019 The evaluation obtained information from Key Informants (KIs) confirming USHA’s contribution towards achievement of improved sanitation and hygiene in communities and schools. The statements below from Key Informants provide detail on how USHA’s interventions are potentially moving villages from OD to ODF status. 12.1 9.5 8.4 16.1 4.7 9.0 0 10 20 Bukomansimbi Gomba Kyotera Lwengo Mpigi Sembabule CW Districts 14.3 19.5 1.4 14.0 9.9 4.4 13.7 0 10 20 30 Bukwe Kaliro Jinja Buyende Kayunga Luuka Namutumba CE Districts 50 16.5 28.9 31.8 29.7 24 55.6 0 50 100 Agago Gulu Kitgum Lamwo Nwoya Omoro Pader North “Before USHA came in Nwoya, communities were doing badly. USHA implemented CLTS+ in 100 villages and the district has achieved 60 percent ODF” KI in Nwoya District. “USHA told us that presence of a latrine does not mean absence of OD. The expected change is all about attitude. Before USHA, the district was at 67 percent latrine coverage and after USHA, the district is at 98 percent latrine coverage and a good number of them are improved toilets” KI in Bukomansimbi District. “There was no project like USHA before. There have been improvements in supported schools and in communities. In 2019, latrine coverage was at 68 percent. After working with USHA in 2020 (one year), latrine coverage is at 79 percent. The district wants to copy what USHA has been doing and scale-up in other sub-counties” KI in Kayunga. District. “Our reporting is not only on latrine coverage but also the type of latrine and status of ODF. Before USHA, two villages were ODF out of 407 villages. Within one year of working with USHA, there are 42 villages declared ODF”. KI in Kayunga District. “USHA works in two sub-counties and the latrine coverage in these sub-counties was at 51 percent before USHA interventions. Currently, latrine coverage is at 96 percent in these sub-counties where USHA works” KI in Pader District. 16 3.2 Sanitation and Hygiene Implementation Approaches Identified The evaluation identified existing implementation approaches to obtain open defecation free status and move towards basic sanitation for communities used in other countries, Uganda and by USHA. 3.2.1 Approaches Identified in Other Countries According to the WHO/UNICEF Joint Monitoring Programme Progress Report, as of 2020, the world was not on track to achieve SDG 6 targets which call for universal access to sanitation and hygiene services. Many countries, including Uganda, face challenges in extending services to rural, poor populations. Achieving universal coverage by 2030 demands innovation and intensifying current rates of progress. The desk review results revealed four major sanitation approaches being implemented in selected countries, as outlined below in Table 2. The four approaches are Community-Led Total Sanitation (CLTS), Market-Based Sanitation Improvement Approach (MBSIA), WASH Friendly Schools, and National Campaigns. The literature contains examples of each of the four approaches being implemented in Ghana, Kenya, the Philippines, and Uganda. For example, the National ODF Kenya 2020 Campaign Framework, is an example of a massive, countrywide sanitation upgrade as a result of the CLTS approach, and advocated for development of the sanitation market and supply chains. The approach was developed as an inclusive, participatory, and transformative process. The framework includes sanitation market development and the engagement of the private sector to build on the behavior change component to enable target audiences to build or upgrade their basic latrines. Ghana achieved the Millennium Development Goal for water supply, but it lagged in sanitation. In 2014, with support from UNICEF, the WASH in Schools National Standards and Implementation Model, an example of the WASH Friendly Schools approach, was developed to enable the Ghana Education Service to coordinate and harmonize school-based WASH interventions. In Uganda, over the past decade, as will be discussed in this report, sanitation approaches have often been blended or integrated to adapt to local sanitation conditions and contexts in order to enhance the effectiveness and sustainability of WASH programs. Table 2: Implementation Approaches in Selected Countries Implementation Approach Overview Example countries from literature search Community-Led Total Sanitation (CLTS) Facilitation of the community’s own observations, appraisal, and analysis of their sanitation profile and their practices of defecation and the consequences, leading to collective action to become ODF. Ghana, Kenya, the Philippines, and Uganda 17 Market-Based Sanitation Improvement Approach (MBSIA) Building of the private sector sanitation market for goods and services for which the customer makes a full or partial monetary contribution (with savings and/or cash equivalents) toward the purchase, construction, upgrade, and/or maintenance of their toilet. Ghana, Kenya, and the Philippines WASH Friendly Schools Creation of schools where everyone– children, teachers, and the wider school-community– carries out three key hygiene behaviors to better secure health. The three behaviors are using improved, accessible sanitation facilities, washing hands with soap at critical times, and drinking safe water. Ghana, Kenya, and the Philippines National Campaigns Use of campaigns to educate, sensitize, and encourage behavior change towards making use of a toilet an automatic and positive behavior and social norm, with the goal of changing mindsets towards ending open defecation. Ghana, Kenya, and the Philippines 3.2.2 Approaches Identified in Uganda The main rural sanitation promotion approach used in Uganda is CLTS and to some extent sanitation marketing. The MOH ODF Roadmap, 2019, identifies CLTS as the most effective approach for implementing sanitation initiatives in rural communities. This approach is effective because it rallies communities to work together to eliminate OD in a short time. The Ministry of Health (MOH), however, recognizes other approaches that complement CLTS, such as Household Improvement Campaigns (HIC), Participatory Hygiene and Sanitation Education (PHASE), Participatory Hygiene and Sanitation Transformation (PHAST), and Child Hygiene and Sanitation Training (CHAST). Community-Led Total Sanitation (CLTS): Evidence shows that CLTS is highly successful in rural settings (MOH ODF Roadmap, 2019). CLTS is preferred by the Government of Uganda (GOU) because it does not use subsidies as an incentive and focuses on behavior change to lead to real and sustainable improvements. However, some challenges to the approach, such as slippage to OD have been documented and pose a challenge to sustainability in various programs and contexts (Sophie H., 2019). Ficek and Novotny, (2018) also note that ODF is sometimes achieved by means of low quality and non-durable sanitation facilities. Findings from national and district key informants indicated that in order to sustain ODF status, there has been a general shift from traditional CLTS to a blend of CLTS and other approaches. It was noted that most regions in Uganda have transitioned from traditional CLTS to including other approaches that enhance CLTS’ efficacy. This is corroborated by the MOH Roadmap, 2019, which states that CLTS is “No single approach can lead to transition from open defecation to open defecation free and eventually to basic sanitation status” -KI in Omoro District. 18 deemed the most effective approach, but there are other approaches such as HIC, PHAST, and CHAST, that can complement CLTS. Application of CLTS varies across different contexts. Fostvedt-Mills et al (2018) documented conventional CLTS tools being successfully used among the Karamojong in Uganda. However, local customs and institutions were used with the Maasai pastoralists in Kenya. For example, community leaders were triggered first, and then local proverbs were used to trigger the community. In Tanzania, the Maasai pastoralists’ close relationship with their animals was used to trigger the community. For example, veterinary officers informed people that their animals eat human feces, and therefore, pastoralists indirectly eat their own feces through eating contaminated meat. Community-Led Total Sanitation Plus (CLTS+): This is a modification of the CLTS approach. It aims at improving the quality of sanitation facilities and reducing relapse of communities to OD after attaining ODF status. Findings from Key Informant Interviews (KII) indicate that combining approaches like PHAST, enforcement, HIC, and elements of sanitation marketing, is necessary to overcome the challenges of CLTS. CLTS+ is an improvement from CLTS because it impacts several factors, such as the mindset of people, including leaders, and income levels that affect the outcomes of sanitation promotion. Examples of success factors for CLTS+ include exemplary leadership, clustering of households, and rural sanitation marketing. Sanitation marketing approach: In addition to CLTS, the GOU Integrated Sanitation and Hygiene Financing Strategy (2017) identifies sanitation marketing as another approach. Evaluation key informants cited the FINISH (Financial Inclusion Improves Sanitation & Health) Diamond approach, promoted by the Caritas Fort Portal Diocese Health through Water and Sanitation (HEWASA) project as an example of a sanitation marketing approach used in Uganda. The approach recognizes four critical aspects for the effective implementation of sanitation marketing. i. Communities: People need to demand improved sanitation services, for which they are willing to pay. Depending on the availability of financial resources in the community, households sometimes mobilize these resources using loans. Demand is created by sanitation promoters who market safe sanitation to communities. ii. Businesses: This involves strengthening the supply chain and reusing human waste. HEWASA offers cheaper, new, and improved products, including toilets and services for safely managed sanitation through local supply networks. iii. Financiers: This involves developing and marketing sanitation credit and other sanitation financial products. This is meant to address the demand for financial services from households and entrepreneurs. The motivation for financial institutions includes business generation, increased market share, and access to social investors. An example of involving financiers is the work done by HEWASA and HOFOKAM Ltd to develop WASH loans. iv. Government: The government creates an enabling environment, formulates and enforces policies and guidelines, regulates standards and quality, supports demand creation, standardizes toilet facilities, and ensures environment and client protection. The FINISH Diamond approach is challenged by limited willingness of financial institutions to extend loans to high-risk customers, such as sanitation focused micro, small and medium enterprises and rural households. Other challenges are the negative attitudes of rural people 19 towards loans and the limited capacity of micro-entrepreneurs engaged in the sanitation sector. Additional sanitation marketing interventions identified through KIIs include work done by organizations, like Water for People and Sanitation Solutions Group, that has implemented a fecal sludge management service chain. Organizations have introduced the gulper (a manually operated pump) and decentralized fecal sludge treatment (DEFAST). The focus of DEFAST is to increase the number of sites for the containment, treatment, and reuse of fecal sludge. In absence of treatment sites, the sanitation service chain is incomplete. National Campaigns In Uganda, different international sector days, such as World Toilet Day, Menstrual Hygiene Management Day, and Global Handwashing Day, are commemorated and celebrated. There are also established national campaigns, including the Annual Sanitation Week and the Uganda Water and Environment Week. These campaigns are used to advocate for sanitation and hygiene and showcase and share innovations and experiences to support the adoption and scale-up of promising interventions. 3.2.3 Approaches Implemented by USHA Based on the findings from the selected districts and Activity progress and performance reports, USHA implemented two sanitation promotion approaches: CLTS+ in the Northern region and MBSIA in Central East and Central West. Both approaches were geared towards achieving ODF communities and transitioning ODF villages to basic sanitation status. With CLTS+ USHA leverages the strength of triggering realizations in communities and integrates elements of sanitation marketing, in an effort to improve the quality of latrines so they meet the basic sanitation status. Findings from the districts in the Northern region indicate there have been great improvements in the status of sanitation and hygiene and in the construction of basic sanitation facilities. Contrary to traditional CLTS, the quality of household sanitation facilities is prioritized in the CLTS+ approach and this led to USHA overcoming the challenge of quality and sustainability associated with the traditional CLTS approach. Improving the quality of toilets in ODF communities involved developing a local sanitation supply chain linking masons, hardware shops, and sanitation promoters. This MBSIA-type of activity allowed households to improve their toilets by themselves. USHA developed a Do-It-Yourself (DIY) toilet product catalogue with a range of products, including those needed for screeding the toilet floor, building the floor with a slab, and establishing the super-structure using either semi￾permanent or permanent materials. USHA only promoted the DIY approach in the Northern region. Compared to CLTS+, which seeks to make an ideal homestead, with the DIY approach USHA focused on the quality of the toilet, especially on moving households from unimproved to basic toilets. The DIY option was a good innovation, but it appears that despite the DIY angle, the information about construction remained with sanitation promoters “For USHA, it’s not only about the presence of latrine in ODF communities. It is all about the quality of the latrine. This has made people move directly from OD to improved sanitation facilities” -KI in Pader District. 20 and masons. The evaluation did not find any people that used the DIY option themselves. Information gathered from Focus Group Discussions (FGDs) indicated that most people used masons for the construction of toilets, and none confirmed having seen the DIY product catalogue. This lack of available information about people taking the DIY option might be due to the limited number of communities FGDs conducted during data collection. However, all masons interacted with were able to show the DIY catalogue, and confirmed they used it to get products to construct toilets for households. This was an indication that people relied on masons to do the construction rather than adopting the DIY approach In the Central East and Central West, USHA focused on the MBSIA approach. The MBSIA approach used had a strong focus on demand creation through community mobilization triggering, awareness creation, and use of sales catalysts and sanitation promoters. USHA’s customer segmentation study was useful in determining the marketing strategies appropriate for the different customer segments. To meet the household demand for sanitation facilities, there were efforts to develop the supply chain in rural areas. The supply chain included masons who had links to aggregators of construction materials and hardware stores. The masons offered a number of products, such as toilet upgrades and new facilities. The masons’ range of products was improved with SaTo (Safe Toilets) products, like the SaTo pan, SaTo plate, and SaTo stools to help meet the needs of the various customer segments. USHA implemented several capacity building initiatives targeting masons to help enable them meet customer needs and address challenges related to the sustainability of facilities. An example of these initiatives is the training on technologies that address collapsing soils, high water tables, and rocky grounds. Information from KIIs indicated that differences in motivation for sanitation improvement exist between Central East and Central West regions. In the Central East, health benefits were mentioned to be the motivation behind people improving their household sanitation and hygiene. In the Central West, dignity was mentioned as a key factor in sanitation and hygiene improvement. Owning a quality toilet in the Central West was associated with upholding one’s reputation in the community. The evaluation noted some benefits of the MBSIA approach over traditional CLTS and CLTS+. These benefits include: i. Product sales at the household level provides a platform for people to know and understand product choices and how best they can buy sanitation products. ii. Households are at liberty to select the mason of their choice and the building negotiations are concluded between the two parties without involvement of the project. iii. A shift from just construction of a toilet to beautifying the home. Some interviewees said their next step after construction will be painting the toilet to make it attractive and to tempt other community members to also construct a toilet. This kind of potential diffusion of toilet facility improvement is important for attainment of ODF and moving to basic sanitation. District KIs were concerned that USHA interventions were implemented in some sub￾There was one of the household head I was convincing to improve the toilet but had refused. When he finally agreed to construct a toilet, his son brought in-laws to visit him, after the function he looked for me to inform me that I saved him from embarrassment FGD participant in Kyotera District 21 counties and not others. Their concern was that there is inconsistent attainment of ODF in districts, and that the positive results achieved in a few sub-counties look insubstantial as the district still has areas with OD. The district KIs believe that a concentration of interventions in a few selected districts has a greater positive impact than interventions spread over only a few sub-counties in a district. They expressed a wish that USHA concentrated its interventions in just one district, which can be used as a model district to inspire others. 3.3 Approaches for WASH in Schools WASH in schools enhances learning environments for children. It is a foundational block for children’s right to live in dignity and is a contributing factor to increasing their prospects for a better future. Schools need inclusive WASH facilities to offer a conducive learning environment for all pupils and those schools with appropriate WASH facilities offer safety and dignity to girls which helps retain them. However, ensuring adequate water, sanitation services, and daily hygiene behaviors in schools in Uganda is a continuous challenge for the government and its development partners. This challenge coupled with the COVID-19 pandemic means that WASH service delivery remains a challenge. The demand for WASH services in schools is higher than the current capacity to supply, meaning there is a need for more innovative school WASH solutions. According to the Ministry of Water and Environment’s Annual Report (2019), 42 percent of schools had available handwashing facilities. The Ministry of Education and Sports’ WASH Report (2016) indicated that the pupil to stance ratio was 71:1, which is well below the recommended national standard of 40:1. The National Micro Planning Handbook for WASH in Public Primary and Secondary Schools in Uganda (2019) noted that access to water in primary schools stood at 58 percent. This implies that there is a significant WASH gap that needs to be addressed. 3.3.1 Approaches for WASH Friendly Schools in Uganda Two approaches for WASH in schools are “hardware” approaches that provide WASH infrastructure that caters for the needs of boys, girls, and people with disabilities and “software” measures that look to behavior change and sustainability. Interventions that encompass both hardware and appropriate software approaches can create the desired impact within schools. The facilities constructed in the hardware approach provide separate stances for boys, girls and people with disabilities and include girls’ washrooms and incinerators for sanitary disposal of used Menstrual Hygiene Management (MHM) supplies. Examples of promising software interventions include school toilet competitions for various school stakeholders that instill the idea of responsibility for ensuring WASH improvements and their sustainability. This competition approach also pushes schools to define their WASH needs and find innovative solutions to address these needs. Another software approach is the “In 2018/2019, GIZ supported Kampala Capital City Authority to implement the Toilet Making the Grade school competition in all the public schools in the city. Schools reported marked improvement in sanitation and hygiene. Cleanliness improved, water bills were paid and school health clubs become functional” KI, National Level. 22 Three-Star Approach that teaches learners about good water, sanitation, and hygiene practices. The Three-Star Approach assumes that once children learn the preferred good behaviors, they will become change agents for the hygiene behaviors of their siblings, parents, and communities. These two software approaches have been implemented in all public primary schools in Kampala by the Kampala Capital City Authority. The GIZ Sanitation for Millions program introduced group handwashing facilities in schools in Uganda, including WASHalot, a water saving technology that can be used by an individual or a group. WASHalot is a deviation from the usual approach of hand basins which are not always appropriate in school settings. The WASHalot provides over ten hand washing points at once and uses low pressure values to release just enough water for each person, reducing water wastage. The WASHalot reduces the time spent by pupils waiting to wash their hands. Innovations like WASHalot makes access to handwashing easier and are suitable for schools with and without a connection to piped water supply systems. 3.3.2 WASH Friendly Schools Approaches by USHA USHA has focused on supporting schools to become WASH friendly through providing hardware (constructing or rehabilitating basic drinking water and sanitation facilities) and using software measures. USHA interventions are based on the eight parameters, shown in Figure 3, required for a school to attain WASH friendly status. Figure 3: USHA Parameters for a WASH Friendly School USHA used the eight parameters to develop four levels of WASH friendliness, a new way of measuring WASH in schools. These levels range from zero points for “No WASH Friendliness” to eight points for “WASH Friendly”. These levels were used to measure the success of the efforts towards attaining WASH friendly status. This developed measurement is a holistic approach that is an improvement on the current measure of pupil-to-stance ratio, “The WASHalot as a hand washing technology is efficient, saves water and several people WASH at the same time” KI National Level. “Five schools in Ongako sub county have received sanitary facilities, six group hand washing facilities and five incinerators from USHA intervention.” - KI Omoro District. 23 which does not give an indication of water access or Operation and Maintenance (O&M). School WASH friendliness was incorporated into MWE’s Joint Water and Environment Sector Performance Report (2020), and according to the Ministry's implementation plans presented in the same report, school WASH friendliness will continue being monitored and measured after USHA closes. This evaluation established through KIIs that the levels of WASH friendliness are measured pre- and post- USHA intervention. Information obtained from USHA confirmed that there is a protocol between USHA and the Uganda Learning Activity, for the latter to monitor WASH friendliness in schools post-USHA intervention to help understand the sustainability of the changes. USHA supported schools in all the nine districts covered by the evaluation to attain WASH friendliness using a mix of governance, service delivery, and social and behavior change interventions. Some schools also received support in the construction of both sanitation and water facilities, while others received support for one of the two. The software approach targeted district staff, School Management Committees, teachers, parents, and pupils. The approach also established or revived School Health Clubs and sector day celebrations, and trained teachers and pupils on making reusable menstrual pads and liquid soap to support hygiene and handwashing. USHA did not work in schools for almost two years due to lockdown of schools as a preventive measure during the COVID-19 pandemic. WASH Operation and Maintenance (O&M) in Schools The sustainable management and use of WASH facilities in schools involves O&M. KIs recognized that USHA developed O&M guidelines that included planning for the O&M resources, and training on how to use the guidelines. This training was provided for local government staff, including District Health Inspectors (DHIs), District Inspectors of Schools, and Health Assistants, and for senior female and male teachers, school officials, Parent Teacher Associations (PTA), School Management Committees, sub county chiefs, and school patrons. This evaluation’s interpretation of the result of the training on guidelines is that target schools can now plan for WASH O&M from an informed point of view. Social and Behavior Change Communication (SBCC) The KIIs revealed that USHA developed Social and Behavior Change Communication (SBCC) messages and materials that include calendars, board games, posters, jingles, badges, and a teacher’s guide on how to use the materials. In line with child centered messaging, these messages were based on “Soapy”, a positive cartoon character. USHA’s SBCC approach is child friendly because it uses child centered messaging and engagement approaches like games. This SBCC messaging was identified by KIs in the districts a promising approach because it is suitable for children as they can easily relate to the messages. Question 2: What are the innovative approaches to improve access to water for rural communities? 3.4 Innovative Approaches for Water Supply The identified innovative approaches for water supply from other countries, Uganda, and 24 USHA are the Community-Based Management System, Direct Local Government Provision, Public Utility Provision, Private Sector Provision, and Supported self-supply. 3.4.1 Innovative Water Supply Approaches in Other Countries Access to safe, reliable, and affordable water remains a major challenge for vulnerable communities in many countries across the world. A variety of approaches is needed to protect and enhance access, especially with the additional impacts of climate change, population growth, and urbanization in the coming decades. Over the years, the Community-Based Management System (CBMS) Model has been questioned as an effective means of service provision due to its lack of professionalism, limited monitoring capacity, and financing deficits. However, according to a study from the Philippines published in 2003 by the International Water and Sanitation Centre (IRC), community-based management with elements of professional support was very effective in small towns that received technical assistance from nearby Water Districts. The technical assistance enabled community-based management bodies to introduce commercial accounting systems, domestic water meters, and well-organized billing and collection systems. Table 3: Water Supply Service Delivery Models Service Delivery Model Overview Community￾Based Management System Relies on voluntary water committees to carry out basic day-to-day O&M and administrative tasks to keep the system going and to address minor repairs. It is a common approach in smaller communities that are typically served by point source supplies. Direct local government provision Local governments plan and deliver water services to communities. They use regulations and guidelines for water supply. Public utility provision Public companies provide water and sanitation services. Private sector provision Private sector firms with specialist skills are awarded contracts for specific service delivery in communities with more complex water supply needs and demands for higher service levels. Supported self￾supply Individual households or groups of neighbors invest their own money and time in and incrementally improve their own services. O&M is done by the households themselves with indirect support from local or external philanthropic or government organizations. 3.4.2 Innovative Water Service Delivery models in Uganda The provision of rural water services is predominantly through local government. Other recognized approaches include the CBMS model for rural point sources and gravity flow schemes and self-supply initiatives by individual users and small groups. Private operators and Water Supply and Sewerage Boards are used for piped water supply in small towns and rural growth centers, but use of private operators is being phased out. Public utilities, such as the National Water and Sewerage Corporation and Umbrella Authorities, are also involved in water service delivery. 25 a) Local government provision: Provision of rural water services is predominantly through local governments with improvements in community-based management systems through a professional performance management approach introduced recently. The Local Government Act (1997) entrusts the provision of water and the maintenance of facilities to district local governments. Through the District Water Offices, districts act as the service authority and have a variety of critical roles, such as budgeting and executing, through the private sector, the establishment and repair of water facilities. The Local Government provides the budget in the form of District Water and Sanitation Conditional Grant. The districts are annually issued with implementation guidelines that set out how the grant should be used by the District Water Offices. For example, the District Implementation Guidelines 2021 instructed districts to spend less than 15 percent of the grant on CBMS interventions and hygiene promotion; 14 percent on the operation of the District Water Office, and 15 percent on rehabilitation of water facilities. Districts are required to provide financial and technical support to sub-counties and to ensure that established standards for O&M are maintained. This support includes providing technical guidance on planning and budgeting, implementation, and monitoring of work plans. The support also includes planning and co-financing the training of Hand Pump Mechanics (HPMs), plumbers, masons, and scheme attendants. The support additionally covers providing tool kits for O&M and selling them on to water user committees (WUCs) and enacting O&M bylaws and ordinances. b) Self-supply: In this approach individuals, households, or community groups build, improve, and manage their own private water supply systems, without help from the government or NGOs. The individual, household, or group provides the investment cost for the water source, either in cash or kind. Self-supply reduces the burden on public resources by reducing the distance water needs to travel and reducing the number of users. The water made available through self-supply may not meet the minimum service level standards in Uganda, so it is essential that district local governments encourage water users to continue to improve their own water supplies. Self-supply guidelines were developed to enable districts to support self￾supply initiatives through provision of technical assistance. c) Community-Based Management System (CBMS): In this approach community members are responsible for the O&M of their water supplies, particularly water sources, such as boreholes fitted with hand pumps, small piped schemes, and gravity flow systems. The participation of communities in decision making and adequate sensitization, training, and follow-up are essential for CBMS to succeed. O&M includes participation in planning, preventative maintenance and repairs, and payment of user fees. Each community should select a competent Water and Sanitation Committee (WSC) or Water User Committee (WUC) and a caretaker. Functionality of WUCs varies from place to place. This approach has had some challenges especially regarding functionality of water facilities. The MWE has taken several steps aimed at addressing the high non-functionality and partial functionality of rural water facilities. Particularly, it set up a new division focused on O&M and encouraged a shift from use of galvanized iron to stainless steel and PVC in boreholes. MWE recognizes that the current rural development agenda necessitates a move beyond CBMS towards a Professional Management Arrangement (PMA). PMA will allow for the long-term functionality and financial sustainability of rural water systems in Uganda. According to MWE’s National Framework for Operation and Maintenance of Rural Water 26 Infrastructure in Uganda. (2020), a PMA is where a sustainable O&M model for water supply infrastructure is guaranteed through formal contract-based performance management arrangements. MWE plans to improve the CBMS model by introducing PMAs for water supply facilities not under the management of Umbrella Authorities or the National Water and Sewerage Corporation (NWSC) in non-gazetted rural areas. MWE’s improved CBMS (known as CBMS+) will include four features: i. An area-based approach. CBMS+ will operate and maintain all rural water facilities within a sub-county, a cluster of sub-counties, a district, or a cluster of districts. The Area Service Provider (ASP) will take responsibility for operating and maintaining all water systems including all point sources in the defined area. ii. Professional management structures with timely follow-up support. These structures will include two distinct aspects: a Community Management structure and the ASP. The Community Management structure has three levels, a) a Water Source Committee) the sub-county water sanitation board, and c) the District Water and Sanitation Board (DWSSB). The ASP is composed of the a) caretaker or scheme operator, b) HPM or technician and c) contract holder. These structures will help ensure timely follow up and regulatory support and increased functionality and water reliability for all users for point and piped water supplies. The PMA signed between the DWSSB and the ASP will have clear performance indicators. iii. Financial sustainability. The PMA will require all water users (households, institutions, and businesses) to pay for water service in addition to the Community Contribution to Capital Costs contributed at the time of construction or rehabilitation. The tariff structures for the operations and capital maintenance expenses that will be covered by the users will consider affordability. The ASP will take responsibility for collecting the user fees and banking them in a revenue collection account. iv. Availability of quality spare parts. The ASP will be responsible for ensuring availability of quality spare parts. The ASP can enter into memorandums of understanding with reliable suppliers or establish a framework contract with a private supplier to continuously replenish spare parts. The MWE is transitioning to CBMS+ through a stepwise process (Table 4) as it implements its National Framework for Operation and Maintenance of Rural Water Infrastructure in Uganda (2020). The roadmap starts by creating buy-in and building capacity (Phase I) and a subsequent transition to Area Service Providers (ASP) as rural utilities (Phase II). Phase I and II are critical interim measures on the road toward the complete implementation of CBMS+ approach. 27 Table 4: Phases in the MWE O&M Roadmap 2030 Phase I (2020 – 2021) Phase II (2022 – 2025) Phase III (2026 – 2030) • Community sensitization • Political and stakeholder buy-in • Development of guidelines and training • Resource mobilization • Rollout of the framework • Review, adaptation and strengthening of systems • Ongoing capacity development • Private sector (ASP) strengthening • Transition to area-based approach • Area-based approach fully integrated • Resource mobilization • Documentation of process The strategic direction of MWE, as described in the O&M Roadmap 2030, will include the design of rural water interventions that fall into three broad but distinct groups. These groups are: i. Supporting public utility models: MWE is phasing out private operators for small towns and rural growth centers and moving to a purely public utility model by scaling up support to and capacity development of Umbrella Authorities so they can scale up their provision of rural water supply. ii. Improved CBMS: The O&M Roadmap 2030 outlines PMAs as the key measure to improve the O&M of rural water facilities. This process of transitioning to PMAs is still in the early phases of development and needs targeted measures that focus directly on institutionalization of CBMS+ from the national to the district level. MWE intends to create district water and sanitation service authorities and also establish ASPs for management of water services in non-gazetted rural areas. iii. Private companies with performance-based contracts: MWE will develop a preventive maintenance model where a private company or operator enters into a performance contract as an Area Service Provider (ASP). In these interventions a private company enters into a performance-based contract with a government actor (such as a district) and a community. The company agrees to deliver a set of maintenance services for a fee. 3.4.3 USHA’s Approaches to Water Supply to Rural Communities USHA has two rural water supply components. One component is in a partnership with Rotary International which uses the local government’s service provision and CBMS. Relying on CBMS for O&M presented challenges as USHA did not improve the approach so that it could manage facilities better than the usual WUC approach. The second and more innovative approach was USHA’s work in supporting two out of the six Umbrella Authorities (UAs) in Central East and Central West to extend their pro-poor water services. USHA supported Umbrella Authorities in Central East and Central West to improve their water supply system capacity and to provide pro-poor water services, such as connection subsidies and a pro-poor water tariff structure. Households in the lowest poverty quintile were able to get a free water connection. An inverted block tariff, where the initial cubic meter is cheaper than subsequent units, targeted poor people. Interviews with KIs at the UAs revealed that this tariff was perceived by the UAs as a good initiative and has been considered for replication in water supply systems operated by UAs. 28 Findings from interviews with KIs at UAs indicated that USHA strengthened the UAs’ internal reporting systems including their billing systems, improved their monitoring through data collection, and updated the Umbrella Performance Management Information System (UPMIS). USHA also reported that it built technical capacity through contracting that allowed staff at the UAs to gain hands-on experience in water systems development and network expansion. Different piped water schemes face different challenges that undermine their commercial viability. The UA support approach used by USHA enabled the UAs to harness existing potential business opportunities through expansion of their customer bases. Question 3: What opportunities exist for WASH systems strengthening? (Cover governance and leadership, information systems, supply chain, civil society participation, financing and accountability, infrastructure operations, and management and regulation) 3.5 Opportunities for WASH Systems Strengthening The evaluation assessed the seven requested WASH systems strengthening opportunity areas: governance and leadership, information systems, supply chain, civil society participation, financing and accountability, infrastructure operations, and management and regulation. 3.5.1 Governance and Leadership Information obtained from district KIIs indicated that USHA built the capacity of district staff to hold regular meetings with different stakeholders. District respondents informed the evaluation that they are now in a better position to present work plans and budgets, which helps rationalize resources. The mandated institutions for WASH have coordination mechanisms and implementation structures from the community to national level and there is a parliamentary WASH forum that indicates that there is recognition and prioritization of WASH in legislation. There are a number of sector-wide approaches, such as the Uganda Water & Sanitation Environment Week, the Joint Water and Environment Sector Review, District Water & Sanitation Coordination Committees (DWSCCs), the Uganda Water & Sanitation Network (UWASNET), National Sanitation Working Groups and sector working groups, and program￾based planning and management through the Third National Development Plan (NDP III). The NDP III human capital development program led by MOES includes the Directorate of Water Development in MWE and MOH. These sector-wide approaches will strengthen the alignment of plans, budgets, and implementation at the macro, sector, and local government levels. Opportunities for future systems strengthening programs under governance and leadership include: i. Empowering women to actively participate in WASH activities and to fulfil leadership and decision-making roles. This could include promoting women who are currently part of water and sanitation committees to leadership roles, promoting peer￾to-peer learning and competition between leaders, supporting institutional 29 coordination and collaboration, and optimizing and leveraging the sector-wide approaches outlined above to sensitize communities on the importance of sanitation and hygiene. ii. Leveraging existing national structures, such as the Water Resources Institute and the Appropriate Technology Centre for Water and Sanitation, to offer face-to-face and online courses for district staff and other actors involved in strengthening WASH systems. iii. Findings from KIIs indicate that the private sector strengthens MBSIA and makes it a potential and promising implementation approach to obtaining ODF status and moving towards basic sanitation. There is an opportunity to engage the private sector for financing and systems strengthening for sanitation marketing. iv. Whilst the government’s systems and capacity are important for creating an enabling service environment, there is an opportunity to focus on strengthening market systems and the capacities of market-based service providers, such as masons and fecal sludge management (FSM) providers. Legally formalized and professionalized FSM service providers would enhance sanitation service provision. Findings indicate that current FSM services are informal and, in most cases, not registered. The government and its partners have invested in FSM facilities, but the utilization is still low. FSM for schools remains a big challenge. 3.5.2 Information Systems Currently, line ministries have parallel reporting lines and use different information systems for data capture, analysis, and reporting. MOES uses its Education Management Information Systems, MOH uses the Health Management Information System (DHIS2), and MWE uses Water Information. In an attempt to harmonize these information systems, UNICEF is currently supporting MWE, MOH, and MOES to develop an integrated information system that is aligned to the SGDs. As part of this harmonization, new data collection tools for sanitation and hygiene that are aligned to the SDGs have been developed. However, the districts do not have the required capacity to use these tools. The ministries are looking to move from manual-based data collection to a digital data collection system. Developing the capacity of districts to be able to use the new data collection tools for sanitation and hygiene is a potential opportunity for USAID. The opportunity for strengthening information systems lies in strengthening monitoring and evaluation processes through capacity development of district personnel. Supporting ministries and districts in their use of digital data collection and analysis platforms will ease the storage, retrieval, and reporting on data from WASH interventions. Other opportunities exist in strengthening the mechanisms for post-ODF monitoring and the regular ongoing monitoring of hygiene practices in communities. USAID could also strengthen the monitoring of FSM services, which may include monitoring by the service authority, FSM associations, and other market players. There are also opportunities to strengthen the DWSCCs’ capacity to facilitate sharing and learning on sanitation and hygiene initiatives and to participate in and benefit from national and regional learning platforms. 30 3.5.3 Civil Society Participation UWASNET has a mandate to influence the WASH sector policies that USHA’s work supports. It also streamlines the coordination of WASH-related Civil Society Organizations (CSOs). UWASNET is currently expanding its mandate to include private sector organizations. DWSCCs include all actors, including CSOs. The DWSCCs have undoubtedly contributed to the scaling up of WASH services in Uganda. There are some challenges in the DWSCCs that need attention, such as their operational coordination at district level with NGOs. The opportunity lies in strengthening UWASNET’s financial sustainability to reduce its dependency on government and increase its ability to provide objective and independent policy advocacy and hold the government accountable. 3.5.4 Supply Chains a) For Water Supply Findings from the literature review indicated that the water supply chain in Uganda is relatively well developed with a number of actors at each stage of the water supply value chain. The supply value chain includes water catchment management; water harvesting and storage; water treatment; water transmission, distribution and utilization for production and domestic use; water collection, treatment and safe disposal of waste water. Figure 4 below presents the water service chain in Uganda. This chain was conceptualized by this evaluation from findings in the available literature. Figure 4: Water Service Chain in Uganda Information obtained from interviews with the UAs indicated that there is a multi-sectoral supply chain for water related products and services at national and district levels. The private sector, including companies, plumbers, manufacturers, drillers, and hand pump mechanics, is active in water supply. The available opportunity lies in developing projects along the water supply value chain. These projects could include the restoration and protection of water catchments, development of catchment and sub-catchment management plans for water for production and rural water utilities, establishment of water treatment plants, operationalization of the O&M framework, facilitating public-private partnerships, establishment of rural water utilities, construction of multipurpose bulk water supplies, financing of capital replacements, establishment of irrigation systems, and establishing waste cycling plants. 31 b) For Sanitation The sanitation supply chain is underdeveloped and there are several constraints within the sanitation value chain that hinder the development of the supply chain. These constraints are discussed in Table 5 below. Table 5: Constraints to and Actors in the Sanitation Supply Chain Based on the Literature Review and Interviews Service Constraints Identified from Interviews and Literature Actors Containment • High cost of sewerage systems and lined pit latrines • Cultural practices which promote OD • Weak enforcement of the Public Health Act • Weak supply chain networks especially in rural areas. MWE, NWSC, MOH, urban authorities, MOES, institutions (schools, health units etc.), households, suppliers of materials, sanitation entrepreneurs, and Civil Society Organizations (CSOs) Emptying • Lack of emptying services in rural areas • High cost of emptying services • Majority of treatment plants not designed to receive sludge from unlined pits • Unregulated pit emptying MWE, NWSC, MOH, urban authorities, MOES, institutions, sanitation entrepreneurs, and CSOs Transportation • Lack of cesspool emptying trucks in rural areas • High costs of trucks NWSC, sanitation entrepreneurs, and CSOs Treatment • Lack of fecal sludge treatment plants in small towns and rural areas MWE, NWSC, MOH, sanitation entrepreneurs, and CSOs Reuse or disposal • Limited awareness of or market for products • Limited knowledge on the technologies needed for production of different products • Existing national fecal sludge management standards do not favor the use of fecal sludge NWSC, urban authorities, sanitation entrepreneurs, farmers, and households Evaluation findings from KKIs and the literature indicate that the sanitation service chain is also relatively well developed, but mainly concentrated in cities and large towns. Some projects focusing on sanitation marketing supported the development of a sanitation supply chain for small towns and rural areas by working with hardware stores to provide appropriate product quantities of materials for toilet construction. For instance, through these projects a person could buy just one or two kilograms of cement and a single wheelbarrow of sand. USHA trained masons and worked with Lixil and Nice House of Plastics to manufacture and market SaTo products for rural households. There are potential opportunities in fecal sludge service chain development and expansion to rural areas. Future programs can further develop the private sector sanitation supply chain. For instance, there are currently only a few actors working in FSM; it is as yet untapped by enterprises. The scaling up of sanitation marketing approaches should be strengthened to develop the sanitation supply chain across the country. Another opportunity is in supporting private sector 32 actors involved in sanitation services to formalize their businesses and to access financing for business expansion and growth. Other opportunities include strengthening the regulation of fecal sludge emptying services in urban authorities, establishing financing options for sanitation entrepreneurs, establishing regional and decentralized fecal sludge treatment plants, and reviewing the standards for the use of fecal sludge. 3.5.5 Financing and Accountability There are budgetary allocations to all districts for WASH. These are the Primary Health Care budget from MOH, the School Facilities Grant from MOES, and the Water and Sanitation Conditional Grant from MWE. However, district officials reported that these resources are not adequate to support sanitation and hygiene interventions in communities and schools, and have conditions attached to them. Rural water supply is financed by the Water and Sanitation Conditional Grant sent to the District Water Office by MWE. Some of the districts recognized the contribution of CSOs and NGOs in the provision of water and sanitation services. However, the district officials complained that most organizations do not share their budgets with the districts, so it is difficult to harmonize plans. A number of regulatory mechanisms are in place to hold officials accountable for public funds. These mechanisms include the Inspector General of Government, Public Accounts Committee, Economic Monitoring Unit, Auditor General’s Office, and the State House Anti￾Corruption Unit. In addition, ministries hold annual sector reviews and report on performance. The evaluation identified the providing WASH conditional grants to district local governments as a potential opportunity for USAID to strengthen WASH interventions at district level. 3.5.6 Infrastructure Operations MWE has several capacity development interventions for WASH infrastructure operations. These include: a. The Appropriate Technology Centre for Water and Sanitation. b. Associations to manage infrastructure, such as Hand Pump Mechanics Associations and Umbrella Authorities. c. The Water Resources Institute that provides a framework for policy dialogue and engagement on key water sector issues. d. Deconcentrated MWE structures, such as urban and small-town water supply systems, Water Management Zones, Regional Water Quality laboratories, and Technical Support Units. USAID can use the deconcentrated MWE structures to design capacity building interventions that support district infrastructure operations. 3.5.7 Management and Regulation There is an enabling policy and regulatory framework to facilitate WASH service provision. Some policies, such as the Water Act and the Water Policy and Public Health Act, are now under review. Future programs could help ensure that the mandates for sanitation and hygiene improvement are clear in these policies and that sanitation and hygiene related activities are efficiently coordinated. 33 The MWE guidelines to the district emphasize the establishment of community management structures. District guidelines aim to ensure software-type steps are implemented before water is supplied to a community. These software steps include Community Contribution to Capital Costs for water infrastructure construction and improving community sanitation and hygiene. There are also sanitation promotion guidelines in place at MWE. Through UWASNET, CSOs advocate for stronger action on WASH policies and regulations. There are village level sanitation structures, such as Village Health Teams, and these are critical in the management of WASH activities. There are opportunities for USAID to strengthen sanitation inspection and enforcement processes, and further engage civil society to hold service authorities and providers to account on sanitation and hygiene issues. 3.6 Sanitation Financing From UNICEF’s Guidance on market-based sanitation (2020) the evaluation identified successful sanitation financing models for big businesses, Micro, Small, and Medium Enterprises (MSMEs), and households. There are three financial models for sanitation financing for big businesses and MSMEs. These are: a revolving fund of temporary capital from which loans are drawn; a guarantee fund that provides assurance to and incentivizes financial institutions and microfinance institutions (FIs/MFIs) to lend to sanitation businesses; and subsidized interest rates to increase the affordability of sanitation loans to borrowers. The three models stimulate sanitation markets by enhancing access to sanitation finance. The household financing models identified in the UNICEF guidance are savings (through Village Savings & Loans Associations; Rotating Savings & Credit Associations; savings groups; and lay-away purchase); borrowing (through consumer loans from MFIs, community or formal banks, revolving loan funds, or self-help groups); and social welfare subsidies to vulnerable households. 3.6.1 USHA’s Sanitation Financing Interventions KIIs revealed USHA carried out the financing activities shown in Table 6. Table 6: Sanitation Financing by USHA Type of Enterprise Sanitation Financing Intervention Big Business • Business-to-business financing. There was technology transfer and partnership with Lixil and Nice House of Plastics. • Commercial loans from financial institutions as indicated by USHA’s partnerships with local banks. Example is partnership with Opportunity bank to finance FSM enterprises. • Asset financing. Example is USHA financing Nice House of Plastics to produce sanitation products. Micro, Small, and Medium Enterprise (MSME) • Blended financing that combined business contribution, commercial loan, and grant. USHA used this approach for FSM enterprises. 34 • Asset financing and business loans. This was evidenced by the support to Musoga Sewerage services in Central West region. Households • Financed through SACCOs and VSLAs and loans from micro finance institutions. • SACCOS and VSLAs identified as promising financing models for households as banks are not willing to work in rural areas. 3.6.2 Barriers to accessing sanitation financing The evaluation’s findings from the literature review and KIIs indicate there are number of barriers to sanitation financing and that the barriers are different for big businesses, MSMEs, and households. The findings are summarized in Table 7 below. Table 7: Barriers to Accessing Finance Type of Enterprise Barriers to accessing sanitation financing Big Business • The sanitation sector is perceived by financiers as a low- or no￾profit sector, with a low capability to repay loans. • High interest rates prohibit business from taking loans because this negatively impacts potential profits. • More established businesses run the risk of being ineligible for any type of formal financing, being too big for MFIs and too small for banks. Micro, Small, and Medium Enterprise (MSME) • MSMEs are often informal, with insufficient collateral and a reputation for high repayment risks. As a result, financial institutions are hesitant to do business in the sector. • MSMEs lack formal structures such as business registration, formal accounts, and collateral which disqualifies them from accessing formal finance. • The sanitation sector is generally viewed as an unprofitable investment because the sanitation business model is not well understood. Many financial institutions are skeptical of the profitability of sanitation businesses. Household • Some rural households have low and seasonal incomes and experience two types of financing barrier: i. Cash flow problems (liquidity) ii. Absolute cash poverty problems (real inability to pay). Question 4: What are the opportunities and lessons for gender integration within the WASH programming in Uganda? 3.7 Gender Integration Within WASH in Uganda The Government of Uganda, through MWE, developed its Water and Sanitation Gender Strategy (2018-2022) with a goal to empower men, women, boys, girls, and vulnerable groups through ensuring equity in the access to and control of resources in the water and sanitation sub-sector, contributing to poverty reduction. 35 USHA in 2019 developed its own gender and youth strategy to guide implementation of its interventions. Despite these strategies, gender gaps in WASH still exist. 3.7.1 Gender Gaps in WASH Findings from key informant interviews and FGDs indicate that gender gaps exist in the appropriateness of WASH technologies and there are limited Menstrual Hygiene Management (MHM) facilities at most institutions and households. The absence of inclusive WASH facilities at the household level, where there are no separated latrines for males and females, remains a gender gap. The evaluation was informed that there is a general lack of provisions in sanitation and hygiene facilities for people with disabilities and other marginalized people. School sanitation structures are shared by students and teachers leading to lack of privacy. Lack of facilities for marginalized people is a result of the absence of an earmarked budget for sanitation and hygiene services at the district level. Amongst the community members interacted with there was a notable lack of awareness on gender issues; most people interpreted “gender” to mean only equality between men and women. Due to the patriarchal culture in Uganda, women often do not have control over household resources, and this limits their ability to influence decisions, including regarding the construction of sanitation facilities. However, women suffer most when there is a lack of sanitation facilities. The achievement of ODF in communities reduces the burden, that mainly falls to women, of looking after children who are sick due to diarrheal diseases. There are good gender-related policies in Uganda, such as the National Equal Opportunities Act and the National Action Plan for Women and Girls, that aim to empower women to take charge of their lives and wellbeing. However, implementation of these policies is limited. For example, there is also low prioritization of gender in planning and financing at the district and household levels. 3.7.2 Approaches that Address Gender-related Gaps The identified WASH gender gaps can be addressed through social mobilization, including conducting community and household outreaches through CLTS and sanitation marketing. During social mobilization, emphasis should be on equal engagement of men and women. Awareness creation, through radio talk shows, community dialogues, radio spots, and campaigns, can also encourage people to appreciate the roles of men and women in WASH interventions. The software-type steps that are included pre- and post-construction should integrate gender issues. Approaches to address gender gaps also include building the capacity of and empowering all WASH actors, both men and women. Increased capacity and empowerment are achieved through establishing and training community-based entities such as Water User Committees, sanitation promoters, Village Health Teams, masons, Hand Pump Mechanic Associations, and using participatory approaches such as PHAST and HIC. 3.8 Enabling Factors for USHA’s Sanitation Approaches The evaluation found several factors that facilitated the implementation and success of USHA’s different sanitation approaches. These are summarized and categorized below. 36 The overarching operating context related factors that enabled USHA’s approaches to work are: i. Presence of national sanitation policies and guidelines that push different government agencies to embrace and support implementation of sanitation promotion approaches. These policies provide a good implementation framework for innovation and adoption of new strategies for sanitation promotion and enabled USHA to implement their innovative MBSIA. ii. Existing government structures, including technocrats in the line ministries, districts, and sub￾counties, that offer the political and technical support needed for implementing agencies to engage communities, so they participate in WASH interventions. iii. Political will from district leadership that enabled USHA to implement interventions without interference. Information from USHA indicated that it takes concerted efforts to secure and sustain political will but once it is secured, interventions tend to be owned by the districts and this eventually promotes sustainability. iv. Celebrating sector days also help people to better understand WASH issues. Sector days include Water Week, Sanitation Week, World Water Day, Toilet Day, Menstrual Hygiene Day, Global Handwashing Day, and Day of the African Child. Factors that enabled the CLTS and CLTS+ approaches to work i. The approaches are low cost in terms of finance and human resources and can be easily adopted by local governments. The approaches encourage community collaboration and engagement. ii. The approaches empower the community to own and improve their common sanitation problem (open defecation) and design their own solutions which leads to ownership of interventions. iii. The inclusiveness of the approaches, as they target children, men, women and youth. Engaging men is key to the success of the approaches because cultural norms mean it is usually their role to construct family latrine; women and their children are often not in a position to make the final decisions on the construction of a toilet. However, cultural norms mean maintaining the cleanliness of the toilet is often the responsibility of women and children. Factors that enabled MBSIA to work i. The engagement of local governments, finance providers and local service providers to create the needed consensus. ii. The willingness and commitment of masons and sanitation promoters to participate in USHA capacity building sessions. iii. The design of the rural based outreach approaches that targeted households directly to reinforce awareness creation and sustain behavior change. iv. USHA’s significant capacity to implement the approaches. This capacity included competent human resources, access to technology including real time data collection systems, and finances. v. Establishing and rewarding local community actors, such as sanitation and hygiene promoters, and equipping them with sanitation product information to make it easy to explain the products and encourage households to adopt them. vi. The collaboration among a wide range of district stakeholders. vii. The strong Local Council Committees that enforced public health laws and facilitated attainment of improved and basic sanitation facilities. 37 viii. The performance-based rewards for implementing partners that encouraged them to meet targets and achieve the required results. 3.8 Challenges for Implementing Sanitation and Hygiene, and Rural Water Supply Approaches Through KIIs and the literature review the evaluation found a number of challenges in implementing WASH interventions. These challenges vary depending on the goal and type of intervention. 3.8.1 Challenges for Sanitation and Hygiene Approaches Challenges from the Operating Context i. MOH does not provide sufficient financial resources to districts, especially for preventive interventions like sanitation and hygiene. Primary Health Care funds, managed by those in charge of health facilities, were reported to be limited. Sanitation is never given the priority it deserves and therefore the population does not appreciate preventive hygiene practices. ii. Sanitation has been politicized. Politicians claim credit for most government sanitation projects to help them build political capital. iii. District staff who are in acting positions do not have the powers or sufficient time to make tangible contributions in designing and implementation of activities since they do not control budget votes. iv. Rural sub-counties have limited local revenue to implement WASH activities. v. There are no management models in Uganda for public toilets in public places such as markets, bus stations, and taxi parks. Challenges when using CLTS and CLTS+ i. Attitudes and cultural norms around sanitation. For instance, in some cultures, pregnant women are prohibited from using latrines and household heads should not share a latrine with their in-laws. In these cases, people look for alternatives and often OD is the quickest option. ii. The poor soil conditions in some areas (e.g., water-logged, sandy or rocky) cause toilets to collapse, sometimes despite construction innovations. Challenges when using MBSIA i. Sanitation financing does not favor the rural poor population. The stringent and bureaucratic requirements to access finances and the fact financial institutions do not reach rural areas mean the rural poor have limited access to financing. ii. Customers are unable to afford sanitation products due to their lack of liquidity as a result of unstable, seasonal cash flows. iii. Market-based approaches are mainly externally driven and require sizable investments that limits them from being brought to scale. iv. The limited collaboration in sanitation financing by the large players in the banking sector, including MFIs and savings and credit cooperative organizations. Information about sanitation financing is not comprehensively disseminated to households. Challenges when using the WASH Friendly Schools approach i. There are no streamlined O&M systems for school WASH facilities like water points and toilets, and handwashing and menstrual hygiene facilities. ii. There is limited prioritization and allocation of resources for WASH in schools at national and district levels. 38 3.8.2 Challenges for Rural Water Supply Challenges in District-led Service Provision i. Water system development challenges lead to lower quality systems that break down easily and regularly. These challenges include the poor-quality design, construction, and installation of infrastructure and low quality or inferior construction materials. ii. Low water quality and limited water quantity resulting from poor water resource management practices render some water sources unusable. Water quantity is supposed to be regulated through abstraction permits but very few developers apply for a permit. Water quality monitoring is supposed to be undertaken by a number of stakeholders, including developers and Water Management Zones, but due to limited funding it is only carried out irregularly. iii. Asset management arrangements and utility performance management information systems do not include rural water facilities as there is no central inventory for these facilities. Information about rural facilities is scattered across various documents kept by different stakeholders who use varying data asset management methodologies. This dispersal and variation make it difficult to benchmark asset management or promote shared learning among the stakeholders. Challenges when using the Community-Based Maintenance System model i. The model has inherent challenges such as weak management structures, unclear lines of responsibility, a declining spirit of volunteerism that leads to lack of participation, and high turnover of WUC members. ii. There are financial and social challenges due to irregular contributions towards O&M, varying community contributions in same the geographical area, misuse of funds, lack of accountability, mistrust between the WUC and the users, absence of financial institutions within close vicinity to communities for safe custody of O&M funds, and political interference. iii. There is poor planning for O&M. Many communities only collect money once a repair is needed leading to a long down time. iv. There is a lack of technical capacity to manage complex water systems, especially motorized systems such as the electro-mechanical components of solar powered systems. v. The reach of the spare parts supply chain is limited, and the quality of the spare parts available is reportedly poor. Spare parts are only available in major towns which delays access and subsequently leads to a long down time of non-functioning water facilities. Challenges when using the Public Rural Utility Approach i. There is limited capacity for the design, development, and management of water systems. This includes limited capacity to establish viable management models for different types of water systems. ii. Limited application of the approach so far. The approach has only been applied in piped schemes in small towns and rural growth centers and has not been used for rural water supply systems, such as point water sources. iii. Some of the systems operated by the water authority have major technical challenges resulting in high levels of non-revenue water and costly O&M. 3.9 Cost of Implementing Sanitation and Hygiene, and Rural Water Supply Approaches 3.9.1 Cost of Implementing Sanitation and Hygiene Approaches Cost information for the implementation of the different sanitation approaches was not readily available from the sources contacted or read during this evaluation. However, the 39 evaluation has estimated the cost for a few services based on the available information in the literature; these are shown in Table 8. According to the MOH ODF Roadmap, 2019, implementing the direct activities of the proposed Roadmap will cost UGX 320.5 billion (USD 89,431,600). This costing is based on using CLTS as the main approach in rural areas. This cost is based on making Uganda ODF. The evaluation was informed that USHA will document the costs of its MBSIA and CLTS+ approaches in FY 2022. Table 8: Unit Costs for Sanitation Service Levels Sanitation Service Cost per person (UGX) Cost per person (USD) Improved sanitation 20,000 5.7 Safely managed sanitation 190,000 53.8 Handwashing at home 20,000 5.7 Handwashing at school 20,000 5.7 Source: Strategic investment plan for the water and environment sector, MWE (2018). USD rates based on OANDA Currency Converter as of 01/30/2022 Cost of WASH in Schools The evaluation did not obtain information from districts on the cost of approaches such as WASH Friendly Schools, SBCC, and school competitions. However, the National Micro Planning Handbook for WASH in Public Primary and Secondary Schools in Uganda (2019) provides unit costs for WASH in schools. These costs, shown in Table 9, give the national average financial figures for both hardware and software items. The unit costs per school relate to the cost of closing WASH gaps. The handbook indicates that these unit costs are likely to vary from one school, district, and region to another, depending on the needs of that area. For instance, the costs of constructing a latrine in a waterlogged area may vary significantly from the cost of constructing a latrine in an arid area. Table 9: Costs of WASH in Schools Item Unit cost per school (UGX) Unit cost per school (USD) Providing water supply (borehole construction) 38,000,000 10,601 Providing sanitation facilities (five stance latrine block) 36,000,000 10,043 Providing hand washing facilities 1,000,000 278.9 Training school health clubs 1,643,000 458.3 Providing a menstrual hygiene facility (incinerator) 7,000,000 1,952.8 Training teachers & girls on WASH 1,245,000 347 Annual support supervision & monitoring 1,025,800 286 Social and behavior change communication materials (posters) 3,950,000 1,101.9 Source: National Micro Planning Handbook for WASH in Public Primary and Secondary Schools in Uganda (2019). USD rates based on OANDA Currency Converter as of 11/26/2021 40 3.9.2 Cost of Implementing Approaches for Rural Water Supply The cost of implementing the different approaches for rural water supply could not be established through primary data collection. However, using the available literature, the cost per capita for rural water supply is calculated to be UGX 114,295 (USD 32). According to the MWE Water and Environment Sector Performance Report, 2017, the average cost per beneficiary of new water and sanitation schemes is USD 54, with the maximum per capita cost being USD 103. It should be noted that 2017 was the last year the sector reported on per capita costs. Beginning in 2018, reporting was aligned to the SDG 6 monitoring framework. In line with the SDGs, the water service delivery approaches in Uganda aim to improve the water service level in the country. The unit costs for services found in the MWE’s Strategic investment plan for the water and environment sector (2018) are summarized in Table 10 below. Table 10: Water Service Unit Costs Service Unit Cost per unit (UGX) Cost per unit (USD) Village water supply Village 25,136,000 7,122.4 Functional rural water sources Water source 5,546,000 1,571.5 Improved drinking water Person 85,000 24. Safely managed drinking water Person 387,000 109.6 Source: MWE’s Strategic investment plan for the water and environment sector (2018). USD rates based on OANDA Currency Converter as of 01/30/2022 3.10 Opportunities for USAID’s Future Programming 3.10.1 Opportunities in Sanitation Programming Using the identified promising approaches, the evaluation identified the critical adaptive changes required in Uganda to achieve and sustain ODF and eventually move to basic sanitation. These identified opportunities for change are described below. i. Findings indicated USHA worked mainly with communities and government, with only minimal involvement with financial institutions like banks and MFIs. Facilitating and coordinating communities, businesses, financiers, and government to work together as the four major elements of sanitation marketing will make the approach work better because of the interdependence of these elements. One of the lessons from the findings is that facilitating organizations (such as NGOs) need to work to coordinate these interdependent actors. Building the capacity of NGOs to do this is very critical. ii. Extensive sensitization and outreaches were identified by this evaluation as important in the marketing and promotion of sanitation products to rural households. From interactions with districts, it is apparent that the districts will not sustain the sensitization and other follow-up activities due to their limited WASH budget. This means there is an opportunity to design a more cost-effective sensitization and outreach approach that can be implemented by districts with minimum expenditure. iii. There were concerns from USHA implementing grantees that some local leaders do not encourage communities to embrace sanitation and hygiene but rather enforce participation 41 through other means. This means there is an opportunity to better engage local leaders, through discussion forums, so they are able to better influence and encourage their communities to take decisions that promote accelerated attainment of ODF. iv. Evaluation findings indicate the sanitation service chain is not fully developed yet, especially in rural and peri-urban areas and in the treatment and reuse of fecal sludge. Therefore, opportunities exist in strengthening the sanitation service chain through increasing the number of waste disposal containment and treatment sites. v. All districts raised the challenge of limited funding for WASH activities. There is an opportunity to engage MOH to persuade it to allocate a sufficient WASH budget to district WASH departments The current WASH budget (of 10 million UGX [2800 USD] per year) from MOH is given to the District Environmental Health Departments as part of the Primary Health Care grant. vi. The WASH budget is controlled and managed by health facilities and is not easily accessed by District Health Inspectors. Another opportunity is to train districts on how to develop operational sustainability plans for interventions implemented by external projects. vii. Sanitation marketing is an emerging intervention area but requires support and leadership from the government for it to thrive. There is an opportunity to support the government as it finalizes and implements guidelines and strategies for district local governments to adopt. viii. Findings indicate that line ministries (MOH, MWE, and MOES) have parallel reporting lines and use different information systems for data capture, analysis and reporting. Additionally, USHA’s failure to obtain documented statistics about OD in the covered districts points to a gap in data storage and the identification of appropriate and useful sanitation indicators. Therefore, there is an opportunity to strengthen information systems. For example, to improve monitoring and evaluation processes at the district level, district personnel should be trained in data collection (such as how to use digital tools for data capture), analysis, storage, reporting, and use. Strengthening information systems will enhance the quality of data reported by the ministries. ix. Based on the finding that USHA did not implement WASH interventions in health facilities, there is an opportunity for USAID to design WASH programs that go beyond households to include more institutional and public settings such as schools, health care facilities, and markets. xii. USHA focused mainly on ODF and basic sanitation, but the opportunity to attain SDG targets also lies in households attaining safely managed sanitation service level. This looks beyond ODF to the containment, treating and safe disposal of human waste etc. x. The National Micro Planning Handbook for WASH in Public Primary and Secondary Schools in Uganda (2019) noted that only 58 percent of primary schools had access to water. This implies that there is a significant gap in water supply to schools that needs to be addressed. 3.10. 2 Opportunities in Rural Water Supply Programming Based on the direction MWE is taking in its design of rural water interventions, this evaluation identified the following opportunities: 42 i. Under CBMS+, there are opportunities to be involved in the Professional Management Arrangements for non-gazetted rural water supply systems. MWE intends to strengthen district local governments and establish Area Service Providers (ASPs) to improve CBMS and establish of professional management structures. These developments are still in the early stages and there is potential space for different actors to participate. ii. Under the private companies with performance-based contract model, the opportunity lies in supporting the development of a guaranteed maintenance model in the form of ASP and in the supporting the formation of rural utilities covering a district or a cluster of districts. iii. Supporting the public utility model is an opportunity to strengthen Umbrella Authorities’ ability to expand and manage rural piped water schemes, because MWE is phasing out private operators and scaling up provision of rural piped water supply. 4.0 EVALUATION TAKE AWAYS 4.1 Take Aways from the Evaluation of Sanitation and WASH Friendly Schools Approaches The evaluation was able to identify take aways of sanitation, WASH friendly school approaches, and rural water supply. Sanitation Approaches i. Market-based approaches alone may not lead to achievement of ODF and basic sanitation without the approaches being blending with CLTS. Some enforcement is also required for the two approaches. ii. Districts receive limited funding for WASH activities so achieving ODF requires districts to prioritize sanitation in their plans. iii. Total sanitation in the target districts and the country as a whole will be better achieved by integrating community-led and market-based approaches. For example, while CLTS has proved to be very effective in rural areas, it may not effectively address the challenges in the urban areas. iv. Sustaining ODF status and preventing slippage requires constant engagement of communities (through mobilization, follow-up, and monitoring). Follow-up should include strong enforcement of sanitation and hygiene practices in communities and sanctions for non-compliance. v. Commitment on the part of community and political leaders is essential for sustainability of achievements. vi. Participation of the private sector in the sanitation supply chain, such as through producing, supplying, and constructing sanitation facilities, is essential for promoting sanitation and hygiene through market-based approaches, but also for offsetting the limited contribution by government towards sanitation. vii. There are various opportunities for the private sector, particularly in the areas of sanitation product marketing and fecal sludge management. viii. The sustainability of community sanitation and hygiene interventions depends on a well￾developed supply chain, beginning with demand creation and community involvement (especially of early adopters) that motivates others. ix. There needs to be community contributions toward adequate revenue generation for O&M of the facilities established. There were encouraging findings in this evaluation that showed community members who share water sources with schools are willing to pay a monthly contribution of UGX 2000 (USD 0.57) towards O&M costs. 43 WASH-Friendly Schools Approaches i. Head teachers are key drivers in achieving WASH friendliness in schools since they control the funds. If head teachers do not prioritize WASH, schools will not achieve WASH friendly status regardless of how good the mobilization and sensitization of WASH efforts are. ii. Involvement of a number of teachers (including sanitation teachers, school health club patrons, and senior male and female teachers) throughout the WASH in Schools implementation cycle increases the likelihood that the WASH friendly status of a school will be sustained even if one teacher is transferred. Rural Water Supply i. The promising direction of water supply to rural communities lies in the activities implemented by Umbrella Authorities (UAs). 4.2 Take Aways from USHA that Inform Future Gender Integration A summary of USHA’s use of participatory approaches is below. Much of the information comes from FGDs in the visited districts. i. USHA works with mostly women as sanitation promoters and according to USHA progress reports, this has enhanced adoption of sanitation messages. ii. USHA provided targeted training in schools, including for male and female pupils in sanitation clubs. iii. Women were the most targeted group at household level for demand creation, because, according to the market survey conducted by USHA women have a greater impact on the hygiene status in a home and they are better than the sanitation promoters at convincing men to construct a latrine. iv. USHA’s structure, from top management through district focal persons to the community, is inclusive of all genders. v. USHA encouraged all community members (both males and females) to participate in sensitization meetings and usually held them at times that were appropriate for women’s schedules. The materials used at the meeting, including songs, games, and visual aids, were all-inclusive. vi. USHA’s training actively engaged boys in MHM activities at supported schools. These activities included music, drama, and making re-usable sanitary pads. vii. USHA provided appropriate technology that includes: a. Latrine blocks that cater for the privacy of both male and female pupils and pupils with disabilities. b. Incinerators for disposing of used sanitary pads. c. The DIY approach to upgrading household sanitation facilities that encouraged men and women to work together to construct improved facilities. 5.0 CONCLUSIONS i. Based on the findings from interviews, literature, and USHA progress reports, the evaluation concludes that the identified sanitation approaches (CLTS, CLTS+, MBSIA, WASH Friendly Schools, and National Campaigns) implemented in Uganda and in other countries are all context specific and their success of in attainment of ODF status therefore depends on the context where each approach works well. However, the MBSIA and other market-based approaches have an edge over other approaches since they go beyond attainment of ODF to the sustainability of ODF and moving towards basic sanitation. MBSIA and other market￾based approaches reduce the possibility of a community relapsing to OD, as it involves 44 developing the sanitation and hygiene supply chain and leveraging the private sector. MBSIA is a promising implementation approach to obtaining ODF status and moving towards basic sanitation. ii. The evaluation concludes that USHA’s blending of CLTS with other approaches to design CLTS+ was a good innovation. CLTS+ is a promising approach to achieve and sustain ODF and will gradually lead communities to moving to basic sanitation. iii. Based on interviews with national level KIs and a review of ministerial documentation, this evaluation concludes that the rural water supply in Uganda is undergoing reform. However, it is concluded that CBMS+ is a promising approach towards professionalizing water supply services. CBMS+ will involve strengthening districts’ capacity, involving the private sector, and providing a conducive environment for the rural water authorities to improve. 6.0 RECOMMENDATIONS FOR USAID FUTURE PROGRAMMING i. Based on the results of the USHA market-based approach to attaining ODF status and eventual moving to basic sanitation, this evaluation recommends USAID scales up and replicates the approach in sub-counties and districts not originally covered by USHA. ii. USAID should commission a comprehensive cost analysis study to get an in-depth understanding of the cost of specific approaches for sanitation and water supply. None of the USHA grantees or districts contacted by the evaluation team had readily available documented costs of WASH activities. iii. Although information from USHA indicated that thinly spread interventions built more capacity at district local governments than concentrated interventions, this evaluation recommends that for future programming, WASH interventions are concentrated in one district as opposed to covering many districts and selecting only a few sub-counties. The current approach used by USHA gave inconsistent results across a district with some areas being ODF while still having OD in other areas. Phased, concentrated interventions would leave a more consistent impact across a district than thinly spread interventions. iv. WASH programs should be comprehensive enough to focus beyond households and selected institutions like schools and health care facilities, to also include other public places such as markets. v. Information systems should be strengthened through monitoring and evaluation processes and training district personnel in data collection, analysis, storage, use, and reporting, especially on sanitation activities. vi. Marketing campaigns and promotion should be continued to nurture positive attitudes, values, and mindsets towards sanitation and hygiene. This is a fundamental element towards achieving the SDG targets. It also aligns with the mission of the MOH ODF Roadmap, 2019, that is to inculcate a mindset that cherishes environmental health through appropriate disposal of human excreta throughout the country. It is apparent that districts will not continue the sensitization and other follow-up activities themselves due to their limited WASH budget. vii. Future USAID programs need to focus on holistic interventions with greater attention on activities that build the practical skills of teachers and pupils that lead to improvements in 45 hygiene practices, such making liquid soap, making bags for MHM supplies, and making sanitary pads. 46 7.0 ANNEXES Annex 1: Detailed Evaluation Questions 1. What are the promising implementation approaches to obtain open defecation free status and move towards basic sanitation for communities in developing countries with a similar context? a. What are the enabling factors, bottlenecks, opportunities, and recommendations for the approaches identified? b. What are the contributions of market-based solutions to the achievement of ODF conversions and movement to basic sanitation? c. What are the enabling factors, bottlenecks, opportunities, and recommendations regarding the MBSIA approach? d. What are the innovative approaches for School WASH? What are the enabling factors, bottlenecks, opportunities, and recommendations regarding the identified approaches? e. What are the costs for implementation of the successful approaches identified? 2. What are the innovative approaches to improve access to water for rural communities? a. What are the enabling factors, bottlenecks, opportunities, recommendations regarding the water approaches? b. What are the successful innovative operation and maintenance systems? c. What are the costs of the innovations identified? 3. What opportunities exist for WASH systems strengthening (Governance and leadership, information systems, supply chain, civil society participation, financing and accountability, infrastructure operations and management and regulation)? a. What are the systemic bottlenecks hindering access to WASH services? b. What are the opportunities to strengthen WASH systems to improve access to WASH services? c. What are the successful sanitation financing models at the different levels of big enterprises, small/medium enterprises, and households in Uganda? d. What are the bottlenecks and recommendations to extend access for WASH financing? 4. What are the opportunities and lessons for gender integration within the WASH programming in Uganda? a. What are the lessons learnt from USHA that can inform gender integration and enhanced participation of women, men, boys, girls, and other marginalized groups in WASH programming? b. What are the existing WASH related gender gaps/disparities/inequalities affecting women, men, boys, and girls and other marginalized populations (including Persons with Disabilities, humanitarian and low-income communities/individuals)? c. What are the gender specific approaches/interventions that have been implemented to address the gender related gaps in regard to social mobilization, service delivery, capacity building and empowerment? 47 Annex 2: District Performance in the WASH Sector Region District Rural water supply Sanitation coverage Number of ODF villages Access Functionality Northern Uganda Gulu 93% 78% 79% 8 Agago 95% 77% 66.5% No data Kitgum 95% 60% 60.5% 42 Lamwo 95% 79% 59% No data Omoro 91% 43% 81% No data Pader 95% 78% 69% 8 Nwoya 65% 78% 67% 11 Central East Buikwe 77% 92% 75% No data Kaliro 50% 95% 72.9% 47 Jinja, 77% 85% 82% 0 Buyende 37% 92% 86.3 2 Namutumb a 61% 88% 78.8% No data Kayunga 69% 87% 73.8% No data Luuka 80% 96% 69.3% 4 Central West Mpigi 83% 72% 66% 3 Kyotera, 63% 67% 78% 16 Lwengo 75% 79% 65% 7 Bukomansi mbi 85% 87% 63.3% No data Gomba 86% 62% 53% No data Sembabule 38% 85% 75% 4 Masaka 78% 81% 84% 4 Source: Ministry of Water and Environment Sector Performance Report, 2020 48 Annex 3: Rationale for Selecting Districts, Countries, and Organizations Selected Districts Performance in Sanitation Coverage Performance in Rural water Supply ODF Villages Northern Uganda Omoro High No data Pader Low High High Nwoya Low Low High Central East Kaliro Low Jinja High No data Kayunga High High No data Central West Kyotera High Low Low Bukomansimbi Low High No data Gomba Low High No data Selected Countries Rationale Africa Kenya For its similarity in context to Uganda and progress in basic sanitation coverage. Kenya has an elaborate focus on sanitation driven by The National ODF Kenya 2020 Campaign Action Framework which envisaged massive sanitation upgrade across the country as a result of CLTS effort and advocacy for development of sanitation market and supply chains3 Ghana Since the early 1990s, Ghana’s water and sanitation sector has seen major reforms to address weaknesses. Appropriate institutional, legal, and regulatory structures are now largely in place, particularly for the urban and rural water supply subsectors4. Asia Philippines For the ‘good progress toward’ its Sustainable Development Goal targets for water supply and sanitation, and for WASH in Schools that is an emerging priority5. Selected Organizations Rationale GIZ For its innovations related to successful programming and implementation of WASH in schools. 3 Republic of Kenya. (2016). National ODF Kenya 2020 Campaign Framework. Nairobi: Ministry of Health Kenya 4 Water Supply and Sanitation in Ghana. Turning Finance into Services for 2015 and Beyond. African Ministers’ Council on Water (AMCOW) Country Status Overview 5 . The Impact of a School-Based Water, Sanitation and Hygiene Intervention on Knowledge, Practices, and Diarrhoea Rates in the Philippines. Int J Environ Res Public Health. 2019 Nov; 16(21): 4056. Published online 2019 Oct 23. doi: 10.3390/ijerph16214056 49 Water for People For its market-based water and sanitation approaches International Water and Sanitation Center (IRC) For water and sanitation systems strengthening. At national level, IRC shares lessons learned and advocates for increased government and political leadership, public finance and the proper management and use of WASH services as part of the Uganda WASH Agenda for Change coalition. 50 Annex 4: Districts Selected Region USHA Implementation Districts Selected Districts Northern Uganda Gulu, Agago, Kitgum, Lamwo, Omoro, Pader and Nwoya Omoro (visited) Pader Nwoya Central East Buikwe, Kaliro, Jinja, Buyende, Namutumba, Kayunga and Luuka. Kaliro (visited) Jinja Kayunga Central West Mpigi, Kyotera, Lwengo, Bukomansimbi, Gomba Sembabule and Masaka Bukomansimbi Gomba Kyotera (visited) 51 Annex 5: List of Key Informants 52 Annex 6: Focus Group Discussions Conducted No. District Location OD or ODF Participants 1 Omoro Ocim village OD Village Members 2 Omoro Lalyaba village ODF Village members 3 Omoro Lalogi sub￾county Redacted 4 Omoro Lalogi sub￾county Redacted 5 Kaliro Bukerehe village ODF Village Members 6 Kaliro Namwiwa sub￾county Redacted 7 Kaliro Namwiwa sub￾county Redacted 8 Kyotera Kabaseke A village ODF Village Members 9 Kyotera Kasaali sub￾county Redacted 10 Kyotera Kirumba sub￾county Redacted 53 Annex 7: USHA Evaluation SOW USAID/UGANDA OFFICE OF HEALTH AND HIV STATEMENT OF WORK FOR EVALUATION OF POST USHA ACTIVITY A. INTRODUCTION Based on the FY20 Annual Water and Environment sector performance report, access to Water, Sanitation and Hygiene (WASH) services is still very poor. The country is lagging behind the Sustainable Development Goal of universal WASH access, with only 34% of the population having access to basic sanitation and up to 20% still practicing open defecation, compared to 69% with basic water supply and 49% of the population practicing basic hand hygiene. The USAID/Uganda Sanitation for Health Activity (USHA), a five-year activity with a TEC of 32 million US Dollars was designed to implement interventions that increase household access to sanitation and water services, improve hygiene behaviors at home, school and health facilities and strengthen district water and sanitation governance systems. The Activity was designed and implemented in 20 Districts in Uganda (7 in Northern Uganda, 6 in Central West, 7 in Central East). The Activity focuses on stopping open defecation and increasing sustainable access to improved sanitation, as defined under the World Health Organization (WHO)/United Nations Children’s Emergency Fund (UNICEF) Joint Monitoring Program for Water and Sanitation (JMP), with the objective of reducing the incidence of diarrhea and other disease related to poor sanitation while contributing to nutritional goals. In Uganda, Community Led Total Sanitation (CLTS) is central to the GOU sanitation approach. CLTS is a community-based approach mainly used in rural communities to achieve behavior change which results in complete elimination of open defecation. The USHA Activity complements CLTS in Northern Uganda with a novel Market Based Sanitation Implementation Approach (MBSIA) in the Central East and Central West that seeks to catalyze the market for rural sanitation and hygiene products and services to reach more households with affordable and quality sanitation facilities than ever before. The Activity also emphasizes strengthening local governance in the context of decentralization. USHA engages and supports local governance structures (Including District Water and Sanitation Department) at all levels during planning, implementation processes and monitoring of implemented activities. The Activity also builds on existing Health information systems to strengthen monitoring of WASH activities across the Districts. USHA’s targeted implementation was designed to complement and expand geographic coverage of existing rural sanitation investments in the WASH sector, particularly those of the Uganda Sanitation Fund. The activities of USHA were linked to and complemented by other USAID activities designed and implemented to support upstream activities including improving health systems, coordination, policy development and advocacy related activities. These include, RHITES, Uganda Health Systems Strengthening (UHSS) activity, Strategic Information Technical Support (SITES) activity, the Family Planning Activity (FPA), Social Behavior Change Activity (SBCA) and the Maternal Child Health and Nutrition (MCHN) 54 activity. Other recently concluded activities complimented by USHA include Uganda Health Supply Chain (UHSC) and Communication for Healthy Communities, Integrated Community Agriculture and Nutrition (ICAN). RATIONALE USAID is committed to investing in WASH activities that will achieve extensive and tangible impacts on Ugandans’ health, education, and nutritional conditions. USAID is designing WASH programs more strategically to intersect with country needs, existing initiatives for advancing water provision and improving sanitation-seeking behaviors. USAID WASH programs will maximize positive benefits accrued through novel implementation approaches, harness demand-led innovative methods to model approaches that will maintain positive sustainable change in the sanitation landscape. This will be accomplished through more thoughtful analysis of existing evidence to optimize WASH programming (both within USAID and externally); rooting approaches with host country partner governments’ development priorities and the needs and wishes of beneficiaries. USAID WASH programs will undertake rigorous analysis of existing evidence to implement relevant approaches, technologies that are scalable and sustainable in the Ugandan context. This evaluation is intended to understand how USHA and other programs have contributed to sanitation improvement, identify key lessons, and explore potential alternative implementation approaches and areas of improvement to inform programs in future. B. PURPOSE OF THE EVALUATION The purpose of the evaluation is to generate insights into how the approaches used by the USHA and other WASH programs in similar contexts in and out of Uganda, have contributed to stopping and sustaining open defecation and increasing basic sanitation services in districts. The evaluation shall focus on geographic areas of USHA and other relevant program implementation areas. The evaluation shall seek to understand the efficiency, success, and cost of WASH intervention models as well as the enablers and constraints that affect scalability. The findings of the review will be used by the Government of Uganda to inform future USAID-funded WASH programming. C. AUDIENCE The findings of the evaluation will be used by the Ugandan Ministries of Health, and Ministry of Water and Environment (MWE), the United States government agencies, USAID funded implementing partners (IPs), and other national and international stakeholders with interest in implementing integrated WASH health programs. D. EVALUATION QUESTIONS Specific questions to be addressed by the evaluation are: 1. What are the promising implementation approaches to obtain open defecation free status and move towards basic sanitation for communities in developing countries with a similar context? a. What are the enabling factors, bottlenecks, opportunities, and recommendations for the approaches identified? 55 b. What are the contributions of market-based solutions to the achievement of ODF conversions and movement to basic sanitation? What are the enabling factors, bottlenecks, opportunities, and recommendations regarding the MBSIA approach? c. What are the innovative approaches for WASH friendly schools? What are the enabling factors, bottlenecks, opportunities, and recommendations regarding the identified approaches? d. What are the costs for implementation of the successful approaches identified? 2. What are the innovative approaches to improve access to water for rural communities? a. What are the enabling factors, bottlenecks, opportunities, recommendations regarding the water approaches? b. What are the successful innovative operation and maintenance systems? c. What are the costs of the innovations identified? 3. What opportunities exist for WASH systems strengthening (Governance and leadership, information systems, supply chain, civil society participation, financing and accountability, infrastructure operations and management and regulation)? a. What are the systemic bottlenecks hindering access to WASH services? b. What are the opportunities to strengthen WASH systems to improve access to WASH services? c. What are the successful sanitation financing models at the different levels of big enterprises, small/medium enterprises, and households in Uganda? What are the bottlenecks and recommendations to extend access for WASH financing? 4. What are the opportunities and lessons for gender integration within the WASH programming in Uganda? a. What are the lessons learnt from USHA that can inform gender integration and enhanced participation of women, men, boys, girls, and other marginalized groups in WASH programming? b. What are the existing WASH related gender gaps/disparities/inequalities affecting women, men, boys, girls and other marginalized populations (including Persons with Disabilities, humanitarian and low-income communities/individuals)? c. What are the gender specific approaches/interventions that have been implemented to address the gender related gaps in regard to social mobilization, service delivery, capacity building and empowerment? E. METHODOLOGY The evaluation will collect data from existing resources using the following methods: ● Internal Desk Research – all related reports, special studies and performance data will be provided. This includes quarterly and annual reports, special studies, implementing partner meeting reports, learning reviews, past evaluations among others. ● External Desk Research – this will include a review of relevant reports and documents outside USAID/Uganda. ● Key Stakeholder Consultations - this will include key informant interviews to collect information from a wide range of people—including community leaders, USHA grantees? District (CAO, USHA Focal Persons, DHI) & Sub county Staff (HA), Ministry Staff (MoH, MWE) professionals, Stakeholders who have firsthand knowledge about WASH programming and targeted beneficiaries. F. DELIVERABLES SCHEDULE 56 Description Date 1 Inception report showing the desk review design, a detailed plan with timelines. October 7, 2021 2 Oral Presentation to USAID: major findings, conclusions, lessons learned, recommendations and relevant annexes. The input from the oral presentation sessions should also be incorporated in the report November 22, 2021 3 First Draft Report: The content should cover all the main elements of the report including major findings, conclusions, lessons learned, recommendations and relevant annexes. November 29, 2021 4 Second: A complete report presented in the agreed-upon format and incorporating initial comments from USAID. December 1, 2021 5 Synthesis Executive Briefer: prepare an executive summary that serves as an infographic that communicates report highlights and key messages (with special emphasis on data visualization). December 8, 2021 6 Final Report: The team leader will submit a final report within five days of receiving final comments from USAID. The core report should not be more than 20 pages excluding annexes. December 14, 2021 G. TEAM COMPOSITION The desk review will be conducted by an external evaluation team that may include international and Ugandan team members. The team must include one evaluation team leader, a WASH expert, and a supporting researcher/analyst. Collectively, the team must have: Team Leader 1. M&E skills with 10-15 years’ specific experience in conducting WASH evaluations in Sub￾Saharan Africa in a similar scope 2. Public health programming experience in integrated WASH programing at national and sub￾national levels in sub-Saharan Africa 3. Excellent understanding of the Ugandan DHIS2(HMIS) and population-based surveys e.g., Lot Quality Sampling Assurance (LQAS), DHS 4. Excellent knowledge of Uganda’s health care and Water system 5. Knowledge of Health Systems Strengthening (HSS) and service delivery in a decentralized setting WASH Expert 1. Experience of 5-10 years in WASH programing in Sub-Saharan Africa with a similar scope 2. Familiarity of Ugandan WASH programming landscape 3. Experience with relevant WASH approaches i.e. MBSIA, CLTS, and WASH governance systems 4. Excellent knowledge of Uganda’s health care and Water system 5. Knowledge of Health Systems Strengthening (HSS) and service delivery in a decentralized setting 57 6. Good understanding of the Ugandan DHIS2(HMIS) and population-based surveys e.g., Lot Quality Sampling Assurance (LQAS), DHS Researcher/Analyst 1. Experience of 5-10 years in research within the health sector, namely WASH programing in Sub-Saharan Africa with a similar scope; experience with gender in WASH also preferred. 2. Strong contextual research and writing skills with an ability to thematically analyze data, especially regarding background literature, literature reviews, and comparative analyses of WASH programs/methods/approaches 3. Strong data visualization skills and presentation skills to support the compilation/copyediting of final deliverables 58 Annex 8: References 1. Ministry of Water and Environment, Water and Environment Sector Performance Reports: 2018, 2019 and 2020 2. Assessing the Affordability of Water, Sanitation and Hygiene (WASH), UNICEF and WHO, 2021 3. National Framework for Operation and Maintenance of Rural Water Infrastructure in Uganda, 2020 4. Republic of Uganda, Third National Development Plan (NDP III) 2020/21-2024/25, June 2020. 5. Republic of Uganda, MoH and UNICEF, (2019): Roadmap for Elimination of Open Defecation and Acceleration of Basic Sanitation in Uganda 2020/21 - 2024/25. 6. Republic of Uganda MoES, (2017): Guidelines for Three Star Approach for Planning and Implementation of WASH in Schools. 7. Republic of Uganda MoES, (2019): National Micro Planning Handbook for Water, Sanitation and Hygiene (WASH) in Public Primary and Secondary Schools in Uganda. 8. Ficek F. and Novotny J., (2018). Comprehending practitioners’ assessments of community-led total sanitation. Health Promotion International journal, Vol. 34 (6) e129–e138, 9. Fostvedt-Mills C., Gebrekidan B., Msuya J., Suten Mwabulambo S., Sigareti E. A., and Sironka K. D. (2018). Pastoralists: East and Southern Africa region. Sanitation and Hygiene hunter gatherer thematic note: www.communityledtotalsanitation.org 10. GoU (2017). 2018 - 2030 Integrated Sanitation and Hygiene Financing Strategy. African Ministers’ Council on Water. 8 December 2017 11. Ministry of Health (2019). Roadmap for elimination of open defecation and acceleration of basic sanitation in Uganda 2020/21 - 2024/25 ODF road map, Government of Uganda 12. MoES, (2019) National Micro Planning Handbook for Water, Sanitation and Hygiene 13. MoES. (2016) WASH in Schools Mapping Report. Kampala, Uganda 14. MWE (2018). Strategic investment plan for the water and environment sector, Uganda (2018-2030). Vol (1). Funding requirements to meet national 2030 targets for the water and environment sector. Kampala Uganda 15. MWE (2020). National framework for operation and maintenance of rural water infrastructure in Uganda. Directorate of Water Development, Rural Water department. Kampala Uganda 16. MWE, (2019) Annual Joint Water and Environment Sector performance report. Kampala Uganda 17. MWE, (2020) Annual Joint Water and environment Sector performance report. Kampala Uganda 18. Sector Performance Report (SPR), (2017). Joint Water and environment Sector performance report. Kampala Uganda. 19. MWE, Water and Sanitation Gender Strategy (2018-2022) 20. Sophie H., (2019). Tackling slippage. Issue 14, September 2019 Frontiers of CLTS: Innovations and Insights. CLTS Knowledge Hub at IDS www.communityledtotalsanitation.org 21. UNICEF (2020). Guidance on market-based sanitation, UNICEF, New York. 22. Stuart K, Peletz R, Albert J, Khush R, Delaire C (2021). Where Does CLTS Work Best? Quantifying Predictors of CLTS Performance in Four Countries: Cambodia, Ghana, Liberia, and Zambia. 23. Oppong TB, Yang H, Amponsem-Boateng C, Duan G. (2019). Hand Hygiene Habits of Ghanaian Youths in Accra, Ghana. 24. Crocker J and Saywell D and Bartram J (2017). Sustainability of community-led total sanitation outcomes: Evidence from Ethiopia and Ghana. 25. Crocker J, Saywell D, Shields KF, Kolsky P, Bartram J. (2017). The true costs of participatory sanitation: Evidence from community-led total sanitation studies in Ghana and Ethiopia. 26. Crocker J and Fuente D and Bartram J. (2021). Cost effectiveness of community led total sanitation in Ethiopia and Ghana. 27. Appiah-Brempong E, Harris MJ, Newton S, Gulis G. (2018). Examining school-based hygiene facilities: a quantitative assessment in a Ghanaian municipality. 28. Lissah SY, Ayanore MA, Krugu JK, Aberese-Ako M, Ruiter RAC. (2021). Managing urban solid waste in Ghana: Perspectives and experiences of municipal waste company managers and supervisors in an urban municipality. 29. Kumbeni MT, Otupiri E, Ziba FA. (2020). Menstrual hygiene among adolescent girls in junior high schools in rural northern Ghana 30. Nunbogu AM, Harter M, Mosler HJ. (2019). Factors Associated with Levels of Latrine Completion and Consequent Latrine Use in Northern Ghana. 31. Osumanu IK and Kosoe EA and Ategeeng F (2019). Determinants of Open Defecation in the Wa Municipality of Ghana: Empirical Findings Highlighting Sociocultural and Economic Dynamics among Households. 59 32. Labi A, Obeng-Nkrumah N, Nuertey BD, Issahaku S, Ndiaye NF, Baffoe P, Duncan D, Wobil P, Enweronu-Laryea C. (2019). Hand hygiene practices and perceptions among healthcare workers in Ghana: A WASH intervention study. 33. Njuguna J. (2019). Progress in sanitation among poor households in Kenya: evidence from demographic and health surveys. 34. Korir E, Okwara FN, Okumbe G. (2018). Menstrual hygiene management practices among primary school girls from a pastoralist community in Kenya: a cross sectional survey. 35. Republic of Kenya, Ministry of Health. National ODF Kenya 2020 Campaign Framework (2016/17- 2019/20). 36. Philippines Country Report (2016): Private Sector Provision of Water and Sanitation Services in Rural Areas and Small Towns: The Role of the Public Sector. 37. Vally H, McMichael C, Doherty C, Li X, Guevarra G, Tobias P. (2019). The impact of a school-based water, sanitation and hygiene intervention on knowledge, practices, and diarrhoea rates in the Philippines. International journal of environmental research and public health. Jan;16(21):4056. 38. Robinson A. (2003. Management models for small towns water supply: lessons learned from case studies in the Philippines (https://www.ircwash.org/resources/management-models-small-towns-water-supply￾lessons-learned-case-studies-philippines ).