MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION AND HYGIENE (KIWASH) PROJECT MARCH 2019 This publication was produced for review by the United States Agency for International Development. It was prepared by Tetra Tech. ACKNOWLEDGEMENTS The mid-term evaluation was conducted by Ken Caplan (Team Leader, Independent), Jonathan Annis (WASH Specialist, Tetra Tech), Charles Oyaya (WASH Governance Specialist, Independent), Evelyn Makena (WASH and Nutrition Specialist, Independent) and Mary Gichuhi (Gender Specialist and Logistics Coordinator, Independent). Each team member made meaningful contributions to this report. The evaluation team would like to acknowledge the invaluable contribution from Siobhan Girling (Tetra Tech) in supporting data collection in one sub-county, Clinton Sears (Tetra Tech) for providing training and guidance in using the Dedoose platform, Elizabeth Kruger (Tetra Tech) for copy editing and Morris Israel (Tetra Tech) and for peer reviewing the final draft report. We extend a sincere thanks to KIWASH staff at national and county levels for their close collaboration and cooperation in the planning and execution of the review. Without their assistance and flexibility, the evaluation team would not have been able to complete the review in the timeframe allotted. The team would also like to thank the numerous key informants at the national, county and community levels who participated openly and thoughtfully in the interviews and focus group discussions, and to USAID/Kenya and East Africa for providing oversight and guidance throughout. Preferred citation: USAID. (2019) Mid-Term Evaluation of the Kenya Integrated Water, Sanitation and Hygiene (KIWASH) Project. Washington, DC., USAID Water, Sanitation, and Hygiene Partnerships and Learning for Sustainability (WASHPaLS) Project. Prepared for the United States Agency for International Development by the Water, Sanitation, and Hygiene Partnerships and Learning for Sustainability (WASHPaLS) project under Task Order number AID-OAA-TO-16-00016 of the Water and Development Indefinite Delivery Quantity Contract (WADI), contract number AID-OAA-I-14-00068. Tetra Tech Contacts: Morris Israel, Project Director morris.israel@washpals.com Jeff Albert, Deputy Project Director jeff.albert@washpals.org Jonathan Annis, Project Manager jonathan.annis@tetratech.com Tetra Tech 1320 N. Courthouse Road, Suite 600, Arlington VA 22201 Tel: (703) 387-2100, Fax: (703) 414-5593 www.tetratech.com/intdev Did you find this report useful? We’d love to hear from you! Please provide feedback/comments here: https://goo.gl/forms/1fSiL634Rqu57r2I2 MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION AND HYGIENE (KIWASH) PROJECT MARCH 2019 DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT i TABLE OF CONTENTS TABLE OF CONTENTS..............................................................................................................................I LIST OF TABLES ........................................................................................................................................II LIST OF FIGURES.......................................................................................................................................II ACRONYMS AND ABBREVIATIONS.................................................................................................... III EXECUTIVE SUMMARY ........................................................................................................................... V 1.0 EVALUATION PURPOSE AND EVALUATION QUESTIONS ..................................................... 1 1.1 PURPOSE .................................................................................................................................................................................1 1.2 AUDIENCE AND INTENDED USES ........................................................................................................................................1 1.3 EVALUATION QUESTIONS ....................................................................................................................................................1 2.0 PROJECT BACKGROUND ................................................................................................................ 2 2.1 DEVELOPMENT HYPOTHESIS ................................................................................................................................................2 3.0 EVALUATION METHODS AND LIMITATIONS ............................................................................ 4 3.1 SECONDARY DATA COLLECTION – DESK REVIEW ..........................................................................................................5 3.2 SELECTION OF KIWASH COUNTIES FOR PRIMARY DATA COLLECTION .....................................................................6 3.3 PRIMARY DATA COLLECTION..............................................................................................................................................7 3.3.1 Key Informant Interviews....................................................................................................................................7 3.3.2 Focus Group Discussions....................................................................................................................................8 3.3.3 KII and FGD Guides..............................................................................................................................................8 3.4 SITE OBSERVATIONS..............................................................................................................................................................9 3.5 DATA ANALYSIS METHODS..................................................................................................................................................9 3.6 CAVEATS AND LIMITATIONS ................................................................................................................................................9 4.0 FINDINGS AND CONCLUSIONS .................................................................................................. 11 4.1 EQ1: HOW APPROPRIATE IS THE KIWASH ACTIVITY DESIGN OF MARKET-BASED APPROACHES IN EXPANDING SUSTAINABLE ACCESS TO WATER AND SANITATION SERVICES IN THE TARGETED COUNTIES?..............................11 4.1.1 The Kenyan Water Service Delivery Context and the KIWASH Design.............................................11 4.1.2 The Kenyan Sanitation Service Delivery Context and the KIWASH Design.......................................15 4.1.3 Conclusions – EQ1..............................................................................................................................................19 4.2 EQ2: HOW APPROPRIATE IS THE KIWASH ACTIVITY IMPLEMENTATION STRATEGY TO THE SUCCESS OF THE MARKET-BASED APPROACHES IN EXPANDING ACCESS TO WATER AND SANITATION SERVICES? .........................21 4.2.1 KIWASH as a Delivery Agent ..........................................................................................................................22 4.2.2 Perceptions of KIWASH as a Program..........................................................................................................23 4.2.3 KIWASH’s Approach to Water Supply Services.........................................................................................25 4.2.4 KIWASH’s Approach to Sanitation Services................................................................................................31 4.2.5 Conclusions – EQ2..............................................................................................................................................33 4.3 EQ3: TO WHAT EXTENT HAVE THE DIFFERENT KIWASH PRIVATE SECTOR APPROACHES FOR WATER SERVICE DELIVERY AND SANITATION ACCESS EXPANSION BEEN SUCCESSFUL? WHICH OF THE KIWASH IMPLEMENTATION APPROACHES FOR EXPANDING SUSTAINABLE WATER SERVICES DELIVERY AND SANITATION ACCESS APPEAR TO BE THE MOST PROMISING AND SHOULD BE PRIORITIZED IN THE REMAINING TWO YEARS OF THE PROJECT?.......................................................................................................................................................................35 4.3.1 Water Services.....................................................................................................................................................35 4.3.2 Sanitation Services...............................................................................................................................................37 4.4 EQ4: TO WHAT EXTENT HAS KIWASH BEEN SUCCESSFUL IN BUILDING COUNTY-LEVEL CAPACITY TO IMPROVE SUSTAINABLE SERVICE DELIVERY?......................................................................................................................39 4.4.1 Strengthening Local Governance and Management....................................................................................39 4.4.2 Strengthening Monitoring, Evaluation, and Learning...................................................................................41 4.4.3 Advancing Policy Reforms.................................................................................................................................42 4.4.4 Conclusions – EQ4..............................................................................................................................................43 4.5 EQ5: IN THE FIRST THREE YEARS OF KIWASH, TO WHAT DEGREE WERE NUTRITION ACTIVITIES SYNERGISTIC WITH WASH ACTIVITIES?..................................................................................................................................................44 MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT ii 4.5.1 Government and Partners’ Perception of KIWASH’s Work in WASH/Agri-Nutrition Integration 45 4.5.2 Concerns about the Wash/Nutrition Exit Strategy ....................................................................................45 4.5.3 A Framework for Assessing WASH and Nutrition Activities..................................................................45 4.5.4 Co-Location: Beneficiary Targeting.................................................................................................................47 4.5.5 Coordination.........................................................................................................................................................47 4.5.6 Cross Training–Capacity Enhancement for WASH and Nutrition Integration ...................................48 4.5.7 Collaboration: Joint Assessment, Planning, Implementation, and Monitoring .....................................48 4.5.8 Complete Integration .........................................................................................................................................50 4.5.9 Conclusions – EQ5..............................................................................................................................................50 4.6 CROSS-CUTTING: REFLECTION ON GENDER MAINSTREAMING IN KIWASH............................................................50 5.0 RECOMMENDATIONS .................................................................................................................... 53 5.1 SHORT-TERM RECOMMENDATIONS..................................................................................................................................53 5.1.1 General...................................................................................................................................................................53 5.1.2 Water .....................................................................................................................................................................53 5.1.3 Sanitation ...............................................................................................................................................................54 5.1.4 County-Level Support ........................................................................................................................................54 5.1.5 WASH/Nutrition Integration............................................................................................................................55 5.2 LONG-TERM RECOMMENDATIONS (USAID)..................................................................................................................55 ANNEXES.................................................................................................................................................. 56 LIST OF TABLES • Table 3.1: Data Sources, Data Collection and Analysis Methods for Each Evaluation Question……..…….4 • Table 4.1: Rates of Open Defecation and Unimproved Sanitation…………………………………………18 • Table 4.2: WASH-Nutrition Integration – Key Activities Under Each Task……………………………….44 LIST OF FIGURES • Figure 2.1: KIWASH Results Framework……………………………………………………………………3 • Figure 3.1: KIWASH Counties Visited……………………………………………………………………….7 • Figure 4.1: Plotting of Synergy in KIWASH WASH and Nutrition Activities………………………………46 MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT iii ACRONYMS AND ABBREVIATIONS AMEP Activity Monitoring and Evaluation Plan BDS Business Development Services CDCS Country Development and Cooperation Strategy CEC County Executive Committee CHEW Community Health Extension Worker CHV Community Health Volunteer CLTS Community-Led Total Sanitation DMA Delegated Management Area DP Development Partner EQ Evaluation Question ERP Enterprise Resource Planning ESHP EWA Environmental Sanitation and Health Policy Essential WASH Action FGD Focus Group Discussion FSM Fecal Sludge Management FY Fiscal Year GWASCO Gusii Water and Sewerage Company GOK Government of Kenya HCF Healthcare Facility HH Household IP Implementing Partner IR Intermediate Result KEWASNET Kenya Water and Sanitation Civil Society Network KII Key Informant Interview KIWASH Kenya Integrated Water, Sanitation and Hygiene Project KIWASCO Kisumu Water and Sewerage Company M&E Monitoring and Evaluation M2M Mother-to-Mother MBS Market-Based Sanitation MCH Maternal and Child Health MOH Ministry of Health NACS Nutrition Assessment, Counseling and Support NCWSC Nairobi City Water and Sewerage Company NRW Non-Revenue Water OD Open Defecation ODF Open Defecation Free OCA Organizational Capacity Assessments ORT Oral Rehydration Therapy PHO Public Health Officer PNC Post Natal Clinic SBCC Social Behavior Change Communication SDG United Nations Sustainable Development Goal(s) SME Small and Medium Enterprises SOW Statement of Work TOT Training of Trainers USAID United States Agency for International Development USAID/KEA USAID Kenya and East Africa MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT iv USAID-NHP+ USAID National Health Promotion Plus Program WASREB Water Services Regulatory Board WSP Water Service Provider WRMA Water Resources Management Authority WASH Water, Sanitation and Hygiene WASH-FIN Water, Sanitation and Hygiene Finance Project WASHPaLS Water, Sanitation and Hygiene Partnerships and Learning for Sustainability Project WRUA Water Resources Users Associations WSUP Water and Sanitation for the Urban Poor WSTF Water Sector Trust Fund WWF World Wide Fund for Nature MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT v EXECUTIVE SUMMARY PROJECT BACKGROUND The Kenya Integrated Water, Sanitation and Hygiene (KIWASH) Project is a $51 million five-year activity (October 2015 to September 2020) funded by USAID/Kenya and East Africa (USAID/KEA) and implemented by a consortium led by DAI. The goal of KIWASH is to improve lives and health through development and management of sustainable water, sanitation and hygiene (WASH) services in Kenya. Its purpose is to institutionalize catalytic models of sustainable service delivery for accelerated water and sanitation access in target counties and to improve complementary hygiene behaviors. The integrated nature of KIWASH’s outputs are reflected in the diverse range of activities the project is implementing, including technical assistance to water utilities and WASH enterprises, Community-Led Total Sanitation (CLTS) and sanitation marketing, and nutrition counseling.1 KIWASH activities are implemented in nine counties in two principal focus areas: Busia, Nyamira, Migori, Siaya, Kisumu and Kakamega counties in the Lake Victoria Focus Area, and Kitui, Makueni, and Nairobi counties in Tana/Athi Focus Area. 1 The contract was amended at the end of FY17 to remove Feed the Future earmark funding, effectively halting all KIWASH activities related to nutrition and increased production and consumption of nutrient-dense foods. EVALUATION PURPOSE, DESIGN AND METHODOLOGY USAID/KEA commissioned this mid-term evaluation at the start of the project’s fourth year. This evaluation’s primary purposes were to: 1) assess the appropriateness of the KIWASH design, 2) assess progress of implementation to date2, and 3) inform future USAID/KEA WASH and nutrition programming. The evaluation team reviewed KIWASH contractual deliverables and other related documents, and conducted Key Informant Interviews (KIIs), Focus Group Discussions (FGDs) and site observations. Following consultations with USAID and the implementing partner (IP), the evaluation team selected five counties to visit for data collection: Kakamega, Kisumu, and Nyamira counties in the Lake Victoria Region and Kitui and Nairobi counties in the Tana/Athi Region. The evaluation team conducted 95 KIIs, 75 percent of which were with stakeholders in the five counties visited during the review. A total of 25 percent of the interviewees were women. 2 This mid-term performance evaluation was not intended to substitute for a Data Quality Assessment exercise to inspect the accuracy of the KIWASH monitoring and evaluation (M&E) system. The evaluation team looked for evidence that KIWASH interventions were of sound design and implementation, and contributed to the project’s wider objectives. FINDINGS AND CONCLUSIONS OVERVIEW KIWASH is an ambitious project with a wide-ranging set of interventions. While KIWASH is making good strides in most areas, an overarching conclusion of the evaluation team is the challenge of delivering tangible impacts in a very emerging context with many component parts across a dispersed and politically varied geography. The KIWASH emphasis on market-based approaches is highly relevant in the Kenyan context. That said, water and sanitation both have their challenges in terms of heavy infrastructure costs to ensure delivery of the whole value chain, establishing cost recovery models, and meeting the needs of a varied customer base. Thus, significant efforts are needed in the initial years to get market mechanisms and their governance structures functioning properly with an emphasis on increasing coverage figures for service delivery coming later. The recent devolution of responsibilities to MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT vi the county level complicates this further. Thus, a number of moving pieces from the outset of the project have required the IP to make difficult decisions to meet the contractual requirements. By design, KIWASH operates at the county level. Indeed KIWASH has no national level project indicators and most of its interactions at Ministry and Development Partner (DP) levels are managed directly by USAID. The project found county governments’ initial expectations difficult to manage, as county governments basically divided the amount of funding advertised by the nine named participating counties3 and assumed that each would receive US$5.6 million. These initial tensions appear to have dissipated, helped also by the arrival of a new cohort of county government officials elected in the 2017 elections. That said, DPs saw a substantial budget but then noted the rather small targets for some activities (e.g., 25 villages per county reaching open defecation free [ODF] status – since increased). DPs also noted that the target counties are not the poorest or least well performing in terms of service coverage, leading to a sense that KIWASH was designed to “target low hanging fruit,” but without the key subsequent tasks to document the lessons learned for expanding the approach to a second generation of counties. Embedding full-time staff in county government offices working alongside county officials on a day-to-day basis gives good insights and access. KIWASH staff are relied on as trusted and go-to members of the county team. They embody the customer care mantra that is a significant part of the KIWASH training modules, with a “proactive and participatory approach.” KIWASH work at the county level was described as “well-structured,” anchoring effectively in county government. As with all projects that embed staff in this way, KIWASH needs to develop an exit strategy so that their activities and approach can remain intact after the project finishes. In essence, KIWASH came at an opportune time and is well placed to influence a path forward at the county level. One DP aptly notes, however, that, “five years is a short time to do anything leading to fundamental transformation of complex systemic, structural and behavioral issues and practices associated with WASH.” The findings and conclusions for each Evaluation Question (EQ) are now presented below. EQ1: HOW APPROPRIATE IS THE KIWASH ACTIVITY DESIGN OF MARKET-BASED APPROACHES IN EXPANDING SUSTAINABLE ACCESS TO WATER AND SANITATION SERVICES IN THE TARGETED COUNTIES? At their core, market-based approaches refer to efforts “to expand greater direct involvement of the private sector in WASH and strengthen the commercial orientation in the sector overall” (as stated in Section C of the KIWASH contract). A review of relevant documents and interviews with stakeholders at all levels support the relevance and appropriateness of market-based approaches in the Kenyan policy context. While not fully developed, clarity around the different ways of fostering and elements required to implement market-based approaches is emerging. Although approved in work plans from Year 2 onwards, Section C of the KIWASH contract does not explicitly mention support to Water Service Providers (WSPs)4 under the types of business development services expected in the project. Similarly, the contract is silent on expectations for the IP to support WSPs to increase their creditworthiness as a prerequisite to delivering the target of $34 million mobilized to reach WSPs to increase access and improve efficiency (as noted in the Activity Monitoring and Evaluation Plan (AMEP) dated September 2015). Given WSPs’ role as the lead provider 3 As per Section C of the contract (pg19), “ USAID/Kenya selected these priority counties based on an analysis of multiple factors that will enable the Mission to address the high priority needs within the WASH sector, while at the same time optimizing the opportunities for synergy with other USAID‐funded development activities in related sectors.” 4 WSPs in the Kenyan context refers to water and sewerage utilities. MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT vii ultimately expected to expand or oversee water and sanitation services to unserved households across the county, KIWASH’s inclusion of utilities as a focal area is logical. Enterprise schemes (private and community) play an increasingly important role in providing services to communities that are off the utility grid or connected through a bulk connection. The emphasis in KIWASH to support them directly makes good sense. That said, the KIWASH contract stipulates that 200 enterprises should be supported to professionalize and expand their business. Such a pre-set target in a rather unknown landscape (without inventories of these emerging businesses, their coverage, or clear guidance at the county level on who and how to oversee and work with them to provide preventative maintenance, for example) seems in hindsight somewhat ill-conceived. Efforts are needed with county governments and utilities to provide greater clarity and transparency on the medium-term vision for these enterprises, particularly for those operating small piped schemes adjacent to the service areas of the utilities. The emphasis in Section C on enhancing the availability of finance for the WASH sector is certainly understandable and feeds logically into the market-based focus of the project. That said, significant impact in this space may have been somewhat unrealistic within a complex and dynamic environment. Regarding sanitation, recent policy documents and the government’s current commitment to achieving ODF status by 2020 suggest that KIWASH’s sanitation interventions are relevant and appropriate for rural areas. Scaling up sanitation and sanitation marketing, including introducing and promoting low cost technologies, remains a much-needed contribution to the sector. However, regarding design, the evaluation team questions the selection of focus counties with four of the nine having an open defecation (OD) rate of two percent or less. A target of 1.2 percent increase per year in household use of basic (stated as “improved” in the contract) sanitation across the target counties (per the AMEP) was an overambitious expectation for a project with such a diverse scope of work, the target counties chosen, and the maturity of rural sanitation markets in Kenya at the time of the design. Establishing sanitation markets is a long and interactive process that should be more at the center of an initiative rather than a small component of a larger program. Expecting the IP to design, test and scale up a sanitation marketing intervention that would achieve the contract’s targets across nine counties along with its other work across the seven outputs, with the budget allocated, was an unreasonable expectation. With the inclusion of Nairobi among the target counties, USAID should have been more explicit about expectations for the IP to work on higher levels of sanitation services beyond containment, recognizing that sanitation marketing, even coupled with CLTS and Social Behavior Change Communication (SBCC), would not meet the needs of more urban settings. Had the contract been more explicit about working in urban areas, the IP may have been in a better position to address longer-term sanitation needs beyond containment creatively. Indeed, Nairobi is probably where significant market-based experimentation could have taken place, linking up with other implementers (e.g., Water and Sanitation for the Urban Poor [WSUP] and Practical Action). The complexity of providing urban sanitation services should not be underestimated, however, and would have required significant time and money. EQ2: HOW APPROPRIATE IS THE KIWASH ACTIVITY IMPLEMENTATION STRATEGY TO THE SUCCESS OF THE MARKET-BASED APPROACHES IN EXPANDING ACCESS TO WATER AND SANITATION SERVICES? KIWASH’s implementation strategy with regard to water services has made significant strides particularly in building capacity at: the county level to understand and more effectively undertake roles and responsibilities, and delivery agent levels to enhance effectiveness and operational viability. KIWASH also works with service providers in a more direct way to expand coverage. According to the KIWASH MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT viii Year 3 Annual Report, such efforts have resulted in over 430,000 people gaining access to water services (slightly half of the 786,000 five-year target). Ultimately, much of the support from KIWASH helps the target WSPs to manage their relationships more effectively with multiple stakeholder groups. WSPs can negotiate and coordinate with counties and DPs from more of a position of strength, and communicate more effectively with customers and suppliers alike. Staff also have a clearer sense of what impacts on the business and how best to ensure its viability (and thus safeguard their jobs). With time, these improvements will be reflected in the Water Services Regulatory Board (WASREB) rankings and affect the creditworthiness of the utilities. With KIWASH and other DPs providing instrumental guidance, commercial banks are seemingly ready to lend. A number of aspects need to be clarified, not least of which is the current debt-service ratios of the target utilities. KIWASH also has a part in helping to define the overarching long-term county and utility strategy regarding the role of enterprises in the sector. That said, the evaluation team notes that the work that KIWASH has been doing to build the capacity of water enterprises is outstanding, having real impact and delivering as intended in the original design. For the sanitation work, the scope understandably became more rural facing based on the links to the nutrition work. Efforts to date have resulted in just over 57,000 people gaining access to basic sanitation services against a target of 237,000 by the end of the project (Year 3 Annual Report). Although slow to start, CLTS is being delivered “by the book” (i.e., according to Government of Kenya [GOK] protocol) and accelerating quickly. Aside from the systematic and well-coordinated way CLTS has been rolled out, the evaluation team questions the appropriateness of focusing exclusively on CLTS in the three counties where baseline OD numbers were two percent or less. In addition, county governments have not yet generally followed through with commitments of public financing to maintain momentum beyond the life of the project, which risks the sustainability of these efforts. EQ3: TO WHAT EXTENT HAVE THE DIFFERENT KIWASH PRIVATE SECTOR APPROACHES FOR WATER SERVICE DELIVERY AND SANITATION ACCESS EXPANSION BEEN SUCCESSFUL? WHICH OF THE KIWASH IMPLEMENTATION APPROACHES FOR EXPANDING SUSTAINABLE WATER SERVICES DELIVERY AND SANITATION ACCESS APPEAR TO BE THE MOST PROMISING AND SHOULD BE PRIORITIZED IN THE REMAINING TWO YEARS OF THE PROJECT? KIWASH has made significant efforts to enhance the professionalism of service providers. This has translated into strategic planning exercises with business plans that are aimed at mobilizing funds to strengthen and expand coverage further. Commercial banks have appreciated KIWASH efforts to educate their institutions on how the WASH sector works and suggested that this should lead to longer-term results. In terms of expanding capacity and reducing costs (i.e., a commercial orientation), KIWASH efforts aimed at enhancing storage capacity and reducing the material costs of providing services through an emphasis on non-revenue water (NRW) and solar-powered pumps make good sense and should continue to be prioritized. Accompanying a traditional grant scheme for such investments and to build experience with and a formal history of borrowing, KIWASH developed a recoverable grants program, often offered in combination with direct investments for water system expansion. Recoverable grants work like interest-free loans that are re-payable once investments start to yield income. Projects eligible for recoverable grants must be income generating, like metering, or income saving, like solar panels, and must help to expand a paying customer base (i.e., helping KIWASH to achieve its targets) with costs below $50 per capita. Documenting and potentially establishing a path for a county government to pilot the approach would be an important way forward. MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT ix With regard to sanitation, KIWASH’s early decision to initiate activities aimed at increasing availability of sanitation products as a post-ODF activity and forgo opportunities to explore market-based sanitation (MBS) outside of CLTS-targeted villages has resulted in limited progress to date in this important area, especially in counties with low ODF rates at baseline. A nuanced and potentially more effective approach would have been to layer MBS work with CLTS or other behavior change interventions from the start, intentionally targeting communities or peri-urban areas with higher rates of OD. As it is, KIWASH’s work in sanitation marketing is just beginning and largely targeted in areas that have achieved ODF. While formative research to explore consumer preferences and understand the value chain has been completed, a clear enterprise model for delivering the two types of products chosen as most appropriate for the KIWASH counties (Safi and Sato) is lacking and should be prioritized. KIWASH is wading into largely unfamiliar territory given the nascent state of rural sanitation markets, so expectations should be set accordingly. The project (and USAID) should anticipate some false starts and generally modest sales over the next 12 months and likely through the end of the project. EQ4: TO WHAT EXTENT HAS KIWASH BEEN SUCCESSFUL IN BUILDING COUNTY-LEVEL CAPACITY TO IMPROVE SUSTAINABLE SERVICE DELIVERY? At the outset, KIWASH found a relative vacuum at the county level in terms of knowledge of how the WASH sector works. As one KIWASH staff member noted, county officials were “really starting from scratch with the governors’ offices [having minimal] understanding of water service provision, regulations and rules in the country.” Few counties had been able to “identify their needs, costed a road map, and invested the time and energy in coordination mechanisms” – all necessary elements for any county-level program to succeed. Thus, county governments are in the midst of developing their own water and sanitation policies, legislation and plans to govern, manage, and provide services as assigned by the Constitution. A change in planning culture and perspectives as well as an openness to more participatory and accountable decision-making has been required to meet the growing expectations of the electorate. Senior county-level department heads are keen to show progress. KIWASH has implemented various capacity building interventions targeting county-level policymakers and practitioners. KIWASH supported WASH taskforces to contribute to the water investment plans of the County Integrated Development Plans (CIDP) 2017-2022 and nascent related monitoring protocols. To support policy development processes, KIWASH contributed to the establishment of various county technical teams and committees drawn from relevant ministries. KIWASH also provided short-term experts for support on policy and legislative processes. Specific to oversight of the WSPs, WSP leadership noted a need to “get past the perception that the water companies were an easy source of revenue for the government,” and curtail micro-management and county government interference. KIWASH work on county contributions to WSP governance has been critical in redressing some of these aspects. KIWASH was expected to improve the effectiveness of local governments to engage and manage the private sector, e.g., through improved regulation, contractual engagement, and enforcement mechanisms, among other methods, to complement the project’s market-driven service delivery approach. Effective as it has been, the work with the enterprises, however, has not appeared to be very aligned with county government capacity building as yet. County official interviews did not suggest sufficient familiarity with how best to support, oversee, regulate, and incorporate WASH enterprises into their planning. This is clearly a work in progress. Despite an appreciation for project activities and the recognition of KIWASH as a respected and trusted advisor at the county level, interviewees and the document review did not reveal a comprehensive and deliberative countywide capacity building strategy for the WASH sector. Nor was it clear to the evaluation team why some strategies and activities were applied in some counties but not in others, what a sequence of activities might look like, or what criteria would be used to determine when certain activities would be carried out or not. MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT x A final aspect revolves around the conundrum for KIWASH in that county-level capacity building is seen as critical to the long-term success of WASH service delivery; “governance is the overriding issue in everything,” noted one KIWASH staff member but also reiterated in different ways by numerous interviewees. Efforts to address these gaps are “long-term processes” and struggle against forces that are “not under any program’s control.” Most importantly for the framing of KIWASH, such activities do not contribute directly to the achievement of the KIWASH targets. There are short-term trade-offs in terms of energy, funding and staffing allocations that must be balanced against the expectations of wider, lasting impact and the opportunity that the transition provides. Working with other DPs and rallying political commitment to policy development and legislative processes are imperative. KIWASH’s role must be to connect the different strands (strategic planning across the county, oversight and regulation, and budget allocations and finance) to ensure success in sustaining and expanding services. EQ5: IN THE FIRST THREE YEARS OF KIWASH, TO WHAT DEGREE WERE NUTRITION ACTIVITIES SYNERGISTIC WITH WASH ACTIVITIES? In its 2015 Water Development Strategy WASH and Nutrition Implementation Brief, USAID outlined co-programming of WASH and nutrition as an area of increasing attention across the Agency. The original KIWASH design included efforts to improve access to integrated programming by working with USAID Feed the Future, health, or nutrition programs; and increasing production and consumption of nutrient dense and diverse foods. However, Feed the Future funding was discontinued at the end of the second year of implementation when community-level interventions were just beginning. After a slow start in some counties, KIWASH’s work was recognized nationally as the first attempt to bring three sectors (WASH, agriculture and nutrition) together to implement an integrated approach. The program of work was well received and momentum was building. The enabling environment provided by both the county ministries and DPs was supportive with potential for KIWASH to have been successful and, given the lack of work in this space, exemplary. An assessment across the five counties suggests the approach was consistently reaching high levels of collaboration and some cross-training across all counties. Co-location was less in evidence. Because the program of work ended early and abruptly, KIWASH and USAID missed an opportunity to learn and document evidence on WASH and nutrition integration, as well as specific methods for successful integration as was initially intended. Such documentation would have been invaluable to inform future programming in Kenya and globally. CROSS-CUTTING: GENDER MAINSTREAMING The KIWASH project has a gender mainstreaming focus in all the stages of the project from development to evaluation. The entire KIWASH consortium received training in gender mainstreaming with a clear plan on who would be the gender focal person responsible for reporting on gender indicators. At the utility (WSP) level, senior-level management received gender mainstreaming training. Interviews and findings from this qualitative review of the project showed increases in women’s skills particularly in terms of operating the enterprises, confidence in leadership roles, and capacity to advocate on their own behalf with decision-makers. KIWASH is proactively striving to meet the 30 percent gender rule in the board composition of WSPs. Community-level stakeholders and water enterprises are well aware of the need to include women in water committees and decision-making organs of the organizations. Both the utilities and enterprises have made conscious efforts to include women not only as consumers, but as part of the service provision value chain (e.g., as water kiosk operators, vendors and employees). KIWASH has clearly considered gender in the WASH facilities that they have designed. The project has made great efforts to make water sources accessible to women so that they feel safe and free from MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT xi harassment. KIWASH has consistently included both men and women in sanitation and hygiene messaging and effectively engaged both men and women as Community Health Volunteers (CHVs). RECOMMENDATIONS Based on findings and conclusions presented in Section 4, this section presents short- and long-term recommendations for KIWASH project management and for USAID. Both sets map specifically to the EQ responses above while offering the evaluation team’s higher-level perspective considering the breadth of the review. While there are areas to improve, a quote from one KIWASH staff member adequately sums up the project at this stage. “We have planted and started weeding, the harvest is coming. If we went to the next stage, everything would be so easy. The demand has been created for the services that KIWASH is providing.” SHORT-TERM RECOMMENDATIONS I) General a) To sustain the gains made by having staff embedded in county governments, develop an exit strategy focused around a dedicated shadowing role for county staff with related commitments from county leadership. b) Forge more linkages across the county-level work (among staff but also between county government partners) to exchange experiences to clarify the types of activities that are having the most impact and the criteria for when such interventions would be implemented. c) Continue to advocate for increases to WASH budget allocations at the county level. d) Clarify a strategy for the remaining period that prioritizes an appropriate strategic balance among macro policy-level work, operational activities and knowledge sharing. e) Provide clear guidance to partners on funding and proposal processes (including with regard to their competitive nature). f) Cascade gender mainstreaming training to all levels of KIWASH engagement from national to community. g) Engage with organizations focused on women’s empowerment to ensure the sustainability of the project’s gender equality goals once the project is over. II) Water a) Continue to support governance improvement activities at the county-WSP level. b) Continue to work with WSPs to articulate Performance Improvement Plans for specific components of their work. c) As possible, work with Water Sector Trust Fund (WSTF), Kenya Pooled Water Fund and WASREB (and others as appropriate, including Agence Française de Développement [AFD], the Dutch PEWAK Program, the Water, Sanitation and Hygiene Finance [WASH-FIN] project, among others) to articulate a clearer national-level strategy for financial instruments for WSPs and financial reporting. (A water.org initiative may be attempting this.) d) Revisit the World Bank-Deloitte financial modeling tool (and compare to that of commercial banks) to advise WSPs of the criteria needed to access commercial finance. e) Document the NRW and other direct impacts of KIWASH’s work with WSPs. f) Work with counties and WSPs to develop a strategy/masterplan for how they engage with localized service providers (enterprises and community-based models) and continue to develop WASH inventories. g) Document the work with enterprises (disaggregating community-managed and private entrepreneur) to underscore both the challenges and the prospects for their success based on KIWASH support. MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT xii h) As affordable, scale up acquisition of efficient and affordable billing software to address part of the operations inefficiencies in the WSPs. i) Articulate an investment strategy with counties for solar energy options for water providers to substantially reduce material costs. j) Ensure work with Water Resources Users Associations (WRUAs) links up with enterprise work to protect sources. k) Explore collaborations with others working on WRUA capacity building (World Wide Fund for Nature [WWF] Kenya, for example) to align approaches. III) Sanitation a) Continue to support final passage of the county-level Environmental Health and Sanitation Bills before the end of the project. b) Continue ongoing CLTS efforts, focusing on achieving ODF status for sub-counties (or smaller areas if resources do not allow) by the end of Year 4. Use Year 5 to document KIWASH’s legacy with CLTS while supporting continued reporting of sanitation data in ODF communities as prescribed in national protocols. c) As resources allow, develop and test a model of pairing new CLTS work with sanitation marketing efforts from entry into new areas (e.g., blending demand activation and supply chain efforts more effectively) rather than introducing sanitation products as part of post-ODF activities. d) Continue to establish or strengthen the supply chain of SATO products into communities. e) Clarify the enterprise model currently being trialed for the SATO products and the SAFI latrine, being more explicit how it works, KIWASH’s inputs, and what is being tested until the end of the project. f) Systematically document the effectiveness of the sanitation marketing approaches being tried. Keep records of buyer profiles, prices and sales margins (among other aspects) to understand how these fluctuate by season and across the diverse project area. Plan follow-up surveys with early adopters to document their profiles and motivations for adoption. Share findings at national level to influence the next generation of projects working in this area. g) Unless already part of the enterprise development work, remove the intention to support fecal sludge management (FSM) improvements across the end stages of the sanitation value chain. IV) County-Level Support a) Based on lessons learned, develop a Capacity Gap/Needs Assessment Framework for county governments with regard to WASH service provision that can be used in future projects and in non-KIWASH counties. b) Showcase each county’s successes and lessons to promote learning, foster exchange and encourage replication. For example, highlight Makueni in terms of leadership as it has a progressive and engaged CEC and Governor with strong public consultation processes, Kakamega, which is further along in policy-level advances and budgeting for WASH; Kisumu for the strength of its WASH Forum. c) Focus at the national level not on overarching national-level issues but advocating specifically on what KIWASH has learned regarding the issues to be resolved at the county level. This includes budget allocations, clearer roles and responsibilities (asset ownership, loan guarantees, county￾wide strategies, budget allocations) and their implications at the operational level, and borrowing protocols and limits, among others. d) Seek to articulate a county level strategy for community-based enterprises with regard to their relationship to county oversight and WSP expansion plans. e) Document the pros and cons of the recoverable grants program and determine whether a pilot county-led program could be developed in a further phase. MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT xiii f) Consolidate the best approaches for county-level monitoring and evaluation (M&E). V) WASH/Nutrition Integration a) Finalize and officially hand over to the counties the training of trainers (TOT) toolkit and the Training Manual for Community-Based Service Providers on Integrated WASH and Agri￾Nutrition. b) Consider an impact assessment of the distribution of hygiene kits on the uptake of maternal and child health (MCH) services. LONG-TERM RECOMMENDATIONS (USAID) a) Recognizing the transition to devolution, consider reducing the number of pre-defined targets in tender documents for a follow-on program related to WASH governance and systems strengthening, allowing respondents to suggest annual and life of project targets that respond to a series of objectives and intermediate results set by USAID. b) Contribute to filling the gap in rural sanitation programming by directing more resources explicitly to developing the sanitation market in Kenya, working simultaneously and interdependently at national, county and community levels. 5,6 c) Invest in working with WSPs, county governments and unregulated service providers to develop, cost and implement new models for FSM, much in the same way that KIWASH currently does on the water side with WSPs. d) Ensure that an environment element is fully integrated to address degradation of water resources. e) Work with the Kenyan government and align future nutrition programs with the government’s support to agribusiness through financing and information systems. f) In future programs, emphasize and facilitate more structured collaboration among United State Government partners to ensure more synergy in the provision of complementary interventions. 5 Refer to the MBS framework introduced in the 2018 USAID/WASHPaLS report entitled Scaling Market Based Sanitation: Desk Review on Market-Based Rural Sanitation Development Programs. 6 The changing living standards and urbanizing demographics in Kenya suggest considerable efforts are needed beyond excreta containment in small towns and peri-urban areas. “KIWASH has just come when devolution started… when they leave, then you leave us in the hands of the devolved systems.” (Focus Group Discussion, WASH Enterprises – Kisumu) MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 1 1.0 EVALUATION PURPOSE AND EVALUATION QUESTIONS 1.1 PURPOSE At the start of the Kenya Integrated Water, Sanitation and Hygiene (KIWASH) project’s fourth year, the USAID Kenya and East Africa (USAID/KEA) mission commissioned a mid-term evaluation to: 1. Assess the appropriateness of the KIWASH design; 2. Assess progress of implementation to-date; and 3. Inform future USAID/KEA water, sanitation and hygiene (WASH) and nutrition programming. The evaluation also informs USAID/KEA if the approaches employed by the implementing partner (IP) are successfully meeting the project’s goal of expanding sustainable access to water and sanitation services through market-based approaches. The evaluation team provides suggestions for any course corrections to the project in the final two years of the contract. 1.2 AUDIENCE AND INTENDED USES The audiences for the evaluation report include: the USAID/KEA Office of Economic Growth and Integration, KIWASH IPs, the Government of Kenya (GOK), and water sector working group/collaborators. To a lesser degree, the evaluation findings may be used by USAID/KEA and Washington technical staff to inform global efforts in the WASH sector. The intent is for the findings of the performance evaluation to be disseminated widely. 1.3 EVALUATION QUESTIONS The evaluation sought to answer the following evaluation questions (EQs), which were proposed by USAID/KEA in the Statement of Work (see Annex I). 1. How appropriate is the KIWASH activity design of market-based approaches in expanding sustainable access to water and sanitation services in the targeted counties? 2. How appropriate is the KIWASH activity implementation strategy to the success of the market￾based approaches in expanding access to water and sanitation services? 3. To what extent have the different KIWASH private sector approaches for water service delivery and sanitation access expansion been successful? Which of the KIWASH implementation approaches for expanding sustainable water services delivery and sanitation access appear to be the most promising and should be prioritized in the remaining two years of the project? 4. To what extent has KIWASH been successful in building county-level capacity to improve sustainable service delivery? 5. In the first three years of KIWASH, to what degree were nutrition activities synergistic with WASH activities? The evaluation team also considered the following lines of inquiry in relation to the five EQs: • Gender Aspects: how KIWASH’s approach is helping to mainstream gender considerations. • Brand Perception: how various stakeholders view KIWASH activities and the reputation of the KIWASH program generally as a proxy indication of KIWASH’s ability for broader influence. MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 2 2.0 PROJECT BACKGROUND KIWASH is a US$51 million project funded by USAID/KEA and implemented by a consortium led by DAI. KIWASH aims to accelerate and sustain improvements in water and sanitation access and services and improve complementary hygiene behaviors in nine counties in Kenya. It is a five-year project that began its fourth year of implementation in October 2018. The goal of KIWASH is to improve lives and health through development and management of sustainable WASH services in Kenya. Its purpose is to institutionalize catalytic models of sustainable service delivery for accelerated water and sanitation access in target counties and to improve complementary hygiene behaviors. KIWASH supports Strategic Objective 1 of the USAID Water and Development Strategy 2013-2018: Improve health outcomes through the provision of sustainable WASH, as well as achievement of the three Development Objectives outlined in the USAID/KEA Country Development and Cooperation Strategy (CDCS) Fiscal Year (FY) 2014 – 2018. The integrated nature of KIWASH’s outputs are reflected in the diverse range of activities the project implements, from technical assistance to water utilities and WASH enterprises to Community Led Total Sanitation (CLTS) and sanitation marketing to nutrition counseling. The seven outputs in the original project design generally reflect the thematic areas understood to influence the sustainability of WASH services. At the end of Year 2 (FY17), USAID amended the contract to reflect the discontinuation of the Feed the Future earmarked funding, effectively halting all activities related to nutrition and increased production and consumption of nutrient-dense foods (Outputs 3 and 4). The overall amount of the funding available to KIWASH remained $51 million. KIWASH activities are being implemented in eight counties in two principal focus areas: Busia, Kakamega, Kisumu, Migori, Nyamira and Siaya counties in the Lake Victoria Focus Area, and Kitui, Makueni and Nairobi counties in the Tana/Athi Focus Area (see Figure 3.1). (Given the complexity of operating there with contested water sources and other factors, Nairobi County is considered a “special case” and unlike the others in the program. It was expected that Nairobi could be the focus of experimentation on market-based approaches.) 2.1 DEVELOPMENT HYPOTHESIS As documented in the approved Activity Monitoring and Evaluation Plan (AMEP), the KIWASH Theory of Change is as follows: IF we build the capacity of relevant private and public stakeholders for improved water and sanitation service provision, and we integrate WASH and nutrition service delivery in target counties, THEN we will have contributed to accelerated and sustained improvements in water and sanitation access and services in the target counties complemented by improvements in hygiene and nutrition behaviors, AND THEREFORE the lives and health of people in the target areas in Kenya will be improved. The Results Framework for the KIWASH activity is aligned with two Intermediate Results (IR). 1. Improved WASH service delivery through strengthened public and private WASH service providers. 2. Improved health and nutrition through integrated WASH and nutrition service delivery. Seven Outputs further specify activities, results, and objectives. The Results Framework from the contract is presented in Figure 2.I and highlights the KIWASH implementation strategy. Neither the MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 3 Results Framework nor the Theory of Change have been updated since cessation of Feed the Future funding at the end of FY17 and the removal of Outputs 3 and 4. Figure 2.1: KIWASH Results Framework MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 4 3.0 EVALUATION METHODS AND LIMITATIONS The evaluation methodology used to answer the EQs is described below. The approach and data analysis methods described herein are generally consistent with the methodology outlined in the Inception Report approved by USAID prior to the start of data collection. The evaluation team employed four data collection methods: a document review of contractual deliverables and other documents produced by KIWASH, Key Informant Interviews (KIIs), Focus Group Discussions (FGDs), and site observations. The evaluation team also used external third-party data to provide additional context on the WASH operating environment in Kenya (e.g., Water Services Regulatory Board [WASREB] reports), public sector investments in the KIWASH counties, and work of other WASH partners in KIWASH counties. Table 3.1 details the data sources, data collection and analysis methods used to answer each EQ. Table 3.1: Data Sources, Data Collection and Analysis Methods for Each Evaluation Question Evaluation Question Data Sources Data Collection Methods Data Analysis Methods 1. How appropriate is the KIWASH activity design of market-based approaches in expanding sustainable access to water and sanitation services in the targeted counties? • GOK WASH strategy and other development partner documents • KIWASH contract • KIIs with USAID and national-level Development Partners (DPs) • Document review • KIIs • Limited review of grey and peer￾reviewed literature • Qualitative data analysis of KII notes/transcripts • Context analysis informed by document review 2. How appropriate is the KIWASH activity implementation strategy to the success of the market-based approaches in expanding access to water and sanitation services? • KIWASH Project documents, including quarterly and annual progress reports, AMEP data and Annual Work Plans. • External GOK documents • KIIs with national DPs, county government staff and service providers participating in the project • Document review • KIIs • FGDs with WASH enterprises • Site observations • Qualitative data analysis of KII and FGD notes/ transcripts and pattern analysis, data disaggregation by gender and location, as appropriate • Context analysis informed by document review 3. To what extent have the different KIWASH private sector approaches for water service delivery and sanitation access expansion been successful? Which of the KIWASH • KIWASH documents, including quarterly and annual progress reports, AMEP data and Annual Work Plans. • Externally produced documents. • Document review • KIIs • FGDs with WASH enterprises, Water Resource User Associations (WRUAs), and Community Health Volunteers (CHVs). • Qualitative data analysis of KII and FGD notes/ transcripts and pattern analysis, data disaggregation by gender and location, as appropriate MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 5 Evaluation Question Data Sources Data Collection Methods Data Analysis Methods implementation approaches for expanding sustainable water services delivery and sanitation access appear to be the most promising and should be prioritized in the remaining two years of the project? • KIIs with KIWASH staff, county government officials, service providers and DPs • Context analysis informed by document review 4. To what extent has KIWASH been successful in building county-level capacity to improve sustainable service delivery? • Quarterly and annual progress reports and AMEP data, Annual Work Plans and project documents. • KIIs with KIWASH staff, county government officials, service providers and DPs • Document review • KIIs • FGDs with WASH enterprises, WRUAs, and CHVs • Site observations • Qualitative data analysis of KII and FGD notes/ transcripts and pattern analysis, data disaggregation by gender and location, as appropriate • Context analysis informed by document review 5. In the first three years of KIWASH, to what degree were nutrition activities synergistic with WASH activities? • Quarterly and annual progress reports and AMEP data. Annual Work Plans. Project documents. • KIIs with KIWASH staff, county government officials, service providers and DPs • Document review • KIIs with KIWASH staff, county Health officials • FGDs with Community Health Volunteers • Site Observations • Qualitative data analysis of KII notes/transcripts and pattern analysis, data disaggregation by gender and location, as appropriate • Context analysis informed by document review In developing the methodology, the evaluation team employed three measures to triangulate and validate data, reduce bias and increase the depth and balance of the findings and analysis (See Annex II for further description): 1) Methodological triangulation: Consulting data from multiple sources: interview data, KIWASH reports and government documents, among others; 2) Investigator triangulation: involving teams of at least two data collectors per interview and having multiple team members involved in data analysis and identification of emerging findings; and 3) Data triangulation: Obtaining the perspectives of multiple similar types of respondents across the sampled counties. 3.1 SECONDARY DATA COLLECTION – DESK REVIEW The evaluation team began the exercise by conducting a desk review of key project documents provided by KIWASH and USAID, along with other GOK policy and strategy documents useful for placing KIWASH and the evaluation in context. The team reviewed all documents listed in Annex IV. Some documents such as KIWASH deliverables were reviewed in their entirety; in most cases team, members MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 6 focused their reviews only on documents or sections of documents related to their areas of expertise. The initial phase of the document review focused on internal KIWASH project documents, such as the AMEP and quarterly and annual reports. Findings from this review shaped the content of the interview guides used during the KII and FGDs and helped to refine the list of high-priority respondents at national and county levels. Collecting and reviewing secondary data proved to be an ongoing and iterative process. Some interviews yielded mention of documents that were previously unknown to the evaluation team or had not been included in the initial request to the IP. These were subsequently requested from KIWASH and were shared with the evaluation team with minimal delay. 3.2 SELECTION OF KIWASH COUNTIES FOR PRIMARY DATA COLLECTION Per the Scope of Work (SOW, see Annex I), the evaluation team was expected to review KIWASH progress in Nairobi (as a special case study) and four of the remaining eight counties targeted by the KIWASH project. At least two counties were to come from the Lake Victoria and one from the Tana/Athi Focus Areas. During entry meetings with USAID and the IP the evaluation team learned that KIWASH is implemented through a similar staffing configuration across seven of the nine counties. Apart from the national project office in Nairobi and a regional project office in Kisumu, KIWASH embeds four full-time staff, supported by interns, in county government offices to coordinate activities. The IP noted that all eight counties were implementing the full complement of KIWASH activities, working through the same types of implementing partners with relatively the same level of project inputs (financing) across the entire portfolio. With these factors in mind, county selection was based on the following criteria: • A mix of better and less well performing counties per data provided by KIWASH from such sources as baseline surveys, and Quarterly and Annual Reports; • At least one county in which Water Service Providers (WSPs or utilities7) are accessing commercial loans or at least accessing recoverable grants; • At least one county in which there is aggregation of small distribution schemes with a major urban utility; • At least one county where there are significant supporting investments from other development partners; and • At least one county where there is progress made on market-based sanitation. Based on preliminary analysis from the document review, the team selected the following counties for team visits: Kakamega, Kisumu, and Nyamira counties in the Lake Victoria Focus Area, and Kitui and Nairobi counties in the Tana/Athi Focus Area (see Figure 3.1). The justification for selecting these counties is provided in Annex II. 7 WSPs in the Kenyan context refers to water and sewerage utilities. The acronym “WSP” and the term “utility” are used interchangeably in the report. MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 7 Figure 3.1: KIWASH Counties Visited 3.3 PRIMARY DATA COLLECTION The evaluation team used three primary data collection approaches: KIIs, FGDs, and a limited number of site observations, as discussed below. 3.3.1 KEY INFORMANT INTERVIEWS The evaluation team conducted a total of 95 KIIs with a broad range of stakeholders to capture sufficiently diverse perspectives to answer the EQs and provide sound, actionable recommendations. Interviewees included staff from KIWASH, USAID/KEA, national development partners (public and independent) working in the WASH sector, and county- and community-level project participants who received technical assistance or funding from KIWASH. Interviewees were selected based on their familiarity (or expected familiarity) with KIWASH either through direct engagement at the county level (ideally for at least one year) or interaction with KIWASH at the national level, and their “expert status” based on their roles and responsibilities at county and national levels. A complete list of KII participants is provided in Annex IV. Approximately 75 percent of interviewees were based in the five counties visited during the review with 25 percent female interviewees. This was done intentionally, recognizing KIWASH’s primary focus (and MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 8 contractual mandate) to influence service delivery at the county level. The team conducted between 12 – 21 interviews per county with KIWASH county staff, senior-level county officials from the Ministries of Health and Water, WSP managers, WASH enterprises, sub-county public health officials, members of WRUAs and community healthcare providers. KIIs were typically between 30 – 75 minutes long and conducted in English. Most were done in-person8, usually by a two-person team, one person conducting the interview while the other took notes. The exception to this was in Kisumu, the first county visited, where most interviews were conducted by a three-person team (the Team Leader, WASH Specialist and Gender Specialist). Subsequently, the WASH and Gender Specialists who participated in interviews in Kisumu were joined by two other team members for interviews in Nyamira and Kakamega, each going to a different county to ensure some consistency of approach was cascaded down and maintained. Annex II includes the breakdown of team members who participated in data collection in each of the five counties. In addition to real-time notetaking, interview teams often requested permission from interviewees to make an audio recording of the discussion. In all cases, discussions were recorded only after gaining the interviewee’s verbal consent to do so. Transcriptions from the recordings were then used to fill gaps in the notes captured during the discussions. As agreed with USAID/KEA and the IP, a handful of national-level KIIs were conducted early in the review to provide the evaluation team with some initial background on the context in which KIWASH is operating, and some sense of where KIWASH was making significant progress. With recommendations from KIWASH on their primary points of contact in each organization, the team interviewed staff from the World Bank, UNICEF, USAID/WASH-FIN, and Water and Sanitation for the Urban Poor (WSUP). The team also interviewed other development partners (DPs) including Vitens Evides International (VEI) PEWAK program, Agence Française de Développement, and Sustainable WASH Services Initiative. 3.3.2 FOCUS GROUP DISCUSSIONS To investigate perceptions of KIWASH performance and progress by community-level project participants, the evaluation team conducted FGDs in four of the five counties (all except Nairobi). Information from the desk review suggested that FGDs were best-designed to hear perspectives from the following types of groups: • Private/community-managed WASH enterprises who have received support from KIWASH; • Community Health Volunteers (CHVs) involved in CLTS and the WASH/nutrition work; and • Members of WRUAs that benefit from KIWASH’s work related to environmental protection. FGDs followed the general flow of the KII guides provided in Annex III. KIWASH staff supported selection of participants for the FGDs and coordinated the time and place for the meetings. However, all FGDs were run independently of KIWASH staff presence and input. Discussions with WASH enterprises consisted typically of three to five enterprises, each represented by two people from their board of directors or technical staff. FGDs with CHVs and their supervisors involved 10-15 people. FGDs with the WRUAs had from 10-30 persons participating at each site. WRUA discussions tended to be led by the WRUA chairperson with some input from members. 3.3.3 KII AND FGD GUIDES Guides used for the KIIs and FGDs are provided in Annex III. The guides were structured to cover key themes of relevance to answer the EQs, as appropriate to the respondent, such as: progress toward IRs, sustainability and capacity development, lessons learned, nature of interaction with relevant 8 Eight interviews were done over the phone. MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 9 stakeholders, and gender-differential participation and results. KII guides were prepared for each respondent type (KIWASH staff, USAID staff, national level official, district level official, service provider, community level beneficiary group, or grantee). Annex II contains further explanation of factors that influenced the design of the guides. Early interviews at the national level, and in Kisumu and Kakamega counties, followed the flow of the questions closely. As the evaluation progressed, the evaluation team became more familiar with the structure of the interviews and patterns began to emerge in the types of responses offered by different categories of respondents at the county level. This resulted in divergence from the KII guide in some of the interviews to probe deeper on issues at the county level. 3.4 SITE OBSERVATIONS The FGDs gave the evaluation team the opportunity to visit a sample of the communities with which KIWASH has engaged. Site visits often scheduled before or after an FGD took the team to tree nurseries and spring protection projects, infrastructure installed to boost WASH enterprise production capacity, and model toilets constructed by masons during recent trainings on sanitation marketing. These visits offered tangible examples of KIWASH interventions discussed during the KIIs, FGDs, and document review, thus enhancing the evaluation team’s understanding of the nature and justification for these types of interventions. The visits (usually walking tours) allowed the evaluation team to discuss issues more one-on-one with a wide range of members. In two instances, the team conducted separate discussions with smaller groups of women from the WRUA for them to share their perspectives in a more conducive setting. 3.5 DATA ANALYSIS METHODS As summarized in Table 3.1, the evaluation team analyzed and synthesized primary and secondary data collected to develop responses to the EQs. Data collected during the KIIs and FGDs was analyzed using the Dedoose application9, a secure fee-based online package to analyze qualitative data. A cloud-based platform, Dedoose facilitated the sharing of data across the evaluation team on a single work space. In the first week of the assignment, the evaluation team took part in a Water, Sanitation and Hygiene Partnerships and Learning for Sustainability (WASHPaLS)-facilitated training on Dedoose and generated a list of interviewee descriptors (such as interviewee sex and location), and codes based on the EQs and KII guides, to use during analysis. The team developed, a priori, a coding tree and set of three to five key codes for each of the EQs, as well as special codes to highlight key quotations from the qualitative data and to allow for identifying leads for follow up. Annex II includes the list of descriptors and an explanation for the coding and the analysis from Dedoose. 9 https://www.dedoose.com/. 3.6 CAVEATS AND LIMITATIONS Scope of Data Collection: Per the SOW, the evaluation team visited a sample of KIWASH counties. While uniform in their implementation structure and the types of activities implemented, the experiences gathered by the evaluation team will have omitted some perspectives from the counties outside of the sample. A careful review of the KIWASH Annual Reports allowed the evaluation team to cross-correlate the findings with activities in other counties. Members of the evaluation team discussed informally with representatives from the remaining counties in various regional and national forums that occurred during the evaluation period. MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 10 Perspectives from Nairobi: The SOW suggested Nairobi as a “special case study.” Due to the different nature of KIWASH’s work in Nairobi, the sample of opinions gathered from Nairobi was more limited than in the other counties. As such, five officials contributed to interviews from the Nairobi County government and the Nairobi Water and Sewerage Company. Perspectives from the Ministry of Health: The evaluation team was unable to secure a meeting with the Ministry of Health during the window of data collection. A meeting was declined without a formal letter from USAID addressed to the Director, Dr Kepha Ombacho. (The generic letter provided by USAID at the start of the evaluation would not suffice.) Timeline: All parties were aware at the outset of the review that the timeline for conducting data collection and analysis was compressed. The team conducted initial data analysis over a three-week period in late November-early December 2018 to meet the Mission’s deadline for a draft report by December 17th. Every effort was made to ensure that the findings were as comprehensive as possible. Weekly summaries were provided to USAID and KIWASH to keep them informed of progress both in terms of delivering on the assignment and the emerging findings. Coding: As described above, the Team Leader read each one of the interview transcripts in their entirety and entered all the codes in Dedoose. Given time constraints and the volume of data generated by over 90 interviews, the team members were unable to validate all of the coding individually. The interpretation of the coded excerpts, however, came at the drafting stage where team members were tasked with drafting sections using the excerpts along with data sourced independently from the desk review and thus were able to frame the findings per their expertise. Having a single coder helped with consistency of code application across the data set. Inspection of KIWASH Monitoring and Evaluation (M&E) System: This review was not intended to substitute for a Data Quality Assessment exercise to inspect the accuracy of the KIWASH M&E system. Achievements reported in project documents (annual reports, quarterly reports) and working program documents were taken largely at face value. The evaluation team looked for evidence that the types of interventions described in the documents were of sound design, implemented and contributed to the project’s wider objectives, without physically verifying if the numbers reported by KIWASH are accurate or substantiated. Findings from a mid-term mixed-methods survey conducted by the Africa Center for Health Systems Research and Management, a Kenyan research firm that assessed progress towards achieving high-level service delivery outcomes on behalf of the IP, was shared with the evaluation team on November 11, 2018. 10 Limitations on recorded data: Not all interviews were recorded for a variety of reasons: eight recordings in Kisumu were lost due to a malfunction of the recording device; some FDGs were not recorded due to logistical reasons; some respondents did not give their consent to be recorded; or the quality of some recordings is poor due to the acoustics in the room or background noise (like rain on a metal roof). Where recordings are absent, the team relied solely on notes from the discussions and used these for coding in Dedoose. Where necessary, follow up emails were sent to clarify any confusion from the notes. 10 Many of the EQs for the IP-supported review and this review are identical. MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 11 4.0 FINDINGS AND CONCLUSIONS11 11 This mid-term evaluation includes a number of quotations from KIWASH county staff, senior-level county officials from the Ministries of Health and Water, WSP managers, WASH enterprises, sub-county public health officials, members of WRUAs, and community healthcare providers. If the quotation captures a general sentiment that was conveyed a number of times in different ways, the evaluation team has not necessarily attributed it to any one interviewee. Findings and conclusions for the five EQs are presented in this section. For each EQ, findings are offered first, often separated by sub-headings to improve readability, followed by summary conclusions. Per the Inception Report, findings are not organized by Output area in the KIWASH Results Framework, although the questions are understood to track broadly to the Output workstreams (EQ1, EQ2, and EQ3 corresponding to Outputs 1 and 2; EQ4 corresponding to Outputs 5 and 6; and EQ5 corresponding to Outputs 3 and 4). That said, the integrated nature of the project, and effective project delivery, demand a certain degree of programmatic intersection across Outputs. The evaluation has tried to capture this in the analysis below. 4.1 EQ1: HOW APPROPRIATE IS THE KIWASH ACTIVITY DESIGN OF MARKET￾BASED APPROACHES IN EXPANDING SUSTAINABLE ACCESS TO WATER AND SANITATION SERVICES IN THE TARGETED COUNTIES? The evaluation team’s response to this question explores whether the directive given by USAID to KIWASH in Section C of the contract to focus on market-based approaches to expand sustainable access to WASH services was appropriate in the Kenyan context. The response is separated into two parts, the first focusing on water services and the second on sanitation. Before focusing specifically on water and sanitation services, it is relevant to offer a few perspectives shared by key informants on the appropriateness of the overarching program design, given the integrated nature of the project. In general, there is a perception, which the evaluation team shares, that the KIWASH approach has been a bit unruly or “scattered in design, perhaps by intention.” This made it appear to some observers, at least at the start, as if KIWASH “wants to be seen as if they can do everything.” The project’s mandate to engage concurrently with county government officials, WSPs, WASH enterprises, WRUAs, and health care professionals across a dispersed geography with varying county-level political contexts in an emerging but not fully articulated devolution process is seen by some as a fundamental challenge for the project. “Integration has been perhaps too much to manage with the time limits compounded by the shift in funding,” with the removal of the Feed the Future funding as well as the emphasis on “hitting the numbers” (i.e., reaching the targets in the contract). One KIWASH staff member suggested that the primary focus could have been placed only on “utility companies as they are the sustainable element... enterprises are emerging but ultimately the responsibility [for service coverage] lies with the utilities.” 4.1.1 THE KENYAN WATER SERVICE DELIVERY CONTEXT AND THE KIWASH DESIGN Market-based approaches underpin USAID’s vision for the expansion of sustainable water service provision in urban and rural Kenya. The relevance and appropriateness of market-based approaches in the Kenyan policy context has been clearly endorsed in a review of relevant documents and interviews with stakeholders at any level. However, the caveat may be, as noted in the inception report, around the interpretation The Kenyan policy environment explicitly endorses and fosters market-based approaches for both water and sanitation services. MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 12 of the term “market-based.” At its core, market-based approaches refer to efforts “to expand greater direct involvement of the private sector in WASH and strengthen the commercial orientation in the sector overall” (from Section C of the KIWASH contract). Thus, the IP is expected to foster WASH services based on financially viable providers (whether public or private) as a result of efficient infrastructure and a paying customer base (albeit with likely mechanisms to determine alternative financing or cross-subsidies for poorer households who may not be able to cover the full life-cycle costs of the service). The Kenyan policy environment explicitly endorses and fosters market-based approaches for both water and sanitation services. Clarity around the different ways of fostering and elements required to implement market-based approaches is emerging. Implications for different stakeholders with regard to how best to apply the approach in the counties are emerging. Other DPs appear to be moving in this direction. Water Providers Section C of the KIWASH contract makes clear reference to market-based approaches under Output 1 related to the provision of business development services for WASH services and products, and to an increase in sustained access to financing and credit for WASH under Output 2. However Section C does not explicitly mention support to utilities under the types of business development services expected under Output 1. Similarly, the contract does not direct the IP to support utilities to increase their creditworthiness as a prerequisite to delivering the Output 2 target of $34 million mobilized to reach water service providers to increase access and improve efficiency (as noted in the AMEP dated September 2015). Approved subsequent KIWASH Work Plans from Year 2 onwards reference support to utilities, but it remains unclear whether this was a gap or intentional with regard to USAID’s initial thinking when the SOW was developed. Given their role as the lead provider ultimately expected to expand or at least oversee water and sanitation services to unserved households across the county, KIWASH’s inclusion of utilities as a focal area makes good sense. To fill a gap in the market, WASH enterprises that focus on operating water systems have emerged in primarily two forms: community-managed and privately operated. In recent years, efforts have been made to blend the two with a professional private operator being hired by a community management committee to run the service. Such professionalization is aimed at instilling business-like management practices that prioritize efficiencies and cost recovery. This would help avoid “local social dynamics and political interference” which sees the service falling “victim to seizure of power by a handful of self-interested individuals.” The community angle is to ensure that the service provider does not neglect the less profitable areas where there is need.12 A further permutation in dense urban settlements is through a Delegated Management Area (DMA), which sees a neighborhood management committee provided with a metered bulk water supply from a municipal utility (see the KIWASH-documented support to Nyalenda in Kisumu in the Annual Report #3 as an example). Although not new, these enterprise constructs are becoming more prevalent. At the start of the KIWASH project, it appears that in most cases counties did not have a ready-made inventory of water enterprises except for kiosks that were connected to the utility. Thus, an initial step for KIWASH was to embark on a substantial survey to understand the scale of this aspect of the market in the different counties. With over 600 enterprises identified as possible targets for KIWASH support, western counties like Kisumu in particular (and Nairobi) were found to have a greater number of somewhat more advanced and a more bankable set of enterprises. KIWASH appropriately set criteria around viability and current/potential coverage with regard to those enterprises that would be prioritized for support, but also with the need to balance out the WASH enterprise interventions across 12 Water and Sanitation for the Urban Poor (WSUP), Hybrid management models: blending community and private management, WSUP Topic Brief, TB#009 – Feb 2013. MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 13 the participating counties. A number of factors, however, were noted as “still to be determined” with regard to how these enterprises fit into the service delivery system. In terms of accountability, interviewees noted that apart from the licensing, it was unclear to whom these enterprises reported, or which specific department was tasked with overseeing or regulating them within the new county structure after devolution. The evaluation team understands that the enterprises all set their own by-laws with minimal guidance from local authorities. For example, overarching rules around whether committee members are allowed to derive an income or allowance from their oversight role is not clear. (Many committee members are retirees in the community.) Finally, and perhaps most importantly is the enterprise’s relationship with the utility, which appears to have overarching responsibility in its jurisdiction to provide services. Thus, there is a sense both that the utility might be required to provide support not only to the DMAs (distributing water to the community through a bulk water connection either through kiosks and/or reticulated pipe systems), but also that the utility and/or county might want to take these systems over once they become profitable. As such, it has not gone unnoticed that these enterprises are only issued a temporary license by county authorities to provide services. Ultimately, national government and counties appear to have a vision for utilities to expand services in all peri-urban communities and small towns. Some are more ready for this than others. For various reasons, the most advanced entrepreneurs are likely to be found in the catchment areas nearest to major utilities. With the pre-defined 200-enterprise target and associated access numbers (specified in the AMEP), KIWASH could not forego working with these groups even though in some cases such investments may be in direct competition with the utility (even in some cases with utility pipelines passing through the same communities). Access to Finance Evolving Devolution: Under Output 2, the contract states that, “the sector needs a more sustainable financial model including the mobilization of greater cost recovery from users, increased efficiency in service provision, better targeting of subsidies, and better mobilization of financing.” While all true, a number of contextual factors well beyond KIWASH’s control currently influence the sector’s approach to finance. At the heart of the financing challenge is the ongoing transition of accountability as proscribed in the Water Act of 2016 that puts county government in charge of ensuring water service delivery. While clear in the overarching roles and responsibilities, a number of practical aspects are still being worked out at both the county and national levels with regard to finance and county capacity to manage its oversight function. According to several interviewees and aptly summarized by one DP, there is “no clear policy on government financing support for utilities and market-based interventions.” A few overarching factors should be considered when framing the analysis of KIWASH’s design and expected contribution to the sector. Firstly, the culture of borrowing assumed in the KIWASH design and by numerous other DPs may not be as strong as imagined. There is still an expectation, as one interviewee noted, of “you will give me” and that DPs “have engrained this in people, who are not familiar with paying back loans [or practically expected to]. DPs come to do everything and then hand With regard to utility borrowing, a number of fundamental questions were not easily answered related to who is the borrower, who authorizes the loan, and who holds the liability in case of default or delay on payments. MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 14 over the project.” As such, the country has perhaps not adjusted to its middle-income status, which does not attract the same kind of donor assistance as in the past. 13 Secondly, in light of the recent devolution of responsibilities, interviewees suggested that county government is “still not fully trusted [by GOK and donors] to channel money in the right way if funds are channeled through them.” The perception of mismanagement of funds persists with WASH services being caught up in county political machinations with interference that may have serious impact on service delivery, expansion plans, and ringfenced funding. Utilities are meant to operate as semi-autonomous limited companies, but there is still the sense that as the “owner” of the utility, counties can exert more pressure than their 3-4 seats on the 10-12 person board of directors would generally suggest. The perception from a number of stakeholders is that there is a rather constant threat of micromanagement and interference in utility operations on the part of the county government. There also is the sense that county government drives unnecessary procurements as that is where the opportunity for corruption lies: “they just want to buy a bunch of pipes whether there is a need or not.” With regard to utility borrowing more specifically, a number of fundamental questions were not easily answered or at least not with consistent responses related to who is the borrower, who authorizes the loan, and who holds the liability in case of default or delay on payments. According to an interviewee from a commercial bank, guarantees from county governments also are not straightforward and would require a long drawn-out legal process to set them up for this guarantor role. Thirdly, notwithstanding the above, there is increased pressure on county governments to deliver services to their constituents and several county-level officials noted that their own continued tenure was predicated on having an actual impact. As Koehler notes, there is “great pressure on incumbents to deliver on their agendas and on newly elected candidates to surpass the achievements of their predecessors.” But also with the localization of such politics, the “decentralization of patronage networks” might lead to “certain areas [being] unevenly targeted for investment.” 14 Thus, the pressure to produce results may in fact pose a challenge that USAID funding could be used to perpetuate ethnic or other forms of politics, particularly where there were “tight gubernatorial election margins.”15 Commercial Finance: As reflected in the KIWASH design, much emphasis at the macro level has been placed on private sector finance to fill the funding gap in the water sector. Indeed, commercial banks are looking for new opportunities that will provide stable returns. With interest rates capped at 13 percent, commercial lending is now seen as a viable option, although banks are “still trying to be creative with service charges to recover lost revenue from these lower rates.” As a further option, an emerging bond market (funded from insurance and pension funds) is expected to accommodate slower returns from longer-term investments in the water sector. Commercial banks, however, are interested primarily in “programs where there is technical assistance and a guarantee for the repayment of loans.” Several commercial banks (KIWASH identified Co-operative, ABC, and Family) are seeking opportunities but also “waiting for the dust to settle” on a number of the questions noted above around lending at the county/utility level. This would hopefully result in clearer commitments being made by county governments presumably in the form of a guarantee of some kind. Historically, the national government would have been the main guarantor, but as one WSP 13 “Kenya has progressed to become a middle-income country, and therefore – in donor circles – is no longer eligible to secure grants and other lower-cost funding lines previously open to it. So there are very few options other than to access commercial funding in order to expand services.” (Bonds and Loans (2018) “Kenya Pooled Water Fund CEO: The Water, Sanitation Sector is Ready for Green Bonds” www.bondsloans.com/news/article/1887/kenya-pooled-water-fund-ceo-the-water-sanitat, accessed on 22 November 2018). 14 Koehler, J. Exploring policy perceptions and responsibility of devolved decision-making for water service delivery in Kenya’s 47 county governments. Geoforum: 19 Feb 2019. (Elsevier Publishing – received by email) 15 Ibid. MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 15 Managing Director noted, they have “no direct relationship [with the WSP] and so don’t feel a part of [what we are doing].” The landscape of WASH sector financing in Kenya is crowded with various government agencies and development partners “tripping over each other” to provide a solution. The range of interviews touching on finance made clear that several actors are working in this space, but it was not apparent “who has the big picture” connecting the different approaches and activities. Some are focused on DP mechanisms and international resource mobilization. Others are focusing on commercial banks and national-level lending; some with national government while others focus at the county and utility level. Beyond this, the different permutations of blended finance with bonds, loans (subsidized and commercial), and grants for short- and long-term finance being promoted concurrently, sometimes to the same utility, further complicate the picture. Given the situation described above, there appears to be unspoken or even outright competition over prominence of ideas, approaches, branding, and recognition. Interviews quickly revealed this aspect even with the increasing emphasis on coordination forums to overcome some of these challenges. DPs (particularly donors) put a clear emphasis on “hitting the numbers” (i.e., reaching set targets), focusing on certain geographies, finding the formula that moves the sector toward more commercial approaches, and tapping into greater levels of private sector finance. This dovetails with the formula used to define the outputs expected in Output 2 of KIWASH’s approved AMEP. According to one KIWASH staff member, as DPs are incentivized in a particular way, there is “too much emphasis on closing deals and not on preparing the ground sufficiently.” One DP interviewee noted that DPs are scrambling “to find the viable service providers and then to get the credit for it. The approach seems to be of cherry picking instead of finding weak utilities and then building them up to borrow.” While competition between financial instruments could be useful, the sector lacks the clarity of the underlying rules of the game that would remove perverse incentives. 4.1.2 THE KENYAN SANITATION SERVICE DELIVERY CONTEXT AND THE KIWASH DESIGN Market-based sanitation (MBS) projects are interventions that support private sector-led supply of household toilets and related sanitation services in order to increase market access to affordable, desirable sanitation options that households purchase directly from a private sanitation enterprise. Market-based approaches to sanitation (e.g., sanitation marketing), which seek to unlock household investment and strengthen the private sector, have been gaining in popularity and are included in the latest strategies of major global sanitation development agencies (e.g., World Health Organization, World Bank, and USAID). 16 MBS solutions increasingly are considered necessary, in part, to complement demand generation (e.g., behavior change) efforts to encourage fixed-point defecation and stimulate household and private sector investments needed to close the funding gap required to reach the United Nations Sustainable Development Goal (SDG) of adequate and equitable sanitation and hygiene for all and ending open defecation by 2030. In Kenya, per the 2016 Environmental Sanitation and Health Policy and Open Defecation Free (ODF) Roadmap, community-focused behavior change efforts (including CLTS) are expected to be paired with sanitation marketing to promote the adoption of durable toilets that meet the SDG standards for basic sanitation. It is in this context that USAID’s inclusion of MBS approaches in the KIWASH SOW under Outputs 1 16 USAID (2018) Scaling Market Based Sanitation: Desk review on market-based rural sanitation development programs. Washington, DC., USAID Water, Sanitation, and Hygiene Partnerships and Sustainability (WASHPaLS) Project. While competition between financial instruments could be useful, the sector lacks the clarity of the underlying. MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 16 (focused on the containment step of the sanitation value chain) and Output 5 (focused on Fecal Sludge management) is understood. To help unpack the appropriateness of including a mandate to stimulate MBS solutions, the following subsections describe the context in which rural sanitation initiatives operate in Kenya, and then the KIWASH design. Sanitation Service Delivery Context Sanitation is a constitutional right in Kenya. Per the 2012 County Government Act, the responsibility for achieving universal sanitation services rests with county governments, under the institutional jurisdiction of the Ministry of Health, as represented by the Public Health Office with staff at county and sub-county levels. Deconcentrating responsibility for sanitation services to the counties, however, has not yet resulted in increased public resource allocations to address the issue. According to one national level respondent, a recent Public Expenditure Review for WASH and Health in 10 counties revealed that only five had any sanitation related specific expenditure in the county budgets. As such, multiple national￾level respondents described sanitation as the domain of partners, without which the counties would not have resources to fund activities. “Financing for sanitation has been too donor dependent” described one interviewee. The “MOH [Ministry of Health] and Public Health can do nothing without partner support. All the allocations from their funds (i.e., public funds) are for salaries. The ODF agenda is fully dependent on which donor will be where.” Another national-level respondent suggested, “county governments are still not yet having clear investment strategies for WASH services. The most affected are sanitation services which do not have defined budgetary allocation.” Creating Demand through CLTS: In rural settings, the GOK adopted CLTS as its foremost programmatic response to eliminating open defecation (OD). The ambitious ODF Rural Kenya 2013 Campaign Roadmap suggests county governments, working primarily through the public health extension services, are duty bearers to lead their counties to ODF status by 202017. According to the Roadmap, CLTS will be rolled-out through the sub-county public health departments, working through cadres of Community Health Volunteers (CHVs). A national CLTS database funded by UNICEF and housed at the Ministry of Health tracks CLTS data reported through county governments. CLTS has slowly diffused as an approach being supported / implemented by the operational staff of the public health system, although not being implemented at nearly the scale that is described in the Roadmap. Public health officers in the county and sub-counties visited during the review are knowledgeable about CLTS, but do not have budgets or sufficient staffing to implement, monitor regularly and report progress systematically in this area. At the time of this evaluation, three counties in Kenya have been declared ODF (Busia prior to KIWASH and Siaya and Kitui both partly because of KIWASH), while many others, particularly those with donor support, are working towards this goal. UNICEF has developed a detailed costed model for CLTS in Kenya, suggesting that it costs US$300 on average to complete all the steps for a village to be certified.18 KIWASH has similar if somewhat lower costings. 17 The definition of ODF in Kenya is that all persons practice fixed-point defecation, irrespective of latrine quality or its characteristics. 18 Other DPs such as Amref (implementers of the Global Sanitation Fund work in Kenya) have similar costing figures for CLTS implementation. Deconcentrating responsibility for sanitation services to the counties has not yet resulted in increased public resource allocations to address the issue. MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 17 Developing Supply Chains: Despite the GOK’s stated commitment to pair sanitation demand creation activities with a ramp up in commitment to achieving ODF counties, the desk review found that donor investment in MBS appears to be extremely limited and public-sector investment in non-sewerage services nonexistent. The evaluation team found only the UK Department for International Development-funded Sustainable Sanitation and Hygiene for All project implemented by SNV (a KIWASH sub-grantee) and the ongoing, but embryonic, work by UNICEF to increase the supply chain of the Sato line of plastic latrine products (designed by the Japanese conglomerate Lixil and manufactured locally by SIL Africa) on a scan of active donor investments in this area. Activities, by Water and Sanitation for the Urban Poor (WSUP), Sanergy, and others, to create entrepreneur-led community toilet blocks and container-based sanitation services in urban slums also present market-based solutions, as does the Dutch-funded work of Amref Health Africa and their FINISH project to work with banks to develop market-based lending products for household investment in sanitation. The scope and maturity of these initiatives, however, generally suggest that developing a viable sanitation market in Kenya is still in very nascent stages and represents a lagging gap in donor support in the sanitation sub-sector. Including an explicit focus on sanitation marketing (with mention of both product development and supporting entrepreneur viability) within KIWASH’s design was, therefore, a justified area of intervention in rural areas. This is consistent with recent global literature that frame CLTS, or other means that aim to change sanitation-related social norms, often as necessary elements of successful market-based approaches to establish latent demand that can be met via private sector services. (Clear caveats around resource requirements, timeframes and the nascent state of the market are captured more in response to EQ3.) WSPs and Sanitation: Although a pillar of the Environmental Sanitation and Health Policy, respondents at both national and county levels recognized that much less donor and political attention has been given to urban and small-town sanitation issues beyond containment. However, as evidenced by the Ministry of Water and Irrigation recently being renamed the Ministry of Water and Sanitation, there is an increasing recognition that sanitation, especially in urbanizing areas where much of the attention to date has been on sewerage, falls under the responsibility of said ministry vis-à-vis the WSPs. The Managing Director of KIWASCO, widely considered to be a progressive and well-managed utility, told the evaluation team that KIWASCO is in the process of formally changing the “S” in their name to refer to Sanitation instead of Sewerage. He acknowledged KIWASCO needs to do more in this area and sees it being within their mandate to do so. Another informant from the same utility suggested the utility “needs to get more into sanitation. We have exhausters. We are concerned with residents in informal areas, improving their quality of health is important for our vision.” To take this service provision forward, as authorized by WASREB, WSPs issue licenses to fecal sludge vacuum truck operators who operate in their service area. Like the expansion of piped-water supply services, under devolution, WSPs are increasingly expected by county governments to expand and diversify sanitation services to include larger service areas. Yet, in Nairobi, one respondent felt that the county was only interested in large investments (e.g., sewerage not community-based interventions). Discussions with WSPs in two other counties also suggested the prevailing interpretation was that the utility’s role was to expand sewerage connections, without considering other fecal sludge management (FSM) solutions that may be more appropriate for the sanitation profile of the towns and certainly less capital intensive to implement than an expansion of sewerage. A viable sanitation market in Kenya is still in very nascent stages and represents a lagging gap in terms of donor support. MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 18 Some respondents noted that county governments follow sanitation protocols from national level, which are mostly rural focused. In Nairobi, the city developed an urban sanitation protocol, but it was never launched. A county official from outside of Nairobi suggested that there should be some innovations to help determine how best to work in urban areas, noting that “the sanitation situation is terrible, and we cannot wait for the national government. If you go to the slums, they know the problems and can come up with local solutions with the community.” KIWASH Design Regarding Sanitation The KIWASH contract sets a mixed level of expectations in terms of MBS results under Outputs 1 and 5. Output 1 focuses on toilet construction in rural areas, positioning sanitation marketing as the mechanism to fulfill the demand for improved services generated by CLTS. The target number of 200 ODF villages is quite modest given the GOK’s ambitious stated commitment to declare all counties ODF by 2020. Table 4.1 shows the rates of OD and unimproved (excluding shared) sanitation for the nine KIWASH target counties as reported by the GOK in 2014 and by the KIWASH baseline in August 2016.19 Notwithstanding intra-county variations that may be masked by countywide statistics, four of the eight KIWASH counties (as well as Nairobi which does not have a CLTS focus for KIWASH) had some of the lowest OD rates in Kenya at the outset of the project. Comparing countywide statistics from 2014 and 2016 suggests the trend toward eliminating OD was well established as KIWASH got underway. Table 4.1: Rates of OD and Unimproved Sanitation in KIWASH Counties County OD Rate (%) Unimproved Rate (%) OD Rate (%) Unimproved Rate (%) MIGORI 33.1 15.5 12.5 30 KITUI 30.9 15.1 7.9 19.1 SIAYA 20.2 31.9 13.2 44.2 KISUMU 12.4 31.3 8.6 19 BUSIA 8.0 31.4 2.1 29.3 MAKUENI 2.0 41.6 0.0 18.3 KAKAMEGA 1.5 15 1.4 49.5 NAIROBI 0.7 12.4 0.5 7.6 NYAMIRA 0.5 32.6 1.2 57.6 Source: County Sanitation Profiles, Ministry of Health/Water and Sanitation Program, 2014 Source: KIWASH Baseline, August 2016 19 The KIWASH baseline survey detected extremely low rates of shared sanitation across all counties, including zero percent in Nairobi, which is odd given that shared facilities are common in urban areas and well documented in certain low-income neighborhoods in the capital. The United Nations-UNICEF Joint Monitoring Program (JMP) estimates that nationally 43 percent of Kenyans use shared facilities, including 50 percent of urban residents, leading to high rates of “limited” services, or those where an improved facility is shared by more than one household. The baseline survey reported that 64.2 percent of households across the KIWASH implementation counties use an improved sanitation facility. This differs substantially from the national average of 53 percent as indicated in JMP estimates from 2015. The evaluation team suspects this is explained by a difference in definitions of improved or basic sanitation used by the two surveys. Other targets are more ambitious, like 237,000 people (or approximately 47,000 households) gaining access to basic sanitation services and an average annual increase of 1.2 percentage points of households having basic across all KIWASH counties (the latter target is unrealistic). This requires movement to higher service levels than are typically reached with traditional CLTS and thus should not be expected in a project operating in an undeveloped MBS context like rural Kenya. In Nairobi County, achieving this target would have required an explicit focus on reducing shared facilities, focusing on wider systems issues such as land tenure, rental/ownership, and space constraints that were beyond the activity’s SOW. Furthermore, a recent literature review looking at characteristics of MBS programs targeting MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 19 rural areas that have sold more than 10,000 toilets (far less than KIWASH’s target) suggest an average duration of 5.8 years and a consistent pattern of sales beginning to accelerate four to five years after the initiation of the project. 20 From this perspective, the targets for households with basic sanitation are higher than what could have been reasonably expected by an integrated program like KIWASH. In addition to focusing on toilet construction in rural areas, Output 5 of the KIWASH contract describes USAID’s original intention, through the lens of environmental sustainability, to support a limited number of private/public sector de-sludgers in small towns “to explore improvements in safe collection, transport, and treatment; including disposal and reuse of waste as a potential for income generation.” Given the extensive range of activities expected of KIWASH already, this specialist area of working across the sanitation value chain was probably best left out of the equation. Notably, the KIWASH AMEP from January 2017 does not include the standard USAID foreign assistance indicator that measures access to safely managed sanitation services that go beyond the containment step on the sanitation value chain. The AMEP aligns the indicator for the number of sanitation providers addressing sludge management under Output 1, which fits better with the types of business development support activities KIWASH should have been expected to provide. 4.1.3 CONCLUSIONS – EQ1 An overarching conclusion of the evaluation team is the challenge posed to KIWASH of operating in an emerging context across so many component parts. In terms of market-based approaches, water and sanitation both have their complications. Significant efforts are needed in the initial years for market mechanisms and their governance to function properly. Significant investments to expand service delivery coverage should then come later. In Kenya, this has been further complicated by transitioning devolved responsibilities to the county level. Thus, a number of moving pieces from the outset of the program have required difficult decisions to meet the design requirements. Water: The emphasis on market-based approaches in the water sector aligns well with the policy context in Kenya and thus is logical as a key thrust of the SOW. The evaluation team notes, however, that there was a gap in the original SOW (since updated through work plans) that relates to KIWASH’s expected work with utilities. Encouraging utilities to adopt a more market-based (i.e., commercial) approach that emphasizes efficient infrastructure and a paying customer base is well recognized as an important component of a viable water sector. The regulator, WASREB, places a clear focus on increasing cost recovery and reducing non-revenue water while still addressing service inequalities. Financing agencies also are looking for progress on ensuring that utilities are run more effectively and efficiently. Ultimately, the enterprise schemes (private and community) play an increasingly important role in providing services to communities that are off the utility grid or connected through a bulk connection. The emphasis in KIWASH to support them directly makes good sense. Yet without a clear grasp on the context, the KIWASH contract stipulates (seemingly arbitrarily) that 200 enterprises should be supported to professionalize and expand their business. Such a pre-set target for enterprise development in a rather unknown landscape (without inventories of these emerging 20 USAID (2018) Scaling Market Based Sanitation: Desk review on market-based rural sanitation development programs. Washington, DC., USAID Water, Sanitation, and Hygiene Partnerships and Sustainability (WASHPaLS) Project. Significant efforts are needed in the initial years for market mechanisms and their governance to function properly. MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 20 businesses, their coverage, and clear guidance at the county level on who and how to oversee and work with them to provide preventative maintenance, for example) seems somewhat ill-conceived. Efforts are needed with county governments and utilities to provide greater clarity and transparency on the medium-term vision for these enterprises, particularly for those operating small piped schemes adjacent to the service areas of the utilities. In terms of finance, the emphasis seems to be more on the larger utilities. If these enterprises are truly meant to be part of the solution, then small lending programs, connected to a support project such as KIWASH, would go a long way in ensuring their viability. Future programs should help deliver this clarity and vision at the county level and then develop appropriate support mechanisms depending on the designated way forward. As suggested, the role of county government in these programs, particularly after KIWASH ends, also needs to be determined. As noted by a county government official, county-level responses, at present, are particularly slow to resolve issues with these enterprises if it is known that programs like KIWASH will step in (to repair breakdowns or help resolve minor local conflicts, among other issues). The project’s focus on market-based approaches, as defined above, is sound and necessary to ensure more functional, responsive and sustainable service delivery. Efforts to improve the technical and commercial operational efficiencies of WSPs will help lead them to creditworthiness. The emphasis in the KIWASH design from Section C on enhancing the availability of finance for the WASH sector is certainly understandable and feeds logically into the market-based focus of the program. The expectations for impact around finance may have been somewhat unrealistic within a complex and dynamic environment of counties becoming responsible for service delivery, the shifting donor landscape, and the other factors noted above. While still identifying opportunities for lending that “push the needle forward”, the emphasis on commercial finance appears to be somewhat premature, as noted in different ways by a wide range of interviewees. More work is needed across the sector to connect the different approaches into a coherent strategy incorporating commercial financing as one of a series of options that need to be tailored for different service provider capacities and contexts. As attempted by the Water Sector Trust Fund (WSTF) and the Kenya Pooled Water Fund, there is a move to create a clear first generation of creditworthy service providers (including KIWASH target utilities) that are effectively borrowing money. Lessons then need to be extracted methodically from these early adopters with a view to seeing how best to extend their readiness for and uptake of commercial finance, and then shared to reach a second generation of service providers. A new water finance initiative has been convened in Kenya by Water.org to map the work being done by different organizations in the finance space. The team understands that three or four meetings have been held since June 2018. KIWASH should continue to participate, bringing its knowledge about the WSPs and enterprises with which it has been working. Sanitation: The direction of recent policy documents and the sector’s current commitment to achieving ODF status by 2020 suggest the design of KIWASH’s sanitation interventions is appropriate for rural areas of the target counties (excluding Nairobi). Output 1’s directive for the IP to scale up sanitation and sanitation marketing, including introducing and promoting low cost technologies, remains a needed contribution to the sector. However, the evaluation team questions the selection of focus counties with four of the nine having had an OD rate of two percent or less at the start of the project. The evaluation team suggests that a target of 1.2 percent increase per year across all target counties was an overambitious expectation for a project with such a diverse scope of work (including a more limited focus on sanitation as compared to water), the target counties chosen, and the maturity of rural sanitation markets in Kenya at the time of the design. Recent research by WASHPaLS and others MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 21 suggests that establishing sanitation markets is a long and interactive process requiring this to be more at the center of an initiative rather than a small component of a larger program. Expecting the IP to design, test and scale up a sanitation marketing intervention within the nascent context and in the timeframes allocated was perhaps an unreasonable expectation. Including Nairobi among the target counties should have prompted USAID to be more explicit about expectations for the IP to work on higher levels of sanitation services beyond containment, recognizing that CLTS and Social Behavior Change Communication (SBCC), even coupled with sanitation marketing, would not meet the needs of these more complicated settings. This would have added more complexity and it may not be appropriate with the general rural-facing orientation of the project. It was unrealistic to include Nairobi as a special case, with the expectation of meeting the target of 1.2 percent annual increase in sanitation coverage, without clear guidance in terms of what was expected from the IP. Had the contract been more explicit about working in urban areas, with less emphasis on achieving a pre-defined target number of households reached across the entirety of the target counties, the IP may have been in a better position to address longer-term sanitation needs beyond containment creatively. That said, the complexity of providing urban sanitation services should not be underestimated and would have further complicated the ambitious design. Indeed, Nairobi is probably where significant experimentation could have taken place on market￾based approaches, connecting with others (e.g., WSUP or Practical Action) in this space. The complexity of providing urban sanitation services should not be underestimated, however, and would have required significant time and money.21 Ultimately, the evaluation team questions the value added to the project by including a small and largely inconsequential remit to work on FSM solutions vis-à-vis the overarching core rural focus of the KIWASH contract. While FSM is certainly an area of concern in a rapidly urbanizing context such as Kenya, the appropriateness of adding this niche and specialized area of concentration into a mechanism with such a wide scope is questionable. While including FSM is certainly not contrary to the project’s goals, global experience suggests that developing FSM services well requires skill sets beyond what would have been staffed for an integrated and ultimately more rural￾facing project such as KIWASH. Likewise, focusing resources in this area without a commensurate indicator for reporting results is not an attractive proposition for an IP intent on reaching contractual targets. As has been shown, the risk is then of a component being ultimately de-emphasized (and largely forgotten) by the project management team and the Contracting Officer’s Representative. 21 WSUP documents the requirements of working with different government offices in Nairobi noting that it “has taken nine years to reach a certain level of impact.” See: WSUP (October 2018) A journey of institutional change: Extending water services to Nairobi’s informal settlements. KIWASH also notes the particularly political nature of working in Nairobi with various shifts in management and other disruptions. 4.2 EQ2: HOW APPROPRIATE IS THE KIWASH ACTIVITY IMPLEMENTATION STRATEGY TO THE SUCCESS OF THE MARKET-BASED APPROACHES IN EXPANDING ACCESS TO WATER AND SANITATION SERVICES? Having explored the appropriateness of focusing on market-based approaches in EQ1, the response to this second question focuses on the implementation of the market-based strategy that the KIWASH IP has adopted to expand access to sustainable water and sanitation services in targeted counties. The response first considers KIWASH as a delivery agent, the effectiveness of its county-centric implementation structure, and strength of collaboration with other DPs at the national level. Then, looking at water and sanitation separately, the wide range of approaches and tools KIWASH uses to deliver market-based services are described. MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 22 4.2.1 KIWASH AS A DELIVERY AGENT Interviews at national and county level confirm that KIWASH operates primarily at the county level, as designed. In seven of the nine counties, KIWASH’s approach to embed four full-time staff in county government offices working alongside county officials on a day-to-day basis provides good insights and access. 22 Each of the seven counties has a dedicated WASH Governance Specialist and Service Provider Capacity Building Specialist. Four WASH Infrastructure Specialists are designated two counties each. Six counties have dedicated Sanitation and Hygiene Specialists. A number of other specialists (WASH Finance, Community Outreach, Organizational and Gender, Utility Finance and Credit Risk) have one dedicated staff member based in Nairobi who provides support across the project. This approach appears to work well with on-call support available to the KIWASH county-level staff. Relationships are generally very positive with appreciative county partners. KIWASH staff have become trusted and go-to members of the county team. Although not verified directly by the evaluation team, a small number of national level KIWASH staff members expressed an opinion that staff in some counties are perhaps too close to government officials and thereby risk being coopted or losing some objectivity. KIWASH is mindful of this and of the need to ensure that KIWASH staff do not substitute for what local government staff should be doing as part of their principal responsibilities. As with all programs that embed staff, KIWASH needs to develop an exit strategy such that their activities and approach can remain intact after the project finishes. Indeed, work at the county level was described as “well structured”, anchoring effectively in county government. KIWASH brings solid engagement with emerging WASH Forums at the county level, which they helped to establish, structure and facilitate in many instances. As described by one DP, ultimately the success of staffing at the county level is based 20 percent on technical knowledge and 80 percent on forging effective relationships. As confirmed by county-level counterparts during the visits, the KIWASH staff in the counties are seen (as a composite of impressive attributes noted by various interviewees) as professional, approachable, passionate, and knowledgeable (with regard to technical aspects as well as local politics) and bringing a strong team spirit that is infectious for county government staff and other counterparts. The evaluation team fully agrees with this assessment. The embedded staff embody the customer care mantra that is a significant part of the KIWASH training modules, with a “proactive and participatory approach.” KIWASH staff and processes are seen as organized and methodical with clear timelines for activities. “You need to be ready to work with high accountability. There are no shortcuts with KIWASH,” noted a WSP staff member. One county-level partner notes that, “the quality of their work is different. [They put] proper systems in place to manage the development of infrastructure.” Similarly, as stated by a separate county￾level partner, once commitments are made and funding has been assured, “you can plan with KIWASH. If we agree a 20-month program of work, that is what we work towards.” As noted by one CHV in reference to ODF triggering activities, “This has been a new experience for me compared with others I 22 Work in Nairobi and Kisumu counties is led by KIWASH staff based in standalone project offices that also are responsible for regional oversight and personnel management. The Kisumu office is a short distance from county offices and there is regular contact between KIWASH staff and county officials. KIWASH staff are very responsive: “Even if they are passing by, they just stop by to check in.” (WASH Enterprise - Kisumu) “You can plan with KIWASH. If we agree a 20- month program of work, that is what we work towards.” County-level partner MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 23 have worked with. You have to plan your work by procedure… You don’t do things hurriedly. You have to do it in stages with a plan.” There are some relatively minor shortfalls with the perception that KIWASH staff are more inclined to water than sanitation and thus a bit slower on the uptake of sanitation work. Familiarity with related activities of other DPs often seemed limited in the interviews mainly at the national level (although this is perhaps changing with the exposure through the WASH Forums). While scoping may have occurred at the outset of the project, further familiarization around what others are doing might be helpful to avoid duplication of efforts, coordinate approaches, and build on successful models. Although perhaps reflecting a more robust and thorough set of procedures than partners may be used to, some concerns were raised around KIWASH approvals for activities, delayed payments to CHVs, and unsent certificates to show participation in training. “They have a lot of bureaucracy issues so approvals for things take a lot of time. Procurement of even simple things takes time,” noted one respondent. The unpredictability of whether activities will be funded and in a timely manner undermines confidence in KIWASH’s ability to deliver and was noted by some respondents as causing them to waste time on proposals that may or may not get funded for reasons that were not always clear. Indeed, the proposal stage “is a required step in the identification, evaluation, selection and implementation of infrastructure projects,” and not every project can be funded. That said, some clarity was needed to frame the process more for partners. 4.2.2 PERCEPTIONS OF KIWASH AS A PROGRAM Expectations at the outset were difficult to manage as county governments, many new to USAID contract-based project delivery mechanisms, essentially divided the amount of funding advertised by the nine named participating counties23 and assumed that US$5.6 million would be coming to each of them. It appears that these initial tensions have dissipated, helped also by the arrival of a new cohort of county government officials after the 2016 elections. High expectations also needed to be managed after the launch at the national level of the US$51 million program. Admittedly many organizations wanted to get access to some of the funding. According to one DP, “KIWASH was launched on the platform of being a major sector catalyst with the promise of sector transformation leading to a significant increase in access to improved WASH services. It was expected that with big money and smart strategies, KIWASH would unleash major impacts in the target counties.” While clearly a major player in the target counties (apart from Nairobi), the general consensus is that KIWASH has “not much visibility or influence at the national level” with national-level DPs describing a mixed level of understanding of KIWASH’s approach and activities. KIWASH has no national level project indicators and most of its interactions at Ministry and DP levels are managed directly by USAID. National-level actors were keen to learn more about what KIWASH is doing and learning, suggesting that USAID could be more proactive in promoting KIWASH activities and facilitating KIWASH participation at the national level. As an integrated program, national level actors suggested that KIWASH could play a useful role in, for example, supporting: the development of an overall sanitation sector financing strategy, the department of sanitation services within the Ministry of Water and Sanitation, and “facilitating national dialogue on the intergovernmental functional linkages and division of labor between different government agencies and levels.” This is clearly beyond the 23 Per Section C of the contract (p. 19), “USAID/Kenya selected these priority counties based on an analysis of multiple factors that will enable the Mission to address the high priority needs within the WASH sector, while at the same time optimizing the opportunities for synergy with other USAID‐funded development activities in related sectors.” MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 24 scope of the project as currently designed, but could be worth considering for the final period of the project and certainly factored in to future programs. In terms of design, DPs saw a substantial budget but then noted the rather minimal targets for some activities (e.g., 25 villages per county reaching ODF status, which KIWASH already well exceeded). The target counties are not the poorest or worst performing in terms of service coverage levels and so there is a sense that USAID through KIWASH was “targeting low hanging fruit.” There is some understanding between USAID and KIWASH that the strategy involves experimenting with models that can be methodically adapted and replicated in other counties, but this does not seem to have been explicitly addressed by KIWASH to date. Given KIWASH’s emphasis, there was some surprise that they have not been involved in national policy discussions such as the national Sanitation Bill, being validated at the time of this report, or, as noted above, that USAID was not representing KIWASH’s work and learning more effectively. KIWASH has no national level targets and limited contractual remit to operate or be visible at the national level. For some DPs, this has resulted in a general sense that the KIWASH approach is to work alone in terms of project implementation with a “don’t call us, we’ll call you…” attitude. Seeing the significant sum of funding involved, some DPs sought out KIWASH in the early days to see if they could leverage KIWASH funding for their own activities. While maintaining its focus, there might have been scope for wider partnerships. One DP described the importance of intentionally seeking collaboration within the wider WASH network stating that, “five years is a short time to do anything leading to fundamental transformation of complex systemic, structural and behavioral issues and practices associated with WASH.” As such, some suggested that KIWASH could be more plugged in at the national level, using partnerships to coordinate both messaging and approach. Coordinated messaging would help leverage greater influence and not to “confuse players in the market” further. Efforts could have been made at the design stage to highlight this aspect and there is undoubtedly a conversation that could be had between USAID and KIWASH about whether it is necessary at this stage and, if so how best, to overcome these perceptions. At the county level, one KIWASH staff member noted that KIWASH would want to see different DPs promoting the same messages with government officials to help ensure coordination rather than competition. The emerging WASH Forums will no doubt help at the local level with shared work plans emerging (in Kakamega at least) and DPs becoming more familiar with each other’s activities and contributions. One DP suggested that, “it is not a problem that they are less familiar with KIWASH work as it should be the responsibility of the utility [or the county] to coordinate and keep Development Partners aware of others’ activities.” Being a consortium of a prime and subcontractors, KIWASH is working through partnerships with SNV primarily focused on sanitation, Big Five Africa (BFA) primarily focused on finance and governance, and EXP primarily focused on behavior change, and having the subcontractor hire staff who then get seconded to work under the KIWASH umbrella. While blended staffing is a common approach to implementing bi-lateral programs of this size, interviews with subcontractor staff suggested that early on perhaps KIWASH had not taken sufficient advantage of the knowledge and wider expertise that the subcontractors brought from across their largely non-USAID portfolios. Managing a complex program does indeed require clear leadership and definitions of roles and hierarchies. That said, the consortium partners work in a diverse set of programs and projects that could inform the KIWASH approach. “Five years is a short time to do anything leading to fundamental transformation of complex systemic, structural and behavioral issues and practices associated with WASH.” (DP interviewee) MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 25 4.2.3 KIWASH’S APPROACH TO WATER SUPPLY SERVICES As noted in the response to EQ1, the KIWASH strategy focuses predominately on strengthening the governance, management and technical capacity of the service providers (both at the utility and the enterprise levels) with which the project has been working. A key element of the approach has been on changing mindsets to emphasize that service providers, large and small, need to operate more as a business (i.e., through market-based approaches). KIWASH has also been working with service providers in a more direct way to expand coverage. According to the KIWASH Year 3 Annual Report, such efforts have resulted in over 430,000 people gaining access to water services (slightly half of the 786,000 five-year target). While the evaluation team spent some time during the county visits seeking to understand KIWASH’s work with the WRUAs, these were found to be generally unrelated to market-based services and the focus of the Mid-Term Performance Evaluation questions. As such, this line of work is not addressed in this report. The work is much appreciated at the local level. That said, while not exploring in great detail, the evaluation team found that the linkages between the interventions with the WRUAs and those with the WSP and WASH enterprise work were not clear. KIIs and FGDs revealed that there were both limited geographic overlap and familiarity between the two different types of organization, a missed opportunity for KIWASH to connect elements of the project. Support to Water Service Providers Since its inception, KIWASH has provided a wide range of support to WSPs in the target counties. Support is comprised of a series of integrated components that include policy and strategy support, capacity building, technical support, grants for refits, and mentoring. From interviews with KIWASH staff at the county level, KIWASH made significant effort at the outset to understand the primary issues facing the utilities and the evolving context under which they were operating as a result of the transitions in authority and responsibilities mandated by the Water Act. This understanding that all WSPs are not at the same point in their development helped to frame and tailor KIWASH’s offer to WSPs. As a result the feedback the evaluation team received in interviews was overwhelmingly positive. Strategic Planning and Policy: Interviews with WSPs highlighted the support that KIWASH provided in developing five-year strategic plans. This effort focused on the mission of the WSPs and the short-term activities and Key Performance Indicators needed to achieve it. In some cases, KIWASCO and Gusii Water and Sewerage Company (GWASCO) were mentioned, these were followed by investment plans and financing proposals to help the WSPs to mobilize funds, including from county government. KIWASH has supported WSPs in developing internal management and operational policies including for gender mainstreaming, metering, human resource management, procurement and financial management, amongst others. The project also has supported structural changes, convincing at least one utility (Kitui Water and Sewerage Company [KITWASCO]) to separate the finance and commercial/billing departments for greater transparency and effectiveness. Training and Capacity Building: A significant part of the training and capacity building support offered by KIWASH has been in response to the shifting roles and responsibilities of county governments under the new Water Act. According to interviewees, KIWASH effectively delivered training for county staff and new board members on utility organizational structures, the role of directors/trustees to fulfill strategic oversight functions, how utilities operate, the code of ethics for directors, and other related topics. According to one WSP interviewee, one clear benefit has been that the training helped directors to understand the importance of hiring qualified people to manage the different departments in the utility, rather than bringing in “friends and family”. MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 26 (Kisii is a case in point where after the governance training, the existing board dissolved and a new board was formed.) Although anecdotal, this appears to be slowly translating into an emphasis on hiring qualified suppliers as well. Other wider considerations related to effectiveness also arose from these discussions. Given that county governments subsidize the energy bills for the companies, a new interest has emerged to save money by ensuring utility efficiency and effectiveness, which stems from good governance, (e.g., good management, incentives, and performance management, among others). While this has resulted in a better appreciation of the requirements of operating a utility more generally, a DP provided a key example of progress regarding Kakamega. “In terms of attitudes and management, something has really changed and come together – the utility has fully turned around in the past few years. This has been interesting to see as there has been a lot of attention to governance from our side and from the KIWASH side.” DPs appear to be clearly and intentionally reinforcing the same messages in their engagement with Kakamega County and utility officials. Given the shift in utility management towards performance-based contracts, there is more emphasis on delivering as departments and individuals in the utility. The introduction of performance targets and appraisals in some of the companies appears to have motivated staff to engage more effectively in delivering on their responsibilities. The timing has thus been right to introduce a series of training activities that support employees in fulfilling their duties. Training topics for senior management and staff have focused on business models, customer care, gender mainstreaming, information and communications technology use, and motivation in the workplace, as well as technical topics like NRW and water quality. Training has helped to clarify the best methods to deal with both commercial (e.g., non-payment, illegal connections, and even cartels from within the utility) and technical losses (like burst pipes and identifying hot spots where challenges frequently occur). Benchmarking tours to well-performing utilities have been instrumental in teaching utility staff on how to improve performance, including investing in Majisoft, a billing software used in Nyeri. 29 staff from five Kitui and Makueni WSPs participated in a benchmarking visit to two of the best-performing WSPs in Kenya (per WASREB ranking tables), Nyeri Water and Sewerage Company (NYEWASCO) and Embu Water and Sewerage Company (EWASCO), with the goal of understanding strategies used in non￾revenue water management and efficiency in general management. Some utilities appear to be producing more accurate data on NRW, coverage figures, and other operation and management functions as a result of KIWASH and other DP support. That said, this increased accuracy in data collection has adversely affected the utility’s ranking in the WASREB annual review. The proof of impact will be whether in the coming years these same utilities improve their performance. According to WSP interviewees, the key is to follow (or help shape) utility perceived priorities to ensure greatest application of training efforts. Several interviewees noted that capacity building has been KIWASH’s most significant and practical contribution to utility performance. Although articulated in many ways, these activities have been seen by many as changing mindsets toward an understanding of the need for a more viable approach through a commercial or business-minded attitude to service provision. “Training is only one thing. Systems are what matter… I took [the WSP staff] for benchmarking in Nakuru and the Nakuru team was telling them they did it with their own money. They were so impressed that they just copied Nakuru and when they came back they 0were telling me about automating their systems, E-payment modes, E￾billing, E-meter reading, and so on. They even have smart meters now.” (KIWASH staff member) MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 27 Leveraging Finance: KIWASH seeks to leverage financing for water sector investments, e.g., through help in preparing proposals to the Water Service Trust Fund and commercial banks. KIWASH has made significant efforts to this end including supporting Nairobi City Water and Sewerage Company (NCWSC) in holding their first ever Sanitation Sub-sector Investment Forum, which was co-organized with USAID/WASH-FIN. NCWSC developed a proposal of KES 3-4 billion for loan financing through the African Development Bank (under review at the time of this report). KIWASH initiated discussions with NCWSC on possible partnership opportunities for a Water Operator Partnership (WOP) program. As mentioned above and per the Year 3 Annual Report, technical support was provided to GWASCO to “formulate a concept paper for a proposal to be presented to the County Government of Nyamira for a NRW project.” The Annual Reports provide other similar examples where KIWASH supported WSPs in Busia, Migori, Kisumu and Kakamega counties with related interventions. Providing support to develop tailored investment strategies that stem from upgraded business and strategic plans has been an important KIWASH contribution. Again it is early for impacts to be seen, but, based on a number of KIIs, the shift in mindset as a result of these interventions appears clear. The proof will be in the success rates of the proposals prepared. Technical Support and Grants for WSP Extension and Refits: KIWASH has provided technical support through direct investment for infrastructure as well as grants for refits, largely aimed at delivering savings to WSPs by reducing NRW and energy costs. These are documented in the annual reports with investments including: • Undertaking feasibility studies and engineering designs on proposed water infrastructure projects, • Test pumping of boreholes to establish reliable yields, • Pre-construction water testing to establish water quality, • Construction of water intakes and installation of water pumping systems, • Construction of main water transmission pipelines, • Construction of water distribution pipelines, • Installation of solar power units for water pumping, • Installation of water treatment systems, • Construction of water storage tanks, • Construction of water kiosks, • Installation of bulk and zonal water meters, and • Rehabilitation and improvement of existing water supply infrastructure. As an example, to reduce the monthly power bill for Kiambere-Mwingi Water and Sewerage Company (KIMWASCO) in Kitui, KIWASH provided a grant to procure an energy efficient high-lift pump and solar array. The new pump will save an estimated KES60,000 (approximately US$600) per month and enable the company to increase pumping from 16 to 21 hours per day. In addition to improvements needed to increase energy efficiency, support also is needed to help utilities better understand and address their NRW needs. KIWASH support in procuring meters to address NRW has enabled the WSP in Kitui, for example, to increase its monthly revenue from KES 8 to KES 10 million. Another county Director of Water noted that KIWASH provided zonal meters so the WSP knows how much water they are pumping and selling. In Kakamega, the utility links GIS meters to accounts so customers can be more easily identified. “They have even gone ahead and procured Enterprise Resource Planning software (ERP), [all of which] has enhanced their revenue collection. They were collecting between KES8-10 million per month. Now they have reached highs of KES22 million in collections.” A KIWASH county coordinator reported that meter testing equipment had been bought but not yet been used. KIWASH planned to do on-the-job training by establishing a pilot project on NRW so staff can be trained and then start monitoring effectively. Thus attitudes to NRW have changed markedly in recent years with more understanding of its importance across the MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 28 different departments of the utility. One DP noted that, “for KIWASCO, every Monday morning they start with a full staff meeting at 7:30 and every department has to give a small update on what has happened in the past week and NRW is always a big topic… So they are talking through the key performance indicators… every single week for at least a year.” While the support is certainly welcomed, the decision to support the purchase of metering or other related equipment does not appear to be consistent or at least the logic as to when KIWASH would or would not support such procurement has not been clear to the evaluation team. WSPs also noted that the KIWASH policy was not clear on this aspect, thinking that KIWASH would only fund “existing projects.” Several noted that they had spent significant amounts of time preparing proposals for KIWASH without clarity on what would be funded and what would not. While there is significant appreciation for KIWASH’s financial support, some communication gaps remain around the competitive nature of the process that interviewees flagged as unhelpful. A Focus on Customer Care: A primary KIWASH message that resonated with all service providers with which the evaluation team came into contact, stemmed from the notion that the viability of service providers lies largely in customer care. In other words, a viable business is one in which customers are treated with respect and their issues are dealt with promptly or a reason is provided to explain any delays. With a view to financial viability, this allows the business to foster a culture of payment for service, which increases revenues that allows for service improvements and ultimately expansion. An Overarching Emphasis on Service Provider Creditworthiness As a middle-income country with limited or modified forms of development assistance expected in the coming years, numerous efforts in the water sector revolve around creating creditworthy service providers. Interviewees began to suggest a way of looking at creditworthiness that included a series of component parts. In the first instance, creditworthiness requires a level of transparency where funders could “trust the numbers”. The performance that is reported needs to tally with the reality on the ground (with regard to staffing, customers served, and NRW calculations, among others). This dovetails with the more accurate reporting noted earlier in this report. With regard to finance specifically, several interviewees noted that financial reporting is not yet standardized particularly around how loans are captured on the books. As such, new approaches to financial modeling from the World Bank and Deloitte should reportedly help WSPs to log debt-service ratios more clearly as it gets rolled out to more utilities. Ultimately, some question whether utilities are still giving numbers that they think DPs want to hear (either because it shows neediness or creditworthiness, depending on the funding formula). As one interviewee from a commercial bank noted, “utilities are very slow to come back with information [and when they do] it is often incomplete,” which does not bode well for boosting a financier’s confidence. Creditworthiness also suggests responsiveness and whether service providers have the governance elements in place to respond sufficiently to risk and opportunity in a timely fashion. Responsiveness also suggests that the service provider has a plan in place to ensure sustainability of its services and a credible expansion plan to meet the needs of all in its service area. Having solid management in place, with decreasing losses (NRW) as evidence of one potential proxy and the ability to fend off county government interference as another, was also seen as an important element of creditworthiness. MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 29 This emphasis has been very much appreciated across the board. As one WSP Managing Director noted, “[we have] seen positive results from positive customer experiences – bill collection is on time and customer service exit interviews are greatly improved.” One KIWASH staff member had a direct experience that appropriately sums up the approach and the emphasis on “downward accountability.” “A customer came in to complain and found me and it was an eye opener... So I decided to focus [KIWASH efforts] on the commercial department, which is where you interact with your customers. You represent the company so you have to carry yourself with decorum, professionalism and respect. The branding, the corporate management, communications all had to change. Even when you go to Facebook you will now find managers there responding to the issues of the customer directly. The billing has improved greatly, and this is evident on the social media with customers saying the billing is fine, even if the supply is a problem still. We have also done customer satisfaction surveys and are doing another one this year.” (KIWASH Staff Member) According to WSPs and DPs alike, an outgrowth of customer care also means encouraging staff to understand the cost to the company of losses stemming from NRW, both in terms of physical as well as commercial losses. This then translates into technical fixes that look to construction specifications and standards, anticipating maintenance requirements, metering issues, and other aspects, as well as addressing billing losses. It also means addressing small-time corruption that relates to meter tampering at the customer level or even dealing with staff who have created their own customer base. KIWASH training and support has helped utility staff to understand the secondary effects of poor customer service and clear ways to address these issues. Support to Water Enterprises With a contractual target of 200 enterprises to support (20+ enterprises per county), a set of criteria was needed to determine where KIWASH could have the greatest impact. An early landscape assessment of enterprises operating in KIWASH counties found the type (non-governmental, community-managed, private entrepreneur), size and capacity of the enterprises varied significantly. (Far fewer were found operating in sanitation related services.) Most enterprises were fairly challenged in terms of their ability to manage a sustainable business that covers operations and maintenance costs (at a minimum). Criteria for receiving KIWASH support included: 1) those who appeared to have the ability to sustain the gains made through the project interventions, and 2) those that could expand because of the potential size of their market and due to the capacity of the water source over which they ideally had control. Ultimately 231 WASH enterprises were chosen to be included in the program in two phases (107 in the first group and 124 in the second group). Most are small self-help groups and community-based organizations. Some are private entrepreneurs who had drilled a borehole with their own money. Most do not have more than three staff members – a field person, a billing clerk and a project manager. Of these, 25 to date have benefitted from KIWASH infrastructure investments. KIWASH conducted a needs assessment and gap analysis. From that, KIWASH developed a training curriculum and a program to support selected enterprises to become legally registered so that they could apply for other sources of funding (commercial or otherwise). The initial three-day training with two follow-up training sessions covered a wide range of themes including: • Governance and oversight; • Strategic planning; • Compliance with laws on service provision and standard operating procedures; MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 30 • Financial management (such as billing, income and expense control, and reconciliation of income and expenses) and record keeping; • Customer care: know your customers and customer service • Marketing and business planning; and • Accountability: “talking wall” that summarizes in poster form the operational data needed “to keep the enterprise focused” (for example, by showing the amount of money billed and collected month by month). Training and capacity assessments through pre- and post-training tests allowed KIWASH staff to see who was committed by how they engaged and how long they took to develop plans. It also provided insights into the enterprise’s management structures, levels of infrastructure and physical challenges (like borehole capacity). This led to an understanding of what kinds of support were required (minimal capital investments or major investments, significant governance work, etc.) to ensure greater viability. One utility, KIWASCO in Kisumu, noted that much of the training materials from KIWASH are being incorporated into the KIWASCO core training package that revolves around how they work with DMAs. The trainings brought enterprises together with others operating in similar ways so they could compare notes and learn from each other. In Kisumu, supported enterprises have created a WhatsApp group to stay in touch. A simple monthly monitoring tool, primarily for those with the ability to track water volumes produced and sold, aims to help the enterprises and KIWASH to plot and track progress. This includes the number of breakdowns and complaints so informed decisions can be made on any remedial measures to be taken. Much of this information had not been previously logged. These reporting tools also helped KIWASH to then tailor its coaching and mentoring to the specific and step-by-step needs of each enterprise. KIWASH staff have dedicated significant amounts of time and energy to supporting these enterprises and as a result engendered enormous good will. As different enterprises stated, “they come 2-3 times a month” and “I talk to KIWASH almost on a daily basis.” KIWASH gives assignments and expects the enterprises to follow through. Yearly assessments are then meant to be developed by KIWASH with the enterprises to capture their progress. This focused attention has yielded clear benefits: “We now have a basis for growing due to the investment KIWASH has made in us. We engage in workshops and trainings sponsored by KIWASH. We are now able to dream, and we are seeing the results of this. For example, in 2009, there were only 29 direct connections – imagine that low number to the 283 we are now.” (WASH enterprise, Kisumu) Interviewer: So in looking at your books further, I can see that in 2016-2017, there was a revenue increase by 300 percent. Enterprise: Most of that was in connection fees. Then because we were meeting demand, the customers we already had started paying their monthly bills and their outstanding bills. This led to the increase in revenue from KES 3m to KES 9m. The customers just started receiving water regularly and so they felt that they could pay as they continued to use the water. KIWASH really helped us identify the issues we had… We could not even quantify how much water was being consumed monthly. In some cases, enterprises requested further training to deepen their skills in certain areas. One mentioned that it would be helpful to be trained to recognize when contractors use substandard tools. Several mentioned software that would help them to more effectively manage the enterprise. The quality and appreciation of the training notwithstanding, a number of enterprises noted that capacity building is not enough, and finance is actually required to improve and expand their services. For some this means increasing their storage capacity or the capacity of their MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 31 boreholes. For others, it means investing in solar energy to reduce otherwise significant energy charges for pumping. In some cases, county governments have taken an interest and expressed interest in supporting schemes with solar power. Based on per capita cost, expressed need, and perceived competence, some enterprises have qualified for KIWASH recoverable grants (described in Section 4.3.1). The typical approach has been to blend direct infrastructure investment with recoverable grants based on the ability to repay. Where possible, KIWASH has also sought outside support. For example, KIWASH collaborated with the Dow Foundation to offer solutions to the challenge of high fluoride content in certain water sources and secured technical and financial support from Dow for two target water schemes: Little Sisters of St. Francis in Kasarani, which will have 7 cubic meter of water treated per hour to serve a hospital, schools and the neighboring community; and Geoseismic, a private WASH enterprise operating a borehole that will treat 2.5 cubic meter of water per hour to serve more than 3,000 households. For both projects, Dow Chemicals will invest US$ 160,000 ($115,000 for Little Sisters’ project and $45,000 for Geoseismic) to install fluoride treatment rigs and support monitoring, evaluation, and capacity-building activities. KIWASH proposed a modification to the budget for this kind of support, which was approved in Year 3. Funding for infrastructure and grants thus increased from US$ 1 million in the original budget to US$ 8 million. Discussions with enterprises revealed some confusion around the criteria for which a bid for KIWASH funding would be accepted and thus expectations or promises have not always been met. One enterprise stated that, “maybe KIWASH wants to see if we can manage our finances first.” Several mentioned that they would be happy to get equipment like meters on credit, because “no metered water means that we can’t track water wastage.” 4.2.4 KIWASH’S APPROACH TO SANITATION SERVICES Apart from a few vacuum truck service providers being supported through the WASH enterprise work under Output 1, KIWASH’s programmatic focus in sanitation has been on rural provision, justified by the absence of contractual targets for safely managed sanitation services. According to the Year 3 Annual report, efforts to date have resulted in just over 57,000 people gaining access to basic sanitation services. KIWASH staff express confidence in their ability to hit the five-year target of 237,000, but there is work to be done to meet this goal. Internal strategy documents on CLTS and sanitation marketing produced in September 2016 describe the project’s sanitation efforts as concentrating on expanding CLTS through the community level workers in the Kenyan public health system and gradually expanding into sanitation marketing after initial market surveys and supply chain mapping. Neither document explicitly describes how the CLTS and sanitation market interventions would be layered and sequenced although multiple county-level respondents explained that the KIWASH approach was to start with CLTS and SBCC, and then move to sanitation marketing as part of a post-ODF strategy. The evaluation team found that KIWASH’s sanitation programming was late in starting. While according to the results framework (Figure 1) sanitation work falls under Output 1, much of the project’s early focus was on WASH-Agriculture-Nutrition Integration under Output 3, of which sanitation was only mentioned through a package of SBCC messages delivered by CHVs. A dedicated Sanitation and Hygiene Leader with specific expertise in rural sanitation joined the project based in the Kisumu KIWASH Regional Office did not join the project until August 2017. Findings from interviews and the document review suggest it took nearly a year for KIWASH to introduce CLTS, work with subcontractor EXP to develop sanitation-related communication materials, and engage with county and sub-county Public Health Officers (PHOs) across eight counties, excluding Nairobi, to begin work in this area. Momentum for the sanitation work notably accelerated in October 2017 when USAID withdrew Feed the Future funding from the project, precipitating a re-organization and staff MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 32 changeover whereby Nutrition Specialists embedded in county government offices were replaced by staff focusing entirely on Sanitation and Hygiene. Raising Awareness and Promoting Demand: KIWASH’s support to the environmental health and hygiene policymaking process appears to be moving well in Kakamega, Migori, Busia and Nyamira, where policies and bills are on track to be passed sometime in 2019. In part due to KIWASH’s advocacy, Kisumu County included a budget line for sanitation in the 2018 budget and the County Executive Committee is pushing for this at sub-county level. County-to-county peer pressure also has played a role in increasing the prioritization of sanitation by county leaders. As one county official notes, “immediately when Siaya became ODF, we decided we needed to have a comprehensive plan for ODF.” At the national level, while KIWASH has supported attendees from its focus counties to attend national and regional-level validation workshops supporting the Environmental Health and Hygiene prototype bill drafting, senior KIWASH representatives have been largely absent from these discussions, due in part because of strained relations between USAID and the Ministry of Health. PHOs, WASH Officers and other sub-county staff interviewed were extremely appreciative of KIWASH’s efforts in rural sanitation, particularly demand generation through CLTS. According to both KIWASH reports and county-level interviews, the rate of ODF transformation (from triggered to verified ODF) in the last two years has been high in all counties and close to 70 percent overall (although it was beyond the scope of the assessment to verify ODF claims). The enthusiasm shown by the PHOs in the sub-counties sampled suggest that they have seen tangible benefits of KIWASH’s work in this area. The UNICEF representative in Kitui recognized that Kitui West is UNICEF’s strongest team because of the partnership with KIWASH. “KIWASH has exhibited quality implementation in terms of delivering results. KIWASH’s capacity building sessions contributed as well as their M&E process.” The evaluation team registered a potential concern with the sustainability of the CLTS work, not in KIWASH’s approach, but the GOKs dependence on CHVs for follow-up and reporting without dedicated resources from county government to do so. FGDs with CHVs suggested they are overloaded with work. As one CHV described, “you have to go to every single patient and ensure they have taken their medications. And that means taking some means of transport which you pay for. Sometimes you don't have the money so you have to walk very long distances.” Some CHVs noted issues with delayed payments of KIWASH travel allowances to collect the monitoring data as a demotivating factor. The CHVs that KIWASH engages are focusing on sanitation partly because it is the “theme of the day” with active, albeit limited, financial support from the project. As county government public health budgets are inadequate to pay CHV transport allowances for CLTS reporting, there is a good chance for CLTS reporting to become far less regular without project support. In at least one county the evaluation team found that KIWASH is promoting inclusion of a monthly allowance to CHVs as part of the public health bills process. In Siaya, Busia and Kitui, KIWASH’s support is transitioning to assist counties to consider post-ODF monitoring through the creation of a quarterly monitoring and reporting plan with CHVs. All counties expressed interest in CLTS activities being scaled up to cover more sub-counties. As described previously, CLTS was late to begin but has picked up pace over the past two fiscal years, with the project reportedly having triggered 931 villages in seven counties (excluding Nairobi and Busia) by the end of Year 3 (with roughly 70 percent, 643 villages, having achieved ODF status). County governments are receptive to CLTS and its success suggests it is an area to prioritize KIWASH’s approach has recognized that counties are in different places in terms of CLTS coordination and execution and has tailored their programmatic response accordingly. MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 33 over the final 20 months of the project along with post-ODF monitoring in the recently certified counties of Kitui and Saiya. From the outset, Nairobi County was an outlier as its urban landscape did not fit well with KIWASH’s rural facing strategy of CLTS and sanitation marketing. Initial thinking to adopt “Urban” CLTS in Nairobi stalled in the absence of guidelines for such an approach in Kenya (or globally). The decision ultimately was made in FY17 that KIWASH would not engage in any community-level sanitation work in Nairobi but focus on supporting policy coordination meetings with the county. This approach also proved challenging because of shifting political priorities and frequent county government staff turnover. One respondent noted, “There is a tug of war between them [Nairobi County] and the national government. The County feels partners have gone to support the national government.” While relationships with Nairobi County and the NCWSC are positive, KIWASH often “felt like it was caught in the crossfire.” Although there is openness to work together, KIWASH has for the time being reduced its time and energy commitments with the county. The story of KIWASH’s work in sanitation has evolved very differently in the other eight counties. The evaluation team understands that initially the strategy was to focus CLTS efforts on 25 villages per county, clustered around health facilities benefiting from the agri-nutrition work. This has since evolved to take a more sub-county focused approach. The evaluation team notes that KIWASH is effectively implementing CLTS “by the book” (i.e., according to GOK protocols) working through government extension agents (CHVs) and following government procedures for triggering and follow up, using standard government forms for monitoring and reporting, both to the project and into the national database. KIWASH’s approach has recognized that counties are in different places in terms of CLTS coordination and execution and has tailored their programmatic response accordingly. Across the surveyed counties, the evaluation team saw examples of where KIWASH has catalyzed the creation of county coordination platforms (e.g., Kisumu and Nyamira). In Kitui, KIWASH successfully coordinated with other implementers like UNICEF and contributed tangibly to the county’s recent ODF declaration. In addition to community level work, as part of Outputs 6 and 7, KIWASH is helping many counties adapt and adopt the national prototype policies and bills for environment health and hygiene. This work is described further in the response to EQ4. [See response to EQ3 regarding private sector approaches for an analysis of sanitation marketing] 4.2.5 CONCLUSIONS – EQ2 In general, the evaluation team was impressed with much of the work of the KIWASH operations team and, as noted, they come across as professional, competent and approachable. The positive response from stakeholders at the county level confirms this. However, KIWASH needs to be mindful of not substituting for what county government staff should be doing and then being careful about operating in a way that when they leave, their activities and approach can remain. An exit strategy is clearly needed. “We are just an investor that doesn't really need to get paid…You could over time tell a longer story about how these investments will make county allocations more productive in the future. So spend less on the electricity bill for the utility…and more on building infrastructure and expanding the market place. The twelve utilities we have worked with may not be able to close deals yet, but we will leave them at a level where in a couple of years they are able to.” (KIWASH Staff member) “Most of the things we are doing it’s because they keep on reminding us.” (WSP Managing Director) MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 34 In terms of the activities, a few aspects stand out. As noted, the roles and responsibilities of different stakeholders at the operational level are still being defined. KIWASH came at an opportune time and is well placed to influence a wide range of issues at the county level. Supporting the professionalization of service providers will go a long way in strengthening the sector. That said, the evaluation team suspects that more could be done with other non-county stakeholders to reinforce messages. The WASH Forums may be the place for these synergies to take hold. Ultimately much of the support from KIWASH helps the utility to manage its relationships more effectively with multiple stakeholder groups. WSPs can negotiate and coordinate with county and DPs from more of a position of strength and communicate more effectively with customers and suppliers alike. Staff also have a clearer sense of what impacts on the business and how best to ensure its viability (and thus safeguard their jobs). With time, these improvements will start to appear in the WASREB rankings and affect the creditworthiness of the utilities. Through the efforts of KIWASH and others to provide an induction, commercial banks are seemingly ready to lend. A number of aspects need to be clarified further, not least of which is the current debt-service ratios of the target utilities. KIWASH also has a role to play in helping to define the overarching long-term county and utility strategy with regard to the role of enterprises in the sector. The evaluation team notes that the work that KIWASH has been doing to build the capacity of water enterprises is outstanding, having a real impact and delivering as intended in the original design. More analysis, however, is needed to understand how the direct and recoverable finance was allocated and how many of the enterprises are now “investible” or bankable. Good work notwithstanding, having to reach the rather arbitrary 200-enterprise target may have tied KIWASH’s hands, spreading too thinly across too many disparate geographies. It is presumed that for the sanitation work, the scope understandably became more rural facing based on the presumed links to the nutrition work. The evaluation team notes the delay in starting sanitation-related activities, and the correlation in timing between dismissing the nutrition based staff and on-boarding staff dedicated to sanitation. It is unclear whether, as noted by some interviewees, this is due to USAID delays in staff approvals or to KIWASH administrative delays. Aside from the systematic and well-coordinated way CLTS has been rolled-out, the evaluation team questions the appropriateness of focusing exclusively on CLTS in the three counties where baseline OD numbers were less than five percent. (This is noted also in response to EQ1 above.) While consistent with national policy for each county to reach ODF status by 2020, the verification and certification process risks becoming little more than a “tick box exercise” in areas with such low rates. In addition, county governments have not followed through with commitments of public financing for sanitation to maintain momentum beyond the life of the project, which risks the sustainability of these efforts. The evaluation team saw no evidence that KIWASH specifically targeted villages in sub-counties that were known to have higher OD rates than the county-wide averages. This raises the question of the usefulness of traditional CLTS in these areas, particularly in the absence of a complementary sanitation marketing component to assist households in investing in more durable options to move up the sanitation ladder. Thus, KIWASH could have let the CLTS work play out in the few counties with high ODF rates while diving into market-based sanitation solutions from day one focusing on the small towns. Although there are not that many to work with, KIWASH could have been more intentional to make sure that the enterprise work was more inclusive of sanitation entrepreneurs from the start. MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 35 4.3 EQ3: TO WHAT EXTENT HAVE THE DIFFERENT KIWASH PRIVATE SECTOR APPROACHES FOR WATER SERVICE DELIVERY AND SANITATION ACCESS EXPANSION BEEN SUCCESSFUL? WHICH OF THE KIWASH IMPLEMENTATION APPROACHES FOR EXPANDING SUSTAINABLE WATER SERVICES DELIVERY AND SANITATION ACCESS APPEAR TO BE THE MOST PROMISING AND SHOULD BE PRIORITIZED IN THE REMAINING TWO YEARS OF THE PROJECT? Building on an understanding of KIWASH’s approaches to WASH service-delivery described above, the response to this question focuses primarily on interventions to prioritize in the remaining period related to the private sector. The evaluation team notes a handful of activities and focuses on evidence of whether these interventions have been (or have the potential to be) successful. As elements of the sections above are repeated, the sub-sections below should be treated as conclusions in their own right. 4.3.1 WATER SERVICES Building WSP and WASH Enterprise Capacity: In an effort to make them ready to access commercial finance, KIWASH has undertaken a number of activities with WSPs and WASH enterprises, as noted throughout this document. KIWASH has supported WSPs and WASH enterprises to develop strategic plans with a list of overarching activities aimed ultimately at mobilizing funds to further strengthen and expand their coverage. These would then be translated into business plans that could be presented to potential funders. The business plans form the basis for Performance Implementation Plans for specific components and areas to redress to ensure that further funding would be used well. From this sequenced process, the emphasis then is on supporting WSPs and enterprises to develop fundable proposals. This is a helpful and much appreciated set of exercises, which goes a long way toward strengthening emerging capacity in the utilities; focusing county governments and boards of directors on short-, medium- and long-term priorities; and channeling the contributions of other DPs more effectively. The work with the utilities is not private sector focused. The private sector angle on this set of interventions is to build capacity to engage more effectively with commercial finance. Continuing the work with county governments to ensure solid governance and oversight structures are in place as well as the enabling environment work will be crucial to setting a solid foundation for WSPs’ enhanced capacity and effectiveness. KIWASH’s role in connecting these different strands (strategic planning across the county, oversight and regulation, and budget allocations and finance) will make the difference between more successful and less successful counties in terms of sustaining and expanding services. The response to EQ2 provides analysis on the wide range of activities KIWASH is undertaking to build capacity of WASH enterprises to operate more efficiently and effectively and to expand their reach. This set of interventions around small business development is critical to building an effective cadre of entrepreneurs, many of whom will likely take the skills into other areas of community development. Apart from the business planning and management, coverage rates and revenue generation, commercial banks are also looking for standardized recordkeeping to understand financial viability. As noted by an interviewee from a water enterprise, “KIWASH has been helping us to put our books in order so we can look at how to get money in future for more development.” The process of keeping accurate records serves many other purposes as well including to build up capacity of community members and private entrepreneurs to plan better, communicate more effectively, monitor and evaluate, and refer back to decisions taken. Recoverable Grants: KIWASH makes direct investments through a KIWASH traditional grant scheme for projects for WASH enterprises to solve immediate and essential needs. To build experience with and a formal history of borrowing, KIWASH also developed a recoverable grants program (i.e., short-term loans), often offered in combination with direct MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 36 investments for water system expansion. Recoverable grants have been offered to those enterprises that have been through KIWASH training modules and are “showing promise in terms of operating a market-based scheme.” The key target for this mechanism is enterprises “who are able to engage more knowledgeably in the design of the project that suits their needs.” Recoverable grants work like interest free loans that are re-payable once investments start to yield income. Projects eligible for recoverable grants must be income generating, like metering, or income saving, like solar panels, and must help to expand a paying customer base (i.e., helping to reach KIWASH targets) with costs below US$50 per capita. The recipients must clearly exhibit the ability to operate the system effectively once commissioned, thus a “judgment [of eligibility] is made based on how well they are doing on the capacity building trajectory.” The evaluation team understands that funds that are recycled back into the project will be used to support software, metering or other strategic investments for partners. The amount available for recoverable grants is not substantial (US$2.2 million of which approximately two-thirds has been allocated to date). Only a few have started to repay their loans, e.g., the Kithambangii enterprise in Kitui. It is early for the recoverable grants approach to show progress and impact. This component perhaps started a bit late due to technical and legal challenges in terms of administering a new mechanism. Although KIWASH issued the recoverable grants on the basis that they would all be repaid in time, only two years remain for enterprises to procure the supplier, construct the project, start to yield an income and then begin to pay back the loan. Presumably this approach is no longer viable at the four-year mark, given the decreasing timeframe for repayment. For WSPs, it is still emerging how recoverable loans fit under procurement rules for public entities. This was a stumbling block faced by KIWASH in establishing and administering this mechanism. As stated by one KIWASH staff member, such mechanisms “are a new concept, which takes a while for people to understand and in some cases a bit of bravery for them to apply.” Enterprises understand that the recoverable grant provides access to funds and helps them to establish a credit rating. Although offered in good faith that they will be repaid in full and monitored in that way, KIWASH also has taken the approach that the recoverable grants program is experimental and may not be repaid in all cases. That said, staff noted that “even if the grant is not repaid [in full], then the project has still been built and is thus providing a benefit” to communities as a traditional grant would have. Thus the risks are relatively minimal given the scale of this component of the project. In terms of feedback, the criteria for a successful application do not appear to be widely understood by all KIWASH staff and certainly not the applicants. The evaluation team understands that the funds are available on a competitive basis, but enterprises expressed a sense that KIWASH was overpromising on the prospects of such finance being available and not necessarily delivering. Nonetheless, the recoverable grants component appears to show promise as a mechanism for strengthening small business capacity in terms of more commercially oriented competence and establishing a track record of borrowing for service providers. Building Understanding of Commercial Banks: Through Big Five Africa, KIWASH has operated in an advisory capacity for commercial banks, effectively training staff from three banks (43 to date from ABC, Co-operative Bank and Family Bank) regarding the workings of the water sector. They also advised on new products as they are developed, particularly with Co-operative Bank with a half-day workshop, to make sure that they effectively meet the needs of the target borrowers. MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 37 “We’ve appreciated the work from KIWASH, especially when we came out with our internal water strategy for the board. We sat with them and they gave us data and insights into the water sector. They provided training on how to lend into the water sector... KIWASH has been one of our greatest partners in the water sector. It is out of these discussions… that we as a bank made the decision to focus on the water sector and see how we can grow and support the sector.” (Commercial Bank) KIWASH will continue to provide a loan portfolio review function for commercial banks. As appropriate under the KIWASH banner, KIWASH will seek to provide future support to facilitate relations after loans are signed, as requested by Co-operative Bank. (For example, in Kisumu, KIWASCO has only drawn half of the loan amount and so the bank is keen to try to move that process along.) In the remaining period, KIWASH will help to facilitate meetings between counties (Kitui and Makueni) and lenders to allow counties to talk about their vision and strategy for the sector. The evaluation team perceives that this level of effort is appropriate given the context described above in response to EQ1. This level of activity will help to position a few commercial banks with a greater knowledge as commercial finance becomes more of a prospect for the sector. 4.3.2 SANITATION SERVICES Facilitating Supply through Sanitation Marketing: Sanitation marketing efforts rely heavily on the outputs of significant efforts to create latent demand, such as through the targeted KIWASH CLTS activities in specific sub-counties (see response to EQ2). KIWASH began sanitation marketing efforts in earnest in early FY18, a full two years after the start of the project, with formative research on the demand side in three counties (Kisumu, Busia and Siaya), and a market supply study in eight counties of sanitation product suppliers in sub-counties where CLTS efforts were ongoing. Building on experience from the World Bank, UNICEF and SNV, the sanitation marketing work is presented to counties as a post-ODF activity. One collaborator described the sanitation marketing efforts as being “focused on equipping artisans with knowledge and skills on simple latrine construction so that we climb up the ladder from any type of toilet (bora choo) to the better type of toilet (choo bora), attaining flushing toilets with time.” Although not self-evident to the evaluation team, another county-level respondent felt that “the uptake of sanitation marketing will be quick now that ODF is realized in these communities.” KIWASH’s decision to delay sanitation marketing efforts and focus them in ODF communities is worth noting. KIWASH seemed to prioritize water in the early days, did not staff up the sanitation component initially, and likely saw CLTS as a safer initial investment. While the evidence on how to phase CLTS and sanitation marketing is thin, research suggests that sanitation marketing can and perhaps should be incorporated into CLTS (or related approaches to create and activate demand) as an integral part of the process as opposed to being thought of as a post-ODF activity. Recent evidence suggests programs introducing supply-side interventions years after triggering is largely failing to move communities up the ladder. 24 The team found that KIWASH settled on supporting two types of latrine technologies: the SATO Pan, a durable plastic interface sold as a pan or stool solution, and the SAFI latrine, an emptiable offset latrine model with a reinforced concrete pit lining and pre-cast interface first introduced in Kenya by KIWASH sub-contractor SNV. The extent to which findings from KIWASH’s formative research into consumer preferences for sanitation drove the decision to focus efforts on these models, or if these were low￾hanging opportunities because of SNV’s presence in the consortium and Sato being manufactured in Nairobi, is unclear. Furthermore, the evaluation team notes that a more comprehensive sanitation marketing strategy is needed that uses the formative research findings to: identify household 24 USAID (2018) An Examination of CLTS’s Contributions toward Universal Sanitation. Washington, DC., USAID Water, Sanitation, and Hygiene Partnerships and Sustainability (WASHPaLS) Project. MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 38 archetypes to target with certain products (either the Sato or Safi), understand household liquidity or affordability constraints that may need to be addressed to generate product uptake, and identify barriers to entrepreneur viability. For instance, a detailed design of the enterprise model being tested through the Sato or Safi models is not clearly articulated either in writing or in conversations with KIWASH staff. Project reports and interviews indicate that sanitation marketing efforts began earlier in Busia than other KIWASH counties. Busia was the first county to be declared ODF and other DPs have also been active in this space for several years. Although not visited during the review, the Busia work was mentioned twice by different DPs as an example of KIWASH’s progress in this area. According to KIWASH’s FY18 annual report, following a sanitation marketing training in Busia and Makueni counties, 173 households installed Sato pan latrines. As a result 865 people have access to improved sanitation. (The evaluation team was not tasked with validating this result.) In the last months of FY18 and early FY19, KIWASH began to train masons on the Safi latrine and establish local supply chains for the Sato products sold through local CHVs. Trainings had only recently occurred in counties visited during the review and it was too early to tell if sales would follow. In-depth discussions in one sub-county suggested that KIWASH supplied a single Safi casting mold for a team of 10 masons from different parishes across the sub-county, and that the mold would be kept at the hospital and borrowed for each installation. At this same location, the linkage between the masons and the CHVs, described by KIWASH staff to be one of a sales agent and a supplier, was unclear to the evaluation team when speaking independently to both groups. While the intention to collaborate to sell the Safi latrines was apparent, the mechanisms that would lead to a sustained revenue stream such that both parties would see value in the relationship over the long term was unclear. The sales price of the product was undefined as were mechanisms for service delivery and customer acquisition. Neither the CHV nor the masons group described focusing promotional efforts on a certain market segment or consumer profile that consumer research suggested was most likely to adopt the models promoted, most notably the Safi that could be characterized as a high-end item in the context of rural Kenya. To this point, one County PHO expressed concern that the Safi latrine, for which the substructure and squat platform cost approximately 17,000 KES alone is not affordable for most households. (The cost of the superstructure is assumed by the household and depends on the materials used.) The review did not uncover any evidence of KIWASH working on a financing solution for households interested in investing in a durable sanitation solution. While recognizing that KIWASH’s sanitation marketing component is still in its initial stages, and more generally that the project is operating in counties without established supply chains for improved sanitation products, it is notable that in the counties visited, the evaluation team saw no evidence that the Sato pans were yet available to the CHVs. Demonstration samples were not brought to any of the FGDs and CHVs did not have any marketing materials about the products and did not describe product pricing or where/how to source the products locally without project support. Discussions with KIWASH staff in one county suggested that the project has negotiated with SIL Africa to distribute Sato products at a discounted price to KIWASH-supported CHVs, which may pose a risk to the sustainability of the model once the project subsidy is withdrawn from the marketplace. Details on the delivery model KIWASH is using to deliver Sato pans through CHV networks and the pricing of Sato pans at various points in the value chain were not made available to the evaluation team. KIWASH’s early decision to initiate activities aimed at increasing availability of sanitation products as a post-ODF activity and forgo opportunities to explore MBS outside of CLTS-targeted villages has resulted in limited progress in this important area, especially in counties with low ODF rates at baseline. A nuanced and potentially more effective approach would have been to layer MBS work MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 39 with CLTS or other behavior change intervention from the start, intentionally targeting communities or peri-urban areas with low rates of OD. As it is, KIWASH’s work in sanitation marketing is just beginning and largely targeted in areas that have achieved ODF. KIWASH is wading into largely unfamiliar territory given the nascent state of rural sanitation markets, so expectations should be set accordingly. The project (and USAID) should anticipate some false starts and generally modest sales over the next 12 months and likely through the end of the project. A recent USAID/WASHPaLS global review of MBS suggests efforts such as these can be quite slow to pick up, with considerable experimentation through trial and error required before a project can expect an uptick in sales and market growth. 4.4 EQ4: TO WHAT EXTENT HAS KIWASH BEEN SUCCESSFUL IN BUILDING COUNTY-LEVEL CAPACITY TO IMPROVE SUSTAINABLE SERVICE DELIVERY? The KIWASH Theory of Change holds that building the capacity of relevant private and public stakeholders will contribute to accelerated and sustained improvements in water and sanitation access and services in the target counties. This is to be achieved through two key outputs: Output 6: WASH services and water resources institutions strengthened and appropriately governed, and Output 7: Target policy reforms advanced to stimulate and support access improvements. To achieve these outputs, KIWASH adopted a multi-pronged approach to ensure that WASH institutions are well governed by: 1) Working with county governments and WASH sector stakeholders to support the adoption of policy reforms, 2) Enabling and facilitating county WASH Forums, 3) Contributing to planning for WASH services expansion and improvements, and 4) Institutional strengthening through training of relevant county departments (as well as WSPs and enterprises as discussed in responses to EQ2 and 3). 4.4.1 STRENGTHENING LOCAL GOVERNANCE AND MANAGEMENT As paraphrased from the original SOW, KIWASH proceeded from the understanding that underpinning many challenges facing the Kenyan WASH sector is a lack of clarity around institutional roles and responsibilities, the capacity of institutions to fulfill responsibilities, and weaknesses in the governance of these institutions. As the agent responsible for ensuring the delivery of services, adequate institutional arrangements and capacity therefore need to be in place at the county level for the sustained delivery of high quality WASH services. As Koehler notes, “improving water service delivery begins with the perception of responsibility by those in charge of implementing legal mandates.”25 The initial context for KIWASH was a relative vacuum at the county level. As noted by one KIWASH staff member, there was a “lot of transition in the early days with [2017] elections and new staff coming in. We had to start over again after the elections.” In terms of knowledge of how the WASH sector works, county officials were “really starting from scratch with the governors’ offices [having minimal] understanding of water service provision, regulations and rules in the country.” Few counties had been able to “identify their needs, costed a road map, and invested the time and energy in 25 Koehler, J. Exploring policy perceptions and responsibility of devolved decision-making for water service delivery in Kenya’s 47 county governments. Geoforum: 19 Feb 2019. (Elsevier Publishing – received by email) There was initially a need to “get past the perception that the water companies were an easy source of revenue for the government.” (WSP Managing Director) MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 40 coordination mechanisms” – all necessary elements for any county-level program to succeed, according to one DP interviewee. Specific to oversight of the WSPs, there was initially a need to “get past the perception that the water companies were an easy source of revenue for the government” as well as to find ways to curtail micro￾management and county government interference whereby county officials would dictate to utilities, for example, saying “don’t pay this and don’t pay that.” As noted above, procurement was also an issue of concern with the sense that county government may drive unnecessary procurements as that is where the opportunity for corruption lies; “they just want to buy a bunch of pipes whether there is a need or not.” To do so, explained one KIWASH staff, the project “needed step-by-step to start with the board of directors of the utilities to codify procedures, processes and structures so that they were not at the whim of the politicians.” All that said, ministers at the county level want to be seen as having built something or at least to show progress. Thus, though not so straightforward in the early period, KIWASH was in place embedded in county offices at an opportune time to influence the way forward for the target counties. The document review further confirms this – that governance work has been foundational and explicitly part of the pathway for success for achieving indicators related to water (and to a lesser extent) sanitation provision. KIWASH has made considerable progress in this area. Specifically, in the context of the transition to devolution under the Constitution of Kenya 2010 and with regard to building county-level capacity, Section C of the contract suggest that KIWASH was expected to: • Improve the effectiveness of local governments in engaging and managing the private sector and otherwise developing strong sector partnerships, e.g., through improved regulation, contractual engagement, enforcement mechanisms etc. to complement the project’s market-driven service delivery approach. Indeed interviews revealed scant evidence of KIWASH having a significant impact on county governments’ relationships with the private sector. Effective as it has been, the work with the enterprises, for example, did not appear to be synchronized with county government capacity building as yet. It was not clear from the interviews that county officials were sufficiently familiar with how best to support, oversee, regulate, and incorporate WASH enterprises into their planning. Undoubtedly this is a work in progress and KIWASH is, according to the Year 3 Annual Report, “working closely with the county governments to ensure that the WASH enterprises that have received capacity building support are considered for further support from county budgetary allocations.” Otherwise, KIWASH has been supporting the creation of WASH Forums as a form of county-level knowledge sharing partnerships, but in fairness, the building blocks of the enabling environment (understanding the responsibility, and developing the plans and the roadmaps) appear to be the first priority. A further expectation was that KIWASH would: • Provide support to local governments to more effectively access funding from other financial institutions for water sector investments, e.g., help in preparing proposals to the Water Service Trust Fund and banks. Again this is a work in progress with KIWASH helping to mobilize just over US$15 million in a cumulative Year 3 target of US$24.65 million. Work is ongoing to achieve the target (see responses to EQ2 and EQ3 for analysis on the prospects of commercial finance), but KIWASH may very well be hamstrung by the context described earlier (including county governments needing further capacity building to understand their role, already overstretched borrowing, competition between DP MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 41 mechanisms, etc.). While reflecting the prospect of the Water Services Trust Fund as a clear signal with regard to finance when KIWASH was being designed, the realities quickly overshadowed what has been possible for KIWASH in the interim. Nonetheless, over the last three years, KIWASH has implemented various capacity building interventions targeting county policy-level practitioners and policymakers. KIWASH supported WASH taskforces to contribute to the water investment plans of the County Integrated Development Plans (CIDP) 2017-2022. The CIDP is a key county development blueprint that sets out the development agenda for a period of five years for the counties. Interviews confirm KIWASH’s instrumental role in this process both in terms of facilitating as well as providing technical and content support. According to the Kitui Director of Water Services, “We were operating without a direction before. Without KIWASH support, this would have dragged on and on.” KIWASH also contributed to the establishment of various county technical teams and committees drawn from relevant ministries to support policy development processes and provided short-term experts for support on policy and legislative processes. As one county partner noted, “counties are keen on further training on technical issues, particularly in how to oversee utilities, given their historical role in and thus greater familiarity with rural schemes.” Strengthening the governance of the utilities by educating the new WSP boards of directors on the roles and responsibilities of different levels and the necessary distinction between oversight and management has been a critical part of the project that interviewees suggest is starting to bear fruit. Discussions with WSP Managing Directors confirmed this. KIWASH has facilitated county level engagement in workshops, WASH Forums and conferences either directly or in collaboration with other stakeholders such as Kenya Water and Sanitation Civil Society Network (KEWASNET), UNICEF, and the Water Integrity Network among others. These engagements involved strategy development for reporting on the SDGs and for supporting the government to achieve specific targets; strategies to ending OD, hygiene and menstrual hygiene management, emergency preparedness, response and knowledge management; innovations to reduce fecal sludge in pit latrines; sector financing including the challenge of bankable WASH projects; and county annual development planning and priority investments in operation and maintenance of capital investment. Some of the forums have resulted in policy and governance reforms leading to budget allocation for WASH (e.g., in Siaya, Kakamega and Busia), capacity building of WRUAs and county WASH platforms. Regarding oversight and regulation of rural water services, as per the FY2018 annual report, KIWASH in collaboration with Caritas Switzerland supported 26 KIWASH staff and county directors for water across the nine counties to benefit from a Training of Trainers (TOT) workshop on the Integrity Management Toolkit. The Toolkit aims to improve the performance of rural water supply schemes and WASH enterprises, particularly in the area of WASH governance and accountability mechanisms. 4.4.2 STRENGTHENING MONITORING, EVALUATION, AND LEARNING Monitoring protocols that begin with a baseline and then routinely capture and report performance data to decision makers is a fundamental building block to deliver and expand sustainable service delivery. According to the contract, the KIWASH project aims to provide technical assistance including advice, support, and capacity development to improve mechanisms and processes for sector performance monitoring, use of data for decision-making, transparency of sector data, reporting systems, enforcement mechanisms and use of evaluations for learning. The document review found that KIWASH carried out stakeholder training on improved monitoring and evaluation, reporting and accountability for MOH across all counties and social accountability for M&E training for county WASH partners in 2018 in Kakamega and Siaya counties. In Kakamega County, MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 42 KIWASH co-convened a sector-learning workshop entitled “Social Accountability for Improved Access to WASH” and a workshop on “Transformation of the Kakamega County Water Services Provision Sector.” KIWASH co-convened these workshops to sensitize stakeholders (including the county governor, and executive and senior staff from WSPs) on county WASH sector plans, rural water institutional framework to make rural supply more sustainable, the county water bill and policy, WASH planning, budget utilization, and monitoring. Kakamega County also established a database of WASH actors and published and institutionalized an M&E tool and framework for WASH actors managed by the office of the county WASH coordinator. On peer learning and exchange, KIWASH supported 15 staff from Nyamira County’s health and water departments to undertake a benchmarking exercise with their counterparts in Kisumu County. The focus was on steps to institutionalize a county WASH forum, enhance data and information sharing among WASH stakeholders in the county, resource mobilization and tracking for WASH interventions to promote equity in allocation of WASH resources and using the WASH forum for effective lobbying and advocacy on policies and regulations that have implications in WASH service delivery. These are works in progress and KIWASH should develop a way of monitoring and documenting the impact of these activities to determine whether they should be replicated in other counties. 4.4.3 ADVANCING POLICY REFORMS The KIWASH project policy reforms focus is set against the backdrop of the new Constitution 2010, the Bill of Rights guaranteeing the right of every person to water and sanitation and the transition to devolution. KIWASH recognizes that while there is a strong constitutional and national policy framework in place for WASH services and water resources management, county governments are in the midst of developing their own water and sanitation policies, legislation and plans to govern locally, and oversee and provide services. Ongoing water sector legal and policy reforms at the county level are meant to provide the enabling environment for improving sustainable WASH service delivery. Decentralization has resulted in increased expectations of county government by the electorate to assume this responsibility. A change in planning culture and perspectives as well as an openness to more participatory and accountable decision-making has been required as a result. KIWASH aimed to strengthen the policy and legal environment for WASH services including supporting policy coordination to streamline and scale up development of services and facilitation of private sector engagement. To this end and as described in the preceding section, KIWASH and other WASH sector stakeholders have supported WASH sector policy and legal reforms through technical assistance, workshops, the creation of WASH Forums and advocacy at national, county and institutional levels. As one of the few national-level activities, KIWASH facilitated a meeting with WASREB to initiate the creation of a rural water management framework to facilitate investments in rural water services. KIWASH supported and facilitated board members and senior management staff of the Lake Victoria South Water Services Board to participate in a strategic planning workshop with a number of utilities to develop/finalize and launch their five-year strategic plans. KIWASH has participated in national level events, but is not expected necessarily to collaborate with DPs at this level and thus has focused its energies mainly at the county level. At the county level, KIWASH facilitated and supported several workshops and forums in various target counties to review, draft and/or finalize county water, environmental health and sanitation and environment bills and policies; provided technical assistance for the development of the Kitui County Metering Policy; and facilitated meetings with NCWSC on the development of a Public Private MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 43 Partnership Policy and Nairobi City County on the proposed Sewer Revolving Fund Policy. Barring any unforeseen obstacles, these are all expected to be adopted during the course of the project. While actively supporting the bills and policymaking process in other counties, perhaps most visibly, KIWASH has been at the center with the Makueni County Government in developing the Water Policy through a multi-stakeholder consultation process that has been labor intensive. While referring to national policies and the prototype bills from the national level (with WASREB guidance and building on the Water Act 2016), KIWASH staff interviews suggested that policies were started largely from scratch and thus KIWASH efforts to cross-reference from other KIWASH counties and working through short￾term experts has been critical in moving these county government documents forward. Interviews suggest KIWASH is well positioned and “able to push certain things through the county government red tape.” This has sometimes meant coming in to galvanize initiatives by others at the county level that have stalled for one reason or another, like the example given of the Water Bill and the Environmental Health and Sanitation Bill in Kisumu. 4.4.4 CONCLUSIONS – EQ4 KIWASH capacity building interventions with county government partners have created greater awareness of the responsibilities involved in overseeing, monitoring and supporting sector-wide WASH activities. KIWASH has made significant contributions at county level to WASH policies, laws and strategies to promote sustainable access to improved water supply and sanitation services across all the target counties (with the exception perhaps of Nairobi due largely to “management issues” and subsequent disruption in this environment described earlier in this report). Anecdotally from statements from county and county-level DP interviewees and based on interviewee feedback on WASH Forum meetings26, county governments appear to be more open to facilitating meaningful community participation in policy and decision-making processes, including priority setting, planning and project implementation. The general openness to the creation of WASH Forums further suggests this. The proof, however, will only emerge in the coming years, and how influential KIWASH activities have been will need to be revisited. Despite an appreciated set of activities and the recognition of KIWASH as a respected and trusted advisor at the county level, interviewees and the document review did not reveal a comprehensive and deliberative countywide capacity building strategy for the WASH sector. At the county level, apart from the work with the WSP board of directors, an institutional capacity building program of activities was not clear or seen as sufficiently strategic in its overall purpose especially in how it would result in improving service delivery and assisting counties and WSPs to transition to devolution. Nor was it clear to the evaluation team why some strategies and activities were applied in some counties but not in others, what a sequence of activities might look like, or what criteria would be used to determine when certain activities would be carried out or not. Although there was some evidence of this, interviews with staff at the county level led the evaluation team to question whether sufficient sharing is occurring between county level KIWASH staff to develop a high-level strategy and on the details of activities that have worked. The timing would seem right for this kind of coming together across the county staff. A final aspect revolves around the conundrum for KIWASH in that county-level capacity building is seen as critical to the long-term success of WASH service delivery. As noted directly and indirectly by a number of interviewees, “governance is the overriding issue in everything.” Yet efforts to address these gaps are “long-term processes” and struggle against forces that are “not under any program’s control.” Most importantly for the framing of KIWASH, such activities do not directly contribute to the targets (while contributing to the sustainability of outputs). So there are short- 26 The evaluation team was unable to attend WASH Forum meetings to directly validate this. MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 44 term trade-offs in terms of energy, funding and staffing allocations that need to be balanced against the expectations of wider, lasting impact and the opportunity that the transition provides. An interesting question for KIWASH is whether the legal frameworks (such as policies and bills) are crucial to establishing the more nuts-and-bolts components of fulfilling the county oversight function. This would involve more emphasis on developing asset inventories, establishing monitoring systems, and agreeing oversight mechanisms. The evaluation team was not clear on how best to prioritize the more macro-level issues with the more operational components and whether there were contextual factors that would help make that determination. Presumably this kind of framing from KIWASH would be useful to inform wider processes beyond the KIWASH counties. A key lesson is that policy and law making are process intensive requiring political goodwill, interest and involvement of all stakeholders (as the KIWASH-supported WASH Forums have done). Working with other partners and rallying political commitment to policy development and legislative processes are imperative. Clear policy statements at the county level are a work in progress and support on these could be a significant legacy for KIWASH. 4.5 EQ5: IN THE FIRST THREE YEARS OF KIWASH, TO WHAT DEGREE WERE NUTRITION ACTIVITIES SYNERGISTIC WITH WASH ACTIVITIES? In the 2015 Water Development Strategy WASH and Nutrition Implementation Brief, USAID outlined opportunities for co-programming WASH and nutrition, suggesting this to be an area of increasing attention across the Agency. The original KIWASH design included two Outputs related to WASH and nutrition integration: Output 3, which aimed at improving access to integrated WASH and nutrition services by working with USAID Feed the Future, health, or nutrition programs; and Output 4, focused on increased production and consumption of nutrient dense and diverse foods (see Figure 2.1). This review focuses on the degree to which nutrition and WASH activities were synergistic during the first two years of the KIWASH program focusing on intended areas of integration under Output 3 as described in the two tasks in the contract to: a) Integrate and promote essential nutrition and WASH interventions, and b) Improve integration of WASH, maternal and child health (MCH), and nutrition services. Key activities under each task are summarized in Table 4.2. Table 4.2: WASH-Nutrition Integration – Key Activities Under Each Task Task Activities Integrate and promote essential nutrition and WASH interventions 1. Develop tools and training materials based on a review of literature from the WASH and nutrition sectors; 2. Train trainers in Essential Nutrition Action and Essential WASH Action; 3. Integrate combined Essential Nutrition Action and Essential WASH Action promotion in community and facility‐based programs; and 4. Develop and implement a monitoring and evaluation strategy to assess program impact. Improve integration of WASH, MCH and nutrition services 1. Integrate WASH in Nutrition Assessment, Counseling and Support (NACS); 2. Integrate WASH in Oral Rehydration Therapy (ORT) corners in health facilities; and 3. Use Safe Water and Hygiene Kits as an Incentive for Early and Frequent Post-natal Care Feed the Future funding for WASH and nutrition was removed from the KIWASH contract at the end of the second year of implementation, at a time when community-level interventions were just gaining momentum. WASH and nutrition activities were implemented in the counties for approximately one-and-a-half years. The challenge for the evaluation team was to square the reported investments made by KIWASH much earlier in the program with the evidence provided by interviewees at the county level during the time of the review. MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 45 4.5.1 GOVERNMENT AND PARTNERS’ PERCEPTION OF KIWASH’S WORK IN WASH/AGRI￾NUTRITION INTEGRATION Interviews were unambiguous that KIWASH’s efforts in WASH and agri-nutrition integration were well regarded by county governments and DPs at the national level. After a slow start in some counties, government officials in all counties surveyed welcomed the approach and expectations were high. KIWASH’s work in this area was recognized as the first time an attempt was made to bring three sectors (WASH, agriculture and nutrition) together to implement an integrated approach. In interviews, the WASH and nutrition departments in Nyamira and Kitui counties, where stunting rates are high, noted that an integrated approach was essential for improvement of nutrition outcomes. In Kisumu County, KIWASH is reported to have been the only project that had an integrated approach to addressing community issues and the removal of the nutrition component was disappointing. As described in EQ2 for WASH services delivery, some DPs were concerned that the project had minimal engagement at the national level and missed opportunities to engage and influence policy. Engagement with the Ministry of Health is circumscribed by USAID at present, which has limited this outreach. 4.5.2 CONCERNS ABOUT THE WASH/NUTRITION EXIT STRATEGY KIWASH staff and partners were universally concerned with the way the WASH and agri-nutrition integration component abruptly ended. The evaluation team understands that KIWASH leadership sent notification to the KIWASH county teams to halt activities immediately after USAID decided to remove Feed the Future funds from the contract. Staff involved in these components were given six months to complete administrative and documentation activities. Many of the county office teams suggested they were expecting official communication to county partners to be delivered by KIWASH senior management or even USAID. The team noted that such formal communication has not happened to the disgruntlement of many. Interviews with government partners echoed similar frustration with the way the situation was handled. Some partners expressed that the suspension of the ag-nutrition work was too abrupt, suggesting that a more phased close-out would have been a more appropriate approach. “In the future take some time to do this. Make it more gradual and communicate. Make sure to complete what they start.” (DP at county level) 4.5.3 A FRAMEWORK FOR ASSESSING WASH AND NUTRITION ACTIVITIES To assess the degree that nutrition activities were synergistic with WASH activities, the team adopted a framework developed by FHI360 that uses characteristics of integrated programming to consider the extent WASH and nutrition integration had occurred in the five counties before the funds were withdrawn.27 The categories of integration noted in the framework from lowest to highest include: coincidence, co-location, coordination, cross training, collaboration, and complete 27 This framework was proposed in the approved Mid-Term Performance Evaluation Inception Report We stopped the program very quickly. Soon after the training things stopped. The technical person from KIWASH was no longer there… nothing more happening, especially on technical aspects” (Agriculture Extension Director, Nyamira) MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 46 integration (or joint implementation at community-based service delivery level, and facility-based service delivery level). A description of each category is included in Figure 4.1. Through the desk review and interviews, the team gathered information to contextualize KIWASH’s WASH and nutrition activities in each of the five counties according to the framework. Individual county analyses are summarized in Annex V. Figure 4.1 shows the results of this analysis. More marks in one category demonstrate the team found substantial evidence to suggest this level of integration had occurred. Fewer or no stars are given where evidence was either lacking or inconclusive. The findings below reflect what was observed directly and the information provided by interviewees in the five counties visited during the review. Figure 4.1: Plotting of Synergy in KIWASH WASH and Nutrition Activities The following sections summarize the evidence of WASH and nutrition integration for each of the framework’s five categories (apart from coincidence) as detailed in Annex V. MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 47 4.5.4 CO-LOCATION: BENEFICIARY TARGETING Integrated WASH and nutrition interventions should ideally focus on vulnerable populations with limited access to WASH facilities and high rates of malnutrition. While KIWASH hosted inception meetings with county partners, the team found no evidence of purposeful targeting of WASH/nutrition activities to focus in areas with both low rates of WASH access and poor nutrition status of children under five. This was not mandated by USAID to be part of the project design, but best practice and USAID global guidance suggest that resources and benefits would be maximized through intentional targeting of communities that had both low WASH and nutrition indicators. To the contrary, some understood co￾location or “overlap” as a duplication to avoid rather than a coordinated reinforcement to be intentionally delivered. In Kitui, KIWASH and the USAID National Health Promotion Plus (NHP+) program agreed to change locations so as to intentionally avoid co-location. There also was no evidence found for common beneficiary targeting by the CHVs and agriculture extension agents after jointly being trained. In Nairobi County, however, in addition to WASH and nutrition messages, CHVs were reported to have delivered messages on agriculture and would identify and link households with kitchen gardens with agriculture extension workers covering those areas. 4.5.5 COORDINATION Per Section C of the contract, the IP was expected to work with other USAID Feed the Future, health, and nutrition programs to implement WASH and nutrition activities. While the evaluation team did not find evidence of joint planning with USAID or non-USAID programs at either the county or national levels, KIWASH notes that it worked closely with Feed the Future projects such as Kenya Agricultural Value Chain Enterprises (KAVES) and Accelerate Value Chains Development (AVCD) programs to ensure effective coordination of WASH, nutrition and agriculture initiatives across programs. Coordination platforms at the county level provide forums for information sharing and learning, a space for providing harmonized interventions to beneficiaries, and other benefits. Kakamega and Kisumu counties reported to have WASH coordination forums, while Nairobi County has a Nutrition Specific Technical Working Group whose mandate is to develop the Implementation Guide for WASH and Nutrition Integration. The membership of this Technical Working Group is comprised of both development partners (Afya jijini, KIWASH, Oxfam, Concern Worldwide and UNICEF) and the Nairobi County Ministry of Health. KIWASH participated actively in this group until the Feed the Future funding was removed from the project. At the sub-county level, joint action plans were developed in some sub-counties by participants of integrated WASH, agriculture, and nutrition training workshops after an initial county-level Training of Trainers (TOT). “We did it [action planning] as a sub county together as a team – the 3 departments, WASH, nutrition, agriculture.” (Sub county PHO, Kitui West) “There was an area where both KIWASH and NHP+ had targeted for kitchen gardening and establishment of mother-to-mother support groups. To avoid an overlap, KIWASH moved to another location since NHP+ had started the process.” (KIWASH Nutrition Specialist) MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 48 4.5.6 CROSS TRAINING–CAPACITY ENHANCEMENT FOR WASH AND NUTRITION INTEGRATION KIWASH has developed a TOTs facilitation manual and a Training Manual for Community-Based Service Providers on Integrated WASH and Agri-Nutrition (in the final stages of development at the time of this evaluation). The project’s SBCC strategy and supporting toolkit include integrated WASH and nutrition messages. All counties visited provided evidence of having conducted TOT workshops on WASH and nutrition integration for government staff drawn from Ministries of Health (Offices of WASH and Nutrition) and Agriculture. These events are documented in KIWASH’s Year 2 Annual Report. The cadre of trainers, in turn, trained CHVs and agriculture extension agents who in most cases began implementing a suite of WASH and nutrition activities for a brief time until the nutrition component was terminated. 4.5.7 COLLABORATION: JOINT ASSESSMENT, PLANNING, IMPLEMENTATION, AND MONITORING Joint Monitoring: Development and implementation of a M&E strategy to assess impact of the WASH/nutrition integration was projected to be a key activity under Output 3. Ideally this would have measured both WASH and agri-nutrition activities and results, with the aim of assessing the benefits of integrated WASH and nutrition programming. To this end, the KIWASH baseline (August 2016) and commissioned mid-term evaluation (November 2018) included WASH and nutrition indicators (e.g., minimum acceptable diet for children and dietary diversity for women of reproductive health). KIWASH developed project-specific reporting forms for CHVs to capture process- and output-level indicators for both WASH and agri-nutrition. However, according to KIWASH staff, this system was in place for only six months before funding was withdrawn. None of the counties visited reported any intention of self-initiating joint monitoring systems. While WASH/nutrition activities were ongoing, data was reported to KIWASH for each component through separate M&E systems managed by the relevant ministries. 28 The evaluation team notes that the standard GOK CHV reporting templates for community strategy tools (MOH 513) and CLTS do not capture agri-nutrition indicators. There were no reports or documentation of joint monitoring activities by the ministries of health and agriculture at the county level. Joint Implementation at Health Facilities: Noting the limited integration of WASH in the provision of MCH and nutrition services within Kenya’s healthcare system, KIWASH was designed to implement activities that would improve MCH and nutrition outcomes through incorporation of WASH at service delivery points. Specific activities included: integration of WASH in Nutrition Assessment, Counseling and Support (NACS); integration of WASH in Oral Rehydration Therapy (ORT) corners in health facilities; and use of Safe Water and Hygiene Kits as an incentive for early and frequent postnatal care. KIWASH’s Year 2 Annual Report indicates that the project had updated protocols and guidelines for NACS and ORT service provision and developed job aids for ORT corners. However, none of the counties visited reported to have had updated NACS or ORT protocol guidelines to include Essential WASH Actions. Additionally, no trainings were reported for CHVs on integrating 28 Data ends up in the county records health information office and then its fed to the national system. As KIWASH works with the same CHVs, they are able to take data on WASH and nutrition from the CHVs. “There was no coordinated or joint M&E. We are reporting separately.” (WASH coordinator, Nairobi County) MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 49 WASH in NACS. Further, the assessment did not see any evidence indicating nutrition counseling was offered either during the distribution of the hygiene kits or provision of water at the ORT services. The evaluation team confirmed that KIWASH supplied selected health facilities with ORT corner equipment. At least one health facility per county was stocked with ORT corner supplies before the WASH and nutrition component stopped. Other than simply providing ORT supplies per the national guidelines,29 the team saw no evidence of KIWASH engaging in other approaches to promote Essential WASH Actions and Essential Nutrition Actions through the ORT corners. The health facilities visited have limited spaces for services, and so ORT supplies and services share space with rooms used for other services including the postnatal care (Nyamira) and HIV testing and counseling (Kitui). All five counties visited had evidence of hygiene kit distribution. The evaluation team had opportunities to review the distribution records. In some counties distribution of hygiene kits continues to date with funding from KIWASH. 30 The strategy for distributing the limited number of kits differed across the counties. In Kitui, selected health facilities together with the KIWASH team designed a distribution approach that would increase the uptake of postnatal care and facility-based deliveries in the targeted communities by distributing the kits on the first and second postnatal care visits and after delivery at the facility. One clinic’s records in Nyamira showed that hygiene kits were fully distributed within two or three days after being stocked at the health facility. Health facilities have verbally reported increased uptake of MCH services because of the hygiene kits distribution. Although not new, kitchen gardening is an approach that created a lasting interest with some county government partners. Through a different funding source, Kisumu County plans to continue supporting kitchen gardening technologies at health facility and community levels in two sub-counties next year. Joint Implementation of Community-Based Programs: KIWASH facilitated the training of CHVs on WASH and nutrition integration through the cascaded TOTs. CHVs would then deliver the same messages, including agri-nutrition messages, to the households. The Ministry of Agriculture trained their extension workers independently of the CHVs31 and the evaluation team saw no evidence of the integration of WASH messages within agricultural activities at the community level. In Kitui and Nyamira, CHVs trained mother-to-mother support groups on WASH, kitchen gardening and preparing good diet foods (Output 4). Demonstration kitchen gardens were established in a few CHV households in Nyamira. CHVs reportedly reached out to mothers on dietary diversity messages, which has improved the uptake of nutritious meals. 29 GOK (2013) Oral rehydration therapy corner operational guidelines. 30 Hygiene kits contain a 20-liter Jerrican, a 17-liter bucket, bar of soap and water guard. 31 This is perhaps understandable given the different levels of the agriculture extension workers (operating at ward level, with higher education qualifications, and being on government payrolls than the volunteer village-level CHVs). The first generation of the SBCC job aid developed by KIWASH for CHVs included nutrition messaging. The evaluation team found that the job aid has since been updated to remove all mention of nutrition messages leaving the CHVs to now focus entirely on WASH messages. “We have trained mothers on kitchen gardening and preparing good diets, e.g., how to plant vegetables, carrots, pumpkins, Irish potatoes... Before they used to go to markets to buy a variety of foods and vegetables. Now they have kitchen gardens because of KIWASH.” (CHV FGD in Nyamira) MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 50 As discussed in EQ2, KIWASH implements CLTS programs as per the national CLTS protocol, which is prescriptive in nature and, by some accounts, not to be altered in delivery. Following GOK protocols, KIWASH did not explore CLTS as an opportunity to integrate agri-nutrition messages into the community mobilization process. While further analysis is needed, recent studies looking at delivering nutrition messages through CLTS based in Kitui County have shown that “child-focused messaging can potentially be integrated into CLTS programming” to positive effect. 4.5.8 COMPLETE INTEGRATION Given the short period of implementation, KIWASH did not have a chance to have complete integration of WASH and nutrition in the respective counties. 4.5.9 CONCLUSIONS – EQ5 The WASH and nutrition integration program was well received and momentum was building by the end of Year 2 of KIWASH. The enabling environment provided by both the county ministries and DPs was supportive and there was a potential for KIWASH to have been successful and, given the lack of work in this space, exemplary. Again the challenge for the evaluation team was to find a legacy of evidence over a year after the funding was discontinued. An assessment of the degree of integration across the five counties summarized in Figure 4.1 suggest the approach was generally well integrated, consistently reaching high levels of collaboration across all counties. There appear to have been opportunities missed related to co-location, which is considered a lower characteristic of integrated programming. Notwithstanding lower scores in these areas, the evaluation team found that KIWASH was making concerted programmatic efforts to integrate WASH and nutrition at the county and sub-county levels. According to one of the national DPs, KIWASH’s contribution to the revision of the National Agri￾Nutrition Resource Manual and development of the minimum package for agri-nutrition is a key contribution to the sector. At the county level, KIWASH managed to set up systems that brought relevant sectors together to discuss WASH and nutrition integration. The pool of trainers, training manuals and other resource materials will be useful for county governments or other development partners interested in taking up integrated programs. Ultimately, because the program ended early and abruptly, KIWASH and USAID missed an opportunity to learn and document evidence on WASH and nutrition integration, as well as specific methods for successful integration as was initially intended. Such documentation would have been invaluable to inform future programming in Kenya and globally. 4.6 CROSS-CUTTING: REFLECTION ON GENDER MAINSTREAMING IN KIWASH The KIWASH project is to be commended for having a gender mainstreaming focus in all the stages of the project from development to evaluation. During project development, KIWASH included gender assessments in the baseline research findings. The entire KIWASH consortium was trained in gender mainstreaming with a clear plan on who would be the gender focal person responsible for reporting on gender indicators. Training staff on the importance of gender in WASH set the stage for consistency in messaging at all levels throughout the project cycle. Gender sensitive indicators “KIWASH was also involved in the review of the Agri-Nutrition Resource Manual… and determined the minimum Agri-Nutrition Package… Their input is still felt in the form of the manual.” (DP, National level) MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 51 also were included in the project AMEP. Data is disaggregated by sex in all the reporting tools, which is critical for identifying needs and rights of all beneficiaries. During the planning of the project, KIWASH also recognized the possibility of unintended consequences of their implementation and put in place measures to address these are they arise. Although project implementation is still ongoing, KIWASH has been able to include both men and women in all project activities thus far. There are already gender stories emerging from the counties in which the project works. This is all in line with international standards set for gender mainstreaming in programs. Women are represented in all levels of the project, from national to community. At the national level, gender sensitization training has been carried out by KIWASH with increasing awareness of the constitutional mandate of gender and social inclusion. At the WSP level, senior level management has been trained in gender mainstreaming. Many of the informants interviewed were proud to point to the progress they were making in incorporating gender as part of their management and decision-making processes. Interviews and findings from this qualitative review showed increases in women’s skills particularly in terms of operating the enterprises32, confidence in leadership roles, and capacity to advocate on their own behalf with decision-makers. KIWASH is “proactively striving to meet the 30 percent gender rule in the board composition” of WSPs. Community-level stakeholders and water enterprises are well aware of the need to include women in water committees and decision-making bodies of the organizations. Despite cultural barriers to gender inclusion, communities are moving forward with this “new” KIWASH requirement in all their operations. Both the utilities and enterprises have made conscious efforts to include women not only as consumers but as part of the service provision value chain. For example, women and women’s groups are identified and targeted to benefit from water provision as water kiosk operators, vendors and employees. To what extent this can be attributed to KIWASH in particular is not clear, but it is evident that the project has consistently made it an issue at all levels of their community engagement. At the community level, inclusion of women at the water committee level is on the rise, although women are still “finding their voices” in these positions. The inclusion of women sends very positive images to the communities about their abilities and capabilities. Conversations at all levels suggested that there is a general sense that women are more trustworthy and sensitive than men in certain positions, even at the community level. Very powerful stories are emerging in the WASH sector in the KIWASH counties of women participating in positions previously only thought to be suitable for men. One example stood out in Nyamira of a woman plumber who is not only effective at her job but is thought to be more sensitive and responsive to the community’s needs by better understanding the issues they face. Women will likely need additional training to increase their confidence to participate meaningfully in committees, but the first steps have been made by KIWASH by insisting on their inclusion from the outset. KIWASH has insisted on community management groups being gender balanced. The KIWASH project also has clearly considered gender in the WASH facilities that they have designed so that they can be used by women more easily. Emphasis is placed on ensuring adequate space for women to be able to clean, dry laundry and fetch water more easily. The project has made great efforts to make water sources accessible to women so that they feel safe and free from harassment. Going forward, KIWASH could consider including private washing cubicles and laundry areas for women to wash and dry undergarments and sanitary cloths at water distribution points and community laundry facilities. This recommendation came out clearly from discussions with women at the community level. KIWASH has 32 Although the numbers are limited, KIWASH does seek to support women-run WASH enterprises. MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 52 also focused implementation work on upgrading WASH facilities within or close to learning environments, health facilities and where larger populations are concentrated. KIWASH has been consistent in including both men and women in sanitation and hygiene messaging. This is crucial given that although women are the primary caregivers in the family, men are largely responsible for education and discipline. Focusing messaging on both men and women is likely to have a more positive impact. At the community level, KIWASH engaged both men and women as CHVs, which is an effective strategy. This moves beyond simply focusing on women’s participation in WASH to opening up discussion between women and men about WASH needs. In discussions with CHVs, the evaluation team learned that there is a deeper community understanding on the rights of women, men, boys and girls. Both men and women need to engage in WASH activities as a community. From the focus group discussions, this sort of engagement and support was noted as leading the community to imagine and realize positive change toward gender equality even while not identifying it as such. To them it is simply working together to make their community better. MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 53 5.0 RECOMMENDATIONS Based on findings and conclusions presented in Section 4, this section presents short- and long-term recommendations for KIWASH project management and for USAID. Both sets map specifically to the EQ responses above while offering the evaluation team’s higher-level perspective considering the breadth of the review. While there are areas to improve, a quote from one KIWASH staff member adequately sums up the project at this stage. “We have planted and started weeding, the harvest is coming. If we went to the next stage, everything would be so easy. The demand has been created for the services that KIWASH is providing.” 5.1 SHORT-TERM RECOMMENDATIONS 5.1.1 GENERAL a) To sustain the gains made by having staff embedded in county governments, develop an exit strategy focused around a dedicated shadowing role for county staff with related commitments from county leadership. b) Forge more linkages across the county-level work (among staff but also between county government partners) to exchange experiences to clarify the types of activities that are having the most impact and the criteria for when such interventions would be implemented. c) Continue to advocate for increases to WASH budget allocations at the county level. d) Clarify a strategy for the remaining period that prioritizes an appropriate strategic balance among macro policy-level work, operational activities and knowledge sharing. e) Provide clear guidance to partners on funding and proposal processes (including with regard to their competitive nature). f) Cascade gender mainstreaming training to all levels of KIWASH engagement from national to community. g) Engage with organizations focused on women’s empowerment to ensure the sustainability of the project’s gender equality goals once the project is over. 5.1.2 WATER a) Continue to support governance improvement activities at the county-WSP level. b) Continue to work with WSPs to articulate Performance Improvement Plans for specific components of their work. c) As possible, work with Water Sector Trust Fund (WSTF), Kenya Pooled Water Fund and WASREB (and others as appropriate, including Agence Française de Développement [AFD], the Dutch PEWAK Program, the Water, Sanitation and Hygiene Finance [WASH-FIN] project, among others) to articulate a clearer national-level strategy for financial instruments for WSPs and financial reporting. (A water.org initiative may be attempting this.) d) Revisit the World Bank-Deloitte financial modeling tool (and compare to that of commercial banks) to advise WSPs of the criteria needed to access commercial finance. e) Document the NRW and other direct impacts of KIWASH’s work with WSPs. f) Work with counties and WSPs to develop a strategy/masterplan for how they engage with localized service providers (enterprises and community-based models) and continue to develop WASH inventories. g) Document the work with enterprises (disaggregating community-managed and private entrepreneur) to underscore both the challenges and the prospects for their success based on KIWASH support. h) As affordable, scale up acquisition of efficient and affordable billing software to address part of the operations inefficiencies in the WSPs. MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 54 i) Articulate an investment strategy with counties for solar energy options for water providers to substantially reduce material costs. j) Ensure work with Water Resources Users Associations (WRUAs) links up with enterprise work to protect sources. k) Explore collaborations with others working on WRUA capacity building (World Wide Fund for Nature [WWF] Kenya, for example) to align approaches. 5.1.3 SANITATION a) Continue to support final passage of the county-level Environmental Health and Sanitation Bills before the end of the project. b) Continue ongoing CLTS efforts, focusing on achieving ODF status for sub-counties (or smaller areas if resources do not allow) by the end of Year 4. Use Year 5 to document KIWASH’s legacy with CLTS while supporting continued reporting of sanitation data in ODF communities as prescribed in national protocols. c) As resources allow, develop and test a model of pairing new CLTS work with sanitation marketing efforts from entry into new areas (e.g., blending demand activation and supply chain efforts more effectively) rather than introducing sanitation products as part of post-ODF activities. d) Continue to establish or strengthen the supply chain of SATO products into communities. e) Clarify the enterprise model currently being trialed for the SATO products and the SAFI latrine, being more explicit how it works, KIWASH’s inputs, and what is being tested until the end of the project. f) Systematically document the effectiveness of the sanitation marketing approaches being tried. Keep records of buyer profiles, prices and sales margins (among other aspects) to understand how these fluctuate by season and across the diverse project area. Plan follow-up surveys with early adopters to document their profiles and motivations for adoption. Share findings at national level to influence the next generation of projects working in this area. g) Unless already part of the enterprise development work, remove the intention to support FSM improvements across the end stages of the sanitation value chain. 5.1.4 COUNTY-LEVEL SUPPORT a) Based on lessons learned, develop a Capacity Gap/Needs Assessment Framework for county governments with regard to WASH service provision that can be used in future projects and in non￾KIWASH counties. b) Showcase each county’s successes and lessons to promote learning, foster exchange and encourage replication. For example, highlight Makueni in terms of leadership as it has a progressive and engaged CEC and Governor with strong public consultation processes, Kakamega, which is further along in policy-level advances and budgeting for WASH; Kisumu for the strength of its WASH Forum. c) Focus at the national level not on overarching national-level issues but advocating specifically on what KIWASH has learned regarding the issues to be resolved at the county level. This includes budget allocations, clearer roles and responsibilities (asset ownership, loan guarantees, county-wide strategies, budget allocations) and their implications at the operational level, and borrowing protocols and limits, among others. d) Seek to articulate a county level strategy for community-based enterprises with regard to their relationship to county oversight and WSP expansion plans. e) Document the pros and cons of the recoverable grants program and determine whether a pilot county-led program could be developed in a further phase. f) Consolidate the best approaches for county-level monitoring and evaluation (M&E). MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 55 5.1.5 WASH/NUTRITION INTEGRATION a) Finalize and officially hand over to the counties the training of trainers (TOT) toolkit and the Training Manual for Community-Based Service Providers on Integrated WASH and Agri-Nutrition. b) Consider an impact assessment of the distribution of hygiene kits on the uptake of maternal and child health (MCH) services. 5.2 LONG-TERM RECOMMENDATIONS (USAID) a) Recognizing the transition to devolution, consider reducing the number of pre-defined targets in tender documents for a follow-on program related to WASH governance and systems strengthening, allowing respondents to suggest annual and life of project targets that respond to a series of objectives and intermediate results set by USAID. b) Contribute to filling the gap in rural sanitation programming by directing more resources explicitly to developing the sanitation market in Kenya, working simultaneously and interdependently at national, county and community levels. 33,34 c) Invest in working with WSPs, county governments and unregulated service providers to develop, cost and implement new models for FSM, much in the same way that KIWASH currently does on the water side with WSPs. d) Ensure that an environment element is fully integrated to address degradation of water resources. e) Work with the Kenyan government and align future nutrition programs with the government’s support to agribusiness through financing and information systems. f) In future programs, emphasize and facilitate more structured collaboration among United State Government partners to ensure more synergy in the provision of complementary interventions. 33 Refer to the MBS framework introduced in the 2018 USAID/WASHPaLS report entitled Scaling Market Based Sanitation: Desk Review on Market-Based Rural Sanitation Development Programs. 34 The changing living standards and urbanizing demographics in Kenya suggest considerable efforts are needed beyond excreta containment in small towns and peri-urban areas. “KIWASH has just come when devolution started… when they leave, then you leave us in the hands of the devolved systems.” (FGD, WASH Enterprises – Kisumu) MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE (KIWASH) PROJECT 56 ANNEXES [Added under separate cover] • Annex I: Evaluation Statement of Work • Annex II: Evaluation Methods and Limitations • Annex III: Data Collection Instruments • Annex IV: Data Sources • Annex V: WASH-Nutrition Integration Profiles from Five Counties • Annex VI: Disclosure of Any Conflicts of Interest U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523 Tel: (202) 712-0000 Fax: (202) 216-3524 www.usaid.gov