KIWASH MIDTERM EVALUATION REPORT ANNEXES 1 ANNEXES TO THE MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION, AND HYGIENE PROJECT (KIWASH) MARCH 2019 This publication was produced for review by the United States Agency for International Development. It was prepared by Tetra Tech. ACKNOWLEDGEMENTS The mid-term evaluation was conducted by Ken Caplan (Team Leader, Independent), Jonathan Annis (WASH Specialist, Tetra Tech), Charles Oyaya (WASH Governance Specialist, Independent), Evelyn Makena (WASH and Nutrition Specialist, Independent), and Mary Gichuhi (Gender Specialist and Logistics Coordinator, Independent). Each team member made meaningful contributions to this report. The evaluation team would like to acknowledge the invaluable contribution from Siobhan Girling (Tetra Tech) in supporting data collection in one sub-county, Clinton Sears (Tetra Tech) for providing training and guidance in using the Dedoose platform, Elizabeth Kruger (Tetra Tech) for copy editing, and Morris Israel (Tetra Tech) for peer reviewing the final draft report. We extend a sincere thanks to KIWASH staff at national and county levels for their close collaboration and cooperation in the planning and execution of the review. Without their assistance and flexibility, the evaluation team would not have been able to complete the review in the timeframe allotted. The team would also like to thank the numerous key informants at the national, county, and community levels who participated openly and thoughtfully in the interviews and focus group discussions, and to USAID/Kenya and East Africa for providing oversight and guidance throughout. Preferred citation: USAID. (2019). Annexes to the Mid-Term Evaluation of the Kenya Integrated Water, Sanitation and Hygiene Project. Washington, DC., USAID Water, Sanitation, and Hygiene Partnerships and Learning for Sustainability (WASHPaLS) Project. Prepared for the United States Agency for International Development by the Water, Sanitation, and Hygiene Partnerships and Learning for Sustainability (WASHPaLS) project under Task Order number AID-OAA-TO-16-00016 of the Water and Development Indefinite Delivery Quantity Contract (WADI), contract number AID-OAA-I-14-00068. Tetra Tech Contacts: Morris Israel, Project Director morris.israel@washpals.com Jeff Albert, Deputy Project Director jeff.albert@washpals.org Jonathan Annis, Project Manager jonathan.annis@tetratech.com Tetra Tech 1320 N. Courthouse Road, Suite 600, Arlington VA 22201 Tel: (703) 387-2100, Fax: (703) 414-5593 www.tetratech.com/intdev Did you find this report useful? We’d love to hear from you! Please provide feedback/comments here: https://goo.gl/forms/1fSiL634Rqu57r2I2 ANNEXES TO THE MID-TERM EVALUATION OF THE KENYA INTEGRATED WATER, SANITATION AND HYGIENE PROJECT (KIWASH) MARCH 2019 DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT i REPORT ANNEXES REPORT ANNEXES ....................................................................................................................................I ACRONYMS AND ABBREVIATIONS......................................................................................................II ANNEX I: EVALUATION STATEMENT OF WORK.............................................................................. 1 A.1 BACKGROUND.......................................................................................................................................................................2 A.2 STATEMENT OF WORK.........................................................................................................................................................3 A.2.1 Evaluation Purpose ................................................................................................................................................3 A.2.2 Dissemination and Utilization Plan....................................................................................................................3 A.2.3 Key Evaluation Questions....................................................................................................................................4 A.2.4 Methodology ...........................................................................................................................................................4 A.2.5 Operating Constraints..........................................................................................................................................6 A.2.6 Operational Considerations ...............................................................................................................................6 A.2.7 Participation ............................................................................................................................................................6 A.2.8 Standards..................................................................................................................................................................6 B.1 PERIOD AND PLACE OF PERFORMANCE..............................................................................................................................6 C.1 DELIVERABLES.........................................................................................................................................................................7 C.2 REPORTING GUIDELINES.......................................................................................................................................................8 D.1 EVALUATION TEAM COMPOSITION ....................................................................................................................................9 D.1.1 Team Leader - Evaluation Specialist..................................................................................................................9 D.1.2 WASH Specialist..................................................................................................................................................10 D.1.3 WASH Governance Specialist..........................................................................................................................10 D.1.4. Nutrition Specialist..............................................................................................................................................10 E.1 EVALUATION MANAGEMENT ............................................................................................................................................10 ANNEX II: EVALUATION METHODS AND LIMITATIONS .............................................................. 12 JUSTIFICATION FOR COUNTY SELECTION..................................................................................................................................12 DATA TRIANGULATION METHODS ............................................................................................................................................12 INTERVIEW TEAMS PER COUNTY.................................................................................................................................................13 INTERVIEWS CONDUCTED AT COUNTY AND NATIONAL LEVELS..........................................................................................13 DESIGN CONSIDERATIONS FOR KII AND FGD GUIDES...........................................................................................................13 DESCRIPTORS USED IN DEDOOSE ...............................................................................................................................................14 DATA CODING AND ANALYSIS IN DEDOOSE ...........................................................................................................................14 ANNEX III: DATA COLLECTION INSTRUMENTS ............................................................................. 15 KEY INFORMANT INTERVIEW TOPIC GUIDE: KIWASH STAFF ...............................................................................................15 KEY INFORMANT INTERVIEW TOPIC GUIDE: COUNTY-LEVEL STAFF (NOT WSPS)............................................................18 KEY INFORMANT INTERVIEW TOPIC GUIDE: NATIONAL-LEVEL COLLABORATORS AND DEVELOPMENT PARTNERS .....21 KEY INFORMANT INTERVIEW TOPIC GUIDE: WASH SERVICE PROVIDERS (COUNTY-LEVEL OR UTILITY) RECEIVING KIWASH SUPPORT.............................................................................................................................................................23 KEY INFORMANT INTERVIEW TOPIC GUIDE – WASH ENTERPRISES RECEIVING KIWASH SUPPORT (PRIVATE SECTOR AND COMMUNITY-MANAGED SERVICE PROVIDERS)......................................................................................................26 KEY INFORMANT INTERVIEW AND FGD GUIDES: WASH AND NUTRITION INTEGRATION STAKEHOLDERS .................28 KEY INFORMANT INTERVIEW TOPIC GUIDE: USAID...............................................................................................................34 ANNEX IV: DATA SOURCES ................................................................................................................. 36 LIST OF KEY INFORMANTS............................................................................................................................................................36 DOCUMENTS CONSULTED IN DESK REVIEW.............................................................................................................................39 ANNEX V: WASH-NUTRITION INTEGRATION PROFILES FROM FIVE COUNTIES................... 40 ANNEX VI: DISCLOSURE OF ANY CONFLICTS OF INTEREST...................................................... 44 ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT ii ACRONYMS AND ABBREVIATIONS AMEP Activity Monitoring and Evaluation Plan ANC Antenatal Care CAP Competency Assessment Program CEC County Executive Committee CHV Community Health Volunteer CLTS Community-Led Total Sanitation COP Chief of Party COR Contracting Officer’s Representative DAI Development Alternatives Inc. DDL Development Data Library DEC Development Experience Clearinghouse ENA Essential Nutrition Action EQ Evaluation Question EWA Essential WASH Action FGD Focus Group Discussion GWASCO Gusii Water and Sewerage Company GOK Government of Kenya HCF Healthcare Facility HH Household IP Implementing Partner IR Intermediate Result KEWASNET Kenya Water and Sanitation Civil Society Network KII Key Informant Interview KIWASH Kenya Integrated Water, Sanitation and Hygiene Project KIMWASCO Kiambere-Mwingi Water and Sanitation Company KIWASCO Kisumu Water and Sewerage Company M&E Monitoring and Evaluation M2M Mother-to-Mother MOA Ministry of Agriculture ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT iii MOH Ministry of Health NACS Nutrition Assessment, Counseling, and Support NRW Non-Revenue Water ODF Open Defecation Free OEGI Office of Economic Growth and Integration RAPID Kenya Resilient Arid Lands Partnership for Integrated Development ORT Oral Rehydration Therapy PNC Post Natal Clinic SBCC Social Behavior Change Communication SME Small and Medium Enterprises SOW Statement of Work TAMIS Technical and Administrative Management Information System TL Team Leader TOT Training of Trainers TWG Technical Working Group USAID United States Agency for International Development USAID/KEA USAID Kenya and East Africa USG United States Government WARMA Water Resources Management Authority WASH Water, Sanitation and Hygiene WASH-FIN WASH Financing Project WASH PAD WASH Project Appraisal Document WASHPaLS Water, Sanitation and Hygiene Partnerships and Learning for Sustainability Project WASREB Water Services Regulatory Board WRUA Water Resources Users Associations WSP Water Service Provider WSUP Water and Sanitation for the Urban Poor WSTF Water Sector Trust Fund ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 1 ANNEX I: EVALUATION STATEMENT OF WORK STATEMENT OF WORK Mid-Term Performance Evaluation of Kenya Integrated Water, Sanitation and Hygiene (KIWASH) Under Contract AID-623-I-12-00001 Identifying Information Program: Office of Economic Growth and Integration Activity Name: Kenya Integrated Water, Sanitation, and Hygiene Project (KIWASH) Implementing Partner: Development Alternatives Incorporated (DAI) Award Number: AID-615-TO-15-00001 Activity COR/AOR: Martin Mulongo Award Dates: September 2015 – September 2020 Activity Funding: $50.9 million Period to be Evaluated: September 2015 – September 2018 Date Completed Evaluation is Needed: December 2018 REQUIRED ATTACHMENTS (Embed documents in cells indicated below) AAD or PAD Approving this Activity or Project, including development hypothesis on which it is designed WASH PAD Relevant Mission Strategy CDCS Office PMP with Results Framework Assessments or Studies Completed to Inform this Activity Design Previous Evaluations of this Activity (indicate if none done) Baseline Assessment (if Impact Evaluation is requested) YEAR 1 YEAR 2 YEAR 3 YEAR 4 YEAR 5 Activity Description from Agreement with Partner and ALL Modifications (For ILs must include project proposals) Activity M&E Plan with Results Framework ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 2 Activity Annual Work Plans Activity Performance Reports (write number per year: 2-4) Site Visit & Other Monitoring Reports (indicate number provided per year) A.1 BACKGROUND KIWASH aims to accelerate and sustain improvements in water and sanitation access and services in the target counties identified herein and improve complementary hygiene behaviors. In Kenya, expansion in improved water and sanitation access has barely kept pace with population growth, respectively growing by only about 0.9 percent and 0.2 percent annually over the past decade according to Joint Monitoring Program (JMP 2013) estimates. The Contractor will, at a minimum, achieve the following targets against these relevant indicators: 1. Percent of households using an improved drinking water source in target counties/1 percent per year coverage increase above recent national trend of approximately 0.9 percent increase per year (1.9 percent per year total increase of coverage). 2. Percent of households using an improved sanitation facility in target counties/1 percent per year coverage increase above recent national trend of approximately 0.2 percent increase per year (1.2 percent per year total increase of coverage). 3. Percent of population in target counties practicing open defecation/2 percent decrease per year in prevalence above historic trend of approximately 0.2 percent decrease per year (2.2 percent per year total decrease in prevalence). 4. Percent of households with water and cleansing agent at a hand‐washing station/5 percent increase over baseline. 5. As part of its proposal, the Contractor will propose an indicator and result to be achieved against Output 4 (detailed below). The proposed result must reflect USAID’s KIWASH objective of achieving county-level improvements, which are systemic and sustainable. The results will be specific to improving the ability of the specific institutions, which are key to supporting/delivering the expected results specified in Section C.3.1. 6. Prevalence of children 6‐23 months receiving a minimal acceptable diet (result TBD). To ensure that improvements in access are accelerated and sustained, KIWASH is guided by a development hypothesis that requires execution of activities which contribute to six distinct types of outputs. These six areas of outputs and their associated activities are reflective of the main areas of vulnerability (technical, environmental, financial, governance/institutional and social) in relation to sustainable WASH services: 1. Market-based WASH service delivery models scaled up 2. Sustained access to financing/credit for WASH services increased 3. Access to integrated WASH and nutrition services improved 4. Environmental sustainability of WASH services improved 5. WASH services and water resources institutions strengthened and appropriately governed 6. Targeted policy reforms advanced which stimulate and support improvements in access ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 3 A.2 STATEMENT OF WORK A.2.1 EVALUATION PURPOSE USAID seeks the services of a contractor to evaluate the performance of the USAID/Kenya and East Africa (KEA) KIWASH Activity. The primary purpose of the evaluation is to assess the activity’s design and implementation, in order to inform future USAID/KEA WASH and nutrition programming. It will also inform management on whether the approaches employed by the activity are successfully meeting its goals of expanding sustainable access to water and sanitation services through market-based approaches. Being a mid-term evaluation, the evaluation will further inform any midcourse corrections to be done to the activity in the final two years. The evaluation findings will also inform the development of future WASH investments. The primary audience for the evaluation report will be the Office of Economic Growth and Integration (OEGI) and the current implementing partners, Government of Kenya, and water sector working group/collaborators. To a lesser degree, the evaluation findings may be used by USAID/Washington technical staff to inform efforts in the WASH sector. A.2.2 DISSEMINATION AND UTILIZATION PLAN USAID/KEA will disseminate the report widely with relevant stakeholders and the implementing partner, and it will also be made publicly available for other interested parties. The findings will be used by each of the relevant audiences to influence the current and future activities, as described in Table 1. Table 1: Audiences and Dissemination Plan for Mid-Term Evaluation Audience Purpose Dissemination Plan OEGI staff and USAID/KEA Mission Management • Assess the success of the activity • Validate the theory of change • Inform a new WASH PAD • Draft report will be shared with OEGI and USAID/W staff • Initial presentation to OEGI staff prior to a presentation to Mission management • Mission out-brief presentation • Final Evaluation report shared with the Mission and USAID/W Implementing Partner • Understand what is working well and not • Expand, continue, or modify approaches in the remaining two years of the activity • A findings validation workshop will be held between the IP/USAID and contractor at the IP’s office • COP will attend the mission out-brief presentation • The draft and final reports shall be shared with the COP for comment Other WASH implementing partners/ USAID Washington • Provide lessons learned with other WASH programs (in Kenya and globally) • COPs of Kenya RAPID and WASH-FIN will attend the presentations • Final report provided to USAID/W for information Government: • Ministry of Water and Sanitation • Ministry of Health • Ministry of Environment and Forestry • County governments • Ministry of Devolution • Encourage them to adopt promising approaches • Provide lessons learned • Copies of the final report will be shared with them Other Public • For learning • The report to be uploaded in the DEC ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 4 A.2.3 KEY EVALUATION QUESTIONS 1. How appropriate is the KIWASH activity design of market-based approaches in expanding sustainable access to water and sanitation services in the targeted counties? 2. How appropriate is the KIWASH activity implementation strategy to the success of the market￾based approaches in expanding access to water and sanitation services? 3. To what extent has KIWASH been successful in building county capacity to improve sustainable service delivery? 4. To what extent have the different KIWASH private sector approaches for water service delivery and sanitation access expansion been successful? Which of the KIWASH implementation approaches appear to be the most promising and should be prioritized in the remaining phase, for expanding sustainable water services delivery and sanitation access? 5. In the first two years of KIWASH, to what degree were nutrition activities synergistic with WASH activities? A.2.4 METHODOLOGY This will essentially be a performance evaluation. The evaluation team is expected to propose an evaluation methodology and analysis tools that guarantees the highest degree of rigor so as to ensure credible findings and recommendations. A mixed-method approach is highly recommended with the following key data gathering techniques. See Table 2 for details. Desk review: The consultants or evaluation firm will undertake a detailed desk review to inform the evaluation of the project. The review will cover all the relevant project documents and reports. The review will also cover relevant government of Kenya national and county government documents, relevant donor-funded programs and published WASH literature. Key internal documents/sources of information to be reviewed include but not limited to: • The KIWASH Scope of Work • DAI proposal to USAID/KEA • Approved Activity Monitoring and Evaluation, Pan (AMEP) • Annual work plans • Quarterly progress reports to USAID/KEA • Annual reports • Project success stories, case studies, presentations, snapshots, and other communication material such as videos • Project deliverables and activity reports • DAI Technical and Administrative Management Information System (TAMIS) • Contextual and sectoral reports Key Informant Interviews: The evaluation team will conduct interviews with USAID/KEA team, relevant Government of Kenya staff both at county and national levels, KIWASH Staff, government partners, private sector, water resource user associations (WRUAs), relevant Water Management Boards (Lake Victoria and Tana/Athi) and knowledgeable beneficiary communities members. A list of proposed interviewees will be submitted with the inception report. The final list of interviewees shall be agreed upon in consultation with USAID and KIWASH team. Focus Group Discussions: The evaluation team may also consider carrying out Focus Group Discussions (FGDs) to obtain the required data from project beneficiaries where applicable. The groups shall be constituted in consultation with USAID and the KIWASH team. ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 5 Site Visits: To further compliment information/data generated through key informant interviews and literature review, field visits shall be carried out. The evaluation team is expected to come up with a suitable sampling methodology to identify the locations or organizations to be visited within five selected counties within the project area. In selecting the various locations for site visits, the evaluation team should ensure that the two geographical project locations, the project components and/or sectors are catered for and that the logistics and cost involved with the different sites are put into consideration. Prior to the start of data collection, the evaluation team will develop and present, for USAID/KEA review and approval, a data analysis plan that details how data generated will be transcribed and analyzed; and what procedures will be used to analyze qualitative data from key informant and other stakeholder interviews. Table 2: Proposed Data Sources, Data Collection and Analysis Methods Questions Suggested Data Sources (*) Suggested Data Collection Methods Data Analysis Methods 1. How appropriate is the KIWASH design in expanding sustainable access to water and sanitation services through market-based approaches, in the targeted counties? Review quarterly and annual progress reports and AMEP data, program descriptions, WASH PAD, GOK WASH strategy and other development projects Key informant, Ministry of Water and Sanitation, county governments, Ministry of Health, WARMA, WSP in the selected counties, IP, USAID, private sector actors Qualitative data analysis, including context and pattern analysis, data disaggregation by gender and location, appropriate 2. To what extent has KIWASH been successful in building county capacity to improve sustainable service delivery? Review quarterly and annual progress reports CEC water in selected counties and county WASH personnel. WSPs and the IP. focus groups - local community beneficiaries Qualitative data analysis, including context and pattern analysis, data disaggregation by gender and location, appropriate 3. To what extent have the different KIWASH private sector approaches for water service delivery and sanitation access expansion been successful? Review, quarterly and annual progress reports and AMEP data. Program descriptions, WASH PAD. GOK WASH strategy and other development projects Key informant, Ministry of Water and Sanitation, CEC county governments, WARMA, Water Service Board, WSP in the selected counties, IP, USAID, private sector actors, focus groups - local community beneficiaries Qualitative data analysis, including context and pattern analysis, data disaggregation by gender and location, appropriate 4. Which of the KIWASH implementation approaches appear to be the most promising and should be prioritized in the second phase, for expanding sustainable water services delivery and sanitation access? Review quarterly and annual progress reports and AMEP data, program descriptions, WASH PAD, GOK WASH strategy and other development projects Key informant, Ministry of Water and Sanitation, county governments, Ministry of Health, WARMA, WSP in the selected counties, IP, USAID, private sector actors, focus groups - local community beneficiaries Qualitative data analysis, including context and pattern analysis, data disaggregation by gender and location, appropriate 5. In the first two years of KIWASH, to what degree were nutrition activities synergistic with WASH activities? Review quarterly and annual progress reports and AMEP data. program descriptions, WASH PAD, GOK WASH Key informant, county governments (Department of Agriculture, Nutrition), Ministry of Health, IP, Qualitative data analysis, including context and pattern analysis, data disaggregation by gender and location, appropriate ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 6 Questions Suggested Data Sources (*) Suggested Data Collection Methods Data Analysis Methods strategy and other development projects USAID, focus groups - local community beneficiaries A.2.5 OPERATING CONSTRAINTS The offeror should outline any anticipated constraints in designing the methodology to be used or the operations and logistics of implementation. For example, any unique situations related to locations that the team must visit, any security concerns or otherwise? A.2.6 OPERATIONAL CONSIDERATIONS Offeror should be aware that once a proposal is received, it will be shared for peer review with USAID/Washington for a period of five days, after which some revision may be required before a task order is issued. A.2.7 PARTICIPATION The primary point of contact for this evaluation will be Martin Mulongo, the activity COR, and he will serve as a resource person to the evaluation team. He, along with other USAID staff, will provide technical guidance pertaining to questions the evaluation team might have during the implementation of the SOW, and approve the evaluation methodology prior to the start of any fieldwork. USAID staff may accompany the evaluation team to selected site visits as observers, but will not be full members of the evaluation team. Implementing partner staff will also support the evaluation team by providing project documentation and information about project implementation. KIWASH staff will also be available to facilitate connections with key informants and beneficiaries for interviews or focus group discussions. A.2.8 STANDARDS The task order proposal, including methods, and the final evaluation must be consistent with and meet the standards of USAID’s Evaluation Policy (especially Appendix I). B.1 PERIOD AND PLACE OF PERFORMANCE 1) The evaluation to be conducted between September and December 2018. 2) KIWASH program is implemented in the following counties: Focus Area‐Lake Victoria Focus Area‐Tana/Athi Busia Kitui Nyamira Makueni Migori Nairobi Siaya Kisumu Kakamega The evaluation team must sample at least five counties, two from the Focus Areas (Lake Victoria and Tana/Athi). A possible distribution of the counties to sample would be: • 3 counties in the Western region (Kisumu, Kakamega, Nyamira, Migori, Busia and Siaya) • 1 county in the Eastern region (Kitui, Makueni), and • Nairobi county as a special case ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 7 C.1 DELIVERABLES Inception Report: The offeror must submit a detailed inception report to USAID within 5 days after the first meeting between USAID and evaluation team. The report shall detail the evaluation methodology, limitations, and ways to mitigate these, and operational work plan, which must include the proposed data collection and analysis methods to address the key evaluation questions. The inception report shall also include questionnaires and interview protocols and must not exceed 15 pages. Weekly reports during field work to update on progress and any logistical challenges to mitigate. Preliminary Draft Evaluation Report: Within two weeks after the end of fieldwork, the offeror must submit a draft evaluation report for of preliminary comments. This will facilitate effective review by the Mission in preparation for the presentation during the validation meeting. Debriefing/Presentation of Report: Approximately five workdays after submission of preliminary draft, and before the offeror's team embarks on doing the final report, a debriefing/presentation of the results will be done. During this meeting the team will present the major findings of the evaluation to key stakeholders in Nairobi. A PowerPoint presentation will be made by the evaluation team and submitted in a day before the presentation to USAID. The debriefings shall include discussions of methodology, limitations, key findings, conclusions, and recommendations. Final Report: Within 5 workdays of USAID and the IP’s comments on the revised draft Evaluation Report, and based on the provisions of the USAID evaluation policy, a final evaluation report shall be presented to USAID/KEA. The final report shall incorporate the team responses comments and suggestions. The format shall include an executive summary (highlighting key lessons learned), table of contents, list of acronyms, evaluation design and methodology, limitations, findings, conclusions, and recommendations and lessons learned. The report shall be submitted in English, in both electronic and three bound hard copies. The Final Report must not be more than 40 pages excluding annexes. A brief summary of this report (the popular version), not exceeding 15 pages, excluding any potentially procurement-sensitive information shall be submitted (also electronically, in English) for dissemination among implementing partners and stakeholders. The report must meet standards outlined in the evaluation policy. One Page Summary: To be submitted three days after USAID’s approval of the final report, back and front; for easy reading and distribution. The final Evaluation Report must conform to the standards set forth in the Agency Evaluation Policy. 1 In addition, the contractor must submit a one-page abstract of the findings/recommendations considered most important for USAID senior managers. If the report contains any potentially procurement-sensitive information, a second version report excluding this information shall be submitted (also electronically, in English). All primary source data, both quantitative and qualitative, generated during the course of evaluation must be provided to USAID in an electronic file in an easily readable format; organized and fully documented for use by those not fully familiar with the activity or the evaluation. In addition, all background documents collected for this evaluation must be provided to USAID on CDs, organized by implementing mechanism, along with the final report. The report and the data should also be submitted to the Development Experience Clearinghouse (DEC) and the Development Data Library (DDL). ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 8 Table 3: Contract Deliverables # Illustrative Deliverables and Timelines Note: All days are calendar days Illustrative Due Dates 1 Initial meeting between USAID, contractor, and evaluation team to clarify evaluation questions, proposed methodology, roles and responsibilities, logistical issues, and timelines. Day Two that evaluation team is in￾country. 2 Inception Report: This includes a work plan that provides: • detailed evaluation methodology • final list of proposed key informants, including IP staff • draft data collection tools • limitations and mitigation measures • timeline. No later than 7 days after #1. 3 USAID/Partner/Stakeholder Workshop (half day) to review, refine, and accept the work plan and tools. Within 3 days after # 2. 4 Updates on progress: Weekly update reports are required. In addition to a face-to-face or teleconference briefing midway of data collection. Updates should include any issues or problems encountered; initial findings that may warrant adaptive management to the evaluation process. Weekly and Midway data collection. 5 Preliminary draft Evaluation Report submitted. Two weeks after completion of fieldwork. 6 Debrief/Presentation of preliminary findings and conclusions i. Finding Validation workshop with OEGI/IP ii. Then to USAID/KEA and possibly key stakeholders. Approximately 5 days after #5 and as scheduled by USAID. 7 Revised draft Evaluation Report incorporating USAID and IP comments. No later than 5 days after receiving comments from USAID and the IP. This may mean more than one round of comments. 8 Final Evaluation Report and a One-Page (back to back) Brief due to USAID: submitted to USAID along with all raw data on a hard device. No later than 3 calendar days after receiving final approval from USAID. 9 Final report uploaded in DEC. No later than 5 days after # 8. 10 Possible dissemination meeting with the IP and other stakeholders. To be discussed and agreed upon with USAID. C.2 REPORTING GUIDELINES USAID Evaluation Policy standards must be met by the offeror throughout the contract. In particular: • The Evaluation Report should represent a thoughtful, well-researched and well-organized effort to objectively evaluate what worked in the project, what did not, and why. • Evaluation Reports shall address all evaluation questions included in the statement of work. • The evaluation report should include the statement of work as an Annex. All modifications to the statement of work, whether in technical requirements, evaluation questions, evaluation team composition, methodology, or timeline need to be agreed upon in writing by USAID. • Evaluation methodology shall be explained in detail and all tools used in conducting the evaluation such as questionnaires, checklists, and discussion guides will be included in an Annex in the final report. • Evaluation findings will assess outcomes and impact on males and females. ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 9 • Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.). • Evaluation findings should be presented as analyzed facts, evidence, and data and not based on anecdotes, hearsay, or the compilation of people’s opinions. Findings should be specific, concise, and supported by strong quantitative or qualitative evidence. • Sources (including people and organizations contacted) of information need to be properly identified and listed in an Annex. • Recommendations need to be supported by a specific set of findings. • Recommendations should be action-oriented, practical, and specific, with defined responsibility for the action. D.1 EVALUATION TEAM COMPOSITION The offeror must propose the most effective team composition based on the proposed methodology. The Evaluation Team shall be comprised of a Team Leader (TL) and at least three experts. All team members must have relevant prior experiences in Africa, familiarity with USAID’s business models, and prior evaluation/assessment experience. The team must have sufficient relevant experience in water, sanitation, private sector, gender, and governance. In addition, individual team members should have the technical qualifications identified for their respective positions. The TL is ultimately responsible for the overall management of the evaluation team and the final products. In addition, the TL is responsible for coordinating evaluation activities and ensuring the production and completion of an Evaluation Report in conformance with this scope of work and timelines. The TL will also ensure high quality analysis, writing quality, and report integration. The following is the recommended team composition: D.1.1 TEAM LEADER - EVALUATION SPECIALIST The Team Leader (TL) will be responsible for the overall management of the evaluation team, and delivery of the final report. In addition, the TL will coordinate the evaluation activities and ensure the production and completion of an evaluation report in conformance with the scope of work and timelines. The TL will ensure data integrity, high quality analysis, written reports, and report integration. S/he is also responsible for quality assurance and timeliness of all deliverables. All team members report to the Team Leader. Required Qualifications: • A minimum of a Master’s degree from accredited college or university in public health, statistics, or any closely related field • At least 10 years’ experience in program evaluation of donor-funded development programs in WASH; Priority will be given to those who have evaluated programs of similar nature in Kenya • Demonstrated knowledge of the methodologies, design, and evaluation process for conducting donor-funded project evaluations in WASH • Excellent report writing skills (evidence of prior evaluation reports or published work is a plus) • Demonstrated experience in Evaluation Team Management • A 5/5/5 level of English proficiency is required ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 10 D.1.2 WASH SPECIALIST The WASH Specialist will be responsible for providing technical expertise in analysis of overall water and sanitation sector. Required Qualifications: • Advanced degree (MEng or MSc) in relevant field (e.g., water, engineering, public health) • Minimum of seven years’ experience in water, sanitation, and hygiene development programming, with experience in Kenya preferred • Experience in evaluating development projects and programs • Fluent in English and Kiswahili D.1.3 WASH GOVERNANCE SPECIALIST The WASH Governance Specialist will be responsible for providing expertise in analysis of governance and strengthening of water resources institutions, as well as policy reforms advanced to stimulate investment in the WASH sector. S/he will also look at the business and financing models employed by the activity. Required Qualifications: • A minimum of a Master’s degree from accredited college or university in economics, law, business, social science or any closely related field • Minimum of seven years’ experience in corporate governance, business sector, or market development, with experience in Kenya preferred • Knowledge of policy and governance necessary to encourage Public Private Partnerships • Experience in evaluating development projects and programs involving institutional governance or business financing modeling • Fluent in English and Kiswahili D.1.4. NUTRITION SPECIALIST The Nutrition Specialist will support the evaluation team leader in analysis of the nutrition issues within the activity. Required Qualifications: • A minimum of a Master’s degree from accredited college or university in (applied human nutrition, nutrition and dietetics, food science and nutrition or any closely related field) • Minimum of seven years’ experience in nutrition sensitive programming in Kenya • Demonstrated experience in undertaking evaluations of nutrition programs • Fluent in English and Kiswahili The offeror is strongly encouraged to consider local experts. E.1 EVALUATION MANAGEMENT A. Logistical Support: USAID/KEA will provide other relevant documents and information for this assignment. USAID/KEA Strategic Planning and Analysis Office will manage this contract and coordinate with the technical office, Office of Economic Growth and Integration (OEGI). The offeror will be fully responsible for all logistical and administrative support including local and regional travel and will be required to demonstrate ability to obtain any security and medical clearances required by USAID. ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 11 B. Scheduling: USAID/KEA expects this evaluation to take place between September 1 and December 30, 2018. USAID Evaluation Policy standards must be met by the offeror throughout the contract. ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 12 ANNEX II: EVALUATION METHODS AND LIMITATIONS JUSTIFICATION FOR COUNTY SELECTION County Justification for Selection Kisumu Chosen over Siaya County, which has also achieved ODF status, Kisumu County hosts the third largest city in Kenya and the WSP there is supporting an aggregated model with smaller adjacent systems. It is also a regional KIWASH hub for the Lake Victoria region. In addition, it has a range of other international donors involved in the WASH sector (e.g., the French, the Dutch, and others). It was instructive to understand how KIWASH fits in to this multi￾stakeholder environment and the influence that KIWASH brings to the table. The Kisumu WSP (KIWASCO) has also accessed a loan from the Co-operative Bank. Kakamega Kakamega County has been profiled to be relatively successful in policy and legislative efforts. Kakamega and Busia County used to share a WSP, which has since split but the Kakamega WSP appears to have established itself well. Kakamega has not achieved ODF status while Busia is ODF. Kitui was already chosen based on having recently achieved ODF status, hence there was no need to include Busia for a county visit. Nairobi Nairobi is a special case that has a mix of all KIWASH activities. It is also the KIWASH head office and regional office for Tana/Athi region. It is a capital city hosting all the national government ministries, departments and agencies, and the main offices of key international partners. Kitui Chosen over Makueni in the Tana/Athi region apart from Nairobi. Kitui was selected due to the recent ODF declaration (World Handwashing Day, October 15, 2018). A specific line of inquiry revolved around understanding KIWASH’s role in the lead up to the certification and announcement. Kitui is also one of the counties that was able to implement an early program on nutrition before this component of the program was effectively discontinued. Nyamira Chosen over Migori, Nyamira shares a WSP with Kisii County (not a KIWASH county). Thus the line of inquiry was around managing in a cross county service delivery environment. Nyamira also has a Gender Mainstreaming Strategy and reportedly successful CLTS efforts. DATA TRIANGULATION METHODS With regard to triangulating data from key informant interviews, the team often requested key informants to provide evidence in the form of internal project reports (see Annex IV) with data to corroborate statements of fact or suggested where evidence may be available. In cases when the team felt there was an evidence gap or the evidence was contradictory, the team referred to KIWASH to provide evidence that confirms or counters the claims made. When analyzing the qualitative data, the evaluation team included both complementary and conflicting accounts of KIWASH implementation. Both accounts have been analyzed and weighed against each other to reach a nuanced determination in this evaluation report. Further, any qualitative scripts that a) were internally inconsistent during the interviews, or b) included probes that resulted in inconsistent remarks were reviewed by the team and follow-up with interviewees was conducted where necessary. Triangulation was also used across county-based interviewees to note consistencies across key informants. ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 13 INTERVIEW TEAMS PER COUNTY County Evaluation Team Members Kisumu Team Leader, WASH Specialist, Gender Specialist Kakamega WASH Governance Specialist, Gender Specialist Nairobi Gender Specialist, WASH Nutrition Specialist, WASH Governance Specialist** Kitui WASH Nutrition Specialist, WASHPaLS Technical Operations and Finance Manager Nyamira WASH Specialist, WASH Nutrition Specialist INTERVIEWS CONDUCTED AT COUNTY AND NATIONAL LEVELS Level Number of Interviews* Kisumu County 21 Kakamega County 9 Nairobi County 3 Kitui County 24 Nyamira County 14 National Level 24 Total 95 *Note that several interviews at the county level in particular were conducted with 3-4 county government staff in attendance. **The WASHPaLS Technical Operations and Finance Manager joined a few interviews in Nairobi the week of December 5. DESIGN CONSIDERATIONS FOR KII AND FGD GUIDES While the KII and FGD guides vary based on respondent categories, they all have some variation of the following sections: • Basic demographic and context data, including interviewee’s sex, role, and location; • Respondent’s relationship to KIWASH: ‒ How the respondent fits within USAID activities, specifically the KIWASH structure or as a partner, beneficiary, or stakeholder, as applicable; • Project background based on respondent experience; • Experience with USAID, KIWASH, and/or partner coordination/outreach, as applicable; • Perception of successes and failures; • Challenges and opportunities; • Relationship with other activities, government, or stakeholders and other factors that could affect outcomes; • Promising practices that should be retained and why, as applicable; and • Sustainability: ‒ Mechanisms for sustainability, and ‒ Challenges to sustainability. For relevant respondents such as staff from KIWASH and USAID/KEA, the evaluation team incorporated other questions to cover the following key themes: • The KIWASH approach to deciding what needs to be done and how; • The KIWASH management and staffing structure over time, what has and has not worked; • The KIWASH thinking on sustainability and the necessary mechanisms to achieve it; ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 14 • The KIWASH approach to gender integration; • The KIWASH ability to modify and adapt—including changes to the KIWASH contract and SOW (e.g., removing nutrition funding after FY17); • The KIWASH identification, interaction, and accountability with partners and key stakeholders; and • KIWASH brand perception and the GOK’s appreciation for KIWASH’s contributions to the sector. DESCRIPTORS USED IN DEDOOSE Field Option List Interviewer CO/KC/EM/MG/JAE/SG and all combinations Org Affiliation County Official/Development Partner, County or National/ KIWASH, County, National, or Regional/USAID/WASH Enterprise/WRUA/WSP Location Kakamega/Kisumu/Kitui/Nairobi/National/Nyamira Level Community/County/National/Regional/Sub-county Sex Female/Male/Mixed Focus Environment/Health/Sanitation/WASH/Water DATA CODING AND ANALYSIS IN DEDOOSE The interview team uploaded transcripts to Dedoose as they were completed. Coding began the week of November 26 and ended on December 7, 2018. To minimize bias and increase consistency in the application of the codes, the Team Leader was the exclusive coder of all data sets collected during the exercise, though all team members reviewed and analyzed coded transcript text related to their area of expertise. In total, 49 codes were applied 1,840 times across the data sets. As would be expected, some codes entered in Dedoose at the outset of the exercise were applied infrequently if at all. Reasons for this are twofold: in some cases these codes proved to be irrelevant and were extraneous to the evaluation; in others, the lack of code application signaled areas where data was missing (either because it was captured elsewhere in the coding or did not come up during the interviews, contrary to what the team anticipated). Beginning the week of December 3, after 80 percent of the data had been coded, the team began to isolate and download coded transcript text related to each evaluation question (EQ) to begin the analysis. Each member of the evaluation team worked with the coded text related to their area of expertise to identify patterns, comparisons, and substantive quotations; eventually generating an initial outline of a response for each EQ. It was at this time that secondary data from the desk review and project monitoring data was again consulted to provide further contextual background, fill in gaps in understanding, and help triangulate findings from the qualitative analysis. The team also attempted to identify patterns in the data to highlight convergence or divergence of opinion across counties, activity types, and stakeholder groups. ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 15 ANNEX III: DATA COLLECTION INSTRUMENTS KEY INFORMANT INTERVIEW TOPIC GUIDE: KIWASH STAFF I. Introduction (10 minutes) a. Thank the respondent for taking the time to participate in the interview. b. Introduction to the researcher and the research: i. Introduce yourself: I am a Consultant residing in ______. I represent an evaluation team fielded by Tetra Tech, a Washington, DC-based firm that has been contracted by the U.S. Agency for International Development (USAID) to conduct an independent evaluation of the USAID KIWASH program. i. As part of its own planning for the next few years, USAID has asked us to conduct a mid￾term evaluation of the KIWASH program to assess its progress toward achieving its objectives. So, what we are trying to assess are the strengths and weaknesses of the program, its accomplishments and best practices, as well as any obstacles and shortcomings faced and how it could be more effective. ii. Ultimately the research will be used to ascertain the extent to which KIWASH activities have improved lives and health through the development and management of sustainable water, sanitation, and hygiene services in Kenya. It will also be used to propose recommendations based on the findings to inform future WASH programming by USAID. c. We will follow privacy protocols to protect your anonymity: i. Explain confidentiality and anonymity and note whether the respondent would like to remain anonymous, and that the assessment team will ask permission if would like to use a quote from the respondent in the final report. ii. Explain how collected data will be stored without identifying information. iii. Ask if the respondent is willing to be recorded and note their response. d. Explain recording, length, and nature of discussion. e. Check whether respondents have any questions. II. KIWASH’s approaches and wider contribution to the sector ( 45 minutes) Transition: I would like to spend some time speaking with you about your knowledge of the USAID￾supported KIWASH program. a. What notable milestones and achievements has the WASH sector realized in Kenya since WASH services were devolved to the counties? b. Does the Kenyan policy environment explicitly endorse and foster market-based approaches for both water and sanitation services? Is there clarity in the Kenyan context about the different This Topic Guide is intended for use in terms of data collection from KIWASH staff. This evaluation will assess the effectiveness of KIWASH’s technical assistance activities to increase viability and professionalism of WASH service providers and support the enabling environment for market-based approaches more generally in Kenya. The evaluation will also help inform USAIDs future decisions on WASH programming in Kenya. It should be followed as closely as possible to guide key informant interviews with these respondents. There are a few questions where a specific set of responses are provided to assist in the collection of data that can be more easily quantified. Instructions to the interviewer are in red. Interviewers will be trained to be compliant with USAID policy in regard to the "USG Common Rule" for the protection of human subjects. ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 16 ways of fostering and the elements required to implement market-based approaches in the WASH sector? c. For WASH service delivery, which KIWASH counties or WSPs do you think are making most significant progress? (i.e., Which counties / WSPs are the ones to watch?) What is driving the observed good progress? How has KIWASH helped to build county capacity as a service provider or as an enabler for service providers in the county in which you are working [or] in the 8 counties in which it is working plus Nairobi? Probe for evidence of effectiveness. d. How has KIWASH helped to build capacity of enterprises (SMEs) that deliver sustainable WASH services in the county in which you are working [or] in the 8 counties in which it is working plus Nairobi? Probe for water, sanitation, and hygiene + evidence of effectiveness. e. In terms of finance, how has KIWASH helped to ensure greater financial viability for service providers in the county in which you are working [or] in the 8 counties in which it is working plus Nairobi? Probe for evidence of effectiveness. f. How has KIWASH helped to enable financing for service providers (i.e., WSPs, enterprises) in the county in which you are working [or] in the 8 counties in which it is working plus Nairobi? g. How specifically has KIWASH worked with sanitation service providers to expand their services? h. How specifically has KIWASH contributed to expanding sanitation services in the county/counties from a demand side (i.e., working with households and communities)? Probe for CLTS and the interaction between supply and demand approaches. i. What has the KIWASH project has done to contribute to gender equality and mainstreaming gender considerations into its approach? j. What is the likelihood of KIWASH achievements being sustained? Probe for: a. At the project level, and b. Influencing stakeholders at the county or national levels. k. What internal factors do you think have enabled KIWASH to be successful in its efforts? [factors within KIWASH control— staffing, budgeting, sequencing of activities…] l. What external factors do you think have enabled KIWASH to be successful in its efforts? [factors beyond KIWASH control—policy environment, support of other like-minded actors, …] m. Are there any internal or external factors (i.e., both within or beyond KIWASH’s direct control) that you think are undermining or affecting their efforts and performance? n. How specifically has KIWASH influenced government policy with regard to WASH services? Probe for water, sanitation, and hygiene o. For county-level KIWASH staff: Are you working with other development partners in your county around WASH service delivery? Describe any overlap or complementarity in what other organizations do and what KIWASH does? Probe for capacity building, finance, construction. p. What do you feel has been KIWASH’s most significant accomplishment to date? q. What would you like to see KIWASH build on in its final two years to have greatest impact at the county level? At the national level? r. What recommendations, if any, do you have for USAID with regard to the design and delivery of future programs to strengthen the WASH sector to ensure and expand access to sustainable WASH services? III. [As appropriate, specific questions to emerge for each county based on the document review – lending, grants, sanitation specific, working with WRUAs, 1 i.e., particular progress in certain areas…] 1 KIWASH work with WRUAs may require a further (limited) set of questions. To be determined at a later stage. ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 17 IV. Conclusion: • Thank the respondent for their time. • Tell the respondent they are welcome to contact you to ask questions at a later date. Ask permission of the respondent to use their name in the report if you might use a quote. Note their response: ____________. ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 18 KEY INFORMANT INTERVIEW TOPIC GUIDE: COUNTY-LEVEL STAFF (NOT WSPS) I. Introduction (10 minutes) a. Thank the respondent for taking the time to participate in the interview. b. Introduction to the researcher and the research: i. Introduce yourself: I am a Consultant residing in ______. I represent an evaluation team fielded by Tetra Tech, a Washington, DC-based firm that has been contracted by the U.S. Agency for International Development (USAID) to conduct an independent evaluation of the USAID KIWASH program. ii. As part of its own planning for the next few years, USAID has asked us to conduct a mid￾term evaluation of the KIWASH program to assess its progress toward achieving its objectives. So, what we are trying to assess are the strengths and weaknesses of the program, its accomplishments and best practices as well as any obstacles and shortcomings faced and how it could be more effective. iii. Ultimately the research will be used to ascertain the extent to which KIWASH activities have improved lives and health through the development and management of sustainable water, sanitation, and hygiene services in Kenya. It will also be used to propose recommendations based on the findings to inform future WASH programming by USAID. c. We will follow privacy protocols to protect your anonymity: i. Explain confidentiality and anonymity and note whether the respondent would like to remain anonymous, and that the assessment team will ask permission if would like to use a quote from the respondent in the final report. ii. Explain how collected data will be stored without identifying information. iii. Ask if the respondent is willing to be recorded and note their response. d. Explain recording, length, and nature of discussion. e. Check whether respondents have any questions. II. KIWASH’s contribution to the sector ( 45 minutes) Transition: I would like to spend some time speaking with you about your knowledge of the USAID￾supported KIWASH program. a. What notable milestones and achievements has the WASH sector realized in your county since WASH services were devolved to the counties? b. For WASH service delivery, which counties or WSPs do you think are making most significant progress? (i.e., Which counties/WSPs are the ones to watch?) Why do you think this is the case? Probe briefly to understand how much they know about other counties in the program. c. Does the Kenyan policy environment explicitly endorse and foster market-based approaches for both water and sanitation services? Is there clarity in the Kenyan context about the different This Topic Guide is intended for use in field research with representatives of county-level staff that are decision makers for water service delivery and oversight, health, and finance. This evaluation will assess the effectiveness of KIWASH’s technical assistance activities to increase viability and professionalism of WASH service providers and support the enabling environment for market-based approaches more generally in Kenya. The evaluation will also help inform USAID’s future decisions on WASH programming in Kenya. It should be followed as closely as possible to guide key informant interviews with these respondents. There are a few questions where a specific set of responses are provided to assist in the collection of data that can be more easily quantified. Instructions to the interviewer are in red. Interviewers will be trained to be compliant with USAID policy in regard to the "USG Common Rule" for the protection of human subjects. ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 19 ways of fostering and the elements required to implement market-based approaches in the WASH sector? d. How familiar are you with the KIWASH program? e. In what areas has your institution worked/collaborated with the KIWASH program (i.e., water, sanitation/health, finance)? f. In your opinion, what are the challenges that a project like KIWASH should be addressing? g. Do you feel KIWASH has worked collaboratively with county government and local stakeholders to define its priorities and align with government planning cycles and strategies? h. In your view, in what area of intervention has KIWASH made the greatest contribution and been particularly successful in your country? Probe for examples: i. County capacity building for monitoring service delivery, planning, investment, monitoring and regulatory functions, etc.; ii. Financing models; iii. WSP and enterprise support; iv. Government policy influencing; v. Nutrition’ vi. Construction, extension of water services’ vii. Sanitation’ and viii. Gender mainstreaming. i. In terms of finance, how has KIWASH helped improve access to financing or increased revenues for service providers in your county? j. How has KIWASH contributed to expanding water and sanitation services in the county from a supply side (i.e., working with providers)? Probe for examples of both water and sanitation. k. How has KIWASH contributed to expanding water and sanitation services in the county from a demand side (i.e., working with households and communities)? Probe for examples of both water and sanitation. l. In your view what specific aspects of the WASH and nutrition intervention were successful? Do you think there is evidence to back this up? Probe for: i. WASH & nutrition integration capacity enhancement activities; ii. WASH & nutrition integration interventions at healthcare facilities (HCFs); iii. WASH & nutrition integration interventions at the community level; iv. Promotion of sanitation and hygiene at the community level through CLTS and sanitation marketing; and v. Gender mainstreaming. m. What is the likelihood of KIWASH’s achievements being sustained? Probe for: i. At the service level, ii. Influencing stakeholders at the county or national levels, and iii. Around gender mainstreaming. n. What would you like to see KIWASH build on to have greatest impact at the county level? At the national level? o. Is there anything that you think KIWASH could be doing differently to have greatest impact at both the national level and in your county? p. Are you working with other development partners in your county around WASH service delivery? Is there any overlap in what other organizations do and what KIWASH does? q. What is the reputation and “brand” of KIWASH amongst you and your colleagues at the county level? r. What recommendations, if any, do you have for USAID with regard to the design and delivery of such programs to strengthen the WASH sector to ensure and expand access to sustainable WASH services? III. Conclusion: ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 20 • Thank the respondent for their time. • Tell the respondent they are welcome to contact you to ask questions at a later date. Ask permission of the respondent to use their name in the report if you might use a quote. Note their response: ____________. ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 21 KEY INFORMANT INTERVIEW TOPIC GUIDE: NATIONAL-LEVEL COLLABORATORS AND DEVELOPMENT PARTNERS I. Introduction (10 minutes) a. Thank the respondent for taking the time to participate in the interview. b. Introduction to the researcher and the research: i. Introduce yourself: I am a Consultant residing in ______. I represent an evaluation team fielded by Tetra Tech, a Washington, DC-based firm that has been contracted by the U.S. Agency for International Development (USAID) to conduct an independent evaluation of the USAID KIWASH program. ii. As part of its own planning for the next few years, USAID has asked us to conduct a mid￾term evaluation of the KIWASH program to assess its progress toward achieving its objectives. So, what we are trying to assess are the strengths and weaknesses of the program, its accomplishments and best practices, as well as any obstacles and shortcomings faced and how it could be more effective. iii. Ultimately the research will be used to ascertain the extent to which KIWASH activities have improved lives and health through the development and management of sustainable water, sanitation, and hygiene services in Kenya. It will also be used to propose recommendations based on the findings to inform future WASH programming by USAID. c. We will follow privacy protocols to protect your anonymity: i. Explain confidentiality and anonymity and note whether the respondent would like to remain anonymous, and that the assessment team will ask permission if would like to use a quote from the respondent in the final report. ii. Explain how collected data will be stored without identifying information. iii. Ask if the respondent is willing to be recorded and note their response. d. Explain recording, length, and nature of discussion. e. Check whether respondents have any questions. II. KIWASH’s contribution to the sector ( 45 minutes) Transition: I would like to spend some time speaking with you about your knowledge of the USAID￾supported KIWASH program. a. What notable milestones and achievements has the WASH sector realized since WASH services were devolved to the counties? b. For WASH service delivery, which counties or WSPs do you think are making most significant progress? (i.e., Which counties / WSPs are the ones to watch?) Why do you think this is the case? c. Does the Kenyan policy environment explicitly endorse and foster market-based approaches for both water and sanitation services? Is there clarity in the Kenyan context about the different This Topic Guide is intended for use in field research with representatives of national-level collaborators and Development Partners. This evaluation will assess the effectiveness of KIWASH’s technical assistance activities to increase viability and professionalism of SME WASH service providers and support the enabling environment for market-based approaches more generally in Kenya. The evaluation will also help inform USAID’s future decisions on WASH programming in Kenya. It should be followed as closely as possible to guide key informant interviews with these respondents. There are a few questions where a specific set of responses are provided to assist in the collection of data that can be more easily quantified. Instructions to the interviewer are in red. Interviewers will be trained to be compliant with USAID policy in regard to the "USG Common Rule" for the protection of human subjects. ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 22 ways of fostering and elements required to implement market-based approaches in the WASH sector? d. How familiar are you with the KIWASH program, its approaches and achievements to date? e. In what areas has your institution worked/collaborated with the KIWASH program? f. In your opinion, what are the challenges in the WASH sector that a project like KIWASH should be addressing? In your opinion, is KIWASH designed to address these issues? g. In your view, in what area of intervention has KIWASH made the greatest contribution and been particularly successful? Probe for: a. County capacity building for service delivery, investment, oversight, and regulatory functions, etc.; b. Financing models; c. WSP and enterprise support; d. Service expansion; e. Gender mainstreaming; and f. Government policy influencing. h. Is there any indication that KIWASH counties are performing better than non-KIWASH counties? i. What is the likelihood of KIWASH achievements being sustained? Probe for: a. At the project level, b. Influencing stakeholders at the county or national levels, and c. Around gender mainstreaming. j. What internal factors do you think have enabled KIWASH to be successful in its efforts? [factors within KIWASH control—staffing, budgeting, technical approach, sequencing of activities…] k. What external factors do you think have enabled KIWASH to be successful in its efforts? [factors beyond KIWASH control—policy environment, support of other like-minded actors, …] l. Are there any internal or external factors (i.e., both within or beyond KIWASH’s direct control) that you think are undermining or affecting their efforts? m. Is there any overlap in what other organizations do and what KIWASH does? n. What would you like to see KIWASH build on in its final two years to have greatest impact at the national level and at the county level? o. Is there anything that you think KIWASH could be doing differently to have greatest impact at both national and county levels? p. What is the reputation and “brand” of KIWASH in Kenya generally? q. What recommendations, if any, do you have for USAID with regard to the design and delivery of future programs to strengthen the WASH sector to ensure and expand access to sustainable WASH services? III. Conclusion: • Thank the respondent for their time. • Tell the respondent they are welcome to contact you to ask questions at a later date. Ask permission of the respondent to use their name in the report if you might use a quote. Note their response: ____________. ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 23 KEY INFORMANT INTERVIEW TOPIC GUIDE: WASH SERVICE PROVIDERS (COUNTY-LEVEL OR UTILITY) RECEIVING KIWASH SUPPORT I. Introduction (10 minutes) a. Thank the respondent for taking the time to participate in the interview. b. Introduction to the researcher and the research: i. Introduce yourself: I am a Consultant residing in ______. I represent an evaluation team fielded by Tetra Tech, a Washington, DC-based firm that has been contracted by the U.S. Agency for International Development (USAID) to conduct an independent evaluation of the USAID KIWASH program. ii. As part of its own planning for the next few years, USAID has asked us to conduct a mid￾term evaluation of the KIWASH program to assess its progress toward achieving its objectives. So, what we are trying to assess are the strengths and weaknesses of the program, its accomplishments and best practices, as well as any obstacles and shortcomings faced and how it could be more effective. iii. Ultimately the research will be used to ascertain the extent to which KIWASH activities have improved lives and health through the development and management of sustainable water, sanitation, and hygiene services in Kenya. It will also be used to propose recommendations based on the findings to inform future WASH programming by USAID. c. We will follow privacy protocols to protect your anonymity: i. Explain confidentiality and anonymity and note whether the respondent would like to remain anonymous, and that the assessment team will ask permission if would like to use a quote from the respondent in the final report. ii. Explain how collected data will be stored without identifying information. iii. Ask if the respondent is willing to be recorded and note their response. d. Explain recording, length, and nature of discussion. e. Check whether respondents have any questions. II. Capacity Building of WASH Service Providers ( 45 minutes) Transition: I would like to spend a little time speaking with you about your experience working with the USAID-supported KIWASH program. a. Please describe the WSP—note: size, scope of services, location, other information about maturity. Is the WASH enterprise managed by the county, city, or independently run? Does the WSP handle both water and sanitation services? b. What is the catchment area for your service? Technologies used? How many customers do you have? Is there room for growth? c. When were you first introduced to KIWASH and how were you selected to receive KIWASH support? Could you describe your level of interaction with KIWASH-funded activities? d. Based on your experience, please describe KIWASH’s approach to capacity building of WSPs? (key words: CAP analysis tool.) This Topic Guide is intended for use in field research with representatives of WASH service providers at the county or utility level. This evaluation will assess the effectiveness of KIWASH’s technical assistance activities to increase viability and professionalism of WASH service providers, and further, will inform USAID’s future decisions on WASH programming in Kenya. It should be followed as closely as possible to guide key informant interviews with these respondents. There are a few questions where a specific set of responses are provided to assist in the collection of data that can be more easily quantified. Instructions to the interviewer are in red. Interviewers will be trained to be compliant with USAID policy in regard to the "USG Common Rule" for the protection of human subjects. ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 24 e. Has your WSP substantially benefited from KIWASH’s technical assistance? If yes, please provide examples. If yes, what was the most useful skill or capacity KIWASH has enhanced? Probe for business and financial management. f. Do you feel the WSP has benefited from individual coaching and mentoring from KIWASH? Has KIWASH’s approach been responsive to the specific needs of your WSP? g. Are there other types of technical assistance that KIWASH could have provided that would have been beneficial to the WSP? h. Did the WSP apply to the KIWASH grants program? Why or why not? i. Did the WSP receive a grant? If yes, what was the scope of the project? If the WSP was not awarded a grant, probe why the application was unsuccessful. j. If you did receive a grant, are you paying back the grant on schedule? k. Without access to the grant funding for infrastructure expansion, would KIWASH’s approach to capacity building have been as effective? l. Has the WSP been able to access commercial loans as a result of KIWASH support? Is this something that you and your colleagues in the WSP are keen to do? m. How, if at all, would you recommend KIWASH adjust its programming to better support WSPs like yours? n. How, if at all, do you address gender in your programming or planning? Were gender aspects emphasized by KIWASH? Probe for: female employment, outreach to female customers. o. For WSPs providing sanitation services: In what ways has KIWASH helped you to expand your customer coverage of sanitation services [supply side]? p. What is KWASH’s progress in helping the WSP to promote appropriate low-cost sanitation products and services [demand creation]? q. Has KIWASH been able to help you identify gaps in addressing women’s needs in sanitation and hygiene products being marketed? r. Has the viability of the WSP [NAME] improved since benefiting from KIWASH support? Do you have evidence of: a. Expanded coverage, b. Increased revenues, or c. Reduced non-revenue water? s. What is your overall view of KIWASH? Under what circumstances would you recommend that other WSPs engage with KIWASH programming? III. Other questions ( 15 minutes) a. Do you feel that KIWASH’s approach to strengthen WASH service providers is complemented by other efforts to increase oversight and monitoring capacity of Water Boards and county governments? b. In terms of sustainability, what are the main challenges you face beyond areas where KIWASH has provided support? c. What other donor organizations/NGOs does your WSP work with? What do these other organizations do? d. Is there any overlap in what these organizations do and what KIWASH does? e. What is the reputation and “brand” of KIWASH among in your county? f. What activities do you think should be scaled up to maximize project results during the remainder of its implementation? IV. Conclusion: • Thank the respondent for their time. • Tell the respondent they are welcome to contact you to ask questions at a later date. ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 25 Ask permission of the respondent to use their name in the report if you might use a quote. Note their response: ____________. ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 26 KEY INFORMANT INTERVIEW TOPIC GUIDE – WASH ENTERPRISES RECEIVING KIWASH SUPPORT (PRIVATE SECTOR AND COMMUNITY-MANAGED SERVICE PROVIDERS) I. Introduction (10 minutes) a. Thank the respondent for taking the time to participate in the interview. b. Introduction to the researcher and the research: i. Introduce yourself: I am a Consultant residing in ______. I represent an evaluation team fielded by Tetra Tech, a Washington, DC-based firm that has been contracted by the U.S. Agency for International Development (USAID) to conduct an independent evaluation of the USAID KIWASH program. ii. As part of its own planning for the next few years, USAID has asked us to conduct a mid￾term evaluation of the KIWASH program to assess its progress toward achieving its objectives. So, what we are trying to assess are the strengths and weaknesses of the program, its accomplishments and best practices, as well as any obstacles and shortcomings faced and how it could be more effective. iii. Ultimately the research will be used to ascertain the extent to which KIWASH activities have improved lives and health through the development and management of sustainable water, sanitation, and hygiene services in Kenya. It will also be used to propose recommendations based on the findings to inform future WASH programming by USAID. c. We will follow privacy protocols to protect your anonymity: i. Explain confidentiality and anonymity and note whether the respondent would like to remain anonymous, and that the assessment team will ask permission if would like to use a quote from the respondent in the final report. ii. Explain how collected data will be stored without identifying information. iii. Ask if the respondent is willing to be recorded and note their response. d. Explain recording, length, and nature of discussion. e. Check whether respondents have any questions. II. Capacity Building of WASH enterprises ( 45 minutes) Transition: I would like to spend a little time speaking with you about your experience working with the USAID-supported KIWASH program. a. Please describe your WASH enterprise—note: size, scope of services, location, other information about maturity, woman or men owned. Are you a formal business? Is the business legally registered? Is the WASH enterprise a Community-Managed Scheme or a private business? Provide water and sanitation services? b. What is the catchment area for your service? Technologies used? How many customers do you have? Is there room for growth? c. When were you first introduced to KIWASH and how were you selected to receive KIWASH support? Could you describe your level of interaction with KIWASH-funded activities? This Topic Guide is intended for use in field research with representatives of SME WASH service providers. This evaluation will assess the effectiveness of KIWASH’s technical assistance activities to increase viability and professionalism of SME WASH service providers, and further, will inform USAID’s future decisions on WASH programming in Kenya. It should be followed as closely as possible to guide key informant interviews with these respondents. There are a few questions where a specific set of responses are provided to assist in the collection of data that can be more easily quantified. Instructions to the interviewer are in red. Interviewers will be trained to be compliant with USAID policy in regard to the "USG Common Rule" for the protection of human subjects. ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 27 d. Based on your experience, please describe KIWASH’s approach to capacity building of WASH enterprises? (key words: CAP analysis tool.) e. Has your business tangibly benefited from KIWASH’s technical assistance? If yes, please provide examples. If yes, what was the most useful skill or capacity KIWASH has enhanced? f. Do you feel your business has benefited from individual “mentoring” and “incubation”? Has KIWASH’s approach been responsive to the specific needs of your enterprise? g. Are there other types of technical assistance that KIWASH could have provided that would have been beneficial to you? h. Has KIWASH linked you up with other likeminded businesses operating in your county? i. Did you apply to the KIWASH grants program? Why or why not? j. Did you receive a grant? If yes, what was the scope of the project? k. If you did receive a grant, are you paying back the grant on schedule? l. Without access to the grant funding for infrastructure expansion, would KIWASH’s approach to capacity building have been as effective? m. How, if at all, would you recommend KIWASH adjust its programming to better support entrepreneurs like yourself? n. How, if at all, do you address gender in your programming or planning? Were gender aspects emphasized by KIWASH? o. Has this approach helped you see the benefit of inclusivity as a profitable business model? p. How exactly did KIWASH encourage women-led businesses to apply for funding or bid on tenders? How many so far have succeeded in doing so directly from KIWASH’s assistance? q. Have KIWASH programs resulted in more women being employed? r. How have KIWASH market-based activities fostered the expansion of sustainable services to women and girls? s. Has the viability of your business improved since benefiting from KIWASH support? Do you intend to remain in the WASH services business for the foreseeable future? t. What is your overall view of KIWASH? How has KIWASH impacted the effectiveness and viability of service providers in your county? III. Other questions ( 15 minutes) g. What interaction do you have with county government, Water Boards, or other regulators? How is your relationship with these bodies? Is this formal or informal? h. Do you feel that KIWASH’s approach to strengthen WASH service providers is sufficiently linked with an effort to increase capacity with the Water Boards and county governments? i. In terms of sustainability, what are the main challenges you face beyond areas where KIWASH has provided support? j. What other donor organizations/NGOs does your enterprise work with? What do these other organizations do? k. Is there any overlap in what these organizations do and what KIWASH does? l. What is the reputation and “brand” of KIWASH among the private sector providers in your county? m. What activities do you think should be scaled up to maximize project results during the remainder of its implementation? IV. Conclusion: • Thank the respondent for their time. • Tell the respondent they are welcome to contact you to ask questions at a later date. Ask permission of the respondent to use their name in the report if you might use a quote. Note their response: ____________. ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 28 KEY INFORMANT INTERVIEW AND FGD GUIDES: WASH AND NUTRITION INTEGRATION STAKEHOLDERS Interview questions to be merged with other KII guides above depending on the interviewee. I. Introduction (10 minutes) a. Thank the respondent for taking the time to participate in the interview. b. Introduction to the researcher and the research: i. Introduce yourself: I am a Consultant residing in ______. I represent an evaluation team fielded by Tetra Tech, a Washington, DC-based firm that has been contracted by the U.S. Agency for International Development (USAID) to conduct an independent evaluation of the USAID KIWASH program. ii. As part of its own planning for the next few years, USAID has asked us to conduct a mid￾term evaluation of the KIWASH program to assess its progress toward achieving its objectives. So, what we are trying to assess are the strengths and weaknesses of the program, its accomplishments and best practices, as well as any obstacles and shortcomings faced and how it could be more effective. iii. Ultimately the research will be used to ascertain the extent to which KIWASH activities have improved lives and health through the development and management of sustainable water, sanitation and hygiene services in Kenya. It will also be used to propose recommendations based on the findings to inform future WASH programming by USAID. c. We will follow privacy protocols to protect your anonymity: i. Explain confidentiality and anonymity and note whether the respondent would like to remain anonymous, and that the assessment team will ask permission if would like to use a quote from the respondent in the final report. ii. Explain how collected data will be stored without identifying information. iii. Ask if the respondent is willing to be recorded and note their response. d. Explain recording, length, and nature of discussion. e. Check whether respondents have any questions. II. WASH and Nutrition Integration ( 45 minutes) Transition: I would like to spend a little time speaking with you about your experience working with the USAID-supported KIWASH program. A) KIWASH Staff focused on WASH & Nutrition (National & regional staff)2 2 As appropriate, these questions can be integrated into interview guides for KIWASH staff above. This Topic Guide is intended for use in field research with KIWASH staff working on synergies between WASH and nutrition, representatives of Public Health Officials at national and county levels, and healthcare workers. This evaluation will assess the effectiveness of KIWASH’s technical assistance activities to increase viability and professionalism of SME WASH service providers, and further, will inform USAID’s future decisions on WASH programming in Kenya. It should be followed as closely as possible to guide key informant interviews with these respondents. There are a few questions where a specific set of responses are provided to assist in the collection of data that can be more easily quantified. Instructions to the interviewer are in red. Interviewers will be trained to be compliant with USAID policy in regard to the "USG Common Rule" for the protection of human subjects. ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 29 1. Kindly describe the KIWASH management and staffing structure that supports/supported the WASH and nutrition integration component. 2. During the years that the nutrition component was implemented, what was the program’s approach to integrating WASH and nutrition interventions? 3. What are the key WASH and nutrition interventions implemented through KIWASH? Which do you feel were most successful? 4. How were sector-specific strategies and targets established for each? 5. Has KIWASH worked with other USAID health and nutrition programs? If so which ones, and to what extent. Probe for: • National-level coordination, • Joint programming/planning, • Co-location, • Capacity enhancement (training, development of guidelines), and • Joint monitoring and evaluation (M&E). 6. How did KIWASH work with county and sub-county government institutions in the implementation of the WASH and nutrition integration? Probe for: • National-level coordination, • Joint programming/planning, • Co-location, • Capacity enhancement (training, development of guidelines), and • Joint M&E. 7. How did/does KIWASH track WASH and nutrition results? 8. In your view, in what area of WASH and nutrition integration has KIWASH made the greatest contribution and been particularly successful? Is there any indication/evidence that KIWASH counties are performing better than non-KIWASH counties because of taking an integrated approach? 9. What factors internal to KIWASH do you think led to this success? 10. What factors external to KIWASH do you think could have helped? 11. With the focus of KIWASH having shifted away from integrated WASH and nutrition work, do you think the outcomes realized in the first 2½ years of implementation will be sustained? 12. How did the discontinuation of the funding affect the program? Probe for: o Management o Implementation 13. How did KIWASH adapt/modify implementation strategies following the changes? B) Government Institutions (MOH & MOA/ other USAID WASH, Nutrition partners (National Level) 1. How familiar are you with the KIWASH program? 2. In what areas has your institution worked/collaborated with the KIWASH’s WASH and nutrition program? Probe for: • Participation in training, and • Adaptation of guidelines/development of job aids. 3. What is your role within KIWASH’s WASH and nutrition activities as a partner? 4. In your view, in what area of WASH/nutrition integration has KIWASH made the greatest contribution and been particularly successful? Probe for: • WASH & nutrition integration capacity enhancement activities, • WASH & nutrition integration interventions at HCFs, • WASH & nutrition integration interventions at the community level, and ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 30 • Gender aspects and gender mainstreaming. 5. With regard to gender specifically, how has collecting gender-disaggregated data helped in program implementation planning and what course corrections and budget reallocations have been made as a result of using this data in M&E systems? 6. How is KIWASH using the Gender Assessment Tool in assessing performance of the program so far and what changes, if any, have been made by the program as a result of using this tool? What is the progress of assisting counties to implement their gender framework? Probe for: training, refined design, sustainability. 7. Are you familiar with any implementation changes on WASH and nutrition within KIWASH? 8. How did the changes affect the program? 9. How did or is the ministry/partner adapting to ensure gains are not lost? What from the KIWASH program is likely to be sustained? 10. Any suggestions for the design of future programing of a similar program? C) KIWASH staff at county level 1. What is the strategy used for implementing WASH and nutrition interventions? 2. For the first 3 years of implementation, kindly describe how KIWASH managed the implementation of WASH and nutrition integration in your county. How is/was the staffing structure? Have there been changes in the structure? If so, why? 3. Are there other USAID health, Nutrition programs in your county? 4. If yes, how has KIWASH worked with other USAID partners in this county? Probe for: • Joint programming/planning, • Co-location, • County-level coordination, • Capacity enhancement (training, development of guidelines), and • Joint M&E. 5. How has KIWASH worked with government institutions (collaborations) in the implementation of the WASH and nutrition integration? Probe for: • Joint programming/planning, • Co-location, • County-level coordination, • Capacity enhancement (training, development of ORT, NACS guidelines, WASH strategy, and job aids), and • Joint M&E. 6. What integrated WASH and nutrition activities have KIWASH sponsored in this county? Probe for: • Who/how many were trained (professionals); • The expected roles of trainees after the trainings; • Action plans developed during the training, how were they implemented, and the impact of those actions.; and • Whether KIWASH followed up with the trainees in any way. 7. How did KIWASH track WASH and nutrition results? 8. What has been the impact so far? 9. Were WASH and nutrition activities linked to issues of gender, and women and girls’ empowerment? 10. How has collecting gender-disaggregated data helped in project implementation planning and what course corrections and budget reallocations have been made as a result of using this data in M&E systems? ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 31 11. How is KIWASH using the Gender Assessment Tool in assessing performance of the program so far and what changes, if any, have been made by the program as a result of using this tool? What is the progress of assisting counties to implement their gender framework? Probe for: training, refined design, sustainability. 12. In your view, what specific aspects of the WASH and nutrition intervention were successful? Do you think there is evidence to back this up? Probe for: • WASH & nutrition integration capacity enhancement activities, • WASH & nutrition integration interventions at HCFs, and • WASH & nutrition integration interventions at the community level. 13. What factors internal to KIWASH do you think led to this? 14. What factors external to KIWASH do you think could have helped? 15. For the first 3 years of implementation, what did KIWASH put in place that would ensure sustainability of WASH and nutrition outcomes? 16. How did the limitation of nutrition funding affect the WASH and nutrition integration program (staffing, implementation)? 17. What would you say were the missed opportunities? 18. What would you like to see KIWASH build on to have greatest impact/sustainable impact? 19. Any suggestions for the design of future programing of a similar program? D) County-Level Government Institutions (Public Health, Nutrition, Agriculture) and implementing partners 1. How familiar are you with the KIWASH program? 2. What is your role within KIWASH’s WASH and nutrition activities as a partners? 3. In what areas has your institution worked/collaborated with the KIWASH’s WASH and nutrition program? Probe for: • Joint programming/planning, • Co-location, • County-level coordination, • Capacity enhancement (training, development of guidelines), and • Joint M&E. 4. Have you participated in any of WASH and nutrition integration training? What was the training about? Was the training organized by KIWASH? 5. If yes, what were the planned rollout actions? Which of the action plans have been implemented since the training? 6. How were the results monitored? Probe for: • Joint monitoring activities, • MOH monitoring, or • KIWASH monitoring. 7. What do you think KIWASH could be doing differently to integrate WASH/nutrition to have greatest impact at the county level? 8. Any suggestions for the design of future programing of a similar program? E) Healthcare workers (ANC, PNC, ORT) 1. How familiar are you with the KIWASH program? 2. What is your role within KIWASH’s WASH and nutrition activities at the health facility? Probe for: • EWAs are integrated in the NACS guidelines, • EWAs and ENAs are integrated in the ORT guidelines, • EWA messaging (health talks, posters, brochures) at the ANC service delivery point, ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 32 • EWA messaging (health talks, posters, brochures) at the PNC service delivery point, • EWAs messaging (health talks, posters, brochures) at the ORT service delivery point, • Public handwashing facilities and safe drinking water points at the ORT corners, and • Water and hygiene kits are distributed through the ANC corners. 3. Have you participated in any of WASH and nutrition integration trainings? What was the training about? Was the training organized by KIWASH? a. Who was trained (cadres)? b. How many were trained in this healthcare facility? c. What were the expected roles after the trainings? d. Any action plans developed during the training? e. Did KIWASH follow up with the trainees in any way? f. How have the trainees implemented the actions? g. What has been the impact? 4. What were the planned rollout actions? Which of the action plans have been implemented since the training? 5. How did KIWASH track WASH and nutrition results? Did KIWASH follow up with you and your colleagues in any way regarding the training? 6. What has been the impact so far? 7. What do you think the program has done to contribute to gender mainstreaming? What is the likelihood of these achievements being sustained? 8. In your view, what specific aspects of the WASH and nutrition intervention were successful at the health facility level? Is there evidence to back this up? 9. What specific aspects of the WASH and nutrition interventions do you think would have been improved? 10. What do you think KIWASH could be doing differently to have greatest impact in WASH and nutrition integration at the HCF level? 11. Any suggestions for the design of future programing of a similar program? F) Focus Group Discussions with CHV and Farmers’ Groups 1. How familiar are you with the KIWASH program? 2. What is the role of CHVs/farmers’ groups in the community? 3. What is your role within KIWASH’s WASH and nutrition activities at the community level? Probe for: • Promotion of EWAs and ENAs at the community level through CHVs, • Promotion of EWAs and ENAs at the community level through lead farmers, • Promotion of sanitation and hygiene at the community level through CLTS and sanitation marketing, and 4. Have you participated in any of WASH and nutrition integration trainings? What was the training about? Was the training organized by KIWASH? 5. What were the planned rollout actions? Which of the action plans have been implemented since the training? 6. Did KIWASH follow up with you and your colleagues in any way regarding the training? How did KIWASH track WASH and nutrition results? Probe for: • Joint monitoring activities, • MOH monitoring, and • KIWASH monitoring. 7. What has been the impact so far? 8. In your opinion, what has been the impact of the training to the CHVs/farmers’ work at the community level? Is there evidence to back this up? ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 33 9. In your view, what specific aspects of the WASH and nutrition interventions were successful? Is there evidence to back this up? 10. How has KIWASH engaged with and influenced stakeholders around gender issues and women and girls’ empowerment? 11. How has KIWASH encouraged men to participate in activities that are considered “female” responsibilities like handwashing, nutrition, diseases like diarrhea, etc.? 12. What have been the challenges faced in the implementation? 13. Are there particular challenges that a project like KIWASH should be addressing? 14. What do you think KIWASH could be doing differently to have greatest impact at both national and county levels? ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 34 KEY INFORMANT INTERVIEW TOPIC GUIDE: USAID I. Introduction (10 minutes) a. Thank the respondent for taking the time to participate in the interview. b. Introduction to the researcher and the research: i. Introduce yourself: I am a Consultant residing in ______. I represent an evaluation team fielded by Tetra Tech, a Washington, DC-based firm that has been contracted by the U.S. Agency for International Development (USAID) to conduct an independent evaluation of the USAID KIWASH program. ii. As part of its own planning for the next few years, USAID has asked us to conduct a mid￾term evaluation of the KIWASH program to assess its progress toward achieving its objectives. So, what we are trying to assess are the strengths and weaknesses of the program, its accomplishments and best practices, as well as any obstacles and shortcomings faced and how it could be more effective. iii. Ultimately the research will be used to ascertain the extent to which KIWASH activities have improved lives and health through the development and management of sustainable water, sanitation and hygiene services in Kenya. It will also be used to propose recommendations based on the findings to inform future WASH programming by USAID. c. We will follow privacy protocols to protect your anonymity: i. Explain confidentiality and anonymity and note whether the respondent would like to remain anonymous, and that the assessment team will ask permission if would like to use a quote from the respondent in the final report. ii. Explain how collected data will be stored without identifying information. iii. Ask if the respondent is willing to be recorded and note their response. d. Explain recording, length, and nature of discussion. e. Check whether respondents have any questions. II. KIWASH’s approaches and wider contribution to the sector ( 45 minutes) Transition: I would like to spend some time speaking with you about your knowledge of the USAID￾supported KIWASH program. a. What notable milestones and achievements has the WASH sector realized in Kenya since WASH services were devolved to the counties? b. Does the Kenyan policy environment explicitly endorse and foster market-based approaches for both water and sanitation services? Is there clarity in the Kenyan context about the different ways of fostering and the elements required to implement market-based approaches in the WASH sector? c. For WASH service delivery, which KIWASH counties or WSPs do you think are making most significant progress? (i.e., Which counties / WSPs are the ones to watch?) Why do you think this is the case? This Topic Guide is intended for use in terms of data collection from USAID technical staff working on WASH and nutrition. This evaluation will assess the effectiveness of KIWASH’s technical assistance activities to increase viability and professionalism of WASH service providers and support the enabling environment for market-based approaches more generally in Kenya. The evaluation will also help inform USAID’s future decisions on WASH programming in Kenya. It should be followed as closely as possible to guide key informant interviews with these respondents. There are a few questions where a specific set of responses are provided to assist in the collection of data that can be more easily quantified. Instructions to the interviewer are in red. Interviewers will be trained to be compliant with USAID policy in regard to the "USG Common Rule" for the protection of human subjects. ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 35 d. How have you seen KIWASH linking in with other USAID or development partner programs to ensure that the policy, capacity and investment environment is enabling expanded and sustained service delivery for water and sanitation? Probe for evidence of effectiveness. e. Is there any overlap in what other organizations do and what KIWASH does? Probe for capacity building, finance, construction f. Do you think KIWASH has the balance right between water and sanitation? g. For sanitation, do you think KIWASH is appropriately linking up the supply and demand sides? h. What internal factors do you think have enabled KIWASH to be successful in its efforts? [factors within KIWASH control—staffing, budgeting, sequencing of activities…] i. What external factors do you think have enabled KIWASH to be successful in its efforts? [factors beyond KIWASH control—policy environment, support of other like-minded actors, …] j. Are there any internal or external factors (i.e., both within or beyond KIWASH’s direct control) that you think are undermining or affecting their efforts and performance? k. How specifically has KIWASH influenced government policy with regard to WASH services? Probe for water, sanitation and hygiene. l. What is the reputation and “brand” of KIWASH amongst you and your colleagues at the county level? m. What do you feel has been KIWASH’s most significant accomplishment to date? n. What would you like to see KIWASH build on in its final two years to have greatest impact at the county level? At the national level? o. What recommendations, if any, do you have for future programming with regard to the design and delivery of future programs to strengthen the WASH sector to ensure and expand access to sustainable WASH services? III. Conclusion: • Thank the respondent for their time. • Tell the respondent they are welcome to contact you to ask questions at a later date. Ask permission of the respondent to use their name in the report if you might use a quote. Note their response: ____________. ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 36 ANNEX IV: DATA SOURCES LIST OF KEY INFORMANTS Below are the names of the key informants consulted during the review categorized by type Respondent Name Title Location / Geographic Focus KIWASH Staff (25) Joe Sanders Chief of Party Nairobi / National Japheth Mbuvi Deputy Chief of Party Nairobi / National Fredrick Odhiambo M&E Specialist Nairobi / National Timothy Grayling Water and Sanitation Advisor (former) Nairobi / National Nancy Ndirangu Services Capacity Building Manager Nairobi / National Diana Mulatya Agriculture and Nutrition Advisor/Gender focal person Nairobi / National Roselyne Okwiri Sanitation and Nutrition Manager Kisumu / Regional Calvince Odera WASH Governance Specialist Kisumu / Regional Ronald Kubasu Service Provider Capacity Building Manager – Western Region Kisumu / Regional Miriam Adongo Environmental Specialist Kisumu / Regional Morris Muteti WASH Governance Specialist / Team Lead Kitui Josephine Kithinji Sanitation and Hygiene Specialist Kitui Faith Mumbua Environment & Climate Change Sustainability Specialist Kitui Euphrasia Luseka Team Leader Kakamega Joel Kiarie Specialist and Hygiene Focal Point Kakamega Samuel Eshitemi WASH Enterprise Development Specialist Kakamega Bernard Kipkurui WASH Governance Specialist Nyamira Mika Mitoko WASH Enterprise Specialist Nyamira Oscar Nzoka Sanitation and Hygiene Specialist Nyamira Stephanie Schwartkopf Project Management Nairobi / National Joshua Kibet WASH Finance Specialist Nairobi / National Anabel Gender Nairobi / National Eric Adams Team Leader for Component 2 Nairobi / National Erik Environmental Officer USAID Albert Waudo USAID - Monitoring and Evaluation Specialist Nairobi / National Martin Mulongo USAID – Contract Officer Representative (COR) KIWASH Nairobi / National Nicholas Owuor USAID – WASH Specialist Nairobi / National Development Partners – National Anthea Mannasseh AFD Nairobi / National Beverly Mademba AMREF Program Officer Nairobi / National Jacquelyne Waithaka Cooperative Bank Nairobi / National Eng. Fancis Kimanthi Kyengo Deputy Director, Water and Sanitation Development Programs, MoWS Nairobi / National J Orenge Family Bank Nairobi / National Steve Mcharo FHI 360 – Agriculture Specialist Nairobi / National CEO Robert Bunyi Kenya Pooled Water Fund Nairobi / National Matthew Okello Kisumu Practical Action Kisumu Jacob KIWASNET Kisumu Kisumu Lieke Nijk PEWAK Project Manager Nairobi / National Pauline Kiamba Rural Focus Limited (USAID SWS Program) Kitui Benjamin Cutner SNV – WASH Team Leader Nairobi / National ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 37 Respondent Name Title Location / Geographic Focus Eduardo Perez Global Communities Washington, DC / National Andrew Trevett UNICEF Nairobi / National Jimmy Kariuki UNICEF Regional Sanitation Officer Kitui Esther Njuguna WASH-FIN Nairobi / National Eng. Francis Maluki WASREB – Inspectorate Services Officer Nairobi / National Eng. Robert Gakubia Water Sector Regulatory Board (WASREB) - CEO Nairobi / National Ishmael Shaiye Water Services Trust Fund - CEO Nairobi / National Willis Ombai Water Services Trust Fund - Chief Manager, Investments and Programmes Nairobi / National Lewnida Sarah World Bank Nairobi / National Cliff Waithaka World Bank Nairobi / National Mugo Kariuki WSUP – National Nairobi / National Emmanuel Owako WSUP – Regional Manager Kisumu County Counterparts (Government and Civil Society) Mr. Joseck Maloba Chief Officer Water Kakamega Paul Manyasi County Public Health Officer Kakamega Paul & Kevin Likuyani Public Health Officers Kakamega Mr. Salmon Orimba CEC for Water Kisumu Kisumu Dr. Rosemary Obara CEC for Health Kisumu Kisumu Mr. Maurice Owino Chief Officer Water Kisumu Kisumu Ms. Anne Kombija Director of Water (Ag.) Kisumu Mr. Arthur Shikanda County Public Health Officer Kisumu Dr. Mary Obade County Nutritionist Kisumu Mr. Peter Sirima WASH Coordinator Kisumu Mr. Thomas Odongo KIWASCO – Managing Director Kisumu Eng. Moses Jura KIWASCO – Technical Manager Kisumu Kevin KIWASCO – Pro-Poor Unit Coordinator Kisumu Ms. Maureen Ondeng GULF – Team Leader Kisumu Mr. Benard Ondego NYANAS – Team Leader Kisumu Mr. Jacob Achollah KEWASNET – Coordinator Kisumu Mr. Okuta Ogacho Chairman – Boya Community Water Project Kisumu Emmanuel Kisangu CEC for Water Kitui Benjamin Kioko Kiilu Chief Officer Water Kitui Kennedy Mutati Director of Water Kitui Johnson Muinde County Public Health Officer Kitui Jackson Matheka County Nutritionist Kitui Augustus Nding’o WASH Coordinator Kitui Geoffrey Kitonga Water Service Provider – Managing Director (KITWASCO) Kitui Stephen Mutua Water Service Provider – Technical Manager (KITWASCO) Kitui Paul Kinuba Water Service Provider – Managing Director (KIMWASCO) Kitui David Nzuki Water Service Provider – Technical Manager (KIMWASCO) Kitui Eng. Joseph Kimanga Chief Officer Environment Kitui Margaret Muthiani WRA Sub-Regional Manager Kitui Kitui Eric Enda WASH Coordinator Nairobi Mario Kainga Director for Water, Nairobi County Nairobi Philip Oruoch Ag. Donor Project Manager, NWCS Nairobi Esther Muthoni Head of the Informal Settlement Region - Water and Sewerage, NWSC Nairobi Moses Kimani Technical Officer, Informal Settlement – Water section, NWSC Nairobi Samwel Maiko CEC for Water Nyamira ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 38 Respondent Name Title Location / Geographic Focus Lawrence Mokaya Director of Water Nyamira Thomas Nyangau County Public Health Officer Nyamira Geoffrey Deputy County Nutrition Coordinator Nyamira Mr. Munyi Agriculture Extension Director Nyamira Esther Nyambati WASH Coordinator Nyamira WASH Enterprises Stanley Omung’ala Khwisero North Community Water Supply Kakamega Samson Winja Ahero Catholic Water Project Kisumu Albert Nyawara Nyalenda Consumers Water Project Kisumu Okuta Ogacho Boya Community Water Project Kisumu Titus Kakuthu Kithambangii Water Project Kitui Rogers Mugambu Musengo Water Project Kitui John Mwendwa Mumbuni/Katalwa Water Project Kitui Angela Muma Machururiati/Karantini CBO Nyamira Wilson Nyaribo Ekerenyo/Emmo Nyamira ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 39 DOCUMENTS CONSULTED IN DESK REVIEW KIWASH Contractual Deliverables Activity Monitoring and Evaluation Plan (updated January 2017) Annual work plans for FY16, FY17 and FY18 Annual Reports: FY16, FY17, FY18 Quarterly Reports: FY16, FY17, FY18 (Q1, Q2 and Q3 each year) Grants Manual Revised Construction Plan KIWASH Programmatic Plans – Strategy Documents – Guides Gender Equality and Women’s Empowerment Plan Capacity Development Action Plan (for WASH enterprises) Training of Trainers Manual on Integrated WASH and Agri-Nutrition. Training Manual for Community Based Service Providers on Integrated WASH and Agri-Nutrition Sanitation Market Assessment Report CHV Discussion Guide WRUA Prioritization Process and Tool CHV Agri-Nutrition SBCC tools Training Materials for Sanitation Marketing County-specific KIWASH Reports Kisumu County Water Policy (Final Draft) KIMWASCO Strategic Plan KITWASCO Strategic Plan Capacity Development Action Plan for Kitui County Kitui West Sub-County Artisan Training Report GWASCO Strategic Plan Capacity Building Road Map for GWASCO Non-Revenue Water Performance Improvement Plan, GWASCO (draft) Nyamira County Water and Sanitation Bill (draft) Nyamira County Water and Sanitation Policy (draft) Social Behavior Change Communication Campaign Design, Nyamira County GOK policy and strategy documents Constitution of Kenya (2010) County Government Act (2012) Water Act 2016 Kenya Environmental Sanitation and Hygiene Policy (KESH) 2016-2030 Kenya Environmental Sanitation and Hygiene Strategic Framework (KESSF) 2016-2020 ODF Policy 2016-2020 Public Health Act Chapter 242 Vision 2030 WASREB Impact Report # 9 WASREB Impact Report # 10 Other documents Fundifix Leaflet and Booklet Exploring policy perceptions and responsibility of devolved decision-making for water service delivery in Kenya’s 47 county governments, Koehler, 2018. Kenya Pooled Water Fund CEO: The Water, Sanitation Sector is Ready for Green Bonds, Bonds & Loans, 22 June 2018 (accessed 22 November 2018) Sustaining WASH services through market- based approaches, experiences by USAID’s KIWASH project, Kubasu, et al., 2018. Water Policy Choices in Kenya’s 47 Counties, Policy Brief, 2016, Oxford University. Situational Analysis of Women and Children in Kenya, 2018, UNICEF ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 40 ANNEX V: WASH-NUTRITION INTEGRATION PROFILES FROM FIVE COUNTIES Kakamega Kisumu Kitui Nairobi Nyamira Co-Location • Targeting of same HHs is rather coincidental through CHVs during their household routine visits to provide WASH, nutrition, and other health services. • Targeting of same HHs is rather coincidental through CHVs during their household routine visits to provide WASH, nutrition, and other health services. • Unclear if HHs targeted by Agriculture extension workers are the same as those targeted by CHVs. • Targeting of same HHs is rather coincidental through CHVs during their household routine visits to provide WASH, nutrition, and other health services. • Unclear if HHs targeted by Agriculture extension workers are the same as those targeted by CHVs. • Targeting of same HHs is rather coincidental through CHVs during their household routine visits to provide WASH, nutrition and other health services. • CHVs and ag extension agents targeting of same HHs is rather coincidental through CHVs while providing WASH, nutrition, and other health services. • Unclear if HHs targeted by Agriculture extension workers are the same as those targeted by CHVs. Coordination • Lacking information on whether there was any joint planning between the WASH and nutrition departments before implementation of the program. • Information lacking on whether there was action planning after the TOT. • Information lacking on coordination forums. • Information lacking on whether there was any joint planning with the WASH and nutrition departments before implementation of the program. • Information lacking on whether there was action planning after the TOT. • Information lacking on inclusion of nutrition aspects in existing WASH forums. • No deliberate joint planning with government or other partners before implementation of the program at county level. Targeting same beneficiaries was avoided. • Joint action plans were developed during the TOT based on the sub-counties. • No information on coordination forum for WASH and nutrition integration. • Information lacking on whether any joint planning between WASH and nutrition departments before implementation of the program. • Information lacking on whether there was action planning after the TOT. • Consortium (Concern Worldwide, Oxfam, UNICEF, KIWASH, USAID Afya Jijini) organized coordination meetings to develop WASH and nutrition Implementation Guidelines. KIWASH was a member. • A nutrition TWG was initiated in Nairobi. • No evidence for joint planning of WASH, nutrition, and Agriculture ministries before implementation of WASH/nutrition Component. However, joint action plans were developed during the first TOT. • No evidence for joint planning with other USAID Health and nutrition programs at county level, (but reported joint planning with other USAID programs at national level). • Joint action plans were developed during the TOTs. • Coordination through the WASH forum—the ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 41 Kakamega Kisumu Kitui Nairobi Nyamira Agriculture Department participated in the forum before the nutrition component was discontinued. Cross Training • Training provided to government staff from Ministries of Health, Nutrition, and Agriculture as TOT. • Same training manual used for all counties. • Training covered WASH, nutrition, and agriculture (kitchen gardening). • WASH SBCC strategy includes nutrition messaging for all counties. • Training provided to government staff from Ministries of Health, Nutrition, and Agriculture as TOT. • Same training manual used for all counties. • Training covered WASH, nutrition, and agriculture (kitchen gardening). • WASH SBCC strategy includes nutrition messaging for all counties. • TOT of public health, nutrition, and agriculture departments completed. • Draft TOT and community￾based manuals developed. • Training covered WASH, nutrition, and agriculture (kitchen gardening). • WASH SBCC strategy includes nutrition messaging for all counties. • CHVs received training in nutrition, WASH, and agriculture after the TOT. • Training provided to government staff from Ministries of Health, Nutrition, and Agriculture. • Developed draft training manuals. • Same training manual used for all counties. • Training covered WASH, nutrition, and agriculture (kitchen gardening). • WASH SBCC strategy includes nutrition messaging for all counties. • Trained CHV promoted WASH and nutrition messages to HHs. • TOT conducted at county level with participants drawn from Agriculture, Nutrition and WASH. • Training covered WASH, nutrition, and agriculture (kitchen gardening). • Draft WASH and nutrition training manuals at final stages of development. • Cascaded integrated WASH and nutrition training for CHVs. • Training of farmers happened separately from WASH and nutrition (trained by the Agriculture department) and focused on agricultural activities. • WASH SBCC strategy includes nutrition messaging. Capacity strengthening on WASH and nutrition integration. Collaboration Joint M&E • CHVs collect data through community strategy. • CLTS tools do not capture agri-nutrition. • No integration of agri￾nutrition data in the WASH • CHVs collect data through community strategy. • CLTS tools do not capture agri-nutrition. • No integration of agri￾nutrition data in the WASH • There were two separate M&E tools for sanitation and agri-nutrition activities. • Data on nutrition collected by KIWASH MUS staff. • The CLTS tools do not • CHVs collect data using the community strategy tools. • No integration of agri￾nutrition data in the community WASH tools. • KIWASH had a tool that • M&E and reporting for WASH activities is through the CHVs and CLTS reporting tools. • No integration of agri￾nutrition data in the WASH ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 42 Kakamega Kisumu Kitui Nairobi Nyamira tools for CHVs through CHS. • KIWASH drafted monitoring form to capture both WASH and nutrition indicators through CHVs, which was only used briefly after program discontinued. • KIWASH staff conducted joint M&E. • No reported joint monitoring activities by the relevant government Ministries of Health and Agriculture. • Joint implementation in evidence during capacity building/sensitization of extension workers and community groups. tools for CHVs through CHS. • KIWASH drafted monitoring forms to capture WASH and nutrition indicators through the CHV for some time. • KIWASH staff conducted joint M&E. • No reported joint monitoring activities by the relevant government Ministries of Health and Agriculture. capture nutrition. • KIWASH drafted monitoring forms to capture WASH and nutrition through CHVs, which was only used briefly after program discontinued. • KIWASH staff conducted joint M&E visits. • No reported joint monitoring activities by the relevant government Ministries of Health and Agriculture. CHVs used to collect data on kitchen gardening. Only in use for the time the components were running. • KIWASH staff conducted joint M&E field visits. • No reported joint monitoring activities by the relevant government Ministries of Health and Agriculture. tools. • CHVs do not collect data for nutrition and Agriculture. • For internal project reporting, KIWASH used stand-alone reporting templates for agri￾production and nutrition monitoring; these have since been discontinued. • KIWASH staff conducted joint M&E. • No reported joint monitoring activities by the relevant government Ministries of Health and Agriculture. Joint Implementation: Community Level • Trained CHV promoted WASH and nutrition messages at community level. • The initial CHV SBCC job aid integrated nutrition messaging. nutrition messages have since been removed from the job aid. • Trained CHV promoted WASH and nutrition messages. • The initial CHV SBCC job aid integrated nutrition messaging. nutrition messages have since been removed from the job aid. • The CHVs developed mother-to-mother support groups. • Original cohort of CHVs do not work in the county any more, following a ban by the governor. New recruitment in progress. • Initial CHV SBCC job aid integrated nutrition messaging, which CHVs used to sensitize HHs. • KIWASH supports CHVs with a stipend. • CHVs trained HHs/mothers on WASH, kitchen gardening, and preparing good diets. • Facility Level • No evidence for updated NACs and ORT protocols and guidelines. • One ORT corner was established. • Distribution of hygiene kits to five health facilities. • No evidence for updated NACs and ORT protocols and guidelines. • The nutrition officer reportedly had trainings, posters, and training manuals developed by • No evidence showing NACs and ORT protocols/ guidelines updated to include WASH. • Supported eight HC facilities with ORT supplies. • Distributed hygiene kits • No evidence for updated NACs and ORT protocols and guidelines. • Two ORT corners were established. • Distribution of hygiene kits to five health facilities. • No evidence showing NACs and ORT protocols/ guidelines updated to include WASH. • Two health facilities were supplied with ORT equipment, including ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 43 Kakamega Kisumu Kitui Nairobi Nyamira • Information lacking on nutrition counseling during distribution. KIWASH and distributed during the Malezi Bora week, but the evaluation team could not physically verify this. • One ORT corner was established. • Distribution of 500 hygiene kits. Plans to distribute further kits to other sub counties as well • Information lacking on nutrition counseling during distribution. through four health facilities since project inception. • KIWASH worked with the health facility to identify the indicator they want to improve. For example, some want to improve ANC attendance, others focused on heath facility delivery. • Conducted health talks on nutrition and sanitation to mother-to-mother support groups that met at the HCF. • CHV coordinates the M2M groups and monitors distribution of hygiene kits at the HCF. • Information lacking on nutrition counseling during distribution of hygiene kits. hygiene items. • Distribution of hygiene kits was done through the ANC and PNC clinics. • Information lacking on nutrition counseling during distribution. • CHVs conduct promotion of WASH and nutrition messaging at the ANC. Complete Integration • No evidence. • No evidence. • No evidence. • No evidence. • No evidence. ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 44 ANNEX VI: DISCLOSURE OF ANY CONFLICTS OF INTEREST ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 45 ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 46 ANNEX VI: DISCLOSURE OF ANY CONFLICTS OF INTEREST Name EVELYN MAKENA MUGAMBI Title WASH AND NUTRITION SPECIALIST Organization Tetra Tech Evaluation Position? Team Leader Team member√ Evaluation Award Number (contract or other instrument) AID-OAA-TO-16-00016 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) KIWASH / DAI I have real or potential conflicts of interest to disclose. Yes No√ If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date February, 8, 2019 ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 47 ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 48 ANNEX VI: DISCLOSURE OF ANY CONFLICTS OF INTEREST Name Jonathan Edward Annis Title WASH Specialist Organization Tetra Tech Evaluation Position? Team Leader Team member Evaluation Award Number (contract or other instrument) Prepared by the Water, Sanitation, and Hygiene Partnerships and Learning for Sustainability (WASHPaLS) project under Task Order number AID-OAA-TO-16-00016 of the Water and Development Indefinite Delivery Quantity Contract (WADI), contract number AID-OAA￾I-14-00068. USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) Kenya Integrated Water, Sanitation and Hygiene Project (KIWASH) I have real or potential conflicts of interest to disclose. Yes No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date 2/9/19 ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 49 Annex VI: Disclosure of any Conflicts of Interest Name Charles Oyaya Title WASH Governance Specialist Organization Independent Evaluation Position? Team Leader Team member Evaluation Award Number (contract or other instrument) AID-OAA-TO-16-00016; AID-OAA-I-14-00068 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) Kenya Integrated Water, Sanitation and Hygiene Project (KIWASH) I have real or potential conflicts of interest to disclose. Yes No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. N/A I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date 11th February 2019 ANNEXES TO THE MID-TERM EVALUATION OF THE KIWASH PROJECT 50 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523 Tel: (202) 712-0000 Fax: (202) 216-3524 www.usaid.gov