PERFORMANCE EVALUATION OF THE USAID/ETHIOPIA TRANSFORM WATER, SANITATION, AND HYGIENE ACTIVITY Final Report Annexes JANUARY 2023 This report is made possible by the support of the American People through the United States Agency for International Development (USAID). The author’s views expressed in this report do not necessarily reflect the views of USAID or the United States Government. Preferred citation: USAID. (2023). Performance Evaluation of the USAID/Ethiopia Transform Water, Sanitation, and Hygiene (WASH) Activity. Washington, DC, USAID Water, Sanitation, and Hygiene Partnerships and Learning for Sustainability (WASHPaLS) #2 Activity. Cover photo credit: Retailer in Dodola Woreda, Oromia Region (Photo taken by Desta Dimtse) Prepared for the United States Agency for International Development by the Water, Sanitation, and Hygiene Partnerships and Learning for Sustainability (WASHPaLS) #2 project, contract number 7200AA21C00079. Tetra Tech Contacts: Morris Israel, Chief of Party morris.israel@tetratech.com Carolien van der Voorden, Deputy Chief of Party c.vandervoorden@tetratech.com Lucia Henry, Project Manager lucia.henry@tetratech.com Tetra Tech 1320 N. Courthouse Road, Suite 600, Arlington VA 22201 Tel: 703-387-2100 Fax: 703-414-5593 www.gllobalwaters.org/washpals-2; www.tetratech.com/intdev ANNEXES ANNEX A: USAID EVALUATION STATEMENT OF WORK (SOW) .......... 1 ANNEX B: EVALUATION METHODOLOGY................................................. 10 ANNEX C: DATA COLLECTION INSTRUMENTS........................................ 17 ANNEX D: HOUSEHOLD SURVEY FINDINGS ............................................. 73 ANNEX E: LIST OF KEY INFORMANT INTERVIEWS AND FOCUS GROUP DISCUSSIONS ................................................................................ 119 ANNEX F: FULL LIST OF TRANSFORM WASH WOREDAS AND SELECTION OF WOREDAS FOR PE TEAM VISIT ................................. 128 PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | i ANNEX A: USAID EVALUATION STATEMENT OF WORK (SOW) A-1.0 INTRODUCTION A-1.1 BACKGROUND OF USAID/TRANSFORM: WASH ACTIVITY Activity Description Program/Project Title United States Agency for International Development (USAID) Transform: (Water, Sanitation, and Hygiene) WASH Prime Partner Population Services International (PSI) Activity Start Date January 2, 2017 Activity End Date December 31, 2022 Life of Activity Budget $27,537,2018 (ceiling increased) Name(s) of Subcontractors/Sub-awardees Stichting Nederlandse Vrijwilligers (SNV), International Water and Sanitation Centre (IRC), and Plan International USAID Transform: WASH is a six-year (one-year extension) activity awarded to PSI and consortium partners SNV, IRC, and Plan International (“the Consortium”) to test and develop market-based models that will increase demand for and supply of quality and affordable WASH products and services. Through the development and testing of four scalable and replicable business models, the first-of-its-kind activity is intended to support and increase the capacity of the Government of Ethiopia’s (GOE) One WASH National Program (OWNP) to market, promote, and improve utilization of improved specific WASH products and services. The Transform: WASH Activity operates across all nine regions where either Transform/Primary Health Care or Transform/Health in Developing Regions interventions are taking place and Dire Dawa City Administration. Transform: WASH Activity’s Goal and Intermediate Results The goal of the activity is to reduce preventable death and illness in Ethiopia due to diarrheal disease, particularly among under five children. To achieve this health goal, the Transform: WASH activity includes four intermediate results (IRs) that appear in diagram 1 below: IR1) increased WASH governance and management capacity at the sub￾national level; IR) increased demand for low-cost, quality WASH products and services, with a focus on sanitation; IR3) increased supply for low-cost quality WASH products and services, with a focus on sanitation; and IR4) increased knowledge base to bring WASH innovations to scale. While USAID/Ethiopia health office has worked for several decades supporting social marketing of key life￾saving health products such as contraceptives, oral rehydration salts, water treatment tablets, and various nutritional supplements and nutritious foods for women and children, the Transform: WASH Activity represents a first-of-its-kind pilot program to harness the power of marketing to build commercially viable suppliers of sanitation products.1 The Transform: WASH Activity contains a demand creation and social and behavior change communication program that seeks to catalyze the existing cadre of health extension workers (HEWs) to promote WASH products to health consumers. The activity has a robust component that focuses on PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 1 1 USAID Transform: WASH Activity Design Document, 2016 _NFO no. RFA-663-16-000007 Transform/Water, Sanitation, Hygiene (WASH). building the business and marketing skills of sanitation product suppliers and partners, including masons and community-level sales agents, and a component to promote various models for community financing for WASH businesses. The Transform: WASH Activity also is designed to test four business models ranging from the door-to-door simple upgrades of WASH products, basic and advanced manufacturing and construction, and plastic slab sales and installation via retailers. The Transform: WASH Activity is operating in nine regions of Ethiopia and one City Administration, targeting 41 woredas selected from the targeted intervention regions that include Amhara, Oromia, Southern Nations, Nationalities, and Peoples’ Region (SNNP), Sidama, Southwest Ethiopia Peoples’ Region (SWEPR), Tigray, Afar, Benishangul Gumuz, Gambella, Somali, and Dire Dawa. TABLE A-1: NUMBER OF WASH WOREDAS BY REGION AND PHASE OF EXPANSION REGION PHASE 1: JAN 2017 PHASE 2: OCT 2018 PHASE 3: OCT 2019 TOTAL SNNP including Sidama and SWEPR 9 9 Oromia 6 7 13 Amhara 4 5 9 Somali 3 3 Gambella 1 1 Afar 1 1 B/Gumuz 1 1 Dire-Dawa 1 1 Tigray 3 3 Total 9 11 21 41 Diagram 1: Transform: WASH Development Hypothesis Results Framework A-1.2 PURPOSE AND USE OF THE FINAL PERFORMANCE EVALUATION OF USAID TRANSFORM: WASH ACTIVITY USAID seeks to get quality evidence on the major achievements, best lessons and from the Transform: WASH activity interventions from this final performance evaluation. The overarching purpose of this evaluation is to examine the validity of the overall Transform development hypothesis(es); effectiveness and sustainability of approaches and generate quality data for the decision-making process regarding future USG investment and programming. The specific objectives of the final performance evaluation of USAID Transform: WASH are: a. Understand the effectiveness of USAID/Ethiopia’s Transform: WASH approaches and strategies to stated objectives and results; Assess factors influencing achievement or non-achievements of intended results; b. Estimate the broader outcomes of USAID/Ethiopia’s Transform: WASH including the overall change in sanitation markets, changes in sanitation coverage, viability of supported enterprises, ability of the market to meet the needs of the underserved, and the likely sustainability of these changes; and c. Identify and document key learning and recommendations to inform future programming. PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 2 A-1.3 AUDIENCE AND INTENDED USES The primary intended audience for the Final Evaluation report is USAID/Ethiopia, who will use the evaluation findings and recommendations to review USAID/Ethiopia’s investment and future programming in the WASH sector. GOE/Ministry of Water and Electricity (MoWE), OWNP Coordination Office, and the Ministry of Health (MOH) (as applicable), will use the findings for planning and monitoring WASH activities and future design. A-1.4 EVALUATION QUESTIONS The final performance evaluation results will address the suggested key evaluation questions outlined below. The Contractor can propose modifications of these questions and respective indicators in consultation with USAID Health and Economic Growth and Trade offices. The Contractor will populate the matrix below with the relevant information and submit it to USAID for review. 1. How effective were the Transform: WASH Market-Based Sanitation (MBS) approaches and strategies for achieving intended results? What were the major factors influencing the achievement or non-achievement of the project objectives, outcomes, and outputs? How efficient were the interventions at achieving results compared to other approaches? 2. How effective have the Transform: WASH Activity’s approaches been contributing to improving WASH outcomes? a. To what extent has sanitation markets and target enterprise viability changed? What is the likelihood of those changes being sustained? b. To what extent has sanitation coverage changed? What is the likelihood of those changes being sustained? 3. To what extent Transform: WASH implementation is coordinated/integrated/aligned with other initiatives/programs and facilitate local Ownership and Sustainability? a. To what extent OWNP-supported Transform: WASH activities aligned with ongoing programs/initiatives? b. To what extent was it aligned with other USAID initiatives and programs? 4. How did Transform: WASH Activity consider gender and social inclusion in activity design and implementation? Were there any missed opportunities? A-2.0 EVALUATION DESIGN AND METHODOLOGY A-2.1 EVALUATION DESIGN The evaluation team will be responsible for developing an evaluation strategy and methodologies that include a mix of qualitative and quantitative data collection and analysis methods. The evaluation team should triangulate the data obtained from different primary and secondary sources to ensure quality. The team should review the evaluation questions matrix presented in this SOW and present a revised version with relevant additions showing the source of data, method of data collection and the tool to be used to answer each of the evaluation questions. The methodology will be presented as part of the draft work plan (inception report) as outlined in the deliverables below and included in the final report. USAID will review the proposed methodology’s strengths and weaknesses and provide feedback as necessary. PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 3 A-2.2 EVALUATION METHODOLOGY Data collection and sources The evaluation should utilize a mixed approach of qualitative and quantitative methods. A qualitative approach should include, but not limited to literature review, key informant interviews, semi-structured interviews, focus group discussions with beneficiaries, and direct observation. The respondents should include the MOH Hygiene and Environmental Health Directorate, USAID Health and EG&T offices, Regional Water Bureaus/WASH Coordinators, regional OWNP coordination office, Transform: WASH staff (Central Office and regional team), woreda WASH offices, private sector enterprises, and sales agents. The quantitative data will largely come from household surveys (HHSs), implementing partner reports, and WASH reports from different sources. The Contractor’s proposal should include detailed sampling procedures for the household data collection in the inception and final reports. The final methods will be finalized in consultation with USAID. Transform: WASH activity baseline and mid-term evaluations were conducted, and the results of this final evaluation are expected to be compared to the baseline. Hence, as far as possible, the final evaluation methodology should be similar to the baseline so that comparison of results will not be impacted by the methodological differences. Data Analysis While developing the work plan, the Contractor is expected to provide a detailed data collection and data analysis plan; for example, what quantitative and qualitative analysis techniques will be used, how focus group responses will be documented and analyzed, etc. As this evaluation also collects qualitative data, the Contractor must employ an appropriate analysis tool for qualitative data in order to categorize, rank and rate the responses of the interviewees and discussants. Very insightful or special descriptions of interviewees and discussants should be quoted as appropriate to highlight findings. All data collected and presented in the evaluation report must be disaggregated, as appropriate, by sex and geographic areas. The household data analysis will mainly focus on comparing final performance to baseline data. Method of data analysis for the quantitative data should be explicitly described including data quality assurance procedures that will be employed. The following table provides a summary of evaluation design and methodology that should be populated by appropriate information and presented to USAID for review. TABLE A-2: SUMMARY OF EVALUATION DESIGN AND METHODOLOGY QUESTIONS SUGGESTED DATA SUGGESTED DATA ANALYSIS INDICATOR (IF SOURCES DATA METHODS ANY) COLLECTION METHODS 1. [Evaluation Identify data sources. For [TBD by evaluation [TBD by evaluation Include the indicator, if question] instance, Project team] team] any, the evaluation documents (incl. PMP, question will address previous evaluations, etc.), national statistics, project staff, stakeholders, expert knowledge, beneficiaries…] PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY –REPORT ANNEXES |4 A-3.0 DELIVERABLES AND REPORTING REQUIREMENTS TABLE A-3: DELIVERABLES AND REPORTING REQUIREMENTS # DELIVERABLES DESCRIPTION TIMELINE RESPONSIBLE BODY 1 In-Briefing: An in-brief meeting will be held with USAID for introductions, presentation of the team’s understanding of the assignment and initial assumptions. Within 48 hours of the availability of the evaluation team Contractor 2 Evaluation Inception Report The Contractor shall submit an evaluation inception report to USAID. This inception report will include: (a) the overall evaluation design, including the proposed methodology, data collection and analysis plan, and data collection instruments; (b) a list of the team members and their primary contact details while in-country, including the email address and mobile phone number for the Team Leader; (c) the team’s proposed schedule for the evaluation; and (d) the estimated cost for the evaluation. USAID and relevant stakeholders will have five working days to review and consolidate comments through Contracting Officer Representative (COR)for this evaluation. Once the Contractor receives the consolidated comments on the inception report, they are expected to return with a revised inception report within two working days. The revised work plan shall include the list of potential interviewees and sites to be visited. USAID will send final comments on the Contractor’s re-submitted documents within two working days. Upon receipt of USAID approval, the Contractor can proceed with the activity. Revise work plan-2 days (Contractor) Review revised workplan- 2 days (USAID+ Stakeholders) Contractor/ USAID/ Implementing partner (IP)/GOE 3 Fieldwork Debrief The Contractor will provide the Mission activity manager for this evaluation with biweekly written briefings and feedback on the progress of the evaluation. If desired or necessary, weekly briefings by phone can be arranged with USAID to provide updates on field progress and any problems encountered. Immediately after the team’s completion of the fieldwork, the team shall provide a debrief to USAID about field level data collection experiences and any preliminary evaluation findings. Biweekly Contractor 4 Final Exit Presentation (PowerPoint Presentation) The Contractor makes 2 final presentations. One as a final exit to USAID and one to USAID, the IP, and relevant partners. The presentation should include a summary of key evaluation findings and conclusions relative to the evaluation questions. The timing and format (in-person and/or virtual) for both presentations will be scheduled as agreed upon during the in￾briefing. A copy of the PowerPoint file will be provided to the USAID at least three working days before the final exit presentation day. The COR shall compile comments from USAID and submit it to the Contractor for revisions to the final exit presentation and the final report. Within one month following completion of data collection Contractor 5 Draft Evaluation Report The Contractor will submit the draft evaluation to USAID within 10 working days after the exit presentation and should incorporate comments made during the exit presentation. USAID and other partners will have 15 working days to review and comment on the draft report and the COR shall submit consolidated comments to the Contractor. The Contractor will then have 10 working days to make appropriate edits and revisions to the draft and re-submit the revised final draft report to USAID. USAID and other partners will have 10 working days after the submission of the second revised Submission￾within 10 working days after the exit presentation First Review￾15 working days Contractor/USAID/ Stakeholders PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY –REPORT ANNEXES |5 6 7 TABLE A-3: DELIVERABLES AND REPORTING REQUIREMENTS # DELIVERABLES DESCRIPTION TIMELINE RESPONSIBLE BODY draft to again review and send any final comments. The content of the draft evaluation report is outlined in Annex A below, and all formatting shall be consistent with the USAID branding guidelines. The focus of the report is to answer the evaluation questions and may include factors the team considers having a bearing on the objectives of the evaluation. Any such factors can be included in the report only after consultation with USAID Second review- 10 working days Final Evaluation Report The Contractor should submit the final report to USAID Within 10 days Contractor within 10 days of receipt of comments. The report will incorporate final comments provided by the USAID/Ethiopia and other stakeholders as appropriate. The length of the final evaluation report should not be more than 45 pages, not including Annexes and Executive Summary. The Final Evaluation Report submission should also include a two-pager briefer on key qualitative and quantitative findings and conclusions relative to the evaluation questions. Submit data and records All project data and records will be submitted in full and shall Following be in electronic form in easily readable format; organized and approval of fully documented for use by those not fully familiar with the Final report project or evaluation; and owned by USAID and made available to the public, barring rare exceptions, on the USAID Development Experience Clearinghouse (http://dec.usaid.gov). A-4.0 EVALUATION TEAM COMPOSITION, ROLES, AND RESPONSIBILITIES OF MEMBERS USAID evaluation guidance for team selection strongly recommends that at least one team member have credentials and experience in evaluation design and methods. It is also highly recommended to utilize a local capacity as much as possible including not just local experts but learning organizations as appropriate. The evaluation team shall consist of a Team Leader and other experts defined by the contractor. Note that it is assumed to be required that the team includes proficiency in English and Amharic. The Contractor is also responsible for ensuring quality team composition and appropriate diversity. Women in the leadership position/management roles within a core evaluation team are welcomed. USAID may propose participation from USAID to accompany the team during site visits and/or to participate in key evaluation activities (to be finalized in discussions with the contractor and the Team Leader). A statement of potential bias or conflict of interest (or lack thereof) is required of each team member. Team Leader (One): The Team Leader should be experienced in WASH evaluation in Ethiopia and/or other African countries. Experience in market-based sanitation is preferred. The Team Leader must have strong team management skills and sufficient experience with evaluation standards, practices, and tools to ensure a credible product. The Team Leader must also be fluent in English and have strong writing skills. The Team Leader, in consultation with other team members, will be responsible for team coordination and performance and for ensuring the timeliness and quality of deliverables. The Team Leader is also responsible to ensure that the evaluation report meets the USAID Evaluation Standards (see Annex A.) They will also lead the design and implementation of data quality management and dissemination of the evaluation’s findings and recommendations to USAID/Ethiopia, Transform: WASH, MOH, and MoWE. PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 6 The Contractor may also hire other technical and administrative personnel (e.g., data collector and/or interviewer) as required and appropriate with defined responsibilities. A-5.0 EVALUATION SCHEDULE AND BUDGET The estimated time period for undertaking this evaluation is 180 working days including time for review of products. The evaluation and all deliverables are to be finalized and completed by September 30th. Please note that July–August is expected to be the rainy season with very limited mobility for field work and the contractor should likely target completion of field work prior to this period. The budget is expected to be between $300,000–500,000. The team is required to travel to selected zones/woredas/Kebeles in each region where program activities are being implemented. The evaluation team will prepare an exit briefing and presentation of the findings, which it will deliver to USAID and to other stakeholders as appropriate before expat team members leave Ethiopia. Following the briefing, the consultants will put together the draft report. USAID and other stakeholders will have 2 weeks to provide comments on the report for incorporation into the final report draft. Transform: WASH Evaluation must be completed by September 30, 2022. A-6.0 MANAGEMENT The Contractor will identify and hire the evaluation team, pending the Contracting Officer’s Representative (COR) and relevant technical office’s concurrence and CO approval for the rate, assist in facilitating the work plan, and arrange meetings with key stakeholders identified prior to the initiation of the fieldwork. The evaluation team will organize other meetings as identified during the course of the evaluation, in consultation with USAID/Ethiopia. The Contractor is responsible for all logistical support required for the evaluation team, including arranging accommodation, security, office space, computers, Internet access, printing, communication, and transportation. The evaluation team will officially report to the USAID COR, who will liaise with and coordinate communication with USAID technical offices. A-7.0 LOGISTICS The contractor will be responsible for all travel and logistics associated with conducting the evaluation. A-8.0 USAID EVALUATION REPORT STANDARDS 1. Identify the evaluation as either an impact or performance evaluation per the definitions in ADS 201. 2. Include an abstract of not more than 250 words briefly describing what was evaluated, evaluation questions, methods, and key findings or conclusions. The abstract should appear on its own page immediately after the evaluation report cover. 3. Include an Executive Summary 2–5 pages in length that summarizes key points (purpose and background, evaluation questions, methods, findings, and conclusions). 4. State the purpose of, audience for, and anticipated use(s) of the evaluation. 5. Describe the specific strategy, project, activity, or intervention to be evaluated including (if available) award numbers, award dates, funding levels, and implementing partners. PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 7 6. Provide brief background information. This should include country and/or sector context; specific problem or opportunity the intervention addresses; and the development hypothesis, theory of change, or simply how the intervention addresses the problem. 7. Identify a small number of evaluation questions. 8. In an impact evaluation, identify questions about measuring the change in specific outcomes attributable to a specific USAID intervention. 9. Describe the evaluation method(s) for data collection and analysis. 10. Describe limitations of the evaluation methodology. 11. In an impact evaluation, use specific experimental or quasi-experimental methods to answer impact evaluation questions. 12. Include evaluation findings and conclusions. 13. If recommendations are included, separate them from findings and conclusions. 14. Address all evaluation questions in the SOW or document approval by USAID for not addressing an evaluation question. 15. Include the following annexes: a. Evaluation SOW. If the SOW is revised, the evaluation report should include the updated SOW as an Annex rather than the original SOW. b. A description of evaluation methods (if not described in full in the main body of the evaluation report). c. All data collection and analysis tools used, such as questionnaires, checklists, survey instruments, and discussion guides. d. All sources of information—properly identified and listed. e. Any “statements of differences” regarding significant unresolved differences of opinion by funders, implementers, and/or members of the evaluation team. f. Signed disclosures of conflicts of interest from evaluation team members. g. Abridged bios of the evaluation team members, including qualifications, experience, and role on the team. 16. Include enough information on the cover of the evaluation report so that a reader can immediately understand that it is an evaluation and what was evaluated. The evaluation cover should: a. Include a title block in USAID light blue background color. b. Include the word “Evaluation” at the top of the title block and center the report title underneath that. The title should also include the word “evaluation.” c. Include the following statement across the bottom of the cover page: “This publication was produced at the request of the United States Agency for International Development. It was prepared independently by [list authors and organizations involved in the preparation of the report].” For an internal evaluation team, use the following statement: “This publication was PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 8 produced at the request of [USAID/Mission] and prepared by an internal evaluation team composed of [list authors and affiliation].” d. Feature one high-quality photograph representative of the project being evaluated and include a brief caption on the inside front cover describing the image with photographer credit. e. State the month and year of the report. f. State the individual authors of the report and identify the evaluation Team Leader. A-9.0 USAID CRITERIA FOR QUALITY EVALUATION 1. Evaluation reports should represent a thoughtful, well-researched, and well-organized effort to objectively evaluate the strategy, project, or activity. 2. Evaluation reports should be readily understood and should identify key points clearly, distinctly, and succinctly. 3. The Executive Summary should present a concise and accurate statement of the most critical elements of the report. 4. Evaluation reports must address all evaluation questions included in the SOW, or the evaluation questions subsequently revised and documented in consultation and agreement with USAID. 5. Evaluation methodology must be explained in detail and sources of information properly identified. 6. Limitations to the evaluation must be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.). 7. Evaluation findings should be presented as analyzed facts, evidence, and data and not based on anecdotes, hearsay, or simply the compilation of people’s opinions. 8. Findings and conclusions should be specific, concise, and supported by strong quantitative or qualitative evidence. 9. If evaluation findings address person-level outcomes and impact, they should be assessed for both males and females. 10. If recommendations are included, they should be supported by a specific set of findings and should be action oriented, practical, and specific. PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 9 ANNEX B: EVALUATION METHODOLOGY This section describes the overall evaluation methodology applied to answer the EQs. Primary and secondary data collection methods are presented in Sections B-1 and B-2, respectively; and the team’s approach to data analysis is provided in Section B-3. The evaluation team employed multiple data collection methods including: (a) a desk review of performance reports and other project documents and data combined with a review of wider academic, grey and government literature; (b) Key Informant Interviews (KIIs); (c) Focus Group Discussions (FGDs); (d) site visits; and (e) a household survey (HH). Wider literature was used to provide additional context on the WASH operating environment in Ethiopia as well as information on other related or complimentary programs in Transform: WASH woredas. The evaluation team used methodological triangulation (i.e., consulting data from multiple sources, such as interview data, Transform: WASH reports, government documents, etc.), investigator triangulation (i.e., involving teams of two data collectors per interview and having multiple team members involved in data analysis and identification of emerging findings), and data triangulation (i.e., obtaining the perspectives of multiple similar types of respondents across woredas in which Transform: WASH operates). Key informants for the evaluation included staff from Transform: WASH and USAID/Ethiopia as well as Transform: WASH stakeholders and beneficiaries and individuals not involved or connected to the Activity to explore the context in which Transform: WASH is operating. In terms of document review, the evaluation team referred to Transform: WASH quarterly and annual progress reports, dashboard data and other written information from the Activity, GOE documentation (from FMoH and OWNP in particular), documentation from MSP meetings including agendas and minutes, woreda strategies and related data sources B-1.0 PRIMARY DATA COLLECTION As noted, the evaluation team used four primary data collection approaches: KIIs, FGDs, HHSs, and site visits where appropriate. Each is discussed in the following sections. B-1.1 KEY INFORMANT INTERVIEWS Selection of Interviewees. The evaluation team conducted over 150 KIIs with a wide range of stakeholders to ensure that sufficiently broad perspectives were captured to answer the EQs and provide sound, actionable recommendations. The evaluation team solicited PSI and the consortium partners for initial suggestions on whom to interview and then, being well connected in the sector, also cast the net wider by soliciting inputs from team members’ own WASH networks. Thus, the evaluation team conducted interviews with key informants from USAID/Ethiopia, GOE both at national and regional/woreda levels, Transform: WASH staff, government partners (for WASH, healthcare, private sector engagement/entrepreneur development, GEFE, amongst other areas), and private sector enterprises. Some interviewees, like the Transform: WASH Chief of Party (COP) and Deputy Chief of Party (DCOP), were interviewed more than once during different stages of the data collection to validate emerging or shifting lines of inquiry. Specific to triangulating data from KIIs, the team requested key informants to provide evidence for statements or suggest where evidence may be available, or to denote whether their response is based on perceptions where evidence may not be directly forthcoming. Where there were evidence gaps or the evidence was contradictory, the Team reverted initially to Transform: WASH partners to provide evidence that confirms or counters the claims made. PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 10 Key Informant Interview Guides. KII guides were provided in the inception report and structured to cover key themes of relevance to answer the EQs, as appropriate to the respondent, such as: the relevance, effectiveness, and likely sustainability of the Transform: WASH approach; capacity development efforts; key lessons learned; nature of interaction with relevant stakeholders; GEFE-related approaches and results; amongst other aspects. The KII guides combined questions on the different EQs and were prepared for each respondent type. While the KII guides varied based on respondent categories and the EQs, all included some variation of the following: a) Basic demographic and context data, including interviewee’s sex, age, location (as well as some indication of “believability” in case there is reason to believe that the respondent may not respond truthfully) b) Respondent’s relationship to Transform: WASH – as a staff member, policymaker, government partner, beneficiary, stakeholder, as applicable c) Project background based on respondent experience d) Perception of successes and failures, challenges, and opportunities e) Relationship with other activities, government, or stakeholders and other factors that could affect responses f) Promising practices that should be retained and why, as applicable Most KIIs were conducted by two evaluation team members. B-1.2 FOCUS GROUP DISCUSSIONS The evaluation team conducted 46 FGDs with local beneficiaries, woreda WASH steering committees (WWSCs) and woreda WASH technical teams (WWTTs), private sector entrepreneurs, and Transform: WASH staff. The evaluation team prepared a preliminary FGD guide that followed the framing of the key questions in the KII guides. At the community level, the evaluation team conducted FGDs in each of the visited Transform: WASH woredas focusing specifically on FGDs with representatives of households who have been mobilized and adopted improved sanitation, and separate FGDs with representatives of households who had been mobilized but did not adopt improved sanitation. Discussions were designed with the latter group to understand their reluctance or inability to invest. Transform: WASH staff supported in organizing FGDs, but FGDs were run independently of Transform: WASH staff and input. B-1.3 SITE VISITS In two sub-teams, the evaluation team visited seven of the nine Transform: WASH regions, including the city administration of Dire Dawa. Scheduled over seven weeks, field visits were used to gather inputs directly from Transform: WASH activity participants and beneficiaries, seeking to triangulate and complement information generated through national level KIIs, the desk review, and the HHs. Final selection of woredas visited were made in consultation with USAID/Ethiopia and the IP and with consideration for logistical issues, security concerns, and other factors. A Community Observation Guide was developed to frame rapid observations undertaken at the community level to gauge general accessibility, the availability of kiosks and local retail outlets, the general environment of the community, and related aspects. Observations were then used in the framing of informal discussions to probe understanding and perceptions of community members to triangulate data with other data collection sources. Efforts were made to balance information collected from older and younger members of the community as well as women and men. PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 11 B-1.4 SELECTION OF WOREDAS FOR PRIMARY DATA COLLECTION The evaluation team visited 16 woredas in seven regions. This includes 12 Transform: WASH woredas and four control woredas as per the table below. The selected woredas were also the target of HHSs by the subcontracted quantitative data collection team (PRIN International) as well as KIIs, FGDs, and community observations by the evaluation team. Efforts were made to ensure contact between the evaluation team and the HHS teams to monitor the HHS team’s progress, capture initial impressions by the evaluation team of emerging findings in the HHS team, and an opportunity to check whether the HHS needed any adjustments. Intervention and Control Woredas Selected for Primary Data Collection (Control woredas noted in gray) Selection of Intervention Woredas Given the sample size of 31 woredas deemed to be safe for evaluation team visits at the start of the assignment, this selection more than adequately provided the evaluation team with sufficient opportunity to review the Transform: WASH activities in-depth across a range of criteria noted below. The evaluation team considered various assumptions resulting in the inclusion of the 12 intervention woredas including balancing the intervention phases, accessibility for the evaluation team, livelihood differences, remoteness and market connectivity, presence of other WASH development partners, and overarching woreda performance with regard to WASH. (See Annex F for more detail.) Selection of the Control Woredas The evaluation team’s primary objectives in conducting a performance assessment in the control woredas was to understand horizontal diffusion of the project model to adjacent (neighboring) woredas and explore differences in performance of MBS, sanitation demand and enabling environments. Logistical considerations have been key in determining the four control woredas. Therefore, four control woredas (two adjacent and two non-adjacent woredas) were selected based on the following assumptions: 1. Woredas located adjacent to the Transform: WASH intervention woredas but not receiving direct Transform: WASH project support. This provided the evaluation team with the opportunity to explore any spillover effects of the Transform: WASH activity – looking at PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 12 whether markets are indeed developing in response to consumer demand and whether woreda officials are aware of the phenomenon and, if so, how they are encouraging a market response. 2. Woredas not adjacent to the Transform: WASH intervention zone and not receiving direct or indirect Transform: WASH project support or similar support from other development partners. These two woredas were chosen to understand whether there is a natural progression towards MBS through a general increase in awareness of the need for and benefits of improved sanitation and general shifts in the functionality of WASH governance and institutional arrangements related to sanitation provision. The evaluation team recognizes that some care has been needed in drawing conclusions from the four control woredas vis-à-vis the analysis of the Transform: WASH woredas. B-1.5 HOUSEHOLD SURVEYS A HHS was conducted to understand levels of (sustained) access to and use of sanitation services, exposure to program activities, and practice of key hygiene behaviors linked to the Transform: WASH outcomes. The overall objective of the HHS was to explore whether the MBS approach taken by the program has resulted in increased uptake of improved latrines and changes in household behaviors. Sample size and design took into account statistical significance and comparability of the findings, time and budget limitations, and guidance from USAID/Ethiopia and the IPs on accessibility of program woredas for data collection. Survey questions focused on program outcomes and were informed by baseline and mid-line surveys conducted as well as the need to collect other, program-relevant information. PRIN International was subcontracted to carry out the HHS data collection and support the analysis. 3076 HHSs were conducted, providing sufficient data for adequate comparison between control (768) and intervention woredas (2308) and to establish an understanding of the influence of Transform: WASH on outcomes. B-2.0 SECONDARY DATA COLLECTION The evaluation team conducted a desk review of available key project documents and undertake content analysis of all available data relevant to the evaluation from both Transform: WASH and outside sources. This document review permitted the evaluation team to better understand where existing gaps in information could be filled through the fieldwork and provided a key resource in the contextualization of evaluation data for answering the EQs. B-2.1 ANALYSIS OF HOUSEHOLD SURVEY DATA As noted above, local research firm PRIN International was subcontracted to conduct and analyze the HHS, including finalizing sample design, recruitment, training and coordination of enumerators, data collection, data cleaning, and analysis using appropriate software. Overall, the aim of the HHS data analysis was to estimate outcomes related to the goals of Transform: WASH; compare these outcomes between intervention and control woredas; determine differences in outcomes between regions, woredas, and Kebeles; and determine any factors influencing the outcomes and their achievement or lack thereof. From the data collected through the HHS, key outcomes were estimated and assessed in Transform: WASH intervention woredas and control woredas, e.g., sanitation market access and overall sanitation coverage metrics. These metrics were compared with those in the control woredas to determine if differences are statistically significant. This was done using Difference in Differences, a respected method for examining differences in an intervention’s effect between groups.. PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 13 Outcome metrics were also compared to determine both within-woreda and between-woreda effects, as Transform: WASH interventions have seen varying levels of success from one region, woreda, and Kebele to the next. To the extent possible, differences will be identified between outcomes in intervention woredas and their bordering control woredas and control woredas that do not border the intervention woredas, to capture possible spillover effects. Causal factors influencing the outcomes were identified, including not only anticipated variables such as the woreda or Kebele or socio-economic attributes of the households, but also emergent factors from the KIIs and focus group discussions cited to influence Transform: WASH’s interventions and performance. The most ideal form of statistical regression analysis was then used based on the data obtained. For example, logistic regression was used for binary outcome values, and linear regression was used for continuously measured outcome values. Statistical analysis was conducted using Statistical Product and Service Solution. All data was cleaned and screened for outliers prior to analysis by examining their leverage (relative influence on any relationships uncovered) compared to other data points. Missing data points were confirmed to be missing at random to ensure unbiased results. Continuous variables were mean-centered prior to analysis to obtain more interpretable results. Multicollinearity was examined between factor variables included in the analysis. Statistical significance was determined based on comparison with the standard p-value threshold of 0.05. In addition to quantitative data collected and analyzed in the HHS, open-ended questions in the HHS were asked to provide additional detail and nuance for interpreting and triangulating the results. B-3.0 DATA ANALYSIS METHODS B-3.1 QUALITATIVE CODING AND CONTENT ANALYSIS To expedite the coding and analysis of qualitative information, the evaluation team developed, a priori, a coding tree and set of three to five key codes for each of the EQs, as well as a code to highlight key quotations from the qualitative data. The team used the Dedoose software platform for data analysis. Dedoose is a secure, cloud-based platform that facilitates the sharing of data across the evaluation team and allows the Team Leader easy access to spot-check coding as it is completed. (Various online media such as Skype, Zoom, and others have a record function that was used. Interviewees were informed of and asked for consent prior to recording the conversation.) Using the Dedoose platform allowed for text to be grouped by domains (codes) and respondent type (quotations). This supported efforts to arrange the data into the final evaluation report and was helpful in identifying clear patterns and recommendations. For the sake of consistency, the Team Leader and one other designated team member from the WASHPaLS #2 home office coded the interview transcripts. Dedoose calculates simple descriptive statistics that allowed the team to identify and analyze trends in codes based on KII or FGD metadata. For example, if data from KII with men tend to be coded one way, and KII with women tend to be coded differently, this was readily apparent and used by the team in analysis and development of recommendations. The evaluation team examined qualitative data from KIIs and FGDs to identify patterns in the data to highlight convergence or divergence of opinion across districts, activity types, and stakeholder groups. Where divergence in responses wase found, the team explored possible reasons for divergence in fact, perception, and opinion both internally and through follow-up interviews, where necessary. Secondary data from the desk review and project monitoring data was used to provide contextual background to help explain the results and to triangulate findings from the qualitative analysis. PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 14 Descriptors used to categorize respondents in Dedoose is as follows: • Data Collector/Interviewer Name • KII/FGD • Sex/Gender - Female - Male - Mixed Majority Male - Mixed Majority Female - Mixed Equal • Location (Area Focus) - National - Regional - Woreda • Primary Role - Overarching - MBS - GEFE - Demand Creation - Governance - M&E - Other Codes used to categorize respondents in Dedoose is as follows: GENERAL CODES USED • Useful quotes • Respondent type • Transform WASH Activities (used to double code alongside any stated changes brought about by said intervention, so that we can count total statements where respondents attribute a certain activity to a certain change) - Activities listed out as child codes here • Influences on achievement or non-achievement of goals (used to double-code alongside any referenced activities—see the code directly above—to identify the primary drivers of and barriers to success for each activity/intervention) - Emergent influences listed out as child codes here CODES USED FOR EQ1—EFFECTIVENESS OF TRANSFORM: WASH DEMAND CREATION APPROACHES • Changes in demand for improved latrines • Changes in community behavior (open defecation, for example) • Changes in sanitation coverage • Evidence of long-term sustainability CODES USED FOR EQ2—MARKET-BASED SANITATION RESULTS • Changes in availability of affordable and quality WASH products • Changes in equity of sanitation market access • Changes in demand for non-latrine sanitation and hygiene products • Changes in supply chain and long-term business viability (including customer satisfaction, etc.) PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 15 CODES USED FOR EQ3—CONTRIBUTION TO GOVERNANCE, ALIGNMENT WITH OTHER GOE, AND USAID INITIATIVES • Changes in government ownership and uptake • Changes in community ownership and uptake • Changes in private sector ownership and uptake • Changes in coordination among key sector actors • Remaining gaps in government ownership and uptake • Remaining gaps in community ownership and uptake • Remaining gaps in private sector ownership and uptake • Evidence of influence of learning activities CODES USED FOR EQ4—GEFE AND INCLUSIVE DEVELOPMENT • Changes in cultural norms or beliefs • Changes in gender roles and responsibilities • Changes in access to resources or control over assets • Changes in involvement and decision-making power • Changes in laws, policies, and planning PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 16 ANNEX C: DATA COLLECTION INSTRUMENTS Annex C-1: Key Informant Interview (KII) Guides Annex C-2: Household Survey (HHS) Data Collection Tool Annex C-3: Focus Group Discussion (FGD) Guides PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 17 C-1.0 KEY INFORMANT INTERVIEW (KII) DATA COLLECTION INSTRUMENTS The examples below are illustrative of the nature and flow of the questions to be asked of different stakeholder groups. These will be modified, as needed, after testing them. The initial set of questions (for Transform: WASH Staff/Consortium Partners) are obviously the most comprehensive and can be used to delve deeper with different stakeholders as appropriate. C-1.1 KEY INFORMANT INTERVIEW TOPIC GUIDE—TRANSFORM: WASH STAFF/CONSORTIUM PARTNERS (NATIONAL AND REGIONAL/WOREDA LEVELS)—GENERAL QUESTIONS This Guide is intended for use in terms of data collection from Transform: WASH staff. This evaluation will assess the effectiveness of Transform: WASH’s technical assistance activities to increase viability and professionalism of WASH service providers and support the enabling environment for market-based approaches more generally in Ethiopia. The evaluation will also help inform USAIDs future decisions on WASH programming in Ethiopia. It should be followed as closely as possible to guide key informant interviews with these respondents. There are a few questions where a specific set of responses are provided to assist in the collection of data that can be more easily quantified. Instructions to the interviewer are in red. Interviewers will be trained to be compliant with USAID policy regarding the “United States Government (USG) Common Rule” for the protection of human subjects. Introduction (~10 minutes) 1. Thank the respondent for taking the time to participate in the interview. 2. Introduction to the researcher and the research below: a. Introduce yourself: I am a consultant residing in ______. I represent an evaluation team fielded by Tetra Tech ARD, a Washington DC-based firm that has been contracted by the US Agency for International Development (USAID) to conduct an independent evaluation of the USAID Transform: WASH program. b. As part of its own planning for the next few years, USAID has asked us to conduct a final performance evaluation of the Transform: WASH program to assess its progress toward achieving its objectives. So, what we are trying to assess are the strengths and weaknesses of the program, its accomplishments, and best practices, but also any obstacles and shortcomings faced and how it could be more effective. c. Ultimately the research will be used to ascertain the extent to which Transform: WASH activities have improved lives and health through the development and management of sustainable water, sanitation, and hygiene services in Ethiopia. It will also be used to propose recommendations based on the findings to inform future WASH programming by USAID. 3. We will follow privacy protocols to protect your anonymity: a. Explain confidentiality and anonymity and note whether the respondent would like to remain anonymous, and that the assessment team will ask permission if would like to attribute a quote directly from the respondent in the final report. b. Explain how collected data will be stored without identifying information. c. Ask if the respondent is willing to be recorded and note their response. d. Explain recording, length, and nature of discussion. e. Check whether respondents have any questions. PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 18 Transform: WASH’s approaches and wider contribution to the sector (~45 minutes) Transition: I would like to spend some time speaking with you about your knowledge of the USAID￾supported Transform: WASH program. 1. What notable milestones and achievements has the WASH sector realized in Ethiopia in recent years? 2. Does the Ethiopian policy environment explicitly endorse and foster market-based approaches for sanitation services? Is there clarity in the Ethiopian context about the different ways of fostering and the elements required to implement market-based approaches in the WASH sector? 3. For WASH service delivery, which Transform: WASH woredas do you think are making most significant progress? What were the major favorable factors influencing this achievement? 4. In your view, in what area of intervention has Transform: WASH made the greatest contribution and been particularly successful? What were the major favorable factors influencing this achievement? Probe for examples: a. Expanding sanitation coverage b. Ensuring sustainable access to sanitation services c. Building woreda capacity for monitoring service delivery, planning, investment, monitoring and regulatory functions, etc. (e.g., through the WWTs/WTTs) d. Financing models e. Service provider and enterprise support f. Influencing government policy g. Gender, female empowerment, and social inclusion mainstreaming (Women included female-headed households, men, girls, boys, people living with disabilities, elderly people) h. Other 5. In your view, in what area of intervention has Transform: WASH had the most challenge in terms of impact? What were the major hindering factors around this intervention area? 6. How has Transform: WASH helped to build woreda capacity as an enabler for service providers? Probe for evidence of effectiveness. 7. How has Transform: WASH helped to build capacity of small and medium enterprises that deliver sustainable sanitation products and services in the woredas in which you are working? Probe for sanitation and hygiene and evidence of effectiveness. 8. In terms of finance, how has Transform: WASH helped to ensure greater financial viability for service providers in the woredas in which you are working? Probe for evidence of effectiveness. 9. How has Transform: WASH helped to enable financing for service providers in the woredas in which you are working? 10. How specifically has Transform: WASH contributed to expanding sanitation services from a supply side (i.e., working with providers)? Probe for inclusion of more businesses and business approaches in sanitation, the introduction and uptake of new technologies, linking with financing institutions, etc. PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 19 11. How specifically has Transform: WASH contributed to expanding sanitation services in the woredas from a demand side (i.e., working with households and communities)? Probe for community-led total sanitation and the interaction between supply and demand approaches as well as sales campaigns, etc. Also, whether any sales/product promotion campaigns have been conducted directly with WASH business enterprises. 12. How has the Transform: WASH project contributed to gender equality? What is the likelihood of Transform: WASH achievements in this area being sustained? Probe for: a. At the project level b. Influencing stakeholders at the region/woreda and national levels c. At the community level (project beneficiaries) 13. How has the Transform: WASH project contributed to social inclusion? What is the likelihood of Transform: WASH achievements in this area being sustained? Probe for: a. At the project level b. Influencing stakeholders at the region/woreda and national levels c. At the community level (project beneficiaries) 14. What internal factors do you think have enabled Transform: WASH to be successful in its efforts? (Factors within Transform: WASH control: staffing capacity, staffing skill set, budgeting, sequencing of activities, nature of collaboration between members of the implementing consortium, etc.). 15. How effective has the collaboration been between the consortium partners? Probe for joint implementation road map, joint performance review, availing of technical expertise, etc. 16. What external factors do you think have enabled Transform: WASH to be successful in its efforts? Probe for factors beyond Transform: WASH control: policy environment, support of other like-minded actors, donor relations, COVID-19, foreign exchange issues, conflict, etc. 17. Are there any internal factors (i.e., within Transform: WASH’s direct control) that you think are undermining or affecting efforts and performance? 18. Are there any external factors (i.e., beyond Transform: WASH’s direct control) that you think are undermining or affecting efforts and performance? 19. How specifically has Transform: WASH influenced government policy (specifically through OWNP) with regard to sanitation services? Probe for specific contributions. 20. How specifically has Transform: WASH supported increased government capacity and performance improvements with regard to the design and implementation of sanitation services through an MBS approach? 21. As relevant, how has Transform: WASH worked with other development partners at the national level and in different woredas around sanitation service delivery and MBS approaches? What challenges have been encountered, if any? Probe for duplication as well as complimentary and contradictory approaches to capacity building, finance and subsidy, construction, etc., and how Transform: WASH built on or addressed these. 22. What do you feel has been Transform: WASH’s most significant accomplishment to date? PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 20 23. Are there any missing elements in the Transform: WASH activity that you would recommend for improving effectiveness? Probe for specific recommendations and examples. 24. What recommendations, if any, do you have for USAID with regard to the design and delivery of future programs to strengthen the WASH sector to ensure and expand access to sustainable sanitation services? (Questions for Transform: WASH staff [PSI and consortium] focusing on specific areas) Market-Based Sanitation 1. How do you assess Transform: WASH’s activities and impact in terms of: (Explicitly for Transform: WASH marketing team [PSI]) a. Technology feasibility—What possible products and services have been introduced and sold significantly? Why do you think these have been in greater demand? b. Business viability—What is financially possible for local businesses? 2. What key achievements would you attribute to Transform: WASH implementation in terms of market enablers—such as changes to market rules, government policies, standards, and regulations? 3. Was there any trial/achievement in optimizing market interaction to overcome high transaction cost and to balance out supplier and buyer risks? If so, how? 4. Are businesses making enough profit to sustain their operations? Explore. And if not, why not? 5. Can business enterprises easily access final products for retail or molds if they want to produce based on consumer preference at an affordable price? 6. Are material suppliers making it easy and affordable for customers to buy and transport the right number of materials and labor they need to install the final product? 7. How have efforts for toilet upgrading compared to new consumer sales and installation? Explore levels of uptake in the two consumer groups, levels of emphasis in the project, etc. 8. Are there any aspects that are more or less conducive or more or less enabling for women entrepreneurs? Probe for access to finance, licensing, customer outreach, access to training, ability to hire, etc. 9. How will the Transform: WASH project approach (particularly around supply aspects) be sustained beyond the project phase out? a. At the national government level b. At the region/woreda level c. At the level of private sector businesses d. At the community level (with local partners) Demand Creation 1. What specific progress has been made against the Intermediate Results stated at the outcome and output level improving sanitation? 2. Do you think that sanitation coverage has changed since the project started its implementation? PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 21 3. Were planned sanitation activities and outputs achieved in a timely manner and at reasonable cost? 4. Who are the private sector actors working with Transform: WASH, any other actors in increasing demand for sanitation products and services? 5. Who are the key champions for sanitation demand creation? What role is the private sector playing in creating demand for sanitation products and services? Explore how the private sector role has changed, if at all, since the start of the project. 6. What capacity-strengthening support has the government at federal-, regional-, woreda-, and Kebele-level WASH sectors received from Transform: WASH to support OWNP activities? Has there been performance improvement as a result of the capacity building? If yes, can you give evidence and examples? 7. How will the Transform: WASH project approach (particularly around demand creation) be sustained beyond the project phaseout? a. At the national government level b. At the region/woreda level c. At the level of private sector businesses d. At the community level (with local partners) 8. Is there evidence of contribution and ownership to sustain Transform: WASH approach interventions at these different levels? Governance and Enabling Environment 1. What were the key achievements of the Transform: WASH project in supporting the GOE with MBS-related policy and guidelines? Probe for tax related policy, MBS implementation guidelines, sanitation key performance indicator definitions, subsidy policy, etc. 2. What approaches were used to work with/convince GOE of these approaches? (What might you have done differently in hindsight?) 3. What progress have you observed in terms of demand creation and supply-driven approaches as a result of these efforts? 4. How specifically has the OWNP been supported to solve the challenges of: a. Foreign currency for sanitation product importers? b. Land and other resources (like access to finance) for small and medium enterprises? 5. What capacity-strengthening support has Transform: WASH provided to the federal, regional and woreda levels? a. For OWNP specifically, how has Transform: WASH support increased the functionality of the cross-Ministry set-up as per the WIF? b. Does OWNP have effective capacity to influence WASH-related decision making? c. What evidence do you have that this is likely to be sustained? PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 22 6. How effective have the national and regional multi-stakeholder learning platforms been in supporting implementation of the national MBS implementation guidelines by bringing lessons learned and key evidence? (What specific support was provided? How has participation changed across the stakeholder groups?) Probe for evidence of likely impact and sustainability of the approach. 7. What changes have you observed at different levels due specifically to the Transform: WASH capacity support program? (Implementation capacity/institutional capacity) 8. In terms of Consolidated WASH strategic vision and plans, have all project woredas developed these and is MBS well addressed in them? 9. Do the woredas produce consolidated OWNP WASH reports that include additional reporting and indicators on MBS performance? 10. How have the Institutional Strengthening Indexes been introduced and how effective has this process been in introducing benchmarking, capacity needs assessments, and other mechanisms to strengthen functionality? What evidence is there that they are likely to continue to be used? As the Institutional Strengthening Index (ISI) is a self-assessment tool, what verifications mechanisms have been used to avoid bias? Probe for whether local governments see these tools as a helpful contribution. Gender Equality/Female Empowerment 1. How have gender equality and women’s empowerment aspects been considered in Transform: WASH? 2. How did Transform: WASH integrate gender equity/women’s empowerment in the activity design? 3. How did you ensure gender equality/women’s empowerment aspects are considered throughout implementation? (e.g., is there any planning checklist? How about gender equality and women’s empowerment responsive budgeting?) a. Which approaches have been most successful for gender inclusion in terms of advancing women led enterprises? b. What are the opportunities and challenges for gender equality with regard to success in approaches to MBS? to engage and succeed in MBS? 4. How are gender equality/women empowerment incorporated into the Transform: WASH monitoring and evaluation (M&E) framework? Are gender equality/women empowerment progress, achievements, and planned activities of Transform: WASH a standard part of reporting (e.g., included in the quarterly, bi-annual, and annual reports)? How has information generated from these processes influenced the activity in terms of advancing gender equality and women’s empowerment? 5. What are the expected intermediate level and immediate level results on gender equality/women’s empowerment aspects? Are these realistic as well as sufficiently ambitious? 6. Are all technical teams of Transform: WASH fully aware of how best to consider gender equality/women empowerment aspects in the project during planning, implementation, and reporting? PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 23 7. What strategies have worked most effectively for the different technical teams to ensure gender equality/women empowerment are part and parcel of Transform: WASH intervention areas? Probe at both the enterprise level (supply) and the household level (demand). 8. What challenges have you faced while supporting implementation of gender equality/women empowerment activities under Transform: WASH? 9. What lessons have you learned from these challenges? 10. What suggestions do you have that would improve/strengthen the gender equality/women empowerment approaches and interventions of Transform: WASH? 11. How would you measure the achievement of the Transform: WASH project with regards to gender equality/women empowerment? (Please give a rating from 1–10 with 10 being the highest.) M&E/Knowledge Management (KM) 1. Please describe the KM policy framework of the Transform: WASH Project for different project component. Probe for demand creation/social mobilization, MBS, and governance. 2. How did the Transform: WASH project set a learning agenda (develop learning question)? Was it jointly or individually? Did you involve the government in setting a learning agenda? 3. What systems did Transform: WASH put in place to manage and share knowledge (system for creation, processing, storing/retaining, retrieving, sharing and use) at community, district, region/woreda and national levels? Probe for: Between consortium partners, government/political leaders, other development partners, private sector. Ask for evidence. 4. Was there a culture of providing feedback and a system for joint performance reviews of the Transform: WASH project activity by/within the consortium? With the government at all levels? If yes, how frequent and effective have these practices been? 5. Was there a culture of providing feedback and a system for joint performance reviews of the Transform: WASH project activity with the government at all levels? If yes, how frequent and effective have these practices been? 6. How did you observe adoption of best practices (use of the research findings, innovations) by Transform: WASH project stakeholders (consortium members, government, other development partners, etc.)? Have research findings contributed to changes of policies, practices, monitoring indicators, improving program management? If yes, provide examples: _______ 7. How frequently did you provide Transform: WASH project performance report/update to the woreda WASH sector offices, regional bureaus, and federal sector ministries? Did the Transform: WASH project get feedback from the government at all levels? Was there government feedback on the content, quality, and timeliness of the report/outputs/results? If yes, what was the Transform: WASH reflection? 8. Did the Transform: WASH project adopt program/project methodologies/approach of other development partners and make any modifications to Transform: WASH project activities as a result? If yes, mention_____ 9. Did the Transform: WASH project establish KM capacity building for the government in terms of technical aspects or strengthening the existing/establishing innovative system and structures including data quality assessment and verification? If yes, mention: _______ PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 24 10. What are the key success factors for development and implementation of KM systems? 11. What are the common challenges and areas of future improvement related to the KM? 12. Any additional points/issues that you may consider important related to KM going forward? Conclusion 1. Thank the respondent for their time. 2. Tell the respondent they are welcome to contact you to ask questions at a later date. 3. Suggest that you may be in contact to follow up on specific issues. 4. Ask permission of the respondent to use their name in the report if you might use a quote. Note their response: ____________. PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 25 C-1.2 KEY INFORMANT INTERVIEW TOPIC GUIDE—REGIONAL/WOREDA-LEVEL GOVERNMENT STAFF/WWT AND WTT MEMBERS This Guide is intended for use in terms of data collection from regional and woreda-level government staff and WWT/WTT members. This evaluation will assess the effectiveness of Transform: WASH’s technical assistance activities to increase viability and professionalism of WASH service providers and support the enabling environment for market-based approaches more generally in Ethiopia. The evaluation will also help inform USAIDs future decisions on WASH programming in Ethiopia. It should be followed as closely as possible to guide key informant interviews with these respondents. There are a few questions where a specific set of responses are provided to assist in the collection of data that can be more easily quantified. Instructions to the interviewer are in red. Interviewers will be trained to be compliant with USAID policy regarding the “USG Common Rule” for the protection of human subjects. Introduction (~10 minutes) 1. Thank the respondent for taking the time to participate in the interview. 2. Introduction to the researcher and the research a. Introduce yourself: I am a consultant residing in ______. I represent an evaluation team fielded by Tetra Tech ARD, a Washington DC-based firm that has been contracted by the US Agency for International Development (USAID) to conduct an independent evaluation of the USAID Transform: WASH program. b. As part of its own planning for the next few years, USAID has asked us to conduct a final performance evaluation of the Transform: WASH program to assess its progress toward achieving its objectives. So, what we are trying to assess are the strengths and weaknesses of the program, its accomplishments, and best practices, but also any obstacles and shortcomings faced and how it could be more effective. c. Ultimately the research will be used to ascertain the extent to which Transform: WASH activities have improved lives and health through the development and management of sustainable water, sanitation, and hygiene services in Ethiopia. It will also be used to propose recommendations based on the findings to inform future WASH programming by USAID. 3. We will follow privacy protocols to protect your anonymity. a. Explain confidentiality and anonymity and note whether the respondent would like to remain anonymous, and that the assessment team will ask permission if would like to attribute a quote directly from the respondent in the final report. b. Explain how collected data will be stored without identifying information. c. Ask if the respondent is willing to be recorded and note their response. 4. Explain recording, length, and nature of discussion 5. Check whether respondents have any questions. Transform: WASH’s contribution to the sector (~45 minutes) Transition: I would like to spend some time speaking with you about your knowledge of the USAID￾supported Transform: WASH program. 1. What notable milestones and achievements has the WASH sector realized in your region/woreda in recent years? Open with WASH broadly and then probe for sanitation specifically. PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 26 2. Does the Ethiopian policy environment explicitly endorse and foster market-based approaches for sanitation services? Is there clarity in the Ethiopian context about the different ways of fostering and the elements required to implement market-based approaches in the WASH sector? 3. How familiar are you with the Transform: WASH program? 4. In what areas has your institution worked/collaborated with the Transform: WASH program (i.e., sanitation, finance)? 5. In your opinion, was Transform: WASH addressing the right challenges with regard to expanding and ensuring sanitation service coverage? 6. Do you feel Transform: WASH has worked collaboratively with regional/woreda government and local stakeholders to define its priorities and align with government planning cycles and strategies? If so, can you provide examples and evidence? 7. In your view, in what area of intervention has Transform: WASH made the greatest contribution and been particularly successful in your region/woreda? Probe for examples of: a. Expanding sanitation coverage b. Ensuring sustainable access to basic sanitation services c. Building woreda capacity for monitoring service delivery, planning, investment, monitoring and regulatory functions, etc. (e.g., through the WWTs/WTTs) — probe explicitly for ISI uptake, CWA support, and OWNP reporting. d. Financing models e. Service provider and enterprise support f. Influencing government policy g. Gender, female empowerment, and social inclusion mainstreaming h. Using MSPs to advance understanding across stakeholder groups i. Other 8. In terms of finance, how has Transform: WASH helped to ensure greater financial viability for service providers in your region/woreda? 9. How has Transform: WASH contributed to expanding sanitation services in the woreda from a supply side (i.e., working with product and service providers)? Probe for examples of both water and sanitation. 10. How has Transform: WASH contributed to expanding sanitation services in the region/woreda from a demand side (i.e., working with households and communities)? Probe for examples of both water and sanitation. 11. What is the likelihood of these achievements being sustained? Probe for: a. At the product and service provider level b. At the household/community level c. At the institutional level (where appropriate) PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 27 d. Influencing stakeholders at the woreda or national levels e. Around gender mainstreaming 12. Is there anything that you think Transform: WASH could have done differently to have greatest impact at both national level and in your region/woreda? 13. Are you working with other development partners in your region/woreda around sanitation service delivery? Is there any overlap in what other organizations do and what Transform: WASH does? 14. What is the reputation and “brand” of Transform: WASH amongst you and your colleagues at the region/woreda level? 15. Beyond financing, what recommendations, if any, do you have for USAID with regard to the design and delivery of such programs to strengthen the WASH sector to ensure and expand access to sustainable sanitation services? 16. Revisit general question guide to determine if there are specific aspects that this stakeholder could usefully respond to, particularly with regard to gender equality and female empowerment (GEFE). Conclusion: 1. Thank the respondent for their time. 2. Tell the respondent they are welcome to contact you to ask questions at a later date. 3. Suggest that you may be in contact to follow up on specific issues. 4. Ask permission of the respondent to use their name in the report if you might use a quote. Note their response: ____________. PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 28 C-1.3 KEY INFORMANT INTERVIEW TOPIC GUIDE—NATIONAL LEVEL COLLABORATORS AND DEVELOPMENT PARTNERS (INCLUDING CORPORATE PARTNERS [FINANCIAL INSTITUTIONS, LIXIL, SILAFRICA, ETC.]) This Guide is intended for use in terms of data collection from national level collaborators and development partners. This evaluation will assess the effectiveness of Transform: WASH’s technical assistance activities to increase viability and professionalism of WASH service providers and support the enabling environment for market-based approaches more generally in Ethiopia. The evaluation will also help inform USAIDs future decisions on WASH programming in Ethiopia. It should be followed as closely as possible to guide key informant interviews with these respondents. There are a few questions where a specific set of responses are provided to assist in the collection of data that can be more easily quantified. Instructions to the interviewer are in red. Interviewers will be trained to be compliant with USAID policy regarding the “USG Common Rule” for the protection of human subjects. Introduction (~10 minutes) 1. Thank the respondent for taking the time to participate in the interview 2. Introduction to the researcher and the research: a. Introduce yourself: I am a consultant residing in ______. I represent an evaluation team fielded by Tetra Tech ARD, a Washington DC-based firm that has been contracted by the US Agency for International Development (USAID) to conduct an independent evaluation of the USAID Transform: WASH program. b. As part of its own planning for the next few years, USAID has asked us to conduct a final performance evaluation of the Transform: WASH program to assess its progress toward achieving its objectives. So, what we are trying to assess are the strengths and weaknesses of the program, its accomplishments, and best practices, but also any obstacles and shortcomings faced and how it could be more effective. c. Ultimately the research will be used to ascertain the extent to which Transform: WASH activities have improved lives and health through the development and management of sustainable water, sanitation, and hygiene services in Ethiopia. It will also be used to propose recommendations based on the findings to inform future WASH programming by USAID. 3. We will follow privacy protocols to protect your anonymity. a. Explain confidentiality and anonymity and note whether the respondent would like to remain anonymous, and that the assessment team will ask permission if would like to attribute a quote directly from the respondent in the final report. b. Explain how collected data will be stored without identifying information. c. Ask if the respondent is willing to be recorded and note their response. 4. Explain recording, length, and nature of discussion. 5. Check whether respondents have any questions. Transform: WASH’s contribution to the sector (~45 minutes) Transition: I would like to spend some time speaking with you about your knowledge of the USAID￾supported Transform: WASH program. 1. What notable milestones and achievements has the WASH sector realized in recent years (2016–2020)? Open with WASH broadly and then probe for sanitation specifically. PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 29 2. For sanitation service delivery, which regions/woredas do you think are making most significant progress? Why do you think this is the case? 3. Does the Ethiopian policy environment explicitly endorse and foster market-based approaches for sanitation services? Is there clarity in the Ethiopian context about the different ways of fostering and elements required to implement market-based approaches in the WASH sector? 4. How familiar are you with the Transform: WASH program, its approaches, and achievements to date? 5. In what areas/ways has your institution worked/collaborated with the Transform: WASH program? Probe for specific mechanisms. 6. In your opinion, what are the challenges in the WASH sector that a project like Transform: WASH should be addressing? In your opinion, has Transform: WASH been designed to address these issues? 7. In your view, in what area of intervention has Transform: WASH made the greatest contribution and been particularly successful? Probe for examples of: a. Expanding sanitation coverage b. Ensuring sustainable access to basic sanitation services c. Building woreda capacity for monitoring service delivery, planning, investment, monitoring and regulatory functions, etc. (e.g., through the WWTs/WTTs) – probe explicitly for ISI uptake, CWA support, OWNP reporting d. Financing models e. Service provider and enterprise support f. Influencing government policy g. Gender, women’s empowerment, and social inclusion mainstreaming h. Using MSPs to advance understanding across stakeholder groups i. Other 8. As far as you are aware, is there any indication that Transform: WASH woredas are performing better than non-Transform: WASH woredas? If so, what reasons would you attribute to these differences? 9. What is the likelihood of Transform: WASH achievements being sustained? Probe for: a. At the enterprise level b. At the household/community level c. Influencing stakeholders at the county or national levels d. Around gender mainstreaming and equity and social inclusion 10. What internal factors do you think have enabled Transform: WASH to be successful in its efforts? Probe for factors within Transform: WASH control: staffing capacity, staffing skill set, budgeting, sequencing of activities, nature of collaboration between members of the implementing consortium, etc. PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 30 11. Are there any internal factors (i.e., within Transform: WASH’s direct control) that you think are undermining or affecting efforts and performance? 12. What external factors do you think have enabled Transform: WASH to be successful in its efforts? Probe for factors beyond Transform: WASH control (e.g., policy environment, support of other like-minded actors, donor relations, COVID-19, FOREX issues, and conflict). 13. Are there any external factors (i.e., beyond Transform: WASH’s direct control) that you think are undermining or affecting efforts and performance? 14. How specifically has Transform: WASH influenced government policy (specifically through OWNP) with regard to market-based sanitation services? Probe for specific contributions. 15. How specifically has Transform: WASH supported increased government capacity and performance improvements with regard to the design and implementation of sanitation services through an MBS approach? 16. As relevant, how has Transform: WASH worked with other development partners at the national level and in different woredas around sanitation service delivery and MBS approaches? What challenges have been encountered, if any? Probe for duplication as well as complimentary and contradictory approaches to capacity building, finance and subsidy, construction, etc., and how Transform: WASH built on or addressed these. 17. What is the reputation and “brand” of Transform: WASH in Ethiopia generally? 18. Is there anything that you think Transform: WASH could have done differently to have greatest impact at both national and regional/woreda levels? Probe for policy/strategy and operational levels. 19. What recommendations, if any, do you have for USAID with regard to the design and delivery of future programs to strengthen the WASH sector to ensure and expand access to sustainable sanitation services? Conclusion: 1. Thank the respondent for their time. 2. Tell the respondent they are welcome to contact you to ask questions at a later date. 3. Suggest that you may be in contact to follow up on specific issues. 4. Ask permission of the respondent to use their name in the report if you might use a quote. Note their response: ____________. PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 31 C-1.4 KEY INFORMANT INTERVIEW TOPIC GUIDE—SANITATION ENTERPRISES RECEIVING TRANSFORM: WASH SUPPORT This Guide is intended for use in terms of data collection from Sanitation Enterprises receiving Transform: WASH support. This evaluation will assess the effectiveness of Transform: WASH’s technical assistance activities to increase viability and professionalism of WASH service providers and support the enabling environment for market-based approaches more generally in Ethiopia. The evaluation will also help inform USAIDs future decisions on WASH programming in Ethiopia. It should be followed as closely as possible to guide key informant interviews with these respondents. There are a few questions where a specific set of responses are provided to assist in the collection of data that can be more easily quantified. Instructions to the interviewer are in red. Interviewers will be trained to be compliant with USAID policy regarding the “USG Common Rule” for the protection of human subjects. Introduction (~10 minutes) 1. Thank the respondent for taking the time to participate in the interview. 2. Introduction to the researcher and the research: a. Introduce yourself: I am a consultant residing in ______. I represent an evaluation team fielded by Tetra Tech ARD, a Washington DC-based firm that has been contracted by the US Agency for International Development (USAID) to conduct an independent evaluation of the USAID Transform: WASH program. b. As part of its own planning for the next few years, USAID has asked us to conduct a final performance evaluation of the Transform: WASH program to assess its progress toward achieving its objectives. So, what we are trying to assess are the strengths and weaknesses of the program, its accomplishments, and best practices, but also any obstacles and shortcomings faced and how it could be more effective. c. Ultimately the research will be used to ascertain the extent to which Transform: WASH activities have improved lives and health through the development and management of sustainable water, sanitation, and hygiene services in Ethiopia. It will also be used to propose recommendations based on the findings to inform future WASH programming by USAID. 3. We will follow privacy protocols to protect your anonymity. a. Explain confidentiality and anonymity and note whether the respondent would like to remain anonymous, and that the assessment team will ask permission if would like to attribute a quote directly from the respondent in the final report. b. Explain how collected data will be stored without identifying information. c. Ask if the respondent is willing to be recorded and note their response. 4. Explain recording, length, and nature of discussion 5. Check whether respondents have any questions. Capacity Building of WASH enterprises (~45 minutes) Transition: I would like to spend a little time speaking with you about your experience working with the USAID-supported Transform: WASH program. 1. Please describe your sanitation enterprise. Note size, scope of services, location, other information about maturity, woman or men owned. Are you a formal business? Is the business PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 32 legally registered? Is the sanitation enterprise a Community-Managed Scheme or a private business? Provide water and sanitation services? Provide other services (e.g., construction)? 2. What is the catchment area for your service? Technologies used? How many customers do you have? Is there room (opportunity) for growth? 3. When were you first introduced to Transform: WASH and how were you selected to receive Transform: WASH support? Could you describe your level of interaction with Transform: WASH-funded activities? 4. Based on your experience, please describe Transform: WASH’s approach to capacity building of WASH enterprises. 5. Has your business tangibly benefited from Transform: WASH’s technical assistance? If yes, please provide examples. If yes, what was the most useful skill or capacity Transform: WASH has enhanced? 6. Do you feel your business has benefited from individual “mentoring” and “incubation”? Has Transform: WASH’s approach been responsive to the specific needs of your enterprise? 7. Are there other types of technical assistance that Transform: WASH could have provided that would have been beneficial to you? If yes, mention: ______ 8. How, if at all, would you recommend Transform: WASH adjust its programming (activity) to better support entrepreneurs/enterprises like yourself? 9. How, if at all, do you assess whether Transform: WASH is addressing gender in its programming or planning? Were gender aspects emphasized by Transform: WASH? 10. Has this approach helped you see the benefit of inclusivity as a profitable business model? 11. How do you assess what exactly the Transform: WASH has done in encouraging women-led businesses to advance their enterprise? 12. Have Transform: WASH programs resulted in more women being employed? 13. How have Transform: WASH market-based activities fostered the expansion of sustainable services to women and girls? 14. Have Transform: WASH efforts specifically to create demand in your area enhanced your business viability? 15. Has the viability of your business improved since benefiting from Transform: WASH support? What challenges did you face working in a sanitation business? Do you intend to remain in the WASH services business for the foreseeable future? 16. What is your overall view of Transform: WASH? How has Transform: WASH impacted the effectiveness and viability of sanitation product and service providers in your county? Other questions (~15 minutes) 1. What interaction do you have with WWTs/WTTs? How is your relationship with these bodies? Is this formal or informal? 2. Do you feel that Transform: WASH’s approach to strengthen WASH service providers is sufficiently linked with an effort to increase capacity with the Water Boards and county governments? PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 33 3. In terms of sustainability, what are the main challenges you face beyond areas where Transform: WASH has provided support? 4. What other donor organizations/nongovernmental organizations (NGOs) does your enterprise work with (if any)? What do these other organizations do? 5. Is there any overlap in what these organizations do and what Transform: WASH does? 6. What is the reputation and “brand” of Transform: WASH among the private sector providers in your woreda? 7. What activities do you think should be scaled up in order to maximize project results during the remainder of its implementation? Conclusion: 1. Thank the respondent for their time. 2. Tell the respondent they are welcome to contact you to ask questions at a later date. 3. Suggest that you may be in contact to follow up on specific issues. 4. Ask permission of the respondent to use their name in the report if you might use a quote. Note their response: ____________. PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 34 C-1.5 KEY INFORMANT INTERVIEW GUIDE – USAID This KII Guide is intended for use in terms of data collection from USAID staff. This evaluation will assess the effectiveness of Transform: WASH’s technical assistance activities to increase viability and professionalism of WASH service providers and support the enabling environment for market-based approaches more generally in Ethiopia. The evaluation will also help inform USAIDs future decisions on WASH programming in Ethiopia. It should be followed as closely as possible to guide key informant interviews with these respondents. There are a few questions where a specific set of responses are provided to assist in the collection of data that can be more easily quantified. Instructions to the interviewer are in red. Interviewers will be trained to be compliant with USAID policy regarding the “USG Common Rule” for the protection of human subjects. Introduction (~10 minutes) 1. Thank the respondent for taking the time to participate in the interview. 2. Introduction to the researcher and the research: a. Introduce yourself: I am a consultant residing in ______. I represent an evaluation team fielded by Tetra Tech ARD, a Washington DC-based firm that has been contracted by the US Agency for International Development (USAID) to conduct an independent evaluation of the USAID Transform: WASH program. b. As part of its own planning for the next few years, USAID has asked us to conduct a final performance evaluation of the Transform: WASH program to assess its progress toward achieving its objectives. So, what we are trying to assess are the strengths and weaknesses of the program, its accomplishments, and best practices, but also any obstacles and shortcomings faced and how it could be more effective. c. Ultimately the research will be used to ascertain the extent to which Transform: WASH activities have improved lives and health through the development and management of sustainable water, sanitation, and hygiene services in Ethiopia. It will also be used to propose recommendations based on the findings to inform future WASH programming by USAID. 1. We will follow privacy protocols to protect your anonymity. a. Explain confidentiality and anonymity and note whether the respondent would like to remain anonymous, and that the assessment team will ask permission if would like to attribute a quote directly from the respondent in the final report. b. Explain how collected data will be stored without identifying information. c. Ask if the respondent is willing to be recorded and note their response. 2. Explain recording, length, and nature of discussion. 3. Check whether respondents have any questions. Transform: WASH’s approaches and wider contribution to the sector (~45 minutes) 1. Transition: I would like to spend some time speaking with you about your knowledge of the USAID-supported Transform: WASH program. 2. What notable milestones and achievements has the sanitation sector realized in Ethiopia since the start of the Transform: WASH project? 3. In terms of the enabling environment, does the Ethiopian policy environment explicitly endorse and foster market-based approaches for sanitation services? Is there clarity in the Ethiopian PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 35 context about the different ways of fostering and the elements required to implement market￾based approaches in the WASH sector? How do the governance structures promote or hinder sanitation service delivery and sustainability? 4. What do you feel has been Transform: WASH’s most significant accomplishment to date? Probe for examples of: a. Expanding sanitation coverage b. Ensuring sustainable access to sanitation services c. Building woreda capacity for monitoring service delivery, planning, investment, monitoring and regulatory functions, etc. (e.g., through the WWTs/WTTs) d. Financing models e. Service provider and enterprise support f. Influencing government policy g. Gender and social inclusion mainstreaming h. Other 5. For sanitation service delivery and sustainability, which Transform: WASH regions/woredas or which parts of the country do you think are making most significant progress? Why do you think this is the case? 6. How have you seen Transform: WASH linking in with other USAID or development partner programs to ensure that the policy, capacity and investment environment is enabling expanded and sustained service delivery for sanitation? Probe for evidence of effectiveness. 7. Is there any overlap in what other organizations do and what Transform: WASH does? Probe for capacity building, finance, construction. 8. Do you think Transform: WASH is appropriately linking up the supply and demand sides – and balancing work across enabling environment, supply, and demand aspects? 9. What internal factors do you think have enabled Transform: WASH to be successful in its efforts (e.g., staffing, budgeting, and sequencing of activities)? 10. What external factors do you think have enabled Transform: WASH to be successful in its efforts (e.g., policy environment, and support of other like-minded actors)? 11. From your vantage point, are there any internal that you think are undermining or affecting their efforts and performance? 12. Are there any external factors (i.e., both within or beyond Transform: WASH’s direct control) that you think are undermining or affecting their efforts and performance? 13. How specifically has Transform: WASH influenced government policy with regard to sanitation services? 14. How specifically has Transform: WASH influenced USAID/Ethiopia approaches and investments with regard to sanitation services? 15. What is the reputation and “brand” of Transform: WASH amongst you and your colleagues? And in the sector more broadly? PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 36 16. What might Transform: WASH have done differently to maximize impact at region/woreda level? And at national level? 17. Based on what you have seen in Transform: WASH, what recommendations, if any, do you have for future programming with regard to the design and delivery of future programs to strengthen the WASH sector to ensure and expand access to sustainable WASH services? Conclusion: 1. Thank the respondent for their time. 2. Tell the respondent they are welcome to contact you to ask questions at a later date. 3. Suggest that you may be in contact to follow up on specific issues. 4. Ask permission of the respondent to use their name in the report if you might use a quote. Note their response: ____________. PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 37 C-2.0 HOUSEHOLD SURVEY PRIN International was subcontracted to carry out the HHS data collection and support the analysis. 3076 HHSs were conducted, providing sufficient data for adequate comparison between intervention (2,308) and control woredas (768) and to establish an understanding of the influence of Transform: WASH on outcomes. (See table below for summary of sample households selected by location.) G￾Power software was used to calculate the sample size with assumptions of effect size= 0.2, power= 0.85, margin of error= 0.05, design effect=2, non-response rate=0.08, and allocation ratio (I:NI= 4:1). Multi-stage cluster sampling was used for HHS data collection, where regions, zones, districts, Kebeles, and villages will be selected, respectively. A total of 12 intervention woredas and 4 control woredas were targeted for data collection, including woredas that border intervention Kebeles to capture spillover effects of Transform: WASH interventions. The number of households surveyed from each Kebele was determined based on the relative population size of the Kebele, and households were randomly selected from each village. The process for household selection from both intervention and control woredas was carried out in the exact same manner. The HHS was written in English and translated to Amharic and other local languages before being programmed into the mWater data collection and management app. SUMMARY OF SAMPLE HOUSEHOLDS BY LOCATION2 REGION ZONE WOREDA INTERVENTION STATUS #KEBELES #GOTTS #HHS Hadiya Merab Badiwacho T: WASH 1 6 12 180 SNNPR Wolayta Kindo Dedaye T: WASH 2 6 12 180 Gamo Mirab Abaya Control (Non-adjacent) 6 12 192 Sidama Sidama Aleta Wondo T: WASH 3 7 14 217 East Arsi Wondo T: WASH 4 6 12 180 Oromia South West Shewa Woliso T: WASH 5 6 12 180 West Arsi Dodola T: WASH 6 6 12 180 Gedeb Assesa Control (Adjacent) 6 12 192 East Gojjam Guzamen T: WASH 7 6 12 180 South Wollo Kalu T: WASH 8 6 12 180 Amhara Were Elu T: WASH 9 6 12 180 Alibuko Control (Adjacent) 6 12 192 North Shewa Angollala Tera Control (Non-adjacent) 6 12 192 Somali Dega Bour Dega Bour T: WASH 10 7 14 217 Afar Zone 1 Mille T: WASH 11 7 14 217 Dire Dawa 1 Sub-City Biyo Awale T: WASH 12 7 14 217 Total 13 16 12+4 124 248 3,076 PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 38 2 Reader should note that since the original selection of the woredas for visits, one initial woreda (Debre Elias from East Gojjam) has been replaced by Ali Booko woreda in South Wollo of the Amhara Region. Debre Elias has been part of the Growth through Nutrition activity and thus was determined as not appropriate to serve as a control woreda. HOUSEHOLD SURVEY QUESTIONNAIRE INTERVIEW FOR ADULT HOUSEHOLD MEMBER SERIAL NUMBER……… INFORMED CONSENT Hello. My name is _______________________________________. I am working with the USAID￾supported WASHPaLS 2 project. We are conducting a national survey to assess the knowledge, attitude, behavior, and access to sanitation services in seven regions plus the Dire Dawa Administrative Area. The information we collect will help the Ethiopian Federal Ministry of Health and USAID/Ethiopia to plan, monitor, and evaluate the Transform–WASH program. Your household was randomly selected for the survey. All of the answers you give will be confidential and will not be shared with anyone other than members of our survey team. You don't have to be in the survey, but we hope you will agree to answer the questions since your views are important. If I ask any question you don't want to answer, just let me know and I will go on to the next question or you can stop the interview at any time. I have gone through many of the experiences we will be talking about. You are encouraged to answer as many questions as possible. Your participation is voluntary and, if you are too uncomfortable with a question, you may choose not to answer. I will first ask you my questions, which will take up to 1 hour. After that, if you have any questions for me, we can discuss things that concern you. If you have any further questions about the study, you may contact ______________, with telephone number __________ Do you agree to participate in the study? 0 = No…………………. End Interview 1= Yes………………… Continue PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 39 I D E N T I F I C A T I O N C O D E S Region ……… 1= Oromiya 2= Amhara 3= SNNP 4=Sidama 5= Afar 6= Somali 7= Gambella 8=Dire Dawa Zone ………………………………………… Woreda …………………………………… Kebele ……………………………………. Name Of Village/Gott……………………………………… Structure/Household Number …………………………… Number of Household Members Who Are Eligible For The Survey: Aged 18 and over: Female_________ Male _______ Total Number In Household: Male_____ Female _____ Elderly ______ PWDS (unable to hear, talk and walk, etc) _________ Number Of Children Under 5 _______ Name Of Interviewer……………………………………………… Name Of Supervisor ……………………………………………… ………………… ………………… ………………… ………………… ………………… ………………… ………………… ………………… ………………… MAP INFORMATION IDENTIFICATION LABEL CODES GPS Unit Tracking Number Waypoint name (entered in GPS unit) Latitude (North/South) N/S . Longitude (East/West) E/W . Altitude / Elevation (Meters) Note: GPS will be also taken at end of the interview Interview Date ___________ PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 40 Language Used in Interview 1 = Amharic 2 = Afan Oromoo 3=Tigrigna 4 = Somali 5 = Afar 6=Other (Specify) ________________ ___________________ Supervisor Signature and Date __________________ ____________________ SECTION 100: INTERVIEWEE / FAMILY BACKGROUND 101 Sex of the Respondent 1 = Male 2 = Female 102 Are you the head of household? 1 = Yes 2 = No 103 Family Type 1 = non-polygamous 2 = Polygamous 104 Enumerator to note if female-headed household or male-headed household 1 = Male headed household 2 = Female-headed household 105 How old were you on your last birthday? Record completed years [____|____] Years If Less Than 15, End Interview. 106 What is your religion? If Respondent answers “Christian,” ask “are you Catholic, Or Protestant?” 1 = Muslim 2 = Protestant 3 = Catholic 4 = Orthodox 5 = Traditional 6 = No religion 7 = Other (Specify) ___________________ 107 What is your marital status? 1 = Single 2 = Married or Cohabitating 3 = Divorced 4 = Widow 5 = Separated PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 41 108 How many years have you lived in this location/Kebele? If less than 1 year, record ‘00’ circle ‘97’ if respondent was born in this location and has lived there for her entire life other than temporary absence. [____|____] years 97 = Whole Life 109 Can you read and write? 1 = Yes 0 = No (If No, Skip to Q112) 110 If you can read and write, what types of education did you attend? 1=Formal education 2=Non-formal education (If Non-formal, Skip to Q112) 3= Others (Specify_________________) 111 What is the highest [GRADE] you completed? If completed less than one year at that level, record '00' [____|____] Completed Grade 97 = TVET Graduate or student 98 = College graduate or student 99 = University graduate or student 112 What is the primary source of income for the household? 1 = Paid labor 2 = Farming 3 = Own enterprise 4 = Other (Specify_______________) 113 Have you worked for pay in the last three months? 0 = No (Go to Q201) 1 = Yes 114 Are you currently working for pay or have you worked for pay in the last three months? 0 = No (Go to Q201) 1 = Yes PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 42 115 What type of work for pay are you 1 = Farming currently doing or done in the last three months? Probe two times ‘anything else?’ 2 = Domestic work /cleaner 3 = Construction/ pottering 4 = Petty trade circle all that apply 5 = Dress making /hairdressing 6 = Waitress / barmaid 7 = Small scale manufacturing 8 = Salesperson 9 = Tradesman (plumber, elect, carpenter etc.) 10 = Professional (health care, teacher, manager, etc.) 11 = Any other work? (Please specify) ___________ SECTION 200: HOUSEHOLD LIVING CONDITIONS 201 Observe the main material of the floor of the dwelling. Record Observation. Natural Floor Earth/Sand . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11 Dung . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12 Rudimentary Floor Wood Planks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 Palm/Bamboo . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 Finished Floor Parquet Or Polished Wood . . . . . . . . . . . . . . . . . 31 Vinyl Or Asphalt Strips/Plastic Tile . . . . . . . . . . . .32 Ceramic Tiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Cement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 Carpet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 Other . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 202 Observe the main material of the roof of the dwelling. Record Observation. Natural Roofing No Roof . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Thatch/Mud . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Sod . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 43 Rudimentary Roofing Rustic Mat/ Plastic Sheet. . . . . . . . . . . . . . . . . . . . 21 Reed/Bamboo . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 Wood Planks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 Cardboard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 Finished Roofing Metal/Corrugated Iron . . . . . . . . . . . . . . . . . . . . . 31 Wood . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 Roofing Shingles . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Other (specify______________) . . . . . . . . . . . 98 203 Observe the main material of the exterior walls of the dwelling. Record Observation. Natural Walls No Walls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11 Cane/Palm/Trunks/Bamboo/Reed . . . . . . . . . . . . .12 Dirt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Rudimentary Walls Bamboo With Mud . . . . . . . . . . . . . . . . . . . . . . . . 21 Stone With Mud . . . . . . . . . . . . . . . . . . . . . . . . . . 22 Uncovered Adobe . . . . . . . . . . . . . . . . . . . . . . . . 23 Plywood . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 Cardboard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 Reused Wood . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 Finished Walls Cement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 Stone With Lime/Cement . . . . . . . . . . . . . . . . . . 32 Bricks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Cement Blocks . . . . . . . . . . . . . . . . . . . . . . . . . . .34 Covered Adobe . . . . . . . . . . . . . . . . . . . . . . . . . . 35 Wood Planks/Shingles . . . . . . . . . . . . . . . . . . . . . 36 Corrugated Sheet. . . . . . . . . . . . . . . . . . . . . . . . . 37 Other . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 204 How many people usually live in your household? (Total number of people) [____|____] Persons 205 What are your sources of energy? 1 = Biogas PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 44 TICK ALL THAT APPLY 2 = Solar 3 = Wood 4 = Animal dung 5 = Liquefied Petroleum Gas (LPG) 6 = Kerosene 7 = Electricity 8 = Other (Specify____________________) 206 Does your household have: a) Electricity? b) Radio? c) Television d) Non-mobile telephone e) Computer? f) Refrigerator? g) Table? h) Chair? i) Bed with cotton/sponge/spring mattress? j) An electric mitad? k) A kerosene lamp/pressure lamp? Yes No a) Electricity . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 0 b) Radio . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 0 c) Television . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 0 d) Non-Mobile Telephone . . . . . . . . . . . . . . . . 1 0 e) Computer . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 0 f) Refrigerator . . . . . . . . . . . . . . . . . . . . . . . . . . .1 0 g) Table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1 0 h) Chair . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 0 i) Bed with cotton/sponge/spring. . . . . . . . . . . . 1 0 j) Electric Mitad . . . . . . . . . . . . . . . . . . . . . . . . . 1 0 k) Kerosene lamp/pressure lamp. . . . . . . . . . . . 1 0 207 Does any member of this household own: a) Wall clock? b) A mobile phone? c) A bicycle? d) A motorcycle or motor scooter? e) An animal-drawn cart? f) A car or truck? g) A boat with a motor? h) A bajaj? Yes No a) Wall Clock . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 0 b) Mobile Phone . . . . . . . . . . . . . . . . . . . . . . . . . 1 0 c) Bicycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 0 d) Motorcycle/Scooter . . . . . . . . . . . . . . . . . . . . 1 0 e) Animal-Drawn Cart . . . . . . . . . . . . . . . . . . . 1 0 f) Car/Truck . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 0 g) Boat With Motor . . . . . . . . . . . . . . . . . . . . . 1 0 h) Bajaj . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 0 208 How many of the following animals does this household own? If None, Record '00'. If 95 Or More, Record '98'. PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 45 209 210 211 If Unknown, Record '99'. a) Milk cows, oxen or bulls? b) Other cattle? c) Horses, donkeys, or mules? d) Camels e) Goats? f) Sheep? g) Chickens or other poultry? h) Beehives? Does any member of this household have a bank or microfinance savings account? What other savings mechanism do any members the of household have? Is your household enrolled in Community Based Health Insurance (CBHI) schemes? a) Cows/Bulls __________ b) Other Cattle __________ c) Horses/Donkeys/Mules __________ d) Camels __________ e) Goats __________ f) Sheep __________ g) Chickens/Poultry __________ h) Beehives __________ 0 = No 1 = Yes 1= Iquib 2= Others Specify_______) 0 = No 1 = Yes 9 = Don’t Know SECTION 300 – ACCESS TO WATER 301 What is the main source of PIPED WATER drinking water for members of Piped Into Dwelling . . . . . . . . . . . . . . . . 11 GO your household? Piped To Yard/Plot . . . . . . . . . . . . . . . . .12 TO Q303 Piped To Neighbor . . . . . . . . . . . . . . . . .13 Public Tap/Standpipe . . . . . . . . . . . . . . . 14 Shallow Borehole With Hand Pump . . . 21 Deep Borehole (Motorized/Powered). . 22 Dug Well Protected Well . . . . . . . . . . . . . . . . . . . . 31 Unprotected Well . . . . . . . . . . . . . . . . . 32 Water From Spring Protected Spring. . . . . . . . . . . . . . . . . . . .41 Unprotected Spring . . . . . . . . . . . . . … . 42 PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 46 Rainwater . . . . . . . . . . . . . . . . . . . . . . . 51 Tanker Truck . . . . . . . . . . . . . . . . . . . . . 61 Cart With a Small Tank . . . . . . . . . . . . . .71 Surface Water River/Lake/Pond/Stream/Dam . . . . . . . . . 81 Water Kiosk. . . . . . . . . . . . . . . . . . . . . . . 90 Bottled Water . . . . . . . . . . . . . . . . . . . . . 91 Other (Specify) . . . . . . . . . . . . . . . . . . . . .98 _______________________________ 302 Where is that water source located? 1 = In Own Dwelling 2 = In Own Yard/Plot 3 = Elsewhere 303A How long does it take to go to the water source, get water, and come back? Minutes [____|____|____] 303B How many days in a week can you get water from its source? ______days 303C Do you pay for water? 1= Yes 0= N0 303D If Yes, how much do you pay for water monthly? __________BIRR 304 What is the main source of water used by your household for other purposes a) Cooking____________ b) Hand washing________ c) Bathing and showering_____ Ask these questions separately and write the code on blank space. Note: Multiple response is possible Piped Water Piped Into Dwelling . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Piped To Yard/Plot . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Piped To Neighbor . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Public Tap/Standpipe . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Shallow Borehole With Hand Pump. . . . . . . . . . . . . . . 21 Deep Borehole (Motorized/Powered) . . . . . . . . . . . . 22 Dug Well Protected Well . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 Unprotected Well . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 Water From Spring PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 47 Protected Spring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 Unprotected Spring . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 Rainwater . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 Tanker Truck . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 Cart With Small Tank . . . . . . . . . . . . . . . . . . . . . . . . . 71 Surface Water River/Lake/Pond/Stream/Dam . . . . . . . . . . . . . . . . . . . 81 Other (Specify) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 _____________________________ 305 Totally how many liters of water does the household consume per day? (Observe the type of container, ask or read how much liter the container can hold) Note: This question includes water consumption for cooking, body washing, toilet washing, watering plants, consumption by animals and other purposes at home 1. Number of container: ____________ 2. Other (Specify_________) Amount of water in liters ________ divide liters by household size _________ (per person per day) 306 How much liter of water does your household use for food cooking? (Ask it by using the container the household they use to collect water and convert to the liter) _____________________ 307 How much liter of water does the household use for body washing (Ask it by using the container the use to collect water and convert to the liter) _____________________ PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 48 308 Who usually goes to this source to 1 = Adult Woman fetch the water for your 2 = Adult Man household? 3 = Female Child Under 15 Years Old 4 = Male Child Under 15 Years Old Note: All that apply 5 = Other (Specify)____________________ 309 Do you do anything to the water to make it safer to drink? 0 = No (Go To Q401) 1 = Yes 9 = Don’t Know (Go To Q401) 310 What do you usually do to make the water safer to drink? ANYTHING ELSE? NOTE all that apply 1 = Boil 2 = Add Bleach/Chlorine/ Water Guard/Pure/ Bishan Gari/Aquatabs 3 = Strain Through A Cloth 4 = Bio Sand /Composite/Ceramic Pot Filter 5 = Solar Disinfection 6 = Let It Stand and Settle 7 = Don't Know 8 = Other (Specify) __________________ 311 Where do you get the materials 1 = Retail outlets / Shops you use to make the water safer to 2 = Pharmacies / Drug outlets drink? 3 = WASH Business centers 4 = Health centers 5 = Other sources 312 Observe the water storage container a) The water container is clean b) The water container has narrow neck and protection cover c) The container has a tap or narrow mouth for drawing the water or Water storage container is 1 = Narrow necked (<=3 inches) 1=Yes 0 = No 2 = Covered with lid 1= Yes 0 =No 3 = Kept clean (inner and outer surface) 1 =Yes 0= No 4 = Placed off-floor (<=30 cm above the ground) 1 =Yes 0 =No 5 = Has no any cracks (no leakage) 1 =Yes 0 =No 6 = Pour water by tilting 1 =Yes 0 =No PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 49 another safe way of scooping water from the container SECTION 4: ACCESS TO SANITATION AND HANDWASHING BEHAVIORS 401 Is there a latrine or other toilet facility that member of your household use? 0 = No 1 = Yes (Go To 403) 402 If your answer to Q401 is “No”, why not? Then go to 408-410 Probe the respondent to memorize and don’t read it, if the respondent gives response out of listed options, write it on others. 1 = Not able to afford 2 = No space to construct 3 = Difficult to acquire appropriate materials 4 = Not our land / no permission 5 = Soil structure or other technical constraint 6 = Not seen as a priority by head of household 7 = Other (Specify): ________________ PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 50 403 What type of toilet facility does the household have? If the respondent does not understand which type of toilet they have, ask to see the toilet facility and indicate the appropriate code. FLUSH OR POUR-FLUSH TOILET Flush to piped sewer system ………….…………11 Flush to septic tank………….………….……….12 Flush to pit latrine………………………….……13 Flush to soak away pit………………………….. 14 Flush to somewhere …………….……………….15 Flush, don't know where………………………....16 PIT LATRINE Ventilated improved pit latrine (VIP) ....………...21 Pit latrine with plastic &/or concrete slab………22 Pit latrine w/plastic &/or concrete slab & closing lid ……………………………………… 23 Pit latrine with self-closing/sealing …………….. 24 Pit latrine riprap/mud plastered floor ………….25 Pit latrine without slab/open pit…………………26 Composting toilet…………………………..........31 Bucket toilet ………………………………..........41 Hanging toilet/hanging latrine……………………51 No facility/bush/field……………………….…….61 Other (specify)………………… ……………….98 _________________________ 404 Where is the toilet facility located? 1 = In Own Dwelling 2 = In Own Yard/Plot 3 = Elsewhere 405 Do you share this toilet facility with other households that are not part of your family? 0 = No (Go To Q407) 1 = Yes 406 If your answer is yes to QQ 405, Including your own household how many households use this toilet facility? No. of Households 0 If Less Than 10 . . . . . . . . 10 Or More Households . . . . . . . . . . . . . . . . 95 Don't Know . . . . . . . . . . . . .. . . . . . . . . . . . . . 99 PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 51 407 If observation to the HH latrine (Q403) shows the latrine slab is not washable, ask the respondent why they did not make it washable? Note: All that apply Probe the respondent to memorize and don’t read it, if the respondent gives response out of listed options, write it on others. 1 = Sanitation Product supply outlet is not accessible 2 = Not heard or seen improved sanitation product 3 = Not know where to buy it 4 = I have no disposable cash (money) 5 = I am satisfied with existing unimproved toilet (no need to change the slab and no bother for feces) 6 = I can see added value of improving the floor 7 = Slab is not a priority expenditure 8 = Other (Specify): ___________ 408 How many people within your household regularly use the bush / field at home? Number Of People Do Not Know …………………………………….………… 99 No Response ………………………………. 98 409 Why do they prefer to use the 1 = Latrine does not provide privacy bush? 2 = Bush is easier to access Probe the respondent to memorize 3 = Bush is more comfortable/ personal preference and don’t read it, if the respondent 4 = Consider the latrine as dirty gives response out of listed 5 = No access to a latrine options, write it on others. 6 = Other (_____________________________) 410 How many people within your household regularly use the bush / field at work? Number Of People Do Not Know ………………… 99 No Response ………………………….……. 98 411 Has anyone ever visited your home 0 = No to talk with you or another 1 = Yes member of your household about toilet facilities and open defecation? 2 = Unsure PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 52 412 If your answer is yes to Q 411, who visited your home? Note: All that apply 1 = Sales Agent 2 = Kebele Leaders 3 = Health Workers 4 = Health Development Army Leader 5 = NGO workers 6 = WDLA 7 = Other(specify)___________ 413A Whose decision was it to construct the toilet facility? 1 = Man of the household 2 = Woman of the household 3= Other family members like children 3 = Joint decision 4 = Other (specify)___________ 413B At that time whether it is decided by you or other family member, why you/family member decided to construct a toilet facility? Probe the respondent to memorize and don’t read it, if the respondent gives response out of listed options, write it on others. 1= We understand the advantage of having latrine 2=The health workers enforced us to construct 3= The Kebele leader enforced to us to construct 4=I don’t know the reason 5=Others (Specify_____________________) 414A Who constructed the toilet? 1 = Self / Family 2 = Private company 3 = Local Mason 4 = Government support 5 = Other Community / Charity Support 6 = Neighbors / Community Members 7 = Other (specify_____________) 414B When was the toilet constructed? 1 = In last 12 months/year 2 = In last ___ years 3 = Not sure 415A Have you upgraded the latrine recently? 0 = No [Go to 415 C] 1 = Yes PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 53 415B If your answer is yes to Q415, when did you upgrade your latrine? 1 = Last year 2 = Over a year ago 415C What additional benefits did you get after upgrading/ changing the latrine slab with washable materials? Note: All that apply The slab (material) is 1 = Comfortable and Attractive to use 2 = Easier to clean/wash 3 = Easy to transport (size, weight, etc.) 4 = It is durable (floor not decayed or damaged by termite) 5 = Not expensive (affordable) 6 = It can be reused (transferable) 7 = Floor no more muddy 8 = Safe to use 8 = Prevents bad odor 9 = Prevents fly nuisance 10 = Sales outlet accessible 11 = Other (Specify _________________) 415D If your answer is yes to Q 415A, why you were not able to renovate it? Note: All that apply 1 = It is not important to change the existing latrine, no problem with it 2 = It is expensive to renovate 3 = I have no disposable cash to buy 4 = Concrete slab is heavy (not suitable) to transport 5 =Fear of pit collapse if I put concrete slab on floor 6 = I am looking for free distribution of the slab (subsidy) 7 = I have other spending priorities (food, cloth, food etc.) 8 = Other (specify ______________ 416 Did you have another toilet facility before the one that members of your household currently use? 0 = No (Skip to 418) 1 = Yes 2 = Unsure (Skip to 418) PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 54 417 What type of facility did you have before the current one? Flush Or Pour Flush Toilet Flush to the piped sewer system…………………11 Flush to septic tank………………………………12 Flush to pit latrine……………………………… 13 Flush to soak pit …………………………………14 Flush to somewhere else……….………… ……15 Flush, don't know where…………….………… 16 Pit Latrine Ventilated improved pit latrine (VIP) ……………21 Pit latrine with plastic/concrete slab…………… 22 Pit latrine with plastic/concrete slab and closing lid ……………………………………… 23 Pit latrine with self-closing/sealing ……………… 24 Pit latrine with riprap /mud plastered floor…… 25 Pit latrine without slab/open pit…………………26 Composting toilet……………………………… 31 Bucket toilet…………………………………… 41 Hanging toilet/hanging latrine……………………51 No facility/bush/field…………………………………61 Other (specify) ………………………………… 98 PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 55 418 Enumerator Should Ask to See the Latrine Observe Latrine for Technology OBSERVATION 1 = Sato pan 2 = Aim plastic slab 3 = Concrete slab 4 = Sato retrofit to concrete slab 5 = Other (Specify_____________) Not Observed Not In Dwelling/Yard/Plot …97 GO Not Observed TO Q420 No Permission To See ………98 Not Observed, Other Reason…………………99 419A Latrine use and Cleanliness Enumerator to Observe Observe also the presence of feces in the compound Tick All That Apply OBSERVATION to proxy indicators for hygienic use of latrine [Multiple response] Latrine appears to be unused 1 Yes 0 No Existence of cover to the hole 1 Yes 0 No Presence of Feces on the Slab 1 Yes 0 No Feces Smear on Squat-Hole / Hole Cover 1 Yes 0No Fresh foot path leading to the latrine 1 Yes 0 No Fresh Feces in the Pit 1 Yes 0 No Fecal Sludge 50 cm below the Slab 1 Yes 0 No Presence of feces in the compound 1 Yes 0 No PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 56 419B Does the latrine have a wall? a) No walls 1= Yes 0= No b) Stone or brick or cement 1= Yes 0= No c) Iron sheet 1= Yes 0= No d) Timber 1= Yes 0= No e) Mud 1= Yes 0= No f) Bamboo or sticks 1= Yes 0= No 1= Yes 0= No g) Tarpaulin 1= Yes 0= No h) Other i) Door/Curtain 1= Yes 0= No 419C Does the toilet have a roofing? a) No roof 1= Yes 0= No b) Stone or brick or 1= Yes 0= No cement 1= Yes 0= No c) Iron sheet 1= Yes 0= No d) Timber 1= Yes 0= No e) Wood 1= Yes 0= No 1= Yes 0= No f) Mud 1= Yes 0= No g) Tarpaulin h) Straw/Bamboo/Grass i) Other 1= Yes 0= No 420 What do you do when the pit is full? 1 = Abandon/change the pit 2 = Prepare a new pit, move the slab and superstructure 3 = Empty the pit ourselves 4 = Get help from members of the community to empty 5 = Hire a person in the community to empty 6 = Hire a pit emptier 7 = Other (Specify): ______________ PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 57 420A Did the pit of your latrine fill before? 1=Yes 0=No (If no, skip to 421) 420B If the answer of the respondent to Q420A is yes, then ask what respondent did? 1 = Changed the pit 2 = Prepared this pit and moved the slab to here 3 = Emptied the pit by ourselves 4 = Got help from members of the community to empty 5 = Hired a pit emptier 6 = Other (Specify ______________) 421 Do you have hand-washing facility where members of your household most often wash their hands? 0 = No (GO TO Q423) 1 = Yes 422 Please show me the handwashing place. Observed ……………..………1 Not Observed Not In Dwelling/Yard/Plot ……2 GO TO Not Observed Q424C No Permission To See ….……3 Not Observed, Other Reason ………………..4 423 Observation Only: Observe presence of water at the specific place for hand washing 1 = Water Is Available 0 = Water Is Not Available 424A Observation Only: Observe Presence of Soap (Bar, Liquid, Powder, Paste) 1 = Soap Or Detergent 2 = Ash, Mud, Sand 3 = None PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 58 424B Observation only: 1=Fixed facility observed (sink/tap) in dwelling Place where members of your 2=Fixed facility observed (sink/tap) in compound HH most often wash their 3=Bucket or Jug near latrine hands? 4=Bucket or Jug near in Household 5=No Hand washing place in dwelling 424C With what did you wash your 1=Water and soap hands? 2= Water and ash/ sand 3= Only water 4= Other (Specify__________________) 425 On which occasions did you wash 1 = Before eating your hands? 2 = Before food handling/preparation Please mention all the moments 3 = Before feeding a small child you remember. 4 = After latrine use/visit Probe: "Think about all the 5 = After cleaning child bottom/disposing child feces different things you did in the past 6 = After cleaning house two days. Are there any other 7 = Before Prayer moments before or after which 8 = Early in the morning you washed your hands?" 9 = After touching dirt Do not read the pre-defined 10 = After work answers Tick All That Apply 11= Other (Specify): ________________ 425A Why you wash your hand at this occasion? Probe the respondent to memorize and don’t read it 1=Health workers taught us to do it 2= It is our culture 3=Our religion order us to do it 4=It prevents the transmission of communicable disease 5= I don’t now 6=Others (Specify): __________________________ 426 Was any money paid to construct the toilet facility that members of your household currently use? 0 = No [If No, Go To 433] 1 = Yes 2 = Unsure PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 59 427 If your answer is Yes to Q 426, What was the total cost spent to construct the latrine? Enter Amount BIRR ______ 428 How much was paid by your household? Enter Amount BIRR ______ 429 Was this a difficult amount for your household to pay? 1 = Yes 0 = No 430 Was any money paid by another entity or organization? 0 = No [If No, Go To 433] 1 = Yes 2 = Unsure 431 Who else paid aside from your household? 1 = Relative 2 = Edir 3 = PSNP 4 = Other (Specify____________________) 432 How much did they pay? Enter Amount BIRR _____________ 433 Does your household have access to financial support or a loan from financial institutions to construct a latrine, repair a latrine, or purchase hygiene products? TICK ALL THAT APPLY 0 = No 1 = Yes, 0 = No To Construct A Latrine 1 = Yes, 0 = No To Upgrade Or Repair A Latrine 1 = Yes, 0 = No To Purchase Hygiene Products 9 = Don’t Know (Go To Q437) 434 If your household is not accessible to financial support or a loan from financial institutions to construct a latrine, repair a latrine, or to purchase hygiene products, why not accessible? 1=Lack of interest 2= Not Affordable 3= I can't get loan 4= Taking loan is forbidden by our religion 5 = Fear of inability to repay the loan 6= Other (Specify):_______________ PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 60 435 If your answer is Yes to Q433, 1 = Bank from which financial institution did 2 = Micro finance institution, specify _______ you receive the financial 3 = Village saving and loan associations (VSLA) support/loan? 4 = Sanitation services provider, specify ______ 5 = Other, specify____________________ 436 For what purpose did you use the loan/financial support? TICK ALL THAT APPLY 1 = To dig the pit 2 = To line latrine pit 3 = To purchase the latrine slab 4 = To build the superstructure (wall, door and/or roofing) 5 = To repair/upgrade the latrine 6 = To purchase Hygiene materials 7= Treatment of illness 8 = Other (Specify): ____________ 436A Where did you purchase the 1 = Retail Outlet materials for the pit, the slab, and 2 = Direct from mason other parts of the sub-structure? 3 = Health Centers Probe: More than one answer is 4 = WASH Business Centers possible 5 = Sales Agent 6 = Other (Specify): ____________ 437 Did you or someone else install the 0 = No slab or its parts that you did purchase (hole cover = SATO pan and/or SATO stool)? 1 = Yes PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 61 437A If your answer is NO to Q 437, why not installed? TICK ALL THAT APPLY 1 = Lack of installation skills 2 = Lack of access to installation services 3 = Shortage of disposable cash (money) to pay for the mason/carpenter 4 = Inappropriateness of floor materials to install 5 = Existing toilet is about to fill and wait to install it on the new toilet, 6 = Doubt on the strength of the slab 7 = Lack of money to pay for installation, 8 = Other (Specify): _________ 437B Where did you purchase / procure 1 = Local materials used – no purchase required the materials for the 2 = Retail Outlet superstructure (above ground)? 3 = Direct from mason 4 = Health Centers 5 = WASH Business Centers 6 = Sales Agent 7 = Multiple sources 439 How easy is it to transport 1 = Easy to organize sanitation construction products 2 = Somewhat easy from the market to construct or 3 = Somewhat difficult repair the latrine? 4 = Difficult 440 Is there repairing of the latrine if it is needed? 0= No – If no, skip to 447 1= Yes 441 How would you repair the latrine if 1 = Self repair there was damage? 2 = Will hire a local mason 3 = Will bring in a construction company 4 = Will ask for help from members of the community 5 = Other 442 What parts of the latrine most 1 = No repairs required to date frequently require repair? 2 = Collapse / sub-structure / pit 3 = Slab 4 = Superstructure PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 62 443 Is it easier, more difficult, or the same to find help to repair the latrine today compared to 5 years ago, or when you constructed the latrine (if less than 5 years ago)? 1 = Easier 2 = More difficult 3 = About the same 4 = Unsure 444 Is it easier, more difficult, or the same to find parts to repair the latrine if there is damage? 1 = Easier 2 = More difficult 3 = About the same 4 = Unsure 445 How much would it cost to repair: Probe: Ask for each option 1 = Sub structure/ pit BIRR ______ 2 = Slab BIRR _________ 3 = Superstructure BIRR _________ 4 = Don’t know 446 Is the repairing cost affordable for you? 0 = No 1 = Yes 447 Can everyone in your household access the latrine? 0 = No 1 = Yes [GO to 449] 448 If your answer is NO to Q 447, why not? 1 = People living with disability: Latrine is too far 2 = People living with disability – Latrine is not appropriate for them to use 3 = Small children not accessing 4 = Lack of sufficient privacy 5 = Other (Specify________________) 449 Would you tell me at where children were defecating in the last two weeks? 1= Latrine 2= Not at latrine, but around the home on the floor 3= They went to the bush/field 4= They were using baby toilet seat(pan) 5= I am not sure 6= I don’t know 7= Others (Specify): ___________ PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 63 449A If the children are not using latrine, what methods, if any, does your household use to dispose of children's waste? Do not read the possible answers out loud. Probe: Multiple response is possible 1 = No children under age 5 in the household 2 = Children use a latrine / toilet 3 = Bury waste in field / yard 4 = Dispose of waste in latrine / toilet 5 = Dispose of waste with rubbish / garbage 6 = Dispose of waste with wastewater 7 = Use it as manure 8 = Burn it 97 = Don’t know 98 = No response 99 = Other (Specify): _______________ 450 What do you see as the main benefits of having your own improved latrine? PROBE 2 TIMES AND TICK ALL THAT APPLY 1. Prevent flies 2. Contains/bury feces 3. Prevents environmental contamination 4. Prevents diseases 5. Convenient and comfortable to use as needed 6. Provides privacy 7. Other (Specify): __________________ 451 What are your main sources of information about sanitation and hygiene? PROBE 2 TIMES AND TICK ALL THAT APPLY 1. Health Extension Workers 2. Sales Agents /Marketers 3. Health Centers 4. Sanitation/hygiene Product manufacturers 5. Sanitation Service providers 6. Health Development Armies 7. Kebele/GOT Leaders 8. Religious/cultural Leaders 9. School Children 10. Printed materials 11. Mass Media (Radio, Television spots) 12. Other (Specify): _________________ PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 64 SECTION 500: CHILD HEALTH 501 Have any children in the home had diarrhea in the last two weeks? 0 = No (Go To Q503) 1 = Yes 502 How many out of the total number of children? _______________ 503 Have you noticed a change in the frequency of diarrhea among the children in the home? 0 = No change 1 = More frequent 2 = Less frequent 504 Have any adults in the home had diarrhea in the last two weeks? 0 = No 1 = Yes 505 Have you noticed a change in the frequency of diarrhea among the adults in the home compared to previous time? Probe the respondent by specifying time like last year, two year three year 0 = No change 1 = More frequent 2 = Less frequent 506 Do you think that having toilet has an impact on the exposure to diarrhea 0 = No 1 = Yes 507 If yes to question 506, how it has impact on an exposure to diarrhea? Probe: Multiple response is possible 1= It reduces an exposure to diarrheal diseases 2=It reduces frequency of diarrhea occurrence 3= It reduces an exposure different kinds of disease 4=Other (Specify): ______________________ PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 65 SECTION 600: GENDER NORMS & PARTICIPATION 601 Who usually decides how the man of the house’s earnings will be used: you, your husband/partner, or you and your husband/partner jointly? 1 = Man Of The House 2 = Woman Of The House 3 = Joint Decision 4 = Husband/Partner Has No Earnings 5 = Other (Specify): __________________ 602 Who usually decides how the woman of the house’s earning’s will be used: you, your husband/partner, or you and your husband/partner jointly? 1 = Man Of The House 2 = Woman Of The House 3 = Joint Decision 4 = Husband/Partner Has No Earnings 5 = Other (Specify): ________________ 603 Who usually makes decisions about making major household purchases? 1 = Man Of The House 2 = Woman Of The House 3 = Joint Decision 4 = Someone Else 5 = Other (Specify): _______________ 604 Who usually makes decisions about purchasing water, sanitation and hygiene products for your household? 1 = Man Of The House 2 = Woman Of The House 3 = Joint Decision 4 = Someone Else 5 = Other (Specify): _________________ 605 Who is mainly responsible to repair latrine? 1 = Man Of The House 2 = Woman Of The House 3 = Joint Decision 4 = Husband/Partner Has No Earnings 5 = Other (Specify): ________ 606 If it is women of the house, why? 1 = It is a responsibility given to them 2 = Because most of the time they are around home 3 = Any other reason_______________ 4 = I don’t know PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 66 606 For women respondents only: The toilet is located in a place where I feel 1 = Strongly Agree 2 = Agree safe and dignified to use it even during the 3 = Disagree evening. 4 = Strongly Disagree Thank you for your time. Do you have any questions for me? INTERVIEWER NOTES (USE BACK OF PAGE) PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 67 C-3.0 FOCUS GROUP DISCUSSION (FGD) GUIDE Goal The objective of the FGD is to: 1. Understand how the woreda OWNP structure supports the Transform: WASH MBS approach to improve basic sanitation at the household level. (Weak OWNP structure is one of the main challenges of the enabling environment.) 2. Understand how the Transform: WASH MBS approach improved intervention woredas’ basic sanitation coverage at the household level. 3. Understand how different sanitation products are produced and accessed by households, their affordability, and the relationship among the microfinance institutions, small and medium enterprises, and households (HHs). Facilitation The focus group will be facilitated by one evaluation team member while the other evaluation team member takes notes. The facilitator’s role is to lead the discussion as informally as possible, orienting them through an open and fairly general discussion on people’s knowledge, awareness and priorities with regard to investing in improved sanitation and the use of different sanitation products. S/he will guide the discussion, ask focused questions, and draw out views from all the participants. The evaluation team should note the contributions of each participant as carefully as possible. Participants Focus groups will consist of five to seven people. Selection of participants will be done randomly. However, the below criteria will be applied in the selection of participants and/or discussion with community members: • Female: mothers with an under-five child • Male: only heads of household will be selected • Female-headed households • People with disability Three focus group discussions will be held in the 12 Transform: WASH intervention woredas and four control woredas: • Woreda WWT members (Health, Water, Education, Women Affairs) • HHs mobilized and adopting improved sanitation • HHs mobilized and but not adopting improved sanitation Conduct The facilitator will set expectations at the beginning of the session. S/he will make it clear that there are no right or wrong answers. Participants will be given numbered labels, and everyone will be asked to address one another using numbers rather than names. Questions will be asked, and participants will be given enough time to think before they answer. The facilitator will check with other participants if there is a consensus or disagreement in viewpoints. Considering the contextual cultural, social and religious situations at the community level, the facilitator should ensure usage of gender sensitive and inclusive language to ensure inclusiveness among FGD participants as well as encourage women and girls active participation. The discussion will be recorded, PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 68 and the assistant facilitator will take detailed notes of the discussion. Discussion time will be around one hour. Data Analysis Discussion records will be transcribed and stripped of non-essential words. Comments will be assigned to each participant, according to their label. An Excel database will be prepared for each group presenting the questions and answers. Answers will then be grouped into common categories in order to extract consensus. Powerful quotes will be identified. Data analysis will be synthesized by category, and short paragraphs will be written summarizing findings. C-3.1 QUESTIONS FOR FGD WITH WOREDA WASH TEAM/WOREDA TECHNICAL TEAM 1. What capacity-strengthening support has Transform: WASH provided to the woreda level? Does this capacity-strengthening support include a GEFE component? 2. What changes have you observed at the regional and woreda levels due to the Transform: WASH capacity-support program? (Implementation capacity/institutional capacity). 3. Does the woreda OWNP structure function well (presence, conducting regular meeting, and staff turnover)? If so, why? If not, why not? 4. Does the OWNP structure have the capacity to influence WASH-related decision making? 5. How effective has the regional Multi-Stakeholder Learning Platform been in supporting implementation of the national market-based sanitation implementation guideline? 6. What has the program put in place to sustain the operationalization of OWNP in the woredas? 7. Do you have any recommendations for the coordination among government, private sector, actors, consortium organizations, and other WASH sector actors? 8. Has your woreda developed a consolidated WASH strategic vision/plan? Is MBS well addressed in the strategy? 9. Has your woreda produce a consolidated OWNP WASH report? Does the report include MBS performance? 10. How is the ISI being used in your woreda and what is the local government’s reflection on its value? Do you think ISI helped the local government to see their weakness and strength? 11. What is the role of woreda Job Opportunity Creations agency in strengthening WASH businesses? 12. What is the role of the woreda sector bureaus in promoting WASH products and services as part of their standard activities and routines? 13. How would you consider gender equality and female empowerment (GEFE) in all Transform: WASH annual planning? Is there any planning checklist? How about GEFE responsive budgeting practices? 14. What approaches are you using to measure/monitor the achievement of planned activities? Do you have any GEFE related indicators? PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 69 15. What challenges have you faced, if any, while supporting implementation of gender equality and female empowerment activities under Transform: WASH? What lessons have you learned from these challenges that you faced related to GEFE? 16. What suggestions do you have to improve/strengthen GEFE approaches and interventions of Transform: WASH? 17. Have you seen changes in child mortality in your woreda in the last two years? Do you think there is a connection to the Transform: WASH for any changes you have seen? (Please check data if available). C-3.2 QUESTIONS FOR FGD WITH HHS—NON-ADOPTERS OF IMPROVED SANITATION Prior to engaging community members in the FGD, the facilitators should conduct a quick village walk to observe the following: FACILITATOR OBSERVATIONS FOR FGD: NON -ADOPTERS QUESTION ANSWER OBSERVATIONS What is the general poverty/wealth level of the community? Do households appear to have televisions, modes of transport, and cement structures? What is the general accessibility of the community? (Note ease of access by transport). Are there kiosks or local retail outlets that sell hygiene products, construction products, and related products? Yes/No Are there any shops/WASH business centers that sell WASH products in the area? If yes, are these accessible for transporting products? Are they visible to buyers? Yes/No Does the general environment of the village look clean (e.g., solid waste management, and roaming animals)? Yes/No Are there signs of open defecation/human feces lying around the village? Yes/No 1. Have you ever had health-education messages about latrine construction and its use in your village? Do you know who disseminated these messages? 2. Do you know anyone (enterprise) who does latrine construction? 3. Are sanitation products and services accessible in your area? 4. Can you construct or install a latrine by yourself? 5. When you want to build an improved toilet, do you know how to do it or who can do it for you? 6. Do you know how much an improved toilet costs? How much will it cost you if you prefer to construct by yourself? 7. Do you think that most families can afford to build an improved latrine? Is it expensive? PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 70 8. What is the main constraint for families without household toilet to install one? 9. Are you willing to borrow money for latrine investment? 10. Are you willing to purchase latrine on credit basis? 11. Have you ever tried to access money from financers for WASH investment? 12. Which financing institution lends money for WASH facility construction? 13. Have members of your community made a specific plan to stop defecating in the open? 14. Has your community achieved this goal? 15. Have you ever seen promotional materials on latrine products/latrines that inspired you to have one for your family? 16. Who usually makes decisions about purchasing water, sanitation and hygiene products for your household? 17. Do you believe women, girls, elderly people, and persons with disabilities (PWDs) have unique needs of improved sanitation services? If yes, how? 18. Have you seen changes in child mortality in your woreda in the last two years? Do you think there is a connection to the Transform: WASH for any changes you have seen? C-3.3 QUESTIONS FOR ADOPTERS Prior to engaging community members in the FGD, the facilitators should conduct a quick village walk to observe the following: FACILITATOR OBSERVATIONS FOR FGD: ADOPTERS QUESTION ANSWER OBSERVATIONS What is the general poverty/wealth level of the community? Do households appear to have televisions, modes of transport, and cement structure? What is the general accessibility of the community? (Note ease of access by transport) Are there kiosks or local retail outlets that sell hygiene products, construction products, and related products? Yes/No Does the general environment of the village look clean (solid waste management, roaming animals, etc.)? Yes/No Are there signs of open defecation/human feces lying around the village? Yes/No 1. What inspired you to invest in constructing or improving your latrine? 2. Where did you buy? Who constructed it to you? 3. How easy is it to transport sanitation construction products from the market to construct or repair the latrine? PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 71 4. Now is it easier or more difficult, or the same as in the past to find help to repair the latrine? 5. Now is it easier, more difficult, or the same as in the past to find parts to repair the latrine if there is damage? 6. Do you know who sets the price of different components (pit excavation, pit lining, the slab, slab installation, and construction of superstructure)? 7. Have you had any financial support (loan) for latrine building? Who supported you? 8. Do you think it is possible to easily access WASH products and services in your village? 9. How do you assess costs to construct or repair /upgrade your HH latrine? and for your community at large? Do you think it is expensive to construct /upgrade latrine for the community at large? 10. What possible solution would you recommend to help your village in having improved latrine? Have you ever seen a promotional campaign which aspires /motivate your community to invest on latrine? Who conducted it? What was it about? Are these campaigns considered inclusive of all community members (women, female-headed households, PWDs, etc.)? 11. How do you observe (assess) hygiene and health status of your family members after your household started using improved sanitation facility/toilet? Mention benefits that your family has gained: _______ Has the overall hygiene, sanitation and health of your family members including female, children, elderly people and PWDs (if there are any) improved after you started using improved sanitation? If yes, please give examples. If no, why? 12. Have you made the decision to buy improved sanitation products/service? Was it in consultation with your wife/husband or you made the decision alone by yourself? 13. Do you think the availability of improved sanitation service at your household has reduced the workload of women and girls in the household? Please provide details. 14. Have you seen reduction in deaths among children under-five years of age in your village/locality woreda in the last two years? Mention how this reduction has come about? Do you think there is a connection to the Transform: WASH for any changes you have seen? PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 72 ANNEX D: HOUSEHOLD SURVEY FINDINGS TRANSFORM WASH PERFORMANCE EVALUATION HOUSEHOLD SURVEY Document submitted to WASHPaLS #2 by PRIN International Consultancy & Research Services. Core Research Team: Dr. Dessalegn Tamiru, Abel Weldetinsae, Dr. Tilaye Kassahun, and Ermias Kibreab (PhDC). PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 73 ACRONYMS AND ABBREVIATIONS AOR Adjusted Odds Ratio COR Crude Odds Ratio FGD Focus Group Discussion GPS Geospatial Positioning System HH Household JMP Joint Monitoring Programme MBS Market-Based Sanitation ODK Open Data Kit SNNP South Nations Nationalities and Peoples SNNPR South Nations Nationalities and Peoples Region USAID United States Agency for International Development WASH Water, Sanitation, and Hygiene WASHPaLS Water, Sanitation, and Hygiene Partnerships and Learning for Sustainability PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 74 EXECUTIVE SUMMARY Introduction The USAID Transform WASH Activity aims to end preventable maternal and child death in Ethiopia. It is being implemented in nine regions and focuses particularly on interventions that contribute to the Transform WASH goal to reduce preventable death and illness in Ethiopia due to diarrheal disease, particularly among children under five years of age. Given that Transform WASH is now nearing its completion, USAID Ethiopia has requested that the USAID Water, Sanitation, and Hygiene Partnerships and Learning for Sustainability #2 (WASHPaLS #2) conduct a performance evaluation of the activity. The team selected PRIN International Consultancy to design, conduct, and analyze data from a household survey conducted across several intervention and non-intervention areas. Study Design and Analysis The PRIN team employed a mixed method, cross-sectional study design using a structured household survey questionnaire, document review, and key informant interviews. The team conducted the study from May to August 2022 in six Transform target regions of Ethiopia (South Nations, Nationalities, and Peoples [SNNP], Oromia, Somali, Amhara, Afar, and Sidama regions). A total of 3,076 study participants were randomly selected and included in the study, proportionally allocated to each selected woreda. The evaluation team used a structured and pre-tested survey questionnaire to collect data for the performance evaluation in areas where the project was conducting sanitation (and related water and hygiene) interventions as well as in those where it does not operate (non-intervention woredas). The team collected data through the Kobo Toolbox platform and exported it to SPSS version 20 for analysis. To filter out extreme values or outliers, the team carried out appropriate data cleaning and computed descriptive statistics, such as mean, frequencies, and percentages. They also conducted binary logistic regression and chi-squared tests to measure association between explanatory and dependent variables at p < 0.05. Data Presentation and Analysis The PRIN team collected data from 3,076 study participants, 2,308 participants from Transform WASH intervention areas and 768 from non-intervention areas. The response rate was 99.9%. The average family size (±SD) of study participants was 5.69 ± 2.62, with a range of 1 to 22. About one-third of households from Transform WASH intervention (33%) and non-intervention areas (33.3%) had children under five years of age. Agriculture was the main source of income for participants drawn from the Transform WASH intervention (65.2%) and non-intervention (87%) areas. Regarding access to water supply, 2,447 (79.6%) households had access to an improved water source (basic and limited water services). Comparatively speaking, more households in the intervention woredas (37.7%) have achieved the basic level on the water service ladder than the non-intervention woredas (27.7%). Regionally, the households in the SNNP Region had the highest access to improved water sources (91.5%) while those in the Somali region had the least access (56.9%). Relatively, more households from non-intervention areas (17.1%) had access to more than 25 liters (L) per day than those households from Transform WASH intervention woredas (10.8%) (p < 0.005). In total, 340 (11.06%) households treated their water; households from Transform WASH woredas more properly handled and stored water than households from non-intervention woredas (P<0.002). A large number (70.9%) of households possessed latrines, of which 16.1% shared their latrine with others. Having a toilet was positively associated with working for payment (AOR = 1.70 [1.20, 2.40]). However, it was negatively associated with poor perception of the toilet’s role in prevention of diarrheal disease (AOR = 0.69 [0.54, 0.8]), being a male headed household (AOR = 0.67[0.49, 0.92]) and low PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 75 economic status (AOR = 0.04 [0.02, 0.08]). Location in a Transform WASH intervention area was not significantly associated with having a toilet facility (p > 0.05). More than three-fourths (76.7%) of families had access to a toilet. The main challenges to latrine accessibility were distance from home (9.8%), toilet’s unsuitability for disabled people (12.4%), incompatibility for small children (87.8%), and lack of privacy (4.7%). Having a toilet was not uniform throughout the regions, with Sidama (96.3%) scoring highest and Somali (24%) region scoring the lowest in number of households utilizing a toilet. In total, 993 (45.5%) study participants in the study area used pit latrine without a slab or open pit, whereas 662 (41.71%) households from intervention areas and 331 (55.82) households from non-intervention areas used a pit latrine without a slab/open pit (p < 0.001). More than one-third (38.1%) of respondents from Transform WASH intervention areas and 102 (30.9%) from non-WASH intervention areas reported the lack of a toilet as the main reason to carry out open defecation (p < 0.01). Only 589 (19.2%) households had access to basic sanitation services, and 46.9% of households only able to access unimproved sanitation services. More than one-fifth of households (22.62%) could not construct a latrine due to lack of deposited cash, lack of space, difficulty acquiring materials, and technical challenges. Though the difference was not significant (p > 0.05), 74.3% of households from Transform WASH intervention areas and 73.2% of households from non-intervention areas had no money to buy sanitation products. More than one-fifth of households (22%) from Transform WASH intervention areas and 109 (22.6%) households from non-interventions areas did not have access to sanitation products (p > 0.05). In total, 1,100 (62.1%) participants mentioned that the man of the household was responsible for decisions regarding toilet construction. About 13.4% of households in Transform WASH areas had recently upgraded their toilets while only 6.9% of households in non-intervention areas had done so (p < 0.001). Among those who had an upgraded toilet, longevity of the floor (8.75%), affordability (8.4%), odor prevention (22%), and fly nuisance prevention (17.8%) were reported as advantages of their improvements. More than two-thirds of respondents (68.3%) from both Transform WASH intervention areas (65.7%) and non-intervention areas (75.4%) were prepared to repair their latrine if needed (p < 0.001). In total, the 468 (21.5%) households’ slabs were smeared by feces, with almost equal proportions seen between Transform WASH intervention areas (338, 21.3%) and non-intervention areas (130, 21.9%) (p = 0.752). A very small number of participants (2.9%) had handwashing facilities (e.g., soap and water). Among those households who had soap or detergent (5.9%), about 132 (5.7%) were from Transform WASH intervention and 49 (6.4%) were from non-intervention areas. A large proportion of households had no handwashing place location in the compound/dwelling (58.1%), but 54.8% of them reported washing their hands after toilet visit. The children of some households (11.5%) were exposed to diarrheal disease in the last two weeks, and a large proportion of households perceived having a toilet as a positive influence to reduce exposure to diarrheal and other diseases. Generally, Transform WASH interventions have contributed highly to improvement of basic sanitation and hygiene services. A meaningful difference was observed between the activity’s intervention areas and non-intervention areas on improved water and toilet utilization. However, no significant difference was observed between Transform WASH intervention and non-intervention areas on accessibility of sanitation products, and very small number of participants had access to sanitation products in the market. The majority of participants complained that sanitation products were inaccessible and expensive. Therefore, there is a need for program revision to enhance water and sanitation services as well as make affordable sanitation products available. PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 76 1.0 INTRODUCTION 1.1 BACKGROUND The USAID Transform WASH Activity aims to end preventable maternal and child death in Ethiopia. It is being implemented in nine regions and one city administration and has three principal areas of intervention: 1) primary health care, in Oromia, Amhara, Tigray, and South Nations Nationalities and Peoples (SNNP) regions; 2) health in developing regions, in Somali, Afar, Gambella, and Benishangul Gumuz; and 3) water, sanitation, and hygiene (WASH) spread across all nine regions. USAID’s Water, Sanitation, and Hygiene Partnerships and Learning for Sustainability #2 (WASHPaLS #2) project is a five-year contract that Tetra Tech implements in collaboration with partners FHI 360, FSG, IDinsight, and Iris Group. This project focuses particularly on interventions that contribute to sustainable access to sanitation services and improved hygiene behaviors in rural areas. Additionally, the project aims to generate and facilitate WASH sector research and learning that result in sustainable, at￾scale, and equitable improvements in key services, behaviors, and environmental conditions at the community and household (HH) levels. In March 2022, WASHPaLS #2 received a request from USAID/Ethiopia to carry out a performance evaluation of the Transform WASH Activity. Transform WASH is a six-year activity implemented by Population Services International with partners Netherlands Development Organization (SNV), IRC WASH, and Plan International to test and develop market-based sanitation (MBS) models that will increase demand for and supply of high-quality and affordable WASH products and services. The goal of Transform WASH is to reduce preventable death and illness in Ethiopia due to diarrheal disease, particularly among children under five years of age. The Transform WASH activity has been designed to test four business models: 1) door-to-door; 2) basic manufacturing and construction; 3) advanced manufacturing and construction; and 4) plastic slab sales and installation. Since Transform WASH ends in December 2022, USAID requested that WASHPaLS #2 conduct a performance evaluation. WASHPaLS#2 selected PRIN International Consultancy to conduct the assignment, which included data collection and production of a high-level report for a large-scale HH survey. 1.2 RATIONALE FOR THE STUDY The performance evaluation comes at the end of the Transform WASH Activity. As such, it is timed to help determine which components of each intervention worked well and why and which ones did not and why, and to suggest recommendations for future activities that employ similar strategies. Results from the Mission’s investment in the Transform WASH Activity effectiveness need to be measured and demonstrated. The overarching purpose of the final evaluation is to understand the key results in Transform WASH program intervention woredas and kebeles. To do so, the endline evaluation examined the following key areas: • Validity of the overall Transform WASH development hypothesis(es) • Effectiveness of MBS approaches (what works/did not work and why) • Sustainability of MBS approaches Beyond the “what” of Transform WASH results, the final evaluation also examined the “why,” including but not limited to factors that largely contributed to these results and factors that may have inhibited achievements. The end-line evaluation also explored unintended results of Transform WASH interventions, both positive and potentially negative. PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 77 1.3 OBJECTIVES OF THE ASSIGNMENT The main objective of the performance evaluation was to examine the validity of the overall Transform WASH development hypothesis(es), assess effectiveness and sustainability of approaches, and generate quality data for the decision-making process regarding future United States Government investment and programming. The performance evaluation sought to obtain quality evidence on the major achievements and best lessons from the Transform WASH interventions. The specific objectives of the final performance evaluation of the Transform WASH interventions were to: • Understand the effectiveness of USAID/Ethiopia’s Transform WASH approaches and strategies in relation to the activity’s stated objectives. • Estimate the broader outcomes of Transform WASH, including the overall change in sanitation markets, changes in sanitation coverage, viability of supported enterprises, ability of the market to meet the needs of the underserved, and the likely sustainability of these changes. • Identify and document key learning and recommendations to inform future programming. PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 78 - - - 2.0 METHODOLOGY 2.1 STUDY DESIGN The evaluation team employed a cross-sectional study design using mixed methods: a structured questionnaire, document review, and key informant interviews. The team was tasked with carrying out the quantitative HH survey while WASHPaLS #2 was tasked with carrying out the qualitative study. 2.2 STUDY AREA AND POPULATION The study was conducted from May to August 2022 in six Transform WASH target regions of Ethiopia (Table 1). Included in the study were two zones from the SNNP, Oromia, and Somali regions; three zones from the Amhara region; and one zone from Afar and Sidama regions. For each selected zone, the team selected woredas in which Transform WASH operates (intervention areas) and woredas in which the activity does not operate (non-intervention areas). Following the woreda selection, the team selected the lowest administrative unit (Kebele), considered a sub-community in this study, in which the surveys would be conducted. Table 1: List of areas where survey was conducted in both Transform WASH intervention and non￾intervention areas, Ethiopia, 2022 Region Zone Selected woredas Study area Remoteness Liveli hoods High/Low perform ing Consol idated WASH Account SNNPR Hadiya Mierab Bediwacho Intervention 1 More connected Agrarian Medium Yes Walaita Kindo Dedaye Intervention 2 Remote Agrarian Medium Yes Gamo Mierab Abaya Non-intervention 1 Non-adjacent Sidama Sidama Aleta Wondo Intervention 3 More connected Agrarian High Yes Oromia East Arsi Wondo Intervention 4 More connected Agrarian High No S/Mierab Shoa Woliso Intervention 5 More connected Agrarian Medium No West Arsi Dodola Intervention 6 More connected Agrarian Medium No Gedeb Assasa Non-intervention 2 Spill-over Amhara East Gojam Guzeman Intervention 7 More connected Agrarian High Yes Ali Booko Non-intervention 3 Spill over South Wollo Kalu Intervention 8 More connected Agrarian High Yes North Shoa Were Elu Intervention 9 Remote Agrarian High No Angolalla Tera Non-intervention 4 Non-adjacent Somali Dega Bour Dega Bour Intervention 10 More connected Pastoralist Medium No Gursum Gursum Intervention 11 More connected Sem￾Pastoralist Low Yes Afar Zone 3 Mille Intervention 12 More connected Pastoralist Low Yes PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 79 2.3 SAMPLE SIZE AND SAMPLING PROCEDURE The assessment team conducted a multi-stage cluster sampling technique to select the study participants. The team calculated a total sample size of 3,076 (intervention HHs = 2,308 and non￾intervention HHs = 768) using G-Power version 3.1.9.4, with an assumption of 0.2 effect size, 0.05 margin of error, 85% power (1-B error prob), allocation ratio 0.25, 9% non-response rate, and design effect of 2. Based on this total sample size, the team allocated numbers of study participants proportionally to each selected woreda. In this study, the team employed a multi-stage cluster sampling study design. First, they selected regions, zones, woredas, and kebeles. In each woreda, the team then selected six to seven kebeles (the lowest administrative unit consisting of, on average, 500 HHs) on a random sampling basis. From each kebele, the team then selected two gotts, or villages (the lowest structure unit of kebele). Finally, the team listed the number of HHs in each selected gott and selected the desired number of HHs using a simple random sampling method. From the fresh HH list, the team selected 30 from each intervention kebele and 32 from non-intervention kebeles in the three regions (Amhara, Oromia, and SNNP) and 31 HHs from each kebele of the remaining three regions (Afar, Sidama, and Somali). The selection of kebeles and HHs was made based on the principle of fair representation. Households were selected from approximately two gotts per kebele. From each HH, the team selected for an interview an eligible candidate (young or adult family member) who could give complete information about WASH within the HH.3 The HH survey also included sampling from non-intervention woredas. The team selected a total of four woredas from three Transform WASH intervention regions (Amhara, Oromia, and SNNP), two that were adjacent to Transform WASH intervention areas, and two that were not adjacent to intervention woredas. The team randomly selected kebeles from the non-Transform WASH intervention woredas of the selected regions (see Table 1). The sampling procedure for selecting the HHs in comparison woredas proceeded in the same manner as for the Transform WASH intervention woredas. A summary of the sample HHs broken down by location is presented in Table 2. Table 2: Summary of sample HHs by location from both Transform WASH intervention and non￾intervention areas, Ethiopia, 2022 Region Zone Woreda Intervention Status # Kebeles # Gotts #HHs SNNPR Hadiya Mierab Bediwacho T:WASH 1 6 12 180 Wolayta Kindo Dedaye T:WASH 2 6 12 180 Gamo Mirab Abaya Non-intervention (Non-adjacent) 6 12 192 Sidama Sidama Aleta Wondo T:WASH 3 7 14 217 Oromia East Arsi Wondo T:WASH 4 6 12 180 South West Shewa Woliso T:WASH 5 6 12 180 West Arsi Dodolla T:WASH 6 6 12 180 Gedeb Assesa Non-intervention (Adjacent) 6 12 192 Amhara East Gojjam Guzamen T:WASH 7 6 12 180 South Wollo Kalu T:WASH 8 6 12 180 Were Elu T:WASH 9 6 12 180 PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 80 3 A household consists of a person or a group of persons, related or unrelated, who live together in the same dwelling unit; who acknowledge one adult male or female as the head of household; who all share the same living and eating arrangements; and who are considered one unit. Region Zone Woreda Intervention Status # Kebeles # Gotts #HHs Ali Booko Non-intervention (Adjacent) 6 12 192 North Shewa Angollala Tera Non-intervention (Non-adjacent) 6 12 192 Somali Dega Bour Dega Bour T:WASH 10 7 14 217 Gursum Gursum T:WASH 11 7 14 217 Afar Zone 1 Mille T:WASH 12 7 14 217 Total 13 16 12+4 124 248 3,076 2.4 DATA COLLECTION TOOLS The PRIN team used a structured and pre-tested survey questionnaire to collect data for the Transform WASH Activity performance evaluation in areas where the activity operates. The team uploaded structured questionnaires to smartphones and used Kobo Toolbox (which uses an Open Data Kit [ODK]) to collect the data. The team designed a data collection form with a drag-and-drop user interface to ensure data collection and entry could be accomplished accurately and in a timely fashion. Once checked for their accuracy and completeness, finalized submissions were sent to the data server. 2.5 FIELD DATA COLLECTION PROCEDURES A team of enumerators and supervisors conducted quantitative and qualitative data collection concurrently. Each data collection team included an average of two (majority female) interviewers and a supervisor. The first task data collection teams undertook after arriving at the enumeration area or kebele was to identify local guides. The survey team, with the assistance of the guides, then identified the available number of gotts in the kebeles and listed them on the gott registration form. From these, the teams randomly selected two gotts for inclusion in the survey. The survey team then demarcated the borders of the gotts and listed all HHs using a HH listing form to establish a sampling frame. The HH listing was done at the kebele level with the help of health extension workers, local administrators, and kebele guides. The guides assisted interviewers to ensure that all HHs were covered, and field supervisors performed random checks to ensure accuracy and coverage of HHs. Once HHs were listed, the supervisor randomly selected HHs in collaboration with the concerned enumerator(s). In cases where the selected HH had more than one eligible respondent, the enumerator used a Kish Grid method to randomly select only one respondent to be interviewed. 2.6 QUALITY ASSURANCE The team deployed mechanisms to maintain study quality during the design, data collection, and post￾fieldwork stages. To assure the collection of high-quality data, the PRIN team conducted a three-day training for all enumerators and supervisors. Enumerators that administered HH surveys had at least a bachelor’s degree and were fluent in both the local language and English. They also had prior experience collecting similar data and an understanding of the culture and traditions of the communities they were visiting. Supervisors, who had at least a masters-level degree in the health or social science fields, coordinated and controlled the enumerators and gathered the qualitative data. In cases where the respondents were illiterate or less-educated adults and to ensure participants would be able to speak comfortably with interviewers, the field team was gender-balanced. The three-day training covered topics such as research ethics, rights of human subjects during research, sampling procedures, informed consent, data collection tools, interviewing techniques, data handling, security and quality, mobile data collection procedures, and gender considerations during data collection. In particular, the training has focused on: • Description of the survey: why and how it was being carried out and for what purpose the information would be used; PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 81 • Presentation of the survey objectives to ensure clear and thorough understanding on the part of the data collection teams; • Review of HH selection methods and establishing contact with respondents and maintaining cooperation; • Review of the questionnaire to confirm that the questions were appropriate and understandable to the local population; • Role play of HH selection, informed consent, questionnaire administration, and logging of visit outcomes; • Role play and practicing of questionnaire administration (on the use of tablets for data collection); and • Review of informed consent and other ethical considerations. Once the data collectors completed the training, they received instruction on how to collect data using the electronic data collection template and how to upload the collected data onto the server. Data collectors pre-tested the survey tools through mock interviews and pilot tests in the vicinity of Addis Ababa using various languages. Moreover, to ensure the quality of collected data, supervisors conducted spot-checks during the data collection and re-interviewing, especially at the beginning (the first three days) of the data collection. Supervisors reviewed samples of completed questionnaires on a daily basis before uploading them onto the server. In addition, they provided feedback to the teams on any problem identified from the review for the next day’s field work. To validate the quality of submitted data, the team used an integrated platform called ODK audit to check if the data was collected in the designated area and the required amount of time was spent to administer each question. For this, a data manager was assigned to continuously follow the quality assurance process. To ensure the proper functioning of the ODK audit, enumerators were encouraged by their supervisor or prompted by the data collection software to take two geospatial positioning system (GPS) points—one at the beginning of the interview and one when finalizing the interview. As a result, the data manager measured the distance between the two coordinates. Furthermore, to avoid unethical misconduct, enumerators were blocked from taking the second GPS and finalizing the questionnaire until they had administered all the interview questions. The teams recorded audio of all interviews with relevant key informants and transcribed and translated the interviews into English. Five percent of the transcriptions were checked against the audio file for accuracy. Finally, the team cleaned up and cross-checked the data before conducting the data analysis. 2.7 TRANSLATION AND PRE-TESTING OF QUESTIONNAIRE The team reviewed the draft questionnaire to check its comprehensiveness and contextual appropriateness. Following the approval of the English version by WASHPaLS#2, the team translated the HH questionnaire to the local languages of the study areas (i.e., Amharic, Afan Oromo, Afamboo [Afar], and Somali for the identified woredas). Then independent translators then translated the questionnaires back to English to check consistency. In addition, the team conducted a pre-test in the vicinity of Addis Ababa with a small number of respondents. The pre-test helped to ensure that interviewers and respondents understood the questions and answers and that the questionnaire worked in terms of skips, filters, and pre-coded categories. The team carried out a quick analysis of the data collected at the pre-test to check for any problems that might otherwise be overlooked. Based on the pre-test results, the team adjusted and made corrections to the contents and order of some of the questions. 2.8 MONITORING AND SUPERVISION OF FIELD DATA COLLECTION The team supervisors submitted regular reports to the WASHPaLS#2 performance evaluation team. The PRIN team developed a checklist regarding the field monitoring to use for this purpose. The enumerators reported to their immediate supervisors, and the supervisors reported to the co￾PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 82 investigators according to the hierarchy of duties and responsibilities. The supervisors organized and directed data collection and were responsible for the technical and logistical issues encountered by interviewers during data collection. The supervisors were involved in the following activities in an effort to monitor and supervise the data collection: • Draw up a fieldwork plan • Organize survey staff into teams led by a supervisor • Organize travel schedules, accommodations, and per diems for field staff • Establish systems for communication between field staff and central office • Liaise with community authorities where necessary • Organize and oversee the daily work of the interviewers • Review completed work • Provide ongoing training to maintain quality and commitment • Troubleshoot to solve problems with implementation • Implement data control procedures 2.9 DATA MANAGEMENT AND ANALYSIS The team exported the data collected through the Kobo Toolbox platform to SPSS version 20. Special arrangements were made to enforce referential integrity of the database so that all data tables were related to each other without problem. Appropriate data cleaning was carried out to filter out extreme values. After the data cleaning, categorical variables were summarized as numbers and percentages, whereas normally distributed continuous variables were presented as means and standard deviations. Descriptive statistics such as frequencies and percentages for discrete data and the mean values for continuous data was computed. Independent t-tests were done to compare means between groups. Relationships between the variables were checked by chi-squared tests and Pearson’s r. Binary and multivariable logistic regression analyses were used to assess the association between the explanatory and the outcome variable. Variables with a p value < 0.25 in bivariate logistic regression analyses were selected as candidates for the multivariable logistic regression model to control for all possible confounding effects. Crude and adjusted odds ratios along with 95% confidence intervals were used to estimate the strength of association between dependent and independent variables. The goodness of fit for the final logistic model was tested using the Hosmer and Lemeshow test at > 0.05. Multicollinearity was checked by using Variance Inflation Factor. Finally, results with a p value of < 0.05 were considered statistically significant. 2.10 ETHICAL CONSIDERATIONS In this assignment, all those who participated as consultants, supervisors, enumerators, and others were committed to protecting the rights of children, women, and others. Before starting data collection, all stakeholders at the region, zone, and woreda levels were informed of this commitment. Finally, the relevant woreda health officer wrote a letter to each kebele requesting permission for data collection. Once permission was secured from all higher officials, data collection officially started in all woredas. The primary caregiver of each household was notified of the purpose of the study and after informed consent was secured, the surveys were administered by trained enumerators. Recruitment of respondents was guided by ethical standards that obtain informed consent and ensure confidentiality. Interviewers guided respondents to help create a private place for the interview. Variability in the location and social circumstance of the interview was minimized. Questions were delivered according to the exact wording and order of the questionnaire. Privacy and confidentiality were maintained at all times, particularly when the topics were sensitive. In addition, PRIN and its staff did not violate the copyright and ownership of the study report. PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 83 3.0 DATA PRESENTATION AND ANALYSIS 3.0 SUMMARY OF RESULTS • A large proportion (79.6%) of HHs had access to an improved water source. • More HHs in the Transform WASH intervention areas (37.7%) have attained the basic level on the water service ladder than those residing in non-intervention areas (27.7%) (p < 0.001). • In total, only 12.4% of HHs were utilizing greater than 25L of water per person per day, and more HHs from non-intervention areas (17.1%) had access to more than 25L per day than HHs from Transform WASH intervention woredas (10.8%) (p < 0.004). • HHs from Transform WASH woredas (1.8%) handled and stored water more properly than HHs from non-intervention woredas (0.3%) (p < 0.002). • A large number of HHs (70.9%) had a latrine, of which 16.1% shared their latrines with others. • Having a toilet was positively associated with working for payment [IN PAID EMPLOYMENT] (AOR = 1.70 [1.20, 2.40]). However, it was negatively associated with a poor perception of a toilet’s role in preventing diarrheal morbidity (AOR = 0.69 [0.54, 0.8]) and poor economic status (AOR = 0.04 [0.02, 0.08]). • Being located in a Transform WASH intervention area was not significantly associated with whether or not a HH had a toilet (p = 0.169). • The main challenges to latrine accessibility were listed as distance from home (9.8%), toilet unsuitability for disabled people (12.4%), and incompatibility for small children (87.8%). • Having a toilet was not uniform across the region; Sidama region (96.3%) scored highest, and Somali region (24%) scored the lowest. • In total, 993 (45.5%) HHs use a pit latrine without a slab or open pit; 662 (41.71%) HHs from Transform WASH intervention areas and 331 (55.82%) HHs from non-intervention areas use a pit latrine without a slab/open pit (p < 0.001). • More than one-third (38.1%) of HHs from Transform WASH intervention areas and 102 (30.9%) from non-intervention areas reported lack of a toilet as the main reason for open defecation (p < 0.014). • Only 589 (19.2%) HHs had access to basic sanitation services, and 46.9% of HHs had access to unimproved sanitation services. • More than one-fifth of HHs (22.62%) were unable to construct a latrine due to lack of deposited cash, lack of space, difficulty in acquiring materials, and technical challenges. • A very small number (15.9%) of participants from both Transform WASH intervention (17.6%) and non-intervention areas (11.3%) had upgraded their toilets. • More than two-thirds of respondents (68.3%) from both Transform WASH intervention areas (65.7%) and non-intervention areas (75.4%) were prepared to repair their latrines if needed (p < 0.001). • The slabs of 468 (21.5%) HHs were smeared by feces, distributed almost equally between Transform WASH intervention areas 338 (21.3%) and non-intervention areas 130 (21.9%) (p > 0.05). • A large proportion of HHs had no handwashing facilities in their compound/dwelling (58.1%), but 54.8% of respondents reported washing their hands after a toilet visit. • In total, 1,100 (62.1%) participants stated that the man of the HH was responsible for decisions regarding toilet construction and only 30 (1.7%) stated that women in HHs had decision-making power on toilet construction. • In total, 1,468 (82.8%) HHs had constructed a latrine in the last five years, with more HHs from Transform WASH intervention areas constructing (84.9%) latrines than HHs from non￾intervention areas (77.6%) (p = 0.001). PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 84 - - • In the last year, HHs in the Transform WASH areas (13.4%) had upgraded their toilets more than HHs in non-intervention areas (6.9%) (p = 0.001). • Some HHs (11.5%) reported that their children had been exposed to diarrheal disease in the last two weeks, and a large proportion of HHs perceived (78.7%) having a toilet as a way to reduce exposure to diarrheal and other diseases. 3.1 SOCIO-DEMOGRAPHIC CHARACTERISTICS This community-based, cross-sectional assessment was conducted in selected areas of Ethiopia. The six regions included in the study were: 1) Amhara, 2) Oromia, 3) Afar, 4) Somali, 5) SNNP and 6) Sidama. The assessment was conducted in areas where Transform WASH operates and does not operate. In each selected region, both rural and semi-urban villages were randomly selected. Table 3 presents the distribution of background characteristics for respondents of the endline survey. Of the total 3,076 study participants, the team collected endline data from 2,308 from Transform WASH intervention woredas and 768 from non-intervention woredas.4 The response rate was 99.9%. Findings of this assessment showed that the large proportion of HHs were male-headed in both Transform WASH Intervention areas (77%) and non-intervention (80.6%) woredas. The average family size (±SD) of study participants was 5.69 ± 2.62 with a range of 1 to 22. A significant number of study participants had large families (5–8) in both intervention (55.1%) and non-intervention (45.4%) woredas. About one-third of (33%) and 256 (33.3%) HHs from Transform WASH intervention and non￾intervention woredas had children under five, respectively. Proportionally, 65.4% of respondents from Transform WASH intervention woredas and 64.4% of respondents from non-intervention woredas were females. More than three-fourths of HHs from both intervention (77.3%) and non-intervention (82%) woredas had a non-polygamous type of family. An equal proportion (48.2%) of respondents from both non-intervention and Transform WASH intervention areas were Muslim followers. 1,889 respondents (82.3%) from intervention woredas and 599 (78%) respondents from non-intervention woredas were married. The same proportion of HHs from intervention (54%) and non-intervention (54%) woredas did not have any formal education. However, among those who had received formal education, 310 (88.8%) and 872 (93.1%) respondents from intervention and non-intervention woredas, respectively, attended school from Grades 1 through 12. Agriculture was found to be the main source of HH income for both Transform WASH intervention 1,504 (65.2%) and non-intervention 668 (87%) areas. A very small number of study participants (345, 11.23%) from both intervention (233, 10.1%) and non-intervention (59, 7.7%) areas reported that they were employed by different organizations and working for payment. Table 3: Socio-demographic characteristics of study participants from Transform WASH intervention and non-intervention areas, Ethiopia, 2022 Variables Categories Intervention area (%) Non intervention woreda (%) Total (%) P value HH Head Female-Headed 530 (23%) 149 (19.4) 679 (22.1) 0.038 Male-Headed 1775 (77%) 619 (80.6) 2394 (77.9) Family size 1-4 697 (30.3) 343 (45.1) 1040 (33.8) 0.001 5-8 1266 (55.1) 345 (45.4) 1611 (52.4) 9 and above 336 (14.6) 72 (9.5) 408 (13.3) Under-five children No under-five 1032 (44.8) 387 (50.4) 1419 (46.2) 0.002 One 760 (33) 256 (33.3) 1016 (33.1) Two 385 (16.7) 97 (12.6) 482 (15.7) PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 85 4 In this report, the words “woredas” and “areas” are used interchangeably. Variables Categories Intervention - - area (%) Non intervention woreda (%) Total (%) P value Three 100 (4.3) 17 (2.2) 117 (3.8) Four and above 28 (1.2) 11 (1.4) 39 (1.3) Sex of respondent Female 1507 (65.4) 498 (64.8) 2005 (65.3) 0073 Male 798 (34.6) 270 (35.2) 1096 (35.7) Family type Non-polygamous 1782 (77.3) 630 (82) 2412 (78.5) 0.006 Polygamous 523 (22.7) 138 (18) 661 (21.5) Religion Muslim 1111 (48.2) 370 (48.2) 1481 (48.2) 0.001 Orthodox 437 (19) 260 (33.9) 697 (22.7) Protestant 728 (31.6) 138 (18) 866 (28.2) Others 29 (1.3) - 29 (0.9) Marital status Married 1889 (82.3) 599 (78) 2495 (81.2) 0.001 Separated 15 (0.7) 10 (1.3) 25 (0.8) Single 170 (7.4) 47 (6.1) 217 (7.1) Divorced 48 (2.1) 29 (3.8) 77 (2.5) Widow 176 (7.6) 83 (10.8) 259 (8.4) Types of education attended No education 1245 (54) 418 (54.4) 1663 (54.1) 0.975 Non-formal education 77 (3.3) 26 (3.4) 103 (3.4) Formal education 983 (42.6) 324 (42.2) 1307 (42.5) Highest educational status College graduate/student 8 (6.1) 60 (2.4) 68 (2.2) 0.094 Grade 1-12 310 (88.8) 872 (93.1) 1182 (38.5) TVET Graduate/student 6 (2.6) 25 (1.8) 31 (1.0) University graduate/student 9 (2.6) 25 (2.7) 34 (1.1) Primary source of HH income Farming 1504 (65.2) 668 (87) 2172 (70.7) 0.001 Own enterprise 191 (8.3) 48 (6.3) 239 (7.8) Paid labor 275 (11.9) 34 (4.4) 309 (10.1) Other 335 (14.5) 18 (2.3) 353 (11.5) Worked in paid employment Yes 282 (12.2) 63 (8.2) 345 (11.2) 0.002 No 2023 (87.8) 705 (91.8) 2728 (88.8) Currently in paid employment Yes 233 (10.1) 59 (7.7) 292 (9.5) 0.001 No 2072 (89.9) 709 (92.3) 2728 (88.8) Enrolled in Community Based Health Insurance (CBHI) Yes 554 (72.1) 1176 (51) 1730 (56.3) 0.001 No 206 (26.8) 1121 (48.6) 1327 (43.2) Unknown 8 (1) 8 (0.3) 16 (0.5) Wealth index Poorest 525 (22.8) 92 (12) 617 (20.1) 0.001 Poor 428 (18.6) 184 (24) 612 (19.9) Medium 430 (18.7) 185 (24.1) 615 (20.0) Rich 479 (20.8) 136 (17.7) 615 (20.0) Richest 443 (19.2) 171 (22.3) 614 (20.0) 3.2 HOUSEHOLD LIVING CONDITIONS A large proportion of HHs from both Transform WASH intervention areas (72%) and non-intervention areas (80.7%) had an earth/mud type of floor in their houses (p = 0.001). Though there were differences between HHs from intervention and non-intervention woredas (p = 0.001), 1,382 (60%) participants from Transform WASH intervention areas and 681 (88.7%) participants from non-intervention areas had house walls made from wood and mud. However, 1,766 (76.6%) participants from intervention woredas and 700 (91.1%) from non-intervention woredas had houses made from metal or corrugated sheets (Table 4). PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 86 - - Table 4: The household living characteristics of study participants in Transform WASH and non￾Transform WASH Activity areas, Ethiopia, 2022 Housing condition Categories Intervention woredas (%) Non intervention woredas (%) Total (%) P value House floor type Carpet 115 (5) 48 (6.3) 163 (5.3) 0.001 Cement 96 (12.5) 385 (16.7) 481 (15.7) Earth/Sand/Mud 1658 (72) 620 (80.7) 2279 (74.2) Plastic/wood/Bamboo/ Cloth/Iron 147 (6.4) 4 (0.6) 150 (4.9) House wall type Bricks/Cement/Cement Block 132 (5.7) 30 (3.9) 162 (5.3) 0.001 Cane/Palm/Trunks/ Bamboo 53 (2.3) 2 (0.0) 55 (1.8) Corrugated Sheet 52 (2.3) - 52 (1.7) Mud 149 (6.5) 6 (0.8) 155 (5.0) Wood and Mud 1382 (60) 681 (88.7) 2063 (67.1) Reused wood/wood planks/shingles 178 (7.7) 9 (1.2) 187 (6.1) Stone with lime/Cement 58 (2.5) 13 (1.7) 71 (2.3) Stone With Mud 122 (5.3) 21 (2.7) 143 (4.7) Bamboo/Adobe/Cloth/ No wall/Grass 179 (7.8) 6 (0.8) 185 (6.0) Type of house roof Metal/Corrugated Iron 1766 (76.6) 700 (91.1) 2466 (80.3) 0.001 Wood 89 (3.9) 1 (0.1) 90 (2.9) Thatch/Mud 149 (6.5) 57 (7.5) 206 (6.7) Sod 21 (0.9) 8 (1) 29 (0.9) Others 258 (11.2) 1 (0.1) 259 (8.4) No Roof 22 (1) 1 (0.1) 23 (0.8) 3.3 ACCESS TO WATER SERVICES: JMP/NATIONAL SERVICE LADDER To determine the status of HHs’ access to clean water, the study team adopted a global indicator, the WASH service ladder, designed by World Health Organization/UNICEF Joint Monitoring Programme (JMP) (Figure 1). The service ladder also aligns with the national service ladder developed by the Ethiopia Ministry of Health. As shown in Figure 1, the water service ladder is divided into basic, limited, unimproved, and surface water. More HHs in the Transform WASH intervention woredas (37.7%) have attained the basic level on the water service ladder than those of the non-intervention woredas (27.7%) (p = 0.001). Only 3% of the survey HHs have access to surface water. In total, 533 (17.3%) HHs surveyed are served from unimproved sources (unprotected well and unprotected spring combined). PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 87 Figure 1: The water service ladder in Transform WASH intervention and non-intervention areas of Ethiopia, 2022 An equal proportion of HHs between Transform WASH (2,447 or 80%) and non-intervention woredas (607 or 79%) are served with improved water sources, such as from piped, spring, borehole, tube well, or delivered water. Regionally, the SNNP region had more access to improved water sources (91.5%) than other regions. Study participants in Somali region had the least access to improved water sources (56.9%) (Figure 2). PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 88 - - Figure 2: The distribution of water service ladder at regional level in Ethiopia, 2022 Only 16% (489) of HHs have their water source in their compounds and almost no survey respondents have water sources in their dwelling. A large number (2,571 or 83.7%) of HHs were collecting water from elsewhere, with similar numbers for Transform WASH intervention woredas (83.6%) and non￾Transform WASH woredas (84%). Though the difference is not significant (p = 0.246), more than half of participants from Transform WASH intervention woredas (56.5%) and 274 (43.6%) from non-Transform WASH woredas were able to get water on average within 30 minutes. The mean and median time it took to collect water was 48.36 ± 48.417 and 30 minutes, respectively (Table 5). Table 5: The location of water and water fetching duration in Transform WASH areas and non￾Transform WASH woredas, Ethiopia, 2022 Variables Categories Intervention woreda Non intervention woreda Total p value Location of Elsewhere 1926 (83.6) 645 (84) 2571 (83.7) 0.137 water sources In own dwelling 11 (0.5) 2 (0.3) 13 (0.4) In Own Yard/Plot/ Compound 368 (16) 121 (15.8) 489 (15.9) Water fetching duration < 30 Minutes 1082 (56.5) 274 (43.6) 1356 (53.3) 0.246 31-60 Minutes 532 (27.8) 239 (38) 771 (30.3) 61> 302 (15.8) 116 (18.4) 418 (16.4) Findings of this assessment indicated that adult women were more often responsible (80.9%) for collecting water than men and youth, with similar figures for Transform WASH intervention areas (89.9%) and non-intervention areas (83.5%). Only 29% of adult men from intervention areas and 34.2% from non-intervention woredas were responsible for collecting water. In total, female adolescents were more responsible for collecting water (25.5%) than male adolescents (15.6%) (Figure 3). PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 89 80.8 79.9 90 80 70 60 50 40 30 20 10 0 Adult Woman Adult Man Under 15 Years Under 15 Years Male Others Female Total Intervention district Non-intervention district 30.33 2934.2 25.51 25.6 25.1 15.55 14.7 18.1 1.85 2.3 0.5 83.5 Figure 3: Responsibility for collecting collect water in Transform WASH areas and non-WASH A significant number of HHs (1,483, 48.3%) reported that they collected water without making any payments. More than half of participants from non-intervention woredas (54%) and 1,068 (46.3%) from Transform WASH intervention areas obtained water free of charge (p = 0.001). However, the majority of those who paid for water from both intervention areas (44.6%) and non-intervention (44.2%) woredas paid between 1 and 50 ETB per month. One-fourth of participants from Transform WASH intervention woredas (24.8%) and 193 (25.1%) participants from non-intervention woredas reported consuming between 21 and 40 liters of water per day. One-third (32.9%) of participants from Transform WASH intervention woredas and 340 (44.3%) participants from non-intervention woredas used between 11 and 20 liters of water per day for handwashing (Table 6). Table 6: Water consumption at household level in Transform WASH intervention areas and non￾intervention areas, Ethiopia, 2022 Water Consumption Categories Intervention woredas Non -intervention woredas Total P -value Pay for water No 1068 (46.3) 415 (54) 1483 (48.3) 0.001 Yes 1237 (53.7) 353 (46) 1590 (51.7) Bill for water per month 1-50 552 (44.6) 156 (44.2) 708 (23.0) 0.012 (ETB) 51-100 224 (18.1) 87 (24.6) 311 (10.1) >100 461 (37.3) 110 (31.2) 571 (18.6) Water consumption for 1-10 489 (21.2) 122 (15.9) 611 (19.9) 0.001 handwashing per liter 11-20 758 (32.9) 340 (44.3) 1098 (35.7) per a day 21-30 326 (14.1) 133 (17.3) 459 (14.9) 31-40 267 (11.6) 115 (15) 382 (12.4) 41-50 144 (6.2) 24 (3.1) 168 (5.5) >50 321 (13.9) 34 (4.4) 355 (11.6) The JMP standard is that individuals have access to a minimum of 25L of water per day for domestic purposes (including drinking and cleaning). Of those surveyed, only 12.4% of participants were getting >25L of water per person per day. However, a relatively larger proportion of participants from non￾intervention areas (17.1%) were accessing >25L per day than participants from Transform WASH intervention woredas (10.8%) (p = 0.004) (Figure 4). PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 90 87.6 89.2 82.9 12.4 10.8 17.1 0 10 20 30 40 50 60 70 80 90 100 Total Intervention Non-intervention <25L/P/day ≥25L/P/day - - day in Figure 4: Proportion of households who were getting minimum standard of water for person per Transform WASH intervention areas and non-intervention woredas, Ethiopia, 2022 Some HH respondents reported treating water at home, with significant differences between Transform WASH intervention and non-intervention woredas. In total, only 340 (11.1%) HHs treat water at home. More participants from Transform WASH intervention woredas (12.1%) reported treating water at home than participants from non-intervention woredas (7.9%) (p = 0.005). More than one-fourth (27.6%) of HHs from Transform WASH intervention areas and 33 (54.1%) HHs from non-intervention woredas treated water by boiling it. More than one-third (35.1%) of HHs from intervention areas and almost one-third (32.8%) of HHs from non-intervention areas treated water at home using Aquatabs. A significant number of participants from Transform WASH intervention areas (44.4%) and non￾intervention areas 15 (24.6%) obtained materials for water treatment from health centers (Table 7). Table 7: Water treatment and handling at Transform WASH areas and non-Transform WASH woredas, Ethiopia, 2022 Water Treatment Categories Intervention Non intervention Total p value Water treatment at home Yes 279 (12.1) 61 (7.9) 340 (11.1) 0.005 No 2002 (86.9) 701 (91.3) 2703 (88.0) Don’t Know 24 (1) 6 (0.8) 30 (1.0) How treat water Boil No 202 (72.4) 33 (54.1) 235 (69.1) 0.005 Yes 77 (27.6) 28 (45.9) 105 (30.9) Aquatabs No 181 (64.9) 41 (67.2) 222 (65.3) 0.728 Yes 98 (35.1) 20 (32.8) 118 (34.2) Strain through cloth No 251 (90) 38 (62.3) 289 (85) 0.001 Yes 28 (10) 23 (37.7) 51 (15.0) Let it stand and settle No 267 (95.7) 58 (95.1) 325 (95.6) 0.832 Yes 12 (4.3) 3 (4.9) 15 (4.4) Lemon, bishan gari, water filter, use cover No 61 (100) 204 (73.1) 265 (77.9) 0.001 Yes - 75 (26.9) 75 (22.1) Providers of substance for treatment Retail outlets / Shops No 233 (83.5) 32 (52.5) 265 (77.9) <0.001 Yes 46 (16.5) 29 (47.5) 75 (22.1) Pharmacies / Drug outlets No 209 (74.9) 55 (90.2) 264 (77.6) 0.010 Yes 70 (25.1) 6 (9.8) 76 (22.4) PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 91 Water - - Treatment Categories Intervention Non intervention Total p value WASH Business centers No 277 (99.3) 60 (98.4) 337 (99.1) 0.485 Yes 2 (0.7) 1 (1.6) 3 (0.9) Health centers No 155 (55.6) 46 (75.4) 201 (59.1) 0.004 Yes 124 (44.4) 15 (24.6) 139 (40.9) Other sources No 195 (69.9) 47 (77) 242 (71.2) 0.264 Yes 84 (30.1) 14 (23) 98 (28.8) Findings from this assessment showed that 10.7% of all households properly stored their water. Figure 5: Composite score of water storage, treatment and handling at Transform WASH intervention areas and non-intervention woredas, Ethiopia, 2022 89.3 10.7 89.7 10.7 96.8 1.4 89.7 10.3 88.5 11.5 98.2 1.8 87.9 12.1 93.2 6.8 99.7 0.3 0 20 40 60 80 100 120 Poor water storage Good water storage Poor water treatment Good water treatment Poor water storage and handling Good water storage and handling Total Intervention area Non-intervention area 10.7% of all households treated their water. Only 1.4% of HHs overall properly stored and handled water at home. HHs from Transform WASH woredas more properly handled and stored water, 1.8% from Transform WASH woredas and 0.3% from non-intervention woredas, a significant difference (p = 0.002) (Figure 5). 3.4 LATRINE AVAILABILITY AND UTILIZATION The findings of the assessment showed that 2,180 (70.9%) surveyed HHs have any kind of toilet (improved/unimproved and shared/unshared), while 893 (29.1%) HHs had no type of toilet facility. The multivariate binary logistic regression analysis showed that participants who did not believe that having a toilet could reduce diarrhea morbidity were 31% less likely (AOR = 0.69 [0.54, 0.8]) to have a toilet than those who perceived that having a toilet could mitigate diarrheal morbidity. Similarly, HHs who were not headed by males were 33% less likely (AOR = 0.67 [0.49, 0.92]) to have a toilet than female￾headed HHs. Participants who were currently working for payment were 70% more likely (AOR = 1.70 [1.20, 2.40]) to have a toilet than those who were not working for payment. Higher economic status of study participants was significantly associated with having a toilet. HHs who were the poorest (AOR = 0.003 [.001, 0.01]), poor (AOR = 0.04 [0.02, 0.08]), and rich (AOR = 0.20 [0.11, 0.38]) were less likely to have a toilet than those who were the richest. Being the beneficiary of Transform WASH projects was not significantly associated with having toilet or not (p = 0.169) (Table 8). PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 92 Table 8: Factors associated with having toilet in Ethiopia, 2022 Variables Had toilet (%) No toilet (%) COR (95%CI) AOR (95%CI) Sig. Types of education attended College graduate or student 61 (2.8) 7 (0.8) 028 (0.11, 0.74) 1.39 (0.39, 4.96) 0.611 Completed Grade (1-12) 972 (44.6) 210 (23.5) 1.50 (0.44, 5.14) 1.94 (0.40, 9.30) 0.410 TVET Graduate or student 24 (1.1) 7 (0.8) 080 (0.31, 2.09) 1.41 (0.40, 50.04) 0.594 University graduate/student 29 (1.3) 5 (0.6) 0.59 (0.17, 2.10) 0.78 (0.14, 4.45) 0.781 No formal education 1094 (50.2) 664 (74.4) 1 1 WASH intervention area Yes 1587 (72.8) 718 (80.4) 0.65 (0.54, 0.79)** 0.83 (0.64, 1.08) 0.169 No 593 (27.2) 175 (19.6) 1 1 Believe having toilet reduces risk of diarrhea No 584 (26.8) 282 (31.6) 0.79 (0.67, 0.94)** 0.69 (0.54, 0.88) 0.003 Yes 1598 (73.2) 611 (68.4) 1 1 Male-headed HH No 408 (18.7) 271 (30.3) 0.53 (0.44, 0.63) 0.67 (0.49, 0.92) 0.012 Yes 1772 (81.3) 622 (69.7) 1 1 Currently in paid employment No 763 (85.4) 1965 (90.1) 1.56 (1.23, 1.97) 1.70 (1.20, 2.40) 0.003 Yes 130 (14.6) 215 (9.9) 1 1 Religion Catholic 2 (0.2) 13 (0.6) 0.63 (0.14, 2.82) 1.04 (0.16, 6.91) 0.970 Muslim 579 (64.8) 902 (41.4) 0.15 (0.12, 0.19)** 0.86 (0.62, 1.21) 0.390 Orthodox 233 (26.1) 464 (21.3) 0.19 (0.14, 0.25)** 0.48 (0.34, 0.68) 0.001 Others 3 (0.3) 11 (0.5) 0.35 (0.10, 1.29) 0.43 (0.08, 2.41) 0.098 Protestant 76 (8.5) 790 (36.2) 1 1 Marital status 0.068 Divorced 28 (3.1) 49 (2.2) 0.66 (0.39, 1.13) 0.51 (0.25, 1.05) 0.067 Married/cohabitating 728 (81.5) 1767 (81.1) 0.92 (0.69, 1.22) 0.78 (0.50, 1.23) 0.283 Separated 13 (1.5) 12 (0.6) 0.35 (0.15, 0.80) 029 (0.09, 0.87) 0.027 Single 53 (5.9) 164 (7.5) 1.17 (0.77, 1.77 0.58 (0.32, 1.05) 0.070 Widow 71 (8) 188 (8.6) 1 1 Wealth index Poorest 541 (60.6) 76 (3.5) 0.03 (0.02, 0.01)** 0.003 (.001, 0.01) 0.001 Poor 200 (22.4) 412 (18.9) 0.04 (.02, 0.08)** 0.04 (0.02, 0.08) 0.001 Medium 86 (9.8) 529 (24.3) 0.12 (0.07, 0.23)** 0.12 (0.06, 0.23) 0.001 Rich 54 (6) 561 (25.7) 0.21 (0.11, 0.39)** 0.20 (.11, 0.38) 0.001 Richest 12 (1.3) 602 (27.6) 1 1 Having a toilet was not uniform across the regions. A large proportion of respondents from Sidama (96.3%), SNNPR (92.2%), and Oromia (83%) reported that they had a latrine, while a small proportion of HHs from Somali (24%) and Afar (32.1%) had a latrine (Figure 6). Additional details about the types of toilets participants are presented in Table 9 and Figure 6. PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 93 - Figure 6: The proportion of households who had latrine in different regions, Ethiopia, 2022 In total, 993 (45.5%) study participants reported using a pit latrine without a slab or open pit. Findings of this study indicated that 662 (41.71%) participants from intervention areas and 331 (55.82%) participants from non-intervention areas were using a pit latrine without a slab/open pit, while 279 (17.6%) participants from Transform WASH intervention areas and 156 (26.31%) participants from non￾intervention woredas were using a riprap/mud-plastered floor type of pit latrine (p = 0.001) (Table 9). Table 9: Types of Toilet Distributions in WASH and non-WASH Project areas, Ethiopia, 2022 Categories Category of study area Total Intervention (%) Non intervention (%) Water flush 240 (15.1) 21 (3.5) 261 (12.0) Ventilated improved pit latrine 32 (2.0) 6 (1.0) 38 (1.7) Pit latrine riprap/mud plastered floor 279 (17.6) 156 (26.3) 435 (20.0) Pit latrine without slab/open pit 662 (41.7) 331 (55.8) 993 (45.6) Pit latrine with plastic &/or concrete slab 241 (15.2) 65 (11.0) 306 (14.0) Pit latrine with self-closing/sealing 38 (2.4) 3 (0.5) 41 (1.9) Pit latrine w/plastic &/or concrete slab & closing lid 93 (5.9) 10 (2.0) 103 (4.7) Others 2 (0.1) 1 (0.2) 3 (0.1) P < 0.001 The HH survey identified that 893 (29.1%) HHs have no toilet facility and are still near the bottom of the JMP sanitation ladder. More HHs (1,441 or 46.9%) had some sort of unimproved latrine facility (open pit without slab or slab material not easily washable). Only 589 (19.2%) observed HHs reached the status of basic sanitation facility. Compared to non-Transform WASH intervention areas (10%), study participants from Transform WASH areas (22.2%) had better access to basic sanitation facilities (p = 0.001). Findings of this study indicated that proportionally more HHs in the non-intervention woredas (63.6%) are in the unimproved section of the sanitation service ladder than HHs in intervention woredas (41.3%) (p = 0.001) (Figure 7). PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 94 Figure 7: Access to sanitation services in Transform WASH and non-intervention woredas, Ethiopia, 2022 The zonal estimate for sanitation services have identified woredas in Dega Bour zone of Somali regional state with more HHs (180, 83%) in the lower ladder (no toilet facility) than any other zones or woredas. On the contrary, more HHs in the Transform WASH surveyed woredas of Sidama (37.3%) had basic sanitation facility (Table 10). Table 10: Access to Sanitation services in selected zones of Ethiopia, 2022 Zones / Woreda Category (Phase) Sample Size Sanitation Service Basic Limited Unimproved Open defecation Dega Bour (3) N=217 14 (6.5) 16 (7.4) 7 (3.2) 180 (82.9) Gursum (3) N=217 44 (20.3) 7 (3.2) 16 (7.4) 150 (69.1) East Gojjam (2) N=180 28 (15.6) 3 (1.7) 92 (51.1) 57 (31.7) Gamo (1) N=192 27 (14.1) 17 (8.9) 136 (70.8) 12 (6.3) Hadiya (1) N=180 59 (32.8) 4 (2.2) 96 (53.3) 21 (11.7) North Shewa (C) N=192 3 (1.6) 3 (1.6) 82 (42.7) 104 (54.2) Sidama (1) N=217 81 (37.3) 4 (1.8) 124 (57.1) 8 (3.7) South Wollo (2/3) N=552 106 (19.2) 24(4.3) 341 (61.8) 81 (14.7) Southwest Shewa (3/C) N=180 53 (29.4) 2 (1.1) 56 (31.1) 69 (38.3) West Arsi (2/C) N=551 145 (26.3) 16 (2.9) 335 (60.8) 55 (10.0) Wolaita (1/C) N=180 14 (7.8) 6 (3.3) 150 (83.3) 10 (5.6) Afar (Zone1) (3) N=215 15 (7.0) 48 (22.3) 6 (2.8) 146 (67.9) Non-intervention woredas N=768 77 (10.0) 28 (3.6) 488 (63.5) 175 (22.8) PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 95 - - - Zones / Woreda Category (Phase) Sample Size Sanitation Service Basic Limited Unimproved Open defecation Intervention woreda N=2,305 512 (22.2) 122 (5.3) 953 (41.3) 718 (31.1) Total N=3,073 589 (19.2) 150 (4.9) 1,441 (46.9) 893 (29.1) PRIN assessed the differences among the woredas based on the phases when the Transform WASH started to operate at different woredas, but no significant difference was observed among the woredas on basic sanitation services (p = 0.054), limited sanitation services (p = 0.661) and unimproved sanitation services (p = 0.205). Other indicators like water services, hand washing practices, and accessibility to sanitation products were also assessed, but all these indicators did not fit statistical assumptions to conduct analysis (Table 11). Table 11: Differences among the woredas on sanitation services by phases of Transform WASH interventions, 2022 Access to Sanitation services Phase of woredas Yes (%) No (%) P value Basic sanitation services Phase 1 156 (29.1) 380 (70.9) 0.054 Phase 2 128 (24.6) 392 (75.4) Phase 3 132 (23) 443 (77) Limited sanitation services Phase 1 41 (7.6) 495 (92.4) 0.661 Phase 2 32 (6.2) 488 (93.8) Phase 3 38 (6.6) 537 (93.4) Unimproved sanitation services Phase 1 339 (63.2) 197 (36.8) 0.205 Phase 2 360 (69.2) 160 (30.8) Phase 3 405 (70.4) 170 (29.6) 3.4.1 CHALLENGES TO LATRINE CONSTRUCTION As shown in Table 12, among those who did not construct latrines, a large proportion of HHs (695, 77.8%) reported that they could not construct a latrine due to lack of capacity and a few HHs due to insufficiency of land (3.1%), difficulty in acquiring materials (7.9%), and technical challenges (4.8%). More HHs in Transform WASH intervention areas (573, 82.4%) reported they had no capacity to build a latrine than those residing non-intervention areas (17.6%) (p = 0.001). Table 12: Challenges to latrine construction in Transform WASH and non-Transform areas in Ethiopia Challenges to latrine construction Intervention (%) Non intervention (%) Total p value Lack of capacity 573 (82.4) 122 (17.6) 695 (77.8) 0.001 Lack of space 19 (67.9) 9 (32.1) 28 (3.1) 0.089 Difficult to acquire materials 54 (76.1) 17 (23.9) 71 (7.9) 0.336 Lack of permission 14 (66.7) 7 (33.3) 21 (2.3) 0.109 Technical constraints 21 (48.8) 22 (51.2) 43 (4.8) 0.001 Not seen as a priority problem 35 (51.5) 33 (48.5) 68 (7.6) 0.001 Others* 80 (87) 12 (13) 92 (10.3) 0.095 *Damaged, filled, on construction, lack of time, has plan to construct, using others’ toilet, lack of attention, culture Overall, 1,382 (87.1%) HHs from intervention areas and 484 (83.3%) HHs from non-intervention areas had latrines in their own yards or compounds (p = 0.052) (Figure 8). PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 96 = - = - Figure 8: Toilet locations in Transform WASH intervention areas and non-Transform WASH areas of Ethiopia, 2022 As shown in Table 13, 350 (16%) HHs share their toilets with other HHs. 241 HHs from Transform WASH intervention areas (15.2%) and 109 (18.6%) HHs from non-intervention areas reported sharing toilet with their neighbors (p > 0.05). Table 13: Toilet sharing practices among Transform WASH interventions and non-intervention HHs in Ethiopia, 2022 Toilet sharing Categories Intervention areas (n 1587) Non intervention areas (n 593) Total P value Share toilet No 1346 (84.8) 484 (81.6) 1830 (83.9) 0.071 Yes 241 (15.2) 109 (18.6) 350 (16.1) Number of people Less than 10 157 (9.9) 88 (14.8) 245 (11.2) who share toilet 10 and above 77 (4.9) 21 (3.5) 98 (4.5) 0.003 Don’t know 7 (0.4) - 7 (0.3) 3.4.2 CHALLENGES OF SANITATION PRODUCT SUPPLY UTILIZATION The survey teams asked HHs what hindered them from accessing sanitation products. As shown in Table 14, a high proportion of HHs have listed financial limitations as a factor. Comparatively, a similar proportion of HHs in Transform WASH woredas and non-intervention woredas reported financial limitation as a reason (p > 0.05). Though there were no significant differences (p > 0.05) between intervention (78%) and non-intervention areas (77.4%), over one-fifth (22.2%) of the respondents reported that sanitation products were not easily accessible. Some participants from Transform WASH interventions areas (9.4%) and non-intervention areas (12%) reported that purchasing of a toilet slab was not a priority of the HH (p > 0.05). However, 10.8% and 12.7% of participants from intervention and non-intervention woredas respectively did not know from where to purchase sanitation products (p > 0.05). Though a significant difference was not observed (p = 0.746), some respondents from both the intervention (26, 1.6%) and non-intervention areas (21, 3.5%) had no disposable cash and did not have access to sanitation products (Table 14). PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 97 Table 14: Challenges of Sanitation Product Supply Outlet Utilization in Ethiopia, 2022 Challenges Intervention Non - intervention Total P - value Not accessible 206 (22) 109 (22.6) 315 (22.2) 0.803 Not heard or seen about the product 67 (7.2) 88 (18.3) 155 (10.9) 0.001 Not know where to buy 101 (10.8) 61 (12.7) 162 (11.4) 0.299 Lack of disposable cash 695 (74.3) 353 (73.2) 1048 (74) 0.656 No need to change the slab 39 (4.2) 20 (4.1) 59 (4.2) 0.985 Slab is not a priority expenditure 88 (9.4) 58 (12) 146 (10.3) 0.124 No disposable cash and sanitation product is not accessible 26 (1.6) 21 (3.5) 47 (2.2) 0.746 No disposable cash and not know where to buy 11 (0.7) 3 (0.5) 14 (0.6) 0.061 Satisfied with existing unimproved toilet and no disposable cash 3 (0.2) 1 (0.2) 4 (0.2) 0.792 Not heard about or seen improved sanitation product and not know where to buy 5 (0.3) 4 (0.7) 9 (0.4) 0.071 Not heard about improved sanitation, not accessible and not know where to buy 2 (0.1) 22 (3.7) 24 (1.1) 0.421 Slab is not a priority expenditure and have no disposable cash 16 (1) 7 (1.2) 23 (1.1) 0.763 - - 3.4.3 OPEN DEFECATION AND ITS REASON Overall,1,374 (44.7%) HHs reported that all HH members always utilize latrines. However, 548 (38.1%) respondents from Transform WASH woredas and 102 (30.9%) participants from non-Transform WASH woredas reported that HHs have neither shared nor unshared latrine and practice open defecation for this reason. More than one-third (38.1%) of respondents from Transform WASH intervention and 102 (30.9%) from non-WASH intervention areas reported toilet deficit as main reason for open defecation (p < 0.05). Some respondents from WASH intervention areas (2.4%) and non-intervention areas (6.7%) reported that defecating in bush give them more comfort than using toilet. More than half of respondents (53.9%) from Transform WASH intervention areas and 484 (63%) respondents from non￾intervention areas reported that they have attended behavioral change education on the importance of using toilet and negative consequence of open defecation. Health workers were the main source of information about toilet facility and open defecation for both Transform WASH intervention areas (89.5%) and non-intervention areas (94%) (Table 15) (p < 0.01). Table 15: Open Defecation and Reason of Open Defecation in Ethiopia, 2022 Open defecation Intervention Non Intervention Total P value Number people practicing open defecation No OD 1177 (5.1) 197 (25.7) 1374 (44.7) 0.001 1-3 176 (7.6) 71 (9.2) 247 (8) 4-6 311 (13.5) 86 (11.2) 397 (12.9) 7-9 133 (5.8) 19 (2.5) 152 (4.9) Don’t know 508 (22) 395 (51.4) 903 (29.4) Reason for OD Latrine does not provide privacy 23 (1.6) 9 (2.7) 32 (1.8) 0.166 Bush is easier to access 24 (1.7) 8 (2.4) 32 (1.8) 0.354 Bush is more comfortable 34 (2.4) 22 (6.7) 56 (3.2) 0.001 Consider latrine as dirty 5 (0.3) 1 (0.3) 6 (0.3) 0.899 No access to latrine 548 (38.1) 102 (30.9) 650 (36.8) 0.014 No 1048 (45.5) 278 (36.2) 1326 (43.2) 0.001 PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 98 - - - - Open defecation Intervention Non Intervention Total P value Attended BCCC about toilet and OD Yes 1242 (53.9) 484 (63) 1726 (56.2 Don’t remember 15 (0.7) 6 (0.8) 21 (0.7) Source of information about toilet and OD Sales agent 12 (1) 2 (0.4) 14 (0.8) 0.250 Kebele leaders 235 (18.9) 150 (31) 385 (22.3) 0.001 Health workers 1111 (89.5) 455 (94) 1566 (90.7) 0.003 Health development army 227 (18.3) 96 (19.8) 323 (18.7) 0.456 NGO 94 (7.6) 11 (2.3) 105 (6.1) 0.001 Women development army 29 (2.3) 1 (0.1) 30 (1.7) 0.027 Others* 25(2) - 25 (1.4) 0.002 *I don’t know, I don’t remember, Guest 3.4.4 GENDER AND HOUSEHOLD DECISION ON TOILET CONSTRUCTION As part of the analysis, PRIN compared Transform WASH intervention and non-intervention areas on the HH decision of toilet construction and upgrading. A similar proportion of participants from both Transform WASH intervention (423, 33%) and non-intervention areas (159, 32.4%) reported that they jointly decide on toilet construction (p = 0.031). In total, 1,100 (62.1%) participants noted the man of the HH was responsible for decisions on toilet construction. Only 30 (1.7%) women had decision￾making power on toilet construction. More than half of participants from both Transform WASH intervention (61.3%) and non-interventions areas (64.2%) reported that decisions regarding toilet construction was made by the man of the HH. Almost all respondents (1,651, 93.2%), including 1,186 (92.6%) in Transform WASH intervention woredas and 465 (94.7%) in non-interventions woredas, reported the HH has built the existing toilet because of a recognition of health benefits associated with constructing a toilet. A large proportion of participants (88%) reported that they constructed the toilet by themselves. In total, 1,468 (82.8%) of HHs constructed a latrine in the last five years with more construction in Transform WASH areas (84.9%) than HHs from non-intervention areas (77.6%) (p = 0.001). In the last year, HHs in Transform WASH areas (13.4%) upgraded their toilets more than HHs (6.9%) in non-intervention areas (p = 0.001) (Table 16). Table 16: Toilet construction in Transform WASH intervention areas and non-intervention areas, Ethiopia, 2022 Categories Intervention Non intervention Total P value Decision on toilet construction Joint decision 423 (33) 159 (32.4) 582 (32.8) 0.031 Man of the HH 785 (61.3) 315 (64.2) 1100 (62.1) Family members 22 (1.7) 11 (2.2) 33 (1.9) Woman of HH 25 (2) 5 (1) 30 (1.7) Other+ 26 (2) 1 (0.2) 33 (1.9) Reason for toilet construction Enforced by HEW 71 (5.5) 16 (3.3) 87 (4.9) 0.326 Kebele leader enforced 14 (1.1) 7 (1.4) 21 (1.2) Know the advantage 1186 (92.6) 465 (94.7) 1651 (93.2) Don’t have reason 5 (0.4) 1 (0.2) 6 (0.3) Others 5 (0.4) 2 (0.4) 7 (0.4) Who constructed toilet Self/Family 1101 (85.9) 458 (93.3) 1559 (88) 0.001 Community/Private/ /Charity organization 11 (0.9) 7 (1.4) 18 (1) Local Manson 139 (10.9) 23 (4.7) 162 (9.1) Government support 12 (0.9) 2 (0.4) 14 (0.8) Others+ 18 (1.4) 1 (0.2) 19 (1.3) Time when the toilet was 1-5 1087 (84.9) 381 (77.6) 1468 (82.8) 0.001 6-10 122 (9.5) 83 (16.9) 205 (11.6) PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 99 - - - - Categories Intervention Non intervention Total P value constructed >11 59 (4.6) 26 (5.3) 85 (4.8) Don’t know 13 (1) 1 (0.2) 14 (0.8) Time when toilet was upgraded Not upgraded 1308 (81) 434 (88.7) 1472 (8.31) 0.001 In the last year 172 (13.4) 34 (6.9) 206 (11.6) Before a year 71 (5.5) 23 (4.7) 94 (5.3) +Kebele leader, renter of house, relatives, NGO Generally, a very small proportion (346, 15.9%) of respondents have reported that they have upgraded their toilets. Of those who have upgraded their toilets, 279 (17.6%) of them were from Transform WASH interventions areas and 67 (11.3%) were from non-intervention areas (p = 0.001) (Figure 9). Toilet upgrading Figure 9: Toilet upgrading in Transform WASH intervention areas and non-intervention areas, Ethiopia, 2022 In total, 296 (39.1%) HHs reported that upgraded toilets were more comfortable to use. A significant number of HHs from Transform WASH intervention areas (255, 38.5%) and non-intervention areas (41, 43.6%) reported that using the upgraded toilet was attractive and comfortable. More than one-fourth of participants (27.3%) perceived that upgrading the toilet would make it easy to clean. Additionally, ease of transportation (5.5%), durability/longevity of floor (8.75%), affordability (8.4%), prevention of odor (22%), and prevention of fly nuisance (17.8%) were reported as advantages of having upgrading toilet (Table 17). Table 17: Reason for upgrading toilet in Transform WASH intervention areas and non-intervention areas, Ethiopia, 2022 Reason for toilet upgrading Categories Categories of study area Total P value Intervention Non intervention Comfortable and attractive to use No 408 (61.5) 53 (56.4) 461 (60.9) 0.338 Yes 255 (38.5) 41 (43.6) 296 (39.1) Easier to clean/wash No 481 (72.3) 71 (75.5) 552 (72.7) 0.514 Yes 184 (27.7) 23 (24.5) 207 (27.3) Easy to transport No 629 (94.6) 88 (93.6) 717 (94.5) 0.700 Yes 36 (5.4) 6 (6.4) 42 (5.5) Floor not degrade No 610 (91.7) 83 (88.3) 693 (91.3) 0.269 Yes 55 (8.3) 11 (11.7) 66 (8.7) PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 100 - - - - Reason for toilet upgrading Categories Categories of study area Total P value Intervention Non intervention Not expensive No 609 (91.6) 86 (91.5) 695 (91.6) 0.977 Yes 56 (8.4) 8 (8.5) 64 (8.4) It can be reused No 631 (94.9) 89 (94.7) 720 (94.9) 0.932 Yes 34 (5.1) 5 (5.3) 39 (5.1) Not muddy No 618 (92.9) 86 (91.5) 704 (92.8) 0.613 Yes 47 (7.1) 8 (8.5) 55 (7.2) Safe to use No 553 (83.2) 72 (76.6) 625 (82.3) 0.118 Yes 112 (16.8) 22 (23.4) 134 (17.7) Prevents bad odor No 519 (78) 73 (77.7) 592 (78) 0.933 Yes 146 (22) 21 (22.3) 167 (22) Prevents fly nuisance No 539 (81.1) 85 (90.4) 624 (82.2) 0.026 Yes 126 (18.9) 9 (9.6) 135 (17.8) Sales outlet accessible No 660 (99.2) 93 (98.9) 753 (99.2) 0.749 Yes 5 (0.8) 1 (1.1) 6 (0.8) 3.4.5 CHALLENGES TO RENOVATE TOILET In total, 581 (31.7%) respondents, in which 34.2% from intervention and 25.5% from non-intervention areas, replied that they did not face major challenges while renovating their toilet. A significant number of participants (46%) complained that renovation of toilet was expensive where relatively participants from non-intervention (51%) reported the expensiveness of upgrading toilet than participants from intervention areas (44%) (p < 0.01). Lack of disposable cash (47%) was one challenge identified overall from 45% of HHs from Transform WASH intervention areas, and 51.9% of the non-intervention area identified financial limitations as a challenge (p = 0.007). Moreover, not seeing toilet upgrades as a priority issue (11.3%) was identified as a challenge to renovate the toilet. Table 18: Challenges of upgrading toilet in Transform WASH intervention areas and non￾intervention areas, Ethiopia, 2022 Challenges Categories Category of study area Total P value Intervention Non intervention No problem with renovating No 861 (65.8) 392 (74.5) 1253 (68.3) 0.001 Yes 447 (34.2) 134 (25.5) 581 (31.7) Expensive to renovate No 733 (56) 258 (49) 991 (54) 0.007 Yes 575 (44) 268 (51) 843 (46) Lack of disposable cash to buy materials No 719 (55) 253 (48.1) 972 (53) Yes 589 (45) 273 (51.9) 862 (47) 0.008 Not suitable to transport slab No 1290 (98.6) 521 (99) 1811 (98.7) 0.459 Yes 18 (1.4) 5 (1) 23 (1.3) Fear of pit collapse slab No 1289 (08.5) 522 (99.2) 1811 (98.7) 0.228 Yes 19 (1.5) 4 (0.8) 23 (1.3) Looking for free distribution No 1243 (95) 487 (92.6) 1730 (94.3) 0.041 Yes 65 (5) 39 (7.4) 104 (5.7) Not priority problem No 1176 (89.9) 451 (85.7) 1627 (88.7) 0.011 Yes 132 (10.1) 75 (14.3) 207 (11.3) The findings of this assessment indicated that female-headed HHs were 54% less likely to have upgraded their toilet (AOR = 0.46 [0.29, 0.74]) than male-headed HHs (p = 0.001), which might be due to lack of commitments by males, as they mostly stay out of home or focus on field activities. Similarly, HHs where toilet construction is primarily decided by the man of the HH were 48% less likely to have upgraded their toilet (AOR = 0.52 [0.31, 0.88]) than HHs where women were the primary decision￾makers on toilet construction (p = 0.015). HHs from Transform WASH intervention areas were 70% PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 101 - - more likely to have upgraded their toilet (AOR = 1.7 [1.27, 2.27]) than HHs from non-intervention areas (p = 0.0001) (Table 19). Table 19: Factors associated with upgrading of toilet in Transform WASH intervention and non￾intervention areas, Ethiopia, 2022 Variables Upgraded Toilet COR (95%CI) AOR(95%CI) P value Yes (%) No (%) Head of HH Female-headed HH 46 (13.3) 362 (19.7) 0.62 (0.45, 0.87)** 0.46 (0.29, 0.74) 0.001 Male headed HH 300 (81.3) 1472 (80.3) 1 1 Can you read and write No 161 (46.5) 881 (48) 0.94 (0.75, 1.19) 1.01 (0.84, 1.39) 0.531 Yes 185 (53.5) 953 (52) 1 1 In paid employment recently No 312 (90.2) 1653 (90.1) 1.01 (0.68, 1.48) 1.01 (0.68, 1.49) 0.978 Yes 34 (9.8) 181 (9.9) 1 1 Toilet construction decision Joint decision 129 (37.3) 515 (28.1) 1.41 (0.95, 2.09) 0.79 (0.46, 1.35) 0.384 Man of HH 170 (49.1) 1010 (55.1) 0.94 (0.64,1.38) 0.52 (0.31, 0.88) 0.015 Health worker/relative/ neighbor 2 (0.6) 30 (1.6) 0.37 (0.09,1.63) 0.20 (0.04, 0.89) 0.065 Other family member 7 (2) 66 (3.6) 0.59 (0.25,1.39) 0.46 (0.19, 1.11) 0.082 Women of the HH 38 (11) 1 1 Percentile wealth index Poorest 11 (3.2) 65 (355) 0.77 (0.39,1.50) 0.73 (0.37, 1.47) 0.380 Poor 59 (17.1) 353 (19.2) 0.76 (0.54,1.07) 0.73 (0.51, 1.10) 0.089 Medium 79 (22.8) 450 (24.5) 0.79 (0.58,1.09) 0.79 (0.57, 1.10) 0.164 Rich 88 (25.4) 473 (25.8) 0.84 (0.62, 1.15) 0.83 (0.60, 1.13) 0.237 Richest 109 (31.5) 493 (26.9) 1 1 Category of the study area Intervention 279 (80.6) 1308 (71.3) 1.68 (1.26, 2.23)** 1.7 (1.27, 2.27) 0.0001 Non-intervention 67 (19.4) 526 (28.7) 1 1 In total, 1,381 (63.3%) respondents from both Transform WASH intervention areas (54.9%) and non￾intervention areas (59.2%) reported that they had a different latrine before the current one. Some participants from Transform WASH intervention areas (39.1%) and non-intervention areas (42.7%) said that their prior latrine was filled before constructing the current one. Some respondents (42.2%) from non-intervention areas and some respondents (37.5%) from intervention areas have covered the filled toilet and dug a new toilet (p = 0.017) (Table 20). Table 20: History of having toilet and emptying it in Transform WASH intervention areas and non￾intervention areas, Ethiopia, 2022 History of Categories Category of study area Total P value having toilet Intervention Non Intervention Had toilet before Yes 1030 (64.9) 351 (59.2) 1381 (63.3) 0.045 No 543 (34.2) 237 (40) 780 (35.8) Don’t know 14 (0.9) 5 (0.8) 19 (0.9) Latrine filled No 966 (60.9) 340 (57.3) 1306 (59.9) 0.134 before Yes 621 (39.1) 253 (42.7) 874 (40.1) PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 102 - - - History of Categories Category of study area Total P value having toilet Intervention Non Intervention How filled toilet Changed pit 595 (37.5) 250 (42.2) 845 (38.8) 0.002 is emptied Emptied by themselves 7 (0.4) 1 (0.1) 8 (0.4) Moved slab to new pit 15 (09) - 15 (0.7) Others 4 (0.3) 2 (0.3) 6 (0.2) 3.4.6 PAYMENT FOR TOILET CONSTRUCTION Findings of this report revealed that 860 (39.4%) HHs paid for their toilet construction. The proportion of HHs who paid for toilet construction were 55.2% in Transform WASH intervention areas and 32.5% in non-intervention areas (P=0.001). More than one-third (240, 37.6%) of participants from Transform WASH intervention areas and 84 (43.5%) participants from non-intervention areas said they paid more than 2,000 birr to construct a toilet. More than one-fourth of respondents (25.6%) reported that it was difficult for the HH to pay to construct a toilet, though some organizations also contributed to their payments. Proportionally, participants from Transform WASH areas (422, 26.6%) reported they have faced more burden of payments than participants from non-intervention areas (22.8%). A very small number of participants (20, 0.7%) reported that they have access to loans for toilet construction. A large proportion of respondents expressed a fear of ability to repay loans (15.2%) and lack of interest (26%) as their main reason why they were not benefiting from the accessible financial loan for toilet construction. Of those who received a loan, a negligible proportion of the money was used to buy the latrine slab (8, 40%), to build superstructure (6, 30%), to repair/upgrade the latrine (4, 20%) and to buy hygiene materials (7, 35%). Respondents were also asked for the readiness of the HH to repair their existing toilet. More than two-third of the respondents (68.3%) revealed that they are ready to repair their latrine. The proportion in Transform WASH intervention areas was 1,043 (65.7%) to 447 (75.4%) in non-intervention areas (Table 21). Table 21: Payment for Toilet Construction in Transform WASH intervention areas and non￾intervention areas, Ethiopia, 2022 Payment for toilet construction Categories Category of study area Total P Value Intervention Non Intervention Paid to construct current toilet Yes 667 (42) 193 (32.5) 860 (39.4) 0.001 No 876 (55.2) 386 (65.1) 1262 (57.9) Don’t know 44 (2.7) 14 (2.4) 58 (2.7) Amount of money paid to construct toilet 1-500 175 (27.4) 32 (16.6) 207 (24.9) 0.028 501-1000 122 (19.1) 39 (20.2) 161 (19.4) 1001-1500 55 (8.6) 17 (8.8) 72 (8.7) 1501-2000 46 (7.2) 21 (10.90 67 (8.1) >2000 240 (37.6) 84 (43.5) 324 (39) Amount of money paid by toilet owner to construct 1-500 180 (28.3) 34 (17.6) 214 (25.8) 0.047 501-1000 125 (19.7) 41 (21.2) 166 (200) 1001-1500 58 (9.1) 20 (10.40 78 (9.4) 1501-2000 49 (7.7) 21 (10.9) 70 (8.4) >2000 224 (35.2) 77 (39.9) 301 (%) Amount paid by other organization 1-2000 7 (38.9) 2 (40) 9 (39.1) 0.860 >2000 11 (61.1) 3 (60) 14 (60.9) Difficult to pay No 221 (13.9) 58 (9.8) 279 (12.8) 0.002 Yes 422 (26.6) 135 (22.8) 557 (25.6) Accessibility to financial support/loan Don’t know 91 (3.9) 10 (1.3) 101 (3.3) 0.001 Not accessible 1416 (61.4) 575 (74.9) 1991 (64.8) Accessible 18 (0.8) 2 (0.3) 20 (0.7) Reason of financial inaccessibility Fear of ability to pay back 306 (13.3) 160 (20.8) 466 (15.2) 0.001 I can’t get loan 329 (14.3) 78 (10.2) 407 (13.2) PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 103 - - Payment for toilet construction Categories Category of study area Total P Value Intervention Non Intervention Lack of interest 562 (24.4) 238 (31) 800 (26) Not Affordable 219 (9.5) 79 (10.3) 298 (9.7) Other 21 (0.9) 2 (0.3) 23 (0.7) My religion forbid loan 35 (1.5) 26 (3.4) 61 (2) Loan givers Bank/Microfinance 4 (0.1) 1 (0.1) 5 (0.2) 0.457 VSLA and others 12 (0.5) 1 (0.1) 13 (0.3) Expenditure of loan To buy the latrine slab 7 (38.9) 1 (50) 8 (40) 0.001 To build superstructure 5 (27.8) 1 (50) 6 (30) 0.515 To repair/upgrade latrine 4 (22.2) - 4 (20) 0.456 To buy hygiene materials 6 (33.3) 1 (50) 7 (35) 0.639 Sources of sanitation products to purchase Retail outlet 12 (66.7) 1 (50) 13 (65) 0.639 Others+ 9 (33.3) 1 (50) 10 (35) 0.389 Slab installation or its parts Another person 9 (0.6) 1 (0.2) 10 (0.5) 0.126 Myself 7 (0.4) - 7 (0.3) Easiness to transport sanitation product No 5 (0.3) - 5 (0.2) 0.269 Yes 2 (0.1) 2 (0.1) Repairing of the latrine if it is needed No 544 (34.3) 146 (24.6) 690 (31.7) 0.001 Yes 1043 (65.7) 447 (75.4) 1490 (68.3) * Village saving and loan associations + Mason, WASH business, Health centers Sales agent As shown in Table 22, of the HHs with toilet maintenance needs, a total of 679 (31.1%) HHs reported that superstructure of toilet mostly needed maintenance. The proportion in both groups was approximately two times less than the overall proportion where 294 (18.5%) from intervention areas and 88 (14.8%) non-intervention areas (p = 0.004). A higher proportion of respondents from WASH intervention areas (80.7%) had complained about toilet maintenance compared to participants from non￾intervention areas (79.8%) (p = 0.049). Overall, more than three-fourth (78.3%) of respondents reported that it was more difficult to find maintenance parts compared to five years back. In total, 1,264 (79.6%) respondents from Transform WASH intervention areas and 442 (74.5%) of respondents from non-intervention areas have faced problem in accessing toilet maintenance parts. Respectively, 42.9%, 36.7%, and 30.4% of respondents reported the maintenance of the superstructure, slab, and substructure on average it costs more than 1,500 Ethiopian birr. In total, 279 (12.8%) respondents reported that the cost of toilet maintenance was affordable. Some study participants from both Transform WASH intervention areas (12.4%) and non-intervention areas (14%) reported that the cost of maintenance was affordable. Table 22: Latrine maintenance in Transform WASH intervention areas and non-intervention areas, Ethiopia, 2022 Latrine maintenance Categories Category of study area Total P value Intervention Non Intervention Parts of latrine which frequently require repair No repairs required to date 481 (30.3) 165 (27.8) 646 (29.6) 0.004 Sub-structure/pit collapse 315 (19.8) 158 (26.6) 473 (21.7) Slab 294 (18.5) 88 (14.8) 382 (17.5) Superstructure 497 (31.3) 182 (30.7) 679 (31.1) Easiness of latrine repair compared to five years About the same 42 (2.6) 27 (4.6) 69 (3.2) 0.049 Easier 187 (11.8) 74 (12.5) 261 (12) More difficult 1281 (80.7) 473 (79.8) 1754 (80.5) Unsure 77 (4.9) 19 (3.2) 96 (4.4) PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 104 - - - Latrine maintenance Categories Category of study area Total P value Intervention Non Intervention Easiness to find parts to repair the latrine About the same 35 (2.2) 28 (4.7) 63 (2.9) 0.001 Easier 163 (10.3) 96 (16.2) 259 (11.9) More difficult 1264 (79.6) 442 (74.5) 1706 (78.3) Unsure 104 (6.6) 20 (3.4) 124 (5.7) Cost to repair substructure No payment 191 (12) 20 (3.4) 211 (9.7) 0.001 1-500 191 (2.9) 66 (11.1) 270 (12.4) 501-1000 300 (18.9) 107 (18) 407 (18.7) 1001-1500 118 (7.4) 34 (5.7) 152 (7) >1500 485 (30.6) 177 (29.8) 662 (30.4) Don’t know 289 (18.2) 189 (31.9) 478 (21.9) Cost to repair slab No payment 174 (11) 19 (3.2) 193 (8.9) 0.001 1-500 166 (10.5) 37 (6.2) 203 (9.3) 501-1000 184 (11.6) 49 (8.3) 233 (10.7) 1001-1500 175 (11) 40 (6.7) 215 (9.9) >1500 553 (34.8) 246 (41.5) 799 (36.7) Don’t know 335 (21.1) 202 (34.1) 537 (24.6) Cost to repair superstructure No payment 218 (13.7) 50 (8.4) 268 (12.3) 0.001 1-500 118 (7.4) 26 (4.4) 144 (6.6) 501-1000 214 (13.5) 35 (5.9) 249 (11.4) 1001-1500 141 (8.9) 46 (7.8) 187 (8.6) >1500 635 (40) 300 (0.6) 935 (42.9) Don’t know 261 (16.4) 136 (22.9) 397 (18.2) Affordability of repairing cost No 1391 (87.6) 510 (86) 1901 (87.2) 0.001 Yes 196 (12.4) 83 (14) 279 (12.8) More than three-fourths of respondents (76.7%) reported that their families had access to a toilet, where 1,250 (78.8%) were from Transform WASH intervention areas and 422 (71.2%) from non￾intervention areas (p = 0.035). The main challenges to latrine accessibility were distance from home (50, 9.8%), inappropriate for disabled people (63, 12.4%), unsuitable for small children (446, 87.8%) and lack of privacy 24 (4.7%). More than half (50.3%) of respondents disposed children’s feces into the toilet/latrine and 350 (27.4%) into rubbish (Table 23). Table 23: Latrine maintenance in Transform WASH intervention areas and non-intervention areas, Ethiopia, 2022 Latrine accessibility Categories Categories of study area Total P value Intervention Non intervention Toilet Accessibility No 337 (21.12) 171 (28.8) 508 (23.3) 0.035 Yes 1250 (78.8) 422 (71.2) 1672 (76.7) Reason of in accessibility Latrine is too far 20 (5.9) 30 (17.5) 50 (9.8) 0.001 Not appropriate for disabled 30 (8.9) 33 (19.3) 63 (12.4) 0.001 Not good for small children 297 (88.1) 149 (87.1) 446 (87.8) 0.746 Lack of sufficient privacy 19 (5.6) 5 (2.9) 24 (4.7) 0.174 Sickness and no reason 9 (2.7) - 9 (1.8) 0.031 Where to dispose of children’s waste Children use a latrine/toilet 31 (3.2) 13 (4.2) 44 (3.4) 0.422 Bury waste in field/yard 268 (27.8) 64 (20.5) 332 (26) 0.011 Dispose in latrine/toilet 431 (44.7) 211 (67.6) 642 (50.3) 0.001 Dispose with rubbish 275 (28.5) 75 (24) 350 (27.4) 0.125 PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 105 - - - - Latrine Categories Categories of study area Total P value accessibility Intervention Non intervention Dispose with wastewater 43 (4.5) 8 (2.6) 51 (4) 0.138 Use it as manure 8 (0.8) 3 (1) 11 (0.9) 0.826 Burn it 4 (0.4) 1 (0.3) 5 (0.4) 0.817 Don’t know/No response 33 (3.5) 4 (1.3) 37 (2.9) 0.642 In total, 1, 562 (71.7%), 1,063 (48.8%), 1,381(63.3%), and 765 (35.1%) of respondents believed that having an improved latrine is useful to prevent flies, maintain sanitation, prevent disease, and protect privacy of the user, respectively (Table 24). Table 24: Community perception on having improved latrine in Transform WASH intervention areas and non-intervention areas, Ethiopia, 2022 Categories Categories of study area Total P value Intervention Non intervention Prevent flies 1119 (70.5) 443 (74.7) 1562 (71.7) 0.053 Contains/bury feces 1119 (70.5) 443 (74.7) 1562 (71.7) 0.053 Keep sanitary 698 (44) 365 (61.6) 1063 (48.8) 0.001 Prevents diseases 987 (62.2) 394 (66.4) 1381 (63.3) 0.067 Convenient to use 1316 (82.9) 477 (80.4) 1793 (82.2) 0.177 Provides privacy 530 (33.4) 235 (39.6) 765 (35.1) 0.007 Findings of this survey showed that 2,294 (74.7%) of respondents got information about sanitation from health extension workers. In non-intervention areas, more HHs (616 (80.2%)) accessed information about sanitation and related services from health extension workers. This proportion is lower in Transform WASH intervention woredas, 1,678 (72.8%) (P=0.002). About 30% of the respondents revealed that health centers were their main source of information about sanitation and hygiene while sanitation service providers served only 3% of the respondents (Table 25). Table 25: Main sources of information about sanitation and hygiene in Transform WASH intervention areas and non-intervention areas, Ethiopia, 2022 Categories Category of study area Total P Value Intervention Non intervention Health Extension Workers 1678 (72.8) 616 (80.2) 2294 (74.7) 0.002 Sales agents/marketers 33 (1.4) 9 (1.2) 42 (1.4) 0.591 Health centers 708 (30.7) 213 (27.7) 931 (30) 0.118 Sanitation service providers 82 (3.6) 11 (1.4) 93 (3) 0.003 Health development armies 319 (13.8) 143 (18.6) 462 (15) 0.001 Kebele leaders 365 (15.8) 199 (25.9) 564 (18.4) 0.001 Religious/cultural leaders 178 (7.7) 58 (7.6) 236 (7.7) 0.878 School children 196 (8.5) 93 (12.1) 289 (9.4) 0.003 Printed materials 19 (0.8) 10 (1.3) 29 (0.9) 0.236 Mass media (Radio, TV) 277 (12) 205 (26.7) 482 (15.7) 0.001 Others* 273 (11.8) 7 (0.9) 280 (9.1) 0.001 *Family, School, Children, Friends, Neighbor, Not Sure, Husband, NGO 3.4.7 TOILET HYGIENE OBSERVATION Toilets of some HHs (7.7%) were observed where the toilet was not used yet. About one-fourth of HHs’ toilet (24.8%) from both Transform WASH intervention areas (24%) and non-intervention areas (26.8%) had a hole-cover. In total, the slab of 468 (21.5%) toilets was smeared by feces and proportionally was nearly equal among Transform WASH intervention areas (338, 21.3%) and non￾PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 106 - intervention areas (130, 21.9%) (p = 0.572). A total of 148 (6.8%) HHs had their toilet squat hole smeared with feces. 1,263 (57.9%) toilets in total and 896 (54.8%) from Transform WASH intervention areas and 394 (66.4%) from non-intervention areas had an observed foot pathway leading to the toilet. Findings of this study revealed that more than one fifth (20.8%) of the toilets were observed with fecal sludge appearing less than 50 cm down the slab. About 6.0% of HHs were identified with feces in their home compound. The proportion was higher in non-intervention areas (8.3%) than Transform WASH intervention areas (5%) (p = 0.005) (Table 26). Table 26: The hygiene of toilet in Transform WASH intervention areas and non-intervention areas, Ethiopia, 2022 Hygiene of toilet Categories Category of study area Total P Value Intervention Non Intervention Latrine appears to be unused No 1463 (92.2) 549 (92.6) 2012 (92.3) 0.759 Yes 124 (7.8) 44 (7.4) 168 (7.7) Existence of cover for hole No 1206 (76) 434 (73.2) 1640 (75.2) 0.177 Yes 381 (24) 159 (26.8) 540 (24.8) Presence of feces on slab No 1249 (78.9) 463 (78.1) 1712 (78.5) 0.752 Yes 338 (21.3) 130 (21.9) 468 (21.5) Feces smear on squat hole/ hole cover No 1481 (93.3) 551 (92.9) 2032 (93.2) 0.739 Yes 106 (6.7) 42 (7.1) 148 (6.8) Fresh foot path leading to latrine No 718 (45.2) 99 (33.6) 917 (42.1) 0.001 Yes 896 (54.8) 394 (66.4) 1263 (57.9) Fresh feces in the pit No 1161 (73.2) 334 (56.3) 1495 (68.6) 0.001 Yes 426 (26.8) 259 (43.7) 685 (31.4) Fecal sludge 50 cm below the slab No 1285 (81) 441 (74.4) 1726 (79.2) 0.001 Yes 302 (19) 152 (25.6) 454 (20.8) Presence of feces in the compound No 1507 (95) 544 (91.7) 2051 (94.1) 0.005 Yes 80 (5) 49 (8.3) 129 (5.9) None of the above No 1367 (86.1) 580 (97.8) 1947 (89.3) 0.001 Yes 220 (13.9) 13 (2.2) 233 (10.7) 3.4.8 HOUSEHOLD ACCESS TO HYGIENE FACILITIES To evaluate HHs’ access to hygiene facilities, HHs were subject to observational checklists. According to the JMP, a HH is considered to have access to a hand washing facility when there is a mobile or fixed hand washing facility within the premises. Nonetheless, the facility should be supplemented with soap and water. In this regard, a negligible number of HHs in the intervention and non-intervention woredas have fulfilled this definition. As shown in Figure 10, more HHs (77%) had no hygiene facility. Similarly, 75% and 63% of the HH in non-intervention and intervention woredas had no fixed or mobile hand hygiene facility, respectively (Figure 10). PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 107 - - Figure 10: The distribution of household hygiene facilities in Ethiopia, 2022 Only 242 (7.9%) HHs had water for hand washing around the toilet which is proportionally equal in both Transform WASH intervention areas (7.8%) and non-intervention areas (8.1%) (P=0.791). In addition, only 5.9% of HHs in total, 5.7% (132) in WASH intervention areas and 6.4% (49) in non-intervention areas, had soap or detergent. Table 27: Hand washing facilities in Transform WASH intervention areas and non-intervention areas, Ethiopia, 2022 Hand washing Categories Category of study area Total P Value Intervention Non Intervention Have hand￾washing facility No 2088 (90(6) 695 (90.5) 2783 (90.6) 0.941 Yes 217 (9.4) 73 (9.5) 290 (9.4) Water is available No 235 (92.2) 706 (92) 2831 (92.1) 0.791 Yes 180 (7.8) 62 (8.1) 242 (7.9) Presence of soap or detergent Ash, Mud, Sand 4 (0.2) 2 (0.3) 6 (0.2) 0.755 Soap /Detergent 132 (5.7) 49 (6.4) 181 (5.9) Place of hand washing Bucket or Jug near in HH 629 (27.3) 248 (32.3) 877 (28.5) 0.127 Bucket or Jug near latrine 27 (1.2) 6 (0.8) 33 (1.1) No hand washing place in compound/dwelling 1359 (59) 427 (55.6) 1786 (58.1) 3.4.9 HAND WASHING PRACTICE Findings of this assessment revealed that 2,920 (95%), 1,764 (57.04%), 1,556 (50.6),1,685 (54.8%), and 1,700 (55.3%) study participants reported that they were washing their hands before eating food, before food handling/preparation, after visiting the toilet, early morning, and after work, respectively. This study also revealed that more respondents from Transform WASH intervention areas (51.8%) were practicing handwashing after visiting toilet than respondents from non-intervention areas (47%) (P=0.020). Slightly more respondents from Transform WASH intervention areas (16.3%) reported that they have good PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 108 - - handwashing practice than non-intervention areas (14.8%) after cleaning a child’s bottom, though this difference is not significant (p = 0.350). Many participants (1,821, 59.3%) from both intervention areas (54.4%) and non-intervention areas (73.8%) reported that they practice handwashing with water and soap (Table 28). Table 28: Hand washing occasions in Transform WASH intervention areas and non-intervention areas, Ethiopia, 2022 Handwashing occasions Categories Category of study area Total P Value Intervention Non Intervention Before eating food No 122 (5.3) 313 (4) 153 (5) 0.166 Yes 21 (83) 737 (96) 2920 (95) Before food handling/ cooking No 1080 (46.9) 229 (29.8) 1309 (42.6) 0.001 Yes 1225 (53.1) 539 (70.2) 1764 (57.04) Before feeding child No 1772 (76.9) 585 (76.2) 2357 (76.7) 0.689 Yes 533 (23.1) 183 (23.8) 716 (23.3) After latrine use No 1110 (48.2) 407 (53) 1517 (49.4) 0.020 Yes 1195 (51.8) 361 (47) 1556 (50.6) After cleaning child bottom No 1930 (83.7) 654 (85.2) 2584 (84.1) 0.350 Yes 375 (16.3) 114 (14.8) 489 (15.9) After cleaning house No 1678 (72.8) 464 (60.4) 2142 (69.7) 0.001 Yes 627 (27.2) 304 (39.6) 931 (30.3) Before prayer No 1512 (65.6) 543 (70.7) 2055(66.9) 0.009 Yes 793 (34.4) 225 (29.3) 1018 (33.1) Early in the morning No 1103 (47.9) 285 (37.1) 1388 (45.2) 0.001 Yes 1202 (52.1) 483 (62.9) 1685 (54.8) After touching dirt No 1269 (55.1) 325 (42.3) 1594 (51.9) 0.001 Yes 1036 (44.9) 443 (57.7) 1479 (48.1) After work No 1173 (50.9) 200 (26) 1373 (44.7) 0.001 Yes 1132 (49.1) 568 (74) 1700 (55.3) How to wash hands Only water 919 (39.9) 197 (25.7) 1116 (36.3) 0.035 Water & ash/sand 88 (3.8) 4 (0.5) 92 (3) Water and soap 1254 (54.4) 567 (73.8) 1821 (59.3) Others 44 (1.9) - 44 (1.4) 3.5 DIARRHEAL MORBIDITY AMONG YOUNG CHILDREN AND ADULTS Table 29 shows diarrheal morbidity and study participants’ perceptions toward diarrheal morbidity and its frequency. 266 (11.5%) participants reported that in the last two weeks, their children were exposed to diarrheal morbidity. A relatively larger proportion of participants from Transform WASH intervention areas (11.5%) were exposed to diarrheal disease than children from non-intervention areas (6.6%) (p = 0.001). On the other hand, overall, 285 (9.3%) adults had history of diarrheal disease in the last two weeks. An almost equal proportion of adults from HHs in the intervention woredas (9.4%) and non-intervention woredas (9%) were exposed to diarrheal disease (p = 0.749). HHs were also asked if they noticed any change in diarrheal frequency. From all study participants, 627 (20.4%) reported that the frequency of diarrhea among young children was lower compared to previous scenarios. Only 46 (1.5%) respondents described that the frequency of diarrhea was significantly reduced among adults compared to previous encounters. PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 109 - - - - Table 29: Diarrheal morbidity report among young children and adults in Transform WASH intervention and non-intervention areas, Ethiopia, 2022 Diarrhea occurrence Categories Category of study area Total P Intervention Non Value Intervention Children had diarrhea in the last two weeks No 1007 (43.7) 330 (43) 1337 (43.5) 0.001 Yes 266 (11.5) 51 (6.6) 317 (10.3) Adults had diarrhea in the last two weeks No 2089 (90.6) 699 (91) 2788 (90.7) 0.749 Yes 216 (9.4) 69 (9) 285 (9.3) Diarrhea frequency among children Less frequent 423 (18.4) 204 (26.6) 627 (20.4) 0.001 More Frequent 121 (5.2) 33 (4.3) 154 (5) No Change 463 (20.1) 93 (12.1) 556 (18.1) Diarrhea frequency among adults Less frequent 112 (4.9) 57 (7.4) 169 (5.5) 0.001 More Frequent 62 (2.7) 8 (1) 46 (1.5) No change 42 (1.8) 4 (0.5) 46 (1.5) A large proportion of HHs perceived that toilets are a positive influence to reduce exposure to diarrheal and other diseases. Approximately 1,736 (79%) HHs reported that constructing a toilet has a potential impact to reduce exposure to diarrheal diseases (p < 0.001). Though the responses did not significantly vary between intervention and non-intervention areas, 1,321 (60%) of participants believed that having toilet reduces the frequency of diarrheal incidence (p = 0.087). Table 30: Community’s perception on association between having toilet and diarrhea morbidity in Transform WASH intervention and non-intervention areas, Ethiopia, 2022 Community perception on association between having toilet and diarrhea morbidity Characteristics Categ ories Category of study area Total P Intervention Non Value Intervention Having toilet reduces exposure to diarrhea No 388 (24.1) 83 (14) 471 (21.3) 0.001 Yes 1225 (75.9) 511 (86) 1736 (78.7) Reduces frequency of diarrhea occurrence No 665 (41.2) 221 (37.2) 886 (40.1) 0.089 Yes 948 (58.8) 373 (62.8) 1321 (59.9) Reduces exposure to different diseases No 500 (31) 147 (24.7) 647 (29.3) 0.004 Yes 1113 (69) 447 (75.3) 1560 (70.7) Others No 1607 (99.6) 593 (99.8) 2200 (99.7) 0.451 Yes 6 (0.4) 1 (0.2) 7 (0.3) 3.6 GENDER NORMS The analysis of gender and decision-making power was conducted after excluding female-headed HHs. HHs were asked if the husband or wife is responsible for deciding how the HH earnings were spent. More than 50% of respondents reported that the husband’s earnings are spent based on joint decisions made between husband and wife. Only 873 (28.4%) and 539 (17.5%) of the HHs relied on husband’s and wife’s independent decisions, respectively. However, no significant association with study groups was found. On the contrary, women (959, 31%) had more decision-making power than men (701, 23%) when purchasing water, hygiene, and sanitation services (Table 31). PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 110 - - = = = Table 31: Decision on individual income and toilet construction in Transform WASH intervention and non-intervention areas, Ethiopia, 2022 Variables Response Intervention woredas Non intervention Woredas Total P value N 2,305 N 768 N 3,073 Who usually decides how the man of the house’s earnings will be used Husband/Partner Has No Earnings 13 (5.6) 21 (2.7) 151 (4.9) <0.001 Joint Decision 1,074 (46.6) 421 (54.8) 1,495 (48.6) Man Of the House 669 (29.0) 204 (26.6) 873 (28.4) Other 9 (0.4) 6 (0.8) 15 (0.5) Woman Of the House 423 (18.4) 116 (15.1) 539 (17.5) Who usually decides how the woman of the house’s earnings will be used Husband/Partner Has No Earnings 136 (5.9) 38 (4.9) 174 (5.7) <0.001 Joint Decision 1,129 (49.0) 438 (57.0) 1,567 (51.0) Man Of the House 523 (22.7) 125 (16.3) 648 (21.1) Other 7 (0.3) 6 (0.8) 13 (0.4) Woman Of the House 510 (22.1) 161 (21.0) 671 (21.8) Who usually makes decisions about making major HH purchases Husband/Partner Ha 1 (0.01) - 1 (0.01) <0.001 Joint Decision 1,053 (45.7) 398 (51.8) 1451 (47.2) Man Of the House 618 (26.8) 206 (26.8) 824 (26.8) Other 2 (0.1) 1 (0.1) 3 (0.1) Someone Else 3 (0.1) 7 (0.9) 10 (0.3) Woman Of the House 628 (27.2) 156 (20.3) 784 (25.5) Who usually makes decisions about purchasing water, sanitation and hygiene Husband/Partner Ha 2 (0.1) - 2 (0.1) 0.22 Joint Decision 1,000 (43.4) 362 (47.1) 1362 (44.3) Man Of the House 540 (23.4) 161 (21.0) 701 (22.8) Other 5 (0.2) 1 (0.1) 6 (0.2) Someone Else 37 (1.6) 6 (0.8) 43 (1.4) Woman Of the House 721 (31.3) 238 (31.0) 959 (31.2) Mainly responsible person to repair latrine Husband/Partner Has No Earnings 130 (5.6) 32 (4.2) 162 (5.3) <0.001 Joint Decision 794 (34.4) 283 (36.8) 1,077 (35.0) Man Of the House 913 (39.6) 344 (44.8) 1,257 (40.9) Man of The House 1 (0.01) - 1 (0.01) Other 161 (7.0) 13 (1.7) 174 (5.7) Woman of the House 306 (13.3) 96 (12.5) 402 (13.1) PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 111 HHS REPORT ANNEXES HHS ANNEX I: DESCRIPTIVE STATISTICS PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 112 HHS ANNEX II: WEALTH INDEX ANALYSIS PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 113 - - HHS ANNEX III: THE SOURCE OF WATER AND WATER ACCESSIBILITY DISTRIBUTIONS IN WASH AND NON-WASH PROJECT AREAS, ETHIOPIA, 2022 Variables Categories Intervention woreda Non intervention woreda Total P value Source of drinking water Piped Into Dwelling 11 (0.5) 2 (0.3) 13 (0.4) 0.001 Piped To Neighbor 70 (3) 25 (3.3) 95 (3.1) Piped To Yard/Plot 318 (13.8) 112 (14.6) 430 (14.0) Protected Spring 279 (12.1) 78 (10.2) 357 (11.6) Protected Well 37 (1.6) - 37(1.2) Public Tap/Standpipe 718 (31.1) 232 (30.2) 950 (30.9) River/Lake/Pond/Stream 64 (2.8) 23 (3) 87 (2.8) Shallow Borehole Wit 341 (14.8) 157 (20.4) 498 (16.2) Tanker Truck 41 (1.8) - 41 (1.3) Unprotected Spring 302 (13.1) 131 (17.1) 433 (14.1) Unprotected Well 93 (4) 7 (0.9) 100 (3.3) Others* 31 (1.3) 1 (0.1) 32 (1.0) Source of water for cooking Piped Into Dwelling 15 (0.7) 2 (0.3) 17 (0.6) <0.001 Piped To Neighbor 64 (2.8) 23 (3) 87 (2.8) Piped To Yard/Plot 313 (13.6) 112 (14.6) 425 (13.8) Protected Spring 280 (12.1) 81 (10.5) 361 (11.8) Public Tap/Standpipe 684 (29.7) 190 (24.7) 874 (28.4) River/Lake/Pond/Stream 70 (3) 33 (4.3) 103 (3.4) Shallow Borehole Wit 321 (13.9) 159 (20.7) 480 (15.6) Tanker Truck 44 (1.9) - 44 (1.4) Unprotected Spring 318 (13.8) 160 (20.8) 478 (15.6) Unprotected Well 93 (4) 7 (0.9) 100 (3.3) Others+ 103 (4.5) 1 (0.1) 104 (3.4) Source of water for Hand-washing Piped Into Dwelling 13 (0.6) 2 (0.3) 15 (0.5) <0.001 Piped To Neighbor 60 (2.6) 23 (3) 83 (2.7) Piped To Yard/Plot 312 (13.5) 112 (14.6) 424 (13.8) Protected Spring 268 (11.6) 83 (10.8) 351 (11.4) Public Tap/Standpipe 653 (28.3) 183 (23.8) 836 (27.2) River/Lake/Pond/Stream 94 (4.1) 40 (5.2) 134 (4.4) Shallow Borehole Wit 301 (13.1) 151 (19.7) 452 (14.7) Tanker Truck 43 (1.9) - 43 (1.4) Unprotected Spring 340 (14.8) 166 (21.6) 506 (16.5) Unprotected Well 103 (4.5) 7 (0.9) 110 (3.6) Others 118 (5.1) 1 (0.1) 119 (3.9) *Rainwater, Tanker Truck, Water Kiosk, Deep borehole; +Water Kiosk, Cart with a Small Tank, Deep Borehole, Protected Well, Rainwater PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 114 HHS REPORT ANNEX IV: ACCESS TO WATER SERVICES IN 16 WOREDAS, 2022, ETHIOPIA List of woredas Number of households Basic water Limited water Unimproved water Surface water Aleta Wondo N=217 65 (30.0%) 108 (49.8%) 44 (20.3%) 0 (0.0%) Ali Booko N=192 39 (20.3%) 71 (37.0%) 73 (38.0%) 9 (4.7%) Angollala Tera N=192 95 (49.5%) 92 (47.9%) 4 (2.1%) 1 (0.5%) Dega Bour N=217 71 (32.7%) 41 (18.9%) 58 (26.7%) 47 (21.7%) Dodolla N=179 56 (31.3%) 104 (58.1%) 19 (10.6%) 0 (0.0%) Gedeb Assesa N=192 19 (9.9%) 122 (63.5%) 38 (19.8%) 13 (6.8%) Gursum N=217 53 (24.4%) 82 (37.8%) 72 (33.2%) 10 (4.6%) Guzamen N=180 61 (33.9%) 39 (21.7%) 78 (43.3%) 2 (1.1%) Kalu N=180 49 (27.2%) 131 (72.8%) 0 (0.0%) 0 (0.0%) Kindo Dedaye N=180 124 (68.9%) 36 (20.0%) 20 (11.1%) 0 (0.0%) Merab Badiwacho N=180 73 (40.6%) 103 (57.2%) 1 (0.6%) 3 (1.7%) Mille N=215 69 (32.1%) 81 (37.7%) 62 (28.8%) 3 (1.4%) Mirab Abaya N=192 60 (31.3%) 109 (56.8%) 23 (12.0%) 0 (0.0%) Were Elu N=180 66 (36.7%) 109 (60.6%) 1 (0.6%) 4 (2.2%) Woliso N=180 102 (56.7%) 61 (33.9%) 16 (8.9%) 1 (0.6%) Wondo N=180 81 (45.0%) 75 (41.7%) 24 (13.3%) 0 (0.0%) Total N=3,073 1,083 (35.2%) 1,364 (44.4%) 533 (17.3%) 93 (3.0%) PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 115 HHS REPORT ANNEX V: ACCESS TO SANITATION SERVICES IN 16 WOREDAS, 2022, ETHIOPIA List of woredas Number of households Basic Limited Unimproved Open Defecation Aleta Wondo N=217 81 (37.3%) 4 (1.8%) 124 (57.1%) 8 (3.7%) Ali Booko N=192 11 (5.7%) 0 (0.0%) 140 (72.9%) 41 (21.4%) Angollala Tera N=192 3 (1.6%) 3 (1.6%) 82 (42.7%) 104 (54.2%) Dega Bour N=217 14 (6.5%) 16 (7.4%) 7 (3.2%) 180 (82.9%) Dodolla N=179 31 (17.3%) 1 (0.6%) 130 (72.6%) 17 (9.5%) Gedeb Assesa N=192 36 (18.8%) 8 (4.2%) 130 (67.7%) 18 (9.4%) Gursum N=217 44 (20.3%) 7 (3.2%) 16 (7.4%) 150 (69.1%) Guzamen N=180 28 (15.6%) 3 (1.7%) 92 (51.1%) 57 (31.7%) Kalu N=180 30 (16.7%) 16 (8.9%) 107 (59.4%) 27 (15.0%) Kindo Dedaye N=180 14 (7.8%) 6 (3.3%) 150 (83.3%) 10 (5.6%) Merab Badiwacho N=180 59 (32.8%) 4 (2.2%) 96 (53.3%) 21 (11.7%) Mille N=215 15 (7.0%) 48 (22.3%) 6 (2.8%) 146 (67.9%) Mirab Abaya N=192 27 (14.1%) 17 (8.9%) 136 (70.8%) 12 (6.3%) Were Elu N=180 65 (36.1%) 8 (4.4%) 94 (52.2%) 13 (7.2%) Woliso N=180 53 (29.4%) 2 (1.1%) 56 (31.1%) 69 (38.3%) Wondo N=180 78 (43.3%) 7 (3.9%) 75 (41.7%) 20 (11.1%) Total N=3,073 589 (19.2%) 150 (4.9%) 1,441 (46.9%) 893 (29.1%) PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 116 HHS REPORT ANNEX VI: ACCESS TO MOBILE AND FIXED HAND HYGIENE FACILITY IN 16 WOREDAS, 2022, ETHIOPIA List of woredas Number of households Fixed hand hygiene facility Mobile hand hygiene facility No hand hygiene facility Aleta Wondo N=217 0 (0.0%) 67 (30.9%) 150 (69.1%) Ali Booko N=192 0 (0.0%) 33 (17.2%) 159 (82.8%) Angollala Tera N=192 0 (0.0%) 97 (50.5%) 95 (49.5%) Dega Bour N=217 0 (0.0%) 0 (0.0%) 217 (100.0%) Dodolla N=179 0 (0.0%) 90 (50.3%) 89 (49.7%) Gedeb Assesa N=192 0 (0.0%) 118 (61.5%) 74 (38.5%) Gursum N=217 0 (0.0%) 0 (0.0%) 217 (100.0%) Guzamen N=180 0 (0.0%) 92 (51.1%) 88 (48.9%) Kalu N=180 1 (0.6%) 33 (18.3%) 146 (81.1%) Kindo Dedaye N=180 0 (0.0%) 5 (2.8%) 175 (97.2%) Merab Badiwacho N=180 1 (0.6%) 77 (42.8%) 102 (56.7%) Mille N=215 0 (0.0%) 78 (36.3%) 137 (63.7%) Mirab Abaya N=192 0 (0.0%) 6 (3.1%) 186 (96.9%) Were Elu N=180 0 (0.0%) 23 (12.8%) 157 (87.2%) Woliso N=180 1 (0.6%) 93 (51.7%) 86 (47.8%) Wondo N=180 0 (0.0%) 98 (54.4%) 82 (45.6%) Total N=3,073 3 (0.1%) 910 (29.6%) 2,160 (70.3%) PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 117 HHS REPORT ANNEX VII. OPERATIONAL DEFINITIONS Surface water: Drinking water directly from a river, dam, lake, pond, stream, canal or irrigation canal. Unimproved water service: Drinking water from unprotected sources including dug well or unprotected springs. Limited water service: Drinking water from an improved source for which collection time exceeds 30 minutes for the roundtrip, including queuing. Basic water service: Drinking water from an improved source, provided collection time is not more than 30 minutes for a roundtrip, including queuing. Improved drinking water sources: Are those which by nature of their design and construction have the potential to deliver safe water. This includes piped water, boreholes or tube wells, protected dug wells, protected springs, rainwater, and packaged or delivered water. Basic sanitation service: Use of improved sanitation facilities that are not shared with other households, and that have a superstructure providing adequate privacy and a drop hole cover or a water seal that prevents the passage of flies and odors. Limited sanitation service: Percent of households that use improved latrines that are shared with others. Unimproved sanitation service: Percent of households that use pit latrines without a cleanable floor/ slab/platform or platform, hanging latrines or bucket latrines. Open defecation: Percent of HHs that dispose human faces in fields, forests, bushes, open Improved sanitation facilities: These include wet sanitation technologies such as flush and pour flush toilets connected to sewers, septic tanks or pit latrines, and dry sanitation technologies such as dry pit latrines with slabs and composting toilets. PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 118 ANNEX E: LIST OF KEY INFORMANT INTERVIEWS AND FOCUS GROUP DISCUSSIONS E-1.0 TRANSFORM: WASH CONSORTIUM PARTNERS • Population Services International (PSI) - Transform WASH Chief of Party (Mr. Monte Achenbach) - Transform WASH Deputy Chief of Party (Mr. Mechael Negash) - Transform WASH Project Manager (Mr. Daniel Tesfaye) - Transform WASH Business Manager (Mr. Dagim Demirew) - Gender Expert (Ms. Meseret Desalegn) • Plan International Ethiopia - WASH SBCC Project Manager (Mr. Getachew Asradew) - WASH Advisor (Mr. Tesfaye Mekonnen) • SNV—Transform: WASH Institutional Development and Capacity-Building Manager (Mr. Shegaw Fante) • IRC WASH: - Transform WASH: Increased Knowledge Base to Bring Innovations to Scale Manager (Mr. Lars Osterwalder) - Learning and Engagement Coordinator (Mr. Melaku Worku) - Policy and Research Advisor (Ms. Bethelhem H/Giorgis) - Communications Advisor (Mr. Tesgaye Yeshiwas) E-2.0 NATIONAL-LEVEL KEY INFORMANT INTERVIEWS (KIIS) • Federal Ministry of Health - Environmental Health and Hygiene Directorate, Director (Dr. Ekram Redwan) - Hygiene and Sanitation, Acting Case Team Leader (Mr. Wondayehu Wubie) • Technical And Vocational Educational Training (TVET) Commission (Mr. Ashenafi) • Micro-finance institution (MFI) – Vision Fund (Mr. Hzekiel Aynalem) • Lixil International (Mr. Jason Cardosi) • USAID Ethiopia Mission - Nutrition Coordinator and WASH Team Lead (Dr. Alia El Mohandes) - Strategic Information Advisor (Mr. Gebeyehu Abelti) - WASH Advisor (Mr. Mequanent Fentie) - WASH Advisor (Mr. Yunis Mussema) - WASH Advisor (Mr. Dejene Kumela) - WASH Advisor (Mr. Faisel Hashi) • WaterAid, Program Director (Mr. Abireham Misganaw) • Community-led Accelerated WASH (CoWASH) SME Development and Financial Access Specialist (Mr. Mulate Yinager) • iDE Ethiopia (Ms. Marta Gebeyehu) PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 119 E-3.0 SUBNATIONAL-LEVEL KEY INFORMANT INTERVIEWS (FOR EACH REGIONAL SECTOR OFFICE AND DIRE DAWA ADMINISTRATION) ORGANIZATION RESPONSIBILITY DODOLA WOREDA Herero Heath Center Primary Health Care Unit Manager Sanitation Business Partner Small & Medium Enterprises (SME) Chairperson Health Baka Kebele Health Extension Worker (HEW) Retailer Owner Retailer (Inactive) Owner Health office WASH Neglected Tropical Diseases (NTD) Expert and Transform WASH Focal Person Enterprise Development / Job Creation Office Certification and Legal O fficer WONDO WOREDA Desta and Godegnochu SME Manufacturer & installer Health Office Intaye Kebele HEW Sinke Bank Branch Manager WOLISO WOREDA Health NTD & WASH Focal Person TVET College Construction technology instructor Enterprise Development / Job Creation Office WASH Focal Person Health Tombe Chebi Kebele HEW Building material shop Owner Tejitu Zinash SME Member of SME ALETA WONDO Aleta Wondo TVET Construction technology instructor Health O fficeWASH Focal Person ED&JB SMEs organizer officer Health Dobe Kebele HEW SME SME Chairperson (Female) SME SME Chairperson (Female) OMO micro-finance institution (MFI) Branch Manager Retailer Owner MIERAB BEDEWACHO Health Kachabira Kebele HEW Fetan SME Chairperson (Female) Health WASH Focal Person OMO Bank MFI Branch Manager Enterprise Development Head Women’s Health Development Army Sales Agent (SA) Private Manufacturer Tsige SME Chairperson (Female) PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY –REPORT ANNEXES |120 ORGANIZATION RESPONSIBILITY KINDO DIDAYE Health WASH Focal Person OMO MFI Operation Manager Enterprise Development / Job Creation Office Team leader Water, Irrigation & Mine WASH Focal Person Weebta SME Chairperson Sira Hiwot SME Chairperson MIERAB ABAYA Health Office HEW Hahu SME Chairperson SIDAMA REGION Regional Health Bureau WASH Focal Person Regional Enterprise Development / Job Creation Office Rural Youth Access to Income Development and Market linkage Director Hawassa Sanitation product wholesaler Merchant Regional TVET Bureau Senior Expert SNNP REGION Regional Health Bureau WASH Focal Person Plan International Ethiopia Social and Behavior Change Communication (SBCC) Regional Coordinator OWNP Coordination Office Coordinator TVET Bureau Trainer of Trainees and Leadership Development Officer Enterprise Development / Job Creation Office Job Creation Senior Officer OROMIA REGION Enterprise Development / Job Creation Office and TVET Senior Officer Sinke Bank Program and Stakeholder Relation Director Plan International Ethiopia SBCC Regional Coordinator PSI Associate Business Development Manager SNV Capacity Building Advisor FEDERAL CoWASH SME Development & Financial Access Specialist IDE WASH Manager DODOLA FOCUS DISCUSSION GROUPS (FDGs) Households (HHs) mobilized and adopted improved sanitation – five members of Baka Kebele Community HHs mobilized but not adopted improved sanitation – five members of Baka Kebele Community WONDO HHs mobilized and adopted improved sanitation – five members of Intaye Kebele Community HHs mobilized but not adopted improved sanitation – five members of Intaye Kebele Community Intaye Village Savings and Loan Association (VSLA) FGD – seven members PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY –REPORT ANNEXES |121 ORGANIZATION RESPONSIBILITY WOREDA WASH TEAM (WWT) Administration Deputy chief administrator Health office Head Health office WASH Focal Point and Transform WASH Technical Working Group (TWG) Water & Monitoring and Evaluation WASH Focal Point and Transform WASH TWG Education High School instructor WOLISO HHs mobilized and adopted improved sanitation – five members of the Werabu B. Kebele HHs mobilized but not adopted improved sanitation – four members of the Werabu B. Kebele WOREDA WASH TEAM (WWT) Health OfficeHead Water OfficeHead Women’s Affairs Office Head Education OfficeWASH Focal Person Enterprise Development / Job Creation Office Head ALETA WONDO HHs mobilized and adopted improved sanitation – six members of Habaja Kebele community HHs mobilized but not adopted improved sanitation group interview – five members of Habaja Kebele community Dobe Sedeka cluster VSLA FGD – six members of Dobe Sedeka Kebele WOREDA WASH TEAM (WWT) Administration Chief Administrator Health OfficeHead WASH / Monitoring and Evaluation Office WASH Focal Person Education OfficeDeputy Head Education Office WASH Focal Person Enterprise Development JC Head MIERAB BEDEWACHO HHs mobilized and adopted improved sanitation – six members of Denema Kebele community HHs mobilized but not adopted improved sanitation – five members of the Denema Kebele community Kachabira Kebele VSLA FGD – six members WOREDA WASH TEAM (WWT) Administration Administration Representative Health OfficeDeputy Head Enterprise Development Office Office Water, Irrigation & Energy OfficeHead KINDO DIDAYE HHs mobilized and adopted improved sanitation – five members of the Sime Dolaye Kebele community HHs mobilized but not adopted improved sanitation – five members of the Sime Dolaye Kebele community PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY –REPORT ANNEXES |122 ORGANIZATION RESPONSIBILITY WOREDA WASH TEAM (WWT) Administration Chief Administrator Water, Irrigation & Energy Office Head Education Office WASH Focal Person Education Office Deputy Head Health Office Deputy Head Finance Office Deputy Head SNNPR PSI AND SNV PSI Integrated Field Implementation Manager PSI WASH Business Development Associate Manager PSI Monitoring and Evaluation Advisor PSI Business Advisor PSI Business Advisor SNNPR SNV Capacity Building Advisor WOLISO WOREDA HHs mobilized and adopted improved sanitation – five members of the Worabu Beri Kebele community HHs mobilized but not adopted improved sanitation – four farmer customers WOREDA WASH TEAM (WWT) Health OfficeHead Water OfficeHead Women’s Affairs Office Head Education Office WASH Focal Person Enterprise Development / Job Creation Office Head DODOLA WASH & Monitoring and Evaluation Office Transform WASH Focal Point Health Office Deputy Head Health Office Head WONDO Enterprise Development / Job Creation Office Core Process Head Enterprise Development / Job Creation Office Planning Expert GEDEB ASASSA Health OfficeHead Health Office Sanitation & Health Officer ALETA WONDO Sanitation SME SME Chair Lady Farming Sales Agent Sanitation SME Sales Agent & installer KINDO DIDAYE Health Office HEW PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY –REPORT ANNEXES |123 ORGANIZATION RESPONSIBILITY Health Office HEW MIRAB ABAYA WASH & Monitoring and Evaluation Deputy Head WASH & Monitoring and Evaluation WASH Focal person Health Office Head Health O fficeDeputy Head Enterprise Development / Job Creation Office Head Enterprise Development / Job Creation Office Deputy Head OROMIA Regional Health Bureau Hygiene and Environmental Health (HEH) & NTD Directorate Director Regional Health Bureau Regional Coordination Office (RCO) Coordinator Oromia OWNP RCO Coordinator Oromia OWNP HEH Office Oromia OWNP Water Engineer AMHARA BAHIRDAR (REGIONAL LEVEL) Health OfficeSanitation and Hygiene Case Team Head Health Office Sanitation and Hygiene Officer PSI Regional PSI Office Coordinator PSI Business Advisor PSI Associate WASH business manager TVET/Job Creation Office Expert TVET/Job Creation OfficeSkill and training officer SNV Capacity Building Advisor Cooperative Bureau Head MFI Manager Regional Health Bureau RCO Coordinator Retailer Owner AMHARA DESSIE (ZONE) Health South Wolo Zone WASH Coordinator College College Dean College Construction Department Instructor and MBS Focal Point Union BankHead PSI Regional Coordinator PSI Associate business Facilitator Dessie Amba Saving and Credit Cooperative Union Manager Retailer Owner AMHARA KALU Health OfficeSanitation and Hygiene expertise PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY –REPORT ANNEXES |124 ORGANIZATION RESPONSIBILITY TVET/Job Creation OfficeJob creation and construction sector specialist Retailer Owner Cooperative Manager Sales Agent Sales Agent HEW HEW VSLA VSLA Facilitator VSLA VSLA Facilitator Kebele Chair Kebele Chairman AMHARA WERE ILU Health Sanitation and Hygiene Officer and Technical Team Chair TVET/Job Creation Center Head Retailer Owner Sales Agent Sales Agent HEW HEW AMHARA ALBUKO Health Office Sanitation and Hygiene Officer TVET/Job Creation Center Head HEW Health Center Head MFI Customer Relations Officer AMHARA ANGOLLELA TERA Health Office Sanitation and Hygiene Officer TVET/Job Creation Center Head HEW HEW AMHARA GOZAMEN Health Sanitation and Hygiene Officer TVET/Job Creation Center Job Creation and Construction Specialist and Transform WASH Focal Person Enterprise Enterprise chair Enterprise Member Union Manager Wholesaler Owner Sales Agent/ Mason Sales Agent HEW HEW VSLA VSLA facilitator SOMALI JIJIGA Health HEH Core Process Owner TVET College Construction Teacher and Transform WASH Focal Person Job Creation Office Skill and Job Creation Head PSI PSI Somali Regional Coordinator SNV SNV Somali Regional Advisor PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 125 ORGANIZATION RESPONSIBILITY MFI (Shebele Bank) Operation Department Technical Expert MFI (Shebele Bank) Operation Department Technical Expert Wholesaler Manager RCO Technical Expert SOMALI DEGE HABOUR Health OfficeHealth Promotion Coordinator Health OfficeSanitation and Hygiene Officer TVET/Job Creation Center Head TVET/Job Creation Center Technical Expert Cooperative Head MFI Technical Expert Woreda PSI PSI Business Facilitator Kebele Kebele Chairman SA/Mason SA/Mason Health Office HEW AFAR SEMERA (REGIONAL LEVEL) Health OfficeSanitation and Hygiene Acting Director Health Office Liquide and Solid Waste Case Team Lead TVET/Job Creation Center Technical Expert RWCO Regional Coordinator RWCO Education Bureau WASH Focal Person AFAR MILLE Health Sanitation and Hygiene Officer Health HEW Case Team Lead Job Creation Center Technical Expert SA/Mason Mason HEW HEW Retailer Owner MFI Afar MFI Mille Branch Manager DIRE DAWA Health Sanitation and Environmental Health E xpert Job Creation Center Rural Job Creation Coordinator TVET College Dean RWCO Coordinator MFI Loan Operation Manager Wholesaler Wholesaler PSI Regional facilitator FGD AT DEGE HABOUR WOREDA WWT - Five woreda WASH sector office representatives Hodale Kebele HHs mobilized and adopted improved sanitation - five community members PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 126 ORGANIZATION RESPONSIBILITY Hodale Kebele HHs mobilized but not adopted improved sanitation - five community members FGD AT GOZAMEN WOREDA WWT – six woreda WASH sector office representatives Fendika Kebele HHs mobilized and adopted improved sanitation – eight community members Fendika Kebele HHs mobilized but not adopted improved sanitation - six community members VSLA group – seven VSLA members FGD AT KALLU WOREDA WWT – Seven woreda WASH sector office representatives 01 Addis Mender Kebele HHs mobilized and adopted improved sanitation - eight community members 01 Addis Mender Kebele HHs mobilized but not adopted improved sanitation - five community members VSLA group – two VSLA group FGD AT WERE ELLU WOREDA WWT – 10 woreda WASH sector representatives 09 Kebele HHs mobilized and adopted improved sanitation - six community members 09 Kebele HHs mobilized but not adopted improved sanitation - six community members FGD AT MILLE WOREDA WWT – Five woreda WASH sector representatives Bekeli dear Kebele HHs mobilized and adopted improved sanitation - five community members PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY –REPORT ANNEXES |127 ANNEX F: FULL LIST OF TRANSFORM WASH WOREDAS AND SELECTION OF WOREDAS FOR PE TEAM VISIT TABLE F -1: TRANSFORM: WASH WOREDAS S.N REGION ZONE WOREDA PHASE Southern Hadiya Shashego 1 Suro 1 1 Nations, Nationalities, and Peoples’ Region (SNNPR) Merab Badiwacho 1 Wolayta Kindo Dedaye 1 Boloso Bonbe 1 Humbo 1 Subtotal 6 2 Sidama Sidama Aleta Wondo 1 Aleta Chuko 1 Malega 1 Subtotal 3 3 Oromia West Shoa Ada’a Berga 2 bako Tibbee 2 Ambo Zuria 2 West Arsi Dodola 2 Wondo 2 Heban Arsi 2 Horro/Guduru Wallagga Horro Buluke 3 Chomman Guduru 3 Guduru 3 Southwest Shoa Dawo 3 Gorro 3 Woliso 3 East Wallaga Bonya Boshe 3 Subtotal 13 4 Amhara East Gojam Debaye Tilatehen 2 Guzamen 2 South Wello Desseia Zuria 2 Kalu 2 Tehuledere 3 Were Ellu 3 West Gojam Nort Mecha 3 Quarit 3 Bahirdar Zuria 3 Subtotal 9 6 Tigray South Tigray Raya Azebo 3 East Tigray Kilta Awelalao 3 PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY –REPORT ANNEXES |128 TABLE F -1: TRANSFORM: WASH WOREDAS S.N REGION ZONE WOREDA PHASE Southeast Hintalo Wajerat 3 Subtotal 3 7 Somali Fafan Gursum 3 Jarara Ararso 3 Dega Bour 3 Subtotal 3 8 Afar Zone 1 Mille 3 9 Gambella Anyuaa Abobo 3 10 B/Gumuz Metekel Dibate 2 11 Dire Dawa Byo Awale 3 Total 41 Selection Criteria of Woredas for Evaluation Team Visit Intervention phases. The Transform WASH activity has been implemented in three phases. Phase 1 was delivered in nine woredas in the Southern Nations, Nationalities, and Peoples’ Region and Sidama Regions, Phase 2 in 11 woredas of Benishangul-Gumuz, Amhara and Oromia Regions, and the remaining 21 woredas from Afar, Gambella, Somali, Tigray, Oromia, and Amhara Regions are covered in Phase 3. The evaluation team balanced woreda selection including three from Phase 1, four from Phase 2, and five from Phase 3. Adequacy of technical support. Due to the phased implementation approach, woredas in Phase 3 and located in remote areas did not receive equal technical support from the project, primarily due to the conflict, and thus were not selected by the evaluation team. Livelihood differences. Although all intervention woredas are predominantly rural, there are livelihood differences. Four woredas in Afar and Somali regions are pastoralist, one woreda in Gambella region is semi-pastoralist, and the remaining 36 intervention woredas are agrarian. As a representative sample, the evaluation team chose two pastoralists and the remaining woredas as agrarian. Remoteness and market connectivity. Along with ensuring geographic diversity across the seven regions, the evaluation team recognizes that there might be a key difference between how market-based approaches perform in more remote areas that are less connected to market centers versus more connected areas. Thus, the evaluation team was keen to understand how Transform WASH reached at least two more remote woredas and thus included Kindo Dedaye in SNNPR and Were Ellu in Amhara. Presence of other WASH development partner activities. While some of the intervention woredas are receiving additional WASH project support from other development partners, other woredas are merely supported by the Transform: WASH project. The evaluation team has included woredas from both categories to determine if there are differences in performance. Woreda performance status. According to Transform WASH’s own data analysis, there is some disparity between the performance of woredas across the portfolio within the same and different regions either due to later commencement of project interventions or recurrent instabilities related to conflict and drought. The evaluation team was thus keen to understand factors that might influence performance and thus noted this as a key criterion for selection using data on sales. PERFORMANCE EVALUATION OF ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNEXES | 129 PER XES | 130 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523 Tel: (202) 712 -0000 Fax: (202) 216 -3524 FORMANCE EVALUATION Owww.usaid.gov F ETHIOPIA TRANSFORM WASH ACTIVITY – REPORT ANNE