PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 1 PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY DECEMBER 2019 This publication was produced for review by the United States Agency for International Development. It was prepared by Tetra Tech. ACKNOWLEDGEMENTS The performance evaluation was conducted by Ken Caplan (Team Leader, Independent), Siobhan Girling (WASHPaLS Technical Operations and Finance Manager, Tetra Tech), Benjamin Arthur (WASH Governance Specialist, Independent), Bertha Darteh (Deputy Team Leader and WASH Specialist, Independent), Jemilatu Mashood (Community-Led Total Sanitation and Behavior Change Specialist, Independent), and Emmanuel Baapeng (Research Specialist, Independent). Each team member made meaningful contributions to this report. The evaluation team would like to acknowledge the invaluable contributions from Clinton Sears (Tetra Tech) for providing guidance in using the Dedoose platform and to Morris Israel and Miriam Otoo (WASHPaLS Project Director and Senior Associate, respectively, Tetra Tech) for peer reviewing the final draft report. We extend a sincere thanks to Water, Sanitation, and Hygiene for Health staff at national and county levels for their close collaboration and cooperation in the planning and execution of the review. Without their assistance and flexibility, the evaluation team would not have been able to complete the review in the timeframe allotted. The team would also like to thank the numerous key informants at the national; Metropolitan, Municipal, and District Assembly; and community levels who participated openly and thoughtfully in the interviews and focus group discussions, and to USAID/Ghana—Emmanuel Odotei in particular—for providing oversight and guidance throughout and to Ryan Mahoney of USAID/Washington for thoughtful comments on earlier drafts. Authors: Ken Caplan, Benjamin Arthur, Bertha Darteh, Jemilatu Mashood, Emmanuel Baapeng Preferred citation: USAID, 2019. Performance Evaluation of The Ghana WASH for Health (W4H) Activity. Washington, DC., USAID Water, Sanitation, and Hygiene Partnerships and Learning for Sustainability (WASHPaLS) Project. Prepared for the United States Agency for International Development by the Water, Sanitation, and Hygiene Partnerships and Learning for Sustainability (WASHPaLS) project under Task Order number AID-OAA-TO-16-00016 of the Water and Development Indefinite Delivery Quantity Contract (WADI), contract number AID-OAA-I-14-00068. Cover/Internal Photo Credits: USAID WASHPaLS Project Tetra Tech Contacts: Morris Israel, Project Director morris.israel@washpals.org Jeff Albert, Deputy Project Director jeff.albert@washpals.org Lucia Henry, Project Manager lucia.henry@tetratech.com Tetra Tech 1320 N. Courthouse Road, Suite 600, Arlington VA 22201 Tel: (703) 387-2100, Fax: (703) 414-5593 www.tetratech.com/intdev PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY DECEMBER 2019 DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY i TABLE OF CONTENTS TABLE OF CONTENTS................................................................................................................................................... i List of Tables........................................................................................................................................................... iii List of Figures......................................................................................................................................................... iii ACRONYMS AND ABBREVIATIONS ....................................................................................................................... iv EXECUTIVE SUMMARY................................................................................................................................................ vii 1.0 GLOBAL COMMUNITIES’ W4H ACTIVITY..................................................................................................1 1.1 W4H BACKGROUND .............................................................................................................................1 1.2 W4H DEVELOPMENT HYPOTHESIS...................................................................................................1 2.0 EVALUATION PURPOSE, METHODS, AND LIMITATIONS....................................................................4 2.1 EVALUATION PURPOSE.........................................................................................................................4 2.2 AUDIENCE AND INTENDED USES.....................................................................................................4 2.3 EVALUATION QUESTIONS ...................................................................................................................4 2.4 EVALUATION METHODOLOGY.........................................................................................................5 2.5 SECONDARY DATA COLLECTION – DESK REVIEW..................................................................6 2.6 PRIMARY DATA COLLECTION ...........................................................................................................7 2.6.1 Selection of W4H MMDAs for Primary Data Collection ......................................................7 2.6.2 Key Informant Interviews and Focus Group Discussions......................................................8 2.6.3 Key Informant Interview and Focus Group Discussion Guides.........................................10 2.6.4 Site Observations...........................................................................................................................11 2.7 DATA ANALYSIS METHODS...............................................................................................................11 2.8 CAVEATS AND LIMITATIONS............................................................................................................11 3.0 FINDINGS AND CONCLUSIONS.................................................................................................................13 3.1 OVERVIEW ................................................................................................................................................13 3.2 EQ1: POLICY GAPS AND THE PRO-POOR GUIDELINES .........................................................14 3.2.1 Existing Policy Gaps.......................................................................................................................14 3.2.2 The Pro-Poor Sanitation Guidelines..........................................................................................23 3.3 EQ2: SUSTAINABILITY OF SANITATION ACHIEVEMENTS......................................................25 3.3.1 Durable Construction Materials And Local Market Options For Toilet Construction27 3.3.2 Community-Level Capacity For Messaging And Enforcement............................................28 3.3.3 Involvement Of Local Authorities In Implementation And Monitoring............................29 3.3.4 Sufficient Quality Of The Initial Triggering Effort...................................................................30 3.3.5 Sufficient Post-Odf Follow-Up And Support...........................................................................32 3.3.6 Overarching Conclusions – EQ2 ...............................................................................................32 PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY ii 3.4 EQ3: SUCCESSFUL APPROACHES TO PRIVATE SECTOR ENGAGEMENT FOR SANITATION............................................................................................................................................33 3.4.1 Forms Of Engagement With The Private Sector...................................................................33 3.4.2 Marketing Of The Digni-Loo.......................................................................................................35 3.4.3 Sanitation Subsidies........................................................................................................................37 3.4.4 Other Products On The Market................................................................................................39 3.4.5 Overarching Conclusions – EQ3 ...............................................................................................39 3.5 EQ4: STRENGTHENING LOCAL GOVERNANCE TO SUSTAIN WATER SERVICES........41 3.5.1 MMDA level ....................................................................................................................................41 3.5.2 Community-Level Services..........................................................................................................43 3.5.3 Overarching Conclusions – EQ4 ...............................................................................................43 3.6 EQ5: INCREASE IN BASIC HANDWASHING BEHAVIORS .......................................................45 3.6.1 Positioning Of W4h Interventions Regarding HandWashing ..............................................45 3.6.2 Findings On Progress And Achievement Of Household Hygiene......................................46 3.6.3 WASH In Schools..........................................................................................................................47 3.6.4 WASH In Healthcare Facilities...................................................................................................49 3.6.5 Overarching Conclusions – EQ5 ...............................................................................................50 4.0 RECOMMENDATIONS .....................................................................................................................................52 4.1 SHORT-TERM RECOMMENDATIONS .............................................................................................52 4.2 LONG-TERM RECOMMENDATIONS...............................................................................................55 ANNEX I: EVALUATION STATEMENT OF WORK............................................................................................57 ANNEX II: EVALUATION METHODS .....................................................................................................................71 ANNEX III: DATA COLLECTION INSTRUMENTS..............................................................................................73 ANNEX IV: DATA SOURCES .....................................................................................................................................95 ANNEX V: DISCLOSURE OF ANY CONFLICTS OF INTEREST................................................................... 102 PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY iii LIST OF TABLES Table 1: Data Sources, Data Collection, and Analysis Methods for Each Evaluation Question....................5 Table 2: MMDAs Selected for Evaluation Team Visit...............................................................................................8 Table 3: Key Informant Sources and Observation Sites.........................................................................................10 Table 4: Summary of High-Level WASH Sector Policies.......................................................................................16 Table 5: Institutions with Roles and Responsibilities for WASH.........................................................................20 LIST OF FIGURES Figure 1: W4H Results Framework...............................................................................................................................2 Figure 2: W4H Districts Visited.....................................................................................................................................7 Figure 3: Simplified Framework of WASH Sector Coordination and Institutional Relationships................22 PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY iv ACRONYMS AND ABBREVIATIONS AMEP Activity Monitoring and Evaluation Plan AOB Asikuma Odoben Brakwa BaSIS Basic Sanitation Information System BCC Behavior Change Communication BDS Business Development Services C4H Communicate for Health CHPS Community Health Planning System CLTS Community-Led Total Sanitation CSR Corporate Social Responsibility CWSA Community Water and Sanitation Agency DA District Assembly DACF District Assembly Common Fund DEHO District Environmental Health Officer DESSAP District Environmental Sanitation Strategy and Action Plan DICCS District Inter-Agency Coordinating Committee for Sanitation DP Development Partner DWST District Water and Sanitation Team EHA Environmental Health Assistant EHO Environmental Health Officer EHSD Environmental Health and Sanitation Directorate EHSU Environmental Health and Sanitation Unit EPA Environmental Protection Agency EQ Evaluation Question ESP Education Strategic Plan FGD Focus Group Discussion FMP Facility Management Plan FY Fiscal Year GC Global Communities GES Ghana Education Service PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY v GOG Government of Ghana GWCL Ghana Water Company Limited HCF Healthcare Facility IP Implementing Partner KII Key Informant Interview LEAP Livelihood Empowerment Against Poverty LG Local Government M&E Monitoring and Evaluation M/DESSAP MMDA Water and Sanitation Plan M/DICCS MMDA Inter-Agency Coordinating Committee for Sanitation MDG Millennium Development Goal MLGRD Ministry of Local Government and Rural Development MMDA Metropolitan, Municipal, and District Assembly MoE Ministry of Education MoH Ministry of Health MOU Memorandum of Understanding MSWR Ministry of Sanitation and Water Resources NCWSP National Community Water and Sanitation Programme NESP National Environmental Sanitation Policy NGO Nongovernmental Organization NL Natural Leader NSA National Sanitation Authority NWP National Water Policy O&M Operations and Maintenance OD Open Defecation ODF Open Defecation Free PSI Population Services International PTA Parent Teacher Association PURC Public Utility and Regulatory Commission RCC Regional Coordinating Council RICCS Regional Inter-Agency Coordinating Committee on Sanitation PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY vi RSMS Rural Sanitation Model and Strategy S4H Systems for Health SDG Sustainable Development Goal SHEP School Health Education Program SMC School Management Committee SOW Statement of Work STK Sawla Tuna Kalba TFHO Total Family Health Organization USAID United States Agency for International Development VIP Ventilated Improved Pit W4H Water, Sanitation, and Hygiene for Health WASH Water, Sanitation, and Hygiene WinS WASH in Schools WSMT Water and Sanitation Management Team PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY vii EXECUTIVE SUMMARY ACTIVITY BACKGROUND The Water, Sanitation, and Hygiene for Health (W4H) Activity is a five-year (February 2015–September 2020) US$19M cooperative agreement funded by USAID/Ghana and implemented by Global Communities (GC) in partnership with The Manoff Group and USAID Global Development Alliance partners Rotary International and The Coca-Cola Africa Foundation. USAID/Ghana commissioned the USAID Water, Sanitation and Hygiene Partnerships and Learning for Sustainability (WASHPaLS) to undertake this performance evaluation at the start of the activity’s final year. The goal of W4H is to accelerate sustainable improvement in water and sanitation access and improve hygiene behaviors in 15 target Metropolitan, Municipal, and District Assemblies (MMDAs). By the end of the program, it is expected that 60,600 Ghanaians will have gained access to improved sanitation and 66,600 to an improved drinking water source. This is being achieved through six mutually reinforcing components to: 1. Increase use of improved household sanitation 2. Improve community water supply services 3. Improve sector governance and policies 4. Expand key hygiene behaviors 5. Leverage public-private partnerships to magnify impact of US Government investments 6. Improve water supply and sanitation infrastructure for schools and health facilities While the challenges are multiple and W4H has many component parts, the overarching framing of the activity is fairly straightforward: by enabling effective supply and fostering demand, sustainable access to improved sanitation, safe water, and hygiene products will be enhanced. EVALUATION PURPOSE, DESIGN, AND METHODOLOGY As per the Statement of Work (SOW), the primary objectives of the evaluation were to: 1) inform both the implementing partner and USAID/Ghana if the approaches employed by GC are successfully meeting the activity’s goal of expanding and ensuring sustainable access to water and sanitation services; 2) inform the need for any course corrections or reemphasis of priorities to the activity in its final year of implementation; and 3) assess the approach to and progress of implementation to inform future USAID/Ghana water, sanitation, and hygiene (WASH) programming. The evaluation’s emphasis on ensuring sustainability of the interventions is in line with the technical proposal, which notes that “W4H has been designed to foster long-term sustainable change in the way that communities and government interact to achieve gains in WASH.” The evaluation did not focus on review, validation, or verification of GC targets under the cooperative agreement nor on the internal organization (finance, management, and deployment of staff) of delivery. To achieve its objectives, the evaluation answered the following five evaluation questions (EQs): 1. How successfully has the joint WASH policy developed by USAID and the Government of Ghana (GOG) served the purpose for which it was developed? Are there other policy gaps within the WASH sector in Ghana? 2. To what extent are W4H sanitation achievements likely to be sustained? 3. To what extent have W4H’s approaches to private sector engagement for sanitation access expansion been successful? PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY viii 4. What gaps exist in strengthening local governance systems to sustain water services in the country? 5. To what extent has W4H achieved an increase in schools, health centers, and household members practicing basic handwashing behaviors? The evaluation team reviewed contractual deliverables and related documents produced by W4H, GOG, and other third parties (development partners [DPs]) and conducted national and local-level key informant interviews (KII), focus group discussions (FGDs), and site observations in 10 of the 15 MMDAs in which GC is active. The team conducted a total of 90 KIIs/FGDs, 75 percent of which were with stakeholders in the ten MMDAs visited during the evaluation; approximately 25 percent of all respondents were female. FINDINGS AND RECOMMENDATIONS OVERVIEW By the end of the Millennium Development Goal period in 2015, Ghana had made significant progress in providing access to improved water supply to 89 percent of the population.1 The corresponding figures for access to sanitation, however, have been recognized as woefully inadequate, with 19 percent of the population not having access to sanitation facilities and only 15 percent using improved sanitation facilities.2 The figure on sanitation appears to be well below the sub-Saharan African average of roughly 30 percent,3 notwithstanding the fact that Ghana has achieved low-middle income status. Based on the findings of the evaluation, the W4H activity is making a meaningful contribution to addressing Ghana’s WASH service delivery challenge. The general perception is that GC is innovative and exploratory particularly in its relationships with the private sector, but also cautious in contributing on the national stage. Briefly, in response to the five EQs above: • The GOG’s wide array of WASH policies and strategies needs to be updated into a policy framework appropriate to deliver Sustainable Development Goal (SDG) 6, aimed at ensuring access to WASH services for everyone. The real need, however, is to ensure that institutional structures and linkages are fit-for-purpose to support MMDAs in ensuring service provision. The Pro-Poor Sanitation Guidelines, a USAID/GC-supported GOG framework to guide efforts to target those who cannot afford or manage their basic sanitation services, have potential to support these processes of sector players’ targeting of the poor and approach to expanding sanitation service delivery. More ownership is needed, however, by the government agencies that would ideally champion the approach noted in the guidelines. • Through GC’s efforts with local Environmental Health Officers (EHOs), community members are well aware of the benefits of investing in and maintaining household toilets, as well as the health and environmental benefits of a community-wide approach to sanitation. Durable, easy to install, and affordable, the Digni-Loo developed by GC makes a significant contribution to household options for sanitation sub-structures. Greater local authority involvement and ownership at more senior levels would strengthen the likelihood of sustainable sanitation service delivery. • GC has forged excellent business-based relations with national-level private sector actors to advance their contribution to sanitation service delivery. Strategies are emerging to aggregate 1 World Health Organization/UNICEF Joint Monitoring Program (JMP), “Progress on drinking water, sanitation and hygiene: 2017 update and Sustainable Development Goal baselines,” 2017. 2 Ibid. 3 As per the United Nations Millennium Development Goal Report, Assessing Progress in Africa toward the Millennium Development Goals, 2015 (p. 48). PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY ix demand at the MMDA level and then to determine distribution modalities down to the household level to synchronize supply and demand. • While beneficiaries have appreciated GC-supported water supply infrastructure, the evaluation team noted some gaps in local governance systems, mainly geared around local government engagement, that are likely to affect the sustainability of these services. A promising example with high potential for sustainability is the W4H-supported piped water system implemented in Sawla Tuna Kalba District, which serves communities in the town of Tuna. • Using Natural Leaders (NLs)4 as a key entry point, GC’s implementation approach has been to sensitize communities and emphasize the importance of investing in and maintaining handwashing facilities. Messages appear to have been well-received. That said, a more integrated approach that works with a range of relevant stakeholders to deliver well-sequenced interventions across communities, healthcare facilities (HCFs), and schools should be the ultimate goal. In this way, GC and other DPs can support efforts to strive for “Sustainable Sanitized Communities” status, as per the GOG Open Defecation Free (ODF) Verification Protocol. Findings from each evaluation question are explored in greater detail below. Some high-level recommendations are woven into the narrative and indicated in bold. Section 4 of the full report contains a full list of recommendations. EQ1: POLICY GAPS AND THE PRO-POOR GUIDELINES Over the past two decades, the GOG has developed a wide array of policies and strategies aimed at enhancing the effectiveness and efficiency of the WASH sector. The vast majority of interviewees, however, expressed a general lack of understanding of how the different policy and strategy documents fit together into a coherent whole. Thus, these documents need to be updated into a policy framework appropriate to deliver SDG 6 aimed at ensuring access to WASH services for everyone. Revised policies and strategies would be expected to influence resource allocation at both the local and national levels5; clarify the modalities by which services would be delivered, particularly in poor communities; underscore the expected role of the private sector and ways in which the state would encourage, enable, and also oversee these contributions; and emphasize the exit or at least modified strategies of DPs in the age of Ghana Beyond Aid, or WISER Ghana (a new GOG manifesto that recognizes Ghana’s changed status as a lower-middle income country). 6 The onus of ensuring service provision for WASH is vested in local government with national-level institutions providing policy and monitoring frameworks. The challenge is to ensure that MMDAs are sufficiently versed in their responsibilities, are actively building up their capacity and have obligated (and received) sufficient resources to discharge their roles and responsibilities, and have the means to develop and analyze data that would support decision-making around resource allocation. Thus, although consolidation and refinement of policies would help, the real need is to ensure that institutional structures and linkages are fit-for-purpose. Through the W4H activity, USAID/Ghana and GC supported the Ministry of Sanitation and Water Resources (MSWR) to develop a framework to guide efforts to target those who cannot afford or manage their basic sanitation services. Participatory processes involving sector stakeholders resulted in the development of the Pro-Poor Sanitation Guidelines launched in June 2018 (only a year prior to this 4 Natural Leaders have been identified in each of the communities in which GC works as enthusiastic champions of WASH service delivery. 5 Funding for the WASH sector has seen a downward trend in relation to Gross Domestic Product over the past three years (0.5 percent in 2017, 0.3 percent in 2018, and 0.1 percent in 2019). According to the Sanitation and Water for All High-Level Meeting assessment report, Ghana requires US$386M annually to achieve SDG 6 by 2030. The 2019 budget allocates only US$50M (13% of required amount). 6 Ghana Beyond Aid has been trademarked and branded as the WISER Ghana project (where “WISER” stands for “wealthy, inclusive, sustainable, empowered, and resilient”). PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY x evaluation). The principles underpinning the guidelines assume that the community-led total sanitation (CLTS) approach will be used as the primary entry point to target poor and vulnerable individuals and households on a case-by-case basis. Although there is growing familiarity and recognition of their potential value, with the exception of W4H, there was limited evidence that sector players, including Community Water and Sanitation Agency (CWSA), are using the guidelines to reach the poor and vulnerable. Indeed, all MMDAs are expected to draft and gazette sanitation by-laws to include issues on pro-poor targeting and incorporate a pro-poor targeting strategy into their sanitation plans. This has not happened since the launch of the guidelines. The evaluation team finds a clear need for the guidelines and the multi-stakeholder approach to developing the guidelines was appreciated according to interviewees. While relatively early in implementation, initial signs of the document’s influence on sector players (both in government and amongst DPs) are emerging in terms of flagging a needed conversation around subsidies, highlighting technology choices, and recognizing that a key objective is to ensure communities have a path to rise up the sanitation ladder (beyond basic sanitation). However, the pro-poor guidelines have not been sufficiently owned by the government agencies, including CWSA and MMDAs, that would ideally champion the approach. Incorporating a pro-poor framework into refined and consolidated existing policies and strategy documents would create greater coherence. EQ2: SUSTAINABILITY OF SANITATION ACHIEVEMENTS According to GC, targets for sanitation are being met or exceeded, and the majority of communities visited by the evaluation team have attained ODF status with a clean environment (“no visible signs of excreta within the community” as per the ODF verification protocol), hygienic toilets, and available hand washing stations at the household level. In line with the CLTS emphasis on reducing open defecation, GOG’s emphasis on the ground is on counting toilets and communities that are ODF, without necessarily establishing the longer-term support to ensure these efforts are maintained. Until recently, little attention had been given to the durability of latrines—a key cause of slippage. With this challenge in mind, early in the project GC embarked on research and design of a technology option that could provide affordability, ease of installation, and durability. After design testing and successful field trials, GC prototyped what is now known as the “Digni-Loo.” Interviewees considered the Digni￾Loo to be a “game changer” for household sanitation. The Digni-Loo comes as a slab with one ring and a vent pipe for installation. Additional rings can be purchased to lengthen the lining at the time of installation. Beyond being easy for households to install, the Digni-Loo is appropriate for areas with loose soil and difficult terrains and can be moved when the pit is full. Priced at roughly US$80, the Digni￾Loo is considered to be affordable and more reasonably priced (at point of sale but also in terms of life cycle costs) as compared to other sub-structure latrine options on the market (e.g., masonry ventilated improved pits, biofil digesters, and septic tanks). The Digni-Loo is seen both as an entry point and an aspirational product, with the hope that households will construct solid superstructures to accompany a sub-structure that is less likely to cave in. While strategies to roll out the Digni-Loo are being determined, GC and MMDA Environmental Health Assistants (EHAs) have trained households in the construction of latrines using local materials. More innovations, however, around the use of appropriate local materials for smaller communities and homesteads will support sustainability. In terms of reinforcing messaging and enforcing agreed household-level commitments, the use of more than 6,000 NLs to follow up with households in those communities visited appears to be effective. Community members were well aware of the benefits of investing in and maintaining household toilets, as well as the health and environmental benefits of a community-wide approach to sanitation. Community by-laws are helpful in clarifying expectations on the part of the household. The evaluation PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY xi team found that small remote communities will generally continue to invest in construction of toilets (even if rudimentary) due to community reinforcement and mutual accountability. While EHOs/EHAs are keenly grateful for the support and welcome a solid working relationship with GC staff, local authority involvement and ownership at more senior levels have generally been weak. This will ultimately affect the sustainability of WASH systems. GC’s provision of facilities or support is not tied in any way to the performance of the MMDA, beyond dedicating staff time to the activity. W4H data is captured and reported to the GC head office in real time. The evaluation team sighted copies of weekly reports that confirmed regular capture and reporting of a rich cache of data that should be used to inform and influence decision-making at the local level. The GC and GOG Basic Sanitation Information System (BaSIS) data systems are not currently linked to an effective MMDA monitoring and evaluation (M&E) system. Generally considered to be good practice, MMDAs are meant to integrate data and results into planning, as per the memoranda of understanding signed between GC and each MMDA. The evaluation team did not observe this in any meaningful way, nor was it clear that MMDA budgets were being increased to facilitate M&E going forward. As noted by GC (through email exchange), “using data for planning and decision-making, although generally expected, is rarely practiced by local governments as political economy considerations almost always supersede more logistical data￾driven approaches.” While the evaluation team agrees, undoubtedly some MMDAs are able to take these next steps. Thus, it is less clear how W4H processes and interventions are anchored within MMDA Environmental Sanitation Strategy and Action Plan (M/DESSAP), nor is sustainability for sanitation a strong feature in these documents. The quality of initial triggering stages of CLTS is high. Interviews with community members (particularly in the smaller communities) suggested a high level of participation, particularly of headmen and women. The evaluation team noted that community-level interviewees recalled the messages from triggering. Ultimately the quality of triggering could be affected by limited harmonization of messages of WASH implementers, but also across WASH, health, and education departments within the MMDAs. Interviews with GC staff at the local level suggested that post-ODF sustainability monitoring and support were less in evidence as a systematic element of the W4H activity. EQ3: SUCCESSFUL APPROACHES TO PRIVATE SECTOR ENGAGEMENT FOR SANITATION Most GOG policy and strategy documents refer to the private sector as service providers or latrine artisans with a clear need to build their capacity to carry out these functions. A key goal of the W4H activity is to strengthen “public-private partnerships (PPPs) to magnify the impact of USG investments” and to maximize impact through market-based approaches for sanitation service delivery (W4H Activity Monitoring and Evaluation Plan [AMEP], p. 8). W4H has excellent emerging relations with national-level private sector actors. This provides the corporate sector a strategic partnership that both delivers commercial benefits and perceived reputational gains through contributions to the SDGs. The activity has successfully engaged several large-scale manufacturers. As an example, a strategic relationship with the plastics company Duraplast is based on clear business benefits to the company as a product manufacturer (i.e., a more progressive view of partnership beyond corporate social responsibility [CSR]). The engagement with Duraplast started with the supply of vent pipes to provide discounted products for sanitation facilities. GC then provided specifications to the company to develop the Digni-Loo toilet. GC also has a relationship with Ghacem Limited (a Ghana-based cement producer) to use their distribution networks to supply materials to households at prices discounted from the market rate. The demand is aggregated at the MMDA level through the EHO/EHAs working in coordination with the NLs. Distribution modalities down to the household level are still evolving to synchronize supply and demand. The key challenges are how best to aggregate demand to ensure that PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY xii transaction costs are kept low enough to make it a viable business opportunity and to structure targeting and pricing to meet the needs of poorer households. A more traditional level of engagement is through companies’ CSR activities, with funding provided for the construction of WASH facilities in selected communities. While helpful in meeting short-term targets and community needs, GC rightly notes that these types of arrangements are “nice to have but not necessarily game changers” in the sector. Only by appealing to companies through a clearer commercial business case, as with the GC strategy, are efforts likely to reach scale and have a wider impact. As an example, after extensive research and development work with Duraplast and incorporating community inputs, the Digni-Loo was formally launched and introduced to the Ghanaian sanitation market in June 2018. Duraplast is the only company manufacturing the product and only manufactures based on orders received. Duraplast sees the product as belonging to GC and has not formulated its own plans to produce, market, and deliver the product to suppliers. The interest in Digni-Loo received a major boost when the GOG CWSA, with support from the World Bank’s Rural Sanitation and Water Supply Programme, placed a large order for Digni-Loo toilets. As a result, the manufacturer created more molds and expanded production capacity to approximately 1,800 units per month. At the time of the evaluation, the company was on course to supply an order of 20,000 Digni-Loos placed by CWSA by November 2019. In the evaluated MMDAs, the Digni-Loo is not yet readily available to be sold by local businesses stocking plumbing supplies (vent pipes and other hardware). GC was still refining the strategy and roll￾out through identified distributors and sanitation entrepreneurs at the time of the evaluation interviews. GC has indicated the intention to employ a local distributor model through the use of active NLs as a micro-entrepreneur scheme. At the time of writing, W4H had developed promotional videos to support the marketing of the product. Numerous interviewees, particularly at the District Assembly level (as well as GC staff and DPs), noted that conflicting approaches to subsidies by different stakeholders, even in adjacent communities, are creating confusion for households. Thus, practices of government agencies (including CWSA), other DPs, or CSR efforts have the potential in the short term to distort the market if a heavily subsidized, ill￾targeted approach is adopted. In response, GC, which provides the product to suppliers at cost, is seeking to time its activities after the phase-out of the CWSA-World Bank scheme that is providing the sub-structure Digni-Loo to households for free.7 EQ4: STRENGTHENING LOCAL GOVERNANCE TO SUSTAIN WATER SERVICES The W4H activity installed 203 boreholes (some of which were mechanized) in communities, schools, and health care facilities (HCFs) by the end of 2018. While beneficiaries appreciated these systems, the evaluation team noted some gaps in local governance systems that are likely to affect the sustainability of services supported by the activity. Like most DPs, W4H engaged private sector firms to drill boreholes and install water systems. The W4H Accra or regional offices drew up and monitored construction contracts. During field visits, the evaluation team noted that a few systems were not functioning properly or at all. In two cases, relatively new systems had stopped working a few months after initial installation. Further follow up is needed to determine if the systems are working and, if not, if the Facility Management Plans are 7 The CWSA-World Bank program was to have been completed by the end of November 2019, but even so, although 16,000 (of the 20,000 planned) Digni-Loos had been received by MMDAs at the time of the evaluation, several MMDAs noted that a significant number had not yet been distributed to communities during the time of the evaluation. PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY xiii effectively supporting their upkeep. W4H should revisit any construction contractor obligations through some form of guarantee or delayed final payment. At the community level, Water and Sanitation Management Teams (WSMTs) were formed and trained by GC to manage newly installed facilities. The evaluation team met with several WSMTs during community visits and generally found that committee members understood their responsibilities and took these seriously. In a few communities, the handover process to WSMTs occurred before tariffs and other governance elements had been fully agreed upon within the community. This has led to some confusion or even willful nonpayment by users and conflicts over the times of day when taps would be locked. A promising example of a piped water system has been supported by W4H in the district of Sawla Tuna Kalba, which serves communities in the town of Tuna. Albeit still a new system (having only been handed over to the community in June 2019), the evaluation team noted the competence and professionalism through a local Water Board with oversight from the MMDA with efficient monitoring system, fiscal transparency around tariffs, and satisfied customer base. Further study is warranted on how this particular system has become such a model operation to understand the factors needed for replication. Otherwise, the MMDAs visited by the evaluation team do not appear to have a functioning M&E system for the installed water systems. W4H, on the other hand, has been refining relatively simple and straightforward systems and collecting data through an Open Data Kit tablet-based application on a regular basis. To support local ownership and an emerging exit strategy for W4H, GC should plan to turn over all their data to the MMDAs and train MMDA staff in these ODK-based M&E systems to enhance their oversight and inform how best to allocate scarce resources. Little evidence emerged of MMDAs dedicating resources to monitor water infrastructure. Indeed, capturing more information from the 15 MMDAs that are part of the W4H activity on the allocation and actual release of budget for water services would be helpful. More study is also needed to determine whether communities are charging and collecting tariffs in sufficient amounts to maintain and repair water points. To manage breakdowns, GC could link up with SkyFox8 or other programs aimed at providing communities with spare parts at a reasonable and published price and in a timelier manner using mobile ordering systems. Without such systems in place, repairs could remain untenable and be significantly delayed for many communities. EQ5: INCREASE IN BASIC HANDWASHING BEHAVIORS Communities have by-laws, rules, and regulations discouraging open defecation (OD); however, these are usually silent on the need for handwashing facilities. This could be attributed to minimal emphasis on handwashing in the ODF Verification Protocol. Using NLs as a key entry point, GC’s implementation approach has been to sensitize communities and emphasize the importance of investing in and maintaining handwashing facilities. Messages seem to have been well-received. Clearly tracking increases in hand washing behavior is notoriously difficult and thus proxy indications are needed. Across most communities, handwashing facilities were observed near or adjacent to the toilets. Some instances of cracking containers, children breaking the Tippy-Taps, and containers being sold to recyclers suggest that further messaging and support may be needed. Across all communities, interviews revealed that 8 SkyFox is a registered company in Ghana that has been piloting and expanding a range of WASH and water-related services across Ghana and the region. The SkyFox system was originally piloted under the CWSA-IRC SMARTerWASH program with the aim of aggregating demand for parts suppliers, creating more transparency in terms of prices for communities in need, and feeding work to registered artisans. PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY xiv soap was readily available and inexpensive from local kiosks and shops. Ash, as an alternative, was clearly available. Healthcare professionals noted increased public awareness around the need to wash hands at critical times, and the evaluation team’s numerous informal discussions with men, women and children largely confirmed this. Communities reported specific positive experiences from practicing good hygiene, which included health benefits as well as cost savings from reduced illness. Although community members and health authorities believe behavior change has contributed to reduction in illness, as yet no irrefutable data validate this claim. With regard to WASH in schools, W4H has provided schools with toilet and handwashing facilities and engaged in hygiene education and promotion. All schools under the W4H activity benefited from rainwater harvesting tanks sited close to the latrines. The activity provided 470 Veronica Buckets to beneficiary schools to promote healthy handwashing habits among pupils and teachers (W4H FY18 Annual Report). During the rainy season, water is available for handwashing, but pupils face huge challenges in the dry season, except in schools with a water point nearby or where authorities buy water from private vendors. Some schools task children with fetching water from the community borehole when there is no water in the rainwater harvesting system. Schools depend on capitation grants, set at only GHS4.00 per pupil per year, of which GHS1.00 is allocated to maintain WASH services. Not only are the capitation grants generally insufficient to ensure maintenance of WASH facilities, the release of funds is often delayed. According to people interviewed in OD communities, community members without toilets seek to access the school toilets (new and old) and, in the process, break the locks and doors and destroy handwashing stations when the toilets are locked. Indeed, the evaluation team directly observed two instances where older school toilets had been vandalized by community members. The W4H activity provided mechanized boreholes with water filtration systems for 12 healthcare facilities in different parts of the country. These efforts have brought improved water closer to the health facilities to enhance delivery of health services, including for handwashing. The systems are not always piped into the healthcare facility directly however, so staff and patients may still be required to access water from a tapstand somewhere on the property. In terms of an integrated approach, a number of healthcare facilities visited did not have any or adequate latrines, and staff were not aware of any plans for such investments to be made. Thus, the program of work could be more integrated with the connection between WASH services and effective health services delivery more clearly established. A key observation is that only through the coordination of the provision of WASH across institutions (schools and healthcare facilities) and communities will behaviors, infrastructure, and services be sustained for lasting benefit. For example, W4H worked with some schools located in or near communities where they did not also engage with the wider community around CLTS or with the nearest healthcare facility. As such, the lack of integration may bring, for example, instances of continuous vandalism, fractured community cohesion, and wider knock-on effects. Comprehensive or integrated community WASH programs may reduce instances of vandalism of school toilets by the broader community, and pupils will practice consistent handwashing only where there is sufficient and regular water supply both at home and at school. Data collection by healthcare professionals would then also be more meaningful around the links between handwashing, reduced OD, available water and health outcomes. While the indicators for measuring W4H achievements are helpful, the evaluation team would suggest that future emphasis on sustainability be reflected in the number of people having access to integrated, functioning WASH services across all primary areas of life—at the household/community, school, and PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY xv healthcare facility level as an aspirational target. The ultimate goal would, thus, be to strive for “Sustainable Sanitized Communities,” as per the GOG ODF Verification Protocol. CONCLUSIONS Overall, the evaluation team finds that GC is having a positive impact and contributing to expanding and ensuring sustainable access to water and sanitation services. The evaluation team also recognizes that many of the sustainability challenges noted throughout this report are beyond the control of GC alone. Effective, functioning operating environments require clearer designation of roles and responsibilities, more coordinated linkages and incentives, and greater levels of dedicated resourcing. Instead there is a reliance on programs like W4H to fund and deliver basic services in Ghana without clearly embedding these programs in the wider institutional context. Due to a perceived neutrality with good relationships across the sector, GC could contribute more to national debates based on the learning generated from its own programs and projects. GC local-level staff are viewed by local counterparts as professional and helpful. The evaluation team also experienced local staff members in this way and questioned whether GC staff could be encouraged to make more explicit use of this social capital by seeing themselves as more than project coordinators or administrators, but rather as agents of change in the sector. This would suggest, for example, a greater advocacy role at the local level for the benefits of investing in WASH services by supporting the uptake of data and data collection methods, seeking to participate more actively in local planning exercises, and flagging the critical steps and resources required to sustain the gains made in the communities that have benefited from W4H support. This could help maximize their influence at more senior levels to ramp up local ownership and local problem-solving. PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 1 1.0 GLOBAL COMMUNITIES’ W4H ACTIVITY 1.1 W4H BACKGROUND The Water, Sanitation, and Hygiene for Health (W4H) Activity is a five-year (February 2015–September 2020) cooperative agreement funded by USAID/Ghana and implemented by Global Communities (GC) in partnership with The Manoff Group and USAID Global Development Alliance partners Rotary International and The Coca-Cola Africa Foundation. USAID/Ghana commissioned USAID Water, Sanitation and Hygiene Partnerships and Learning for Sustainability (WASHPaLS) to complete this performance evaluation at the start of the activity’s fourth year. The goal of W4H is to accelerate sustainable improvement in water and sanitation access and improve hygiene behaviors in 15 target Metropolitan, Municipal, and District Assemblies (MMDAs). By the end of the program, it is expected that 60,600 Ghanaians will have gained access to improved sanitation and 66,600 to an improved drinking water source. This is being achieved through six mutually reinforcing components: • Component 1: Increase use of improved household sanitation • Component 2: Improve community water supply services • Component 3: Improve sector governance and policies • Component 4: Expand key hygiene behaviors • Component 5: Leverage public-private partnerships to magnify impact of US Government investments • Component 6: Improve water supply and sanitation infrastructure for schools and health facilities Although not within the remit of this evaluation to validate such achievements, the evaluation team is confident that GC will meet or exceed the targets agreed under the cooperative agreement. 1.2 W4H DEVELOPMENT HYPOTHESIS As documented in Section 2.1 of GC’s Technical Proposal, “weakness in governance underlies many of Ghana’s [water, sanitation, and hygiene] challenges.” As such, “effective service delivery is hampered by limited human and financial resources for guiding, monitoring, and supporting community gains in WASH.” A lack of coordination across ministries “limits the effectiveness of scarce resources” and prohibits the resolution of water, sanitation, and hygiene (WASH) challenges due to a “lack of understanding of the full picture.” Simple tools and effective capacity building are needed to allow for longer-term planning and the scaling of successful interventions through the effective use of data. Using participatory approaches to generate data and then developing a “two-way information flow between local governments and their communities” are clearly needed. The challenges described above combined with a “one-size-fits-all approach” and limited sanitation marketing efforts “marginalize the poorest of households from accessing and maintaining improved sanitation.” While coverage for water service delivery has been far more effective, sustainability remains a key challenge due to “weak capacity of communities to manage operations and maintenance and of MMDA authorities to monitor water supply infrastructure.” With regard to handwashing behavior, community outreach workers have behavior change communication (BCC) mandates related to WASH PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 2 Figure 1: W4H Results Framework USAID Water and Development Strategy 2013 – 2018 SO1: Improve health outcomes through provision of sustainable water, sanitation and hygiene (WASH) USAID GHANA Development Objective 3: Equitable Improvements in Health Status GOAL: Accelerate sustainable improvement in water and sanitation access and improve hygiene behaviors in target districts • US Peace Corps • Bill and Melinda Gates Foundation • Coca Cola Corporation • Rotary Foundation • Others • USAID/People for Health Project • USAID/Innovate for Health Project • USAID/Evaluate for Health Project • USAID/Communicate for Health Project • USAID/Systems for Health Project • USAID/RING • USAID/SPRING • USAID/Deliver Objective 6: Improve water supply and sanitation infrastructure for schools and health facilities IR 6.1 Support rehab or new construction of water and sanitation facilities in schools and health institutions Objective 5: Leverage public/ private partnerships to magnify impact of USG investments IR 5.1 Coordinate existing USAID alliances Objective 4: Expand key hygiene behaviors Objective 3: Improve sector governance and policies IR 4.1 Increase adoption of key hygiene behaviors IR 3.1 Strengthen local governance and management. Objective 2: Improve community water supply services Objective 1: Increase use of improved household sanitation IR 2.1 Develop new and rehabilitate water supply services IR 1.1 CLTS regional￾and district-level facilitators trained to implement demand-led CLTS IR 1.2 Increase access to and improve sanitation products and services IR 2.2 Support scale￾up of Manual Well Drilling (MWD) IR 3.3 Support selected WASH-sector policy reforms IR 4.2 Integrate promotion of essential WASH and nutrition actions IR 4.3 Improve integration of WASH, MCH and nutrition services IR 5.2 Create new USAID alliances IR 3.2 Support broad engagement on WASH sector Key Direct Influence Cross-Cutting Influence USAID GHANA W4H Results Framework PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 3 and health, but “opportunities are being missed to strengthen on-the-ground coordination and to reinforce consistent, integrated messaging that adheres to Government of Ghana (GOG) standards.” While the challenges are multiple and W4H has many component parts, the overarching framing of the activity is fairly straightforward: by enabling effective supply and fostering demand, access to improved sanitation, safe water, and hygiene products will be enhanced. These three essential WASH actions foster healthy behaviors that improve health outcomes. Improving sector governance and fostering public-private partnerships will further strengthen expected outcomes. Albeit somewhat generic in its presentation of the Ghanaian context, the evaluation team finds the results framework (presented in Figure 1 above) to be robust and sound with appropriate linkages being made across components. The evaluation questions (EQs) listed in section 2.3 below track broadly to the W4H objective workstreams (EQ1 corresponds to Objective 3, EQ2 to Objective 1, EQ3 to Objective 5, EQ4 to Objectives 2 and 6, and EQ5 to Objective 4). PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 4 2.0 EVALUATION PURPOSE, METHODS, AND LIMITATIONS 2.1 EVALUATION PURPOSE As per the Statement of Work (SOW), the primary purposes of the evaluation are to: 1. Inform both the implementing partner (IP) and USAID/Ghana if the approaches employed by GC are successfully meeting the activity’s goal of expanding and ensuring sustainable access to water and sanitation services; 2. Inform the need for any course corrections or reemphasis of priorities to the activity in its final year of implementation; and 3. Assess the approach to and progress of implementation to date to inform future USAID/Ghana WASH programming. The emphasis of the evaluation has been on the likelihood of success in ensuring sustainability of the interventions, examination of barriers in the wider context, and W4H’s influence on sustainability in the sector more broadly. This is in line with the technical proposal, which notes that “W4H has been designed to foster long-term sustainable change in the way that communities and government interact to achieve gains in WASH.” The exercise has not focused on a review, validation, or verification of GC targets under the cooperative agreement, nor on the internal organization (finance, management, and deployment of staff) of delivery. This final report provides the team’s methodological approach (evaluation methods) and related limitations in this section, main findings and conclusions in response to the EQs posed by USAID/Ghana in Section 3, and recommendations on strengthening sustainability across W4H’s different component parts in Section 4. Annexes provide the SOW, Key Informant Interview (KII) Guides, a list of key informants (KIs), and related information. 2.2 AUDIENCE AND INTENDED USES The audiences for the evaluation report include: the USAID/Ghana Health Office, GC and their IPs, the GOG, and WASH sector working group collaborators. The evaluation findings may be used more broadly by USAID/Ghana and USAID/Washington technical staff to inform global efforts in the WASH sector. The intent is for the findings of the performance evaluation to be disseminated widely. 2.3 EVALUATION QUESTIONS The evaluation answered the following EQs, which were provided in the approved SOW (see Annex 1). The evaluation team reordered the EQs from the original SOW (without any modifications to the wording) to facilitate the flow of the analysis from policy-making to service provision to household behaviors. 1. How successfully has the joint policy developed by USAID and the GOG (i.e., Pro-Poor Sanitation Guidelines) served the purpose for which it was developed? Are there other existing policy gaps within the WASH sector in Ghana? (Originally EQ3) 2. To what extent are W4H sanitation achievements likely to be sustained? (Originally EQ1) 3. To what extent have W4H’s approaches to private sector engagement for sanitation access expansion been successful? (Originally EQ4) PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 5 4. What gaps exist in strengthening local governance systems to sustain water services in the country? (Originally EQ2) 5. To what extent has W4H achieved an increase in schools, health centers, and household members practicing basic handwashing behaviors? 2.4 EVALUATION METHODOLOGY The evaluation methodology used to answer the EQs is described below. The approach and data analysis methods described herein are generally consistent with the methodology outlined in the Inception Report approved by USAID prior to the start of data collection. The evaluation team employed four data collection methods: (a) review of contractual deliverables and other documents produced by W4H, the GOG, and other third parties (development partners [DPs]); (b) Key Informant Interviews (KIIs); (c) focus group discussions (FGDs); and (d) site observations. The data sources, data collection, and analysis methods used to answer each EQ are provided in Table 1. In developing the methodology, the evaluation team employed three measures to triangulate and validate data, reduce bias, and increase the depth and balance of the findings and analysis: 1) methodological triangulation (i.e., consulting data from multiple sources: interview data, W4H reports, government documents, etc.); 2) investigator triangulation (i.e., involving teams of at least two data collectors per interview and having multiple team members involved in data analysis and identification of emerging findings); and 3) data triangulation (i.e., obtaining the perspectives of multiple similar types of respondents across the sampled MMDAs) (see Annex II for further description). Table 1: Data Sources, Data Collection, and Analysis Methods for Each Evaluation Question Evaluation Question Data Sources Data Collection Methods Data Analysis Methods 1. How successful has the joint policy developed by USAID and the GOG (i.e., Pro-Poor Sanitation Guidelines) served the purpose for which it was developed? Are there other existing policy gaps within the WASH sector in Ghana? • References to pro￾poor guidelines across the sector and any related analysis and critique • Other related GOG strategy, policy, and guideline documents • GOG and MMDA staff • Alliance for WASH Advocacy (Assessment of WASH Sector Policies and Strategic Documents) • KIIs with Ministry of Sanitation and Water Resource (MSWR), Community Water and Sanitation Agency (CWSA), Regional Environmental Health Units, Regional Interagency Coordinating Committee on Sanitation (RICCS), MMDAs, Ministry of Health (MoH), USAID, and DPs • Qualitative data analysis, including context and pattern analysis; data disaggregation by location, gender and role, as appropriate 2. To what extent are W4H sanitation achievements likely to be sustained? • Quarterly and annual progress reports • GOG, MMDA staff • KIIs with MMDAs; site/observation visits • FGDs with local communities and sanitation service providers • Qualitative data analysis, including context and pattern analysis; data disaggregation by location, gender, and role, as appropriate; cross-MMDA comparisons where available PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 6 Evaluation Question Data Sources Data Collection Methods Data Analysis Methods 3. To what extent have W4H’s approaches to private sector engagement for sanitation access expansion been successful? • Quarterly and annual progress reports • Research and think pieces on private sector engagement • KIIs with private sector actors, customers, communities, MSWR, MMDAs, CWSA, other DPs • Qualitative data analysis, including context and pattern analysis; data disaggregation by gender, role, and location, as appropriate; cross￾MMDA comparisons where available 4. What gaps exist in strengthening local governance systems to sustain water services in the country? • Quarterly and annual progress reports and Activity Monitoring and Evaluation Plan (AMEP) data • Program descriptions • GOG WASH strategy, policy, and implementation guidance • MMDAs and CWSA • KIIs with CWSA, MSWR, MMDAs, local-level Water and Sanitation Management Teams (WSMTs), MoH, USAID, and other DPs, private sector providers, and service users • Qualitative data analysis, including context and pattern analysis; data disaggregation by gender, role, and location, as appropriate; and cross-MMDA comparisons where available 5. To what extent has W4H achieved an increase in schools, health centers, and household members practicing basic handwashing behaviors? • Quarterly and annual progress reports • AMEP data • Program descriptions • Institutional budget data • Behavior change messages • National strategies and guidelines (Ghana Education Service [GES] WASH in Schools [WinS] implementation guidelines, BCC strategy, WASH in healthcare facilities [HCFs]) • Observations at institution and household level, KIIs/FGDs with institutional stakeholders (schools, health centers, etc.), School Health Education Program (SHEP) (GES), IPs, USAID, and other DPs; FGDs with community members • Qualitative data analysis, including context and pattern analysis; data disaggregation by gender, role, and location, as appropriate; and cross-MMDA/cross￾institutional type comparisons where available 2.5 SECONDARY DATA COLLECTION – DESK REVIEW The evaluation team began the exercise by conducting a desk review of key project documents provided by W4H and USAID, along with other GOG policy and strategy documents useful for placing W4H and the evaluation in context (see Annex IV for a complete list of documents reviewed). The team reviewed some documents, such as W4H deliverables, in their entirety, but members focused their reviews in most cases only on documents or sections of documents related to their areas of expertise. As the desk review and fieldwork were completed in a relatively short timeframe, secondary data collection was an ongoing process. To the greatest extent possible, the team integrated findings from PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 7 the document review into the primary data collection process (for example, in revising the respondent list or refining the data collection instruments). 2.6 PRIMARY DATA COLLECTION The evaluation team used three primary data collection approaches: KIIs, FGDs, and site observations, where appropriate. 2.6.1 SELECTION OF W4H MMDAS FOR PRIMARY DATA COLLECTION Per the evaluation SOW, the team was expected to review W4H progress in ten MMDAs in five to eight regions in which W4H is active (Figure 2). Primary factors initially discussed with USAID and GC to determine which MMDAs to visit included the following: • Mix of better- and less-well-performing MMDAs (as per GC feedback combined with MMDA coverage statistics, District Performance Assessment Tool scoring, and UNICEF/Centre for Democratic Governance and Centre for Social Policy Studies MMDA League Table ranking [2015 and 2019]) • Regional and poverty considerations • MMDAs in which GC is operating a full complement of activities • MMDAs with significant supporting investments from other DPs to understand W4H’s influence and MMDAs in which there are fewer DP interventions • Districts with piped water service supported by GC • Presumed W4H links to other nongovernmental organizations (NGOs) and USAID programs operational in the MMDA • Where significant progress has been made on market-based sanitation, to understand W4H’s influence in this area The team agreed with GC’s suggestion of five MMDAs of the fifteen in which they are active and a further five based on logistical and language considerations. This allowed for a mix of contextual and operational considerations. Table 2 presents the final list of MMDAs selected for site visits, with GC-suggested MMDAs shown in bold. Given the sample size of two-thirds of active W4H MMDAs, this selection more than adequately provided the team with sufficient opportunity to review the W4H interventions in-depth across a wide range of criteria noted above. The spread across the League Table rankings also provided helpful information on diversity in performance. The evaluation team notes that GC is no longer working in a number of MMDAs. This is either as a function of insufficient interest and commitment on the part of the MMDA and households for latrine construction or to attain Open Defecation Free (ODF) status, or (more positively) that communities have achieved and sustained ODF status, meeting targets agreed Figure 2: W4H Districts Visited PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 8 upon with the MMDA. The evaluation team did not visit any MMDAs in which GC is no longer working due to time constraints, though this may have provided further insights. Table 2: MMDAs Selected for Evaluation Team Visit9 MMDAs Selected for Evaluation Team Visit Percentage of ODF Coverage National Ranking in 2015 League Table10 National Ranking in 2019 League Table 1. Ada West (Greater Accra) 17 147 47 2. Asikuma Odoben Brakwa (AOB) (Central) 2.8 99 82 3. Krachi East (Oti) 11.1 202 56 4. Nkwanta North (Oti) 28 165 32 5. Sawla Tuna Kalba (STK) (Savannah) 79.5 158 4 6. Sefwi Bodi (Western North) 22.6 137 36 7. Sefwi Wiawso (Western North) Unreported 71 — 8. Shai Osudoku (Greater Accra) Unreported 28 — 9. Wassa Amenfi West (Western) 4.5 — 77 10. West Mamprusi (North East) 13.5 138 50 2.6.2 KEY INFORMANT INTERVIEWS AND FOCUS GROUP DISCUSSIONS The evaluation team conducted and captured interview transcripts for a total of 90 KIIs/FGDs11 with a broad range of stakeholders to incorporate sufficiently diverse perspectives to answer the EQs and provide sound, actionable recommendations. Interviewees included staff from the W4H activity and USAID/Ghana, other national stakeholders with a mandate to support the WASH sector, and MMDA￾and community-level activity participants who received technical assistance or funded support from W4H. Interviewees were selected based on their (expected) familiarity with W4H, either through direct engagement at the MMDA level or interaction with W4H at the national level, and their “expert status” based on roles and responsibilities at MMDA and national levels. A complete list of KII participants is provided in Annex IV. Over 75 percent of formal interviewees were based in the ten MMDAs visited during the review with female respondents making up approximately 25 percent of the total. The team captured between 4 and 11 formal interviews per MMDA with the following types of individuals: W4H MMDA staff, MMDA officials, WSMT members, Natural Leaders (NLs), sanitation enterprises, public health officials, Environmental Health Officers (EHOs), school staff, health center workers, and community members. 9 The evaluation team also took advantage of proximity to Wassa Amenfi West to visit Wassa Amenfi Central to interview the local GC WASH Officer at length. 10 Figures have been provided as indicative. Ranking criteria and data collection have changed over the period 2015 to 2019, and there was insufficient reporting for several MMDAs in 2019 to be able to include them. Thus, direct comparisons in terms of positioning may not be appropriate or relevant. 11 In many instances at the MMDA and national levels, two or more government staff engaged in the interview process. PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 9 KIIs/FGDs were typically between 30 and 75 minutes in duration and conducted in English or a mix of English and the local language. Interviews were conducted in-person, usually by a two-person team, with one person conducting the interview and the other taking notes. The exception to this was in Shai Osudoku, the first MMDA visited, where a four-person team that included the Team Leader conducted several interviews. This team effort ensured that the approach was tested jointly and then consistently applied throughout the evaluation. Annex II includes the breakdown of team members who participated in data collection in each of the ten MMDAs. FGDs followed the general flow of the KII guides provided in Annex III. W4H staff supported selection of participants for the FGDs and coordinated the time and place for the meetings. However, all FGDs were run independently of W4H staff presence and input, except in instances where their knowledge of the local language was needed for translation. No FGDs were conducted with private sanitation service providers, as originally intended, due to their absence from most of the MMDAs visited (the team only identified one for a KII). FGDs with the target communities included between ten and thirty participants per site. With support from Natural Leaders (NLs), community discussions were largely directed through the local chief. Community walks then allowed the evaluation team to correlate responses in the FGD with other members of the community. Photo 1: Focus group discussion in Ada West (Mission) In addition to real-time note-taking, interview teams often requested permission from interviewees to make an audio recording of the discussion. In all cases, discussions were recorded only after gaining the interviewees’ verbal consent to do so. The team then used transcriptions from the recordings to fill gaps in the notes captured during the discussions. In many instances, GC staff were present but not vocal during the interviews at the MMDA level. MMDA staff did not appear to be hindered or influenced in any way by GC staff presence. Given that the majority of the discussions were geared around sustainability considerations in a broader sense rather than GC performance, GC staff presence was not deemed to be influencing the discussions materially. On exiting the interview, evaluation team members raised any areas of possible contention or concern with the MMDA interviewees out of earshot of the GC staff. Indeed, on no occasion did this elicit a different response. The evaluation team held a number of informal and follow-up discussions with knowledgeable individuals in the sector and Ghanaian stakeholders at the Water and Health Conference hosted by the University of North Carolina’s Water Institute the week of October 7, 2019. This included discussions with the Honorable Deputy Minister for Sanitation and Water Resources, Mr. Patrick Bouamah (Member of Parliament), and representatives from GC, Population Services International (PSI), SNV Netherlands, the CLTS Knowledge Hub, the Center for Water Security and Cooperation, the Aquaya Institute, and others. PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 10 Table 3: Key Informant Sources and Observation Sites MMDA Total (Captured Interviews) Community FGDs/Visits School Visits12 HCF Visits13 Private Sector Interviews Ada West 6 2 4 – – AOB 5 4 1 1 – Krachi East 9 2 2 2 – Nkwanta North 9 2 – 1 1 STK 6 1 1 1 – Sefwi Bodi 5 2 3 — – Sefwi Wiawso 11 3 2 1 – Shai Osudoku 6 3 2 – – Wassa Amenfi West (+ Central) 8 3 1 – – West Mamprusi 4 3 – – – SUB-TOTAL (MMDA Level) 69 25 16 6 1 Development Partners 11 – – – 3 GC Staff (national) 6 – – – – GOG Staff 4 – – – – SUB-TOTAL (Non-MMDA) 21 – – – 3 TOTAL 90 25 16 6 4 2.6.3 KEY INFORMANT INTERVIEW AND FOCUS GROUP DISCUSSION GUIDES Annex III presents the interview guides the team used for KIIs and FGDs. The guides cover key themes of relevance to answer the EQs, as appropriate, to each type of respondent, including: W4H staff, national and regional-level stakeholder, MMDA level official, service provider, and community-level beneficiary group. Themes include: progress toward targets, sustainability and capacity development, lessons learned, nature of interaction with relevant stakeholders, and gender-differential participation and results. Early interviews in Shai Osudoku, Sefwi Wiawso, and STK MMDAs followed the flow of the guides closely. As the evaluation progressed, patterns began to emerge in the types of responses, and the team used these to explore certain issues in more depth. As the evaluation is framed as a formative rather than summative evaluation and also not designed strictly as a piece of research, the team considered it appropriate to diverge from the KII guides to probe deeper on some issues as needed. 12 Not necessarily captured as KII or FGD but rather as site observations with informal discussions with staff and students. 13 Not necessarily captured as KII or FGD but rather as site observations with informal discussions with HCF staff. PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 11 2.6.4 SITE OBSERVATIONS The FGDs gave the evaluation team the opportunity to visit a sample of the communities with which W4H has engaged. Community site visits often scheduled before or after an FGD entailed a short walk through the community to inspect the status of latrines and handwashing stations. Site visits to schools and health centers provided opportunities to inspect institutional latrines and boreholes and were usually accompanied by KIIs with head teachers, the regional SHEP coordinator, and health attendants. These visits enhanced the evaluation team’s understanding of the nature and justification for W4H types of interventions discussed during the KIIs and FGDs. The visits allowed the evaluation team to discuss issues one-on-one with a wide range of community members, including with women and children. 2.7 DATA ANALYSIS METHODS The evaluation team analyzed and synthesized primary and secondary data collected to develop responses to the EQs, as summarized in Table 1. The team analyzed data collected during the KIIs and FGDs using the Dedoose web-based application, a secure fee-based online system for collaborative analysis of qualitative data. 14 Dedoose facilitated the extraction of qualitative data for the evaluation team on a single work space. In the first weeks of the assignment, the evaluation team generated a list of codes based on the EQs and KII guides to use during analysis. The team developed, a priori, a coding tree and a set of four or more key codes for each of the EQs, special codes to highlight specific areas of GC and MMDA performance, and a code to signify leads for follow-up. An explanation of the coding and the analysis from Dedoose is provided in Annex II. Secondary data from the desk review and project monitoring data also was entered into Dedoose under a separate folder and used to collate contextual background to help explain the results and to triangulate findings from the qualitative analysis. 2.8 CAVEATS AND LIMITATIONS Per the evaluation SOW, the evaluation team visited ten of fifteen MMDAs in which GC is active. While relatively uniform in their implementation structure and the types of interventions implemented, the experiences gathered by the evaluation team will have omitted some perspectives from the five MMDAs outside of the sample and also from MMDAs in which GC is no longer working. All parties were aware at the outset of the assignment that the timeline for conducting data collection and analysis was compressed. Initial data analysis was effectively done over a four-week period in July and August 2019. The team provided weekly summaries to USAID and W4H to keep them informed of progress both in terms of delivering on the assignment and the emerging findings. The Team Leader and one other evaluation team member read the interview transcripts in their entirety and coded them in Dedoose. Given time constraints and the volume of data generated from 90 interviews, the team members were unable to validate all of the coding individually. Interpretation of the coded excerpts, however, came at the drafting stage where the Team Leader tasked team members to draft sections using interview excerpts and data sourced independently from the desk review. Members were thus able to frame the findings as per his/her expertise. Having only two coders helped with consistency of code application across the data set but limited interaction across the whole team during this stage. All team members have reviewed this document and contributed to the analysis. This review was not intended to substitute for a Data Quality Assessment exercise to inspect the accuracy of the W4H monitoring and evaluation (M&E) system. Achievements reported in activity 14 https://www.dedoose.com/ PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 12 documents (annual reports, quarterly reports) and working program documents were taken largely at face value. The evaluation team looked for evidence that the types of interventions described in the documents were of sound design and implementation and contributed to W4H’s wider objectives, without physically verifying if the numbers reported by W4H were accurate or substantiated. The team did not record all interviews for a variety of reasons. Where recordings are absent, the team relied solely on notes from the discussions and used these for coding in Dedoose. Where necessary, team members held follow-up conversations to clarify any confusion from the notes. PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 13 3.0 FINDINGS AND CONCLUSIONS This section presents findings and conclusions for the five EQs. Recommendations are embedded in the narrative and provided in list format in Section 4. 3.1 OVERVIEW Feedback from stakeholders across all levels suggests that GC is making a solid contribution that is appreciated by the sector, particularly for the work done at the local level. By all accounts, GC staff are seen as highly competent, hardworking, and conscientious. The evaluation team found GC staff to be highly knowledgeable about WASH service delivery and the sustainability challenges facing stakeholders. GC staff members based in the MMDAs were viewed by local counterparts as professional and helpful. The evaluation team experienced similar positive interactions with local staff and questioned whether efforts could be made to encourage them to make more explicit use of this social capital by seeing themselves as agents of change in the sector, in addition to serving as project coordinators or administrators. This would suggest room for a greater advocacy at the local level for the benefits of investing in WASH services, by supporting the uptake of data and data collection methods, seeking to participate more actively in local planning exercises, and flagging the critical steps and resources required to sustain the gains made in the communities that have benefited from W4H support. This approach could help to maximize their influence at more senior levels in order to ramp up both local ownership of the activities and local problem-solving. A number of tools like the League Table rankings could help frame these conversations more effectively. STK was last in the League Tables at the start of the W4H activity and was keen to tap into GC support to redress this. Although not fully validated, the assumption by the evaluation team is that GC’s support has contributed significantly in the substantial climb in the MMDA League Table to fourth (out of 216) in the 2019 rankings. Indeed anecdotally, most MMDAs did not seem to believe that GC will in fact exit or scale back interventions in their communities. Thus, different conversations are needed with MMDA staff at all levels to foster the needed ownership to sustain the gains made. Given the timing, this should be coordinated with USAID’s conversations at the national level with regard to any follow-on programs and what criteria would be used to select MMDAs. For the design of further programming USAID may consider a two-tiered strategy of working with high-achieving, first-generation MMDAs to chart a path and show an example for a second tier of local authorities. An observation was made that, whether they like it or not, GC staff at the local level are forced to own all previous interventions in an MMDA, whether by GC, other DPs, or GOG agencies. In other words, most communities and local counterparts are not blank slates, and thus development activities are required to build on or redirect previous efforts and expectations. GC staff at the local level appeared to navigate this space well, managing expectations, explaining opportunities and constraints, and generally relating with people in the local area in a constructive way. The evaluation team understands that MMDAs were selected in consultation with USAID, relevant ministries, and other implementing agencies like UNICEF. It was less clear what analysis was done on the enabling environment, specific criteria used for when GC pulls out of an MMDA, and what lay behind decisions on whether or not to support water supply or sanitation, as well as community-level or school or HCF interventions. At a more macro level, the general perception is that GC is innovative and exploratory particularly in its relationships with the private sector, but also cautious in contributing on the national stage. GC PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 14 coordination with other GOG entities and DPs in the WASH space appears to have been limited. Due to a perceived neutrality with good relationships across the sector, GC could bring more to national debates around the learning that it is generating from its own programs and interventions. Restarted by the MSWR, the re-emerging WASH coordination platform could serve as the vehicle for GC to contribute more to these discussions. In terms of actual approaches to delivery, it is widely noted that W4H’s approach in CLTS delivery differs from other IPs in that W4H controls funding disbursements and works directly with district Environmental Health Units. By contrast, UNICEF channels its funding and implementation through the MMDAs. In some cases, the GC arrangement is appreciated as it streamlines the delivery of CLTS. In others, a lack of coordination with the MMDA may hinder their ability to sustain the program. The variance may be attributable to differences in expectations and preferences from the individual MMDAs. KIIs showed that MMDA staff in Sefwi Wiawso were appreciative of the reports provided by W4H, whereas some District Assembly (DA) staff in AOB complained that implementation updates were not forthcoming. At the same time, AOB district staff directly involved in CLTS implementation were very happy with the working relationship. “The way they disburse funds helps the activities to go on. When we need something, we get it on schedule. It is a good experience working with them.” (CLTS Focal Person, AOB) 3.2 EQ1: POLICY GAPS AND THE PRO-POOR GUIDELINES To put the discussion around WASH sector policy gaps into context, by the end of the Millennium Development Goal (MDG) period in 2015, Ghana had made significant progress in providing access to improved water sources to 89 percent of the population. 15 The corresponding figures for access to sanitation, however, have been recognized as woefully inadequate, with 19 percent of the population without access to sanitation facilities and only 15 percent using improved sanitation facilities. 16 The figure on sanitation appears to be well below the sub-Saharan African average of roughly 30 percent,17 notwithstanding the fact that Ghana has achieved low-middle income status. 3.2.1 EXISTING POLICY GAPS Policies, institutional arrangements, and the resulting accountability mechanisms constitute different dimensions of a governance framework for effective service delivery. For the WASH sector, this includes the formulation, establishment, and implementation of WASH policies and legislation to achieve the GOG’s stated objectives. Ideally, such objectives would be interlinked with other GOG goals around health, education, and the environment. Appropriate resourcing and then assigning of roles and responsibilities around oversight, finance, and implementation for the effective delivery of WASH services are the final step needed to ensure progress toward meeting government objectives. High-Level WASH Sector Policy Documents Over the past two decades, the GOG has developed a gamut of policies and strategies aimed at enhancing the effectiveness and efficiency of the WASH sector. The primary policy documents guiding 15 World Health Organization/UNICEF Joint Monitoring Program (JMP), “Progress on drinking water, sanitation and hygiene: 2017 update and Sustainable Development Goal baselines,” 2017. 16 Ibid. 17 As per the United Nations Millennium Development Goal Report, Assessing Progress in Africa toward the Millennium Development Goals, 2015 (p. 48). PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 15 the sector include the National Water Policy (NWP, 2007), National Environmental Sanitation Policy (NESP, 2010), Riparian Buffer Zone Policy for Managing Freshwater Bodies in Ghana (2011), and School Health Education Programme Policy Guidelines (2012) (see Table 4). These were all developed to inform and guide implementation to meet the MDG targets. With the adoption of the UN Sustainable Development Goals (SDGs) in 2015, these GOG policies need to be reviewed and updated to reflect the new ambitious targets aimed at ensuring access to services for everyone. Indeed, the MSWR has initiated the process of reviewing two of these policies, the NWP and NESP. Terms of Reference for the assignment have been completed and the ministry is at present procuring the services of a consultant to support this process. However, it is not certain when the other policies will be reviewed. Several of the policies lack the required M&E frameworks to track their effectiveness in implementation. Revised policies and strategies would thus be expected to: • Influence resource allocation at both the local and national levels; • Clarify the modalities by which services would be delivered, particularly in poor communities; • Underscore the expected role of the private sector and ways in which the state would encourage, enable, and also oversee these contributions; and • Emphasize the exit or at least modified strategies of DPs in the age of Ghana Beyond Aid or WISER Ghana, a new GOG manifesto that recognizes Ghana’s changed status as a lower-middle income country. 18 As discussed in more detail below, ensuring the delivery of WASH services is ultimately the responsibility of the MMDA. While policies have been “on the books” for some time, discussions at the MMDA level revealed that most staff had not seen or do not have much knowledge of sector policies. As one MMDA officer and one GC staff put it: 18 Ghana Beyond Aid has been trademarked and branded as the WISER Ghana project (with “WISER” stands for wealthy, inclusive, sustainable, empowered and resilient). “Sector policies and strategies have not played any significant role in WASH services in the municipality with the exception of the Pro-Poor Guidelines.” (EHO, Wasa Amenfi West Municipal Assembly, July 31, 2019) “There is not much reflection of national-level policy in the implementation in the MMDA. This stems from the fact that there were limited knowledge and information on national-level policies and strategies, with the exception of CLTS …” (District-Level GC Representative) The implication of these statements is that: 1) not much has been done to disseminate these policies to the MMDA level; and 2) their relevance to the realities of local-level decision-making could be questioned. GC local WASH Officers similarly did not appear to be explicitly framing interventions at the MMDA level through government policy or using these to make the case for increased attention or resource allocation to WASH. PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 16 Table 4: Summary of High-Level WASH Sector Policies Policy Main Highlights National Water Policy (2007) The NWP is a forward-looking document with the overall goal to “achieve sustainable development, management, and use of Ghana’s water resources to improve health and livelihoods, reduce vulnerability while assuring good governance for present and future generations.” The policy includes components on good governance, gender, and water quality and equity in relation to access, especially by the poor and under- and non-served areas including peri￾urban communities. The policy framework: • Targets all water users, water managers and practitioners, investors, and decision and policy makers at national and decentralized levels (MMDAs); NGOs; and international agencies; and • Recognizes the various cross-sectoral issues related to water use and the links to other relevant sectoral policies such as those on sanitation, health, agriculture, transport, and energy. The implementation of the policy is based on recognizing the fundamental right of all people, without discrimination, to safe and adequate water to meet basic human needs; prioritizing the social needs for water, while recognizing the economic value of water and the goods and services it provides; ensuring participatory decision-making at the lowest appropriate level in society; and coordinating water resources planning with land use planning. The NWP was underpinned by the MDGs, the African Water Vision 2025 of the New Partnership for Africa’s Development, and the Growth and Poverty Reduction Strategy II. The main strategic actions of the policy are water resources management, urban water supply, and community water and sanitation. Each of these strategic actions is framed around specific sub-principles, policy objectives, legal and regulatory frameworks, and financing mechanisms designed around actions required to achieve the policy intent. In addition to needing to be updated to fit the SDG context, further weaknesses in the policy include inadequate linkages to the handwashing strategy and inadequate clarity on the role of the Ministry of Lands and Natural Resources, Minerals Commission, civil society organizations/NGOs, the media, and traditional authorities regarding water resources management. National Environmental Sanitation Policy (2010) Originally developed in 1999, the NESP was revised and adopted in 2010. The policy provides a framework for the coordination and implementation of environmental sanitation initiatives from the national to the community level. The overarching goal is to develop a clear and nationally accepted vision of environmental sanitation as an essential social service and a major determinant for improving health and quality of life in Ghana. The NESP has implications for all sectors of the economy including health, environmental protection, improvement of human settlements and services, tourism, and general economic productivity. The main pillars of the NESP include capacity development; information, education, and communication; legislation and regulation; sustainable financing and cost recovery; levels of service; research and development; and M&E. The critical principles underpinning the policy are environmental sanitation as both a public and an economic good, the polluter-pays principle, improvement of equity and gender sensitivity, and precautions to minimize activities with negative effect on environmental resources. (See text box below on the Rural Sanitation Model and Strategy, which forms an integral part of the NESP, for further information.) PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 17 Policy Main Highlights School Health Education Policy (2012) The national SHEP provides a framework for a holistic approach to the implementation of school health interventions and streamlines the various policies and activities of all agencies, departments, NGOs, and individuals who offer school health services. The policy seeks to: • Bring all policy issues on school health together into a comprehensive policy document to give focus and measurable direction to program implementation; • Streamline and enhance program coordination; • Effectively and efficiently maximize the use of available scarce resources; • Provide an institutional framework within which stakeholders will be assigned roles and responsibilities; and • Provide the channel and mechanism through which the commitment of political leaders and stakeholders can be sought. The GES has developed a new Education Strategic Plan (ESP) covering 2018 to 2030 that includes SHEP as a programmatic area. The SHEP policy needs to be reviewed to reflect the contents of the new ESP with particular regard for SDG 4.1 (Ensuring that all girls and boys complete free, equitable and quality primary and secondary education) and SDG 6 (indicators 6.1, 6.2, and 6.6). Riparian Buffer Zone Policy for Managing Freshwater Bodies in Ghana (2012) The framework was designed to harmonize all dormant and fragmented regulations in the country concerning buffers bordering water bodies or river systems. The policy aims to ensure that all designated buffer zones along rivers, streams, lakes, reservoirs, and other water bodies are sustainably managed for all. The objectives include: to protect, restore, and maintain the ecological and livelihood support functions of the buffer zone; to ensure equitable and sustainable utilization and management of buffer zone conservation areas, which will contribute to the long-term well-being of both resident and downstream communities; to intensify capacity building, education, and training of stakeholders and ensure their commitment to the conservation of the buffer zone; and to coordinate and harmonize policies and laws in the area of buffer zones amongst various governmental agencies with the view to achieving maximum synergy. This policy is relevant in that it brings a water resources management dimension to WASH programming. The policy was developed before the SDGs were launched, and baseline data and other information informing the policy therefore may be out of date. There is no clear M&E framework to monitor policy implementation. Agenda for Jobs: Creating Prosperity and Equal Opportunity for All (2018– 2021) In the context of jobs and livelihoods, this policy framework has encapsulated WASH issues through its strategic objectives to: 1) improve access to safe and reliable water supply; 2) enhance access to improved and reliable environmental sanitation services; and 3) promote efficient and sustainable waste management. Observations on WASH Sector Policies. The contents and intentions of the policies presented in Table 4 are still appropriate, with some having commendable forward-looking sustainability dimensions. That said, the primary policies were prepared prior to the adoption of the SDGs and the African Union Agenda 2063, with their ambitious targets and indicators. Thus, the assumptions underpinning these policies, falling within the framework of the MDGs, require further updating to reflect the current development paradigm under SDG6 with its emphasis on access for all. While current policies state a commitment to equity, other aspects around the use of subsidies, government budget allocations and PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 18 resourcing more generally, fecal sludge management and related environmental considerations, and greater linkages to health and education will need to be incorporated into an SDG-appropriate policy framework. Changing dynamics with respect to national and local-level developments have an impact on water quality and quantity, in particular population growth and urbanization and the increase in artisanal mining and other industrial developments and their environmental pollution effects on forestry and fresh water resources. The MSWR process to review these policies also should consider these contextual shifts. While certain institutions have been identified as crucial to achieving specific policy objectives, little clarity emerges on the coordinating mechanisms needed with respect to how different policies fit together. It is also unclear if analysis has been conducted as to whether there are competing agendas across different policy objectives as discussed below. Although potentially cumbersome to create, a comprehensive M&E framework would facilitate the tracking of programs and projects arising from the policy framework. “[GOG policy] provides us with an excuse to do what we want. If the policies are lower than what they should be, it allows me to set lower targets for myself. But if everyone everywhere is to have access to improved water and sanitation service, then the policies have to be reviewed to reflect the current demand. Looks like we’re satisfied with getting 2019 budget of GHS243 million. If we’d set the bar high enough to correspond with our ambitions, we would have been screaming that it’s woefully inadequate.” (DP representative)19 WASH Sector Strategy Documents The sector has a fairly well-developed set of strategy documents derived from and aimed at supporting the achievement of various policy objectives. This started in the 1990s as a response to WASH sector reforms and an emerging legal framework. The most relevant GOG documents include: • Behavior Change Communication Strategy (2009) • Drinking Water Quality Framework for Ghana (2015) • Education Sector Medium-Term Development Plan (2018–2021) • Education Strategic Plan, 2018–2030 (2018) • Guidelines for Targeting the Poor and Vulnerable for Basic Sanitation Services in Ghana (2018) • MDG Acceleration Framework (2011) • MMDA Operational Manual, National Community Water and Sanitation Programme (NCWSP) (2014) • National Community Water and Sanitation Strategy (2014) • National Costed Strategy for WASH in Schools (2017) • National Environmental Sanitation Strategy and Action Plan (2011) • National Integrated Water Resources Management Plan (2012) • Project Implementation Manual, NCWSP (2014) 19 Funding for the WASH sector has fluctuated historically, but with a downward trend in relation to Gross Domestic Product over the past three years (0.5 percent in 2017, 0.3 percent in 2018, and 0.1 percent in 2019). According to the Sanitation and Water for All High-Level Meeting assessment report, Ghana requires US$386M annually to achieve SDG 6 by 2030. However, the 2019 budget allocates only US$50M (13 percent of the required amount). A report from the Parliamentary Select Committee on Works and Housing shows that the total disbursements in 2018 were 45.16 percent (GHS82.92M) of the total budget allocation of GHS183.63M. PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 19 • Rural Sanitation Model and Strategy/Rural Sanitation Model and Scaling-Up Strategy (2012) (see text box below) • Strategic Environmental Sanitation Investment Plan (2012) • WASH in Schools National Minimum Standards (2016) • Water Sector Strategic Development Plan (2012–2025) (March 2014) All national-level stakeholders interviewed noted that strategic documents and policies driving the WASH sector should be updated to reflect the SDG targets and indicators. While the review will ensure that the strategies and policies are situated in current international and national development dynamics, a robust M&E framework with validated data sets is still needed to identify the financial requirements and mechanisms to best meet the targets and indicators. Local government officials noted that the policies may suggest what needs to be done (at least to deliver the MDGs) but do not reflect the challenges of how finances flow through the system and how accountability actually works institutionally. A critical challenge remaining is that no mechanisms ensure that stakeholders comply and work within the policy framework. Observations on WASH Strategic Documents. The vast majority of interviewees expressed a general lack of understanding of how the different strategy documents fit together into a coherent whole. Within the context of updating sector documents to reflect the SDGs, the evaluation team notes that consolidation rather than further proliferation of policy and strategy documents for the WASH sector is required. Consolidation would clarify where there may be competing policy objectives (e.g., water for agriculture and industry versus water for domestic use, how housing and land tenure rights play out against requirements for sanitation for all, and how no levies or school fees affect capitation grant requirements for schools to maintain WASH facilities). Consolidation could also result in a clearer M&E framework that would facilitate data collection and analysis that would be used for decision-making RURAL SANITATION MODEL AND STRATEGY An integral part of the NESP, the MLGRD developed the Rural Sanitation Model and Strategy (RSMS), or the Rural Sanitation Model and Scaling-Up Strategy, in 2012. It emphasizes cost-effective sanitation programs through CLTS processes to reduce the rate of Open Defecation by ensuring that rural communities construct and use hygienic toilets. With a step-by-step approach to implementation, the strategy rests on five mutually reinforcing pillars: creating the enabling environment, strengthening capacity, creating demand, facilitating supply (through sanitation marketing), and monitoring and evaluating. With MMDAs as the focal points, roles and responsibilities of all entities, both state and non-state and from community to national levels, are clearly defined. The strategy identifies a financing mechanism and M&E framework to track its implementation, identifying learning outcomes to improve the process. The strategy also has a five-year implementation plan and a costed model to facilitate effective implementation. Financing plan implementation has not come through the mainstream government budgetary allocations. Concentrating in five out of the previous ten regions, mainstream funding has come directly from UNICEF to Regional Coordinating Councils (RCCs)/Regional Inter-Agency Coordinating Committees on Sanitation (RICCs) and MMDAs/District Inter-Agency Coordinating Committee for Sanitation (DICCS) of the Environmental Health and Sanitation Directorate (EHSD). At the national level, EHSD also receives support for training, backup support to regions and districts, and monitoring. The government pays the salaries of the staff leading and supporting implementation. NGOs/DPs implement programs in line with the strategy, financing program implementation directly and not through the MMDA system. DPs/NGOs do not report through the main reporting system, and it is therefore challenging to assess the impact of implementation of the strategy and plans across the board. This notwithstanding, the fact that Nandom District is declared ODF and that sections of the national budget provide information on communities declared ODF (mainly from the UNICEF￾supported GOG WASH program) indicates that programs implemented within the RSMS framework are making some impacts. No central mechanisms have been created to monitor and coordinate programs of all stakeholders at the national level. However, the situation seems to be better in regions where District League Table competition is effectively implemented. The RSMS could benefit from being positioned more clearly into broader GOG policy objectives to guide its implementation. PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 20 purposes on resource allocation, approaches to use in different contexts, and other factors. Indeed, the existing frameworks reside with different institutions (Ministry of Education [MoE], MSWR, MoH, etc.) without clear linkages and coordination between them in terms of wider policy objectives. Institutional Framework for Delivering WASH Services The wider policy agenda notwithstanding, the onus of ensuring service provision for WASH is clearly vested in local government. Thus, as noted above and as per the 2016 Local Governance Act Article 936, service delivery is decentralized with national-level institutions providing policy and monitoring frameworks. While the MLGRD has a responsibility for implementation at the MMDA level, the MSWR is responsible for policy formulation, harmonization, M&E, and research and coordination of WASH activities. The MSWR has no formal structures at the regional and MMDA levels, however, and thereby relies on MLGRD to implement policies. MLGRD staff do not have reporting responsibilities to the MSWR, and thus prioritizing WASH at the MMDA level can prove difficult. In concrete terms and as noted above, MMDAs respond to directives from MLGRD, and the Local Government Services sub-unit has responsibility over MMDA human resources. The MMDAs’ Health, Education, and Agriculture units report to their responsible ministries through the regional offices. The MSWR, therefore, does not have oversight responsibility for any of the departments at the MMDA level. Driving policies through these structures to MMDA level is therefore somewhat of a challenge. (The primary institutions and their roles and responsibilities are provided in Table 5.) Table 5: Institutions with Roles and Responsibilities for WASH Institution Roles and responsibilities Ministry of Sanitation and Water Resources (MSWR) Formulates sector policies, programs and plans. Ministry of Finance (MoF) Finances the sector through government annual budgets and other funding sources. Ministry of Special Development Initiatives Provides oversight responsibility to the three regional DAs (special purpose vehicles to reduce poverty and enhance development in three geographic zones) to provide basic infrastructure at the constituency level through the Infrastructure for Poverty Eradication Program. National Development Planning Commission Formulates and coordinates Ghana’s spatial and economic plans and provides guidelines for the development of Medium-Term Development Plans for all MMDAs. Ministry of Local Government and Rural Development (MLGRD) Formulates and coordinates MMDAs’ policies and plans. Public Utility and Regulatory Commission (PURC) Regulates financial / economic aspects and quality of service for urban water supply and the country’s electricity. Ghana Standards Authority Sets national water supply standards, including chemicals for purification and sanitation products. Environmental Protection Agency (EPA) Oversees environmental management, including sanitation (solid waste and latrines) and pollution of all forms. MMDAs and beneficiary communities Positioned at the center of WASH planning, management, implementation, and M&E. Food and Drugs Authority Regulates and licenses bottled and “sachet” water. PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 21 Institution Roles and responsibilities Ghana Water Company Limited (GWCL) Responsible for water supply in urban areas through management of 81 systems. Water Resources Commission Regulates and manages Ghana’s water resources (both surface and underground). Ghana Health Services/Ministry of Health Responsible for prevention, promotion, and management of health services and WASH in health facilities. Community Water and Sanitation Agency (CWSA) Facilitates, through MMDAs, the provision of safe drinking water and related sanitation services to rural communities and small towns in Ghana. Civil society organizations (e.g., CONIWAS20, religious groups, and network of journalists in WASH) Engage in advocacy and sector dialogues in areas of planning, research, monitoring, and implementation. Ministry of Education (MoE)/Ghana Education Services (GES) School Health Program Unit In collaboration with stakeholders, establishes standards and guidelines for WASH services in schools. Environmental Sanitation Providers Association Coordinates private sector environmental service providers for solid waste. Development partners Provide support through sector funding and projects and programs, and contribute to sector dialogues, including policy, strategy, plan formulation, and implementation of M&E. Private sector (contractors, consultants) Provides services to support MMDAs/communities to access and use sustainable water supply, sanitation, and hygiene services. Includes capacity building, knowledge management, construction of facilities, supply of goods, and services in general. A simplified schematic framework21 for sector coordination and institutional relationships among sector actors is provided in Figure 3. The actual linkages, in practice, would involve more horizontal interconnections and should demonstrate data filtering up through the chain. While several institutions have a direct and indirect impact on WASH service delivery, interviewees suggested that there is little practical engagement and coordination between national-level entities at either strategic or operational levels. There was much discussion in interviews around the structure of the MSWR itself needing to be reviewed. One respondent noted that the MSWR is a “new ministry superimposed on existing structures without detailed analyses of what the implications would be.” Institutional re-alignment may be required to ensure efficient and effective operations of the ministry. 22 A Ministerial Coordinating Committee or similar structure could be created to foster these linkages. The WASH sector platform, which brings stakeholders together bi-monthly, was re-activated by the MSWR in April 2019 after almost two and a half years of inactivity. While good to see this reinstated, 20 CONIWAS is a coalition of NGOs in the water and sanitation sub-sector. It has a secretariat and is usually represented in sector dialogues. 21 Appiah-Effah, E., Armstrong Duku, G., Yakubu Azangbego, N., Kojo Aduafo Aggrey, R., Gyapong-Korsah, B., and Biritwum Nyarko, K. “Ghana’s post-MDGs sanitation situation: An overview.” Journal of Water, Sanitation and Hygiene for Development (2019). 22 Beyond some discussion among the evaluation team and with USAID about a proposed National Sanitation Authority (NSA), this did not elicit much response from national-level stakeholders during the time of the evaluation interviews. It was understood that the Cabinet was not in favor of the idea. Since then the Vice President and the Minister of Sanitation and Water Resources announced at the most recent Mole Conference (2019) that the NSA would be established. Further consultations are no doubt needed on how it would be structured, and NSA does not appear in the 2020 budget presented to Parliament. PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 22 discussions at present are more about basic information sharing than strategies for coordinating or determining what works best in which contexts. Figure 3: Simplified Framework of WASH Sector Coordination and Institutional Relationships23 23 WMD = Waste Management Department (at the MMDA level). MES = Ministry of Environment, Science, Technology and Innovation. MOFE = Ministry of Finance and Economic Planning (now separated into two separate ministries). Overarching Conclusions on Policy Gaps in the WASH Sector The primary focus in most interviews was not on the gaps in the WASH policy space directly but on the challenges of ensuring a functioning and well-coordinated sector, given the numerous authorities and departments that have some influence over service delivery. Indeed, the policies and strategies are comprehensive but need to be updated to reflect the more ambitious targets of the SDGs as a major development paradigm shift. This effort could provide the opportunity to consolidate and refine, making policies and strategies more accessible to those needing to apply them. The bigger challenge will be to ensure that the ultimate authority over implementation—the MMDAs—are sufficiently versed in their responsibility, have obligated (and received) sufficient resources to discharge their roles and PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 23 responsibilities, and have the means to develop and analyze data that would support decision-making around resource allocation. 3.2.2 THE PRO-POOR SANITATION GUIDELINES Meeting the SDG agenda of ensuring universal access to sustainable hygienic sanitation services will certainly require some level of support particularly for the poor and vulnerable. Through the W4H activity, USAID/Ghana and GC supported the MSWR to develop a framework to guide efforts to target poor and vulnerable households who cannot afford or manage their basic sanitation services. Participatory processes involving sector stakeholders resulted in the development of the Pro-Poor Sanitation Guidelines launched in June 2018. The principles underpinning the guidelines assume that the CLTS approach will be used as the primary entry point to target poor and vulnerable individuals and households. The guidelines underscore that support may be given on a case-by-case basis, based on proven or demonstrated poverty, rather than to all households in a community. As set out in the guidelines, the MMDAs will determine the process of identifying and targeting poor and vulnerable persons in consultation with and led by community members using minimum standards and operation and maintenance (O&M) protocols, as prescribed in the RSMS. The guidelines also make mention of types of services that can be made available for the benefit of the poor. Beyond the criteria provided (largely geared toward vulnerable households who cannot rely on external support), the suggestion is that the community ultimately decides who is poor based on its own criteria. The guidelines attempt to modify the purist approach to CLTS by introducing a range of acceptable support mechanisms, albeit without actually tackling direct subsidies, i.e., cash transfers. Interestingly, the document does not mention the word “subsidies,” with the closest reference being to the “sale of customized/specialized toilets at subsidized rates.” That said, it does flag different modalities of subsidized support that can be used to target the poor and vulnerable. As one respondent suggested: “It’s a good policy because … it guides [us on] how to approach the whole subsidy thing.” (District Environmental Health Assistant [EHA]) The list of types of support is helpful, but there is significant scope for interpretation as to how the support is to be provided (e.g., directly to a household or indirectly through artisans, by other community members or through artisans who are paid to help), in what amounts (e.g., as a percentage of hardware or installation costs), and when. Two situations noted from the fieldwork in AOB suggested that CWSA informs the community members of support at the CLTS triggering stage, while GC staff indicated that the subsidy is provided after ODF status has been achieved. The policy is silent on which approach is preferred. In terms of dissemination, according to the W4H activity’s annual report for Fiscal Year (FY) 2018, “copies of the [Pro-Poor Sanitation Guidelines] have since been distributed throughout the country and workshops held in various regions to build the capacity of implementers.” Interviewees noted that to date, dissemination has been limited to national-level stakeholders and W4H operational MMDAs. This was confirmed in an interview with a staff member of the MSWR who indicated that the ministry has yet to disseminate the document nationwide and that it is the ministry’s responsibility to do so. Interviews found that some of the MMDA teams have not referred to the document after the initial orientation. With the exception of W4H, there was limited evidence that other sector players, including CWSA, are using the guidelines in any meaningful way to facilitate project implementation in support of the poor and vulnerable, although there is growing familiarity with the guidelines and recognition of their potential value. Indeed, it is worth noting that the guidelines are only just over a year old (they were launched in June 2018). PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 24 The W4H activity is implementing the guidelines by supporting the poor and vulnerable with the Digni￾Loo toilet facility. The implementation has followed the prescription made in the guidelines with community engagement leading to community identification, selection, and further validation by the whole community of the poor and vulnerable. NLs are selected from the communities to guide this process and to navigate through community politics. This is how a member of W4H staff put it: “We contacted chiefs and Natural Leaders first, and then they went through the process, and then we went back to the community to verify. I had an experience in one community … when I had the list, the community members said that one person on the list was not poor.” (GC Representative, Wasa Amenfi Municipality) In practice, the W4H activity has been very cautious at implementing this kind of support because, as stated by a W4H staff, “they do not want to create the impression that the project is supporting some people and leaving out others.” The guidelines have identified institutions that have roles to play in its implementation. One DP noted, however, that they tried to use the guidelines, but there was insufficient clarity on the specific roles and responsibilities of different stakeholders to operationalize the principles, beyond that which is stated for the implementing agency, MMDA, and community leaders. For example, beyond contributions made by DPs, it is unclear how budgets will be made available for MMDAs and under what budget lines to implement the guidelines going forward. While micro-finance (mentioned in the Pro-Poor Sanitation Guidelines as not having worked well in Ghana for these kinds of programs) could be an option generally, the very poor and vulnerable are unlikely to qualify for credit and asking them to incur debt to construct latrines would be inappropriate. National-level stakeholders generally agreed that the guidelines have provided an initial framework for the sector to reach the poor and the vulnerable with access to improved WASH services. Controversially though, the GOG’s CWSA, through the World Bank program aimed at increasing sanitation coverage, is not adhering to its own guidelines through targeted distribution of the Digni-Loo toilet, but rather blanketing supply across whole communities (see EQs 2 and 3 for further discussion). Indeed, some noted that if government agencies are not using the guidelines as a reference, then there is less pressure on DPs and others in the sector to use the guidelines. Sanctions on non-compliance by IPs and service providers are seen as a non-starter since the guidelines are a suggested rather than mandated practice. Indeed, all MMDAs are expected to draft and gazette sanitation by-laws to include issues on pro-poor targeting, but this has not happened since the launch of the guidelines. By all accounts, MMDAs have also not incorporated in their sanitation plans (MMDA Water and Sanitation Plan [M/DESSAPs]) a pro-poor targeting strategy as stated as a requirement in the guidelines. While drawing on inputs from different ministries and GOG agencies, the guidelines are largely silent on inter-sectoral coordination or inter-ministry coordination activities. Linkages are drawn between Livelihood Empowerment Against Poverty (LEAP), the Ghana household registry, and other social intervention projects that suggest a clear engagement with the Ministry of Gender and Social Protection is vital. It remains unclear how that ministry or its corresponding local government unit have been involved in the implementation. Overarching Conclusions on the Pro-Poor Guidelines Gaps in the operational considerations for how the guidelines would be applied notwithstanding, the evaluation team finds that the rationale for a multi-stakeholder approach to developing the Pro-Poor Sanitation Guidelines has been robust. While relatively early in implementation, some initial signs of the document’s influence on sector players are emerging in terms of flagging a needed conversation around PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 25 subsidies, highlighting technology choices, and recognizing that a key objective is to ensure communities have a path to rise up the sanitation ladder (beyond basic sanitation). However, the introduction of the guidelines into the sector has not been sufficiently owned by the government agencies that would ideally champion the approach. An option might still be to refine and consolidate existing policies, strategy documents, and guidelines but also to incorporate a pro-poor framework into these revised sector documents. A quick and straightforward checklist for use by practitioners will also help make the Pro￾Poor Sanitation Guidelines more user friendly, while also assuring their usefulness and limiting the level of subjectivity when it comes to their implementation. A simple framework for monitoring and evaluating the implementation of the guidelines should be part of the document. Clear implementation of the guidelines would also help the GOG to track access by wealth quintiles as well as by other marginalized groups. Some additional guidance is needed to implement the community sensitization process outlined in the Pro-Poor Sanitation Guidelines. This would help MMDA staff to manage the expectations of non￾beneficiaries, particularly as the results of the selection process from within the community (which could be manipulated by local political pressures or interests, as discussed with MMDA staff and in local-level FGDs) could differ from those selected strictly through the use of the guidelines criteria. Similarly, the guidelines could more clearly reflect that the process is designed to validate those persons in the community who do not have livelihoods or external support, and then to flag any special cases to be considered. The section on targeting should reference LEAP and other social welfare registers as a first line of inquiry for those entering the community. (Indeed, not all communities have been registered on LEAP or are benefiting from existing subsidy programs.) Ultimately, records should be kept of the households identified for support, any inconsistencies with other social welfare registers, and an explanation of why and what kinds of support were received. This will provide documentation for other IPs working in the community at a later date, either on WASH or other sector programs. 3.3 EQ2: SUSTAINABILITY OF SANITATION ACHIEVEMENTS GC’s track record through the W4H activity is impressive. According to GC, targets for sanitation are being met or exceeded, and most of the communities the evaluation team visited have attained ODF status with a clean environment (“no visible signs of excreta within the community” as per the ODF verification protocol), hygienic toilets, and handwashing stations. While the focus of CLTS interventions is behavior change, the reality on the ground for government is on counting toilets and ODF communities. In terms of framing the evaluation team’s analysis, a 2018 USAID publication on sanitation24 uses the following two definitions of sustainability and slippage: • Sustainability is defined as the persistence of OD reductions attributable to a CLTS intervention or private latrine ownership increases measured over time after the “end” of a CLTS intervention (however defined by the local implementer); and 24 USAID, “An Examination of CLTS’s Contributions toward Universal Sanitation.” Washington, DC: USAID Water, Sanitation, and Hygiene Partnerships and Sustainability (WASHPaLS) Project (April 2018). https://www.communityledtotalsanitation.org/sites/ communityledtotalsanitation.org/files/WASHPaLS%202018_CLTS%20desk%20review.pdf PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 26 • Slippage (or “backsliding”) is defined as the percentage of households found to have reverted to the practice of OD, or the percentage of households no longer served by a household latrine measured at some monitoring interval following an ODF declaration. A recent document published by the CLTS Knowledge Hub emphasized that programs need to “understand that ODF slippage at some level is likely to occur in most CLTS programs. Being able to recognize the patterns and identify factors driving slippage will improve speed and effectiveness of response and ensure slippage is reversed before it becomes permanent.” 25 To understand the likelihood that W4H sanitation achievements would be sustained over time, the evaluation team first reviewed the W4H approach to sanitation delivery to ascertain how sustainability was considered. Secondly, for communities that have attained ODF status (see text box), the evaluation team sought to determine how well the W4H activity is addressing the likelihood of slippage. With these aspects in mind, the evaluation team discussions revolved largely around similar themes to those suggested in the sustainability checks put in place by the Dutch Ministry of Foreign Affairs, which identified the following as critical overarching elements for sustainable rural sanitation: • The use of durable construction materials combined with local sanitation market options for toilet construction; • Sufficient capacity at the community level to reinforce messaging and enforce agreed household-level commitments; • Sufficient involvement of local authorities in program implementation and sustainability monitoring; • Sufficient quality of the initial triggering effort; and • Sufficient post-ODF follow-up and support. 26 The analysis below of W4H programming is framed around these themes. 25 Hickling, S., “Tackling Slippage.” CLTS Knowledge Hub, Issue 14, Sept 2019. 26 Jiménez, A., Jawara, D., LeDeunff, H., Naylor, K.A., and Scharp, C. “Sustainability in Practice: Experiences from Rural Water and Sanitation Services in West Africa.” Sustainability, 9, 403 (2017). https://www.mdpi.com/2071-1050/9/3/403 OPEN DEFECATION FREE DEFINITIONS According to the 2013 revised ODF Verification Protocol, ODF is defined as: • An ODF-Basic Community has no visible feces accessible to flies, or domestic and wild animals in the entire community (can be expected to be achieved in two months); and • An ODF Community has no visible feces, and all households have access to and use a latrine. At least 80% of households own improved latrines with handwashing facilities (can be expected to be achieved in six months). Beyond these two definitions, communities can achieve two other levels in the Verification Protocol: • A Sanitized Community has no visible feces. All structures (houses, schools, marketplaces, churches, mosque, health post, etc.) in the community have improved latrines, as required by law, with handwashing facilities. There is proper refuse and wastewater management in place. All members have adopted good hygiene practices (can be expected to be achieved after 12 months). Wastewater in the ODF verification document refers to graywater from bath houses, kitchen, and laundry. • A Sustainable Sanitized Community has sustained its sanitized community status for three successive years. PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 27 3.3.1 DURABLE CONSTRUCTION MATERIALS AND LOCAL MARKET OPTIONS FOR TOILET CONSTRUCTION Generally, the focus of CLTS in Ghana has been on triggering behavior change and achieving ODF in target communities. Adequate practical consideration has not been given to the enabling environment or infrastructure that will support these communities both in terms of access to products and installation and to follow-on support that anticipates slippage. Until recently, little attention had been given to the durability of latrines and to issues of safely managed sanitation (as part of the SDG 6 goal). Interviewees suggested that lessons learned from a range of CLTS projects identified that communities declared as ODF were regressing to OD in part due to infrastructure not being durable. For example, the CWSA Extension Services Specialist noted that assessments in Upper East recorded that 40 percent of toilets washed away in heavy rains. Thus, although recognizing the focus on using local materials, easy-to-install technologies that are easy to transport and distribute can make a substantial contribution to ensuring safe and sustainable access to sanitation. With this in mind, GC has focused on durability throughout the W4H activity. In their proposal, GC recognized that as “improved latrines are unaffordable to many, the use of inappropriate techniques by untrained pit diggers, masons, and other craftsmen working in isolation from one another often leads to pit collapse, latrine decay, and, ultimately, discouragement on the part of the household.” GC therefore sought to ensure that latrines lasted and embarked on research and design of a technology option that could deal with affordability, ease of installation, and durability. After design testing and successful field trials, GC prototyped what is now known as the Digni-Loo. Interviewees considered the Digni-Loo to be a “game changer” for household sanitation in a number of ways. It responds to the challenge of durability while also being easy for households to install and appropriate for areas with loose soil and difficult terrains. The Ghana Standards Authority has tested and certified the structural integrity of the slab. The Digni-Loo has further undergone certification through the MSWR – Technology Applicability Framework. Apart from providing a durable and stable structure that can withstand the elements, Digni￾Loo can be moved when the pit is full. Priced at roughly US$80, the Digni-Loo is considered to be affordable and more reasonably priced than other sub-structure latrine options on the market (e.g., the masonry ventilated improved pit [VIP], Biofil Digester, and septic tank). As the Digni-Loo requires little to no maintenance and the slab can be re￾used, it has a lower life cycle cost than the other options. The Digni-Loo is seen both as an entry point product and as an aspirational product, with the hope that households will construct solid superstructures to accompany a sub-structure that is less likely to cave in. Photo 2: Household latrines in a range of construction styles in Ada West and Nkwanta North PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 28 While strategies to roll out the Digni-Loo are being determined, GC and EHAs have trained households in the construction of latrines, and as a result, most households have done so on their own using local materials. More innovations, however, around the use of appropriate local materials for smaller communities and homesteads will support sustainability. 3.3.2 COMMUNITY-LEVEL CAPACITY FOR MESSAGING AND ENFORCEMENT To ensure sustainable sanitation service delivery within the communities, W4H deployed a number of strategies including: • The use of NLs based in the community to reinforce messages and promote and support construction of latrines; • The use of trained artisans; and • The development of markets to facilitate supply of materials. The W4H FY16 Annual Report states that the activity has from inception built on local knowledge and structures, working with local government staff, local NGOs, NLs, and latrine artisans to drive household sanitation facility construction and ownership and to improve sanitation behaviors through CLTS. This approach aims to ensure that local capacities are developed to continue CLTS interventions even after activity closure. The FY19 work plan further suggests that W4H has been successful in “using participatory approaches to develop self-reliant… community management structures that hold themselves and their local governments accountable for continued services delivery.” The evaluation team notes that the use of NLs to reinforce and follow up with households in those communities visited appears to be effective. Interviews with GC indicated that more than 6,000 NLs are currently trained, and the activity intends to take a number of them (at least 10 percent) through a sanitation entrepreneur module (discussed further under EQ3). Community members were well aware of the benefits of investing in and maintaining household toilets, as well as the health and environmental benefits of a community-wide approach to sanitation. Households noted such benefits as reduced incidence of illness and disease and the convenience of having facilities closer to living spaces, and respondents provided reassurance that they will continue to invest in sanitation even if their pits collapse. Reflecting the views of a number of community members interviewed, one noted that: “We do not go to the bush any more to defecate. One can now confidently go to defecate in his/her latrine in the night without fear. Cholera cases… have been drastically reduced. Due to the construction and use of household latrines, reported cases of snake bites have eventually ceased. Our domestic birds, especially fowl, used to die, but now we no longer have reported cases of fowl dying in our community. We used to experience certain ailments when we were practicing open defecation, but now we no longer experience such ailments.” (NL, Ataki, Krachi East) The evaluation team found that small, remote communities will generally continue to invest in construction of toilets (even if rudimentary) largely due to mutual reinforcement and mutual accountability. Taking the analysis one stage further, this corresponds to GC’s own findings in which households that strongly identify with their community are more likely to construct latrines after CLTS interventions.27 Community by-laws are helpful in clarifying expectations on the part of the household. 27 Borkowski, J. “What Makes Ghanaians More Likely to Stop Open Defecation and Build Latrines?” Global Communities Research Brief: November 2019. [Based on Harter, M. and H.J. Mosler. Determining the Effectiveness and Mode of Operation of Community-led Total Sanitation: The DEMO-CLTS. EAWAG: October 2018.] Available at: www.ncbi.nlm.nih.gov/pubmed/30946581 PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 29 In some communities, the evaluation team observed that these were posted in public places. The GC representative in Sefwi Wiawso indicated that the “willingness of communities to enforce agreed rules and regulations has supported the sustainability of ODF status.” “Those who got the concept of building the household latrine faster were asked to support and work with their colleagues [neighbors] to grasp the concept. In our community, if a person is not able to build, we [the NL] go to support the individual to build his/her household latrine.” (NL, Krachi East) 3.3.3 INVOLVEMENT OF LOCAL AUTHORITIES IN IMPLEMENTATION AND MONITORING While EHO/As are keenly grateful for the support and welcome a solid working relationship with GC staff, MMDAs have generally not shown leadership and ownership of the project, and this will ultimately affect the sustainability of WASH systems put in place by GC across the board. The basis for MMDA engagement in the W4H sanitation interventions is through a Memorandum of Understanding (MOU) between each MMDA and GC. The MOU is largely geared around activity delivery with an emphasis on ensuring the release of field officers (EHO/As) to support GC fieldwork. Although it shows intent, the document is not legally binding and thus, as currently framed, it would be challenging for GC to hold the MMDAs to their commitments. The provision of facilities or support is not tied in any way to the performance of the MMDA. In the interests of sustainability and a more systemic approach, future MOUs could consider an explicit set of interventions to build capacity for the MMDA to strengthen their role in planning, budgeting, implementing, and monitoring sanitation activities. The MOU could also note an expected increasing contribution over the period year-on-year to show commitment and foster local ownership. This would suggest a de facto emphasis on GC’s role and resource allocation evolving and decreasing over time with regard to these functions. “UNICEF sends direct cash transfer that we plan for. Global Communities doesn’t put money in government accounts, [but] they work directly with their own field facilitators.” (Krachi East Municipal Planning Officer and EHO, referencing the work with the EHOs selected to support GC with fieldwork) For the CLTS interventions, the MMDAs selected the communities while the W4H activity worked with environmental health staff to bring those communities to ODF. MMDA staff are released from other duties, as noted in the MOU, but the activity supports them directly to work alongside GC to implement the interventions. W4H facilitated the establishment of MMDA Inter-Agency Coordinating Committees for Sanitation (M/DICCSs) and RICCSs where none existed to coordinate CLTS activities and, more importantly, validate communities’ claim to be ODF. As per the MOU and generally considered to be good practice, MMDAs are also meant to integrate data and results into planning. The evaluation team did not observe this in any meaningful way, nor was it clear that budgets were being increased by MMDAs to facilitate M&E going forward. Although not mentioned in the MOU, efforts to build capacity and facilitate the integration of data and results into planning would logically run through planning processes. As noted by GC, using data for planning and decision-making regarding interventions, although generally expected, is rarely practiced by local governments (or national governments) as political or economic considerations almost always supersede more logistical data-driven approaches. The commitment and passion demonstrated by the local GC staff and EHAs during the fieldwork was commendable. That said, using resource allocation and forward planning for GC’s exit as proxy indicators, levels of commitment demonstrated by the MMDA level leadership appear limited. Thus, it is less clear how W4H processes and interventions are anchored within the broader MMDA plans. Annual PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 30 W4H plans for each MMDA are not integrated into the MMDA Water and Sanitation Plan (M/DESSAP), nor is sustainability for sanitation a strong feature in MMDA plans. In terms of verification of ODF status, the community first goes through a self-verification process. The M/DICCS then conducts a second verification, and the RICCS conducts the final verification, potentially drawing on third-party verification. Before MMDAs can request for external verification to be undertaken, MMDA information on ODF communi Photo 3: Community member displaying “Updated OD Map” in Ada West ties needs to be uploaded on the Basic Sanitation Information System (BaSIS). 28 IPs such as GC need to engage further with national agencies to ensure that their achievements are included. The W4H Chief of Party confirmed that, given that different stakeholders use different systems for data capture, the BaSIS has not captured all the data for ODF. The MSWR should make sufficient effort to ensure that BaSIS truly becomes a national platform where all stakeholder data are captured. Furthermore, issues of underreporting and mismatch of data need to be systematically resolved. BaSIS is currently limited only to regions where UNICEF is working. GC could help facilitate discussion toward expanding the reach of BaSIS to all regions. USAID is currently supporting a program with MSWR to determine how best to harmonize data. Whether USAID or other DPs will support the GOG to ensure that BaSIS becomes the nationwide sanitation monitoring system it was intended to be remains a question. Ultimately, the MSWR needs to own such processes. Data are captured and reported to GC head office in real time. The evaluation team sighted copies of weekly reports that confirmed regular capture and reporting of a rich cache of data that should be used to inform and influence decision-making at the local level. Beyond the weekly progress reporting, the data captured at the MMDA level provides information for uploading into BaSIS. While this is a helpful structure for M&E at the national level, these systems are not linked to an effective M&E system within the MMDA. 3.3.4 SUFFICIENT QUALITY OF THE INITIAL TRIGGERING EFFORT “A proper triggering outcome is like a matchbox in a gas station. After facilitation, you see some kind of chaos where people are willing to take action.” EHO/CLTS Focal Person, AOB 28 According to www.sanitationghana.org, BaSIS is a “decentralized M&E sanitation system developed to aid in the implementation of the CLTS at both sub-national and national levels. The system is built to populate data collected from approved sources based on some sanitation index in the form of maps, charts and tables.” PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 31 The initial stages of CLTS triggering include pre-triggering with community entry and the identification of NLs. The triggering stage then involves: “Engaging the community to realize the problem and come up with an action plan. [We then] collect baseline data and conduct a transect walk with the community members to know where they dump refuse, where they defecate, etc. We trigger shame.” GC Staff at MMDA level Initially, GC relied on local NGOs to implement CLTS interventions in target communities. As the activity progressed, the approach favored more engagement with EHAs/EHOs, NLs, and, in some instances, artisans as a way of ensuring more impact and sustainability. In this way, GC could monitor costs more effectively, ensure consistency of approach, provide regular follow-up, and collect data to monitor targets. The W4H FY19 Annual Report reemphasizes this approach, noting that “working with employees of GOG and trained artisans is one of the surest ways to have capacity at the MMDA level to continue to implement WASH after the project has ended.” Interviews with community members (particularly for the smaller communities) suggested a high level of participation from all the community members, particularly women. The headmen of the communities were involved in the triggering process and encouraged community members to build latrines. Indeed, the evaluation team noted through engaging with community members that most of them recalled the messages from triggering. Some also recalled training in handwashing and demonstrated for the evaluation team to observe. “We are enjoying the benefits of household latrines in our communities. No odor and fecal matter pollution in the community. No more reported cases of reptile bites, drastic reduction of reported diarrhea cases, and at our convenience, we walk into a latrine.” (Community Member, Matamanu Junction Community, Krachi East) In terms of gender, the W4H FY17 Annual Report indicated that “the CLTS approach enables all genders to participate in their community’s progress toward ODF status.” Indeed, the approach taken by GC did not appear to emphasize a gender angle to any significant degree, but in the communities visited, women were engaged, knowledgeable, responsive, and appreciative. “Sanitation is a human right; this should be reflected in the document so we can use this as a basis to provide for the poor, disabled, etc. [As for] gender in WASH, nothing is stated in any of the [government policy] documents to show that we are serious about gender issues. We need to change this.” (Kweku Quansah, Director at MSWR) Ultimately, the quality of triggering could be affected by the harmonization of messages across implementers. As noted elsewhere in this report, there appears to be limited harmonization across WASH implementers as well as across WASH, health, and education departments within the MMDAs. Early on in the activity, the W4H FY15 Annual Report indicated engagements with projects like Communicate for Health (C4H); Systems for Health (S4H); Strengthening Partnerships, Results, and Innovations in Nutrition Globally (SPRING); and Resiliency in Northern Ghana (RING) to build synergies and harmonize messages so that W4H’s “behavior change strategy reinforces and extends their efforts.” These interactions appeared less readily apparent and influential to the evaluation team, albeit it should be noted that most of these programs have already concluded. PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 32 3.3.5 SUFFICIENT POST-ODF FOLLOW-UP AND SUPPORT W4H work has focused mainly on expanding access. The evaluation team noted consistency in engagement with communities from first entry until ODF status is achieved. Sustainability should then be supported through monitoring by the MMDA Community Development Officers and EHOs. Interviews with GC staff at the local level suggested that sustainability monitoring over subsequent periods was less in evidence as a systematic and well-planned element of the W4H activity. Such checks would ensure the sustained shifts in social norms and note any sanitation sustainability challenges and how best to address them. “In terms of monitoring, whenever we go to an area, I know where all the communities are; I pass by sometimes to do spot-checking on the communities. If there are problems, we meet with the community to discuss.” (EHO from AOB MMDA) The monitoring role of the MMDAs (DEHOs/EHO field staff) is hampered by the lack of resources such as funds and motorbikes. The lack of resources at the MMDA also means that monitoring slippage is not being properly resourced and, thus, not documented. The W4H Deputy Chief of Party noted that GC had started to conduct sustainability surveys to ascertain levels of slippage. A related WASHPaLs study has been validating sustainability in the north while GC has been conducting its own sustainability studies in the south. Initial findings of this GC survey indicated that there was some slippage, largely attributed to children’s relapse, but that these are less common as compared to the sector more generally.29 Indeed, GC should be commended for opening up their activity intervention to such rigorous data collection and analysis as is being conducted by WASHPaLS. The findings from those studies could hopefully inform the selection of appropriate target intervention communities and determine how best to allocate resources to ward against slippage. 3.3.6 OVERARCHING CONCLUSIONS – EQ2 According to GC, targets for sanitation are being exceeded and the majority of communities visited by the evaluation team have attained ODF status with a clean environment, hygienic toilets and hand washing stations. GOG’s emphasis on the ground is towards counting Open Defecation Free communities without necessarily putting in place the longer-term requirements to ensure these efforts are sustained. Until recently, little attention had been given to the durability of latrines, a key cause of slippage. With this challenge in mind, GC embarked on research and design of a technology option that could provide affordability, ease of installation and durability. After design testing and successful field trials, GC prototyped what is now known as the Digni-Loo, considered as a “game changer” for household sanitation. The Digni-Loo comes as a slab with one ring and a vent pipe for installation. (Additional rings can be purchased to lengthen the lining.) The Digni-Loo is appropriate for areas with loose soil and difficult terrains and can be moved when the pit is full. Priced at roughly US$80, the Digni￾Loo is considered affordable and more reasonably priced (at point of sale but also in terms of life cycle costs) compared to other sub-structure latrine options on the market. While strategies to roll out the Digni-Loo are being determined, households have been trained in latrine construction using local materials. Community members are well aware of the benefits of investing in and maintaining household toilets, as well as the health and environmental benefits of a community-wide approach to sanitation. The evaluation team found that small remote communities will generally continue to invest in 29 Indeed, related to the points made in Section 3.3.1 on durability, one could argue that the quality and durability of latrines constructed could provide a predictor of possible slippage. The more durable a latrine is, the less chance of slippage to ODF. Thus, while most toilets installed by households under the W4H were made of local materials, the Digni-Loo may be a solution that directly helps to address slippage. PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 33 construction of toilets (even if rudimentary) due to community reinforcement and mutual accountability. More innovations around the use of appropriate local materials for smaller communities and homesteads will support sustainability. While EHOs welcome a solid working relationship with GC staff, local authority involvement and ownership at more senior levels has generally been weak. This will ultimately undermine the sustainability of WASH systems put in place. The quality of initial triggering stages of CLTS is high, but systematic post-ODF sustainability monitoring and support was less in evidence. 3.4 EQ3: SUCCESSFUL APPROACHES TO PRIVATE SECTOR ENGAGEMENT FOR SANITATION While generally mentioned as having a role in the provision of household sanitation, most policy and strategy documents only refer to the private sector in their role as service providers or latrine artisans. There is often mention of the need to build capacity to carry out these functions. The NCWSP Project Implementation Manual for CWSA indicates that “where needed, capacity of the private sector to construct sanitation facilities is strengthened to benefit households with an interest to construct sanitation facilities and take their household members ‘higher up’ the sanitation ladder” (p. 16). The NESP talks about supporting “local private sector entrepreneurs and artisanal entities to produce machines, equipment, and tools appropriate for local use,” even though this places greater emphasis on solid waste management (p. 20). The RSMS also refers to “facilitating supply through the development of low-cost sanitation technology options, creating and strengthening existing sanitation supply chains, and enhancing the role of the local private sector (simple slab-makers, artisans)” (pp. 2 and 63). The RSMS further expands the role of private sector participation to go beyond the local-level artisans (micro private sector) to what is termed the “macro-level” private sector. However, these engagements are more related to corporate social responsibility (CSR). The RSMS refers to “enhancing the role of the macro private sector to exploit social responsibility funds, e.g., Ghacem, MTN, the Breweries, etc.” (p. 25) and noting that “EHSD will explore the possibility of getting subsidized cement from Ghacem for household latrine construction.” (p. 43) Other activities considered for the private sector are improving supply by focusing on producers, suppliers and stockists of goods, the provision of micro-finance (through rural bank credit lines), and the use of private media for communication and sanitation marketing. 3.4.1 FORMS OF ENGAGEMENT WITH THE PRIVATE SECTOR From the outset, GC sought to forge strategic engagement with the private sector and to select MMDAs in part based on the “potential to stimulate private sector supply of sanitation products due to market towns [and the] presence of related businesses.” A key goal of W4H is to “leverage public￾private partnerships to maximize impact” through market-based approaches for sanitation service delivery” (W4H Technical Proposal, p. 6). The activity has targeted partnerships both at the national and local levels by seeking to engage: • Large-scale, national-level manufacturers; • Other national-level private sector actors; • Regional and MMDA-based suppliers; and • Community-level entrepreneurs (e.g., sales persons and artisans). The evaluation team noted that W4H has excellent and emerging relations with national-level private sector actors. These relations take one of two forms: a business-driven, market-focused approach that is based on stimulating supply to meet demand for products and materials; or an approach that builds on a company’s CSR programs to facilitate provision of WASH infrastructure to communities. PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 34 At the national level, the activity has successfully engaged several large-scale manufacturers. An example of a strategic relationship is with the plastics company, Duraplast, which is based on clear business benefits to the company as a product manufacturer (i.e., a more progressive view of partnership beyond CSR). The engagement with Duraplast started with the supply of vent pipes to provide discounted products for sanitation facilities. GC then provided the specifications for Duraplast to develop the Digni￾Loo toilet (introduced in the previous section). GC also has established a relationship with Ghacem Limited (Ghana cement producers) through their distribution networks to supply materials to households at prices discounted from the market rate. Cement, provided through the Ghacem arrangement, is used to plaster latrines that are constructed with local materials to provide a more robust, waterproof, and aesthetically pleasing finish. The demand is aggregated at the MMDA level through EHAs working in coordination with NLs. MMDAs relay customer names and quantities to Accra for distribution with monies supplied upfront by GC. The distributor then sends the materials to the requesting MMDA directly. Distribution modalities down to the household level are still evolving to coordinate supply and demand. Interviewees confirmed this arrangement indicating that: “We buy the cement, zinc, and vent pipe from Kpassa. If you purchased from the MMDA assembly, the cement cost is reduced to about GHS26, but from the market, cement costs GHS35.” (Member of Gbosike Community, Nkwanta North) A second perhaps more traditional level of engagement with the private sector at the national level is through companies’ CSR activities that provide funding for the construction of WASH facilities in selected communities. GC has engaged with the Unilever and Caterpillar Foundations in particular, and other companies have also expressed interest in this type of arrangement. The FY18 W4H Annual Report indicated that ten institutional facilities with handwashing stations had been completed in selected schools in Tema through collaboration with the Unilever Ghana Foundation. An interview with Unilever indicated satisfaction with the partnership and the quality of work carried out by GC. The FY18 Annual Report also indicated a partnership with the USAID/Coca-Cola Water and Development Alliance, which resulted in the construction of seven institutional latrines in five schools. While helpful in meeting short-term targets and community needs, GC rightly notes that these types of arrangements are nice to have but not necessarily game changers in the sector. They would only start to have wider impact if they can appeal to a clearer commercial business case for a company and thus reach scale. While strong private sector engagements have yielded positive results, private sector approaches are unlikely to reach poorer, more remote communities without clearer efforts to support these market￾based approaches. GC recognized in their proposal that without support in aggregating sales, sparsely populated rural MMDAs are not viable markets for private sector sanitation enterprises. For this reason, the GC approach to private sector engagement has been to work through the EHAs and NLs to aggregate local demand and then to make connections to private businesses in the MMDA capital (as in the case of the Ghacem arrangement). Admittedly, providing cement for some remotely located and sparsely populated communities has not worked too well, particularly when this resulted in repayment delays. Secondly, given that W4H no longer works with local NGOs to transport cement, the additional costs have resulted in a leveling up to the market price. Because sales were not sufficiently frequent, storing cement meant for latrines proved a challenge for some distributors. PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 35 An interview with one DP suggested that the broader sector could benefit from distribution models like that organized with Ghacem and requested that GC explore how to make these opportunities open to other stakeholders. This suggestion is consistent with proposals in the RSMS that encourage the EHSD to “explore the possibility of getting subsidized cement from Ghacem for household latrine construction” (RSMS, p. 43). This may, however, require careful negotiation given that the different implementers have their unique approaches for working within the sector. While perhaps not possible in the remaining time, such arrangements are worth considering in subsequent programming where the private sector could be engaged in a similar manner. The key challenge is how best to aggregate demand to ensure that private provider transaction costs can be kept low enough to make it worthwhile as a viable business opportunity while targeting and structuring pricing to meet the needs of poorer households. At the community level, GC has trained artisans and more than 6,000 NLs in new skills to support community members. In most of the communities that the evaluation team visited, households had used local materials to construct the latrines themselves, sometimes with support from others in the community. These latrines constitute improved services in the Ghanaian context and show clear community commitment. 3.4.2 MARKETING OF THE DIGNI-LOO After extensive research and development work with Duraplast and incorporation of community inputs, the Digni-Loo was formally launched and introduced to the Ghanaian sanitation market in June 2018. The Digni-Loo currently comes as a slab with one ring and a vent pipe for installation. Users can purchase additional rings to lengthen the lining at the time of installation. “Priced at approximately US$81 [for the sub-structure hardware], the Digni-Loo… provides a more durable and hygienic product than traditional latrines.” (GC Case Study: A Market-based, Pro-Poor Approach to Rural Sanitation) The W4H FY17 Annual Report indicated that the Digni-Loo “was widely acclaimed by the GOG as an important sanitation option available for rural populations.” The report further indicated that following the launch, 400 units were initially produced and distributed to entrepreneurs in W4H activity regions with the expectation that the Digni-Loo would be sold to households. By the end of the year, W4H reported that there had been many inquiries about the Digni-Loo and 490 had been distributed to entrepreneurs throughout W4H MMDAs. W4H further reported that the “Digni-Loo continued to penetrate gradually the sanitation market in project regions as households, communities, and their leaders are buying into the Digni-Loo idea.” (GC FY17 Annual Report) The FY18 Annual Report indicates that 560 were produced in the reporting period, with 296 distributed to entrepreneurs in various districts to be sold, but 264 had yet to be distributed or purchased. Photo 4: A cement retailer in Nkwanta North PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 36 The evaluation team notes that GC has been rather cautious in the marketing of the Digni-Loo. GC staff noted that one reason for this is that the initial capacity of the manufacturer to produce large quantities has been limited. Duraplast indeed confirmed and then proved that they would stretch production capacity when clear demand is evidenced. Discussions were also had to determine if Duraplast would consider investing in manufacturing outside of Accra to cut costs for production and transport. To date, sales of the Digni-Loo and additional rings have been varied across the districts. While the GC Business Development Officer indicated that there are well-established markets in Northern and Oti Regions, there have been limited sales in the Western North and Central Regions. In Wassa Amenfi West District, for example, there has been a limited uptake of the Digni-Loo (slab and rings). Discussions with GC staff in the district indicated that the private suppliers who are already selling hardware products were cautious to stock the product for a number of reasons, including limited storage space30, lower profit margins, and potential capital lock-up due to slow sales. As noted, a unique feature of the Digni-Loo is the additional rings that can be easily installed in the pit as an attachment to the slab. This increases the cost, of course, but also reduces the risk of pits caving in. It was not clear, however, how many additional rings had been sold in addition to the one-ring-and-slab package. Visits to the field showed most of the Digni-Loos had the slab, one ring, and vent pipe. In addition to the rings, there is also a detachable plastic seat that has been added to the Digni-Loo slab in FY18 for greater comfort for users generally as well as to support disabled or elderly users. Interviews with GC indicated that they hold ownership of the patent and intellectual property rights for the Digni-Loo, but that USAID has unlimited access to the distribution of the product. The patent is to protect the Digni-Loo from being copied and to forestall a situation where GC could be asked for royalties or said to be copying other people’s products. Duraplast is the only company manufacturing the product, and they only manufacture the product based on orders received (i.e., what it has been instructed to produce). Even though there is a patent, the W4H Chief of Party indicated that any group wishing to use the product could make a request to GC and the activity team would assist in placing or approving the order with Duraplast. GC could provide some guidance to sector players on how they can also access the Digni-Loo for large-scale production. Interestingly, Duraplast sees the product as belonging to GC and has not formulated its own plans to produce, market, and deliver the product to suppliers. In those MMDAs that the evaluation team visited, the Digni-Loo is not yet readily available to be sold by local businesses stocking plumbing supplies (vent pipes and other hardware). GC was still refining the 30 One interviewee suggested that both storage and transportation would be easier if the Digni-Loo could somehow come in a flatpack design. Photo 5: Digni-Loo in a W4H community PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 37 strategy and roll-out through identified distributors and sanitation entrepreneurs at the time of the evaluation interviews. Visits to Shai Osudoku, Bodi, Amenfi, and AOB indicated that beyond pro-poor support, Digni-Loo distribution and marketing have not yet reached these areas. The GC Business Development Officer indicated that the markets were more likely to take off in STK, Nkwanta North, and Krachi East MMDAs due to fewer competing products on the market. Interest in Digni-Loo received a major boost when the government, through the GOG CWSA with support from the World Bank’s Rural Sanitation and Water Supply Programme, placed a large order. As a result, the manufacturer created more molds and expanded production capacity to approximately 1,800 units per month. At the time of this evaluation, the company was on course to supply an order of 20,000 Digni-loos for CWSA by November 2019. GC has indicated the intention to employ a local distributor model through the use of active and effective NLs as a micro-entrepreneur scheme. The W4H FY19 work plan indicated that in the course of 2019, GC would assist with field-level interventions including business development training and consulting for sanitation enterprises. As a result of these trainings, sanitation entrepreneurs would be equipped with the requisite business and development skills to take their businesses to the next level. This scheme, which currently has trained about 30 micro-entrepreneurs, is expected to be able to establish and scale up the supply of the Digni-Loo to meet the demand created from upcoming market activation and promotional activities. GC staff noted that promotional videos had been developed to support the marketing of the product and were to be aired before the end of FY19. 3.4.3 SANITATION SUBSIDIES WASH projects and programs apply financing support in the following ways: 1. Promoting household demand through marketing activities (CLTS, community mobilization) 2. Developing sanitation markets through negotiation of sales at wholesale prices and free transport when demand is aggregated 3. Providing subsidies on hardware and toilet facilities: a. Directly to households (paid to installer/artisan) b. Indirectly paid to suppliers through support for product development c. As (pre)funding for product development 4. Subsidizing financing to support provision of credit to households Numerous interviewees, particularly at the DA level but also including GC and DPs, noted that conflicting approaches to subsidies by different stakeholders, even in adjacent communities, is creating confusion for households who continue to wait for further support. Thus, practices of government agencies (including CWSA) or other DPs, or CSR efforts, have the potential in the short term to distort the market if a heavily subsidized, ill-targeted approach is adopted. In response, GC, which provides the product to suppliers at cost, is seeking to time its activities after the phase-out of the GOG-World Bank scheme mentioned above, which provides the sub-structure Digni-Loo to households for free. Discussions with government representatives at the national level indicate a revision of the GOG’s own stance and approach toward subsidies. It is worth noting that the word “subsidy” is missing from the Pro-Poor Sanitation Guidelines. The only reference is to the “sale of customized/specialized toilets at subsidized rates.” The Pro-Poor Sanitation Guidelines categorize support into hardware, software, and operational. Support is expected to be in the form of construction materials, labor and volunteer costs, special latrines at subsidized rates, and the provision of construction equipment. PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 38 As food for thought, operating in a heavily subsidized sector, Total Family Health Organization (TFHO, formerly PSI) representatives interviewed indicated that the health sector has realized how disruptive subsidies can be if not well designed and that the health sector continues to explore various models for subsidies linked to sustainability. Current approaches include cross subsidy, full subsidy, and time-limited subsidies for essential commodities. Targeting approaches consider geographical location, economic status, and biological status. In terms of sanitation, approaches to subsidies are not aligned across different implementers and efforts to identify what works and what works less well should be a primary topic for discussion at sector coordination forums. Using the Pro-Poor Sanitation Guidelines as the starting point, GC’s relatively neutral position as well as its learning from the W4H activity and other programs could be instrumental in fleshing out the issue. Recent World Bank studies and forthcoming research by WASHPaLS with UNICEF and GC aimed at both qualitative and quantitative analysis of CLTS success factors should shed further light. WASHPaLS’ key research questions for this work are as follows: 1. To what extent do targeted subsidies within ODF communities result in sustained latrine coverage, quality, and use among the most vulnerable households? 2. To what extent do these benefits spill over to the rest of the community? 3. What are the costs and challenges of implementing a post-ODF targeted subsidy program? GC and MMDA staff in districts visited in Western North and Central regions in particular indicated that the blanket distribution by CWSA of Digni-Loos to all households without toilets in some communities had an effect on their own interventions in neighboring communities. For example, in Bodi, GC staff noted that: “CWSA is currently distributing Digni-Loos in Juaboso District, which is close to Amorya where we work, and this creates issues for us.” (GC District Staff, Bodi) Similarly, a DA staff member in AOB noted the following: “CWSA is here in this district … They are providing 100 percent support. They are giving a Digni-Loo to every household. I am the same person who works in Ankase and Kwadwo Kokoo; I tell households to use local materials or local-based resources, then I have to go and distribute Digni-Loos. It is a security issue. The people will think that I am corrupt and that I am favoring some communities.” (DA AOB Staff) He further indicated that the community members raised this as an issue since they did not understand the differences between the two projects but saw the same faces from the DA. As indicated by GC’s W4H Chief of Party, the model districts where implementation is less challenging are those without a lot of different actors. “Model districts, like Sawla Tuna Kalba and Bole are more remote, so no one wanted to work there. We are the pioneers going there. [Community members] were receptive, and they were not receiving conflicting messaging like in West Mamprusi and other places with a lot of donors.” (GC Chief of Party) Indeed, the CWSA-World Bank program is expected to end in November 2019. Although 16,000 (of the 20,000 planned) Digni-Loos have been distributed to MMDAs, several MMDAs visited by the PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 39 evaluation team noted that a significant number have not yet been distributed to communities. One respondent cynically noted that such distribution and installations will likely take place just prior to the next election in December 2020. 3.4.4 OTHER PRODUCTS ON THE MARKET In an interview with TFHO, a USAID-supported social marketing organization, interviewees indicated that, on behalf of the manufacturer, Lixil, they had begun conducting market research, direct marketing, and a sales trial for the SaTo pan. Introduced in other countries in Africa, the SaTo pan has a full cost price in Ghana of GHS25 (US$4.50–5.00). Unlike the Digni-Loo, however, the SaTo does not offer a whole solution for the sub-structure, but is only one component that needs to be fitted into a slab. Given that it can be combined with various options, TFHO is expecting a good level of uptake in some of the communities in the trials. TFHO is licensed to manage the roll-out in Ghana. As part of the trials, a market survey is exploring reasons why households make or do not make investments in toilets. TFHO indicated that 20 masons had been trained to support these trials. Retail shops have been engaged to stock the SaTo pan with promoters going into communities to market the product. It is currently being sold at events such as festivals where other TFHO health products are sold. Partnerships have also been sought with UNICEF, WaterAid, and sanitation entrepreneurs who are providing household toilet facilities both in rural and urban settings. TFHO is planning a large-scale national launch for the SaTo pan; GC could capitalize on this opportunity to market around the SaTo pan and develop a partnership with TFHO to support the marketing of Digni-Loo (since the SaTo pan and Digni-Loo offer different solutions for households). As noted in earlier sections, other alternatives include VIP latrines (constructed using blocks), septic tanks, and biofil digestors, which are likely to be unaffordable for most households. In line with this, GC should catalog the different options, their benefits, and (potentially life cycle) costs to provide further information to households to aid in decision-making. 3.4.5 OVERARCHING CONCLUSIONS – EQ3 While a work in progress, GC’s multi-faceted approach to engaging with the private sector shows promise, particularly in the different ways in which GC is leveraging investments from the corporate sector to support the program. The approach provides the corporate sector with a strategic partnership that not only delivers commercial benefits but also social benefits through contributions to the SDGs. This approach of moving from traditional CSR to a “win-win” partnership shows the potential of providing sustained engagement with companies. The evaluation team is confident that in the final year of the activity W4H will work to sustain the contribution that these players are making to the sector. W4H did not adopt a one-size-fits-all approach but employed flexible and differentiated approaches across corporate organizations, manufacturers, distributors, and local artisans. These approaches need to be clearly spelled out to elaborate the different strategies for CSR and private sector engagement aimed at improving supply chains and market development. PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 40 While competing technologies are emerging, there is sufficient room in the market for a wide range of solutions. The Digni-Loo makes a significant contribution to the range of sanitation solutions for households, particularly given its affordability, durability, and ease of installation. While confident in the product, W4H is rightly sizing up potential market distortions from others’ activities and trying to position its approach accordingly. Developing markets to support uptake will very much depend on the ability of W4H to aggregate demand and to demonstrate to retailers, particularly in areas with some density and where demand may be higher, that they can make sufficient sales. This approach may create the groundswell that GC believes is possible. GC could potentially benefit from social marketing approaches that are being piloted by TFHO and could certainly explore some form of partnership with this organization. Using GC’s own framing (as per the case study entitled “A Market-Based, Pro-Poor Approach to Rural Sanitation”) (see adjacent box), the W4H activity is enabling, facilitating, and engaging the market. Enabling efforts are primarily geared around “social normalizing,” i.e., influencing the knowledge, practices, and experiences of a population with respect to sanitation behaviors. GC has been instrumental in supporting the MSWR in creating policy through the Pro-Poor Sanitation Guidelines, albeit with work still to be done by the ministry to roll these out and ensure their uptake. In terms of facilitating, GC has mostly focused on generating demand and monitoring and evaluating, with fewer efforts looking at financing, coordinating stakeholders, building capacity, and brokering knowledge. GC’s marketing W4H CASE STUDY: A MARKET-BASED, PRO-POOR APPROACH TO RURAL SANITATION Market Enabling • Creating and enforcing policy. Creating and enforcing laws and legal guidance that have an impact on the market (e.g., health laws, technology specifications/standards, certifications, subsidies, vouchers, tariffs, taxes). • Developing infrastructure. Developing and maintaining transportation, utility, and information conduits (e.g., roads, railways, power lines, and information and communication networks). • Social normalizing. Influencing the knowledge, practices, and experience of a population with respect to sanitation behaviors. Market Facilitating • Financing. Creating the availability of funds to market actors (e.g., via microfinance loans, extended payment periods, and revolving funds). • Coordinating stakeholders. Organizing collaboration between market actors. • Building capacity. Increasing market actors’ abilities to perform core functions, solve problems, and define and achieve objectives. • Brokering knowledge. Developing relationships and networks among producers and users of information. • Generating demand. Targeting marketing programs/campaigns to drive general awareness and interest in sanitation products and/or services. • Monitoring and evaluating. Tracking and measuring performance, including quality control. Market Engaging • Conducting customer research. Investigating the needs, preferences, opinions, and behaviors of consumers. • Planning. Determining and balancing aggregate demand and supply to develop a course of action that best addresses market gaps. • Innovating. Researching and developing new or modified sanitation products/services. • Developing products. Piloting, evaluating, and commercializing (including developing market strategies) sanitation products/services. • Sourcing and aligning suppliers. Identifying and coordinating material suppliers. • Supplying materials. Supplying component goods (e.g., concrete, reinforcement bars, toilet pans/bowls). • Coordinating logistics. Transporting materials from suppliers to producers and finally to customers. • Producing. Assembling materials into sanitation products. • Sales and marketing. Implementing the marketing strategy, including collection of orders and payments. • Providing aftersales services and feedback. Providing range of aftersales services to assist customers including: siting, installing, training, maintaining, upgrading, and removing and disposing waste. It also includes the collection of customer information, such as satisfaction and product use. Source: Borkowski, J. and E. Perez. A Market-based, Pro-poor Approach to Rural Sanitation. Global Communities, 2019. PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 41 efforts have been most active in effectively tackling the areas of customer research, planning, and innovation. Ultimately, further market research will be needed to segment providers and customers and to determine which market-based options hold the most promise and how best to catalyze those approaches. As per a respondent from the MSWR: “Disaggregating the market by wealth and ability to pay allows the market to have different strategies and conditions for: (1) the sector of the populations that can afford to pay for the products; (2) the segments that need to have access to credit to purchase the sanitation products; and (3) segments of the population that need subsidies.” (Kweku Quansah, MSWR) 3.5 EQ4: STRENGTHENING LOCAL GOVERNANCE TO SUSTAIN WATER SERVICES According to GC, the W4H activity installed 203 community boreholes (some of which were mechanized) in communities, schools, and health care facilities (HCFs) by the end of 2018. While these systems have been appreciated by beneficiaries, the evaluation team noted some gaps in the local governance systems, which are likely to affect the sustainability of services supported by the activity. (For a discussion on schools and HCFs, see the EQ5 section below.) A few key areas that the evaluation team were expecting to see included MMDA prioritization of water in development plans with increasing budget allocations and evidence of spending, clear ongoing collection and use of data to guide resource allocation, good governance at the level of the community WSMT, and increasing responsibility on the part of MMDAs for strategies to manage service disruptions and foster expansion. 3.5.1 MMDA LEVEL Within annual work plans and budgets, most MMDAs have WASH plans that focus on water with the District Water and Sanitation Teams (DWSTs) and District Planning and Coordinating Units as the lead authorities. Plans and budgets notwithstanding, the release of funds and other resources to support WASH service delivery is a key challenge—often delayed and at reduced amounts to what was originally agreed. MMDAs claimed they do not receive their share of the District Assembly Common Fund on time and the Internally Generated Fund does not provide enough to cover all activities. (Recognizing this challenge, one MMDA, Wassa Amenfi West, was seeking to increase the tax base by assessing property values more accurately. While discussions did not go into much detail on district financing, most MMDAs, however, appeared to be less proactive in their approach.) Photo 6: Water tank in the Tuna system PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 42 In terms of priority expenditures, MMDAs visited by the evaluation team noted that they spend more on health and education. Solid waste management usually came third on the list of expenditures. Thus, the DWST with responsibility for water services delivery faces similar challenges to their EHO counterparts seeking to deliver on sanitation. Minimal resources are provided for monitoring overall WASH activities in the MMDAs, even including those initiated by the MMDAs themselves. In most cases, MMDA staff do not have the necessary logistical means to carry out their responsibilities (especially motor bikes for field activities) and thereby often resort to using their personal motorbikes or taking public transport. This gap not only affects expansion of services, but sustaining existing systems through effective O&M is also affected. “Big bowl with small fufu inside. Everyone wants to eat from the same bowl.” (Environmental Health Officer, Sewi Wiawso) In terms of construction, like most DPs, W4H engaged private sector firms to drill boreholes and install water systems (and construct toilet facilities) in communities, schools, and HCFs. While MMDA officials were initially consulted on which communities to support, as noted by interviewed District Engineers, these construction projects were facilitated from the W4H activity office, generally with little, if any, MMDA involvement. The W4H Accra or regional offices drew up and monitored construction contracts. During their visits, the evaluation team noted a few systems that were not functioning properly or at all. In two cases, new systems had stopped working a few months after initial installation. Further work is needed by GC staff to follow up with beneficiary communities, schools, and HCFs to determine if the systems are working and, if not, if the Facility Management Plans (FMPs) are effectively supporting their upkeep. The activity team should also revisit obligations of the construction contractor through some form of guarantee or delayed final payment. A clear exception to community-level infrastructure, with W4H support and funding, a piped water system was implemented in Sawla Tuna Kalba MMDA, which serves three small towns. A Water Board was established and trained to manage the Tuna system with oversight responsibility from the MMDA Assembly. FMPs were prepared to guide O&M. Albeit still a new system, having only been handed over to the community in June 2019, the evaluation team noted the competence and professionalism of the management team, an efficient water monitoring system, transparency with the communities around tariff issues, and the satisfaction of the customer base in the services received so far. Although early days, further study is definitely warranted on how this particular system has become such a model operation to understand the factors needed for replication. A key challenge for the Tuna system is a proposed takeover by CWSA under its emerging mandate to operate as a quasi-utility of rural/small town water services. Although beyond the scope of this exercise, the evaluation team agrees that solutions are needed for problematic water systems that are not providing adequate services to the community but questions the remit of CWSA to take over well￾Photo 7: Secure pump house and solar panels in the Tuna system PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 43 functioning locally owned systems. A more localized challenge is to incorporate more female representation on the Water Board. At present, only two women are on the Tuna board of fifteen members. In terms of data collection and analysis, MMDAs visited by the evaluation team do not appear to have a well-functioning M&E system with regard to installed water systems. W4H, on the other hand, has been refining relatively simple and straightforward systems and collecting data on a regular basis. (Such systems deploy Open Data Kit [ODK] applications that can be easily used on mobile phones or tablets to collect and collate data.) To support local ownership and an emerging exit strategy for W4H, the activity team should share such systems with those MMDAs with appropriate capacity building to enhance their oversight, as well as to inform them on how best to allocate scarce resources. 3.5.2 COMMUNITY-LEVEL SERVICES At the community level, WSMTs were formed and trained by W4H to be responsible for managing newly installed facilities. The intention is for at least 50 percent of the WSMT members to be women. The evaluation team could not verify exact numbers, but discussions the team conducted with WSMTs often included at least 50 percent women. Funds are raised through pay-per-use or monthly/yearly household contributions depending on the option agreed by community members. Funds raised are paid into bank accounts established for O&M purposes. The evaluation team was able to meet with a number of WSMTs during community visits and generally found that the committees understood their responsibilities and took them seriously. The team visited a few communities for which efforts to guarantee sustainability may be more of a challenge. For instance, a community in Shai Osudoku and another in West Mamprusi each had three boreholes (at least one of which was constructed by GC) located relatively close to each other (within 100–150 yards). One had broken down, and there is no indication when repairs will take place. Because two are still working, there is no motivation to repair the broken-down facility. It also raises an issue of how three boreholes could be sited so close to each other. Pump mechanics exist in most of the activity’s operational MMDAs, though they were not trained by the W4H team. Communities have been linked to these resource persons either directly or more often through the MMDA staff teams. While some communities have sourced services from these mechanics for quick repairs, spare parts are not always readily available in the MMDAs. Some of them have to be procured from the regional capitals or other major cities like Kumasi, which increases transport costs. Communities noted that in some cases, the service costs of mechanics were high. 3.5.3 OVERARCHING CONCLUSIONS – EQ4 The W4H activity has supported a number of communities, schools, and HCFs with improved water services. A number of gaps are likely to constrain the sustainability of services delivered. Photo 8: Borehole and handpump in use in Belma, STK PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 44 For financial aspects, there was little evidence of MMDAs dedicating resources to monitor water infrastructure. It would be helpful to capture more information from the 15 MMDAs that are part of the W4H activity on allocation and actual release of budget for water services. More study is also needed to determine whether communities are charging and collecting tariffs in sufficient amounts to maintain and repair water points. In terms of institutional aspects, most MMDAs visited did not appear to have a sufficiently functional monitoring system in place that is owned by local government and able to collect and analyze data. GC has collected a wealth of data through a simple ODK tablet-based application that should be shared with the MMDAs to assist in their planning and resource allocation. (CWSA uses DiMES and Akvo Flow. It should be possible to incorporate ODK data into these systems.) The exception is the Tuna small piped water scheme appears to have much to offer on this and other aspects. For the purposes of replication, more study on why this system has been successful to date would be instructive. From an environmental perspective, there was no evidence of negative environmental impacts arising from the installation of infrastructure. Water quality testing is conducted as part of the borehole drilling process and the result is provided to GC. Although testing post-construction could no doubt be more frequent, few specific issues were raised with regard to water quality. In the community of Bedii (in Wiawso Municipality), community members complained that water from one of the boreholes in the community is corroding the metal washbasin storage containers, and water from a borehole in Wui in Nkwanta North smelled acidic and sulfurous. (These issues were raised directly with the GC local representative and, if not addressed yet, further testing is certainly needed on these boreholes.) From a technical angle, FMPs were generally in place for institutions but, as noted under EQ5 below, have supported maintenance of water systems only insofar as funding is available. A few communities do not have FMPs to guide O&M of water facilities. In Nuahele (Ada West) in particular, the WSMT was dissolved after the 2016 election and CWSA took over the small town’s water system. Receipts from sales are lodged with the District Assembly. At Holy Rosary/Dambai (Krachi East), according to community members, the “borehole has become a white elephant. There is no water, the borehole never reached the water table.” At Zanguga (West Mamprusi), the borehole had not been functioning for three months at the time of visit despite pump mechanics having worked on it. In situations of this nature, the solution to the problem may be beyond the capacity of the communities and the pump mechanics. It is important that the Assembly steps in to help the community solve the problem. To manage breakdowns, GC could look into linking up with SkyFox or other programs aimed at providing communities with spare parts at a reasonable and published price and in a timely manner using mobile ordering systems. (SkyFox is a registered company in Ghana that has been piloting and expanding a range of WASH and water-related services across the country and the region.) The SkyFox system was originally piloted under the CWSA-IRC SMARTerWASH program with the aim of aggregating demand for parts suppliers, creating more pricing transparency for communities in need, and feeding work to registered artisans. Without such systems in place, repairs could remain untenable and significantly delayed for many communities. From a social perspective, the evaluation team noted that the handover process to WSMTs in a few communities occurred before tariffs and other governance elements had been fully agreed with all stakeholders. This led to some confusion around payment, or even willful nonpayment, by users and conflicts over when the taps would be locked. This may have been more problematic for installations outside of the W4H activity, but it was the evaluation team’s sense that standard practices should be in place across all GC-supported interventions. PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 45 3.6 EQ5: INCREASE IN BASIC HANDWASHING BEHAVIORS 3.6.1 POSITIONING OF W4H INTERVENTIONS REGARDING HANDWASHING As noted in Section 3.2.1, the overarching goal of the 2010 NESP is to develop a clear and nationally accepted vision of environmental sanitation as an essential social service and a major determinant for improving health and quality of life in Ghana. According to the NESP, the Health Inspectorate (Environmental Health and Management Departments of the RCCs) is the first line of enforcement, working through a combination of education and persuasion. The objective is to make the community understand and accept its responsibilities with regard to environmental sanitation. Working in concert with this NESP objective, the W4H activity clearly defines Behavior Change Communication (BCC) as “the strategic, systematic and consultative use of communication to build target audiences’ conviction that an action (behavior) is the best choice for them and motivate them to try it and adopt it” (BCC Roll￾Out Strategy, 2016). The W4H intervention cuts across three main areas: a) household water treatment and safe storage; b) household latrine construction, use, and maintenance; and c) proper handwashing at the critical times. The focus of this evaluation under EQ5 is on handwashing behaviors, even though the evaluation team recognizes the critical role of safe water storage. (Latrine construction and use is covered under the response to EQ2 above.) GC measures its interventions based on the following two indicators: • Number of people practicing handwashing with soap under running water • Percent of households with soap and water at a handwashing station commonly used by all family members The W4H activity was designed to contribute to improved hygiene behaviors in communities, schools, and HCFs through the provision of WASH services. The approach for supporting behavior change in institutions is to provide simple infrastructure that supports handwashing as well as to support instructional agencies with materials and guidance on the practice of handwashing. In communities, W4H is designed to influence collective action on behavior change with the aim of improving health outcomes. Communities have by-laws, rules, and regulations discouraging OD; however, these are usually silent on the need for handwashing facilities. This could be attributed to minimal emphasis on handwashing in the ODF verification protocol. To counteract this and encourage simple hygiene solutions for handwashing, Tippy-Taps have been promoted for households to construct in close proximity to toilets and cooking areas and use with soap or ash. Coordination with statutory government entities and other WASH-related projects was framed in the early W4H activity documents as critical to the success around handwashing. There is evidence of some such coordinated work with MMDAs. The activity has collaborated with CWSA and the GES on BCC in Photo 9: W4H district staff member displaying BCC package PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 46 schools and communities. The latter provides oversight for basic and secondary education in Ghana. The SHEP is an integral part of the strategies for promoting WinS. Similar relationships were observed with the environmental health and sanitation departments (EHSU), whose work falls under the MSWR’s sectoral goal of promoting behavioral change programs and increasing access to safe hygiene practices. Overall, the focus and coordination appear to have been stronger at the sub-national levels (regional and MMDA) to achieve targets at the household level. Collaboration with other projects like S4H and C4H requires further attention, especially in terms of coordinating interventions in healthcare facilities. GC district staff work directly with EHOs to implement activities and maintain day-to-day contact. However, Planning and Coordinating Unit staff in several MMDAs mentioned that they were not sufficiently engaged in the implementation of W4H. Thus, with regard to W4H-constructed facilities, their ability to fulfill their mandate to coordinate and monitor MMDA activities was limited. The number of communities targeted and the logistics for carrying out the work are all agreed between GC staff and EHOs. As such, the MMDA fulfills its part by allocating time of the EHOs to support the activity. Reporting and data go to GC, with copies to the M/DEHO. Steps need to be taken, however, to establish proper reporting lines and to ensure that senior MMDA staff are sufficiently informed. The evaluation team did not explore this in detail, but stronger relationships with UNICEF related to training school, district, regional, and national SHEP coordinators could be beneficial. 3.6.2 FINDINGS ON PROGRESS AND ACHIEVEMENT OF HOUSEHOLD HYGIENE The team’s assessment of progress and achievement focused on five specific areas: the presence of handwashing facilities such as Veronica Buckets (mini-handwashing stations comprised of a bucket and basin on top of a wooden stand) and Tippy-Taps (containers at ground level that can be tipped by a foot pedal to dispense water) in institutions and households; functionality of handwashing facilities; the availability of soap; the evidence of handwashing practices; and school hygiene including support mechanisms, such as action plans and school health committees. Regarding the presence of handwashing facilities at the household level, the activity’s implementation approach has been to sensitize communities and emphasize the importance of investing in and maintaining such facilities. Messages seem to have been well received. Across most communities that the evaluation team visited, handwashing facilities were observed near or adjacent to the toilet. Takuka Community in West Mamprusi District in Northern Region is by all standards a model community for good practice in sanitation and hygiene. Each house had more than one handwashing facility: one close to the toilet and the other close to the household. Some households had more than one latrine, with one designated for males and the other for females, and the team observed good environmental cleanliness with graywater/wastewater properly managed and hygienic surroundings. Nuhale Community Photo 10: A household Tippy-Tap in Krachi East PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 47 in Ada West proved the exception where the evaluation team did not see a single household with handwashing facilities close to their latrine. In FGDs, Nuhale community members indicated that they constructed and used the handwashing facilities at the initial stage of the project. They stopped after they were declared ODF as they thought the project had ended. NLs said they were verbally abused when they tried to tell household members to reconstruct the handwashing facility. (Indeed, even some toilets were converted to rooms for storing farm produce and other household items.) With regard to the technology, the evaluation team observed a few cases where Tippy-Tap containers easily crack after a period. Households with broken handwashing facilities attributed this to children playing with the Tippy-Taps (Nkwantanum, Krobo, and Patakro communities in Wasa Amenfi West MMDA). Thus, further messaging and support may be needed as households may not be replacing or reconstructing broken facilities quickly. Some respondents in West Mamprusi suggested that the containers could be sold to recyclers for cash. These factors have implications for the recurrence of hygiene-related diseases and illness. Across all communities, interviews revealed that soap was readily and inexpensively available in local kiosks and shops. Ash, as an alternative, was clearly available. Healthcare professionals noted increased awareness around the need to wash hands at critical times and the evaluation team’s numerous informal discussions with men, women, and children largely confirmed this. The MMDA Health Officer in Shai Osudoku noted that: “…[we] have not collected data yet, but observing people, there has been a change as a lot of children have become more conscious of washing hands, whereas in the past people would just pour water on their hands before eating. Washing hands was dipping in water without soap.” (MMDA Health Officer, Shai Osudoku) Communities reported specific positive experiences from practicing good hygiene that largely included health benefits, but also cost savings from reduced illness. Although community members and health authorities believe behavior change has contributed to reduction in illness, there is as yet no irrefutable data to validate this claim. The activity identified and trained community-level volunteers in all target communities to provide support in promoting positive behavior change as part of the CLTS process. Recognizing the critical role of these Natural Leaders in facilitating the sustainability of good hygiene behaviors, their continued motivation remains a major challenge for projects and local governments. 3.6.3 WASH IN SCHOOLS W4H provided schools with toilet and handwashing facilities and engaged in hygiene education and promotion. The activity organized training to underscore the management requirements of facilities in conjunction with the SHEP of GES. For toilets, the team supported preparation of Facility Management Plans with the School Management Committees (SMCs) and Parent Teacher Associations (PTAs) to facilitate O&M. In a few instances, the evaluation team encountered visible signs of deterioration. For example, at the Amate Koni Basic School in Ada West the team saw cracks on the school latrine, part of the roof was dislodged, and stones had been placed on top of the building, posing a threat to pupils. Some structural deterioration that the team encountered during the evaluation was due to quality of construction. For others, deterioration may have been due to the inability of schools to maintain the toilet systems. All schools under the W4H activity benefited from rainwater harvesting tanks sited close to the latrines. The activity provided 470 Veronica Buckets to beneficiary schools to promote healthy handwashing PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 48 habits among pupils and teachers (W4H FY18 Annual Report). During the rainy season, water is available for handwashing, but pupils face huge challenges in the dry season, except in schools with a water point nearby or where authorities buy water from private vendors. Some schools task children with fetching water from the community borehole when there is no water in the rain harvesting system. Schools depend on capitation grants that are pegged at only GHS4.00 per pupil per year. Of this, GHS1.00 is expected to be used to maintain WASH services (including maintenance of the infrastructure, detergent/cleaning materials, and buying water from water vendors when needed). Not only are the capitation grants generally insufficient to ensure maintenance of WASH facilities, the release of the funds is often delayed by government. A handful of schools visited had old, non-functional toilet facilities adjacent to new infrastructure constructed under the W4H activity. If the old facility was not maintained, the evaluation team questioned what reassurances there would be that the new system would be well looked after, even with Facility Management Plans in place. According to people interviewed, community members in OD communities without toilets seek to access the school toilets (new and old) and, in the process, break the locks and doors and destroy handwashing stations when the toilets are locked. Indeed, the team observed directly two instances where older school toilets had been vandalized by community members. Schools were provided with gender and disability-friendly VIP latrines. These facilities have changing and washing rooms for girls to facilitate menstrual hygiene management and an attached facility to dispose of used sanitary materials. The evaluation team did not see full view mirrors in the girls’ changing rooms as recommended in the WinS technical guide. In spite of significant progress in hygiene messaging with most interviewees at the school/community level seemingly aware of the importance of handwashing, evidence of the uptake of hygiene practices was mixed. Some challenges were structural, with insufficient clean water to facilitate good hygiene behaviors particularly in the dry season. Some were behavioral (e.g., failing to put up handwashing facilities at key vantage points and showing lukewarm attitudes toward children’s health) or due to limited or lack of knowledge on all critical times to wash hands. Some challenges were operational with unreliable sources of soap. As noted, households in communities visited reported that soap was readily available in local shops, but there were mixed findings around the actual presence of soap to facilitate handwashing practice in schools. Some schools relied on the PTA to provide soap or on parents to send soap in with their children on a periodic basis. Some PTAs and community leaders provide detergents and cleaning materials. However, this support has not been sustained in most schools. Some SHEP coordinators and head teachers stated a reluctance to ask parents or flagged parents’ unwillingness to contribute due to the GES’ two directives (issued in January 25, 2017, and May 7, 2019) that prohibit schools from imposing dues or levies on pupils and parents. In Shai Osudoku, a handwashing facility in one of the three schools visited did not have soap for pupils to use. Indeed, the SHEP coordinator mentioned that parents usually “Provision of soap for all handwashing facilities will be the responsibility of the school with support from the PTA/SMC.”—Technical Guide for WASH in Schools Facilities, January 2014. Photo 11: Institutional latrine at a school in Krachi East PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 49 provided soap and detergents for pupils to use for handwashing, but that no contributions had been made recently—presumably either as it was the end of the term or because of the prohibition on levying fees from families. (Other school visits by the evaluation team were made out of term time.) Skills development and knowledge building for managing facilities represented another important approach adopted by the W4H activity where significant results had been achieved. W4H worked closely with the MMDA SHEP coordinators to ensure government-led hygiene trainings were conducted at beneficiary schools. In addition, a total of 1,160 teachers, SMC representatives, and PTA members of the 232 beneficiary schools were taken through toilet user education to ensure the toilet facilities are used hygienically and can be maintained over a long period of time. Furthermore, requisite guidance documents were developed to facilitate the work of frontline health personnel. A BCC package was developed for use by school staff, health officers, EHAs, and community volunteers to guide the promotion of good hygiene practices. Copies of the materials have been sent to Community Health Planning System (CHPS) compounds, MMDAs, and community-based organizations to make them accessible to hygiene promoters to continue to promote hygiene behaviors after the W4H activity ends. Teachers of some schools visited by the evaluation team indicated that School Health Clubs had been set up and were functional, but the impact of these activities could not be verified because schools were on holiday during the evaluation. Similarly, the evaluation team could not observe handwashing practices, except to assess the facilities and speak with the head teachers and some local children for their account of handwashing facilities in the schools. 3.6.4 WASH IN HEALTHCARE FACILITIES The W4H activity provided mechanized boreholes with water filtration systems for 12 HCFs (health centers and CHPS compounds) in different parts of the country. The aim is that nurses and patients would no longer access water from old wells with potentially circumspect water quality. These efforts were aimed at bringing improved water closer to the HCFs to enhance delivery of health services. The systems are not always piped into the HCF directly, however, so staff and patients may still be required to access water from a tapstand somewhere on the property. Due to the technical nature of the mechanized boreholes, measures for preventative maintenance and repairs generally have to be provided by the contractor who constructed the facilities. Contracts have a liability defect clause, which obliges the contractor to guarantee good working order on handover and to maintain the system for an initial period. That said, at the time of the evaluation team’s visit to the Kabonwule HCF in Nkwanta North and another in Sefwi Wiawso, the water facilities were not functioning and there was no clarity on how to resolve them. These newly installed water points had broken down within four months of installation. HCF senior staff were not clear on aspects of basic functioning of the water facilities installed or indeed on who to call to address any problems. Indeed, the Facility Management Plan was not readily available and actions for recourse were not straightforward. As noted by the health care center representative: “The system has broken down and not been working for the past two months. We have contacted the contractor [outside the region] through a number provided, but they have not come. I hear they will come from Accra or Takoradi.” George Dapilah, Ayinabrem Health Center, Sefwi Wiawso Municipal “Basic Service in Hygiene for HCFs: Functional hand hygiene facilities (with water and soap and/or alcohol-based hand rub) are available at points of care, and within five meters of toilets.” —World Health Organization and UNICEF (JMP): WASH in HCFs: Global Baseline Report, 2019. PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 50 Photo 12: HCF in Krachi East. Mechanisms for long-term sustainability, especially when the defect liability clause expires, are unclear. The health facilities depend on quarterly operational support from the MMDA Health Management team to enable them to operate and maintain the health system, which includes WASH services. Electricity bills from pumping the water are sent to the MMDA Health Management office for payment, which is often delayed. Health centers frequently do not receive their own allocations on time, and the amount of funding is insufficient. As a result, to maintain a hygienic environment, the evaluation team was told of instances in which nurses purchase detergents and undertake cleaning themselves. In terms of an integrated approach, while some had benefited from sanitation-related support from S4H, a number of HCFs that the evaluation visited did not have any or adequate latrines and staff were not aware of any plans for such investments to be made. Thus, the program of work could be more integrated with the connection between WASH services and effective health services delivery more clearly established. The evaluation team noted a collaboration between GC and S4H in the Oti region that led to GC providing mechanized boreholes and S4H providing the health infrastructure. However, there was little evidence of the two working seamlessly to ensure effective software aspects (i.e., Facility Management Plans and proper handover). 3.6.5 OVERARCHING CONCLUSIONS – EQ5 Throughout the evaluation, the team was struck by the importance of intervention sequencing. CLTS approaches are based on a series of interventions that are meant to build on one other. Water service interventions require a series of steps to put in the infrastructure but also to ensure local ownership. For handwashing behaviors, conveying, instilling, and reinforcing messages need to be integrated into a clear step-wise approach that links into and feeds off of other processes that are designed to ensure that water is available and that toilets are functional. Only through the coordination of WASH provision across institutions (schools and HCFs) and communities will behaviors, infrastructure, and services be sustained for lasting benefit. The evaluation team noted that work under the W4H activity may, for example, have been done in schools, but perhaps not in the wider community around CLTS or for the HCF. According to a GC staff member in AOB, some of the communities that benefited from school latrines are not ODF. As such, the lack of integration may bring, for example, instances of continuous vandalism, fractured community cohesion, and wider knock-on effects. Comprehensive or integrated community WASH programs will prevent community members from vandalizing school toilets because they lack toilets, and pupils will practice consistent handwashing only where there is sufficient and regular water supply both at home and at school. A full complement of WASH options reinforces positive hygiene practice in schools, households, and HCFs. Positive hygiene practices then impact on health, livelihoods, education, and other aspects of life. A few practical steps are needed to ensure functioning systems. Operator-friendly Facility Management Plans are a critical part of the process. Planning for an effective and efficient management of WASH facilities in schools and HCFs, according to the Ghana Education Service National Implementation Model PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 51 WASH in Schools (2014), is critical for the long-term sustainability of the facilities provided. Relevant stakeholders must play their roles effectively during the design process and in facility O&M, recognizing that the SMC and PTA are responsible for the overall governance of school WASH. Facility Management Plans were in place in the schools visited; however, one of the major constraints to implementing these was the reliance on capitation grants, which were often delayed and inadequate. In the absence of sufficient capitation grants and in light of the new GOG policy on not levying funds from parents, schools need to find innovative ways to engage PTAs for the O&M functions of school WASH facilities. Regular collection, analysis, and triangulation of information/data to support claims of “social change” and reduction in WASH-related illness are critical. Sources of such data will include facility-level records for reported or treated cases across the various related institutions (HCFs, schools, and households). Claims on reduced illness as a result of WASH project interventions need to be substantiated, first and foremost, by concrete evidence. This data should help make the case for where to invest specifically, but also to highlight the overarching benefits of investing in WASH. Reaching ODF status is only one step to improving health outcomes. For handwashing practices to be sustained, MMDAs and D-SHEP coordinators need to prioritize post ODF monitoring to ensure that ODF status is maintained and, where possible, continuous efforts are made to raise communities up the sanitation and hygiene ladder (from “Basic” to “Sustainable Sanitized”). While the indicators for measuring W4H achievement in this area are helpful, the evaluation team would suggest some reflection on the number of people having access to integrated, functioning WASH services across all primary areas of life—at the household/community, school, and HCF levels as a highly aspirational target. The ultimate goal would be to strive for “Sustainable Sanitized” Communities, as per the ODF Verification Protocol. PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 52 4.0 RECOMMENDATIONS The evaluation team finds that GC is having a positive impact and contributing to expanding and ensuring sustainable access to water and sanitation services. The evaluation team also recognizes that many of the sustainability challenges noted throughout this report are beyond the control of GC alone. The team notes that the questions posed in the evaluation SOW are the correct ones, starting with EQ1, “What are the policy gaps that need to be addressed for a more functioning and integrated system that ensures that gains made in access to services and the subsequent public health and other societal benefits can be sustained?” Effective, functioning operating environments require clearer designation of roles and responsibilities, more coordinated linkages and incentives, and greater levels of dedicated resourcing. Instead there is a reliance on programs like W4H to fund and deliver basic services in Ghana, without clearly embedding these programs in the wider context. With Ghana recently declared as a lower￾middle income country, the contribution of DPs is shifting with interventions like W4H required to position itself going forward as part of the Ghana Beyond Aid/WISER Ghana movement. Work is needed on the part of the GOG at all levels to prepare for these shifts. For programs like W4H, a key focus on MMDA governance and the prioritization of WASH could be included more explicitly for the Ghanaian context. This refers in part to a needed emphasis at the national level on the timely and more predictable release of the District Assembly Common Fund (DACF). An increased prioritization of WASH at the MMDA level should result in increased MMDA budget allocations from the DACF. Both of these actions would allow for more effective and sustainable programming. A further requirement is for data and learning to foster greater coherence of approaches by key stakeholders across the WASH sector. While the W4H activity makes significant strides in reaching communities with services, governance weaknesses need to be constantly flagged with local and national authorities to ensure that roles and responsibilities laid out in numerous policy documents are appropriately resourced and effectively honored. Momentum is easily lost if programs are held up by delayed release of funds, insufficient priority is placed on WASH as a key contributor to achieving health and education gains, and local government staff turnover leads to revisiting local objectives. These challenges are particularly acute for interventions like community-led total sanitation (CLTS) that are timebound and sequential and require follow-up to ensure that sanitation services are sustainable. With a broader view in mind and based on the findings and conclusions presented in Section 3, the evaluation team provides the following recommendations for GC, USAID/Ghana, and the wider DP community going forward. Short-term recommendations are presented in five categories corresponding with the EQs: policy gaps, sanitation sustainability, sanitation and the private sector, local governance and water, and handwashing behavior change and institutional support. Longer-term recommendations propose issues that USAID should consider when designing future investments in the Ghanaian WASH sector and that GC can take forward in advancing the work of W4H. 4.1 SHORT-TERM RECOMMENDATIONS 1) Policy Gaps a) General Policy Recommendations 1. Support the MSWR and Ministry of Local Government and Rural Development through training materials and capacity building to cascade policies and strategies more effectively down to the local level. 2. Systematically track budget allocation and actual spending on WASH at the MMDA level, and support and advocate for efforts for tracking at the national level with other stakeholders. PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 53 b) Pro-Poor Guidelines 1. Specify roles of different stakeholders with regard to how best to implement the guidelines. 2. Work with the MSWR to disseminate the guidelines further to MMDA staff. 3. Develop an M&E framework to monitor implementation of the guidelines. 4. Create a simple standalone checklist to support implementation of the guidelines. 5. Keep records and promote record-keeping regarding guideline implementation, including the households identified, any inconsistencies with other social welfare registers, and an explanation of why and what kinds of support individual households received. 2) Sanitation Sustainability a) Situate the implementation of WASH interventions more effectively within MMDA systems and structures to facilitate greater ownership and more effective planning, monitoring and data collection, and, ultimately, funding allocations. b) Develop a strategy to share the data collected by W4H more effectively with MMDAs. c) Clarify incentives for communities for achieving ODF. Where possible, this could be about sequencing support with other investments (for schools, boreholes for communities, etc.). d) Actively share the findings from the WASHPaLS studies, when completed, on the performance envelope of CLTS effectiveness and the subsidy analysis with MMDAs to inform the selection of and help negotiate targets for intervention communities and to determine how best to allocate resources for follow-up with communities to ensure they remain ODF. e) Strengthen the understanding of EHOs in the various levels of the CLTS ladder to undertake post-ODF monitoring and support communities to move up the sanitation ladder. f) Where possible and relevant, support further sharing between MMDAs (mainly at the EHO level) in adjacent MMDAs to generate cross-learning, greater ownership, and some level of competition. g) With USAID support, advance discussions on expanding the reach of BaSIS to all regions as well as support the process of harmonization of data at the national level. h) Situate the Sanitation League Tables more prominently in discussions with MMDAs to foster competition. i) Based on the W4H experience, revisit the design of the GC-MMDA MOU to consider an explicit set of activities to build MMDA capacity to strengthen their role in planning, budgeting, implementing, and monitoring sanitation activities, as well as to note increasing contributions year-on-year to show commitment and foster local ownership. j) Log specific dates for the CLTS stages (from when triggering started to when the community was declared ODF). This will help to clarify expectations around behavior change in communities and factors that determine relapse. Analysis of timelines could then inform the levels of effort needed for different kinds of community contexts. 3) Sanitation and the Private Sector a) Clarify both the economics and finances of different technologies (Digni-Loo and local construction for sub-structure) in terms of life cycle costs, direct or hidden subsidies in the system at present, and who currently pays for what. As possible, compare to other existing and emerging technologies to understand possible positioning in the market from a financial perspective. PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 54 b) Clarify the strategy for the roll-out of the Digni-Loo and determine which partnerships for social marketing would be of most use. c) Accelerate the process of marketing for wider audiences beyond the pro-poor target. d) Clearly spell out the separate strategies for different private sector engagement (CSR for non￾WASH related companies and market development for companies like Unilever, Ghacem, and Duraplast; local private sector distributors; artisans; and private sanitation entrepreneurs). 4) Local Governance and Water a) Document the (STK) Tuna community water supply model carefully to understand what factors have led to emerging success. b) Clarify through the MOU with MMDAs who is responsible for providing oversight for future systems involving MMDA engineers more consistently from the start. c) As appropriate, link to other parts and artisan maintenance supply systems (like that of the SkyFox SMS reporting system and artisan matching aimed at reducing downtime of handpumps). d) Follow up with beneficiary communities, schools, and HCFs to determine if the systems are working and, if not, whether Facility Management Plans are effectively supporting their upkeep. e) Revisit the obligations of the construction contractors through some form of guarantee or delayed final payment. 5) Handwashing Behavior Change and Institutional Support a) Handwashing 1. Continue to ensure and monitor that water is available through an integrated approach (water, sanitation, and hygiene). 2. As a best practice case, study and document Takuka in West Mamprusi to understand factors of success for handwashing behavior change, maintenance of clean toilets, sustainability of water facilities, and management of wastewater. b) Schools/HCFs 1. Work with MMDAs and District School Health Education Program (D-SHEP) to prioritize post-ODF monitoring to ensure that handwashing practices are sustained and to avoid relapse. (As noted earlier, the ODF verification protocol is largely silent on handwashing.) 2. Clarify roles and responsibilities with education and health authorities about the maintenance of WASH facilities in supported institutions. 3. Ensure that all schools and HCFs have Facility Management Plans in place to comply with implementation requirements and ensure a proper handover to beneficiary institutions through head teachers, Health In-Charge officers, etc. 4. As with recommendation above under Local Governance and Water, revisit contract management for construction to ensure that facility managers have recourse in case of system failure. (This might mean holding back a portion of the final payment for the first three to six months.) 5. Support schools to engage PTAs in innovative ways of operating and maintaining WASH facilities in light of the new government policy of not levying fees for pupils. At the national level, GC and partners should advocate for appropriate funding for WASH in schools by joining ongoing initiatives for increased sector financing. PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 55 6. Ensure that schools with toilet provision have viable access to safe water nearby to facilitate/promote handwashing. 4.2 LONG-TERM RECOMMENDATIONS a) Policy Gaps 1. Update policies and strategies to reflect the more ambitious targets of the SDGs. This effort could provide the opportunity to consolidate and refine, making policies and strategies more accessible to those needing to apply them. 2. With efforts to consolidate the WASH sector policy and strategy documents, incorporate the Pro-Poor Sanitation Guidelines into these documents as a part of a more coherent whole. 3. Track guidance on gender aspects and marginalized households through GOG policies and strategies to clarify guidance. 4. Promote greater structural alignment between MSWR and MMDAs to facilitate effective implementation and monitoring of policy effectiveness. Toward this end, place greater emphasis on the MMDA Water and Sanitation Plans (M/DESSAP) as planning tools and the Regional / MMDA Inter-Agency Coordinating Committee for Sanitation (R/M/DICCs) as maintained forums post-project for guiding implementation at the local level. 5. Put in place structures and systems to ensure that policy monitoring, data capture, and learning influence policy more systematically through support to the MSWR and through more purposeful debate in sector forums. 6. Support the MSWR to strengthen the coordination and alignment of DP programs and activities. Forge linkages to (or at least greater awareness of) health, education, environment, and other programs that intersect with WASH. 7. Provide a facilitated conversation with key players about application of the Pro-Poor Sanitation Guidelines and any gaps in the policy through future WASH sector forums to ensure ownership and uptake. 8. Provide easy access to Livelihood Empowerment Against Poverty (LEAP) and other social welfare databases for community organizers and MMDA officials to cross-check recipients. b) Programmatic Considerations 1. Negotiate reasonable and increasing targets for MMDAs in terms of funding allocations and other resource commitments to be met over a gradual period to foster greater local ownership. Phase out MMDAs not meeting these targets from the program. Review MMDA local ownership through a comparison of UNICEF and other approaches to working with local government. 2. Design post-ODF support and monitoring in collaboration with MMDAs. 3. Track progress on access to sanitation by wealth quintiles. Facilitate this process through implementation of the Pro-Poor Sanitation Guidelines. 4. Ensure a facilitated conversation with key players about the role and use of subsidies to support households through future sector forums. 5. Support the clarification of the CWSA remit and criteria for and approach to taking over viable community-managed schemes. PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 56 6. For WASH interventions, coordinate school provision, healthcare provision, and ODF triggering for more comprehensive coverage in communities to aim more deliberately for “Sustainable Sanitized” communities. PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 57 ANNEX I: EVALUATION STATEMENT OF WORK End-Term Performance Evaluation of WASH for Health I. PURPOSE OF THE EVALUATION The primary purpose of the evaluation is to conduct a performance evaluation of the USAID/Ghana WASH for Health (W4H) Activity to assess if the activity’s design and implementation are successfully meeting its goal to expand sustainable access to water and sanitation services. The evaluation will inform any midcourse corrections for the 13 months of implementation. The evaluation findings will also inform the development of future WASH activities. The primary audience for the evaluation report will be the USAID/Ghana Health Office, current implementing partners, the Government of Ghana, and water sector working group/collaborators in Ghana. To a lesser degree, the evaluation findings may be used by USAID/Washington technical staff to inform efforts in the WASH sector. II. SUMMARY INFORMATION Table 1: Audiences and Dissemination Plan for Mid-Term Evaluation Strategy/Project/Activity Name WASH for Health (activity) USAID Office USAID/Ghana Office Health, Population, and Nutrition Implementer(s) Global Communities (prime) and Manoff Group (sub) Cooperative Agreement/Contract # Cooperative Agreement # AID-641-A-1500005 Total Estimated Ceiling of the Evaluated Project/Activity(TEC) $19,393,256 Life of Strategy/Project/Activity February 2015-September 2020 (includes anticipated cost extension) Active Geographic Regions Five regions: Greater Accra, Western, Central, Volta and Northern Development Objective(s) (DOs) DO3 Required evaluation? No External or internal evaluation? External PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 58 III. BACKGROUND A. DESCRIPTION OF THE PROBLEM AND CONTEXT Ghana accomplished its Millennium Development Goal for drinking water in 2008 (halving the population with an unimproved water source), however this progress was highly inequitable and significant gaps remain. Today, basic water service coverage ranges from 47% to 96% depending on the region. The sector is supported by effective policy and institutional frameworks, but relies heavily on donor support in rural contexts (Monney & Antwi-Agyei, 2018). The sanitation situation is far direr with over 30% of the rural population practicing open defecation and only 14% with access to basic sanitation, nationally. Hygiene, as measured through practices of handwashing with soap, remains poor with only 18% of the population having access to basic sanitation. These poor WASH conditions are known to cause the majority of diarrhea mortality, infections from intestinal worms, and other neglected tropical diseases. Morbidity from these conditions are direct contributors to poor nutrition outcomes such as wasting, stunting and anemia. Poor WASH is estimated to cost the economy in Ghana over US$280M annually. In 2016, the water and sanitation sectors were consolidated into the Ministry of Sanitation and Water Resources (MS&WR). This was a welcome development to those in the sector, however budgets have subsequently been cut for the sector (SEND Ghana). USAID/Ghana is one of many active donors in the sector. The USAID/West Africa Regional Mission also contributes to this work. USAID/Ghana has identified 6 regions in which it focuses on rural water and sanitation services while promoting hygiene. This complements the mostly urban investments by USAID/West Africa. The mission’s flagship WASH program, WASH for Health (W4H), leverages multiple international and local partnerships (such as Rotary International and local plastic and cement companies). Water, Sanitation and Hygiene (WASH) for Health is one of six activities designed to work together under the USAID/Ghana’s Health System Strengthening (HSS) Project to achieve equitable improvement in health in Ghana. W4H allocates 60% of its funds to sanitation activities due to the country’s poor sanitation standing on the SDG and 40% to support the provision of adequate water supply to poor and needy communities and institutions. Being part of the USAID/Ghana HSS Project, W4H supports achievement of Development Objective 3 (DO3) of the USAID/Ghana Country Development and Cooperation Strategy (CDCS). WASH for Health supports DO3 by increasing improved drinking water access in underserved areas, increasing use of improved sanitation, and promoting good hand washing behavior and point-of-use water treatment (Sub IRs 2.3, 2.4 and 2.5). WASH for Health coordinates with other USAID HSS activities, especially Communicate for Health for message development and People for Health for civil society engagement. WASH for Health was specifically designed to support the achievement of the USAID Water and Development Strategy 2013-2018 strategic objective 1 (SO1) “Water for Health” and its three associated intermediate results in the manner that the Strategy envisions for a “transformative impact”. WASH for Health also supports the Mission’s contributions to the Feed the Future objective on improving nutrition, especially among women and children. In all respects, WASH for Health supports the Government of Ghana’s programming in water, sanitation, and health as well as their journey to self￾reliance. The primary health indicator for the Journey to Self-Reliance Country Roadmaps is the Child Health Indicator, which is an index of basic water access, basic sanitation access, and child mortality. Additionally, the activity strengthens government systems to mobilize sanitation campaigns and declare communities and districts open defecation free (ODF). WASH for Health strengthens water governance structures to ensure effective management of government services, and attracts business development. PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 59 B. DESCRIPTION OF THE INTERVENTION TO BE EVALUATED AND THEORY OF CHANGE Instructions: Provide information to help the reader develop a strong understanding of the strategy/project/activity to be evaluated, particularly the primary interventions, strategic approaches, or tasks implemented by the strategy/project/activity. USAID/Ghana designed the WASH for Health activity to address the health effects caused by inadequate or non-existent WASH services for people across Ghana. Health effects are more profound in some regions than in others. Districts in the Northern, Central, Western, and Volta Regions were poorly served in terms of potable water coverage, averaging about 65% according to the rural water supply coverage statistics published by the Community Water and Sanitation Agency (CWSA). Regarding sanitation, virtually all regions were faring very poorly, though poor sanitation was more pronounced in the Northern, Upper East, and Upper West Regions with nearly 75% of all households lacking access to basic sanitation and practicing open defecation (WHO/UNICEF Joint Monitoring Program). It is, therefore, unsurprising that water and sanitation-related diseases were the top outpatient issues at healthcare centers in these areas. The scourge of cholera in recent years has been devastating, taking the lives of over 240 people, and infecting almost 30,000 Ghanaians between 2014 and 2015 in eight regions. Cholera can be prevented by the use of basic improved sanitation facilities, the provision and use of safe drinking water, and good hygiene practices. The goals of the WASH for Health activity are to accelerate sustainable improvement in water and sanitation access and to improve hygiene behaviors in target districts. These goals are achieved through six mutually reinforcing objectives (components), which are listed below: 1. Increased use of improved household sanitation; 2. Improved community water supply services; 3. Improved sector governance and policies; 4. Expanded key hygiene behaviors; 5. Leveraged public private partnership (PPP) investment to magnify the impact of USG funding; and 6. Improved water supply and sanitation infrastructure for schools and health facilities. Global Communities is the lead organization responsible for project management and administration, as well as for the implementation of water and sanitation infrastructure development and Community Led Total Sanitation (CLTS) implementation. The Manoff Group, a sub-recipient on the activity, is responsible for promoting Social Behavior Change Communication (SBCC), which is mainstreamed throughout the above-listed activity components. Other partners include Rotary International (RI) a USAID Global Development Alliance (GDA) partner. RI extends activities into a sixth region (Eastern Region) and works with the CWSA as a consultant/implementing partner. WASH for Health matches one to one funding in the already selected areas across six regions for Water and Sanitation projects with Rotary. WASH for Health engaged Local Non-Governmental Organization (LNGOs) as sub￾recipients to support community mobilization, sensitization, household latrine promotion, and hygiene education promotion for the Water and Development Alliance (WADA) activity at the early stages of the project but that fizzled out due to poor performance. WASH for Health field staff trained Water and Sanitation Management Teams (WSMT) to effectively manage WASH facilities constructed for their sustainability. The activity is implemented in collaboration with other USAID-funded projects for greater impact in the focus districts and region. The six objectives of the activity were translated into components, with particular interventions detailed out in the project implementation plan. Though WASH for Health implements activities in five regions (Volta, Central, Western, Northern, and Greater Accra), the degree and amount of activity varies depending on other previously existing interventions (i.e. existing water and sanitation coverage, prevailing health issues or interventions or other related projects present in the communities). WASH PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 60 for Health worked in the Eastern Region in this fiscal year to complement implementation of RI partnership activities. Target populations in these regions are entire communities, especially as it relates to sanitation results as many health and nutrition benefits are only realized with high coverage rates. This also embeds the principle of ‘leave no one behind,’ ensuring that vulnerable populations are reached. WASH for Health is active in the following Metro/Municipal/District Assemblies (MMDAs). In all the MMDAs, the activity promotes behavior change through information, education, and communication together with the infrastructure stated in the table below: Former Region (current) MMDA Interventions 1 Northern (now Savannah) Sawla-Tuna-Kalba • Community sanitation & hygiene • Point water sources (boreholes) • Piped water scheme • Institutional latrines 2 Northern (now Savannah) Bole • Community sanitation & hygiene • Point water sources (boreholes) • Institutional latrines 3 Northern (now Northeast) West Mamprusi • Community sanitation & hygiene • Point water sources (boreholes) 4 Volta (now Volta) Adaklu • Community sanitation & hygiene • Point water sources (boreholes) • Institutional latrines 5 Volta (now Oti) Nkawanta North • Community sanitation & hygiene • Point water sources (boreholes) • Institutional latrines 6 Volta (now Oti) Nkwanta South • Community sanitation & hygiene • Point water sources (boreholes) • Institutional latrines 7 Volta (now Oti) Krachi East • Community sanitation & hygiene • Point water sources (boreholes) • Institutional latrines 8 Western (now Western) Sekondi Takoradi • Community sanitation & hygiene • Institutional latrines 9 Western (now Western North) Sefwi Wiawso • Community sanitation & hygiene • Point water sources (boreholes) • Institutional latrines 10 Western (now Western North) Sefwi Bodi • Community sanitation & hygiene • Point water sources (boreholes) • Institutional latrines PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 61 Former Region (current) MMDA Interventions 11 Western (now Western North) Amenfi West • Community sanitation & hygiene • Point water sources (boreholes) • Institutional latrines 12 Western (now Western North) Amenfi Central • Community sanitation & hygiene • Point water sources (boreholes) • Institutional latrines 13 Greater Accra Shai Osudoku • Community sanitation & hygiene • Institutional latrines 14 Greater Accra Ada West • Community sanitation & hygiene • Institutional latrines 15 Greater Accra Tema • Institutional latrines 16 Greater Accra Ashiaman • Institutional latrines 17 Greater Accra Kpone Katamanso • Institutional latrines 18 Central Asikuma Odoben Brakwa • Community sanitation & hygiene • Institutional latrines 19 Central Ajumako Eyan Essiam • Point water sources (boreholes) • Piped water scheme • Institutional latrines 20 Central Cape Coast • Cholera prevention activities • Institutional latrines • Community sanitation & hygiene • Point water sources (boreholes) PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 62 Below find the theory of change for achieving improved health and well-being in target communities based on WASH for Health’s defined objectives. C. PROJECT OR ACTIVITY MONITORING, EVALUATION, AND LEARNING (MEL) PLAN WASH for Health has robust systems in place for monitoring and evaluation which will be made available to evaluators. This includes: • WASH for Health Scope of Work • Activity Monitoring, Evaluation and Learning Plan (AMELP) • GIS data information system developed by WASH for Health for infrastructure reporting Number of community that have attained ODF • Quarterly reports • Annual reports • Activity success stories, case studies, presentations and other communication materials • List of contact information for local authorities Additionally, other evaluations can be made available including: • Evaluation of CLTS+ (community-led total sanitation), household action planning and social incentives under the Bill and Melinda Gates Research in Sawla, Tuna Kalba district • WASHPaLS desk review of CLTS and market-based sanitation • Other evaluations done under USAID/Ghana Health System Strengthening project IV. EVALUATION QUESTIONS Evaluators will review and finalize questions in collaboration with USAID prior to finalizing the evaluation design. 1. To what extent are W4H sanitation achievements likely to be sustained? 2. What gaps exist in strengthening local governance systems to sustain water services in the country? PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 63 3. How successful has the joint policy developed by USAID and Government pro-ppor sanitation guidelines served the purpose for which it was developed. Are there other existing policy gaps within the WASH sector in Ghana? 4. To what extent have W4H’s approaches to private sector engagement for sanitation access expansion been successful? 5. To what extent has W4H achieved an increase in schools, health centers and households members practicing basic hand washing behaviors? V. EVALUATION DESIGN AND METHODOLOGY The performance evaluation methodology shall seek the highest degree of rigor so as to ensure credible findings and recommendations. The partner will decide on what method, techniques or approach is feasible the data gathering. Desk Review: A detailed desk review of all the relevant project documents and reports including relevant Government of Ghana national and local government documents, relevant donor-funded program,s and published WASH literature. Key Informant Interviews (KIIs): Interviews with USAID/Ghana team; relevant Government of Ghana staff at national, regional and district levels; W4H staff; other WASH development partners; the private sector; UNICEF, the World Bank; relevant Water Management Boards (Tuna); and beneficiary community members. A list of proposed interviewees will be submitted with the inception report. The final list of interviewees shall be agreed upon in consultation with USAID. Focus Group Discussions (FGDs): To obtain the required data from project beneficiaries, FGDs will be used, where applicable. The groups shall be constituted in consultation with USAID and the W4H team. Site Visits: Conducting KIIs and FGDs will require travel to activity sites. As part of site visits, some observations may be made to inform the evaluation. Site visits can be carried out in up to five selected regions within the activity area. It is important to balance the desire to sample from the five regional geographical activity locations, various activity components, and/or sectors with the logistics and cost implications. The evaluation team, in collaboration with USAID, will finalize the evaluation methods before fieldwork begins. USAID expects that, at a minimum, the evaluation team will: • Upon award, familiarize themselves with documentation about the activity and USAID’s current WASH assistance in the region. USAID will ensure that this documentation is available to the team prior to their arrival to Ghana; • Review and assess the existing performance and effectiveness information or data; • Conduct site visits for field testing survey instruments (when applicable and feasible); • Meet and interview W4H beneficiaries, partners, and host government counterparts at appropriate levels; • Interview USAID staff and a representative number of experts working in the sector; and PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 64 • Spend approximately 30 days in the region carrying out this SOW. The desk review includes at a minimum: • USAID WASH for Health SOW; • WASH for Health materials: Annual and Quarterly Reports, Annual Work Plans, MEL Plans, sector assessments, trip reports, performance reports, gender analyses, relevant sections of the Project Appraisal Document, GIS data information system, contact information for local authorities and miscellaneous thematic reports from other sources as listed above. • USAID Desk reviews on CLTS and Market-Based Sanitation • Government of Ghana standards, guidelines, policies, legislation or progress reports on WASH in households, schools and health centers • Other relevant evaluations from USAID/Ghana’s Health Systems Strengthening project The contractor will submit the preliminary evaluation design in response to the Request for Proposal for review by USAID. The evaluation Contracting Officer’s Representative (COR) will approve the finalized evaluation design two weeks or more prior to the team’s arrival in country. The evaluation design matrix should include a data analysis plan for each evaluation question. Requests of the offeror can include explicit description of major limitations in data collection and analysis. VI. DELIVERABLES AND REPORTING REQUIREMENTS The evaluation should be conducted between June and the summary report and recommendation submitted by end of July 2019. The final evaluation report is expected by the end of September 2019. The evaluator will purposefully select districts from each region where the data will be collected. 1. EVALUATION WORK PLAN Within 2 weeks of Kick-Off Conference Call, the evaluation team must submit an Inception Report to the COR. This will become an annex to the evaluation report. This will include details about the work plan and evaluation design. The work plan will include: 1. Draft schedule and logistical arrangements; 2. Members of the evaluation team, delineated by roles and responsibilities; 3. Evaluation milestones; 4. Anticipated schedule of evaluation team data collection efforts; 5. Proposed evaluation methodology including selection criteria for comparison groups, if applicable; and 6. Evaluation Report outline (if different from the attached template). The evaluation design will include: 1. Detailed evaluation design matrix that links the Evaluation Questions from the SOW (in their finalized form) to data sources, methods, and the data analysis plan; 2. Draft questionnaires and other data collection instruments or their main features; 3. List of potential interviewees and sites to be visited and proposed selection criteria and/or sampling plan (must include sampling methodology and methods, including a justification of sample size and any applicable calculations); 4. Limitations to the evaluation design; and 5. Dissemination plan (designed in collaboration with USAID). PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 65 Unless exempted from doing so by the COR, the evaluation design will be shared with partner country stakeholders as well as with the implementing partners for comment before being finalized. The data analysis plan should clearly describe the evaluation team’s approach for analyzing quantitative and qualitative data (as applicable), including proposed sample sizes, specific data analysis tools, and any software proposed to be used, with an explanation of how/why these selections will be useful in answering the evaluation questions for this task. Qualitative data should be coded as part of the analysis approach, and the coding used should be included in the appendix of the final report. Gender, geographic, and role (beneficiary, implementer, government official, NGO, etc.) disaggregation must be included in the data analysis where applicable. All dissemination plans should be developed with USAID and include information on audiences, activities, and deliverables, including any data visualizations, multimedia products, or events to help communicate evaluation findings. See the Evaluation Toolkit for guidance on Developing an Evaluation Dissemination Plan. If applicable based on the Disclosure of Conflict of Interests Forms submitted with the awardee’s proposal, the evaluation design will include a conflict of interest mitigation plan. USAID offices and relevant stakeholders are asked to take up to five working days to review and consolidate comments through the COR. Once the evaluation team receives the consolidated comments on the initial evaluation design and work plan, they are expected to return with a revised evaluation design and work plan within five working days. 2. WEEKLY REPORTS The evaluation team will update the evaluation work plan (the lists of interviewees, survey participants, the schedule) and submit the updated version to the COR on a weekly basis 3. IN-BRIEF MEETING The evaluation team is expected to hold an inception meeting with the Mission. The purpose of the kickoff meeting is to formally notify all stakeholders that the evaluation has begun and make sure everyone has a common understanding of the project on hand and their roles and expected outcome and also make sure that everybody is on the same page. This meeting will be scheduled as agreed upon by the contractor and USAID/Ghana Health Office. 4. EXIT BRIEFING The evaluation team is expected to hold a final exit briefing to discuss the status of data collection and preliminary findings. This presentation will be scheduled as agreed upon by the contractor and USAID/Ghana Health Office. 5. DRAFT EVALUATION REPORT The draft evaluation report should be consistent with the guidance provided in Section IX, Final Report Format. The report will address each of the questions identified in the SOW and any other issues the team considers to have a bearing on the objectives of the evaluation. Any such issues can be included in the report only after consultation with USAID. The submission date for the draft evaluation report will be determined in the evaluation work plan. Once the initial draft evaluation report is submitted, USAID/Ghana Health Office and USAID/GH Environmental Health team will have 10 working days in which to review and comment on the initial draft, after which point the COR will submit the consolidated comments to the evaluation team. The evaluation team will then be asked to submit a PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 66 revised final draft report within 10 working days, and again USAID will review and send comments on this final draft report within 10 working days of its submission. 6. FINAL EVALUATION REPORT The evaluation team will be asked to take no more than 10 working days (or as agreed upon in the work plan) to respond to and incorporate final draft evaluation report comments from USAID. The evaluation team lead will then submit the final report to the COR. 7. FINAL EVALUATION TWO PAGE BRIEFER The evaluation team will produce a front and back summary of findings and conclusions in a graphical and easy to read format. 8. FINAL EVALUATION REPORT SUMMARY A brief summary of the Final Evaluation Report, which is not to exceed 15 pages, excluding any potentially procurement sensitive information shall be submitted electronically in English for dissemination among implementing partners and stakeholders. The report must meet standards outlined in the evaluation policy. 9. SUBMISSION OF DATASET(S) TO THE DEVELOPMENT DATA LIBRARY Per USAID’s Open Data policy (see ADS 579, USAID Development Data) the contractor must also submit to the COR and the Development Data Library (DDL), at www.usaid.gov/data, in a machine￾readable, non-proprietary format, a copy of any dataset created or obtained in performance of this award, if applicable. The dataset should be organized and documented for use by those not fully familiar with the intervention or evaluation. Please review ADS 579.3.2.2 Types of Data To Be Submitted to the DDL to determine applicability. 10. SUBMISSION OF FINAL EVALUATION REPORT TO THE DEVELOPMENT EXPERIENCE CLEARINGHOUSE Per USAID policy (ADS 201.3.5.18) the contractor must submit the evaluation final report and its summary or summaries to the Development Experience Clearinghouse (DEC) within three months of final approval by USAID. 11. FINAL PRESENTATION The evaluation team is expected to hold a final presentation by virtual conferencing to discuss the summary of findings and conclusions with USAID. This presentation will be scheduled as agreed upon in advance. VII. EVALUATION TEAM COMPOSITION The offeror shall propose a team of individuals including a team leader, and their structure in order to successfully ensure answer the evaluation questions and complete deliverables in the specified timeline. The contractor must provide information about evaluation team members, including their curricula vitae, and explain how they meet the requirements in the evaluation SOW. Submissions of writing samples or links to past evaluation reports and related deliverables composed by proposed team members are highly desirable. Per ADS 201.3.5.14, all team members must provide to USAID a signed statement attesting to a lack of conflict of interest or describing an existing conflict of interest relative to the activity being evaluated (i.e., a conflict of interest form). PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 67 Required qualifications and skills across all team members: 1. WASH programming experience in Sub-Saharan Africa, preferably Ghana 2. Knowledge of Community-Led Total Sanitation and WASH infrastructure design and functionality 3. Awareness of social and behavior change approaches 4. Experience with USAID and/or other donors 5. Experience in evaluation design, methods, management, and implementation; 6. Background in USAID’s cross-cutting program priorities, such as gender equality and women’s empowerment, youth, etc.; 7. Appropriate language skills. Please include CVs of the proposed team lead and primary technical staff in Inception Report. Proposed key personnel are expected to be the people who execute the work of this contract. Any substitutes to the proposed key personnel must be vetted and approved by the COR before they begin work. USAID may request an interview with any of the proposed evaluation team members via conference call, Skype, or other means. The primary point of contact for this evaluation will be Emmanuel Odotei, the WASH for Health AOR, and he will serve as primary resource person to the evaluation team. He, along with other USAID staff, will provide technical guidance pertaining to questions that the evaluation team might have during implementation. The evaluation COR or other USAID representatives may observe some of the data collection efforts. USAID may also delegate one or more staff members to work full-time with the evaluation team or to participate in selected evaluation activities. The evaluation COR will inform the contractor in writing about any full-time or part-time USAID delegates no later than 10 working days after the submission of an Inception Report. USAID will pre-define any staff’s level of involvement by indicating the purpose of their inclusion, their role on the team and in which components of the evaluation they will participate, their expertise in the topic or sector, their expertise in evaluation design or implementation, and their anticipated LOE. USAID maintains primary responsibility for management of its own staff. USAID will outline collaboration, delivery, and performance expectations for its staff as well as reporting lines and how staff management roles and responsibilities will be coordinated between USAID, the contractor, and the evaluation team lead. This plan will be finalized in consultation with the contractor and the evaluation team lead, with final approval by the COR, to ensure it is feasible and appropriate to the evaluation objectives and USAID/Ghana’s needs and that it addresses mitigation of risk of impeding evaluation implementation or biasing findings. All costs associated with the participation of full-time or part-time USAID delegates in the evaluation will be the responsibility of USAID. VIII. EVALUATION SCHEDULE Offeror should be aware that once a proposal is received, it will be shared for peer review with USAID for a period of five days, after which some revisions may be required before a task order is issued. The below evaluation schedule is illustrative and will be updated in collaboration with USAID prior to finalization of the work plan. Performance Evaluation Schedule (all days are calendar days). PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 68 Date or Duration Proposed Activities [Important Considerations] Within 7 days of NTP Kick-off Conference Call between USAID, contractor and Team Leader Conference Call format 14 days Preparation of the Inception report (work plan and evaluation design) Inception report (deliverable #1) due two weeks after kickoff meeting 5 days USAID review of the work plan and evaluation design Day two that team is in-country In-briefing with USAID/Ghana Office of Health Schedule this with mission 30 working days Data collection Mondays OOB Weekly update of progress To WASH for Health AOR and mechanism COR (deliverable #2) Within 5 days of completing data collection Exit Briefing w/ USAID/Ghana Office of Health In-person including preliminary findings (deliverable #3) 28 days Data analysis & report writing Draft Evaluation report due 28 days after data collection complete (deliverable #4) 5 days USAID review of draft report 21 days Incorporate USAID comments and prepare final report Final Evaluation Report (deliverable #5) due 21 days after comments received 28 days Prepare 15 Pages and One pager Summary report by the end of September Final briefer and summary (deliverables #6 and 7) due 7 days after receiving final approval from USAID 90 days Submit dataset(s) to Development Data Library and final report to Development Experience Clearinghouse Deliverables #8 and 9 Within 10 days of final report submitted Dissemination meeting with USAID/Ghana, IP and other stakeholders To be determined and agreed upon in consultation with USAID (deliverable #10) IX. FINAL REPORT FORMAT 1. Abstract 2. Executive Summary 3. Evaluation Purpose 4. Background on the Context and the Strategies/Projects/Activities being Evaluated 5. Evaluation Questions PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 69 6. Methodology 7. Limitations to the Evaluation 8. Findings, Conclusions, and (If Applicable) Recommendations 9. Annexes See the Evaluation Toolkit for the How-To Note on Preparing Evaluation Reports and ADS 201mah, USAID Evaluation Report Requirements. An optional Evaluation Report Template is also available in the Evaluation Toolkit. The evaluation abstract of no more than 250 words should describe what was evaluated, evaluation questions, methods, and key findings or conclusions. The executive summary should be 2–5 pages and summarize the purpose, background of the project being evaluated, main evaluation questions, methods, findings, and conclusions (plus recommendations and lessons learned, if applicable). The evaluation methodology shall be explained in the report in detail. Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methods (e.g., in sampling; data availability; measurement; analysis; any potential bias such as sampling/selection, measurement, interviewer, response, etc.) and their implications for conclusions drawn from the evaluation findings. Annexes to the report must include: • Evaluation SOW (updated, not the original, if there were any modifications); • Evaluation methods; • All data collection and analysis tools used in conducting the evaluation, such as questionnaires, checklists, and discussion guides; • All sources of information or data, identified and listed; • Statements of difference regarding significant unresolved differences of opinion by funders, implementers, and/or members of the evaluation team, if applicable; • Signed disclosure of conflict of interest forms for all evaluation team members, either attesting to a lack of or describing existing conflicts of interest; and • Summary information about evaluation team members, including qualifications, experience, and role on the team. X. CRITERIA TO ENSURE THE QUALITY OF THE EVALUATION REPORT Per ADS 201maa, Criteria to Ensure the Quality of the Evaluation Report, draft and final evaluation reports will be evaluated against the following criteria to ensure quality. • Evaluation reports should represent a thoughtful, well-researched, and well-organized effort to objectively evaluate the strategy, project, or activity; • Evaluation reports should be readily understood and should identify key points clearly, distinctly, and succinctly; • The Executive Summary should present a concise and accurate statement of the most critical elements of the report; • Evaluation reports should adequately address all evaluation questions included in the SOW, or the evaluation questions subsequently revised and documented in consultation and agreement with USAID; • Evaluation methodology should be explained in detail and sources of information or data properly identified; PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 70 • Limitations to the evaluation should be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.); • Evaluation findings should be presented as analyzed facts, evidence, and data and not based on anecdotes, hearsay, or simply the compilation of people’s opinions; • Conclusions should be specific, concise, and include an assessment of quality and strength of evidence to support them supported by strong quantitative and/or qualitative evidence; • If evaluation findings assess person-level outcomes or impact, they should also be separately assessed for both males and females; and • If recommendations are included, they should be supported by a specific set of findings and should be action-oriented, practical, and specific. • See ADS 201mah, USAID Evaluation Report Requirements and the Evaluation Report Checklist and Review Template from the Evaluation Toolkit for additional guidance. PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 71 ANNEX II: EVALUATION METHODS DATA TRIANGULATION METHODS With regard to triangulating data from key informant interviews, the team often requested key informants to provide evidence in the form of internal project reports (see Annex IV) with data to corroborate statements of fact or suggested where evidence may be available. In cases when the team felt there was an evidence gap or the evidence was contradictory, the team referred to W4H to provide evidence that confirms or counters the claims made. When analyzing the qualitative data, the evaluation team included both complementary and conflicting accounts of W4H implementation. Both accounts have been analyzed and weighed against each other to reach a nuanced determination in this evaluation report. Further, any qualitative scripts that were either a) internally inconsistent during the interviews or b) included probes that resulted in inconsistent remarks were reviewed by the team and follow-up with interviewees was conducted where necessary. Triangulation was also used across county-based interviewees to note consistencies across key informants. INTERVIEW TEAMS PER COUNTY District Evaluation Team Members Asikuma Odoben Brakwa (Central) WASH Governance Specialist, Deputy Team Leader/ WASH Specialist Shai Osudoku (Greater Accra) Team Leader, Deputy Team Leader/ WASH Specialist, CLTS and Behavior Change Specialist, Research Specialist Sawla Tuna Kalba (Savannah) CLTS and Behavior Change Specialist, Research Specialist West Mamprusi (North East) CLTS and Behavior Change Specialist, Research Specialist Ada West (Greater Accra) CLTS and Behavior Change Specialist, Technical Operations and Finance Manager Krachi East (Oti) CLTS and Behavior Change Specialist, Research Specialist, Technical Operations and Finance Manager Nkwanta North (Oti) CLTS and Behavior Change Specialist, Research Specialist, Technical Operations and Finance Manager Sefwi Bodi (Western North) WASH Governance Specialist, Deputy Team Leader/ WASH Specialist Sefwi Wiawso (Western North) Team Leader, WASH Governance Specialist Amenfi West (Western) Team Leader, WASH Governance Specialist PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 72 DESIGN CONSIDERATIONS FOR KII AND FGD GUIDES While the KII and FGD guides vary based on respondent categories, they all have some variation of the following sections: • Basic demographic and context data, including interviewee’s sex, role, and location • Respondent’s relationship to W4H • Project background based on respondent experience • Experience with USAID, W4H, and/or partner coordination/outreach, as applicable • Perception of successes and failures, challenges and opportunities • Relationship with other activities, government, or stakeholders and other factors that could affect outcomes • Promising practices that should be retained and why, as applicable • Sustainability - Mechanisms for sustainability - Challenges to sustainability For relevant respondents such as staff from W4H and USAID/Ghana, the evaluation team incorporated other questions to cover the following key themes: • The W4H approach to deciding what needs to be done and how • The W4H management and staffing structure over time, what has and has not worked • The W4H thinking on sustainability and the necessary mechanisms to achieve it • The W4H ability to modify and adapt – including changes to the W4H contract and SOW (e.g. removing nutrition funding after FY17) • The W4H identification, interaction, and accountability with partners and key stakeholders • W4H Brand perception and the GOGs appreciation for W4H’s contributions to the sector. DATA CODING AND ANALYSIS IN DEDOOSE The interview team uploaded transcripts to Dedoose as they were completed. Coding began the week of September 2 and ended on September 27, 2019. To minimize bias and increase consistency in the application of the codes, the Team Leader and the Research Specialist were the exclusive coders of all data sets collected during the exercise, though all team members reviewed and analyzed coded transcript text related to their area of expertise. In total, 40 codes were applied 1067 times across the data sets. As would be expected, some codes entered in Dedoose at the outset of the exercise were applied infrequently if at all. Reasons for this are twofold, in some cases these codes proved to be irrelevant and were extraneous to the evaluation. In others, the lack of code application signaled areas where data was missing (either because it was captured elsewhere in the coding or did not come up during the interviews, contrary to what the team anticipated). Beginning September 27th, after most of the data had been coded, the team began to isolate and download coded transcript text related to each EQ to begin the analysis. Each member of the evaluation team worked with the coded text related to their area of expertise to identify patterns, comparisons and substantive quotations; eventually generating an initial outline of a response for each EQ. It was at this time that secondary data from the desk review and project monitoring data was again consulted to provide further contextual background, fill in gaps in understanding, and to help to triangulate findings from the qualitative analysis. The team also attempted to identify patterns in the data to highlight convergence or divergence of opinion across counties, activity types and stakeholder groups. PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 73 ANNEX III: DATA COLLECTION INSTRUMENTS KEY INFORMANT INTERVIEW (KII) GUIDES POLICY AND POLICY GAPS, GOVERNANCE AND SUSTAINABILITY OF WASH SERVICES AT NATIONAL, REGIONAL, DISTRICT AND COMMUNITY LEVELS I. Introduction (10 minutes) a. Thank the respondent for taking the time to participate in the interview b. Introduction to the researcher and the research i. Introduce yourself: I represent an evaluation team fielded by Tetra Tech ARD, a Washington DC based firm that has been contracted by the U.S. Agency for International Development (USAID) to conduct an independent evaluation of the USAID WASH for Health (W4H) program. The Ghana Water, Sanitation and Hygiene for Health (W4H) Activity is a five-year Cooperative Agreement funded by USAID/Ghana and implemented by Global Communities (GC). The goal of W4H is to accelerate sustainable improvement in water and sanitation access and improve hygiene behaviors in 20 target districts. i. As part of its own planning for the next few years, USAID has asked us to conduct a mid￾term performance evaluation of the W4H program to assess its progress toward achieving its objectives. We are trying to assess the strengths, accomplishments and best practices, as well as any obstacles and shortcomings faced and how it could be more effective. ii. Ultimately the research will be used to ascertain the extent to which W4H activities have improved lives and health through the development and management of sustainable water, sanitation and hygiene services in Ghana. It will also be used to propose recommendations based on the findings to inform future WASH programming by USAID. c. We will follow privacy protocols to protect your anonymity: i. Explain confidentiality and anonymity and note whether the respondent would like to remain anonymous. The assessment team will ask permission to attribute a quote from the respondent in the final report. ii. Explain how collected data will be stored and responses used. iii. Ask if the respondent is willing to be recorded and note their response. d. Explain recording, length and nature of discussion e. Check whether respondents have any questions. This Guide is intended for use in terms of data collection for WASH stakeholders as part of the Performance Evaluation of Global Communities’ WASH for Health (W4H) activity. The primary purpose of the evaluation is to assess if the Activity’s design and implementation are successfully meeting the goals of expanding sustainable access to water and sanitation services. The evaluation will inform any midcourse corrections for the final period of implementation. The evaluation findings also may inform the development of future USAID WASH investments in Ghana and elsewhere. The framing should be followed as closely as possible to guide key informant interviews with these respondents. There are a few questions where a specific set of responses are provided in order to assist in the collection of data that can be more easily quantified. Instructions to the interviewer are in red. Interviewers will be reminded of USAID policy with regard to the "USG Common Rule" for the protection of human subjects. PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 74 II. Conclusion: • Thank the respondent for their time. • Tell the respondent they are welcome to contact you to ask questions at a later date or to add to their responses. Revisit permission of the respondent to use their name in the report if you might use a quote. Note their response: ____________. PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 75 KII GUIDE – GC STAFF (NATIONAL STAFF, DISTRICT STAFF, AND COMMUNITY LEVEL STAFF) [Interviewers should establish the interviewee’s area of expertise and include the topics below accordingly. Areas of fact that have been established in prior interviews should be stated as such to save time. It is not expected that all interviews will include all questions below.] • Name of Key Informant: • Gender: • Position at GC: • Role on W4H Project: / Since when? Policy Gaps 1. Would you say the governance/institutional arrangements for WASH delivery in Ghana are effective? If yes, what strengths do you see? If no, why not? 2. Do you see any gaps in existing WASH sector policies? a. If so, how have these gaps constrained access to services? b. Or how have any gaps affected program design and implementation by Sector Players? 3. How have sector policies influenced WASH implementation at the District level? 4. How have sector policies and government strategies influenced sustainability of service delivery? Pro-Poor Guidelines 5. What role did W4H/GC play in the preparation of the pro-poor sanitation guidelines? 6. To what extent has the guideline been disseminated at national, regional and district levels? Has there been any training to ensure understanding? 7. What role did GC/W4H play in the dissemination process at these levels? 8. To what extent have the guidelines influenced the design of programs in favor of the poor and vulnerable? Can you mention specific projects that have adopted the guidelines? 9. Do the guidelines provide sufficient clarity in terms of roles and responsibilities of different stakeholders particularly with regard to ensuring that sanitation services can be accessed by poor households? 10. How do the sanitation guidelines link to and harmonize with other related government policy? What are the implications, if any, for inter-ministerial coordination? 11. To what degree are the guidelines an actual reference point for local level decision-makers and service providers? Has there been any change in approach, coverage, and/or sustainability of services as a result of the policy being launched? 12. Are you aware of any other pro-poor strategies being implemented in WASH, and how effective are they in reaching their targets? What about in other sectors (health, education, etc.)? 13. What do you see as challenges for implementing/adopting the pro-poor guidelines? 14. What are your recommendation(s) for improving the implementation of the guidelines? PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 76 15. What are your recommendation(s) for improving policies and strategies to support WASH program design and implementation more generally, particularly with regard to sustainability? Sanitation Service Delivery and Engagement of The Private Sector 16. What have been the approaches that GC has used for the provision of sanitation services? a. What has been the balance between providing access and sustaining continued use of sanitation and hygiene facilities? b. What considerations were made regarding sanitation options? (appropriate and affordable sanitation options for communities – cost, ease of installation, durability, availability of parts etc.) c. After several years of experience, what kinds of changes have been made to these approaches in the different districts or areas where W4H is being implemented? 17. Describe how the W4H project has engaged the private sector. How have the options and markets for sanitation been developed? a. Describe the existing supply chain or the supply chain that GC has developed as part of the project. How does this supply chain work? b. How has GC worked on fostering demand? (Link to CLTS/Behavior change) c. How has GC worked to create an enabling environment for supply – what capacity deficiencies were observed and what interventions were made to improve capacities? d. Describe key success gained from this approach e. What kind of progress has been observed? Which districts/communities are performing well and why? f. How will these successes be sustained after the project finishes? What sustainability mechanisms has GC/W4H put in place? g. Describe key challenges – technical challenges to providing sanitation services h. Which communities are not responding well to W4H’s interventions and why? i. What kinds of follow up have there been with the communities to assess sustainability? (Results of follow up) 18. Regarding the enabling environment and engagement of the private sector a. Which categories of private sector has GC engaged as part of the project? (Check for the following: National Level Manufacturer, District Level Distributor, Community level SMEs and artisans?) how many companies have been engaged – check what kinds of support is given to local level artisans and SMEs to expand their services) b. How and why was this engagement done? c. What has been the focus of private sector engagement – more on product development or provision of services? d. What business model has been used? e. What other support (e.g. Capacity building, financing, marketing) has been provided? PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 77 f. Were these companies already involved in sanitation or were they engaged as part of the project? If they were engaged as part of the project, what indications do we have that they will continue working in sanitation services? Behavior Change 19. What kinds of behavior changes have been noted around demand for sanitation services/facilities? 20. Sustaining demand: How will these behaviors be sustained beyond the project? 21. What gender considerations and other technical considerations have influenced the activities? 22. Have you assessed the level of acceptability of sanitation options or products provided as part of the project? What are the major findings? How affordable are these compared to other options? How are households able to afford this investment? How long does it take the average household to pay for product? 23. What other investments do households make with regard to sanitation? 24. What approaches are used for ODF monitoring? What are the outcomes? Are communities able to maintain ODF? Are communities moving up the sanitation ladder? 25. What observations have been made regarding changes or attitudes of households with regard to investment behavior regarding sanitation? Sustainability and Government Linkages 26. From your experience, what are the challenges to sustainability? 27. Can you provide evidence that points to the sustainability of markets? (increasing demand and matching supply) – Examples of how demand has changed. How has supply been enabled to respond to demand? 28. How affordable are these facilities for the poorest or vulnerable households? 29. How has GC engaged the government as part of the project? 30. How has GC’s approach been linked to Government policy for sanitation services? (alignment and institutionalization) 31. How will local governments continue to foster demand creation beyond the project? 32. Influence of Government policy regarding options for sanitation and services - Is there clarity in the government position regarding the different approaches for sanitation delivery using a market-based approach – vis-à-vis pure CLTS approach, subsidies/support, micro-credit support (RSMS), development of markets/supply chain etc. 33. What recommendations would you make to ensure sustained access to improved sanitation services? PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 78 KII GUIDE – NATIONAL & REGIONAL LEVEL STAKEHOLDERS Pro-Poor Guideline and Policy Gaps, Sanitation, Behavior Change [Interviewers should establish the interviewee’s area of expertise and include the topics below accordingly. Areas of fact that have been established in prior interviews should be stated as such to save time. It is not expected that all interviews will include all questions below.] Targets: 1. State Actors: MSWR/EHSD/Water Directorate, CWSA, WRC, RCC/EHSD/RICCS 2. Non-State Actors: Development Partners, INGOs/NGOs/CSOs (local and international), Staff of GC 1. Tell me about yourself (Name/Department/Position) 2. What do you know about the W4H interventions being implemented by Global Communities in Ghana? How were the districts selected to benefit from the W4H Project? How many communities have benefited and how were the communities selected? 3. What successes have W4H achieved? What challenges have they faced? 4. Are there key differences between the W4H approach compared to other DPs? 5. How has the W4H project supported strategy development for WASH in the districts? 6. How has the W4H project ensured sustainable service delivery, particularly for sanitation? 7. What kinds of outcomes or changes has the W4H program brought to sanitation service delivery? How are the approaches being taken up by the districts (alignment and institutionalization)? 8. How will these achievements be sustained beyond the project? 9. How has W4H engaged with other NGOs or DPs working in WASH? Policy Gaps 10. Would you say the governance/institutional arrangements for WASH delivery in Ghana are effective? If yes, what strengths do you see? If no, why not? 11. Do you see any gaps in existing WASH Sector Policies? a. If so, how have these gaps constrained access to services? b. Or how have any gaps affected programme design and implementation by sector players? 12. How have sector policies influenced WASH implementation at the District level? 13. How have Sector Policies and Strategies influenced sustainability of service delivery? 14. What gender considerations and other technical considerations have influenced the activities? Pro-Poor Guidelines 15. What do you know about the Guidelines for Targeting the Poor and Vulnerable for Basic Sanitation Services in Ghana? Were you part of the development process? 16. What role did W4H/GC play in the preparation of the guidelines? Other sector stakeholders? 17. To what extent has the guideline been disseminated at national, regional and district levels? 18. What role did GC/W4H play in the dissemination process at these levels? PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 79 19. To what extent have the guidelines influenced the design of programs in favor of the poor and vulnerable? Can you mention specific projects that have adopted the guidelines? 20. Do the guidelines provide sufficient clarity in terms of roles and responsibilities of different stakeholders particularly with regard to ensuring that sanitation services can be accessed by poor households? 21. How do the sanitation guidelines link to and harmonize with other related government policy? What are the implications, if any, for inter-ministerial coordination? 22. To what degree are the guidelines an actual reference point for local level decision-makers and service providers? Has there been any change in approach, coverage, and/or sustainability of services as a result of the policy being launched? 23. Are you aware of any other pro-poor strategies being implemented in WASH, and how effective are they in reaching their targets? What about in other sectors (health, education, etc.)? 24. What do you see as challenges for implementing/adopting the pro-poor guidelines? 25. What are your recommendation(s) for improving the implementation of the guidelines? 26. What are your recommendation(s) for improving policies and strategies to support WASH program design and implementation more generally, particularly with regard to sustainability? Sanitation Investments and Behavior Change 27. What are the main government targets regarding sanitation across the country? 28. What approaches is the government using to achieve these targets? What has been the progress towards these targets? What have been any major challenges in achieving and sustaining these targets for improved access to sanitation? 29. In your view how might these challenges be addressed? 30. Are there any other policy area gaps particularly regarding sanitation products and markets? 31. What support is provided for private sector service provision? (Any guidelines, regulations, credit and financing, market development, etc.) 32. Is there a clear government position on subsidies and support for hardware for households in the construction of latrines? 33. In your view, what is the effect of this policy on the development of sanitation markets (both demand and supply)? 34. What has been the contribution of the Sanitation Fund to development of sanitation markets or products? Behavior Change 35. What kinds of behavior or attitude changes have been noted around demand for and investment in household sanitation services/facilities? Do you have evidence to back this up? 36. What shifts have you seen in terms of behavior change around hand washing? Do you have evidence to back this up? 37. Can you think of any parallels between other development interventions and products that could inform how we do sanitation delivery? PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 80 38. How will these behaviors be sustained going forward? 39. What approaches are used for ODF monitoring? What are the outcomes? Are communities able to maintain ODF? Are communities moving up the sanitation ladder? Strengthening Governance 40. How do the Districts manage their water resources? (Does the District have a Water Resource Management Plan? Is the plan linked to MTDPs and budgeted for? What is the status of implementation of the plan?) 41. To what extent have District Assemblies prioritized water and sanitation? (Probe for the following:) a. Evidence in reviewed DESSAPS/NESAPs, MTDPs with budget provisions reflecting on sustainable services delivery (O&M, support for monitoring, CAPEX, expansion) b. What is the status of District Assemblies’ by-laws on sanitation and enforcement? Did W4H help develop or review these? c. What role did W4H play in enabling the MMDAs to reflect WASH issues in their MTDPs, DESSAPs, NESAPs, …? d. How have these plans, and budgets affected sustainability of services? OR e. Can the MMDAs provide the support services to enable communities to sustain WASH Services? f. How are the communities sustaining their services? g. Are there trained area mechanics in the district and do you have a register of them? h. How are the communities/pump mechanics assessing spare parts? 42. What are the mechanisms for monitoring service provision in the district a. How does the District monitor service provision in the districts? b. What M&E framework does the District use for WASH Services? Does the district have data on WASH Services in the district? How does the District collect data for M&E purposes? c. How does the district M&E link to regional and national level systems? d. Has the W4H project contributed to strengthening the M&E system in the district? e. What do you know about Participatory Action for Community Enhancement (PACE) f. Have you received any training on this and used this approach to monitor WASH services? How useful was it? g. What is the district plan for monitoring WASH Services towards sustainability when W4H has come to an end? h. What is the Exit Strategy for W4H/GC and how does the District intend to take over the WASH investments? i. What role do Natural Leaders play at the community level with respect to monitoring? 43. What recommendations do you have for improving management of the water systems towards ensuring sustainable services delivery? PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 81 KII GUIDE – DISTRICT/MUNICIPAL LEVEL STAKEHOLDERS Targets: 1. State Actors: DPCU/DICSS/EHSU 2. Non-State Actors: Partner NGOs/CSOs (local), Staff of GC 1. Tell me about yourself (Name/Department/Position) 2. What are the sanitation targets for the DA (Review DESSAP for DA) and what progress has been made? What are the challenges affecting sanitation delivery in your district? 3. How has the private sector been involved in the delivery of sanitation services? Is there any DA Support for private sector in service provision? (Guidelines, regulation, finance, market development, capacity building, …) 4. What do you know about the W4H interventions being implemented by Global Communities in Ghana? How was the district selected to benefit from the W4H Project? How many communities have benefited and how were the communities selected? 5. What successes have W4H achieved? What challenges have they faced? 6. Are there key differences between the W4H approach compared to other DPs? 7. How has the W4H project supported strategy development for WASH in the District? 8. How has the W4H project ensured sustainable service delivery, particularly for sanitation? 9. What has been your level of involvement in the W4H program? How is the program collaborating with the DA? How is the project supporting the district: a. to target services to the poor and vulnerable? b. with demand creation? c. in the development of markets and options for sanitation? d. in the financing of sanitation? 10. What kinds of outcomes or changes has the W4H program brought to sanitation service delivery in the district? How are the approaches being taken up by the district (alignment and institutionalization)? 11. How will these achievements be sustained beyond the project? 12. How has W4H engaged with other NGOs or DPs working in WASH within the district? 13. What are your recommendations for improvement or things to continue to develop or build upon for the project? Policy Gaps 14. Would you say the governance/institutional arrangements for WASH delivery in Ghana are effective? If yes, what strengths do you see? If no, why not? 15. Do you see any gaps in existing WASH sector policies? e. If so, how have these gaps constrained access to services? f. Or how have any gaps affected program design and implementation by sector players? PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 82 16. Influence of Government policy regarding options for sanitation and services - Is there clarity in the government position regarding the different approaches for sanitation delivery using a market-based approach – vis-à-vis pure CLTS approach, subsidies/support, micro-credit support (RSMS), development of markets/supply chain etc. 17. How have sector policies influenced WASH implementation at the District level? 18. How have Sector Policies and Strategies influenced sustainability of service delivery? Pro-Poor Guidelines 19. What do you know about the Guidelines for Targeting the Poor and Vulnerable for Basic Sanitation Services in Ghana? Were you part of the development process? 20. To what extent has the guideline been disseminated at the district level? 21. What role did GC/W4H play in the dissemination process at these levels? 22. Do the guidelines provide sufficient clarity in terms of roles and responsibilities of different stakeholders particularly with regard to ensuring that sanitation services can be accessed by poor households? 23. How do the sanitation guidelines link to and harmonize with other related government policy? What are the implications, if any, for inter-ministerial / inter-departmental coordination? 24. To what degree are the guidelines an actual reference point for you at the local level? Has there been any change in approach, coverage, and/or sustainability of services as a result of the policy being launched? Can you mention specific projects that have adopted the guidelines? 25. What do you see as challenges for implementing/adopting the pro-poor guidelines? 26. Does the District plan to continue supporting the poor using the guidelines after the W4H activity has ended? What does the plan entail? Or does the district plan to integrate the program into the DESSAP/MTDP and budgeted appropriately? 27. What are your recommendation(s) for improving the implementation of the guidelines? 28. What are your recommendation(s) for improving policies and strategies to support WASH program design and implementation more generally, particularly with regard to sustainability? 29. Are you aware of any other pro-poor strategies being implemented in WASH in your district, and how effective are they in reaching their targets? What about in other sectors (health, education, etc.)? Strengthening Governance 30. How does the District manage its water resources? g. Does the District have a Water Resource Management Plan? h. Is the plan linked to MTDPs and budgeted for? i. What is the status of implementation of the plan? 31. To what extent have District Assemblies prioritized water and sanitation? (Probe for the following:) i. Evidence in reviewed DESSAPS/NESAPs, MTDPs with budget provisions reflecting on sustainable services delivery (O&M, support for monitoring, CAPEX, expansion) PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 83 j. What is the status of District Assemblies’ by-laws on sanitation and enforcement? Did W4H help develop or review these? k. What role did W4H play in enabling the MMDAs to reflect WASH issues in their MTDPs, DESSAPs, NESAPs, …? l. How have these plans, and budgets affected sustainability of services? OR m. Can the MMDAs provide the support services to enable communities to sustain WASH Services? n. How are the communities sustaining their services? o. Are there trained area mechanics in the district and do you have a register of them? p. How are the communities/pump mechanics assessing spare parts? 32. What are the mechanisms for monitoring service provision in the district q. How does the District monitor service provision in the districts? r. What M&E framework does the District use for WASH Services? Does the district have data on WASH Services in the district? How does the District collect data for M&E purposes? s. How does the district M&E link to regional and national level systems? Are they using the same indicators? Same software? t. Has the W4H project contributed to strengthening the M&E system in the district? u. What do you know about Participatory Action for Community Enhancement (PACE) v. Have you received any training on this and used this approach to monitor WASH services? How useful was it? w. What is the district plan for monitoring WASH Services towards sustainability when W4H has come to an end? x. What is the Exit Strategy for W4H/GC and how does the District intend to take over the WASH investments? y. What role do Natural Leaders play at the community level with respect to monitoring? 33. (As relevant) Managing and sustaining Small Town Water System a. What is the structure for managing Small Town System and how does it differ from what CWSA recommends? Who are the members on the WSMTs and how were they selected? Any gender considerations? (Female representation is 60% as per GWSA guidelines) b. What are the responsibilities of WSMTs? c. What type of training (contents and practice) was provided to the WSMTs? d. Who supervises/monitor the operations including financial management of the WSMTs and how does the monitoring take place e. How do WSMTs set tariff and who approves the tariff? What is community’s involvement in setting and agreeing on the tariff? f. What are the mechanisms in place to sustain services? PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 84 i. What are the maintenance schedules? ii. Are spare parts readily available (in the district, region)? iii. Does facility have the required accounts set up for the different types of maintenance – O&M, CAPEX and for extension? Are there funds to support sanitation services? iv. Are there plans for scaling up to meet the needs increasing population? Behavior Change 34. What kinds of behavior or attitude changes have been noted at the district level around demand for and investment in household sanitation services/facilities? Do you have evidence to back this up? 35. What shifts have you seen in terms of behavior change around hand washing? Do you have evidence to back this up? 36. Can you think of any parallels between other development interventions and products that could inform how we do sanitation delivery? 37. How will these behaviors be sustained going forward? 38. What approaches are used for ODF monitoring? What are the outcomes? Are communities able to maintain ODF? Are communities moving up the sanitation ladder? 39. What are the indicators of change for the Behavior Change strategy implemented by the W4H project? 40. How has the District supported the W4H work on behavior change? Does it help coordinate messages across different delivery mechanisms? 41. What have you learned about how to get messages out that have the desired impact of changing hand washing behaviors? Sustainability 42. How will the district continue to foster demand creation beyond the project? 43. How will the district continue to support supply beyond the project? 44. Are there particular challenges with regard to sustaining the gains made by the W4H activity? Please suggest how the initiative can be sustained. 45. [Apart from further financing] Do you have any advice for W4H moving forward? How can your collaboration with the project be further improved? PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 85 KII GUIDE – WSMT Targets: 1. WSMT (See also questions from Household / Community Level below regarding sanitation) 2. Tell me about yourself (Name/Position in the WSMT/Community) 3. What do you know about the W4H intervention being implemented by Global Communities in your community? 4. How was your community selected to benefit from the W4H Project? 5. What is the structure of the WSMT? Who does the WSMT report to? 6. (As relevant) Managing and sustaining Small Town Water System a. What is the structure for managing Small Town System? Does it follow the CWSA recommendations? b. Who are the members on the WSMT and how were they selected? [Gender consideration-60% as per GWSA guidelines] c. What are the responsibilities of the WSMT and the individual members? d. What type of training (contents and practice) was provided to the WSMTs? e. Who supervises/monitors the operations including financial management of the WSMTs and how does the monitoring take place? f. How does the WSMT set tariffs and who approves the tariff? What is the community’s involvement in setting and agreeing the tariff? g. What are the mechanisms in place to sustain services? i. What are the maintenance schedules? ii. Are spare parts readily available (in the district, region)? iii. Does the facility have the required accounts set up for the different types of maintenance – O&M, CAPEX and for extension? Are there funds to support sanitation services? 7. What recommendations do you have for improving management of the water systems towards ensuring sustainable services delivery? 8. [Apart from further financing] Do you have any advice for W4H moving forward? How can your collaboration with the project be further improved? 9. [Apart from further financing] Do you have any advice for government moving forward? PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 86 KII GUIDE – PRIVATE SECTOR General Information: • Name of Company • Type of company – (e.g. National Level Manufacturer, Regional Level Distributor, District level Distributor, small enterprise or artisan in community) • Location: • Name of Interviewee: • Contact: • Gender: • Position in the company: • Knowledge about project and Role in the W4H Project: • How long has informant been part of the project? Company Overview 1. When was the company established? What types of service or product is the company providing (sanitation and otherwise)? How long has the company been in the sanitation business? Did W4H encourage you to get involved in sanitation? Since you became part of the project, have there been any new products that you have developed? 2. What specific support, if any, has the W4H activity provided to the company? Distribution and Supply Chain 3. How many regions and districts do you work in? 4. How do you get your products to the users? 5. How many distribution centers are there? 6. Do you have any community-based agents? 7. How is the W4H program supporting the distribution of your products and services? 8. What kinds of challenges do you expect to face when W4H ends and how will you address this assuming you want to continue with the supply of your product/service? Sales and Marketing 9. What is the current customer base? How many clients were you able to reach before W4H and how many clients are you able to reach after engaging with W4H? 10. What is your view on the profitability of sanitation services? 11. How does your pricing structure work? Have you made any changes to your pricing to be more inclusive or to reach the poor? 12. Any major sales or projections for sales outside the project? Or for the next year? 13. What does the company do to market its products or services? How has marketing been supported by W4H, if at all? PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 87 Sustainability and Investment 14. Is the product available on the open market? Do you see any competition for what you offer? 15. What is your view of the future of the product or sanitation markets? Outside of this project, what prospects do you see for sanitation? 16. Are key manufacturers interested in taking the product forward? What would be required and what are they doing regarding taking product forward? 17. Are there any new product lines that you are considering based on your experience with W4H? What influences your decision to go into any particular type of product? 18. Outside support from W4H, are there any investments that the company is making? 19. How does your company raise funds for business? 20. If you consider a product lucrative, how do you do your research and product development? 21. [Apart from financing] Do you have any advice for W4H moving forward? How can your collaboration with the project be further improved? 22. [Apart from financing] Do you have any advice for government moving forward? PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 88 KII GUIDE – REGIONAL LEVEL AND DISTRICT LEVEL DISTRIBUTORS Region / District: Name of Company: Name of Interviewee: Position in company: Role in W4H project: Company Overview 1. When was the company established? What types of service or product is the company providing (sanitation and otherwise)? How long has the company been in the sanitation business? Did W4H encourage you to get involved in sanitation? Since you became part of the project, have there been any new products that you have started distributing? 2. Do you have links to any national level companies or manufacturers? Size of Market 3. How many districts/communities do you serve? 4. Do you have direct customers in the community? How many are these? 5. What type of services / products do you provide? (Sales of sanitation products, building materials or provision of services such as artisan services) 6. How do you get your products to the customers? Engagement with W4H 7. What kinds of challenges have you faced as a business? 8. What support have you received from W4H? (Did you receive any funding support from W4H?) 9. What have been the results of these interventions from W4H? Sustainability and Investment 10. What changes have you noted about the size of the market with regards to sanitation products and services in the district 11. What has been the level of demand; what levels of sales have been made 12. Do you have community-based agents that you work with? 13. What are your plans regarding sanitation business beyond the project? 14. Do you have plans for investment or expansion of the business? How do you raise funds to expand or run your business? 15. If you consider a product lucrative, how do you do your research and product development? 16. Once the W4H project has finished, how will you sustain the gains made? 17. [Apart from financing] Do you have any advice for W4H moving forward? How can your collaboration with the project be further improved? 18. [Apart from financing] Do you have any advice for government moving forward? PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 89 KII GUIDE – COMMUNITY LEVEL SME/ARTISAN General Information on Informant: • District: • Name of Community: • Name of Informant: • Contact; • Type of business: artisan or small SME: • Role in W4H Project: General Questions 1. How long have you been working as an artisan/SME? 2. Are you based in the community? (How often do you travel outside of the community to work?) 3. What type of sanitation products or services do you provide at the community level? 4. How many clients are you serving? 5. What are the key challenges faced in providing sanitation services to the communities? 6. How have you managed or dealt with these challenges? 7. Where do you get your supplies from? And how easy is to get materials? 8. What kind of support did you receive from W4H project? 9. What difference did this make in your business? 10. How do you support households with construction? How are you able to follow up with your clients or installations? 11. Are there some households that are not able to afford services? How do you deal with them? 12. Are there any other competing products or services that households invest in? 13. With the advent of Digni-lo, what effect has it had on your business? 14. What are your plans regarding sanitation business? 15. How do you raise funds to finance your business? 16. Beyond the project, how do you intend to grow your business? 17. [Apart from financing] Do you have any advice for W4H moving forward? How can your collaboration with the project be further improved? Alternate for Group discussions with artisans if they have a good number in district or community PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 90 KII GUIDE – WASH IN SCHOOLS Targets: 1. SMC/School Health Club/SHEP/Head Teacher Observations (refer to WinS guidance) Are there hand washing facilities visible in the school? Are the facilities functional? (Water and soap available for use, etc.) Is there evidence of use by pupils? 1. Tell me about yourself (Name/Department/Position) a. Tell me about your school: How many pupils/students do you have on roll? (Boys/girls) How many teachers do you have on roll? Does the school have a health club or hygiene educators? b. Tell me about the W4H interventions being implemented by Global Communities: How was your school selected to benefit from the project? When were facilities provided? What kinds of behavior change activities have been put in place? c. Are there other related interventions in the school from other Development Partners? How do these differ from what Global Communities is trying to do? School Facilities 2. Do boys and girls have equal access to WASH facilities in the school? 3. Are there changing rooms for girls with full view mirror? 4. How do you maintain your latrines and water points? 5. How do you finance preventive maintenance and major repairs? Is there a budget allocated in the school for O&M of WASH facilities? 6. How many times have your water facilities broken down and how long did it take to be repaired? 7. Who repaired it and did the school find it difficult to have a pump mechanic to repair it? 8. How much did the school pay? 9. Was the school trained to undertake preventive maintenance and does the school have the tools to enable them do that? 10. Do you have a Facility Management Plan (MP) and to what extent are you following the FMP? 11. What role does the PTA/SMC play in the school’s FMPs? 12. To what extent does the school use capitation grant to support operation and maintenance of WASH facilities? Behavior Change 13. How are sanitation and hygiene practices promoted for pupils of different age groups? Please give examples. 14. Where/how do you get soap for hand washing services? Is there a budget allocated for soap? 15. Have you seen a sustained shift in behaviors (social norms) around hand washing and hygiene? What PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 91 factors would continue to support this sustained change in behavior? 16. Are there particular challenges with regard to sustaining the gains made by the W4H activity? Please suggest how the initiative can be sustained. 17. [Apart from financing] Do you have any advice for W4H moving forward? How can your collaboration with the project be further improved? PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 92 KII GUIDE – HEALTH CARE FACILITIES (HCF) Target: 1. In-Charge, Midwives/Nurses Observations Are there hand washing facilities visible in the healthcare facility? Are the facilities functional? (Water and soap available for use, etc.) Is there evidence of use by patients? 1. Tell me about yourself (Name/Department/Position) a. Tell me about your health care facility – How many patients do you attend to in a day? (Male/female) How many staff do you have on roll? What are the three topmost diseases you treat? b. Tell me about the W4H interventions being implemented by Global Communities / How was your HCF selected to benefit from the project? When were facilities provided? What kinds of behavior change activities have been put in place? c. Are there other related interventions in the HCF from GoG or other Development Partners? How do these differ from what Global Communities is trying to do? HCF Wash Facilities 2. Is there equal access for WASH facilities in the HCF for both sexes including children? 3. How do you maintain your latrines and water points? 4. How do you finance preventive maintenance and major repairs? Is there a budget allocated in the HCF for O&M of WASH facilities? 5. How many times have your water facilities broken down and how long did it take to be repaired? Who repaired it and was it difficult to find a pump mechanic to repair it? How much did the HCF pay? 6. Was the HCF trained to undertake preventive maintenance and does the HCF have the tools to enable them do that? 7. Do you have a Facility Management Plan (MP) and to what extent are you following the FMP? 8. To what extent does the HCF use capitation grant to support O&M of WASH facilities? Behavior Change 9. How are sanitation and hygiene practices promoted for patients/clients? Please give examples. 10. Where/how do you get soap for hand washing services? Is there a budget allocated for soap? 11. Have you seen a sustained shift in behaviors (social norms) around hand washing and hygiene? What factors would continue to support this sustained change in behavior? 12. Are there particular challenges with regard to sustaining the gains made by the W4H activity? Please suggest how the initiative can be sustained. 13. [Apart from financing] Do you have any advice for W4H moving forward? How can your collaboration with the project be further improved? PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 93 KII GUIDE – COMMUNITY (HOUSEHOLD AND FGD GUIDE) Target: 1. Community Members – Leaders, NLs, Households, Households benefiting from Pro-poor Guidelines Observations Are there hand washing facilities visible in the house? Are the facilities functional? (Water and soap available for use, for example) Is there evidence of use by household members? 1. Tell me about yourself and the community Managing the Water Supply (FGD) 2. How does the community manage the water facility? 3. Are you aware of a group/individual who is responsible for managing the water system? 4. Has the WSMT been trained and do they have the tools to undertake preventive maintenance? 5. When was the last time your water facility broke down and how long did it take to be repaired? 6. Who repaired it and where did the spare parts come from? 7. How much did the community pay and what was the source of the money? Does the community keep collected tariffs in a bank account? Sanitation and Toilet Provision 8. Do you have a toilet facility in the house? If no, where does the household defecate? If yes, what type of facility do you have? 9. How did you install your latrine a. What types of latrine are used in the community (when they were built?) b. For own latrine (did you build by yourself or used artisan; how easy is it to get an artisan, how affordable are their charges) c. If you built by yourself, where did you buy the parts for latrine installation d. How did you raise funds for latrine construction or purchase of product? e. Any follow up from W4H/GC/DA since you installed? 10. How and when did you get to build it? What is their level of satisfaction with it? How is the latrine maintained and by whom? 11. What difference has the facility made to the household? 12. What will you do if there is a problem, like your latrine collapses or becomes full? Are there private providers who will make repairs for you? (Availability of spare parts for latrines that require them, e.g. vent pipe; door locks etc.) 13. Replacement: a. How long do you hope your latrine will last? b. What will you do when your latrine is full? (for households using digniloo, link this to the idea of taking out the substructure and reinstalling) PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 94 c. Will you prefer new one/land available for new latrines 14. Community members to identify the sanitation products or options currently in use by the community/households (adapt above questions for an FGD) d. How much did people pay for these products? How long did it take to pay for these products or undertake installation (if they built it themselves)? How did they fund it? 15. Have you seen a shift in attitudes towards investing in sanitation facilities in the home? If funds are available, do community members prioritize sanitation as compared to other products? Is it affordable for poorer households to invest in a toilet? 16. What support did you receive for constructing your latrines and who provided the support? Did community leaders play a part? How engaged is the District in ensuring access to sanitation? Do you have natural leaders in the community? What is their role in improving sanitation and hygiene in the community? 17. Has there been special attention paid to poorer members of the community to ensure they have access to sanitation facilities? What role did the community play in supporting the poor and vulnerable to have toilet? Hand Washing and Behavior Change 18. What do you know about hand washing? Why do you wash your hands? 19. Where do you hear about the need to wash your hands? 20. How often do you wash your hands? 21. Have you seen a shift in attitudes about hand washing and hygiene? 22. How is the WSMT supporting community members to maintain the practice of hand washing? For Natural Leaders Specifically 23. What has been your role in the W4H project? 24. How have you created demand in the community? 25. How have you facilitated access to sanitation products for community members 26. What has been the level of acceptability for the sanitation products? Which products or options are mostly patronized by the households? 27. What kinds of support do the household members need and how do you provide support? 28. Beyond the projects, how will you take forward your role as natural leader (explore if there are any project-based incentives that might not be available after the project) Looking to the Future 29. What will the community do after the project has ended and there are new people who need support? 30. Apart from providing further financing, what advice would you provide to Global Communities about how to strengthen these programs and make sure that the projects are sustainable? PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 95 ANNEX IV: DATA SOURCES LIST OF KEY INFORMANTS Below are the names of the key informants consulted during the review categorized by type Respondent Name Title Location / Geographic Focus Global Communities Staff Alberto Wilde Country Director / Chief of Party National Dominic Osei Deputy Director / Technical Director / Deputy Chief of Party National Felix Amofa Business Development National Martha Tia Adjei CLTS/MHM Specialist National Sampson Renner Hydrologist National Dominic Dapaah Regional Boss – Tamale Tamale / National Linda Amponsah Senior BCC Staff National Edward Thompson Senior BCC Staff, Representative of Manoff Group National Albert Gharbin WASH Officer Ada West Derrick WASH Officer Amenffi Central Andy Kontoh WASH Officer Asikuma Odoben Brakwa Kpamba Tanko WASH Officer Krachi East Austin Amoaku WASH Officer Nkwanta North James Lomote Behavior Change Specialist Nkwanta North Josiah Aduah WASH Officer Sawla Tuna Kalba George District Assembly – Field Staff Sefwi Bodi Mattais Binfoh WASH Officer Sefwi Wiawso Philip WASH Officer Wassa Amenfi West USAID Emmanuel Odotei USAID – W4H AOR National Development Partners – National Jessica Tribbe Aquaya National Mireille Hitti Duraplast, Executive Director National PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 96 Respondent Name Title Location / Geographic Focus Vida Duti IRC – Ghana National Veronica Ayibontey IRC – Ghana National Edward K.E. Thompson Manoff Group, Senior Behavior Change Communication Specialist National Ako Odotei Rotary Ghana, Host Committee Chairman National Kate Flemming S4H National Bright Gemegah S4H National Eric Adjei S4H National Demi Duah Total Family Health Organization (TFHO) Technical Director National Lorretta Roberts UNICEF – WASH Officer National Ramesh UNICEF – WASH Chief National Gabriel Adu-Wusu UNICEF – WASH Officer National Henry Herbert Malm Unilever – Head of Corporate Communications and Sustainability National Ann-Sherie Aidoo Unilever – Communications National Abdul-Nashiru Mohammed WaterAid – Country Director National Mathilda Afriyie WaterAid – Performance Monitoring Evaluator National County Counterparts (Government and Civil Society) Community Water and Sanitation Agency (CWSA) Ada West Community Water and Sanitation Agency (CWSA) Ada West Andrew Nii Apai Aborhey District Assembly – District Environmental Health Officer (DEHO) Ada West Ammanuel Annan District Assembly – Environmental Health Officer Ada West Lovina Kwateng District Assembly Assistant Director Ada West Senyo Agdemasu District Assembly Assistant Director Ada West Defali Kofi Mordeghi District Assembly Planning Officer Ada West Emmanuel Tetteh Ambah Sackey Head of Community Development Unit Asikuma Odoben Brakwa PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 97 Respondent Name Title Location / Geographic Focus Emmanuel Seniadza District Focal Person for Global Communities Project Asikuma Odoben Brakwa Solomon Lamon Environmental Health Assistant and M&E Focal Person for Environmental Health Unit and CLTS Asikuma Odoben Brakwa Timothy Gadagbui District Planning Officer and WASH Focal Person Asikuma Odoben Brakwa Alex Damptey District School Health Education Programme (D-SHEP) Sefwi Bodi Paul Dogah District Assebly – DEHO Sefwi Bodi Franklin Boateng District Budget Officer Sefwi Bodi Isaac Akowuah District Finance Officer Sefwi Bodi Theodora Adomako Adjei CWSA National Kweku Quansah Deputy Director of Environmental Health and Sanitation Directorate National Ellen Gyekye Programme Officer for School Health Education Programme National Sulemna Yakubu EHS Regional and REHO National Alabira Osuman Coordinator CLTS National Peninah SHEP Krachi East William Agbodo Holy Rosary Head Teacher Krachi East Winfred Botsyoe Kwami Akura Head Teacher Krachi East Dah Joseph MEHO Krachi East Abus Ibrahim Municipal Planning Officer Krachi East Ostin Amoako District WASH Officer Nkwanta North Abraham Domonja Environmental Health Assistant Nkwanta North Nanaman Shadak Field Officer Nkwanta North Toby Ayim Adzokodo II Head of Community Development Nkwanta North Moses Bitir Environmental Health Assistant Nkwanta North Richard Newel Kabonwule Community-Based Health Planning and Services (CHPS) Compound – Registered Community Nurse Nkwanta North PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 98 Respondent Name Title Location / Geographic Focus Anthony Waasan Cement/Vent Pipe Dealer Nkwanta North Winfred Botswe SHEP Nkwanta North Agnes Shai Osudoku EHP Shai Osudoku Julius Shai Osudoku EHP Shai Osudoku Ebeneezer Asiamah District Health Officer Shai Osudoku Roberta District SHEP Coordinator Shai Osudoku Abubakari Sadique Dawuda DEHO Sawla Tuna Kalba Justice Kusi Gindabour CHPS – Community Health Nurse Sawla Tuna Kalba Seidu Alhassan Kulmasa RC Head Teacher Sawla Tuna Kalba Takuradum Justice Sawla Tuna Kalba Small Town Water System – Management Team Chairman Sawla Tuna Kalba Saaka Ibrahim Municipal Coordinating Director Wassa Amenfi West George Agyiri Municipal Chief Executive (MCE) Wassa Amenfi West Benjamin Baah Municipal Environmental Health Officer (MEHO) Wassa Amenfi West Robert Zory Deputy MEHO Wassa Amenfi West Kingsley Kenneth Nunoo Municipal Budget Officer Wassa Amenfi West Linda Assiedu Municipal Planning Officer Wassa Amenfi West Anas Baba Iddrisu Agriculture District of Environmental Health Officer (DEHO) West Mamprusi Sualisu Abudu Braimah M&E Officer West Mamprusi George Dapilah In Charge of Ayinabirim Health Center Sefwi Wiawso Matthew CWSA Regional Water Safety Specialist Sefwi Wiawso Darkwah MEHO Sefwi Wiawso Asamoah Assistant MEHO Sefwi Wiawso Louis Ouwus Acheampong MCE Sefwi Wiawso Oscar Municipal Finance Officer Sefwi Wiawso PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 99 Respondent Name Title Location / Geographic Focus Hardin Municipal Engineer Sefwi Wiawso Benson Municipal Health Officer Sefwi Wiawso PRIMARY DOCUMENTS CONSULTED IN DESK REVIEW W4H Contractual Deliverables and Related Documents Activity Monitoring and Evaluation Plan (updated January 2017) A Market-based, Pro-poor Approach to Rural Sanitation: Global Communities Case Study, 2019 Annual Reports: FY15, FY16, FY17, FY18 Annual Work Plans for FY15, FY16, FY17, FY19 Borkowski, J. What Makes Ghanaians More Likely to Stop Open Defecation and Build Latrines? Global Communities Research Brief: November 2019. Maps for the Minister of Sanitation April 2017 Memorandum of Understanding – Global Communities and Sefwi Wiawso District Assembly Quarterly Reports: FY15 Q3/Q4, FY16 Q1/Q2, FY18 Q2/Q3 Report on 2017 Global Handwashing Day Updates for Minister for Sanitation and Water W4H Activity Monitoring and Evaluation Plan (AMEP) W4H Facility Data W4H Inception Meetings- Zone 1 Presentation W4H Northern Region Selection Presentation W4H BCC Roll-Out Strategy, 2016 W4H Technical Proposal and Annexes W4H Program Description, Excerpt from Cooperative Agreement No. AID-641-A-15-00005 W4H Success Stories A Functional Latrine in One Week Access to Potable Water Reduces the Occurrence of Diarrhea in Takuka Bye Cholera, Accra Does Not Miss You Community-Led Total Sanitation Helps Improve Nutrition From Open Defecation to ‘Sanitized Community’ — The Journey of Bekuikope PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 100 W4H Success Stories Innovation at Work: Plastic Latrine Slabs Address Ghana’s Sanitation Challenges No More Crying on the Trail: Water in Daboya No. 2 Saving Lives with a Little Soap and Water: New Behavior Change Communication Package promotes healthy hygiene practices Success Story For WADA/USAID/Global Communities Sanitation Project The Challenged Leading Kwaku Ninsin to ODF: The Challenged Curbing Open Defecation With Assistance from WASH for Health, Household Latrines Become a Reality GOG Policy and Strategy Documents Environmental Health and Sanitation Directorate (EHSD) of the Ministry of Local Government and Rural Development (MLGRD) and Water Directorate (WD) of the Ministry of Water Resources Works and Housing (MWRWH), Water, Sanitation and Hygiene (WASH) Behavior Change Communication (BCC) Strategy for the Urban Sub-Sector. June 2011. Environmental Health and Sanitation Directorate, Ministry of Local Government and Rural Development, National Environmental Sanitation Strategy and Action Plan (NESSAP) 2010: MINT, Materials In Transition. March 2010. Environmental Health and Sanitation Directorate, Rural Sanitation Model and Strategy: Abridged version. April 2015. Environmental Health Directorate, Ministry of Local Government And Rural Development, Advisory Notes: Rural Sanitation Model and Strategy. February 2014. Ghana Education Service, National Implementation Model WASH in Schools, 2014. Ghana Education Service, School Health Education Programme: Policy Guidelines. July 2012. Ghana Education Service, Technical Guide for WASH in Schools Facilities, 2014 Ministry of Local and Rural Development, Environmental Sanitation Policy (Revised 2009). April 2010. Ministry of Local Government and Rural Development and the Environmental Health and Environmental and Sanitation Directorate (EHSD), MDG Acceleration Framework (MAF) - Country Action Plan For Sanitation: Go Sanitation Go! October 2011. Ministry of Local Government and Rural Development, Strategic Environmental Sanitation Investment Plan (SESIP): Achieving Minimum Service Options by 2015. April 2011. Ministry of Sanitation and Water Resources, Guidelines for Targeting the Poor and Vulnerable for Sanitation Services in Ghana. June 2018. Ministry of Sanitation and Water Resources, WASH Golden Indicators: A conceptual introduction. October 2018. Ministry of Water Resources, Works and Housing, National Water Policy 2007. June 2007. PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 101 Other Documents Aguaconsult Ltd & Maple Consult. Rural Sanitation Model and Costed Scaling Up Strategy for Community-Led Total Sanitation (CLTS) & Hygiene in Ghana. March 22, 2011. Andres, Luis A.; Thibert, Michael; Lombana Cordoba, Camilo; Danilenko, Alexander V.; Joseph, George; Borja-Vega, Christian. 2019. Doing More with Less: Smarter Subsidies for Water Supply and Sanitation. World Bank, Washington, DC. © World Bank. https://openknowledge.worldbank.org/handle/10986/32277 License: CC BY 3.0 IGO. Coleman & Partners, Public Health Laws Of Ghana: Environmental Public Health Regulation. April 8, 2009. Ghana District League Tables 2015/16/17/18/19: Strengthening Social Accountability for National Development, UNICEF and CDD-Ghana. Hickling, S., Tackling Slippage. CLTS Knowledge Hub, Issue 14, Sept 2019. Jiménez, A.; Jawara, D.; LeDeunff, H.; Naylor, K.A.; Scharp, C. Sustainability in Practice: Experiences from Rural Water and Sanitation Services in West Africa. Sustainability 2017, 9, 403. (Accessed 6 August 2019 at https://www.mdpi.com/2071-1050/9/3/403) National systems to support drinking-water, sanitation and hygiene: global status report 2019. UN￾Water global analysis and assessment of sanitation and drinking water (GLAAS) 2019 report. Geneva: World Health Organization; 2019. License: CC BY-NC-SA 3.0 IGO. USAID, 2018. An Examination of CLTS’s Contributions toward Universal Sanitation. Washington, DC., USAID Water, Sanitation, and Hygiene Partnerships and Sustainability (WASHPaLS) Project. USAID, 2018. Scaling Market Based Sanitation: Desk review on market-based rural sanitation development programs. Washington, DC., USAID Water, Sanitation, and Hygiene Partnerships and Learning for Sustainability (WASHPaLS) Project. USAID, 2019. Research on the Impact of Targeted Subsidies Within Open Defecation Free (ODF) Communities: Inception Report. Washington, DC., USAID Water, Sanitation, and Hygiene Partnerships and Sustainability (WASHPaLS) Project. References Appiah-Effah, E., Armstrong Duku, G., Yakubu Azangbego, N., Kojo Aduafo Aggrey, R., Gyapong￾Korsah, B., and Biritwum Nyarko, K. “Ghana’s post-MDGs sanitation situation: An overview.” Journal of Water, Sanitation and Hygiene for Development, 2019. United Nations Millennium Development Goal Report, “Assessing progress in Africa toward the Millennium Development Goals,” 2015. World Health Organization/UNICEF Joint Monitoring Program, “Progress on drinking water, sanitation, and hygiene: 2017 update and Sustainable Development Goal baselines,” 2017 PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 102 ANNEX V: DISCLOSURE OF ANY CONFLICTS OF INTEREST Conflict of Interest Disclosure Form Note: A potential or actual conflict of interest exists when commitments and obligations are likely to be compromised by the individual's other material interests, or relationships (especially economic), particularly if those interests or commitments are not disclosed. Date: Name: Position: Please describe below any relationships, transactions, positions you hold (volunteer or otherwise), or circumstances that you believe could contribute to a conflict of interest: ___ I have no conflict of interest to report. ___ I have the following conflict of interest to report (please specify other nonprofit and for-profit boards you (and your spouse) sit on, any for-profit businesses for which you or an immediate family member are an officer or director, or a majority shareholder, and the name of your employer and any businesses you or a family member own: 1. ___________________________________________________________________________ 2. ___________________________________________________________________________ 3. ___________________________________________________________________________ I hereby certify that the information set forth above is true and complete to the best of my knowledge. Signature: _____________________________________________________________________ Date: ________________________________________________________________________ PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 103 PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 104 PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 105 PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 106 PERFORMANCE EVALUATION OF THE GHANA WASH FOR HEALTH (W4H) ACTIVITY 107 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523 Tel: (202) 712-0000 Fax: (202) 216-3524 www.usaid.gov