External Program Evaluation Water, Sanitation And Hygiene (WASH) Program In Ethiopia Final Report December 2008 EXTERNAL PROGRAM EVALUATION Of COOPERATIVE AGREEMENT No. 663-A-00-04-00419-00 MILLENNIUM WATER ALLIANCE (MWA) WATER, SANITAION AND HYGIENE (WASH) PROGRAM IN ETHIOPIA The Mitchell Group, December 2008 Bruck Aregai (Team Leader) – Water, Sanitation & Hygiene Specialist Siseraw Dinku - Senior Water, Sanitation & Hygiene Expert Disclaimer: The authors’ views expressed in this publication do not necessarily reflect the views of the U.S. Agency for International Development or the United States Government. Queue for service at Bedowacho A typical household pit latrine in Amhara ii TABLE OF CONTENTS ACRONYMS AND ABBREVIATIONS ........................................................................ iv EXECUTIVE SUMMARY .............................................................................................. 1 1. INTRODUCTION..................................................................................................... 5 1.1 Purpose ................................................................................................................ 5 1.2 Background ......................................................................................................... 6 1.3 Methodology ....................................................................................................... 8 1.4 Limitations ........................................................................................................ 10 2. FINDINGS AND CONCLUSION ......................................................................... 12 2.1 Progress and Achievements Toward Stated Objectives .................................... 12 2.2 Impacts .............................................................................................................. 15 2.3 Coordination and Integration ............................................................................ 16 2.4 Sustainability..................................................................................................... 17 2.5 Linkage With Other Programs .......................................................................... 20 2.6 Engineering Design and Management .............................................................. 21 2.7 Behavior Change Design, Implementation and Result ..................................... 23 2.8 Capacity Development ...................................................................................... 27 2.9 Challenges Encountered.................................................................................... 29 2.10 Efficiency of Resource Utilization .................................................................... 29 3. CONCLUSIONS ..................................................................................................... 31 4. LESSONS LEARNED ............................................................................................ 33 5. PRIORITIZED RECOMMENDATIONS ............................................................ 34 6. FUTURE DIRECTIONS ........................................................................................ 36 iii ACKNOWLEDGMENT The evaluation team wishes to acknowledge the administrative support of Alex Mckenzie and the uninterrupted technical support and undivided attention of Rose Schneider at The Mitchell Group. The evaluation team would like to acknowledge that the work has been facilitated and coordinated through the support provided by Mrs. Anita Gibson, Mrs Sophia Brewer and Dr. Mulugeta Yohannes at United States Agency for International Development /Ethiopia (USAID/E). The evaluation team is deeply thankful to all of them for their diligent, unwavering support and patience throughout this process. We also thank them for their valuable comments and suggestions they made at debriefing sessions which helped us enrich and refine this report. The evaluation undertaking would not have been possible without the conscientious work and strong support of a number of people and organizations. The authors wish to acknowledge Ato Teshome Lemma, the Millennium Water Alliance (MWA) Water Supply, Sanitation and Hygiene (WASH) program coordinator, who was responsible for facilitating our work in the field and in Addis Ababa, and for making sample project documents and supplementary information on the program activities available to us. Most of all, this evaluation would not have been possible without the cooperation, knowledge and wisdom of the households, communities, and concerned government staff and officials and implementing Non Government Organizations (NGOs) in the visited program areas, who have participated and facilitated the process on the ground and provided the evaluation team with insights and valuable facts present in this report. Last but not least, we also wish to acknowledge and express our gratitude for logistical and technical support throughout the assignment in Ethiopia to Dr. Michael Dejene. Bruck Aregai (WASH Specialist – Team Leader) Siseraw Dinku (Senior WASH Expert) iv ACRONYMS AND ABBREVIATIONS AIDS Acquired Immune Deficiency Syndrome CHP Community Health Promoters CIS Corrugated Iron Sheet CLTS Community Led Total Sanitation CRS Catholic Relief Services DA Development Agent ETB Ethiopian Birr FHI Food for the Hungry International FTC Farmers’ Training Center GOE Government of Ethiopia H&S Hygiene and Sanitation HAPN Health, AIDS, Population and Nutrition HEP Health Extension Program HEW Health Extension Workers HIP Hygiene Improvement Project HHs Households IWRM Integrated Water Resource Management LD Line Departments LWI Living Water International MDG Millennium Development Goal M&E Monitoring and Evaluation MOH Ministry of Health MOU Memorandum of Understanding MWA Millennium Water Alliance MWA/E Millennium Water Alliance/Ethiopia MWP Millennium Water Program NGOs Non Government Organizations O&M Operation and Maintenance PHAST Participatory Hygiene and Sanitation Transformation PMG Program Management Group PPP Private Public Partnership PS Private Sector R-WASH Rural Water, Sanitation & Hygiene SNNPRS Southern Nations, Nationalities and Peoples Regional State SOW Statement of Work TOT Training of Trainers TSP Technical Service Providers UAP Universal Access Plan UDC University of the District of Columbia USAID United States Agency for International Development USAID/E United States Agency for International Development/Ethiopia VHP Village Health Promoters VIP Ventilated Improved Pit VLOM Village Level Operation & Maintenance v WASH Water Supply; Sanitation & Hygiene WASHCO Water, Sanitation & Hygiene Committees WHO Woreda Health Office WPI Water Partners International WSDP Water Sector Development Program WSP Water and Sanitation Program - Africa WV World Vision WWO Woreda Water Office WWT Woreda WASH Team Definitions CARE Humanitarian Organization Kebele Part of a woreda (district), smallest administrative unit Motorized schemes Water systems equipped with electrical pump & diesel generator Projects Activities implemented by individual partner NGOs Sites Actual location of water supply and sanitation facilities Woreda District 1 EXECUTIVE SUMMARY Introduction This report presents results of the evaluation of the ongoing (2004-2009) Water, Sanitation and Hygiene (WASH) Program funded by USAID, in cooperation with the Millennium Water Alliance (MWA), under Cooperative Agreement No. 633-A-00-04- 00419-00. The program’s scope and activities have evolved over the last three years (2005-2007) subsequent to agreement modifications because there was no long-term planning. However, the basic elements and stated objectives of the program have remained constant throughout. Currently, the program is being implemented through eight international MWA partners and four local sub-contractor Non Government Organizations (NGOs) in 30 woredas (districts) in Ethiopia. The evaluation was commissioned by the USAID/Ethiopia (USAID/E) Health, AIDS, Population and Nutrition (HAPN) office. The Mitchell Group (TMG) was requested to provide technical assistance under the USAID/E Evaluation Contract Number 663-C-00- 08-00409-00, to design and implement an independent external project evaluation. The purpose of the evaluation was to assess the performance and outcomes of the program, examine the effects it has had on the target beneficiaries, the challenges encountered during program implementation, the weaknesses and strengths of the program, and document the lessons learned for future programming. The evaluation was conducted in November and December 2008 by a team of two independent consultants with expertise in water resources supply and management and public health. The team spent 26 days in the country and 9 working days in the field between November 24 and December 18. Qualitative data was gathered to assess impacts as well as program performance. Data was collected through review of key program documents, key informant interviews with project and partners’ staff, focus group discussions with beneficiaries, and direct observation of activities in selected sites in three regions. Information gathered were analyzed using specific evaluation criteria including output level achievements, effectiveness, impact, and sustainability. Key findings and preliminary conclusions and recommendations of the evaluation were also presented at de-briefing sessions to seven MWA implementing partner organizations and USAID/E Mission staff for their comments and suggestions. The evaluation was constrained by the short time available for fieldwork in project sites and the distances involved. The number of NGOs and other organizations to be interviewed based on the issues to be addressed in the Statement of Work (SOW) was also constraining. The evaluation had to limit field visits to only five projects implemented by four partners in five accessible woredas (districts) of three regions. The selected projects or partners are therefore not necessarily representative, implying that 2 some of the findings at the woreda and kebeles (a smaller unit of woredas) level may not be applicable to all other projects. Service Provision Since the start of the program in 2004 to September 2008, 411 new and existing water supply schemes have been constructed and/or rehabilitated. As a result, about 226,080 people have gained access to a safe water supply. The sanitation and hygiene interventions also created access to individual household sanitation facilities to over 75,000 people in various communities and to 65,284 students in 64 elementary schools. Overall, approximately 150,000 people gained access to safe water supply, sanitation facilities and hygiene education in MWA target areas. Impact In all contacted community members visited areas were very pleased with the improved water facilities provided by the projects, and were observed to have been making good use of them. Both community members and governmental partner staff reported that the availability of safe drinking water alone has already had tangible benefits and impacts on people’s health, especially among children who are now much less likely to suffer from diarrhea. There is also a perception among women that they have benefited most from the water supply interventions as a result of both the reduction in the amount of time and effort they now have to spend collecting water and the improvements in the health of their children. However, the evaluation could not satisfactorily measure and verify these changes due to absence of baseline data at the beginning of projects and monitoring information on key indicators during project implementation (Recommendation 2). Integration Implementation of integrated water supply, sanitation and hygiene at the community level is encouraging. However, more emphasis is required to accelerate especially the sanitation and hygiene component as it still lags behind water supply development. Working directly with health extension workers and community volunteers and health promoters is fundamental to sustainable change in the targeted communities. At the same time, MWA needs to work with the local government to ensure the responsibility of Water, Sanitation and Hygiene Committees (WASHCOs) including clearly defined tasks in hygiene promotion, possibly with one member specifically assigned to supervise and coordinate these activities (Recommendation 11 & 12). Sustainability Communities and/or households in visited sites were highly engaged during the evaluation period, and found by the evaluation team to have been sufficiently involved in the process of project implementation as well as post-implementation management of facilities. Specifically, targeted communities were involved in sharing capital costs through provision of free labor and locally available construction materials. However, their participation in project planning, such as deciding service levels and the design and implementation of hygiene promotion activities, has been minimal. In this regard, while the efforts made to involve communities by all visited projects is commendable, communities should be empowered to be more actively involved in all project phases and 3 in deciding on their service level as the primary owners of WASH interventions (Recommendation 1). In 80% of the visited sites, WASHCOs were organized and trained under the program and have taken over responsibility for the management, operation and maintenance of facilities. All benefiting committees have introduced appropriate financing systems by which they collected user fees (in rural areas) or fees from water sales (at fixed rates in peri-urban areas) monthly. WASHCOs have also started using bank accounts where they regularly deposit money to independently cover operation and maintenance costs. In most of the areas visited by the evaluation team, the committees are functioning in a responsible manner. Furthermore, efforts have been made to link WASHCOs with woreda water and health offices from which they potentially receive follow-up and technical support in matters exceeding their capacities. However, the weak capacity of woreda offices and absence of clear lines of accountability, including the lack of WASHCOs’ legal status and ownership rights of user groups, remains a constant threat to the long-term sustainability of benefits. Behavior Change Design, Implementation and Result The absence of a well articulated IEC/BCC strategy has perhaps limited the program impact on disease reduction (Recommendation 6). Integration of hygiene promotion into water and sanitation program with the aim of bringing about meaningful behavior change requires more effort. It particularly requires a thorough planning of IEC/BCC activities based on context-specific assessments of existing conditions and practices, and development of sound and effective strategies with clearly defined behavioral objectives, target groups, messages, communication channels, as well as Monitoring and Evaluation (M&E) mechanisms taking available resources into account (Recommendations 1 & 8). Capacity Building Capacity building of relevant woreda offices (particularly water, health and education) is one of the most important missing elements in the program. Sectoral capacity is especially weak at the woreda level, with a subsequent ‘down stream’ impact on capacity at kebele and village level. In addition to limitations in technical capacity and mobility, poor recognition, low morale, frequent turn-over and vacant posts are serious problems in the field. In terms of scope, the capacity gap impacts on coordination, planning, implementation, (most notably related to decentralized decision making and hygiene promotion. It is also linked to technology selection), longer-term support to Operation & Maintenance (O&M), and monitoring and evaluation. Support for developing private sector involvement was also limited to only one of the projects (implemented by CARE in Farta woreda of the Amhara region). In a country like Ethiopia where the WASH programs are constantly faced with challenges for timely implementation and quality service delivery, support to small service providers at the woreda level deserves more attention (Recommendation.13). 4 On the other hand, to ensure continuity and sustainable use of WASH facilities, it is equally important to develop the local government capacity to provide technical and administrative supports to communities. It is a fact that local government capacity is inadequate for taking over completed WASH systems and providing support to communities to manage the systems sustainably. Therefore, it is apparent that MWA must devise a clear exit strategy that makes sure a reliable and functional local government counterpart is developed both in terms of skilled human resources and institutional systems is in place before handing projects over to the relevant woreda offices (Recommendation 4). Main Conclusion It should be noted that MWA is making progress in terms of increasing access to safe water supplies, sanitation and hygiene services in its target areas. There are also some good beginnings of the program outcomes that contribute positively to the improved health and education of beneficiaries. MWA peer-reviews, field visits, discussions and exposure to different ways of doing business are forward-looking practices and should be supported further. Discussions about strengths and weaknesses of approaches at Program Management Group (PMG) meetings are important as well. In addition, the partnership has a potential to push the boundaries to further improve quality. For example, promoting the importance of hygiene and sanitation will improve quality much more than just accepting water related projects. In addition, introducing innovative ideas to accelerate the sanitation and hygiene agenda based on agreed priorities/approaches in joint applications will help. The possibilities of research at scale such as baseline findings should be further explored. Because impact studies and set research agendas can influence MWA partners’ work and the work of other partners in the future, they should be further explored as well (Recommendation 13). The evaluation team learned that the MWA is in an excellent position to play a prominent role in changing constraints and challenges to opportunities to support the national WASH program. MWA is therefore encouraged to take advantage of these opportunities, both through service delivery and advocacy. In conclusion, these gains demonstrate a strong commitment, improved harmonization of development efforts by member NGO partners and increased donor support. In addition, the MWA needs to be strengthened both in programming and operations to further its initiatives and role as an important WASH player in the country. 5 1. INTRODUCTION 1.1 Purpose The SOW stipulates that the overall objective of the evaluation of the USAID Supported WASH Program is to implement an independent external assessment of the MWA water supply, sanitation and hygiene program. The immediate objective of the evaluation is to assess: a.) achievements of the stated MWA program objectives; b.) quality and standard of the service as compared with USAID and GOE guidelines; c.) impact of the project; d.) efficiency of resource utilization: and, e.) program sustainability The evaluation is to document the outputs/outcomes, lessons learned, challenges encountered, and the result of the program’s contributions to Ethiopia’s Water Sector Development Program (WSDP) and benefiting communities. The evaluation identified the major achievements, challenges, and lessons learned. To the extent possible, it also identified the impact of the water and sanitation program in achieving the above objectives based on available primary and secondary data. The evaluation analyzes the main findings and reflects on the performance of the program as a whole and offers practical recommendations for improvement. The report describes findings at community, kebele and woreda partners levels. It is a program evaluation, and, as such, does not include detailed analysis of specific projects implemented by individual NGOs. The findings of the evaluation will be used for developing further cooperation: the findings and recommendations will assist to develop agreed upon indicators to monitor and evaluate program performance in the future. This evaluation covers the MWA water supply, sanitation and hygiene program financed by USAID/E and other donors based on interviews at the national level and fieldwork in five woredas of the three regions of Amhara, Oromia and Southern Nations, Nationalities and Peoples Regional State (SNNPRS). The evaluation covers the entire program period since its commencement in 2004 until 2008. The key findings and preliminary conclusions of the evaluation were discussed on 12 December 2008 with MWA partners and with USAID/E on 16 and 18 December 2008. The views and suggestions of seven MWA partners1 made at the debriefing sessions are incorporated into this document to the extent possible. 1 The debriefing session with MWP partners was organized by the MWA Secretariat Office, and held on 12th December 2008, from 02:30 to 04:15 it the Conference Room of CARE - Ethiopia. The session was attended by more than 20 people from 7 partner NGOs and the Secretariat/Program Coordination Office. Specifically, participants were representatives and key WASH officers of Water Action, FHI, World Vision, CRS, EECMY/DASSC, Hope 2020, CARE/E and staff of the MWA Secretariat Office. Although the Program Coordination Office invited all 12 partners and notified each in advance, five did not attend for various reasons. Similar debriefing sessions were held with 10 relevant staff of USAID/E first on 16/12/08 and later with 4 management staff on 18/12/08. Valuable comments and suggestions were also made by the participants at these meetings, which helped refine the presentation of the draft 6 Main Policy Principles in the Sector The overall water sector policy promotes: 9 Water as an economic good. 9 Devolving ownership and management autonomy to the lowest possible level. 9 Promoting involvement of all stakeholders to improve efficiency. 9 Gradual full cost recovery for urban systems and Operation and Maintenance (O&M) for rural systems. 9 Integrating planning for sanitation and hygiene with water supply. The national sanitation strategy seeks to promote: 9 Enabling frameworks to accelerate scaling up. 9 Use of participatory learning, advocacy, and incentive based approaches. 9 Appropriate technology and reliance on local producers and artisans. The immediate objectives of the sanitation protocol are: 9 To promote the integration of Hygiene and Sanitation (H&S) promotion with planning. 9 To improve sector coordination through the preparation of clear guidelines. 9 To define minimum standards and manage information to evaluate performance. 1.2 Background According to USAID/E Statement of Work (SOW) for the MWA WASH program evaluation, “approximately 3.1% of deaths worldwide are attributed to unsafe water, sanitation and hygiene practices. Africa carries the heaviest burden, with 4 to 8% of all disease in Africa being related to poor water, sanitation and hygiene. In Ethiopia, water￾and sanitation- related diarrhea accounts for approximately 20% of all deaths in children under the age of five, taking the lives of close to 100,000 children annually. Thirty-two percent of this diarrhea could be prevented by improving sanitation interventions such as pit latrines, septic tanks and composting toilets.” According to the 2005 Ethiopian Demographic and Health Survey, only 8% of Ethiopian households have water on their premises and only 38% have a toilet. In addition, poor water and sanitation is the source for many other health problems including chronic intestinal parasites that attribute to high prevalence of malnutrition, anemia and retarded growth. Poor water and sanitation are also major causes of blinding trachoma in Ethiopia. In addition, climactic conditions such as floods and droughts, which cause diarrhea because of water scarcity and food insecurity, have become frequent occurrences in the country. Realizing the need for accessible and equitable WASH services, the Government of Ethiopia has begun an ambitious Universal Access2 Programme (UAP) which aims to ensure 98% and 100% coverage for water and sanitation services respectively by 2012, three years prior to the MDG cut off dates. Progress has been made in creating an enabling environment with an appropriate policy and proclamation backed up by the development of a national hygiene and sanitation strategy report. The purpose of the meetings was to present the findings and preliminary conclusions and recommendations of the program evaluation to program partners for joint discussion and comments. As the evaluation exercise including the preparation of the draft report was still in progress, the feedback of partners was expected to help clarify certain program level concerns of the evaluation team (such as what specific experiences could be considered by partners as learning points), and to validate and enrich field findings which were limited to a handful of projects 2 Access to safe water is defined as 15 liters/capita/day within 1,500 meters of the homestead. 7 setting out the key principles. Strategy development was followed by the development of a national ‘step-by-step’ protocol describing what needed to be done to achieve universal access. Ethiopia’s water supply policies follow best international practices and promote the core principles shown in the text box above. The same holds true for the sanitation sub-sector where the national strategy seeks to promote prevention of sanitation related diseases and low cost approaches. The National Hygiene and Sanitation Protocol empowers local offices of health, water and education to undertake health and sanitation programs in a consistent, integrated and co-ordinated manner. It is part of an overall effort of the government to mainstream hygiene and sanitation into key development programs and ultimately siphon funding through a national hygiene and sanitation program. USAID/E has been a substantial supporter of both emergency and regular development programs in the WASH sector for many years. The mission in Addis Ababa is a highly appreciated development partner within the Government of Ethiopia and the NGOs community because of the reliability and the continuity of its projects and for its substantial financial contributions and technical assistance. In the last five years, USAID/E has supported the WASH sector through its financial & technical assistance. USAID/E assistance through the MWA WASH program also contributes towards the achievement of the Millennium Development Goal (MDG) of halving the proportion of the population with sustainable access to safe drinking water, basic sanitation and hygiene by 2015 as compared to 1990. The MWA, formed in 2002, is a US-based alliance of international NGOs with experience and expertise in rural and peri-urban water supply, hygiene education and promotion of sanitation. Members include CARE, Catholic Relief Services, Emmanuel International Mission, Food for the Hungry, Life Water International, Living Water International (LWI), Water for People, Water Missions International, Water Partners International (WPI) and World Vision (WV). UNICEF is an Advisor to the coalition. In Ethiopia, MWA has been implementing the WASH program under the name of MWP since March 2004. It responds to the need for safe water supply and basic sanitation and hygiene for the rural population through advocacy and direct action to reach one million people. The MWP program objectives are: a) to improve access to sustainable and adequate water; b) to improve access to sanitation services; c) to increase community awareness; and d) to promote safe hygiene practices. Currently the MWP is implementing eight WASH projects in 28 woredas and five regions (Amhara, Benishangul-Gumuz, Oromia, SNNPR and Tigrai) of Ethiopia. The MWA/E WASH program has been implemented through a partnership for advocacy and direct action aiming to reach one million people by 2015. The program started implementing its activities in three regions namely, Amhara, Tigrai and SNNPR through its four program Partners with USAID’s one year matching funds of $836,000. Subsequently there were other modifications for funding from USAID (Mod# 2- Mod 8 #4). The current on-going program under Modification #5 runs from January 2008 – December 2009 and the program also expanded into Oromia region. Under the USAID/E evaluation Contract Number 663-C-00-08-00409-00, USAID/E HAPN Office requested technical assistance from TMG, to design and implement an independent external evaluation of the MWA water and sanitation and hygiene WASH program. The USAID/E HAPN office also requested that the evaluation be completed before December 15, 2008, so that the office can use the evaluation findings, conclusions and recommendations to inform a follow-up water and sanitation program. In accordance with the USAID/TMG evaluation Guidelines, this report is structured as follows: 1. The Executive Summary with a summary on the background, major conclusions, recommendations, and lessons learned; 2. An introduction with information on the purpose, background and methodology used for the evaluation (Chapter 1); 3. Finding and Conclusion (Chapter 2) including progress and achievements toward the stated objectives of impact: social and economic life of the beneficiaries; integration and coordination; sustainability; linkages with other programs; water supply and sanitation engineering design and management; behavior change design, implementation and results; capacity development; and challenges; 4. Lessons Learned (Chapter 3); 5. Prioritized Recommendations (Chapter 4); and, 6. Future Direction (Chapter 5) Annexes: Statement of work (SOW), Annotated List of Documents Collected and Reviewed, Persons Contacted, Data Collection Tools 1.3 Methodology The evaluation team mainly relied on qualitative data to assess impact as well as program performance. These data were collected through review of key program related documents, interviews with key informants and beneficiaries, and observations of program activities in the field. Team members also reviewed and assessed the quantitative data available on program performance from the FY 2004 - 2008 periodic reports of the MWA which contained information on program implementation process and accomplishments. The evaluation was conducted by a team of two professional and independent external consultants over a period of approximately four weeks. MWA assigned the program coordinator to join the team to facilitate the evaluation process. Table 1 - Composition of the Evaluation Team: S/N Name Title 1 Siseraw Dinku Senior water, sanitation and hygiene expert 2 Bruck Waregai (Team Leader) Water, sanitation & hygiene (WASH) specialist 9 The assessment was participatory and mainly relied on qualitative information gathered from partners and other stakeholders through discussions and interviews at the various levels. To the extent possible information collected through these means was substantiated and complemented with assessment of secondary data obtained from various sources, including USAID/E, MWP project offices and government institutions. Methodology of data collection3 included: key informant interviews, focus group discussion, and field observations. Interviewees included: beneficiaries and community leaders, representatives from the Government of Ethiopia, MWA staff, MWA consortium NGOs staff, and USAID Mission staff. Focus group discussions included: extensive discussions with beneficiaries and with members of WASHCOs, and with HEWs and staff of woreda WASH teams and Regional Water and Health Bureaus. Reviewed background documents included: Cooperative Agreement No. 663-A-00-04- 00419-00 and all relevant modifications, MWA Semi-Annual Report submissions, MWA Quarterly Reports, USAID trip reports summarizing past field visits to MWA sites, Ethiopia Water and Sanitation Development Program Documents, Sanitation and Hygiene Policy and Strategy of the GOE, Rural Water Supply Policy, and USAID draft water guidelines. Field visits and observations included: field level meeting with Woreda Health Office (WHO), Woreda Water Office (WWO), woreda Council, Region and woreda specific WASH service providers, households and schools. Fieldwork was conducted in Amhara, Oromia and SNNPRS. In Amhara three projects in three woredas were visited. One project in each of the other regions was covered. In Oromia, an additional project of solar-wind hybrid water supply system was visited for special interest of innovative technology. The woredas and projects visited were selected based on the suggestion of the MWA Secretariat Office and approval by the relevant USAID/E staff. In addition to the practical considerations (e.g. distance from regional capitals and road conditions), the selection was made taking diversity of implementing modalities (by international NGOs and in partnership with local NGOs) as well as diversity of intervention types (new/rehabilitation, rural/urban, small/large facilities) into account. Visited projects also show variation in terms of their implementation status (completed/ongoing). Two of the projects are implemented directly by international NGOs and the others in partnership with local NGOs. The projects in Amhara are directly implemented by CARE and by Food for the Hungry International (FHI) in Farta and North Achefer woredas, respectively. The third project in Bure woreda, on the other hand, is implemented by CRS in collaboration with Water Action. All three projects are new projects consisting of spring development and hand dug well construction in addition to hygiene education, private pit latrine and institutional latrines promotion activities. 3 Please refer to detailed data collection tools – Annexes. 10 The project in SNNPRS is a rehabilitation project implemented in the Bedowache woreda by Catholic Relief Services (CRS) through a partnership with Sodo-Hosana Catholic Secretariat. In addition to hygiene and sanitation (H&S) promotion (including public and private eco-san4 latrines), the project involved borehole rehabilitation and distribution network expansion and WASHCO strengthening interventions. The project serves both rural and peri-urban villages. In Oromia, the visited project is a town/peri-urban WASH project implemented in Ginchi and surrounding areas (Dendi woreda) by Water Partnership through Water Action. This project involved a borehole with motorized water system as well as communal latrine facilities construction. The other project visited by the evaluation team was one being done by the Hope 2020 local NGO: a borehole based solar and wind hybrid model project in west Shoa. The projects in SNNPRS and Oromia are completed, while implementation of the others is still underway. 1.4 Limitations Considerable efforts were made to clarify the SOW in advance to allow the evaluation team to focus on the issues that were most important to USAID/E in both the assessment of the current program and to provide guidance on future project directions. The evaluation was constrained by time. Only nine days were available for fieldwork at the project sites resulting in limiting field visits to only five projects implemented by four partners in five accessible woredas of three regions. The selected projects or partners are therefore not necessarily representative, implying that some of the findings at woreda and kebeles level may not be applicable to all other projects. However, all efforts were made to have evaluation findings, conclusions and recommendations serve for the overall program. As pointed out above, this evaluation is an overall program evaluation, and therefore does not focus on the detailed evaluation of individual projects that make up the program. Accordingly, although each project or implementing NGO appears to have certain particular features that distinguish it from others, they are not treated separately. Mention of individual projects or NGOs is thus made only where it was important to illustrate points of particular relevance. The other limitation of this evaluation relates to the assessment of impacts. Evaluation of long-term impacts was partly difficult because of the fact that implementation of most of the projects is still underway or completed only recently. More importantly, however, the true measure of impacts was hindered because of absence of baseline information about the status of the target populations at the beginning of projects and monitoring information on key indicators over the period of project implementation. Baseline surveys reported to have been completed by individual projects were either not available or compiled at the program level. Narrative progress and end-of-project implementation reports heavily describe accomplishment of planned targets at the level of outputs (in 4 4 These documents are available on request with MWA secretariat 11 terms of number of schemes and latrines built, number of people trained, and access created, with little reflection on actual utilization of facilities). Methodologically, it could be possible to evaluate impacts without baseline information through alternative means (such as by establishing retrospective data on past conditions through available secondary sources), but because of time constraints, the evaluation focused on immediate benefits and perceived changes as witnessed by stakeholders and evaluation Team field observations. In addition to review of project objectives and targets, qualitative analysis by this evaluation thus relied mainly on beneficiaries’ perceptions of the situation before and after project interventions. 12 2. FINDINGS AND CONCLUSION 2.1 Progress and Achievements Toward Stated Objectives a) Improved Access to Safe Drinking Water The interventions supported under this cooperative agreement mainly included technology options of spring development, hand dug wells, shallow wells, borehole construction and rehabilitation of existing water schemes. Springs were capped and protected. Hand-dug wells constructed and fitted with hand pumps for on spot water collection. Drilled shallow and deep wells are provided with water collection chambers, reservoirs, and distribution network and water points/stands as required. Almost all the water supply infrastructures visited meet the quality standards established by the Ethiopian government. At the time of the field visit, all but two visited facilities were productive and functional serving the targeted population within the GOE definition of rural water supply coverage (located within 1,500 meters distance and minimum 15 liters/person/day consumption). Where appropriate, the water supply schemes visited included separate troughs for cattle, clothes washing basins and shower houses. The team observed that the facilities were being used well by the beneficiaries. The beneficiaries told the evaluation team that cattle troughs are used mostly in the dry season when access to alternative sources of water for their animals is limited. They also reported that the showers are used by most community members (mainly by men) at least once a week. Access to the shower service is free for those who had participated during construction, while other users are charged for the service (0.25 or 0.50 Ethiopian Birr (ETB) per person per use). The same rule and price applies to the use of clothes washing facilities. Table 2: Program Accomplishment in Water Supply (2004 to September 30, 2008) Activities No. Accomplished Beneficiaries Water facilities constructed 334 178,284 Water facilities rehabilitated / repaired 77 47,833 Total 411 226,080 13 All water supply facilities were constructed applying low cost and appropriate technologies. In addition, 330 WASHCO’s were established and trained to manage the water supply facilities. b) Improved Access to Improved Sanitation Facilities, Improved Personal and Environmental Hygiene The prime focus of the sanitation component of the MWA program was on the promotion of construction and use of household latrines. As indicated above, several MWA partners included other types of interventions such as construction of clothes washing basins and showers housed in the development of their water points to encourage personal hygienic practices. To a certain extent, projects have also attempted to introduce use of surplus and waste water for irrigation purpose. The evaluation team witnessed that this was particularly true for spring development projects implemented by CARE, FHI and CRS/Water Action in the Amhara region. At the present, all MWA programs provide only technical support for construction of household toilets without hardware subsidies except those constructed by earlier projects for demonstration purposes (e.g., by Water Action in Dendi woreda). In some instances, local construction materials such as sand and gravel were supplied by all the households interested in having the pit latrines while the program supplied reinforcing iron bar and skilled labor. The households dug the pits and constructed the superstructure using local materials. In the early years of the MWA program, some implementing partners provided concrete slabs to the households who were interested in having private pit latrines. According to discussions with MWA/E staff, and reviews of the project progress reports, construction of Ventilated Improved Pit Latrines (VIPs) at schools, health posts, Farmers’ Training Centers (FTCs) and peri-urban areas were carried out while household latrines have been constructed by the community, following the program’s sanitation and hygiene education. Table 3: Program Accomplishment in H&S (2004 to September 2008): Activities No. Accomplished Beneficiaries School toilets constructed (VIP) 64 65,284 Communal latrines constructed (VIP) 8 5,182 Number of TPL latrines constructed (HH + demonstration) 14,613 75,114 Total number of people gained access to sanitation facilities and hygiene education 149,850 14 In Amhara, particularly in CARE, FHI and CRS sites, the evaluation team observed that household toilets are being used effectively. Hand washing facilities are also available at the door step of the toilet. Community members interviewed told the evaluation team that they wash their hands either with soap or ashes after using the toilet. On the contrary, only a few households at the CRS site in SNNPR (Badowacho) have started using their toilets and hand washing facilities, even though the structures have been built and are available at the site. The evaluation has observed this was due to the absence of follow-up by woreda health office and kebele health extension workers (HEW). Some projects have also attempted to provide public latrines for non-villager passers-by (e.g., in Bedowacho) with the aim of creating a defecation free environment. According to the woreda health office staff and the kebele administration, such latrines were built using the Productive Safety Net program resources as part of the kebele Safety Net plans. During the time of the visit, the Team has observed these toilets and the surrounding ground were covered with dry feces. While the idea is commendable, the kebeles’s good intention of having such facilities did not, however, take into account a system to ensure their routine upkeep and maintaining the latrines clean for sustainable use. The evaluation team observed that ventilated improved pit (VIP) latrines built at rural primary schools by the program are being used properly. They are clean, provided with hand washing facilities, and well managed by the schools’ Sanitation Clubs or are assigned to classes to manage under the close supervision of designated teachers. However, in one primary school at CARE site in Amhara (Bure woreda), there is a situation to be addressed immediately. The project built only one new block for girls and left the boys to use an old latrine which was in disrepair. The boys’ latrine has no responsible caretaker, no hand washing facility, and old human excreta were observed scattered in and around the latrine rooms. This particular situation implies the focus of the project is more on providing new facilities than changing hygiene practices at a wider level. Similarly, in the case of VIP latrines built for health posts and FTCs, their utilization and management responsibilities are generally entrusted to HEWs and development agents (DAs), who are also the principal users Communal VIP latrines built for poor households (in Dendi woreda/ Ginchi town), with the project providing industrial materials and users contributing local materials and labor, also appear well constructed, intact and well used. Yet there are still ownership and management concerns. The latrines are allocated to groups of user households for their exclusive use and routine maintenance. The groups have informal by-laws and one elected person is responsible to ensure the use and maintenance happen according to the by-laws. Nevertheless, there is a concern that users are in most cases tenants with no rights to the land on which the latrines are built. This entails problems of sense of ownership of the facilities and long-term management responsibility by present users. In addition, a system by which users contribute money or pay fees for the service has not been introduced to cover future costs of major repair and maintenance such as “desludging”. 15 The program’s hygiene promotion and health education component has sets of participatory tools and materials (such as Participatory Hygiene and Sanitation Transformation (PHAST)) developed and distributed. However, there has been no in￾depth analysis of what combination of communication approaches really work in different Ethiopian contexts, with the exception of the government initiative in behavior change negotiations at the household level as part of the Health Services Extension Program. 2.2 Impacts The projects have generally brought about positive impacts on the lives of participating communities. About 226,080 people have gained access to safe water they did not have before. This figure is based on data contained in project completion documents submitted by implementing NGOs. The distance saved for water carried, especially for women, is considerable and varies significantly by type of water service. The largest gains accrue for those who use shared pipe connected tap stands or shared point sources such as hand pump wells built close to villages, followed by those using spot springs. Time savings also varies for type of water service. Time spent collecting water decreased where reductions in distance walked are made possible. According to beneficiaries statement, water collection time in project areas has been significantly reduced for all types of service compared to previous conditions, reduction of waiting time appears more significant for springs as a result of increased yield and night storage facilities. Human Story On the other hand, although users reported significant queuing time reduction, in the borehole based motorized scheme (electrical pump with diesel generator) rehabilitated by CRS (in Bedowacho), people still have to line up their jerry cans for hours long before service hours. Generally, beneficiaries’ testimony revealed that the program have produced significant benefits, particularly for women, who often bear primary responsibility for meeting family water needs. Implementing program staff and community members also perceive that distance and time reductions has given women and girls more time to spend on economic activities, education, and other beneficial activities. ”I am at school regularly because of the private toilets we girls have, the boys do not come to our side of the toilets and we are not afraid to use our toilets any time of the day” (Dasash Bililgn, 13 years old, grade: 5 student of Danbola 1st cycle (1-8) School in Amhara) 16 Although hard data is lacking on water consumption, it is also assessed that increased access has produced other effects such as increases in daily consumption. In some of the sites visited, daily water consumption of households, however, seems still lower than accepted levels, but has shown recent improvements. In all the places visited by the evaluation, there were reports of improved health, although concurrent water activities by other programs and parallel sanitation and health education make it impossible to attribute all benefits to the MWA program. User households particularly reported experiences of reductions in the incidence of diarrhea among children and intestinal parasites among adults. The evaluation team’s field observations and beneficiary interviews revealed that access to safe and more convenient water supply has certainly improved the quality of life of rural women. Easier access to safe water allows women to provide a cleaner home environment with less effort. According to women beneficiaries interviewed by the evaluation team, it is now easier and safer to wash their children. They wash their own and their children’s hands more often. Similarly, clothes washing is simplified and less time consuming, especially in areas where washing basins are provided close to water points. The program also increased access to sanitation through construction of Households (HH) latrines and school toilets. As acknowledged by implementing partners, in most of the projects the progress of sanitation facility construction and use has been much slower than water system development. Therefore, the coverage in sanitation facilities and/or use remained lower. Sanitation coverage figures generally focus more on the number of latrines built than on their proper utilization. The main benefit of the hygiene and sanitation efforts has been the introduction of the practice of latrine construction and use. In schools, the provision of VIPs has significantly contributed to environmental cleanliness. School girls have particularly enjoyed privacy in using the latrines. Similar benefits are accrued to VIPs in poor urban/peri-urban areas (in Ginchi, Dendi woreda). Access to improved sanitation has effects beyond reducing the health impacts of open defecation. In rural villages, according to customs, women can attend to their bodily functions only during the hours of darkness as it is not seen as proper for them to be seen doing so in daylight. Therefore according to women informants, having a private, convenient, and clean place to attend to personal hygiene is particularly beneficial to them. 2.3 Coordination and Integration Sector coordination has been poor in the past but the Memorandum of Understanding (MOU) signed by the Ministries of Health, Water, and Education has provided the foundation for implementation of the National WASH Program in an integrated and coordinated manner. This is a very important milestone and now expected to vastly improve water and sanitation programs, provided the MOU is put into effect and cascaded to lower levels. A multi-stakeholder approach has taken root in the sector and sector partners see its advantages as creating a forum for dialogue to discuss sector 17 specific issues, and to use the platform for experience sharing and learning across sectors players. Within the context of the MWA/E program, woreda Line Departments (LD), mainly Health and Water Offices, have been actively involved in project implementation. They particularly have been engaged in supervision of construction activities and training of field workers (HEWs, DAs, etc.) and WASHCOs. In in four of the five woredas visited project steering committees have been formed to oversee project progress and coordinate activities at the woreda level. As explained above, at the kebele level, HEWs have been actively involved in sanitation and hygiene education activities, which are included as part of five of their health extension packages of activities. Together with DAs, they also have benefited from training activities and support in terms of material provision. In the cases of two of the projects visited (Dendi and Bedowacho woredas), the regional governments of Oromia and SNPRRS have made substantial contributions to investment costs through provision of generator sets and pipes. While the involvement of water and health offices generally appears encouraging, participation of education offices in project activities has been minimal, limited to the involvement of rural schools in taking over and managing VIP latrines. There has, therefore, been poor integration of other needed WASH activities in their annual plans. 2.4 Sustainability In all projects visited, participation of communities in project implementation is observed to be very high. Communities have particularly contributed in construction activities through provision of in-kind (labor and material) contributions and in Dendi, cash.). Although community participation should encompass other areas (planning, monitoring, etc.), this is expected to help to develop a sense of post-implementation ownership and management responsibility. In all the sites visited, WASHCOs (composed of 5-7 members, including 2-3 women) have been formed and trained under the project and have taken over responsibility for the future management and O&M of facilities. Teams of caretakers have also been established, trained, and in most areas, provided with tool kits (except in FHI’s project in Achefer). In larger water supply facilities of boreholes with electric pumps and generators (e.g., Bedowocho), motor operators have also been trained and provided with basic tools. In such facilities, in addition to operators, fee collectors have been employed by the committees to administer day-to-day services. In Ginchi town (Dendi), the water 18 system is being managed by the town’s water office (governed by a board), and individual water points are contracted out to private operators, working for profit (buying water at ETB 2.50/m3 , and selling at ETB 0.20/20 litre). Training of WASHCOs include topics on hygiene and sanitation education in addition to water scheme O&M, water management, financial management, and water source protection and multiple uses of water. According to project staff, their responsibilities for hygiene and sanitation activities include motivating community towards utilization of improved water supply, improved sanitation facilities, good hygiene practice, and undertaking hygiene and sanitation promotion. Nevertheless, the evaluation team observed that while WASHCOs responsibilities for the management of water services is clear and well understood by members, their responsibilities for H&S works do not seem well defined and understood. Interviews with WASHCO and general community members during field visits revealed that WASHCOs involvement in H&S promotion has never been insignificant. The WASHCOs collect user fees and have bank accounts to save the money for future O&M requirements. Caretakers undertake minor repairs, such as hand pump maintenance. However, the committees are receiving continuing support from the local government line offices and the zonal and regional technical government staff for major repairs such as borehole maintenance. Availability and accessibility of hand pump spare parts as well as technical services from higher levels for major repairs is a key concern in all areas visited. In this regard, CARE’s project in Farta is attempting to involve private service providers by training local artisans and assisting them to organize themselves into cooperatives to provide such services to rural communities for a fee. A user fee is introduced in all rural sites (monthly contributions of 1-2 birr/hh/month). Funds are meant to cover O&M costs (such as guard’s salary and future spare part costs). In larger schemes, user fees are set based on amount of water collected (commonly 0.20/0.25 Birr/20 liters jerry can). The user fees and tariffs in all areas are set randomly by the committees without taking actual O&M requirements into account. In all the regions WASHCOs are not formally registered entities. However, WASHCOs have clear linkage with woreda water offices, from which they solicit technical assistances. In all visited areas water office staff also conduct monitoring visits to communities to check proper functioning of the facilities and the performance of committees. In areas where there are larger schemes, WASHCOs are also required to provide periodic reports to the water offices especially on their financial status. Due to their poor capacity in terms of human resource and logistics, however, woreda water offices are not able to effectively follow up a wide network of service facilities and provide effective technical services to rural communities. The line of accountability of WASHCOs, especially to their constituency, also appears not well established in any of to the areas visited. For instance, there is no established system by which WASHCOs report to their respective communities on the amount of money collected, spent or deposited in the bank account. It should also be stated that WASHCOs are not formally 19 linked with woreda health offices and they are not formally accountable to health offices or to the communities for their responsibilities in hygiene and sanitation activities. Construction and maintenance of private latrines in all areas visited are the responsibility of user households, and except in SNNPRS, in all visited villages, latrines are observed to have been properly used and maintained. In all areas visited, household pit latrines are technically simple and built from local materials (wood, mud, straw) using family labor and are found to be safe for all users, including children. School VIPs in Amhara are being managed by the schools’ communities or administrations. Latrines are separated for male and female students and teachers, and compartments within the VIPS are assigned to specific classes for use as well as routine cleaning. The management of public latrines (in Bedowocho) and communal VIP latrines (in Ginchi/Dendi) is a critical concern to which the projects or partner woreda institutions have made little or no effort to change the present situation. As noted above, no system has been put in place to upkeep (to clean and maintain) the public and communal latrines, and to cover future maintenance costs of both the public and communal VIPs. In areas where hygiene education activities were implemented through project staff and not linked with existing health services, the continuation of hygiene education efforts remains a crucial concern. The effort being made by CARE (Farta woreda, Gondar) in involving the private sector is commendable to address some of the bottlenecks of sustainability of rural water and sanitation services. Specifically, according to the project staff, it is attempting to address the problems of availability of local contractors, availability of technical services for rural communities in major maintenance of water supply facilities, and availability of spare parts through training and organizing of local technical service provides, called artisans. So far CARE has trained 25 local artisans who were residents in the villages, interested in taking up the job of serving as technical service providers, and had basic experience in masonry and plumbing works to cope with the shortage of local contractors to construct water facilities. According to the same source, the artisans have been trained on water facilities construction for two months, which encompasses essential technical skills development as well as practical exercises. The artisans are expected to fill in the gaps of government capacities in providing support for rural communities and WASHCOs especially in cases of major repair works which are often beyond the capacity of village caretakers and technicians against reasonable honorarium. Following the training program, CARE contracted construction works to the trained people at internally established fixed work norms to ensure their proper operation in the market and equipped them with basic tool kits. To further ensure their sustained commitment and organizational strength, CARE devised a strategy to organize the artisans into cooperatives. Apart from the provision of technical services, the groups are expected to engage in the supply of spare parts. In this regard, CARE has initiated discussions with concerned offices (such as the office of cooperatives development) to have the groups registered as cooperative societies and obtain licence to involve in business activities. 20 The approach of involving private technical service providers (TSPs) is also being used/promoted by the World Bank financed GOE’s WASH program which is being implemented in several regions of the country. It was initiated based on best practices in the sector to address the above mentioned gaps, and, apart from stimulating private sector involvement, it is believed to contribute in bringing technical services closer to rural communities. The experience of this program and CARE’s project offers good lessons, which other member NGOs could profitably employ. Yet, there are issues that need to be addressed in order to ensure TSPs linked with rural communities and woreda offices function independently and responsibly, and operate profitably in the open market when the coaching and support of projects are withdrawn. Moreover, the role of TSPs in hygiene and sanitation activities, such as in supporting the construction of latrines, is not clearly defined. 2.5 Linkage With Other Programs In all woredas visited, there are other ongoing WASH activities being implemented by the government and other NGOs. However, integration and coordination of MWA programs with these activities was not evident. In this regard, although the task of coordinating activities principally lies within the mandate of the woreda administration, implementing NGOs of the program did not play a proactive role in linking with others to facilitate joint learning, review and experience sharing. Two of the projects (in Achefer and Bure, Amhara) visited by the evaluation team are implemented in woredas where there were earlier USAID financed projects. These projects were reported by relevant project staff to have been built on health and nutrition projects (of ESHE, Pathfinder) in ‘champion’ kebeles. Accordingly, it was noted that the WASH projects have filled in the missing element (i.e., WASH) that was identified by earlier interventions. Nevertheless, understanding of the need for linkages between the different programs appears to be low at both woreda and project levels. Generally, collaborative approaches within the MWA/E have been minimal; the only effort so far being the establishment of a common M&E framework which is still not fully implemented. In all other aspects, the different NGOs follow different approaches to water supply and sanitation development as well as hygiene education. The same is true with other programs implemented in the regions and woredas where the MWA Program operates; each following different approaches in most aspects with a bearing on standardization and sustainability. For instance, the Government WASH program adopts a demand-responsive approach through which communities identify needs and articulate demands as a community. Following acceptance, they are incorporated into a woreda WASH Plan. The program requires communities to make 10% contribution to capital costs (3% cash and 7% in-kind in the Amhara region, and 5% cash and 5% in-kind in SNNP and Oromia regions) and 5% matching funds for capital investment by woreda governments. The program is managed by woreda WASH Team (WWT), and relies on communities in deciding on service levels and in the management of contracts. A similar 21 capital cost sharing arrangement is also used by FINNIDA in the Amhara region. The rural MWA projects on the other hand do not have such requirements and community participation is heavily associated with in-kind contributions for physical activities. Differences in approach are also observed in involving the private sector and in promoting hygiene and sanitation. Similarly, the Ministry of Health (MOH) has, with support of WSP-AF, developed the strategy and implementation framework for hygiene and sanitation in the country based on extensive consultations with a large group of stakeholders including the regions and selected woredas. The “Program to Implement the National Hygiene and Sanitation Strategy through Learning by Doing in Amhara” is jointly implemented by the Regional Bureau of Health, MOH, WSP-AF and the USAID Hygiene Improvement Project (HIP). This program has helped present a practical example for improving stakeholder coordination and collaboration and has developed important resource material on partner and resource mapping, designing strategic behavior change interventions and related training content, development of coordinated action plans. It has also helped develop a participatory process of popularizing the National Hygiene and Sanitation Strategy and protocol throughout the region for immediate replication in other regions. UNICEF’s incremental introduction of an effective water quality monitoring, sanitary surveillance, home-based water treatment, and storage systems can be cost effective, high impact measures, and should be examined further. It is well acknowledged that harmonization and coordination is generally a key problem of the WASH sector and despite some efforts in certain aspects (such as the MOU), efforts in this regard have so far been minimal. NGOs nevertheless could be proactively engaged in facilitating this process. MWA partner NGOs particularly need to initiate and strengthen efforts firstly within themselves and jointly advocate for harmonized and coordinated efforts in their areas of operation to enhance effective delivery of services and bring about sustained impacts. In this context, the MWA needs to link with initiatives such as mentioned above and with others. Nonetheless, this suggestion should not be taken to mean closing opportunities for drawing best practices from other places and piloting innovative approaches. 2.6 Engineering Design and Management All six visited water projects of MWA WASH program are designed and constructed up to the national standard of rural water supply. This requires facilities to be of the Village Level Operation and Maintenance (VLOM) type. Springs were capped and protected accordingly. National standards have also been met for hand-dug wells which have been constructed and fitted with appropriate hand pumps for on spot water collection. Drilled shallow and deep wells are serviced with water collection chambers, reservoirs, distribution network, and water points as required. All visited water supply systems’ infrastructure meets the acceptable government standards and exhibit good 22 workmanship. All visited schemes except one in SNNPRS were functioning, fenced, and well maintained. WASHCOs are established and trained to manage the projects. Construction of VIPs in all visited areas have uniform design and used the same types of construction materials (except in Achefer where wood and mud were used instead of corrugated iron sheet (CIS) to reduce cost). The types of private pit latrines promoted by the projects are the same in all areas, except in the SNNPRS where CRS attempted to introduce Arborloos. In all visited areas, except in SNNPRS, latrines are well built from local materials with super structure and hand washing facilities hung outside the latrines (small jerry cans or plastic bottles with straw). Ash or soap was also available at the door steps of latrines in all areas. The evaluation team also visited an innovative water system introduced by Hope 2020, which is not under the MWA program but partially funded by USAID. The project, ‘Solar and wind hybrid powered water supply model’ is located in Senqele na Faris kebele close to Ambo town (West Ahoa). It is implemented by Hope 2020 in collaboration with the University of the District of Columbia (UDC) for the purpose of further research and community (2,000 people) service. Both solar energy and wind mill are used to draw water from a 72 meterss deep well and pump it to a reservoir which has in-built water distribution point. Primary energy source During the time of this evaluation field visit, the system was not functional even though it was completed recently in July 2008. At the time of the visit, there was no WASHCO established, and no trained personnel have assumed the O&M responsibility of the water supply system. Secondary energy source Although the project is not part of the MWA/E program, it was assessed by the evaluation team in view of the importance of promoting innovative technologies such as this particular one. This was done in response to the comment by one partner (Hope 2020) on the need for the MWA to facilitate learning and experience sharing among organizations working in the water, sanitation and hygiene sector. 23 The technology appears appropriate for evaluative learning and subsequent promotion in the MWA program future interventions. However, it would require establishing a management system and developing the local capacity for its O&M to demonstrate the appropriateness of the technology given the country’s abundant year round sunlight to generate solar energy. Efforts also have to be made to integrate hygiene and sanitation activities with water supply. 2.7 Behavior Change Design, Implementation and Result a) Purpose of Hygiene Promotion within the Context of Program Objectives Hygiene education is an important component of all the projects, aiming at promoting conditions and practices that help to improve the health status of target populations by preventing water and sanitation related diseases. Hygiene promotion activities of the projects were meant to maximize the potential benefits of improved water and sanitation facilities. Although not explicitly articulated in the program documents, the activities also helped users appreciate the need for the proper O&M of facilities and create a willingness to contribute to their costs. Qualitative evidence from evaluation team field visits provide a positive correlation between awareness of the health and other social benefits of improved facilities and a commitment to their proper upkeep, expressed through payment of fees for water services and routine maintenance of latrines by user households and institutions. All projects of the MWA/E program identified that water and sanitation related diseases are prevalent in their areas of operation. According to contacted project staff, this prevalence of basic health problems was further substantiated later through their baseline surveys, topical studies and experiences during project implementation5 . A main objective of the projects, consequently, aimed at reducing water and sanitation diseases. This included a decrease in incidence of diarrheal episodes, intestinal parasites, etc., and an improvement in the living conditions (reduced burden of water collection, time and energy gains for other activities, use of surplus water for economic activities, improved environmental cleanliness, provision of facilities for washing and bathing, provision of private places to relieve oneself, etc.) of targeted communities through the provision of improved water supply and sanitation services. To meet these objectives, it was well recognized in project design that construction of improved water supply and sanitation facilities is only an entry point, but not a sufficient condition, either to ensure their continuous utilization in a safe way, or to improve health. Integration of hygiene education into those activities was therefore designed to be instrumental in establishing the link between improved facilities and user behaviors/practices. Accordingly, construction and rehabilitation of water and sanitation facilities were accompanied by, and coordinated with, hygiene education which is concerned with promoting hygienic behavior, with a focus on safe collection and handling of water; construction, use and 5 According to MWA secretariat office, findings of the base line surveys and studies are not available yet but will be finalized soon for reference purposes. 24 cleaning of latrines; body and clothes washing; hand washing at critical times using soap or substitutes; and maintaining household/domestic hygiene. b) Development of Hygiene Promotion Efforts The objectives of the program as well as specific projects were formulated based on global and national assessments of health problems that deserve attention and could be tackled through water supply and sanitation interventions. Projects had accordingly identified specific behaviors that are directly linked to water and sanitation related diseases. These made up the focus of the hygiene education interventions. Review of documents however shows that the behaviors were not identified in specific terms and prioritized for intervention based on their importance and changeability. As a result, the projects included a list of behaviors including practices of open defecation to breast feeding to be addressed through hygiene education efforts. For instance, a recent program document (MWA Modification # 5, 2007) states that ‘education concerning the prevention of diarrhea and other water-related illnesses will focus on hygienic preparation of foods, hand washing, promotion of exclusive breastfeeding, handling and storage of water, and the use of improved water sources and sanitation/waste disposal facilities’. The relevance of addressing these issues to reduce/prevent the identified health hazards is not questionable. However, given the short timeframe and limited resources at the disposal of projects, the appropriateness of attempting to address all these behaviors without raising unrealistic expectations and running the risk of losing focus was raised by the evaluation team. Immediate behavioral objectives were, in all cases, formulated as part of the broader project objectives, and have been followed during project implementation. This was done, however, without the changes to revise and make them more relevant to the specific conditions identified later at the time of planning and reporting of baseline studies. In practice, projects all appeared to have focused on specific behaviors and fairly well answered questions, such as the number of people expected to achieve a certain level of behavior by the end of the projects. Specifically, it appears that objectives related to the promotion of the construction and use of latrines and promotion of hand washing after defecation were much clearer than others. Most other behaviors relating to food, personal and domestic hygiene were less addressed targeted hygiene education interventions. Perhaps this approach, i.e., detailed planning of hygiene education may not be expected at the time of project preparation when resources were yet to be made available. Yet, this could have been done later (following preparation of modification plans) to better inform the design and implementation of effective behavior change communication strategies. One of the strengths of the projects in addressing water and sanitation problems lies in the integration of technological (provision of facilities) and environmental interventions (measures in improving water quality) with hygiene education, by which both behavioral and non-behavioral causes of health problems are simultaneously attacked to increase the potential for full benefits. Yet, what is lacking throughout the program period is a well developed hygiene education (IEC/BCC) plan to organize and direct actions on the ground. 25 Behavior change efforts of the projects almost uniformly focused on addressing a set of predisposing and enabling factors that contribute to health behaviors. Increasing community awareness and changing attitudes and perceptions through educational activities was a key strategy used to stimulate adaptations in behavior. Provision of water and institutional sanitation facilities and promotion of household latrines together with training activities were also intended to enable and motivate people to adapt improved practices. Some of the projects (e.g., Water Action in Dendi) also included the promotion of latrine use through the provision of technical and material support to model households to build and use latrines. The hygiene education strategies of the projects certainly lacked some basic elements including formulation of specific objectives for each behavior desired to be influenced, development of key messages and appropriate channels of communication, and means of monitoring and evaluation of changes. Nevertheless, it was evident from project documents and the evaluation team’s field visits that project plans included a mix of methods to promote changes in behavior. The methods used vary from project to project, but generally included mass education (lecture), community meetings, group discussions, visual aids, demonstration, household visits, and skill development. All these approaches and efforts seem to have been developed from the experiences of the implementing NGOs and the government health extension program, rather than based on a program supported critical assessment of existing hygiene education activities and experience, and development of studies into local practices, needs and conditions. c) Implementation of Hygiene Promotion Projects followed a variety of organizational structures/approaches for the integration of hygiene education activities. In this regard variations are observed especially in the approaches followed for community level activities. The four types of structures/approaches adopted by the projects were: i. Use of a team of salaried hygiene promoters to conduct the hygiene education component at the field level under supervision of an official appointed by the project or seconded by the health office at the woreda level, ii. Use of permanent project staff to coordinate activities at project level, while woreda health offices handle activities through their woreda experts and village based staff (HEWs), iii. Use of woreda health offices and their health structures at community level to manage the hygiene education component of the projects, and iv. Use of project staff to coordinate activities at woreda level and woreda health offices conduct training activities, while voluntary hygiene communicators are recruited for educational activities at community level in coordination with HEWs. Most of the projects used a combination of the different approaches in undertaking hygiene education. Commonly, projects had senior staff with a social/health background with responsibilities for the development and coordination of hygiene education. The different organizational arrangements had their own strengths and limitations. The use of 26 an organization’s own paid staff for all hygiene education activities at all levels had the advantage of easier project management and implementation efficiency. However, such an arrangement had limitations in developing good communication and rapport between hygiene educators and community groups, which is essential for effective hygiene education. It also had limitations in ensuring the continuity and persistence of efforts that are needed to reinforce changes over time, especially after the project’s end. In view of the importance of sustained hygiene education and expanded outreach, therefore, projects that relied on the use of existing health services and community level promoters (volunteers) were in a better position than others. Coordination of activities with other/ongoing health education activities, such as the government’s health extension program, also had multiple advantages in the economic use of available resources as well as avoiding confusion of the messages to community groups. As it is commonly done by other interventions such as World Bank (WB) and UNICEF supported programs, use of designated staff to liaise and coordinate with woreda health offices helped integrate projects activities with those organized through the regular government health program. While the training of woreda and other health staff (including HEWs and DAs) on new approaches (such as PHAST and Community Led Total Sanitation (CLTS)) were included in MWA program activities, the actual implementation of hygiene education activities at the community level for most of the implementing NGOs involved training of WASHCOs on hygiene and sanitation issues and organization of community hygiene education sessions. As stated above, activities also included house-to-house visits and demonstrations. IEC related activities implemented by some of the projects (e.g., Water Action in Bure and Dendi) were mainly concerned with the distribution of flip charts. Activities at schools and other public institutions appeared to have focused on provision of new sanitation facilities (typically VIP latrines) leaving out essential accompanying BCC activities that would help with the adoption of better hygiene practices by targeted users (e.g., school children). Although NGOs stated they included hygiene activities in their program documents, the evaluation team field visits found that the involvement of communities in developing hygiene promotion plans and in implementing and monitoring them was minimal. d) Results of Hygiene Promotion Efforts Project progress and completion reports mainly focused on reporting accomplishments of planned targets in terms of the number of facilities built, number of people trained, number of people educated, etc.. Unfortunately, such a monitoring system allows little potential to assess outcomes and impacts of hygiene promotion activities in terms of behavior change and health improvement. Health promotion should be one of the crucial aspects of project planning and implementation in the future, and the evaluation team found in its field observations that the projects have registered encouraging results in influencing people’s behavior. It must be noted, however, that since there were, in addition to the MWA project activities, regular health education activities and other WASH projects implemented in all project areas, all results cannot entirely be attributed to the MWA projects. 27 At the level of outputs the projects have succeeded in producing the following results: ƒ Training of 2,740 WASHCO members ƒ Training of 609 community hygiene educators ƒ Training of 185 HEWs ƒ Training of 28 of school sanitation clubs ƒ Educating 150,000 people The above figures are directly taken from the data given by the Program Coordination Office. It was reported by individual projects that trainings have also been given to woreda water and health experts, DAs, school teachers and community leaders. However, data was not available on these activities at the program level. During field visits, the evaluation team also found that encouraging achievements have been made at higher levels as a result of project interventions. Notable results of hygiene education activities observed in most of the visited sites include: ƒ Functioning of all completed water supply facilities and implementation of user fees; ƒ Regular use of improved water sources for drinking, clothes washing and bathing, ƒ Protection of water sources (fencing); ƒ Use of private latrines by all family members, and regular cleaning and maintenance of latrines; ƒ Proper use and maintenance of school and communal latrines; ƒ Availability of hand washing facilities and practice of hand washing after defecation with soap or ash; ƒ Safe disposal of feces from young children and infants; and, ƒ Use of clothes washing and shower facilities at water points (especially by men). In spite of repeated mentions in project documents and reports regarding interventions to influence behaviors in increased water consumption, safe water collection, transportation and storage, and domestic hygiene (including food handling), there was little evidence of change found by the evaluation team. Data from the monitoring system on changes in this regard are also lacking. 2.8 Capacity Development While coordination and collaboration of projects with woreda line departments is generally encouraging, as discussed above, capacity building of relevant woreda offices (particularly water, health and education) is perhaps found to be an important missing element in the program. Where capacity building was provided by the MWA program/projects, it was mainly concerned only with provision of some training of trainers (TOT), training (in PHAST and CLTS) and provision of tool kits to water offices. There is a serious lack of involvement of the Education Office at woreda level in MWA project activities, although schools are often targeted for hygiene and sanitation promotion works. While the project activities including construction of new water and sanitation facilities are well recognized in putting additional strains in the already 28 overstretched capacity of woredas, the failure to address woreda capacity building as a crucial component of the MWA projects is one of the shortcomings of the projects. GOE sectoral capacity is especially weak at woreda (district) level, with a subsequent ‘down stream’ impact on capacity at kebele and village level. In addition to limitations in technical capacity and mobility, poor recognition, low morale, frequent staff turn-over and vacant posts affect the capacity to implement water and sanitation programs. In terms of scope, the capacity gap impacts on coordination, planning, implementation (most notably related to decentralized decision making and hygiene promotion, but also affects to the technology selection), longer-term support to O&M, and monitoring and evaluation. The virtual absence of the private sector to date indicates that there may be considerable potential to do more, if critical issues can be identified and addressed in future projects. For programs such as of MWA/E, the woreda is the key institution for service delivery and it is essential to ensure that the program implementation strategy enables woreda capacity to be enhanced directly from the program intervention. The capacity of the Program’s Coordination Office also deserves some attention. The Secretariat Office of the MWA in Ethiopia is currently staffed with a coordinator, finance analyst and M&E expert. However, it appears that the office is too heavily involved in routine administrative work to play a key role in effective coordination and facilitation. It was stated by partners that coordination of the 12 different organizations (each having a different level of capacities and their own institutional culture, including monitoring of projects and timely undertaking of jointly agreed activities), has proved to be beyond the capacity of a single coordinator who also has to liaise with other partners, compile reports, organize meetings, etc. Improvements have been seen by partners in program coordination since 2004. Commendable efforts are being made through the quarterly Program Management Group (PGM) meetings in setting out new agenda, in developing M&E systems to be implemented across partners, and in jointly initiating research activities. Yet all partners agreed that much remains to be done to strengthen coordination. In addition, the MWA program in its current state has limited staff and capacity to facilitate other needed elements including development and implementation of harmonized approaches among partners; initiation and implementation of innovative methods; effective engagement of partners in learning and experience sharing forums; documentation of program experiences; provision of technical assistance and capacity building support to partners; and creation of linkages with other WASH actors and programs. In the present context, it appears that instead of performing as an integrated program, the NGOs are performing as groups of projects. Opportunities for identifying and addressing advocacy agendas to the sectors, for learning and experience sharing within the program and outside, for documenting and disseminating best practices, etc., have been limited to activities at scheduled meetings. The role of the MWA in these areas thus remains limited 29 and potential areas of adding value by working as an alliance could be much better explored. 2.9 Challenges Encountered The main challenges faced by the Program since its commencement is described as follows: ƒ The new NGO legislation: The new legislation on charities and associations is expected to redefine the operational context and landscape for NGOs, and has created many uncertainties in the future status of NGOs especially those of National NGOs. However ,it is still too soon to know for certain the implementation modality and the effect of this legislation ƒ Short duration of projects: The short period of planning and project implementation (1-2 years) has been stated by implementing agencies as a critical challenge especially for realization of objectives relating to behavior changes and establishment of strong community level service management structures ƒ Escalation of material and labor costs: The unprecedented increase of construction materials and labor coupled with the lack of construction materials around project implementation areas has had a significant impact on program implementation. In this regard MWA has set up a task force to assess the trend of cost inflation and its effects on program activities and attempted to bring the issue to the attention of donors for possible additional funding. ƒ Woreda capacity and staff turnover: The overall poor capacity of partner woredas in terms of skilled human resources, budget and logistics have remained a critical challenge for effective implementation of projects and a constant threat for their sustainability. This problem is further compounded by frequent turnover of woreda officials and experts making smooth coordination of activities with partner NGOs all the more difficult. 2.10 Efficiency of Resource Utilization Assessment of budget allocation by major components of the program shows that the lion share (71%) went to water development, while about 29% went to sanitation and hygiene education activities. Obviously, water supply activities involve much higher investment costs than sanitation and hygiene works as they involve costs for drilling, pump purchase and installation, study and design, etc.. On the other hand, costs of private pit latrine construction are much lower as local construction materials and family labor are used. Similarly, hygiene education activities involve minimum cost since they are carried out as part of routine community level works by project staff and volunteers or village-based extension workers. This makes the percentage of hygiene and sanitation cost much lower as compared to water. The evaluation team observed that MWA’s 70-30 proportion of expenditure to water supply and sanitation & hygiene respectively was consistent with that of government and other major sector player budget allocation. 30 Although budget and expenditures are reported in line with the initial budget description, reports do not give breakdowns by specific activities such as water, sanitation, hygiene education, or training. Understandably, as training activities for WASHCOs and partner staffs involve topics on each component at the same time, it may be difficult to split expenses by activities. Nevertheless, this could be done if some effort is put in calculating costs to provide estimates of fund allocation. In addition, community in-kind contributions are not well recorded and accounted for by projects. This also makes it difficult to assess the level of community involvement in capital cost sharing. Analysis of financial resource utilization was therefore made mainly at the program level. The following table shows program budget and expenditure at the aggregate level. The total financial expenditure as of September 30, 2008 for USAID and Conrad N. Hilton Foundation (CNHF) grant, respectively, is summarized in table above. The remaining balance of the USAID grant will be spent fully by the end of December 2009 and that of CNHF grant will be spent through June 2010. Table 4 – Budget and Expenditure Summary Donor Total Obligated Budget + Cost Share Obligated Grant Budget Cost Share Grant Expenditure Cost Share Expenditure Total Expenditure USAID 6,583,525 4,371,446 2,212,079 3,052,773.30 1,853,933.09 4,906,706.38 CNHF 12,342,179 7,232,322 5,109,857 3,108,693.76 2,466,149.88 5,574,843.64 Total 18,925,704 11,603,768 7,321,936 6,161,467.06 4,320,082.97 10,481,550.02 Utilization of the budget by the program has generally been slow throughout the period covered by this evaluation. This resulted in no-cost extensions of the project implementation period. In spite of the fact that, overall, the organizations have not been able to cope with the expected rate of expenditure (burn rate), they have been faced with several practical challenges that have been affecting their pace of implementation and financial utilization. These include delays in the signing of project agreements with regional government authorities, the time required to setup projects and negotiate with local governmental partners, the extended process of procurements of services and goods, and the time needed to initiate, organize and conduct hygiene and sanitation promotion works. In recent months, the situation has improved considerably and burn rate seems to no longer be an important issue. 31 3. CONCLUSIONS Since the start of the program in 2004 to September 2008, 411 new water supply schemes were constructed and existing schemes were rehabilitated. As a result, about 226,080 people have had access to safe water supply. Over 75,000 people gained access to sanitation facilities and equal number of people received hygiene education. In 80% of the visited sites, WASHCOs were organized and trained under the project and have taken over responsibility for their operation and maintenance. Most committees do collect user fees and have bank accounts to save the money for future operation and maintenance costs. The committees were trained and equipped to carry out bookkeeping and scheme administration functions, while teams of caretakers have been established to undertake minor repairs. Committees are receiving continuing support from woreda water offices and the zonal and regional technical government staff for major repairs such as borehole maintenance. The members of the communities that were visited by the evaluation team were very pleased with the new and rehabilitated water facilities and were making good use of them. Both community members and partner staff reported that the availability of safe drinking water alone has already had tangible benefits and impact on people’s health, especially among the children who are now much less likely to suffer from diarrhea. There is a perception among women that they benefited most from the water supply interventions both because of the reduction in the amount of time they now have to spend collecting water and because of the improvements in the health of their children. Since the beginning of the program in 2004, there has been a general shift in the MWA program approach with regards to addressing sanitation and hygiene challenges. From subsiding latrine construction more recent efforts have moved towards intensive hygiene promotion and community mobilisation. These activities are time consuming and will need additional technical and financial support to ensure that the full benefits of these interventions can be realized and that the activities have an opportunity to bring the required sustainable behavioral changes. The planning horizon of projects lasted only one to two years and therefore did not allow enough time for effective hygiene and sanitation promotion activities. Specifically, projects did not have sufficient time to change people’s harmful hygiene and sanitation behaviors and improved water management practices. A two year period might have been enough time to get a lot of construction successfully built but not enough to cause behavior that would be internalized in the population. This was true whether the program’s objective was to change people’s behaviors with respect to personal and environmental hygiene and sanitation practices or improved water management practices. The program period was also limited in the time to get communities and community groups (on their own) to the point where they have the capacity to manage the facilities and other activities that were initiated under the program. 32 In general, the efforts made by the program to integrate the provision of improved facilities with hygiene education are encouraging, and good results have been registered as a result. Integration of hygiene education with water supply and sanitation is particularly evident in the organizational set up of the program and implementation of activities on the ground. Nevertheless, there is room for improvement in the design of the future program to ensure effective integration and enhance adoption of hygiene practices by target groups. Integration of hygiene promotion into water and sanitation programs with the aim of bringing about meaningful behavior change requires more effort. It requires in particular a thorough planning of IEC/BCC activities based on context￾specific assessments of existing conditions and practices, and development of sound and effective strategies with clearly defined behavioral objectives, target groups, messages, communication channels, as well as M&E mechanisms taking available resources into account. Overall, MWA is making encouraging progress in terms of increasing access to safe water supply and sanitation services in its target areas. These gains demonstrate a strong commitment, improved harmonization of development efforts by member NGO partners and increasing donor funding. Nevertheless, the MWP needs to be strengthened both in programming and operations to further its initiatives and role as an important WASH sector player in the country. 33 4. LESSONS LEARNED Increase Access to Safe Water Supply ƒ Safe water handling in the household, such as hand washing and proper latrine use must be included in every safe water project. Otherwise water protection at source is not effective. ƒ Integration of water supply with hygiene and sanitation needs to be emphasized through MOUs at woreda and kebele level. ƒ Application of Integrated Water Resources Management (IWRM) principles improves water source. Increase Access to Sanitation ƒ Use of low-cost technologies enhances implementation, replication and the acceptance of sanitation facilities among communities. ƒ Householdss decide the preferred sanitation technology options to ensure continuity. This implies the fact that people are put at the center of their development issues and are assisted to analyze their social and economical situations and define their own priorities to decide their service levels on the sanitation ladder. Increase community awareness ƒ Application of CLTS enhances community-wide action. ƒ Mainstreaming good governance has resulted in an increased sense of scheme ownership and increased community contribution. ƒ Use of grass roots government structures, such as Health Extension Workers, accelerate community based processes and changes because of their local knowledge and relationship with the community. ƒ Community monitoring systems ensure accountability. Promote Safe Hygiene Practices ƒ Additional health education efforts are required to cultivate hand washing with soap/ash at critical times. ƒ Household outreach needs to be strengthened to ensure safe water storage & handling. 34 5. PRIORITIZED RECOMMENDATIONS MWA/E is considered one of six major programs working towards achieving the MDG targets for Ethiopia. As MWA develops a more programmatic approach it can be a strong modality for donors to undertake an even larger scale strategic contribution to the sectors’ achievements. Reaching the MDG targets on time will require a more concerted and coordinated effort by all partners including the GOE donors, NGOs, and the private sector. The following are prioritized recommendations directed mainly to the MWA. The recommendations are separated into programmatic and operational areas to which MWA is expected to deliver. MWA Needs To: Programmatic 1. In all WASH interventions ensure that people are put at the center of activities and are assisted preferably by health extension workers and (HEW) community health promoters (CHP) to analyze their problems, define their own priorities and decide their service levels for water and sanitation interventions. Participatory monitoring systems should also be introduced through which target groups evaluate progress and define priority action points on a regular basis. This process, in addition to others, will enhance commitments on the part of communities. 2. Establish joint baseline data at the beginning of future projects including agreed upon key indicators in water, sanitation and hygiene, and monitor change against these at regular intervals including six months and one year following project closure in villages/woredas. 3. Engage more actively with the relevant woreda offices and facilitate improved coordination, experience sharing and joint actions working with other government and non government players within the target woreda. 4. Develop and implement a clear strategy to improve planning, implementation, community support mechanism and monitoring capacity and systems within the woreda for water and sanitation interventions. A starting point for this would be a capacity development plan, informed by a capacity needs assessment. The capacity development plan could be closely associated with the parallel development of a Woreda WASH implementation strategic plan. 5. Provide a clearer framework for evaluating MWA program successes and failures including comparison among different project/partner approaches. This may be combined with improved documentation and real time sharing of experience and promising practices, amongst partners and with others (e.g., through annual/bi-annual workshops, topic based newsletters and peer site visits). 6. Ensure that each organization invests in development of a strategic, reality based communication strategy for hygiene education which is well integrated into other activities with the community. This may involve supporting organizations to access BCC expertise where this is outside of the NGOs current expertise. 7. Develop a clear advocacy strategy for better linkages with and influence on national and regional government and other key players in the WASH sector in country. 35 8. Link directly to the National Plan of Action and engage in the development and implementation of the National Sector Monitoring process while maintaining a MWA program M&E framework. Operational 9. Work directly with existing woreda staff and health extension workers (such as community volunteers and health promoters) to build the capacity of the existing personnel in order to facilitate ongoing activity after the end of the project. 10. Ensure the responsibility of WASHCOs includes clearly defined tasks in hygiene promotion, possibly with one member specifically assigned to supervise and coordinate these activities. This is in addition to the senior project staff with BCC qualification and experience and to involve woreda health staff in the development, implementation and M&E of hygiene promotion activities. 11. Assist WASHCOs in setting more systematic tariffs, taking basic O&M (and, where applicable, replacement and expansion) costs into account (instead of the random setting of rates). 12. Introduce recognition-based incentive systems (such as certificates and recognition signs or flags) for households, groups or villages who have switched over to a new behavior. This would be based on periodic participatory review programs. Such systems will serve as strong motivators for people to adopt and maintain new behaviors. 13. Facilitate learning and experience sharing outside the MWA, study tours and exposure visits should be organized for partners to visit projects where successful participatory hygiene education programs are implemented in the country. Include operational research of best practices from other areas and pilot new approaches. This is especially important in areas of community management, hygiene and sanitation, multiple uses of water IWRM, participatory monitoring and evaluation and in building capacities of the private sector at woreda level, woreda offices and communities. 14. Include detailed financial plans (with schedules of spending with corresponding activities) in project proposals and implement them to avoid possible concerns by donors on rates of spending especially during the early phase of implementation. 15. Strengthen the MWA/E secretariat’s in-house capacity. At the same time, outsourcing certain functions of the program to external experts/consultants who would provide technical assistance including program formulation and periodic monitoring could be one option. 36 6. FUTURE DIRECTIONS → More focus both in scale and scope of interventions is needed. Instead of efforts to increase targets/spread efforts, focus in the future on introducing best practices from other areas and piloting new approaches, especially in the areas of participation, community management, gender, H&S promotion, household water treatment and storage, multiple use of water, and IWRM. It should be noted that, the main responsibility and accountability for the provision of basic social services rest on the government. NGOs should align with that of government programs to fill gaps through leveraging resources to strengthen the sector performance. To this end, MWA could contribute to further develop both local government and community’s capacity to ensure sustainable use of facilities. MWA is also better placed to invest more in R&D to improve quality of services, document best practices, and sharing experience in the sector. → More attention to harmonized approaches and adoption of best practices, with equal attention to the importance of trying out innovative methodologies, introducing new technologies, etc., (setting measurable indictors both for performance and impact, standardizing M&E systems, including calculation of water and sanitation coverage). → Linkage and integration of MWA program with other programs, such as the government’s WASH and HEP, USAID/E Health and Nutrition Program, WSP, and USAID HIP and UNICEF WASH Program. → Capacity building for woreda Water, Health and, if relevant, Education Offices, investing in systems and approaches in recognition that staff turnover will continue for the foreseeable future.