DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. MID-TERM EVALUATION USAID/JORDAN WATER MANAGEMENT INITIATIVE (WMI) WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION July 29, 2018 Contracted under USAID 6084000.01USAID Monitoring & Evaluation Support Project This publication was produced at the request of the United States Agency for International Development. It was prepared independently by LilitMelikyan (Governance Expert);Mohye Al Deen Al Shbool (Water Utility Expert);FerasMatar (IWRM/NRW Expert); RaniaZoubi (WDM/BCC Expert); RaghebFityan (M&E Specialist); and Joe Landis (Junior Evaluator), on behalf of INTEGRATED, partner under USAID Jordan Monitoring and Evaluation Support Program. CONTENTS ACRONYMS 1 EXECUTIVE SUMMARY 5 EVALUATION PURPOSE AND PROJECT BACKGROUND 5 EVALUATION QUESTIONS, METHODS AND LIMITATIONS 5 SELECTED FINDINGS AND CONCLUSIONS 6 RECOMMENDATIONS 11 CROSS-CUTTING 11 GOVERNANCE 12 UTILITY PERFORMANCE 13 CIVIC ENGAGEMENT 14 EVALUATION PURPOSE AND QUESTIONS 15 EVALUATION PURPOSE 15 EVALUATION QUESTIONS 15 EFFECTIVENESS 15 SUSTAINABILITY 15 LEARNING 15 PROJECT BACKGROUND 16 EVALUATION METHODOLOGY AND LIMITATIONS 19 METHODOLOGY 19 LIMITATIONS 21 FINDINGS AND CONCLUSIONS 21 GOVERNANCE 22 FINDINGS: EFFECTIVENESS 22 CONCLUSIONS: EFFECTIVENESS 27 FINDINGS: SUSTAINABILITY 29 CONCLUSIONS: SUSTAINABILITY 32 UTILITY PERFORMANCE 33 FINDINGS: EFFECTIVENESS 33 CONCLUSIONS: EFFECTIVENESS 39 FINDINGS: SUSTAINABILITY 40 CONCLUSIONS: SUSTAINABILITY 42 CITIZEN ENGAGEMENT 43 FINDINGS: EFFECTIVENESS 43 CONCLUSIONS: EFFECTIVENESS 47 FINDINGS AND CONCLUSIONS: SUSTAINABILITY 48 LEARNING 50 RECOMMENDATIONS 53 CROSS-CUTTING 53 GOVERNANCE 53 UTILITY PERFORMANCE 55 CIVIC ENGAGEMENT 55 ANNEX I. EVALUATION STATEMENT OF WORK 57 ANNEXII: EVALUATION DESIGN AND METHODOLOGY 62 ANNEX III: BIBLIOGRAPHY 69 ANNEX IV: LIST OF PERSONS INTERVIEWED 74 ANNEX V: WMI LOGIC MODEL AS OF Q2 2018 83 ANNEX VI: INTERVIEW TOOLS 85 1 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV ACRONYMS AA AssignmentAgreement ABB Activity Based Budgeting ACS American Community School ACWUA Arab Countries Water Utilities Association AfD AgenceFrançaise de Développement AMEP Activity Monitoring and Evaluation Plan AoA Articles of Association AWC Aqaba Water Company BCC Behavior Change Communication BPR Business Process Review CAB Citizen Advisory Board COP Chief of Party DCOP Deputy Chief of Party DMZ EAM EE District Meter Zone Enterprise Asset Management Energy Efficiency ERP Enterprise Resource Planning EU European Union FARA Fixed Amount Reimbursement Agreement FGD Focus Group Discussions GAAP Gender Analysis and Action Plan GAM Greater Amman Municipality GIS Geographic Information Systems GIZ GoJ Deutsche GesellschaftfürInternationaleZusammenarbeit Government of Jordan HHD Hand Held Device HHU Hand Held Units 2| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV HR Human Resources HRM Human Resources Management IA Internal Audit ICTU IMF IMT Information and Communications Technologies Unit International Monetary Fund Irrigation Management Transfer IP Implementing Partner ISSP Institutional Support & Strengthening Program IT IWSMG Information Technologies Improved Water Sector Management and Governance JCC JOD Jordan Cooperative Corporation Jordanian Dinar JREDS Royal Marine Conservation Society of Jordan JVA Jordan Valley Authority KAC KAP KfW King Abdullah Canal Knowledge, Attitudes and Practices Kreditanstalt Fur Wiederaufbau KII Key Informant Interviews KPI Key Performance Indicator MCC M&E MESC Millennium Challenge Corporation Monitoring and Evaluation NRW Management Engineering Services MoEnv MoF MOU MRVM MWA Ministry of Environment Ministry of Finance Memorandum of Understanding Meter Reading & Violation Management System Ma’an Water Authority MWI Ministry of Water and Irrigation 3 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV NGOs Non-Governmental Organizations NOS NRW NWIS National Orthodox School Non-Revenue Water National Water Information System NWS National Water Strategy O&M Operation and Management PBC PFMRAF Performance Based Contract Public Financial Management Risk Assessment Framework PIP Performance Improvement Plan PMU PPP Performance Management Unit Public Private Partnership PSP Private Sector Participation QA Quality Assurance QR Quarterly Report RBAP RC Results-Based Policy Action Plan Regional Company RE ROU Renewable Energy Remote Operation Unit RS Remote Sensing SOP Standard Operating Procedures SOW Scope of Work TA Technical Assistance TOC Theory of Change TOR Terms of Reference WAJ Water Authority of Jordan WDM WEAP WHO Water Demand Management Water Evaluation and Planning World Health Organization 4| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV WIT WMI Water Innovations Technology Water Management Initiative WSD Water Saving Devices WSP Water Safety Plan WUA WUE Water Users Associations Water Use Efficiency WWTP W&WW YWC Wastewater Treatment Plants Water and Wastewater Yarmouk Water Company ZWA Zarqa Water Administration 5 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV EXECUTIVE SUMMARY EVALUATION PURPOSE AND PROJECT BACKGROUND Jordan is recognized as one of the ten most water deprived countries in the world. Domestic water use in Jordan is among the lowest in the world, and barely meets basic household needs for sanitation, cooking, and cleaning. The influx of Syrian refugees living in host communities and refugee camps is further draining the limited water supply. WHO projects Jordan will enter a state of “extreme water poverty” by 2025 should no major advancements be made. However, Jordan currently faces a unique window of opportunity, tied to momentum generated and support for water sector reform, utility performance improvement, reduction in Non Revenue Water (NRW) loss, and efficiency gains. The Water Management Initiative (WMI) Activity is a five-year, $33 million activity (April 2016 – March 2021) implemented by Tetra Tech ARD “to support the Government of Jordan (GoJ) in water sector reform through technical assistance (TA) to strengthen the GoJ’s reform, policy development and implementation, and capacity-building efforts.”1 The scope of of the Water Management Initiative (WMI) Activity is ”to increase accountable, sustainable, water sector management and governance by supporting and strengthening the Government of Jordan’s policy development and implementation, and capacity building efforts.”2 WMI activities are grouped into the following components: Component 1) Water Supply Systems, Component 2) Water Conservation and Demand Management, Component 3) Water Sector Governance, Component 4) Protection of Water Supply, and Component 5) Tools and Equipment (added March 2018). WMI interventions are grouped in five components and working in close collaboration with GoJ through the Ministry of Water and Irrigation (MWI); the Water Authority of Jordan (WAJ) and its Performance Management Unit (PMU); the Jordan Valley Authority (JVA); and water utility companies including Miyahuna, Yarmouk Water Company (YWC), and Aqaba Water Company (AWC). WMI subcontractors include Orient Engineering Consulting and Design for technical and engineering expertise; Jordan Social Marketing Center for expertise in outreach, communications, and behavior change; SEGURA Consulting for utility reform and restructuring; and Water for Life Solutions for expertise in water demand management, agricultural and groundwater management. USAID requested a Mid-Term Evaluation of WMI to guide prioritization of programming initiatives and resources for the remainder of the project to make mid-course corrections to support achievement of WMI objectives. The evaluation results will be used to inform the design, performance, and implementation of activities related to the government counterparts, public utilities, water user associations, and water consumers in enhancing water and wastewater infrastructure, water conservation, and governance and policy reform. EVALUATION QUESTIONS, METHODS AND LIMITATIONS The evaluation employed a qualitative approach to answering the evaluation questions that included an exhaustive review of WMI project documents, research and special studies conducted by WMI, 1 USAID/WMI “Section C- Performance Work Statement.” USAID-WMI Contract. 2 Ibid, 6| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV training curricula, and databases of information maintained by WMI. To maximize the utility of the evaluation to WMI and USAID, and in light of recent ministerial changes resulting in changes in WMI counterparts at MWI and changes in WMI’s scope, the evaluation team used a participatory planning approach that relied heavily upon WMI staff input for identification of key informants and stakeholders. Stakeholders interviewed included government officials, utility companies, NGOs, water user associations (WUAs), WMI staff, and other stakeholder organizations. A purposive sample of KIIs and FGDs was selected to cover all utilities benefiting from WMI support. In addition, MWI, WAJ and JVA interviewees were selected based on their ability to speak to the various WMI interventions. While conducting interviews, utility observations were conducted using a structured checklist to facilitate triangulating data. Data collection was conducted from May 3 through June 11, 2018. The full list of people interviewed is contained in Annex IV. The evaluation design, data analysis approach are included in the Evaluation Design Report in Annex II, and the associated evaluation tools in Annex VI. The evaluation was guided by six evaluation questions grouped under the categories of effectiveness, sustainability and learning (See Evaluation Purpose). SELECTED FINDINGS AND CONCLUSIONS WMI is a complex project with 41 initiatives and activities grouped into five main components: 1) Water Supply Systems, 2) Water Conservation and Demand Management, 3) Water Sector Governance, 4) Protection of Water Supply, and 5) Tools and Equipment. Although WMI operates through Components, the evaluation findings and recommendations lend themselves to presentation through a lens of Governance, Utility Performance, and Citizen Engagement. These categories reflect the key stakeholders in water management, as well as the relationships that connect, as well as the dynamics captured within WMI’s Theory of Change, shown within the below best practice model for water management: Figure 1. Best practice model for water management. 7 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV In keeping with this best practice model,3 the organization of the evaluation report follows the three broad categories of “Governance,” “Utility Performance,” and “Citizen Engagement,” with evaluation questions addressing Effectiveness, Sustainability, and Learning within each broad category. GOVERNANCE Under water sector governance, WMI aims tosupport the GOJ in water sector policy development and implementation; enhance utility internal governance and overall governance environment to improve performance; support improvement of irrigation management; develop and support sector regulation; support the integration of strategic communications, advocacy, gender inclusion, and youth engagement within the water sector; develop a groundwater management framework and sustainable monitoring systems; strengthen wastewater treatment performance and compliance; and improve water quality management. WMI interventionsevaluated in this section include the following: 1.2 International Monetary Fund (IMF) Action Plan Support 3.1 Water Sector Policy Supported 3.2 Utility Assignment Agreements Developed/Modified 3.3 Divestment of Irrigation Management to Water User Associations (WUAs) Supported 3.4 Independent Regulator Developed and Supported 4.1 Groundwater Management Framework Developed 4.2 Wastewater Treatment Performance and Compliance Strengthened 4.3 Water Quality Management Improved INDEPENDENT REGULATOR Progress towards achieving improved governance objectives is significantly hampered by the limited buy-in among some stakeholders who have a vested interest in preserving the status quo by holding multiple positions within MWI, WAJ, and utility Boards. Until governance is improved, the utility performance monitoring is unlikely to improve, with possible consequences on performance improvement. However, with the new water sector administration, WMI and other donors may have a renewed opportunity to see increased buy-in to pursue to establishment of a firewalled regulatory unit as an interim step, and move towards an independent regulator when service providers are fully privatized. While there is potential for great synergies between GIZ and WMI, there is also a need for clearer delineation of roles between WMI and GIZ in supporting this agenda, particularly with respect to duplication of similar activities and some differences in approaches, leading to confusion among stakeholders. RBAP: WMI’s approach to RBP Action Plans development has been effective in securing a high level of buy-in by MWI, where the eight NWS water policies have been translated into plans ready for implementation by MWI units, laying the groundwork for WMI to focus implementation on capacity building of these units in the next stage. The participatory approach in RBP Action Planning actively engaged MWI though working groups that included technical assistance from GIZ and WMI, and will require strengthened organizational and technical capacity to implement these plans. 3 Adopted from: UNDP-SIWI Water Governance Facility & UNICEF, 2015. WASH and Accountability: Explaining the Concept. Accountability for Sustainability Partnership: UNDP-SIWI Water Governance Facility and UNICEF. Stockholm and New York: SIWI and UNICEF.http://www.watergovernance.org/resources/accountability-in-wash-explaining-the-concept/ 8| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV AA: While the AA modifications and recommendations have generated support from MWI and WAJ, there is some resistance by companies leading to delays in adopting amendments to meet AA objectives. AA reform is tied to regulatory reform, inclusive of addressing conflicts of interest at the Board levels between utilities and WAJ, and the coming period should focus on simultaneous implementation of AA and regulatory reform. While Citizen Advisory Boards (CABs) were rejected, alternatives for citizen engagement are not yet in place, and alternatives will need to be sought in the coming period. DIVESTMENT OF IRRIGATION MANAGEMENT TO WUA The potential effectiveness of WMI support to Divestment of Irrigation Management to WUAs continues to be affected by the lack of clarity among stakeholders (and JVA in particular) for the mandate of the WUAs, and the weak buy￾in in relation to the WMI assistance package, which hinders WMI from developing comprehensive interventions to support intended governance arrangements, and instead relegates support to technological interventions for improved data flows (which is needed but not sufficient). IMF ACTION PLAN SUPPORT While support to the IMF Action Plan reporting has been effective for bi-annual reporting to IMF, it could benefit from increased institutionalization and knowledge transfer of technical assistance. While this is already planned, knowledge transfer activities are likely to be most useful to MWI through secondment with a gradual handover of responsibilities, increasing the likelihood of sustainability. Although still in review stages, studies supporting WAJ in improving Cost Recovery and Financial Accounting and Fixed Asset Accounting Policies have been well-received with buy-in for improving operations. Next steps in support could focus on fleshing￾out clear goals and milestones for their achievement (scope and time). While there is clear buy-in by WAJ for promotion of PSP, prioritization of the subsectors, inclusive of mapping and coordination with the various actors in the sector, could help to enhance the likelihood of sustainability. GROUNDWATER WMI efforts to support establishment of a business unit for private wells has seen a number of success factors including partner engagement through the Task Force, technical assistance inclusive of the feasibility study for establishing the private wells unit, the assessment of increased annual revenue through the licensing systems for private wells, and support to tendering for smart meters. There is both a high level of buy-in for establishing a business unit for private wells inside WAJ as well as potential for contribution to improved standards and reduction of illegal use, which WMI should be prioritized in the coming period, supported by trend analysis of monitoring data and WMI follow up. UTILITY PERFORMANCE The original SOW tasked WMI with assisting the GOJ in achieving cost recovery and NRW reduction by providing intensive, tailored support to Zarqa Water Authority (ZWA) and the Yarmouk Water Company (YWC), which have the greatest potential for contributing to this goal. During the first two years, additional activities were added to WMI’s tasks: supporting Ma’an Water Administration (MWA) and assistance through providing tools, equipment and systems for all utilities. WMI interventions evaluated in this section include the following targets: 1.1: Zarqa Key Performance Indicators (KPIs) Improved 1.3: Yarmouk Financial Performance Improved 1.4: Ma’an Operational Performance Improved 4.2: Wastewater Treatment Performance and Compliance Strengthened PERFORMANCE IMPROVEMENT PLANNING Planning activities that have an implementation￾focused approach, such as the one used in Aqaba Water Company (under merger with MWA) to create the Master Plan, were effective in generating stakeholder engagement, giving stakeholders a 9 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV clear idea of how WMI interventions fit into the future of their organization, and such approaches should be replicated. On the other hand, with utilities such as Miyahuna (Amman-Zarqa), planning activities resulted only from dialogue with upper management rather than including a documented plan, and none of the stakeholders interviewed within ZWA or Miyahuna had seen the PIP and KPI targets, nor had a designated unit to manage such a plan. In the coming period, WMI could benefit from replicating effective approaches to planning and/or increasing engagement in planning with stakeholders and plan sharing, inclusive of designating roles and responsibilities, setting timelines and benchmarks for progress, and ensuring necessary budgets are in place. Interventions at utilities that were based upon extensive formal planning with stakeholders are seeing the best results. In the cases of the roll-out of Phase I of the ERP in Miyahuna and the establishment of the Customer Service Control Unit and the Internal Audit Unit in YWC, the project and its partners made steady progress as long as there was a sufficient plan and timeline to guide them. The most effective communication mechanisms observed consisted of regular steering committee meetings, as demonstrated with the WAJ WWTP task force. Where WMI interventions are aligned with utilities’ perceived needs, thereby increasing counterparts’ buy-in and ownership through in-kind contribution, there is increased likelihood of sustainability. This is especially true for the introduction of new systems. Technical assistance to Miyahuna’s SCADA system and the creation of the Wastewater Treatment Plants Directorate’s SOW stand out as examples of effective interventions focused on stakeholder priorities. Conversely, where WMI has introduced new practices outside of requests from partners at utilities, communication practices could be improved in order to generate buy-in and up-take. WMI could benefit from cascading communication to coordinate these activities across steering committees and beyond upper management, in the coming period as a way of ensuring that all interventions are jointly managed. The potential for sustainability for some interventions lies in moving from short-term progress to institutionalization in the coming period. For example, in the case of developing fixed asset registers and GIS systems, WMI could benefit from focusing on generating buy-in for the creation of these units to ensure that roles exist within utilities’ structures to perform up-keep in the future. TECHNICAL ASSISTANCE Short-term technical assistance interventions enabled close collaboration and complemented the already-existing work of utilities, with the best example evidenced by WMI technical assistance to Miyahuna in procuring the SCADA system and expert trainers. The results of other training are mixed, where there are opportunities to improve training outcomes within utilities by designating more staff members to train on-site for extended periods of time (seconded). NRW The issue of double connections has been shown by the pilot study to have great potential for reducing NRW, but will need to be implemented and funded in order to make a contribution to long-term outcomes. WMI assisted ZWA in collecting missing customers’ data in order to increase billing from 91% of the customer base to 98% at the end of March 2018, which according to WMI resulted in a drop in NRW of 5 million cubic meters during FY 2016/2017.4 In the coming period, the ZWA billing intervention could benefit from a focus on optimizing billing SOPs now that the quick win of customer registration has been accomplished. At MWA, introduction of Hand Held Units (HHUs) increased the efficiency of billing, leading to “an increase in revenue by 5%, and reading and billing reached 98%,” according to MWA. The introduction of a new Customer Information System (X7) for digitized processing of billing and collection has potential to improve collection rates, but require additional financial and technical support in the coming period in order to be sustained and improve KPIs. 4 The time lag in NRW reporting captured here demonstrates the difficulty in capturing and attributing performance improvements such as billing interventions in 2018 to NRW savings data linked to 2016/17. 10| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV SYSTEMS Systems-based interventions such as X7 and ERP are likely to be sustained due to widespread buy-in across managerial levels. However, financial constraints within utilities and within WAJ are inhibiting the potential for sustainability of the ERP and X7, which require hardware and licensing, new organizational units that require staffing, and training plans that require consultants. WMI has considered the issue of financial support, and the introduction of a limited amount of direct tools, equipment and systems has led to good results thus far. In the coming period, WMI could increase the likelihood of sustainability by moving beyond technical support to the development of TORs to the next step of active match-making between utilities/WAJ and entities that can supplement their budgets. PILOTS The piloting of the pulsers at MWA has demonstrated potential to improve water meter readings, as evidenced by overall reading rate improvement correlated with the use of pulsers. However, pulsers are only effective when paired with smart meters, not the mechanical meters in place in many areas of MWA, leading to less than optimal outcomes in performance improvement at MWA. The pilot at Jerash Control Center did not advance beyond a simulation before the then￾Minister ordered it to be stopped and shifted to a national scale. While the pilot demonstrated the buy-in of the Ministry, WMI and YWC staff were not able to generate lessons learned for the eventual scale-up of the idea. In the coming period, interventions involving the piloting of certain practices or technologies could benefit from documentation and assessment of lessons learned in order to ensure replicability and sustainability. CITIZEN ENGAGEMENT With respect to citizen engagement, WMI is tasked to support the development of water demand management, inclusive of behavior change communication is included, which starts with the implementation of the Knowledge, Attitudes, and Practices (KAP) survey, in addition to activities related to water sector strategic communications and advocacy with a focus on gender and youth. WMI interventionsevaluated in this section include the following targets: 2.2 Behavior Change Communication (BCC) Supported 3.5 Water Sector Strategic Communications, Advocacy, Gender, and Youth Supported BCC WMI’s initiatives with civic engagement and behavior change communication (BCC) are in early stages of implementation, primarily due to a series of changes and delays. WMI conducted a Knowledge Attitudes and Practices (KAP) survey intended to inform the design of BCC activities however analysis and application of the KAP survey was not yet complete at the time of evaluation. While WMI efforts with Communication departments of water utilities are at the early stages, at the time of evaluation partners at utilities, particularlyYWC, Miyahuna, MWA and ZWA, had disparate views than WMI staff members on objectives and needs. Although WMI has not yet fully engaged utilities in communications directed towards long-term behavior change, WMI has contributed to an initiative to increase awareness of the Amiri law for payment of delinquent water bills in YWC through a mass media campaign reaching an estimated 322,070 people. The campaign focusedon debt collection rather than behavior change for water conservation, leaving BCC for water conservation an area in need of work to meet WMI aims in the coming period. YOUTH: While WMI piloted school-based awareness raising with two private schools in Amman, lessons learned with school piloting contributed to a wise decision to outsource civic engagement through schools to the Royal Marine Conservation Society of Jordan (JREDS) to continue work at Eco-schools and to promote water-use efficiency in touristic facilities in Amman, Dead Sea and Aqaba. WMI was contracting JREDS at the time of evaluation. The internship program, while an effective way to engage youth in utilities, is currently implemented entirely by WMI. At the time of evaluation the internship program did not have institutional arrangements to continue, and in the coming years it will be important to work to institutionalize these, provided utilities are interested. 11 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV GENDER: WMI’s work related to gender hinges on the Gender Analysis and Action Plan (GAAP) conducted in WMI’s first year. The GAAP focuses on activities related to women working in the (utility) sector, grouped around amending HR gender policies intended to hire more women at utilitiesand increase female membership in utility boards, encouraging female participation in consultations, and sex disaggregation of the above data. However, obstacles that hinder gender￾sensitive hiring practices are significant, which negate progress made by WMI in instituting gender sensitive HR policies at utilities, due to WAJ civil service law de facto over-riding utility-based policies. In addition to the GAAP, WMI has provided technical assistance to develop the strategy and workplan for MWI’s Gender Unit, but the Unit as yet has neither financial resources nor MWI management buy-in,to implement the plan. CROSS-CUTTING Across components and stakeholder groups a number of themes were apparent. Perhaps not surprising, WMI’s initiatives with the most effectiveness and sustainability are those that are aligned with partner priorities; have undergone a structured planning process with partners; conduct regular communications with partners throughout implementation; and have been supported with both material and technical assistance. While WMI has made headway in launching 41 initiatives and associated J4 deliverables, where WMI results are currently limited, limitations are most attributable to an absence of full commitment among partners. WMI has had to work with three different administrations in its two and a half years of operations, starting, stopping, and establishing relationships anew, as well asshifting direction in an attempt to be responsive to each successive administration. Shortcomings in effectiveness and risks to sustainability are highly correlated to limited buy-in at both senior and operational levels. While WMI invests resources in building relationships at senior levels and with each new administration, new directions and directives are sometimes not communicated through the ranks to operations-level staff, who are often in the position to make or break implementation. Compounding these challenges is the misperception among partners that as a USAID program, WMI will provide full financial assistance for equipment, systems, and implementation, in addition to technical assistance. The multiple changes over the first half of the project- both within activities modified with the aim of improved performance or modified in response to stakeholder desires, as well as changes to WMI’s operating context - have impacted WMI’s program design. In particular, evaluation findings related to the assumption of “full cooperation and commitment of sector stakeholder staff” demonstrate that this assumption has not always held true – both at the Minister level and utlity mid-level management. RECOMMENDATIONS CROSS-CUTTING 1. Mid-term evaluation offers the opportune time for WMI and USAID to revisit assumptions and, given the current dynamics of commitment, capacity, and resources, fine-tune the way forward. USAID and WMI would benefit from reflecting on how changes, both as a result of modifying and improving activities and as a result of changes in the context in Jordan, impact project logic flows against program design and the Theory of Change, and ensure that assumptions still hold. 2. USAID and WMI should consider developing custom-level outcome indicators to augment existing F-indicators as a means of increasing reliability, timeliness and attribution to project interventions. For example, customer billing data collected by WMI (ie at MWA, ZWA) is both timely and attributable to WMI interventions, but correlating it with NRW datagenerated one to two years back poses challenges to attribution and learning. WMI should consider using utility-level KPI data as interim data points to ensure continuous learning and activity attribution from M&E. 12| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV 3. For activities that are technology related, WMI should put effort into securing funding for hardware required to operate the systems, including leveraging Fixed Amount Reimbursement Agreement (FARA) funding as a strategic resource to avoid delays in procuring hardware. 4. In order to avoid the misperception among partners that WMI will fully fund all equipment, systems, or implementation in addition to technical assistance, WMI should meet with all partners to ensure a common understanding of WMI assistance mechanisms. If initiatives require financial or material assistance that had not been anticipated and are not covered under component five, WMI should prioritize determining whether to add financial support, assist partners in identifying alternative sources of funding, or discontinue supporting the initiative and communicate the decision in writing to both management and operational-level counterparts. 5. For initiatives that involve new systems and technology, WMI should base interventions on pre￾assessments, and include technical assistance for on-the-job training of partner staff, and replicate successes such as those seen with X7. 6. WMI should develop an internal strategy for gaining buy-in for its sector-wide vision for reform among each stakeholder group. The plan should set out expected roles and responsibilities for each group in light of their unique interests and resources and outline how regular communications will be maintained. 7. WMI should deepen participatory approaches with partners through regular meetings and well￾designed communication structures that include top-level and operational-level management, throughout the lifespan of each intervention. 8. WMI and USAID should thoroughly assess risk and develop a contingency plan during the inception phase of interventions to mitigate situations such as changes in staff at utilities and changes in direction at the level of WAJ/MWI that may occur during implementation. 9. USAID should consider leveraging its support of GoJ through MESC, WIT, and WMI as a way to increase timeliness of secondary data that MWI supplies for WMI indicators. GOVERNANCE 1. Finalize the scope and approach of the assistance package to PMU to help its monitoring function of company performance/data, combined with conditionality to require publicizing the KPIs.Clarify the relation between WMI support and GIZ support. 2. As a cornerstone for governance reform, prioritize further corporatization through AA reforms and consider using conditionality mechanisms for adoption of revised AAs/AOAs by Miyahuna and YWC. WMI should also further explore international best practice mechanisms for citizen engagement in utility governance. 3. Prioritize achieving consensus on a vision for changes in WUA mandate and ensure that new leadership at the JVA and Ministerial levels is continuously engaged, in light of March 2019 renewal date of WUA contracts. Continue to ensure close coordination with GIZ to ensure consistency, synergies and mutual reinforcement of interventions. 4. Until JVA clarifies direction for reform, focus on technological interventions. Once clarified, commit significant resources to the JVA pilot in the North using a staggered approach with first steps including the use ofsmart meters at farm gates and building capacity of the JVA control center to use data and GIS mapping. Until JVA clarifies its direction for reform, focus on technological aspects of the pilot in the North without committing significant resources; building capacity of the JVA control center coupled with GIS mapping. Once the vision is clarified, WMI could consider support to institutional reform of the pilot WUA, as well as an assessment of the financial gaps in the sector. 5. Develop a sustainability plan to institutionalize secretariat support for IMF Action plan reporting with a focus on knowledge transfer to MWI staff, including those from the Strategies and Policies Unit of MWI, with a gradual handover of responsibilities. Ensure that linkages to the National Water Information System are in place (including utility level data) as part of the sustainability plan for data use and integration in IMF reporting. 6. Focus on fleshing-out clear goals and milestones for Cost Recovery/Financial Accounting. 13 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV 7. Capitalize on the high levels of buy-in and commitment to Private Wells Business Unit establishment and prioritize support to WAJ. 8. Fast-track the development of a sector-wide NRW masterplan, underway at the time of evaluation, as a matter of priority to meet the November 2018 deadline. 9. Consider supporting two rather than three PSP/PBC projects, prioritizing those sub-sectors in which WMI interventions have the most added demonstration/learning value, in close coordination with donors active in these subsectors to ensurenon-duplication, coordination, synergies and sustainability. 10. Assist WAJ to identify sources of co-funding for ICTU ERP implementation(hardware/licensing). 11. Focus RBAP efforts on building leadership capacity within the Policies and Strategy Unit and building technical capacity that cascades down into business units for each entity in the sector. Support amendment of NWIS to cover utility level data and ensure that interventions related to data systems strengthening at ICTU are well-integrated into with policy development processes, including the monitoring of the implementation of the RBAPs once adopted. 12. Assign a full-time WMI specialist and ensure sufficient internet capabilities to maximize effective use and incorporation of remote sensing data, and coordinate with the World Bank to ensure synergies in support. 13. WMI should replicate training to expand the water audit team and provide a WSP expert to support knowledge transfer within MWI. For continued WAJ buy-in,WMI should ensure that work in WSP reinforces existingJordanian standards and regulations in WSP, and allay (mis)perceptions of replacing existing standards. 14. If WMI intends to make progress in water quality governance, support to the Ministry of Environment needs to be clarified and intensified, and financial support will need to be secured to support GIS infrastructure. Also, WMI should support coordination between all the stakeholders involved in monitoring of reclaimed water quality. UTILITY PERFORMANCE 1. Work with stakeholders within utility companies to agree upon a vision, develop a work plan for each activity, and ensure that the work plan is shared with both management and operations staff. ▪ Ensure review and agreement on action plans via signatures of both WMI and counterparts at utilities. ▪ Designate an official project management team for each activity including both WMI staff and direct counterparts at the implementation level. ▪ Each activity plan should include a description of the distribution of staff time and financial responsibilities and the timeline for implementation ▪ Develop progress indicators for each utility and share these with all relevant stakeholders. 2. In order to ensure that NRW pilots such as the double connections study, HHUs, and pulsers lead to results, WMI should secure funding for the next steps, and ensure that results are measured in order to gauge their contribution to utility-wide NRW reduction. 3. WMI should continue to support WAJ in improving energy efficiency measures in WWTPs, not only through technical assistance but also through securing financial support for implementing recommendations. 4. When creating utility-wide strategic plans, replicate to the extent possible the approach of deep engagement with utility staff that was used in the development of the Aqaba water and wastewater investment Master Plan. 14| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV 5. Prioritize generating buy-in at the upper management and Board levels for interventions involving structural change such as organizational charts, job descriptions, and SOPs. CIVIC ENGAGEMENT 1. WMI should conduct an assessment to determine the real value of the internship program to both interns and employers. If determined to be of significant value, USAID should consider providing an endowment to an entity such as an engineering university to continue the program or consider wrapping the activity into another project focused on youth employment. WMI should consider supporting the GOJ in instituting an incentive schemes for the utilities to have their own internship programs. 2. When designing future BCC activities, an attempt should be made to utilize information gained through the KAP, GAAP, and other BCC studies to target and differentiate BCC activities to various audiences based on their respective roles and motivations. 3. To maximize synergies, increase linkages between awareness and BCC campaigns, and supporting replication/scaling up of the pilot interventions, after rigorously assessing their effectiveness and streamlining the design, rather than supporting activities for awareness that do not necessarily link to BCC activities. 4. In order to address the spirit of the gender equality objective, WMI should broaden the focus of gender initiatives beyond employing women and recast gender-sensitivity as a target marketing approach for BCC, sensitizing the decision-makers to better understand the important role of women as water managers, and the need for policies that reflect those differentials in water use. 5. If WMI intends to increase opportunities for women within public utilities, WMI should focus on implementation, and extend beyond counting and disaggregating to measure the impact of gender-related HR policies relative to women’s representation and engagement in decision-making. In the second half of the project WMI should focus on advancing women’s empowerment through specific actions targeting gender power dynamics and consider targeting the internship program solely to women in order to increase gender parity in employment at public utilities. 6. Provide consultative services to MWI’s Gender Unit and MWI and utilities’ Communications Departments to unpack information in the GAAP and KAP to increase communications effectiveness through leveraging gender and youth-sensitive strategies, and motivating women and youth as agents of change. Technical assistance should include raising awareness among water sector entities about how valuable women and youth can be to achieving sector objectives. 7. Re-scope the BCC component to focus only on direct support to MWI in developing the national-level strategy (at the policy level) and work with each utility company to determine how best to support their WDM communication plans. 15 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV EVALUATION PURPOSE AND QUESTIONS EVALUATION PURPOSE U.S. Agency for International Development (USAID) in Jordan requested an external performance evaluation of the Water Management Initiative (WMI) Project, a five-year project that began in April 2016. This evaluation is a mid-term evaluation, during which the data collection period coincided with the 25-26 month of implementation from May-June 2018. The purpose of the evaluation is to provide USAID with recommendations to effectively allocate resources for programming for the remainder of WMI and to make mid-course corrections to support its ability to achieve its objectives. The evaluation results will be used to inform the design, performance and implementation of activities related to the government counterparts, public utilities, water user associations, and water consumers in supporting the Government of Jordan’s capacity development and policy and implementation efforts.. EVALUATION QUESTIONS The key evaluation questions that guided the mid-term performance evaluation are: EFFECTIVENESS 1. What have been the achievements and deficiencies in each of the components and what were the factors affecting it? In particular: a) Were the achievements in line with the plans? If not why? ▪ How did the strategy and implementation approach enhance or weaken achievement of the anticipated tasks? Were there any alternatives or unexpected causes of documented results or outcomes? ▪ Were the deliverables of sufficient quality? b) b) How were the main gender and youth constraints in the project addressed? 2. What are the activities/sub-activities that need to be revised or enhanced in the next work￾plan in order to boost project performance results? What changes need to be made in terms of modalities of work/approaches? SUSTAINABILITY 3. What technical results of the project can USAID expect to be sustainable (and which will be difficult to sustain)? a) Is there an adequate sustainability plan and is it being well implemented? b) How strong is the national buy-in and the institutional arrangements to promote it? c) What is the likelihood of technical, financial and programmatic sustainability? What lasting benefits can be expected in terms of USAID’s and the project’s relationships with the main project counterparts? 4. What recommendations could be given to ensure the sustainability of the interventions? LEARNING 5. Which incomplete project activities should be prioritized for the remaining project period of performance? 6. Were there strategies or practices implemented that should be replicated in future? 16| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV PROJECT BACKGROUND Jordan is recognized as one of the ten most water deprived countries in the world. The Syrian refugee crisis has further strained the situation, and the World Health Organization (WHO) projects that Jordan will enter a state of “extreme water poverty” by 2025, should no major advancements be made. However, Jordan currently faces a unique window of opportunity. The current Minister of Water and Irrigation is committed to achieving tangible results in non-revenue water (NRW) tariff reform, and is open to exploring opportunities for water supply. The international community is cooperating to achieve synergistic results in water sector reform, utility performance improvement, reduction in NRW loss, and efficiency gains. The Government of Jordan (GoJ) has formally committed to improvements as part of the Millennium Challenge Corporation (MCC) and International Monetary Fund (IMF) agreements. Within this context, WMI aims to support the GoJ to achieve greater sustainability of the water sector by providing technical assistance (TA) to strengthen the GoJ’s reform, policy development and implementation, and capacity-building efforts. The scope of the Water Management Initiative (WMI)Project is to “increase accountable, sustainable, water sector management and governance by supporting and strengthening the Government of Jordan’s policy development and implementation, and capacity building efforts.”5 Through integral partnerships with the Government of Jordan (GOJ) and other private sector allies, USAID WMI aims at addressing Jordan’s serious and growing water scarcity: WMI works to reduce water losses, improve water sector governance, and raise the average liters of (potable) water available per capita per day. WMI activities are grouped in the five components and works in close collaboration with GoJ through the Ministry of Water and Irrigation (MWI); the Water Authority of Jordan (WAJ) and its Performance Management Unit (PMU); the Jordan Valley Authority (JVA); and water utility companies, including Miyahuna, Yarmouk Water Company (YWC), and Aqaba Water Company (AWC). WMI subcontractors include Orient Engineering Consulting and Design, which provides technical and engineering expertise across components and overall sector reform advisory services; Jordan Social Marketing Center, which provides specialized expertise in outreach, communications, and behavior change; SEGURA Consulting, which supports utility reform, restructuring, and performance improvement; and, Water for Life Solutions, which is key in offering expertise in water demand management, and agricultural and groundwater management. 5 Ibid, PROJECT BACKGROUND Activity Name WATER MANAGEMENT INITIATIVE (WMI) Activity Start / End Date April 1, 2016 to March 31, 2021 Prime Implementing Partner Tetra Tech ARD Contract Number AID-278-C-16-00001 Subcontractors Orient Engineering Consulting and Design; Jordan Social Marketing Center (JSMC); SEGURA Consulting; Water4Life Solutions (W4LS) Geographic Coverage Nationwide, with focus on Miyahuna (serving Amman and Zarqa) and Yarmouk Water Company (serving Ajloun, Irbid, Mafraq and Jerash) Funding $33,247,544 17 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV WMI conducts activities through the following five Components: 1. Water supply systems: This component is focused on water utilities management and reform mainly through support for design and implementation of Performance Improvement Plans (PIP). Under this component WMI supports the implementation of the National Action Plan to Reduce Water Sector Losses (in support of its IMF commitments); 2. Water conservation and demand management: This component covers the development of water demand management as well as activities related to the study of opportunities for expanding treated wastewater sales. As part of this component the area of behavior change communication is included, which starts with the implementation of the Knowledge, Attitudes, and Practices (KAP) survey; 3. Water sector governance: This component aims to support water sector policy development and implementation; enhance utility internal governance and overall governance environment to improve performance; support improvement of irrigation management; develop and support sector regulation; and support the integration of strategic communications, advocacy, gender inclusion, and youth engagement within the water sector; 4. Protection of water supply: This component aims to develop a groundwater management framework and sustainable monitoring systems; strengthen wastewater treatment performance and compliance; and improve water quality management; and 5. Equipment and tools: Thiscomponent was added in March 2018 to support utility company requirements for equipment, tools and systems. In March 2018 WMI’s contract was modified to include an additional $4.8 million to support utility company requirements for equipment, tools and systems (Component 5). This contract modification also revised WMI deliverables under each component. This evaluation is based upon WMI’s Activity Monitoring and Evaluation Plan (AMELP)that incorporates these revisions. WMI’s Theory of Change (TOC) recognizes that sustainable, measurable improvements in Jordan’s water sector will not come about in a singular event achieved by a limited set of actors working in isolation, but ratherrequire myriad incremental changes over time involving a broad set of stakeholders. WMI’s TOC is presentedin Figure I, below: 18| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV Figure 2. WMI Theory of Change. SOURCE: ACTIVITY MONITORING AND EVALUATION PLAN (AMEP), MARCH 2018. 19 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV Among activity indicators, key performance indicators include: 1) Amount of water provided or saved increased a) Percentage of NRW (by utility) b) Water conservation improved: number of adoptions of technologies or techniques to save water or increase reuse of water c) Cubic meters of water provided or saved as a result of U.S. Government (USG) assistance d) Cubic meters of water saved annually from water-saving technologies or techniques as a result of USG assistance 2) Water sector management and governance improved a) Number of action plans implemented for water security, integrated water resource management, and/or water source protection as a result of USG assistance b) Percent change in cost recovery of water sector (disaggregated by utility) c) Capacity built: Number of people educated on tools, approaches, and/or methods for water security, integrated water resource management, and/or water source protection as a result of USG assistance 3) Accountable, sustainable management of water and natural resources increased a) Number of policies, regulations and administrative procedures relating to water and natural resources in development, passed, or being implemented as a result of USG assistance b) Percentage of NRW (national average) WMI relies heavily on output indicators, related to outcomes of intended institutional change, which are generated from secondary sources that have questionable reliability, have a one-year reporting time-lag, and do not easily lend themselves to attribution or learning about results of institutional interventions. This approach to monitoring progress towards indicators poses challenges in terms of accuracy, attribution, and making evidence-based decisions for project management, which limits learning and adapting. In the words of one team member, “There is a challenge to ensure that the project and team leads are focusing on results rather than deliverables. We have struggled to link the workplan with results. Let us start with results and plan backwards.” At the outcome level, three out of five indicators stem from secondary data sources, where reliability and timeliness of data is an issue. For example, percentage change in NRW by utility is derived from data reported by utilities with a one year time lag and are limited by “the availability of reliable measurement tools (meters, billing software, etc.) and the availability of timely data.” Meanwhile, the indicator for “cubic meters of water provided or saved as a result of USG assistance (M-PMP 3.3.c)” is an estimation reliant on “water utilities capacity to report the accurate data and the accuracy of estimated saved water.” Stakeholders report that MWI often maintains two sets of data, one official and one unofficial. Although WMI collaborates with USAID/CITIES in achieving Indicator Sub-AO2.2, Engagement in Critical Water Scarcity Issues Strengthened, the modality of collaboration has yet to be defined. EVALUATION METHODOLOGY AND LIMITATIONS METHODOLOGY The evaluation employed a qualitative approach to answering the evaluation questions that included an exhaustive review of WMI project documents, research and special studies conducted by WMI, training curricula, and databases of information maintained by WMI. To maximize the utility of the evaluation to WMI and USAID, and in light of recent ministerial changes resulting in changes in WMI counterparts at MWIand changes in WMI’s scope, the evaluation team used a participatory planning approach that relied heavily upon WMI staff input for identification of key informants and stakeholders. Stakeholders interviewed included government officials, utility companies, NGOs, water user associations (WUAs), WMI staff, and other 20| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV stakeholder organizations. A purposive sample of KIIs and FGDs was selected to cover all utilities benefiting from WMI support. In addition, MWI, WAJ and JVA interviewees were selected based on their ability to speak to the various WMI interventions. The evaluation reached out to a total of 117 people in partner institutions, as follows: KEY INFORMANT INTERVIEWS AND FOCUS GROUP DISCUSSIONS TYPE AGENCY NUMBER OF KIIs CONDUCTED NUMBER OF FGDs CONDUCTED Government Officials MWI 9 0 WAJ 13 0 JVA 4 0 Ministry of Environment 1 0 Greater Amman Municipality 1 0 WDM Task Force WDM Task Force has been dissolved 0 USAID and Implementing Partners (IPs) USAID Water (EDE DO) 1 0 USAID Cities project 1 0 USAID WIT project 1 0 International Organizations IMF 0 (KII with WB instead) 0 IFC 1 0 KfW 1 0 GIZ 4 0 AFD 1 0 World Bank 1 0 Water Companies ZWA 5 1 (interns) YWC 18 1 (interns) Ma’an Water Authority 5 1(billing & collections depts) Miyahuna Water Company 8 0 Aqaba Water Company 4 0 Non-Governmental Organizations (NGOs) ACWUA 2 0 JREDS 1 0 Water User Associations (WUA) Attendees of WMI WUA event 0 1 Chambers of Commerce Businesses WUE and WDM 1 (Amman CoC) Not conducted as planned as there was very little interaction with CoCs WMI and Subcontractors Tetra Tech, Segura, JSMC 10 0 21 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV In-depth interviews were guided by semi-structured questions to capture perspectives and activities of each type of informant. The interview guides were designed to preserve the potential for a relatively free-flowing conversation, while creating a standardized format to facilitate a reliable, comparative analysis of data pertaining to the evaluation questions for triangulation of information from multiple stakeholder perspectives.While conducting interviews, utility observations were conducted using a structured checklist to facilitate triangulating data. Data collection was conducted from May 3 through June 11, 2018. A total of 117 people were interviewed through FGDs and KIIS, of which 27 were female and 90 were male. The full list of people interviewed is contained in Annex IV. The evaluation design, and data analysis approach are included in the Evaluation Design Report in Annex II, while the evaluation tools can be found in Annex VI. LIMITATIONS As the Minister of Water and Irrigation and ministry staff changed in March 2018, many of the MWI staff who were interviewed had limited knowledge of WMI. To the extent that political sensitivities allowed, this was mitigated by interviewing former MWI staff members, where those staff members were available and willing to participate in interviews. This was also mitigated by the engagement of WMI team in identifying key informants at MWI in current and past administrations with whom they had worked. As a mid-term evaluation, monitoring data reflect data only for the first year of performance; the nature of interventions and results are such that some indicators have a one-year time lag for reporting. This limitation was mitigated by triangulating KII feedback on perceived successes on impact, but the assessment of KIP-based performance over time remains an area for further investigation and assessment. The evaluation’s primary source of independent information is based on self-reported feedback from partners and beneficiaries, as well as various experts in the sector. Self-reported data reflects individual perceptions, which may or may not closely reflect reality. The evaluation did not include a structured survey, as its project implementation status to date did not lend itself to such a survey. To mitigate this limitation, the evaluation team conducted a large number of interviews and triangulated findings from multiple sources and used results of the desk review to ensure consistency in findings and results. FINDINGS AND CONCLUSIONS Although WMI operates through Components, the evaluation findings and recommendations lend themselves to presentation through a lens of Governance, Utility Performance, and Citizen Engagement. These categories reflect the key stakeholders in water management, as well as the relationships that connect them. Figure 3. Members of the evaluation team conducting field visits to YWC Customer Service Control Center. 22| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV In keeping with this best practice model, 6 the organization of the evaluation report follows the three broad categories of “Governance,” “Utility Performance,” and “Citizen Engagement,” with evaluation questions addressing Effectiveness, Sustainability, and Learning. GOVERNANCE FINDINGS: EFFECTIVENESS WMI is supporting the IMF plan to reduce financial losses in the water sector by introducing best practices in management across the utilities, including legal and institutional reforms. WMI is tasked to support development and implementation of staffing plan and strengthen MWI capacity in groundwater monitoring, data analysis and modeling, issuing and enforcing well licenses, and other responsibilities as they are transferred from WAJ. WMI is assisting in implementing Assignment Agreements (AA), an internationally recognized best practice that defines the rights and assets conveyed from WAJ toincorporated utilities, with associated terms and time period. WMI is also collaborating with JVA and the Jordan Cooperative Corporation (JCC) to build the momentum toward making WUAs a reality in the Jordan Valley. WMI is further tasked to work with MWI, WAJ, and Ministry of Environment (MoEnv) to streamline groundwater monitoring, assessment and management functions. WMI activities evaluated in this section include the following: 6 Adopted from: UNDP-SIWI Water Governance Facility & UNICEF, 2015. WASH and Accountability: Explaining the Concept. Accountability for Sustainability Partnership: UNDP-SIWI Water Governance Facility and UNICEF. Stockholm and New York: SIWI and UNICEF.http://www.watergovernance.org/resources/accountability-in-wash-explaining-the-concept/ Figure 4. Best practice model for water management. 23 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV 1.2 International Monetary Fund (IMF) Action Plan Support 3.1 Water Sector Policy Supported 3.2 Utility Assignment Agreements Developed/Modified 3.3 Divestment of Irrigation Management to Water User Associations (WUAs) Supported 3.4 Independent Regulator Developed and Supported 4.1 Groundwater Management Framework Developed 4.2 Wastewater Treatment Performance and Compliance Strengthened 4.3 Water Quality Management Improved WATER SECTOR POLICY In its original scope, WMI was tasked to “review and update policy matrix biannually at each IWSMG Steering Committee meeting and provide TA and training in Quarterly and Annual Reports.”7Upon request from MWI another activity was added to support development of results-based water policies action plans, and to support implementation of action plans in coordination with donors (GIZ in particular), and strengthen the institutional framework promoting adoption of action plans through WAJ mandates. In 2016 MWI issued a National Water Strategy (NWS) for which WMI has provided technical specialists to support the development of draft results-based water policy action plans; water policy action plans; NWS – General Action Plan; Water Demand Management Policy; Groundwater Sustainability Policy; Surface Water Utilization Policy; and Water Allocation Policy. MWI also worked closely with GIZ for the completion of the Water Substitution and Reuse Policy Action Plan8, Energy Efficiency, and Renewable Energy Policy Action Plans. Although elaboration of the draft Policy Action Plans took longer than planned due to changes in MWI management and MWI’s vision of the format, all stakeholders interviewed felt that completing the action plans is an important milestone. The concept note for the implementation of these draft action plans was drafted jointly with GIZ and is pending approval by MWI. The Action Plan results￾based approach relies on WAJ, PMU, and JVA and other stakeholders to provide data and on the operationalization of data exchange protocols with various stakeholders. These interventions are supported by an EU project working with NWIS. INDEPENDENT REGULATOR WMI is tasked with providing technical expertise to GOJ for operationalizing an independent regulator (some of whose regulatory functions are currently housed in the PMU) and/or water regulatory unit within a multi-sectoral regulator. Support was intended to include developing a new legal framework that ensures independence, increased transparency, and capacity building. In the 7USAID -278-C-16- 00001- SECTION C - PERFORMANCE WORK STATEMENT 8WMI staff report that the project has since ceased providing technical assistance to this initiative, since GIZ is already providing this assistance. 24| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV original scope, WMI was tasked with delivering consolidated recommendations for an independent regulator, facilitating a path forward with a budget, strategy, organizational structure, SOPs, and technical support for implementation. In the first year of implementation WMI focused on “introducing a clearer framework of regulatory provisions in amendments of Assignment and Development agreements (See AAs), aiming to support the financial independence of the unit inside the WAJ- PMU so it could become a self￾sustained, capable body inside the umbrella of MWI.”9 While WMI embarked on this intervention, MWI, WAJ and the PMU requested WMI cease efforts to create an independent regulator, as “it is not within the sector’s priorities for the foreseeable future.”10 As a result, WMI postponed the start of this intervention to FY19 and has requested support from USAID in getting it reinstated. The PMU monitors the performance of utility operators against the same period in preceding years but does not benchmark operators against each other.11 The PMU relies on the operators to furnish it with data, but the completeness, accuracy, transparency, and reliability of the data is of concern. Both WMI and GIZ note challenges in implementing reforms of the AAs including the lack of transparency in the regulatory framework, conflicts of interest on Boards, and conflicts of interest arising from WAJ being supplier, owner, and regulator of the utilities. In fact, the WAJ Secretary General (SG) is also the Director of the PMU. Stakeholders interviewed highlighted the governance challenge of WAJ “managing the utilities, rather than governing, and the challenge of Boards voluntarily giving up their rights.” While WAJ publicly supports improving the governance of the sector, there appears to be lack of clear commitment to modifications being approved. WMI intends to link utility reporting responsibilities to a regulator through revisions to AAs; however, the revisions cannot be made until regulator responsibilities are better defined. Additionally, stakeholders interviewed within the PMU indicate that there is no buy-in for WMI support to KPIs or to make them publicly available. While stakeholders within WMI, USAID, GIZ, WAJ, and MWI have divergent views on the urgency and timing of establishing an independent regulator, all stakeholders agree that the absence of a firewalled regulatory function poses challenges to improving accountability of operations of the companies; seven out of ten stakeholders felt that KPIs should be publicly available to exert pressure on operators to improve their performance. UTILITY ASSIGNMENT AGREEMENTS WMI is tasked to develop new and/or modified AAs between utilities and WAJ to increase utilities’ independence and their ability to act commercially. WMI is supporting WAJ, Miyahuna and YWC in drafting revised AAs and Articles of Association (AoA) to address gaps and barriers to performance improvement, and to establish mechanisms at utilities for monitoring and tracking compliance with the agreements. AAs are viewed as a cornerstone of governance reform by WMI, and their implementation and effectiveness are clearly tied to regulatory framework reform. WMI started by producing a concept note that evaluates AAs relative to international best practices. In collaboration with Miyahuna, WMI assessed the existing governance framework and implications of merging Zarqa and Madaba, and suggested amendments to the AA and AoA for Miyahuna. For YWC and Miyahuna, WMI drafted a Governance Analysis which was the basis for amending the AA and AoA for YWC. WMI also conducted meetings with WAJ, PMU, YWC, and Miyahuna to secure feedback, and conducted 9WMI Annual Work Plan FY 17. 10WMI Implementation Matrix. 11OECD (2014), Water Governance in Jordan: Overcoming the Challenges to Private Sector Participation, OECD Studies on Water, OECD Publishing. http://dx.doi.org/10.1787/9789264213753-en 25 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV training on the governance good practices for state-owned enterprises. The Governance Analysis Reports for AAs and AoAs for Yarmouk and Miyahuna Water Companies, are under review by the utilities. The proposed modifications to the AAs and AoAs focus on accountability, financial sustainability, and capital investment responsibility. According to WMI reports and KIIs, some proposed modifications have met with resistance, and stakeholders at utilities claim that their “hands were tied” with respect to implementing changes, thus delaying progress. Proposed modifications to the AAs and AoAs for Miyahuna and YWC have been presented to the Minister of Water and Irrigation, and progress has resumed. WMI’s scope included support to establish an internal charter for the first Citizen Advisory Boards (CABs) in Jordan at Miyahuna and Aqaba Water, however, MWI rejected the idea. Alternatives to CABs are not clearly identified as yet, but WMI is exploring the idea of involving the decentralization committees (through collaboration with USAID/CITIES) for citizen engagement. JVA OWNED O&M COMPANY AND DIVESTMENT TO WUAS WMI is tasked with providing technical support to the JVA to divest the retail irrigation management responsibilities to WUAs, developing a new administrative framework to establish and manage retail contracts and building capacity of WUAs. WMI conducted a pre-feasibility study which explored various models for restructuring O&M functions of JVA, concluding that “establishing an O&M company in charge of conveyance and distribution of the Jordan Valley irrigation system us a viable option that JVA should seek and implement.” Throughout FY 2017 WMI cultivated JVA stakeholder buy-in for the O&M company, however, in December 2017 the Minister of Water and Irrigation rejected the creation of an O&M company and requested that WMI instead strengthen existing WUAs through delegation of all the O&M functions, from intakes on the King Abdullah Canal (KAC) pump stations and gravity lines to the Farm Turnouts Assemblies (FTAs). Stakeholders interviewed believe that the Minister was likely not informed of the JVA O&M approach which had been circulated to JVA stakeholders over the course of 2017. According to stakeholders interviewed, management changes at JVA in June 2018 resulted in JVA revisiting the idea of the O&M company with particular concerns about the capacity of WUAs to assume additional responsibilities. In order to clarify direction, the Acting Secretary General of JVA and the Minister of Water and Irrigation requested WMI and GIZ to analyze the effectiveness of the WUAs. Responding to the vision set by the former Minister focused on increasing the mandate and role of the WUAs, and informed by the KAC loss assessment, WMI is supporting the amalgamation of six WUAs in the North to pilot farmgate-level smart meters and upgrade the control center to improve data flow and amending KPIs of WUAs. This initiative is being supported with GIS mapping, and GIZ is piloting a similar initiative in the South. GIZ’s pilot advocates for the adoption of an Irrigation Management Transfer (IRT) Action Plan while WMI’s pilot does not provide clear support for institutional governance/strengthening, according to stakeholdersAt the time of the evaluation, WMI and JVA had a tentative agreement on the scope of WMI assistance. IMF ACTION PLAN WMI is tasked with supporting the achievements of MWI’s plan with the IMF, approved by the Council of Ministers, to reduce losses in the water sector by improving the financial sustainability of the sector though technical assistance and roll-out of improved best management practices across the utilities. In FY 2017 MWIprepared a Structural Benchmark Action Plan to Reduce Water Sector Losses to address IMF requirements, which define actions, schedules, and expected results in terms of cost recovery through increasing revenues and reducing costs. • Secretariat Assistance: WMI supports reporting by compiling data and writing reports. All stakeholders interviewed within MWI noted how this assistance contributes to quality and timeliness of reports. WMI states that the efficiency of IMF support is hindered by weak coordination and communication among WAJ, MWI and JVA. 26| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV • PBC and PSP Support: WMI has completed the “Support Implementation of PBC and PSP to Improve O&M Services Stage 1 - Exploration & Identification of Public-Private Partnership (PPP) Opportunities” which recommends selecting at least three PSP Projects from the following categories: Improvements in energy efficiency throughout the sector/generation of renewable energy; Renewal and maintenance of existing water & wastewater treatment plants (WWTPs); and NRW. While stakeholders indicate buy-in for PSP promotion, six of seven stakeholders questioned the merits of WMI tackling all categories. Stakeholders noted thatNRW has most the tangible value with respect to learning/demonstration,and that energy efficiency (EE) is actively supported by other donors such as GIZ. Stakeholders also noted that KfW and AfD are supporting the waste water treatment sector, and while they believe PSP in WWT is worthsupporting, it should involve an innovative solution such as sludge management and be more closely coordinated with donors. Support to implementation of PBC/PSP is behind schedule, with seven of nine stakeholders attributing the delays to inefficiencies of counterparts and insufficient staffing of the Governance team at WMI. • Support to NRW Reduction Strategies: Modifications to WMI’s original scope added support for NRW reduction strategies inclusive of drafting a NRW reduction master plan; preparing case studies of best practices in Jordan and region; conducting a knowledge sharing workshop on the best practices to reduce NRW in Jordan; and providing short-term technical assistance to the individual utilities. WMI has contracted the Arab Countries Water Utilities Association (ACWUA) to develop the NRW reduction master plan strategy; to conduct a knowledge sharing workshop; and to conduct water utility management training. Beyond the development of a draft approach, implementation has been delayed due to issues with procurement and delays in forming an Expert Working Group at WAJ. The USAID-imposed deadline for the development NRW Action Plan is November 2018. • Improve the Financial Management, Determination of Cost and Subsidy of Services: WAJ stakeholders perceive WMI support for the development of concept notes to inform decision￾making on cost-recovery as useful. As a result, WAJ has given a verbal commitment to implement certain improvements in the next financial year. The Concept Note on the Financial Accounting and Fixed Asset Accounting Policies and procedures for managed assets is in early stages, and WMI has identified several areas where WAJ’s accounting is not consistent with international accounting standards. In relation to supporting WAJ’s FARA assessment, WMI has developed a table highlighting WAJ’s status relative to risks outlined in the “Public Financial Management Risk Assessment Framework (PFMRAF).” While all stakeholders express appreciation for work on the PFMRAF, there is as yet no progress in consumers’ representation at company Boards, and there is little buy-in among government actors for consumer representation through Citizen Advisory Boards. Support to analysis and implementation of tariffs is likely to be cancelled at the request of WAJ, and similarly support to WAJ in debt management is on hold.12 • Support to IT and GIS: According to WMI, all activities supporting the IMF action plan “require significantinformation management capabilities that can only be met with advanced IT and GIS systems.”13WMI has facilitated the formation of the IT Strategy Group whose primary objective is to harmonize IT systemsand use. According to stakeholders, the IT Strategy Group has helped movement towards a common IT strategy. WMI has drafted an initial IT master plan; a draft report on “Available Solutions for Hosting Oracle Applications Suite for WAJ and YWC, and Miyahuna Business Process Review (BPR),” with drafts currently under revision.14 Enhancing the 12 According to WMI team members, WAJ has since agreed with Ministry of Finance to handle the debt management. 13 WMI. Annual Work Plan: October 2017-September 2018 (FY 2018), p, 20. 14 According to WMI, the contract start date is August 9 2018. 27 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV capacity of WAJ’s ICTU data center (including through WAJ’s Enterprise Resource Planning (ERP)upgrade/re-implementation) is still at the concept stage. Meanwhile, promoting GIS in the water sector has started, with an assessment complete and a pilot started at MWA. Groundwater Management WMI is tasked with supporting the GoJ’s implementation of a new legal framework for groundwater management to reduce water theft and groundwater over-extraction. WMI established the MWI Groundwater Task Force to review groundwater monitoring practices and policies. While not a part of the original scope of the committee and initiated by WAJ board, an outcome of the activity was the presentation of options to improve the management and licensing of wellsand feasibility study of establishing a business unit for private wells. Following the presentation, WAJ prioritized the creation of a business unit for private wells, explaining that it is “one of the most important interventions for WAJ and it will help WAJ to double the annual revenue from 20 to 40 million JOD.” WMI is further supporting improved groundwater governance through a series of assessments for MWI and WAJ, which are currently in draft form. In an effort to improve groundwater monitoring standards, WMI planned to embark on a smart meter pilot in the Amman-Zarqa Basin to remotely read and measure abstractions from each well, selecting 20 wells to implement the pilot project and to supply 20 smart meters. However, before the pilot could be implemented, WAJ decided to directly purchase 241 smart meters without implementing the WMI pilot project and its associated smart meters, and WMI then instead preparedtendering documents for the procurement of the smart meters. This was combined with on-the-job training for remote sensing which stakeholders at WMI and WAJ considered to be effective. The World Bank is also actively supporting remote sensing which presents opportunities for coordination. To support Water Safety Planning (WSP) governance functions at WAJ, WMI conducted an assessment on WSP practices at the national level. Action plans were approved by the steering committee for the regions of Tais Spring, Kufranjeh Dam, Zara-Main, WadiEsser, and Mujib Dam. WMI supported external consultants to conduct water audit training which has produced a national water audit team and documentation for good practices titled “Institutional Set Up and Arrangement Plan for Implementation of WSP in Jordan.” WMI has been working with the Water Authority of Jordan (WAJ) and other water utilities to improve water quality risk management. In December 2016 WMI prepared a report onGoJ practices related to the Water Safety Planning “WSP.” The WMI intends to work with MWI/WAJ and water utilities to institutionalize WSP practices within the water sector in Jordan and to develop and support an action plan for WSP implementation. An action plan was developed and initiated in April 2017 to implement Water Safety Plan Approach (based on WHO guidelines) in four water systems located in Amman, Jerash, and Karak governorates. Three technical teams have been formed to undertake activities of the WSP action plan supported by WAJ Labs. WMI conducted four workshops in April 2017-April 2018, and aims to develop the final WSP documents by end of 2018. In an effort to support wastewater enforcement capacity, WMI prepared an evaluation report titled “Water Quality & Reclaimed Water Monitoring Practices in the Ministry of Environment” and conducted an assessment for the MoEnv capacities in GIS. Thus far a work plan has not been shared with the Ministry and no activities on this intervention have begun.The MoEnv and WMI do not share a commonunderstanding of the reason for the delays in progress. CONCLUSIONS: EFFECTIVENESS INDEPENDENT REGULATOR Progress towards achieving improved governance objectives is significantly hampered by the limited buy-in among some stakeholders who have a vested interest in preserving the status quo by holding multiple positions within MWI, WAJ, and utility Boards. Until governance is improved, the utility performance monitoring is unlikely to improve, with possible consequences on performance 28| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV improvement. WMI and other donors may have a renewed opportunity to see increased buy-in with the new water sector administration to pursue to establishment of a firewalledregulatory unit as an interim step, and move towards an independent regulator when service providers are fully privatized. While there is potential for great synergies between GIZ and WMI, the need for clearer delineation of roles between WMI and GIZ in supporting this agenda is contributing to some inefficiencies with respect to duplication of similar activities, differences in approaches, and confusion among stakeholders. RBAPS WMI’s approach to RBP Action Plans development has been effective in helping to translate the eight NWS water policies into plans that are ready for implementation by MWI units. The participatory approach in RBP Action Planning actively engaged MWI though working groups that included technical assistance from GIZ and WMI. Nevertheless, the results are ultimately contingent on both organizational and technical capacity to implement these plans. To this end, focus should be on building leadership capacity and technical capacity that cascade down into business units for each entity of the sector, in alignment with WMI scope of “strengthening the institutional framework promoting the action plans to be adopted in WAJ board mandates to all utilities’ boards and from these to company mandates to be implemented through company’s business plans.” Effectiveness of RBAP monitoring can be maximized by ensuring simultaneous implementation of data systems strengthening (WAJ ICTU support to GIS and IT, as well as EU-supported NWIS). UTILITY ASSIGNMENT AGREEMENTS While the AA modifications that have been recommended have generated support from MWI and WAJ, there is some resistance by companies (to varying degrees) which has led to delays in adopting amendments. Unless the drafts of revised AAs are adopted, project objectives are unlikely to be reached. While Citizen Advisory Boards (CABs) were rejected by MWI, alternatives for citizen engagement in governance are not yet in place, thereby negatively impacting WMI’s best practice-based approach to citizen engagement in AA reform. Unless the role of citizen participation in decentralization committees through decentralization committees or other mechanisms is defined, there is a risk that WMI not meet its citizen engagement objectives. DIVESTMENT OF IRRIGATION MANAGEMENT The potential effectiveness of WMI support to Divestment of Irrigation Management to WUAs continues to be affected by the lack of clarity among stakeholders (and JVA in particular) for the mandate of the WUAs, and the weak buy-in in relation to the WMI assistance package being discussed. The situation hinders WMI from developing comprehensive interventions to support institutional and governance arrangements as intended, and instead relegates support to focus on technological interventions to support improved data flows (which is needed but not sufficient).The approaches ofGIZ and WMI are not entirely aligned, which could result in conflicting advice on non￾technology aspects of WMI assistance, particularly as GIZ is advocating for the adoption of an Irrigation Management Transfer (IRT) Action Plan. WMI’s effectiveness in supporting divestment to WUAs continues to be affected by the lack of clarity in the JVA vision on the WUA mandate. IMF ACTION PLAN SUPPORT While support to the IMF Action Plan reporting has been effective for bi-annual reporting to IMF, it could benefit from increased institutionalization and knowledge transfer of technical assistance. Although still in review stages, studies supporting WAJ in improving Cost Recovery and Financial Accounting and Fixed Asset Accounting Policies have been well-received and seem to have 29 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV generated buy-in for improving operations. Next steps in support could focus on fleshing-out clear goals and milestones for their achievement (scope and time). In light of ongoing investments and donor work within each subsector WMI support to PSP would be most effective if focused on one or two interventions: NRW related PBC in particular. Unless WMI accelerates work on NRW master workplan, there is a risk that the master workplan will not be ready for the USAID-imposed November 2018 deadline. Meanwhile, the cancellation of the tariff analysis (together with affordability tests to ensure that an increase in cost recovery is feasible) and the postponement of the advisory on debt management are likely to negatively impact the effectiveness of both interventions.15 Effectiveness of IMF action plan support can be maximized by ensuring simultaneous implementation of data systems strengthening (WAJ ICTU Support to GIS and IT)and EU-supported NWIS, with WMI amending the current NWIS with links to utility level data. GROUNDWATER MANAGEMENT The high level of buy-in for establishing a business unit for private wells inside WAJ may be the result of partner engagement through technical assistance, inclusive of the feasibility study for establishing the private wells unit, the assessment of annual revenue through the licensing systems for private wells, and support to tendering for smart meters. The potential for contribution to improved standards and reduction of illegal use will require trend analysis of monitoring data and requires WMI follow-up to ensure effectiveness and sustainability. GIS and water quality monitoring seem to have stalled, and expectations on the path forward seem to be unclear to partnering stakeholders. These activities should be prioritized in the coming period. FINDINGS: SUSTAINABILITY To date WMI has not developed sustainability plans for interventions in governance. INDEPENDENT REGULATOR Although the GoJ expresses commitment to improving the sector regulatory framework (as noted through the National Water Strategy and Jordanian Water Policy Dialogue), the demonstrated lack of buy-in and stop-order by former Ministers has impeded progress. Both progress and sustainability of results in the regulatory environment are dependent upon buy-in at the senior levels of MWI and WAJ. WMI has mapped-out its interim and ultimate vision for sector reform, where the regulator is added, (as illustrated in Figure 4) however this vision has not yet been shared with MWI and WAJ as it is still under review with USAID.16 15OECD (2014), Water Governance in Jordan: Overcoming the Challenges to Private Sector Participation, OECD Studies on Water, OECD Publishing. http://dx.doi.org/10.1787/9789264213753-en, p. 60. 16 This vision is planned for presentation to MWI within WMI’s three year plan. 30| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV Figure 5. WMI's Interim and Ultimate Visions for the Restructuring of the Water Sector. 31 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV Sustaining the improved performance results and accountability within the sector is dependent upon progress within the regulatory environment of the sector, which can only occur if there is buy-in among GoJ and sector stakeholders. Water Sector Policy: WMI pursued aparticipatory approach through working groups and close collaboration with GIZ to produce final drafts of eight Results Based Policy Action Plans (RBAPs) that provide guidance to the water sector. Although the process of elaboration of the Policy Action Plans took longer than planned due to changes in MWI management and MWI’s vision of the format, there is consensus among stakeholders interviewed that action plan completion was an important milestone. Stakeholders indicate strong buy-in for the implementation concept note but point out that implementation will require capacity building of the units responsible for implementation, particularly the Policies and Strategies Unit of MWI. Accordingly, WMI plans to conduct an institutional and technical capacity assessment provide assistance to MWI staff to develop and monitor the implementation of the RBAP. Utility Assignment Agreements: WMI approaches to securing buy-in for AA reform have included training and discussion of international best practices. While this approach has mobilized some support by WAJ and MWI to reform AAs, some proposed modifications were not perceived as important by utilities, which resulted in resistance to reforms. While WAJ is supportive of improving the governance of the sector, there seems to be no definitive commitment to see modifications approved. Both WMI and GIZ and the interviewed sectoral expertsbelieve that sustainability of reforms will depend upon governance and regulatory reform, of which the adoption of the revised AAs is the first important step. Citizen Advisory Boards (CABs) were rejected outright by MWI and utility management, and interviews with MWI key informants reveal that there is little to no buy-in for citizen engagement due to a perception that citizens are unable to engage in governance in a meaningful manner. Divestment of Irrigation Management to Water User Associations: While the approach to reaching stakeholder consensus on divestment of irrigation management was participatory and actively engaged key actors and decision-makers, the early anddirect engagement with the Minister of Water and Irrigation impacted the ability of WMI to ensure full buy-in. While the vision put forth by the former Minister focuses on a larger mandate and larger role for the WUAs, there is no clear consensus on the vision for WUAs’ role at JVA. JVA senior managers have expressed concern over entrusting more functions to WUAs due to weak WUA capacity to handle the functions to be transferred. This has also been compounded by unclear strategy of GIZ support to WUAs, leading to uncertainty and stagnation in decision-making. In May 2018 the Acting SG of JVA and the Minister of W&I requested WMI and GIZ to analyze the effectiveness of the WUAs to inform decision-making, while the new JVA leadership (June 2018) expressed potential interest in revisiting the O&M company mechanism. There is less than one year (March 2019) until WUA contracts will need to be renewed. IMF SUPPORT Secretariat Support: According to WMI and key informants at the MWI, secretariat support amounts to compiling the necessary data and report writing by WMI staff for bi-annual submission to the IMF. While MWI expresses appreciation for the report writing, MWI notes the need to improve knowledge transfer to its staff for institutionalization. WMI has plans in place to better institutionalize this secretariat support. Secretariat support is not as yet linked to the Strategy and Policies Unit of MWI but is planned by WMI and MWI. MWI’s preference for knowledge transfer is through on-the job training or secondment of an expert. 32| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV WAJ ICTU/ERP: According to WMI and WAJ, the IT Strategy Group has helped to move stakeholders towards a common IT approach and has generated significant buy-in for the ICTU ERP upgrade. WhileWMI has allocated a budget for hardware for the WAJ ICTU ERP upgrade, it covers only part of the financial needs; the source of remaining funds needed is not yet clear. PSP and PBC: Stakeholders indicate that WAJ is committed to promoting PSP. Of the three subsectors, Non-Revenue Water (NRW) was noted as most relevant by all but one stakeholder interviewed for its tangible value; as GIZ is supporting energy efficiency it had lowest priority for WMI assistance. Moreover, stakeholders reveal that there is little consensus over the prerequisites needed for PSP promotion in the water sector in Jordan, and the trade-offs and alternative scenarios to meet the costs are not clearly identified. NRW Strategies: While there is commitment from WAJ senior management to form the experts’ group and appoint a Head of the NRW Unit, this had not happened at the time of the evaluation. Both WMI and counterparts express concern about risks to sustainability related to large scale and systematic implementation of NRW action plan. Concerns revolved around weak incentives by the utility companies,partly stemming from the high and inexplicit subsidies for bulk water. GROUNDWATER MANAGEMENT While a technical proposal for building MWI and WAJ capacity in remote sensing is underway,WMI currently trains MWI only through on-the-job training; at the time of evaluation, there was no training plan to build the capacity of WAJ.17 Internet capabilities at MWI and WAJ could be improved to provide more efficient services to support remote sensing. The high level of buy-in for the Private Wells Business Unit bodes well for sustainability, as does the perceived success of the WSP training and interventions. WAJ is interested in moving forward with implementation of WSP within utilities but is adamant that WSP should act as a tool for implementation of Jordanian standards and regulations rather than a replacement to existing standards and regulations. Given the need for improved GIS infrastructure of servers and computers at MoEnv, WMI interventions supporting GIS and water quality monitoring at the MoEnv are unlikely to be sustained without financial support to GIS infrastructure. CONCLUSIONS: SUSTAINABILITY RBAPs: WMI’s participatory and collaborative approach to development of RBP Action Plans has supportedthe high level of buy-in by MWI, thereby enabling WMI to focus implementation on capacity building of WMI units (and Policies and Strategies Unit in particular) in the next stage. Utility AAs: Unless buy-in from YWC,Miyahuna and WAJ is secured, and the revised AAs are approved and adopted, sustainability is at risk. AA reform is tied to regulatory reform, and as such these reforms need to be simultaneously implemented. Sustainability of AA reform is also dependent upon addressing conflicts of interest at the Board levels between utilities and WAJ, as well as addressing the low level of buy-in for citizen engagement in governance(as part of revised AoAs). Divestment to of Irrigation Management:The potential sustainability of approaches to divestment of irrigation management to water user associations have been negatively impacted by the lack of 17 According to WMI, the new workplan aims to train WAJ on reading RS maps. 33 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV consensus on a common vision, as well as the perceived insufficient capacity at WUAs to assume irrigation management functions. JVA’s lack of clarity regarding the vision of reform of WUA mandate, together with the large financial gap in the sector (to be assumed by the GoJ), can negatively affect the sustainability of the upcoming WMI interventions. IMF Secretariat Support: While the secretariat support is helping to ensure MWI meets its reporting obligations, there is a need to transfer knowledge to MWI staff. While this is already planned, knowledge transfer activities are likely to be most useful to MWI through secondment with a gradual handover of responsibilities, increasing the likelihood of sustainability. WAJ ICTU: While the high level of buy-in for the ICTU ERP upgrade increases likelihood of sustainability, insufficient funding for the hardware may put sustainability at risk. PSP/PBC: While there is clear buy-in by WAJ for promotion of PSP, prioritization of the subsectors to be supported by WMI could help to enhance the likelihood of sustainability. Simultaneously, mapping and coordination with the various actors in the sector could help better prioritize the PSP opportunities, the approach of WMI support to PSP, and its potential sustainability. NRW Strategies: NRW Strategies are unlikely to be sustained unless stakeholder concerns are addressed. UTILITY PERFORMANCE FINDINGS: EFFECTIVENESS The original SOW tasked WMI with assisting the GOJ in achieving cost recovery and NRW reduction by providing intensive, tailored support to Zarqa Water Authority (ZWA) and the Yarmouk Water Company (YWC), which have the greatest potential for contributing to this goal. During the first two years, additional activities were added to WMI’s tasks: supporting Ma’an Water Administration (MWA) and assistance through providing tools, equipment and systems for all utilities. WMI activities evaluated in this section include the following targets: 1.1: Zarqa Key Performance Indicators (KPIs) Improved 1.3: Yarmouk Financial Performance Improved 1.4: Ma’an Operational Performance Improved 4.2: Wastewater Treatment Performance and Compliance Strengthened ZARQA KEY PERFORMANCE INDICATORS (KPIS) IMPROVED Under this target, WMI seeks to improve annual KPIs at the ZWA through the implementation of improved management practices, especially those related to NRW reduction and cost recovery increases, complementing the MCC Compact. In light of Miyahuna’s status as the manager of ZWA, WMI focused the activities under this target on both ZWA itself and Miyahuna. Performance Improvement Plan (PIP) and utility KPI targets: WMI developed the PIP and KPI targets for ZWA; however, none of the stakeholders interviewed within ZWA or Miyahuna had seen the PIP and KPI targets, and there is no Strategic Planning Unit within Miyahuna or ZWA to manage such a plan. 34| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV Planning activities with Miyahuna resulted from informal dialogue with upper management rather than on a documented plan. The biweekly steering committee meetings, which were meant to allow for joint planning, did not cover all interventions, and stakeholders reported that individual interventions such as the ERP system installation do not have designated points of contact within Miyahuna and WMI. Implement PIP interventions to reduce administrative NRW losses: Although WMI has begun work on all five implementation deliverables, interventions aiming to reformulate SOPs have been delayed. Stakeholders at ZWA report that WMI gathered data on policies and procedures but subsequent steps such as a report or training have not materialized; ZWA staff are unaware of the status of this activity and formulation of SOPs,while WMI staff report that new drafts of SOPs were sent to them by April 2018. The delay in this activity may be attributed to a request from MWI that WMI halt some interventions within ZWA until final agreement is reached on whether Miyahuna will take full ownership of the utility, or whether it will simply extend its management contract.18 Double Connections Survey: Senior management at ZWA and Miyahuna expressed appreciation for the work done thus far covering nine of 31 District Meter Zones (DMZs) and look forward to receiving a follow-up report that will cover all the 31 DMZs affected by double connections and calculate figures for revenue lost due to double connections. At the time of the evaluation, this follow-on study was on hold due to an outside request for prioritization of a different report.Senior management from both utilities emphasize that the study should be used directly by WMI to draft a TOR for disconnecting the old network. Increasing Billing Percentage: WMI assisted ZWA in collecting missing customers’ data in order to increase billing from 91% of the customer base to 98% at the end of March 2018. According to WMI analysis, the increased billing ratio has resulted in a drop in NRW by 5 million cubic meters during FY 2016/2017,19 thereby saving over 2 million cubic meters of water. All ZWA officials interviewed recognized WMI’s support as a factor in the increase in billing ratio, although attribution of the increased ratio is unclear since two of five stakeholders identified the financial incentives that WMI enabled for Jabis as a more significant driver of improved performance. Implement PIP interventions to operationalize financial systems: As preparatory work for the implementation of an integrated ERP system, WMI is currently working with Miyahuna on re-engineering its financial processes. Currently Miyahuna is using several isolated systems in the management of its financial processes, which negatively impacts data updates, cost-recovery calculation, and ultimately the NRW rate due to low accuracy. Miyahuna senior management consider ERP a suitable solution for increasing performance and consider re-engineering Miyahuna’s financial processes a necessary step before ERP deployment. The re-engineering process is expected to last for six months and is being managed by WMI and Miyahuna through joint steering committee meetings and a time-bound work plan that continues through the roll-out of ERP Phase 1. Senior staff at Miyahuna express the need for a time￾bound work plan for Phase II roll-out as these modules play an even larger role in improving company efficiency. “Miyahuna will not benefit until the ERP reaches Phase II,” voiced one senior staff member directly responsible for operations. Implement PIP interventions to outsource service functions: While the WMI Performance Scope calls for this activity to begin in Q6, work has not begun due to a request from MWI to refocus efforts in 18Implementation Issues Matrix, p. 66 19 The time lag in NRW reporting captured here demonstrates the difficulty in capturing and attributing performance improvements such as billing interventions in 2018 to NRW savings data linked to 2016/17. 35 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV ZWA on the pre-feasibility study for merging with Miyahuna. Stakeholders at ZWA report that they were not told why some sub-activities were paused while others continued. Develop customer service-focused business plan: Similar to outsourcing, this activity was scheduled to begin in Q6; however, work has not yet commenced. ZWA senior staff were told that WMI was going to upgrade the X7 system at ZWA but were not informed that the initiative is on-hold until further notice. Pre-Feasibility analysis and implementation of merger of Zarqa and Madaba into Miyahuna: With the merger of the Madaba utility into Miyahuna complete, Miyahuna senior management no longer see a need for WMI support related to Madaba. Regarding the ZWA merger however, Miyahuna management has requested that WMI provide detailed figures for projected losses and subsidies resulting from a merger before making a decision. These figures were not included in the first Pre-Feasibility Analysis, but WMI has provided them in a new draft that was in the approval process at the time of this evaluation. YARMOUK FINANCIAL PERFORMANCE IMPROVED Under this target, WMI was tasked with providing direct management and technical support to YWC to implement institutional reforms and performance improvement measures to move toward commercial viability and cost recovery. The performance and achievements of activities are as follows: Develop Interim Performance Improvement Plan and Support Implementation: WMI progress reports and stakeholder interviews with YWC management confirm that the Interim PIP for the year 2017 was developed and included KPIs, a WMI Work Plan, and approval from the YWC board. WMI hosted senior and mid-level managers, as well as Board members, from YWC at the Dead Sea for two days of workshops to draft the plan. After the planning exercises, however, stakeholders within YWC reported that support to interim PIP implementation did not meet their expectations. According to a member of upper management, the PIP was not implementable since it was not linked to YWC’s budget and would lead to a deficit of about 8 million JOD. Five mid-level managers reported that they did not receive copies of the PIP; one of these managers stated, “The training was for WMI to ask for data and for the WMI team to create and share the PIP with upper and middle management, but they never came back to me and never shared the PIP.” WMI staff expressed a different viewpoint of the Interim PIP process, with one staff member expressing that the development of the PIP was intended to be a training exercise to teach YWC staff about best practices for creating a PIP and that the Interim PIP was not meant to be implemented. Piloting the Operation Control Center in Jerash: WMI’s SOW charges the activity with piloting an Operation Control Center for Jerash and Ajloun with the potential for scaling-up. WMI intended to strengthen the complaint response and repair functions of YWC in this area by creating a centralized office linked to handheld GIS-enabled units. In order to do this, WMI directly funded the creation of a small office with computers and desks for two people and ten tablets for responders to use to report data in real time while responding to complaints. Stakeholders from within both YWC and WMI report that this idea was shown to the Minister of Water and Irrigation in a simulation before any real-world implementation of the Center’s new capabilities occurred, yet the Minister ordered the implementation of the new Center idea to be stopped so that WMI could instead incorporate this idea into Miyahuna’s forthcoming nation-wide Operations Control Center. As such, the tablets have been placed in Jerash storage and the new complaint response mechanism within the Jerash Remote Operation Unit (ROU) is dormant. Establishment of an Internal Audit (IA) unit: WMI has developed an organizational structure and standard operating procedures for the new unit, trained nine IA staff, and developed an annual IA plan 36| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV to guide the department’s day-to-day functions. IA staff reported that WMI sent a full-time consultant to develop the annual IA plan hand-in-hand in a collaborative process with them. Despite a delay in the third part of the training course due to miscommunication, the team has been well-equipped through WMI training and WMI has paid for nine staff members to complete an international certification as Certified Internal Auditors. These interventions allowed for the development of the IA unit, but the unit is still unable to operate since the Board of Directors have yet to formally approve the establishment of the Unit and the Unit’s Annual IA Plan. Commercial Customer Service Center: WMI has established the Center as a unit within YWC, ensured staff is hired and trained, moved a caravan from a YWC ROU to the back of the YWC headquarters in Irbid, and furnished the trailer as the Center’s office space, including equipping it with hardware through the Component 5 Activity funds. The site visit of the evaluation team revealed that staff are using training manuals and procedures provided by WMI in their work but that the Center is not yet fully operational a few months after it opened because it is not yet linked to the data storage and reporting functions of the X7 system. The team there is scheduled to be trained by a seconded WMI consultant for a period of six months on the X7 system beginning in the second half of 2018. The management of this unit also consider buy-in from top management and the Board of Directors as a barrier since the first quarterly report generated by the unit has not been allowed to circulate and the unit as a whole has not yet been approved for functioning. Implement PIP interventions to reduce administrative NRW losses: WMI has introduced a new X7 Customer Information System for digitized processing of billing and collection. To date, only the Billing module of this system has been rolled-out while the Collection module will be rolled-out at a later date. Nineteen of the 20 stakeholders interviewed within YWC see the X7 modules as essential to better business processes, but these stakeholders also voiced that the process of data cleaning, end-user training, and incorporation of digital processes was too rushed and that WMI imposed a roll-out date that was too early. In the case of the Irbid ROU, management sees the new X7 system as responsible for diminishing collection rates from 90% in 2017 to about 70% one month into its implementation (May 2018). The training program used for X7 trainedonly key users, but these users expressed that the classroom-only training was not enough to prepare them to assist other users. The expansion of training to all end-users did not occur until early June 2018, months after the roll-out of the Billing module. Further, there is a lack of printers and other hardware necessary for use of this system, especially in field ROUs. Field managers requested that MWI send a staff member to sit in each ROU in order to provide the opportunity for question-and-answer based training, and HQ staff reacted positively to having Ahmad Saadoun as full-time seconded WMI staff at the main office. Implement PIP interventions to financial management systems: Over the course of the project thus far, WMI has established a new version of YWC’s Financial Policy, which received approval from the YWC Board of Directors at the end of 2017. WMI plans to conduct detailed organizational assessments of business processes within YWC and build the capacity of financial personnel in international best practices in the second half of the project. WMI management views the ERP as the first step before this can be completed: “The ERP is our prep work before we can begin our real work,” comments one manager. While field visits confirmed that the upgraded system has been rolled-out, with seven of nine accounting and finance employees actively using the upgraded system, some problems were frequently mentioned regarding its use. Although Leading Edge Technology (the implementer for the ERP system) trained “super users” and WMI contracted an Application Manager – MohannadAbuNajm – to provide technical support within key departments of YWC, seven of nine ERP end-users told the evaluators that they did not know where to get assistance with technical issues. YWC and WMI did not sign a written agreement to cover costs associated with prolonged software licensing and troubleshooting from 37 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV Leading Edge and to provide devices for the use of the ERP system (including printers and computers), leading to tension and delays as the two parties negotiated these expenses as they arose. HR Assessment and Implementation: WMI’s interventions regarding HR in YWC are directed towards creating a new organizational chart with accompanying job descriptions, policies, and procedures. At first, WMI developed an organizational chart through the use of a consultant charged with redesign based on a secondary review. All relevant YWC stakeholders interviewed found this approach to be less helpful than the approach used by a second WMI consultant who came later and met individually with each upper and mid-level manager to discuss restructuring their departments. This later effort has resulted in a new organizational chart that goes down to the operational level below middle managers, thus providing a deeper picture of the organization than have any of its previous charts, which only went to the middle manager level. The new organizational chart has not yet been approved by the Board, but all interviewees are hopeful that it will pass. Job descriptions and SOPs are perceived as the next steps by YWC middle management and HR staff, who consider the organizational chart only part of a solution for smoother operations. Regarding HR policies, WMI conducted a review of the HR policies, provided targeted financial assistance for printing 300 copies of HR material, and coordinated meetings between upper management and staff to explain changes in the HR policy.. This approach allowed HR staff to circulate hard copies to all upper and mid-level managers for their review and approval. HR management within the company takes credit for changing some lines within the policy prior to printing to adhere to equal opportunity standards for female employment. The new HR policy has been approved by the Board. Fixed Asset Registry & Valuation: WMI is supporting WAJ Headquarters and YWC in conducting an asset survey, the data from which will ultimately be uploaded into the Fixed Asset Module of the ERP system. The survey had been completed for Jerash and Ajloun, and was 70% complete in Mafraq at the time of data collection20Senior staff in Jerash and Ajloun stated that they provided Excel sheets to WMI and are looking forward to accessing the asset data via the ERP system after WMI provides it, but are unaware of the status of the intervention. All stakeholders interviewed at YWC and WMI indicated that the data has not yet been uploaded on the Fixed Asset Module. The data must be approved by WAJ before it is uploaded into the Module. Support for the New YWC Headquarters: WMI is supporting a procurement committee within YWC to tender for the design and construction of a new headquarters that will meet green building standards. TheTOR for design of the building was tendered through an iterative process involving drafting by WMI technical experts and review and feedback by YWC staff. All YWC committee members interviewed expressed appreciation for the technical assistance and stated that their involvement within the process increased their knowledge of best practices for tendering. None of the stakeholders within WMI or 20 According to the WMI team, this was subsequently completed at the end of July 2018. Figure 6. HR booklet on a stakeholder's desk at YWC. 38| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV YWC know where the funding will come from for the actual construction of the building once a design is chosen. Overall Approach: All of the YWC sub-activities except for the Green Building TOR are managed by a small steering committee comprised of WMI managers and the General Manager, IT Manager, and HR Manager of YWC. Upper and mid-level managers from other departments within YWC attend on a case-by-case basis depending upon the topic of a given steering committee meeting. Minutes from the four most recent meetings show that the meetings were for the most part biweekly, but that attendance beyond the three core YWC staff members was irregular and did not cover covered anyone from ROUs. MA’AN OPERATIONAL PERFORMANCE IMPROVED This target was an addition to the original scope of work to support the merger of MWA with AWC through the following activities: Situation assessment and preparatory work for transfer to AWC: After developing a situation assessment for Ma’an, WMI has worked on the following support to MWA: 1) Provision of 20 hand-held units (HHUs) to Jabis and piloting the use of barcodes with HHUs; 2) Establishment of a GIS Unit within MWA;and 3) Piloting the use of new pulser technology. All MWA managers interviewed reported that the HHUs increased the efficiency of billing within MWA by recording the geographic location and time of readings by Jabis in addition to the data itself. “HHUs supported office managers in managing Jabis,” stated one member of upper management, “and this ability to manage Jabis more effectively led to an increase in revenue by 5%. Reading and billing reached 98%.” This statement is consistent with WMI’s report that the HHUs helped to produce a decrease in reading process duration of 38%. All five Jabis interviewed during a field visit expressed that they regularly use HHUs and that they are able to receive technical support when needed. One element of the HHU intervention that received less positive feedback was the usage of adhesive barcodes on meters. While these barcodes were supported by MWA management as useful tools for systematization, the quality of the material resulted in barcodes peeling off of devices. The GIS Unit stakeholders from MWA are enthusiastic about the support they received from WMI. This support includes training on GIS and the creation of a road map with policies and procedures which were observed to be in use during a field visit despite the fact that the Unit is in its earliest stages of functioning and still lacks hardware. The pilot on pulser technology was recognized by MWA managers as an area with great potential, but only if these devices are used alongside “smart” digital meters rather than older-style mechanical meters. “As long as the current mechanical water meters are in place,” one stakeholder stated, “there will not be any improvement in the reading rate from using new pulsers.” Managers also noted that some of the pulsers were stolen or broken over the course of the pilot. WMI managers, on the other hand, were enthusiastic about the pulsers as a pilot intervention, and water meter readings as measured by WMI showed overall reading rate improvement correlated with the use of pulsers. Regarding WMI’s approach in implementing the above interventions, MWA managers voiced that their counterparts in WMI have stopped visiting Ma’an as often as they had at the beginning of the cooperation between the two organizations, without communicating a reason for the decrease in visits,noting that WMI sends seconded staff to YWC but has not done so in MWA despite the need for more support. 39 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV Aqaba Water and Wastewater Investment Master Plan: WMI support to AWC in developing a master plan included participatory sessions with AWC staff, developing indicators and a timeline for achieving results, and linking the plan to AWC’s budget to fund new initiatives. All AWC stakeholders expressed satisfaction with the plan, with one stating, “This was the best master plan we have ever had prepared by an external consultant.” Stakeholders expressed, however, that there is a need for WMI support in securing financial resources to implement the plan. WASTEWATER TREATMENT PERFORMANCE AND COMPLIANCE STRENGTHENED WMI sought to spur progress toward this target by improving wastewater treatment performance through the following sub-activities: Situational analysis and training needs assessment: WMI conducted a full technical assessment for several WWTPs, and stakeholders remarked that the recommendations provided were effective to improve the performance of the WWTPs.At the time of the evaluation, the training needs assessment had not yet been made available to the evaluation team. Training programs and technical assistance regarding processes and upgrades: WMI counterparts at utilities report that the only recommendations from technical assessments that could be implemented are those requiring no additional costs. Onestakeholder at YWC commented that they are fully aware of the problems in their WWTPs but were expecting to receive financial support from WMI, not just an assessment report. WMI assistance consisted solely of trainings on improvement of laboratory procedures and on raising the quality of WWTP operations. Two out of three WAJ stakeholders report that WMI technical support to WAJ helped improve energy efficiency in the South Amman WWTP, with the other stakeholder noting that Miyahuna had already begun work on this goal before WMI assisted. WMI is supporting the WWTPs working group within WAJ through participating in the meetings and providing a seconded expert to support WAJ. The working group meets on a regular basis and documents its decision making through meeting minutes. WMI supported WAJ in creating a SOW to change the Wastewater Treatment Plants Section within the organization into a Directorate to be housed under the Assistant SG. This SOW was the result of a direct request from WAJ, and high-level officials within the Authority expressed appreciation for WMI’s work. CONCLUSIONS: EFFECTIVENESS Interventions that were based upon extensive formal planning with stakeholders are seeing the best results. This planning includes designating roles and responsibilities, setting a timeline for progress with benchmarks, and ensuring the necessary budget is in place. No intervention wholly satisfied all of these components of planning, but in the cases of the roll-out of Phase I of the ERP in Miyahuna and the establishment of the Customer Service Control Unit and the Internal Audit Unit in YWC, the project and its partners made steady progress as long as there was a sufficient plan and timeline to guide them. Many project activities are aligned with utilities’ perceived needs, thereby increasing the investment of counterparts. This is especially true for the introduction of new systems. Technical assistance to Miyahuna’s SCADA system and the creation of the Wastewater Treatment Plants Directorate’s SOW stand out as examples of effective interventions as they are narrowly focused on stakeholder priorities. 40| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV Where WMI has introduced new practices outside of requests from partners at utilities, communication practices could be improved in order to generate buy-in and up-take. This is true of the new units introduced in YWC, which have yet to receive permission to operate from the Board. A good example of participatory communication to generate buy-in is the second approach to revising the organizational chart at YWC. The most effective communication mechanisms consisted of regular steering committee meetings, as demonstrated with the WAJ WWTP task force, with upper management at Miyahuna, and at times in YWC. These meetings allow for a record of progress and regular re-adjustment of WMI interventions. WMI could benefit from cascading these activities across steering committees that are specialized by intervention rather than targeting upper management only as a way of ensuring that all interventions are jointly managed. Planning activities that have an implementation-focused approach, such as the one used in Aqaba to create the Master Plan, were effective in generating stakeholder engagement. This type of approach gives stakeholders a clear idea of how WMI interventions fit into the future of their organization and enables stakeholders to take charge of ensuring the success of the plan. The project could have benefited from using such an approach in strategic planning for ZWA and YWC and the WWTPs. Short-term technical assistance interventions enabled close collaboration and complemented the already-existing work of utilities. The requests from Miyahuna that WMI send experts to help with procuring the SCADA system and that WMI help in procuring trainers demonstrate trust in the project. Where tools, systems, and equipment interventions were introduced, the project did not effectively ensure that the material prerequisites for success were in place, thereby leading to delays. The results of training are mixed due to a lack of on-the-job training opportunities. The project could improve training outcomes within utilities by designating more staff members to train on-site for extended periods of time. While a host of reasons contribute to barriers to employment among women and youth, WAJ HR policies that prevent employment of interns and hinder gender-sensitive hiring practices are significant. WMI’s original design could have benefited from learning from prior projects such as the Institutional Support & Strengthening Program (ISSP), which identified changes in direction at the Ministry level as a threat to effectiveness. FINDINGS: SUSTAINABILITY ZWA & MIYAHUNA Performance Improvement Plan (PIP) and utility KPI targets: Management has already completely disengaged with the PIP and its targets, as evidenced by the fact that no interviewees had seen the plan or its indicators. Senior management at ZWA expressed a desire for WMI’s future work to be bounded by a work plan embedded within a contract with a timeline built on close coordination, needs analyses, and a focused scope of work in order to have a better sense of the working relationship. Additionally, one respondent expressed frustration with a lack of communication about changes in the WMI plan. “At the beginning,” the respondent told evaluators, “WMI was collecting wishes and dreams with ZWA like a honeymoon phase, discussing a new organizational chart, SOPs, X7, HR policy changes...now they have not finished anything.” 41 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV Implement PIP interventions to reduce administrative NRW losses: The issue of double connections has been shown by the pilot study to have great potential for reducing NRW: in the DMZs covered,over 16% of the houses had a double connection. ZWA stakeholders stressed that the work done thus far cannot stand alone, however, and that it will not have any effect on the NRW rate if the survey is not completed thoroughly and the results are not used to create a TOR for disconnecting the double connections. Managers reported that the billing problem has been fixed through the “quick win” of restoring billing records. In the view of ZWA stakeholders, therefore, the current billing rate of 98% will be sustained without further WMI assistance, and these stakeholders are unlikely to buy into reforming procedures. Implement PIP interventions to operationalize financial management systems:The stakeholder in charge of systems in Miyahuna expressed confidence that once the ERP is tendered and rolled-out, this system will be fully integrated into Miyahuna’s business processes and there will be no issue with the hardware or software requirements. Develop customer service-focused business plan:Upper and mid-level managers expressed a lack of certainty about the plan for the future. One reason cited is the abrupt end to interventions on SOPs, the X7, and a GIS remapping initiative. In the case of the X7, mid-level managers reported that ZWA allocated funds to a service contract for their antiquated X7 system in order to ensure its good condition for the WMI X7 upgrade intervention, only to hear silence from the WMI end as time progressed. This systems upgrade remains a top priority for the utility. Miyahuna stakeholders also identified an X7 systems upgrade in their offices as a top priority, since their system is 17 years old. Pre-Feasibility Analysis and Implementation of Merger into Miyahuna: Senior management at ZWA and Miyahuna identified the finalization of the pre-feasibility analysis and continued support to the decision process as their highest priority for WMI’s work moving forward. Miyahuna stakeholders expressed reluctance to the idea of absorbing ZWA because of its large electricity debt. YWC Support Implementation of Interim PIP: IA Unit –KIIs revealed a difference in outlook between what is captured in the IA policies generated by WMI and adopted by the new IA Unit and the views held by upper management of the company and the Board. The Board and the General Manager believe that the IA Unit’s functions should also cover Internal Control functions, whereas the Unit, WMI policy, and international standards dictate that the Unit remains solely dedicated to internal auditing. This difference in opinion was identified as the reason the Unit has not yet been approved. Jerash Control Center –While WMI staff classify this pilot initiative as a success, the idea did notadvance beyond a simulation before the then-Minister ordered it to be stopped and shifted to a national scale. While the pilot demonstrated the buy-in of the Ministry, WMI and YWC staff were not able to generate lessons learned for the eventual scale-up of the idea. Implement PIP interventions to reduce administrative NRW losses:KIIs with senior YWC staff indicated that there is no plan for YWC to supply the computers necessary for efficient functioning of the X7 system. These same staff members stated that there is no budgetary capability for YWC to do so in the future because of the utility’s large electricity debt, thus preventing a threat to the sustainability of X7 use. Implement PIP interventions to operationalize financial management systems:Upper management in YWC reported that the first six months of roll-out contains the most glitches for a system like the ERP. 42| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV As such, these stakeholders stated that the expense of a support contract from Leading Edge will no longer be necessary once this period is over and that YWC IT has the capability to troubleshoot any additional issues after then. Apart from the issue of hardware, the Department of Finance policy was mentioned by stakeholders as a barrier to long-term use of the ERP system, and particularly the procurement module. Although the Board communicated to WMI that this would not be an issue, mid-level managers mentioned that end users are conducting procurement through both the ERP and on the old standard paper forms until the ERP system complies with MOF requirements. HR Assessment and Implementation:The interventions that go beyond developing an updated organizational chart will start after the approval of the chart, according to HR stakeholders. These stakeholders, as well as others in middle- and lower-management, believe that the new organizational chart requires effective completion of job descriptions and functional scopes for each employee in order to truly address the need for more organization within YWC long-term. The HRM intervention that is slated to begin soon is recognized by HR stakeholders as a critical next step for sustainable improvements. Further, the entire department was observed using its office space and new computer equipment enthusiastically, with new systems such as a digital archive now in place. Fixed Asset Registry & Valuation:YWC’s stakeholders in Jerash and Ajloun reported that there is no Asset Management Unit within YWC, nor are there assigned staff to manage maintenance of the asset register once it is created by WMI. The standard procedures for inputting new assets have not yet been created. MWA & AWC Situation assessment and preparatory work for transfer to AWC: MWA directors reported that the Board has fully committed to funding staff salaries for the GIS Unit. However, on the operational level, senior staff in Ma’anreport that GIS data is not systematically checked and cleaned before uploading on the system, and that the Unit is just now getting equipped and staffed. Aqaba Water and Wastewater Investment Master Plan: The plan has recently been completed and implementation has yet to begin; it is too early to tell if the plan will be sustainable. WWTPS Training programs and technical assistance regarding processes and upgrades: Stakeholders expressed a need for outside financial assistance if they are to move into implementing the recommendations held within the WWTP analyses. In regard to the Wastewater Directorate initiative, high-level staff at WAJ communicated to the evaluation team that they have ownership over this idea and will carry it through to realization. CONCLUSIONS: SUSTAINABILITY Research such as that conducted for the double connections survey in ZWA will have little contribution to long-term outcomes unless implementation is planned and funded. The potential for sustainability for some interventions lies in moving from short-term progress to institutionalization. In the case of developing fixed asset registers and GIS systems, WMI could benefit from focusing on generating buy-in for the creation of these units to ensure that roles exist within utilities’ structures to perform up-keep in the future. In other cases, such as the ZWA billing 43 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV intervention, project managers could benefit from a focus on optimizing billing SOPs now that the quick win of customer registration has been accomplished. Financial constraints within utilities and within WAJ are inhibiting the potential for sustainability of the ERP and X7, which require hardware and licensing, new organizational units that require staffing, and training plans that require consultants. WMI has considered the issue of financial support, and the introduction of a limited amount of direct tools, equipment and systems has led to good results thus far. Still, the project could increase the likelihood of sustainability by moving beyond technical support to the development of TORs to the next step of active match-making between utilities/WAJ and entities that can supplement their budgets. In interventions involving the piloting of certain practices or technologies, lessons learned must be documented and plugged into future plans in order to ensure sustainable outcomes. For example, WMI managers could benefit from considering and mitigating the security and hardware-related risks uncovered by the MWA pulsers pilot and conducting a formal assessment of the pilot to ensure other lessons are uncovered. The pilot conducted in Jerash unfortunately did not uncover lessons learned since it was not completed, but WMI can move forward assured of the Ministry’s buy-in of the general idea at the national level. Systems-based interventions such as X7 and ERP are likely to be sustained due to widespread buy-in across managerial levels, although efforts to remove the barriers from MOF will increase long-term institutionalization. CITIZEN ENGAGEMENT With respect to citizen engagement, WMI is tasked to support the development of water demand management, inclusive of behavior change communication is included, which starts with the implementation of the Knowledge, Attitudes, and Practices (KAP) survey, in addition to activities related to water sector strategic communications and advocacy with a focus on gender and youth. WMI activities evaluated in this section include the following targets: 2.2 Behavior Change Communication (BCC) Supported 3.5 Water Sector Strategic Communications, Advocacy, Gender, and Youth Supported FINDINGS: EFFECTIVENESS UTILITY DEMAND-SIDE MANAGEMENT STRENGTHENED Under this target, WMI was tasked with supporting the GoJ and utilities’ efforts to improve the water use practices of citizens and the private sector. The move to better IWRM21, which is the main objective of the GoJ National Water Strategy 2016-2025, requires both a supply management strategy involving the development of new water supplies and a rigorous water demand management (WDM) strategy involving comprehensive reform and behavior change to optimize the use of existing supply. As 21Jordan National Water Strategy 2016 - 2025 44| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV is the case with other targets, the activities under this target have been modified over the course of the project, as reported in WMI quarterly and annual reports. WMI has focused its WDM activities with MWI on developing a WUE plan, a water demand forecast, and a study on treated wastewater sale to the private sector. WMI drafted the WUE plan in collaboration with the Ministry’s WDM Task Force, and the plan put the onus on utilities for implementation. As the only planning document for WDM created through the support of WMI, the WUE plan has a focus on water use, rather than on the reform of MWI policies and regulations. The utilities are tasked in the WUE plan with promoting water-efficient products and distributing Water Saving Devices (WSDs), but these initiatives are not contextualized within a comprehensive strategy that includes long-term targets for sector-wide WDM. There is a lack of agreement between WMI and stakeholders regarding the completion of the water demand forecast, with regard to WMI’s implementation of the planned water demand forecast activity: WMI reports that this intervention is finished, while stakeholders within MWI indicated an expectation for WMI to complete a new forecast. The forecast was to be based on a module created by USAID’s IDARA project, but the forecast module was not updated, as according to WMI staff, the GIZ-backed “Water Evaluation and Planning System” at MWI already provides this service. In terms of approach, project managersin charge of this componentaltered and dropped some activities meant to contribute to WDM. The reasons for these changes ranged from prioritization of other activities in light of KAP survey findings to requests from MWI to remove certain activities. These changes have affected the alignment of WMI activities with the objective of increasing citizen engagement. The study on wastewater sale to the private sector has been well-received by MWI; this activity has now been merged into the IMF PSP support activity that focuses on governance rather than citizen engagement. In terms of support to utilities in implementing the WUE plan, WMI planned to focus on building utilities’ capacity to encourage water reuse. However, utilities were not sufficiently included in the WUE planning process which led to limited ownership of the WUE plan.Campaigns to spread use of WSDs could have benefited from a deeper level of differentiation in the design phase by geographic region and by water use type, sector and gender. BEHAVIOR CHANGE COMMUNICATIONS WMI is tasked with pursuing “state-of-the-art approaches to change behavior relative to water use and public engagement for increased efficiency in the use of water and advocacy for transparency and accountability”22 by supporting the development of a strategic communications plan within GoJ entities and utilities. Interventions slotted for BCC have witnessed many changes since project inceptionas illustrated in Figure 6. These changes have resulted in delays in launching BCC activities, where with the exception of the Amiri Law campaign, citizen-oriented BCC activities had not yet started at the time of evaluation. 22USAID -278-C-16- 00001- SECTION C - PERFORMANCE WORK STATEMENT 45 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV Figure 7. Modifications to Activities Supporting Communications, Advocacy, Gender, and Youth WMI conducted a Knowledge Attitudes and Practices (KAP) survey intended to inform the design of BCC activities. 23The KAP was completed and shared with water utilities and MWI in early 2018. Howeverstakeholders from utilities and MWI expressed reservations about the methodology and the conclusions of the study. MWI and the Communication departments of water utilities claim that they have had little engagement with WMI and as a result, have little awareness of WMI objectives and its role vis-à-vis the sector in general, and vis-à-vis their departments in particular. Stakeholders at YWC and Miyahunawere under the impression that WMI would support implementation of their existing programs and strategies,while WMI views its role as developing strategy rather than supporting those that already exist. Stakeholders within the Communication Departments of the MWI24 and the water companies expressed the unanimous opinion that WMI’s support for developing strategy is misplaced, as they already have strategies that simply require funding, and support in development of new techniques from WMI. Although at the time of evaluation, WMI had not yet fully engaged utilities in communications directed towards long-term behavior change, it has contributed to an initiative to increase awareness of the Amiri law for payment of delinquent water bills in YWC; however, this campaign focused on changing 23 This is the the third KAP study funded by USAID, but the first to take Syrians into account, and reaching 2000 households. 24 MWI has a one year strategy which it publishes on its website. MODIFICATIONS TO ACTIVITIES REVISED ACTIVITIES ADDED ACTIVITIES • Conduct a Gender and Youth Assessment (3.5.1; completed). • Establish a Citizen Advisory Board at Miyahuna (3.5.2. included in Intervention 3.2.3 and proposed to be deleted) • Encourage Urban Youth to Assume Leadership for Advocacy Initiatives (3.5.3. postponed for FY19 due to limitations of resources). • Develop and Implement Training in Social Marketing for CSOs Based in Zarqa and Amman (3.5.4. postponed for FY19 until regional entity is in place). • Revise and Reformulate the Functions Within the New Communications Directorate at MWI (folded with other activities in 3.5.6 and 3.5.7 and proposed to be deleted) • Support Reformulating the Communication Functions within Water Utilities (3.5.6) • Support implementation of Strategic Communication Priorities for the Water Sector (3.5.7) • Develop the Water Sector Gender Policy (3.5.8) • Develop Gender Training (3.5.9) • Implementation progress of the internship program (3.5.10 moved from under component 2) 46| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV payment behavior rather than water use behavior. The campaign message, “Please visit the nearest YWC branch to pay the outstanding balance resulting from your consumption" was conducted through TV, radio, leaflets, and via meetings with tribal leaders, reaching a reported 322,070 people. To date, WMI’s approach has been to work with communications departments across entities individually rather than collectively. While this approach allowed WMI to work with the units towards activity-based interventions, the roles and responsibilities of the various media and communications departments at times overlap, resulting in siloed rather than leveraged efforts.For example, BCC through WMI has targeted different audiences for awareness than it targets for motivating people to change their behavior. Yet, the steps of BCC are commonly recognized as first building awareness and then prompting change within the same target group. WMI has conducted school WUE outreach activities including educational sessions and a social entrepreneurship boot camp on water conservation with ACS, and training on household-level water￾use efficiency techniques and technologies with students from the National Orthodox School. Post￾activity assessment reveals that over half of the participants gained new knowledge as a result of their participation and plans to contract JREDS to conduct activities with other schools in Jordan (JREDS contracting underway at time of evaluation). WMI has initiated several other BCC activities including an initiative proposed by JREDS to promote water-use efficiency in touristic facilities in Amman, Dead Sea and Aqaba and in schools that will be qualified for the ECO-school certification. WATER SECTOR STRATEGIC COMMUNICATIONS, ADVOCACY, GENDER, AND YOUTH SUPPORTED Under the Jordanian Water Policy Dialogue, the GoJ has committed to engage customers more systematically, including through media and communication campaigns that give adequate attention to specific target groups such as women and young people. According to its contract, WMI was to implement activities to achieve the following strategic objectives: • Gender equality and female empowerment enhanced; • Opportunity, well-being, and civic engagement for youth improved; and • Impacts of rapid population growth alleviated and/or mitigated. WMI planned to: • Conduct Gender and Youth Assessment; • Support gender integration in all of the projects activities; • Support the women studies unit in implementing the Gender study and drafting the Gender policy in water sector; and • Provide gender training to address gender disparities in a variety of domains. The Gender Analysis and Action Plan (GAAP)was produced by WMI in the first year of project implementation. While the GAAP speaks to gender power dynamics and representation of women in decision-making and governance, it does not go beyond highlighting disparities to suggesta strategy or 47 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV actions that could increase women’s agency. The GAAPfocuses on activities related to women working in the sector, grouped around amending HR gender policies at utilities, hiring more women in the sectorand including increased female membership in utilitiy boards, encouraging female participation in consultations at utilities and MWI, and sex disaggregation of the above. For example, the GAAP is intended to contribute to Indicator 2.2.1Number of women/youth reached by water conservation behavior change campaigns25and Indicator 2.2Number of new policies, regulations and administrated procedures being adopted. While the indicators measure the number of women and youth reached, they have little support from activities listed in the GAAP to increase the number or percentage of women and youth reached, and the indicator (2.2.1) itself is not sex or age disaggregated. The GAAP does not speak toapplication of gender differentials, gender-sensitive approaches or activities that would effect change in either indicator. TheWomen’s StudiesUnit at MWI was established in 2015; WMI support has included development of the Unit’s mandate, strategic direction, and work plan. These foundational documents were completed in February 2018 and had yet to be approved by the Minister at the time of evaluation. INTERNSHIP PROGRAM WMI’s SOW included an internship program as part of Component 2, “linked to crosscutting USAID/Jordan Mission youth and gender efforts and focused on attracting Jordanian male and female college graduates into the public water sector,” as a way to “increase the number of women and youth engaged in water sector planning and management.” WMI developed an internship program for 100 university graduates to work in the water sector. WMI received more than 2000 applications for the 25 slots available in the first year of the program, and 22 interns were placed for six to nine -month long internships at utilities and WMI, where interns reported that they mainly facilitate the work of WMI in the utilities. The majority of interns are female, which offers an entry point for increased female representation at utilities. While two of the 22 interns have now been employed at utilities, interns expressed a desire to learn more about employment and career pathways at utilities post internship. Meanwhile, sector employers reported that they have had no involvement in recruitment of interns and in interviews did not express an interest in institutionalizing the program. CONCLUSIONS: EFFECTIVENESS WMI’s awareness and BCCinitiatives to different target groups are missed opportunities that could be leveraged to move targeted populations from awareness to long-term behavior change. Thus far, BCC has been conducted through activities, such as the Amiri law campaign, that do not appear to be part of a larger, long-term strategy for water conservation. While some activities, such as Amiri Law and school pilots have been conducted it is unlikely that they have or will result in long-term behavior change within the life of the Activity, due to the delayed start of BCCactivities and time-lag required for behavior change. WMI’s focus on women’s employment primarily through gender sensitization of HR policies at utilities and MWI is not likely to have much effect on gender equality or female empowerment. While WMI- 25 If disaggregated by gender and age. 48| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV supported HR policies within public utilities would be counted in Indicator 2.2, the continued de facto adherence to civil service law26, and lack of full buy-in for implementation renders it ineffective. The lowlevel of awareness and lack of buy-in for the Women’s Studies Unit needs to be reversed in order to move beyond impressions that the Unit is an academic operation or think tank with no substantive involvement or authority in implementing policy.While the strategy developed for the Unit with WMI support is solid, there is a need to ensure that WMI work on gender harness the various spheres of influence and participation of women and youth as change agents. There is also a need to ensure that programming in schools capitalizes on opportunities to use youth as agents of change with respect to both behaviors and water use efficiencies. While interns appreciate the opportunity to have a work experience, the internship program has marginal contribution to increasing opportunities for youth in the sector beyond the 100 positions on offer over the life of the Activity, and currently there is little evidence that sector employers recognize significant value in the internship program. Nevertheless given the large interest in the internship program (2000 applicants) and its potential value, in the coming period work could focus on generating interest in its institutionalization at utilities. FINDINGS AND CONCLUSIONS: SUSTAINABILITY Although civic engagement is critical to water conservation, it is unlikely that activities or results will be sustained due to the limited buy-in for initiatives, and insufficient local financial and human resources to implement strategies or continue activities that have started.Sustainability planswere not developed for implementation of the GAAP, the internship program, and BCC initiatives, and at the time of evaluation, none appear to have buy-in or institutional arrangements to continue. WMI’s current approach encompassing a wide breadth of one-touch activities with students is unlikely to result in lasting benefits with counterparts. Most elements of the WUE plan do not have clear plans for institutionalization, since the adoption and follow-up mechanism for other actors are not clear. One exception is the section on Plumbing Codes. Over the past years there have been several attempts to encourage MWI to adopt a more robust WDM strategy, including former USAID projects IDARA and CBIWDM. The former worked with MWI to establish a WDM unit and a set of WDM policies, while the latter worked at the community level to encourage behavior change among water users. In order to capitalize on buy-in demonstrated during these earlier projects, WMI might benefit from refocusing its activities to act as follow-on interventions to previous USAID projects rather than stand-alone initiatives. Initiatives with students could be sustainable if school administrators and teachers embrace the activities as learning opportunities for civic engagement and voice and providing sufficient positive reinforcement to motivate students. Given the passive involvement of employers in the internship program, sustainability is unlikely. There is a need to re-orient MWI decision-makers on how nuanced approaches to targeting women, men, and youth can translate into improved water governance, water use, and water management. If WMI is genuinely interested in leveraging knowledge gained through the KAP and gender analysis, it 26 The utilities are private companies and as such are not required to adhere to the civil service law. However, in practice adherence to the civil service law is still in effect as most applicants for positions emerge from the civil service bureau, rendering recruitment de facto in adherence with civil service laws, as confirmed by KIIs at utilities. 49 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV needs to help partners develop an understanding of the pivotal role of women and youth as providers and users of water. 50| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV LEARNING Across components and stakeholder groups a number of themes were apparent. Perhaps not surprising, WMI’s initiatives with the most effectiveness and sustainability have the following characteristics, as reflected in Figure 7: • Alignment with partner priorities; • A structured planning process with partners; • Regular communications with partners throughout implementation; and • Are supported with both material and technical assistance. Interventions that were based upon extensive formal planning with stakeholders are seeing the best results. This planning includes designating roles and responsibilities, setting a timeline for progress with benchmarks, and ensuring the necessary budget is in place. As part of the formal planning process, regularengagement and communications with stakeholders is identified as a determinant of effectiveness, such as steering committee meetings which allow for a record of progress and adaptive learning for re-adjustment of interventions. Planning activities that have an implementation-focused approach, are more effective in generating stakeholder engagement than ones that simply create a strategy but do not assist the partner in moving towards or through implementation. Furthermore, the potential for sustainability for some interventions lies in moving from short-term progress to institutionalization. WMI activities that are aligned with partners’ perceived needs increased the in-kind investment of counterparts and are likely to see sustainable results. Initiatives such as research, strategic and action plans that do not have funding for implementation areunlikely to produce results toward project objectives. As a corollary to achievement and sustainability, deficiencies in both effectiveness and sustainability are highly correlated to limited buy-in at both senior and operational levels. While WMI invests resources in building relationships at senior levels with each new administration, new directions and directives are sometimes not communicated through the ranks to operations-level staff who are often in the position to make or break implementation. Initiatives with low levels of effectiveness are characterized by undefined roles and responsibilities of partners and other donors, activities initiated without a formal planning process, and initiatives that are launched despite partners’ lack of financial, material, and human resources to implement. Unless otherwise clarified, partners assume that USAID programs will provide financial assistance for equipment, systems, and implementation; lack of this financial assistance can foster ill will with partners. Figure 8. Characteristics of Effective Interventions. 51 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV Stakeholder cooperation facilitated by ongoing dialogue is key to project performance. By nature of its limited sphere of influence, WMI cannot fulfill all of the roles and responsibilities required to ensure smooth implementation and progress on project indicators on its own. The GoJ and the utilities, as well as USAID, each hold influence and capacity within their own institutions/constituencies. The roles and responsibilities needed to fulfill WMI’s vision are captured by stakeholder in Figure 8. Continuous dialogue and communication are vital to ensuring that elements of effectiveness and sustainability are in place. The multiple changes over the first half of the project- both within activities modified with the aim of improved performance or modified in response to stakeholder desires, as well as changes to WMI’s operating context - have impacted WMI’s program design. In particular, evaluation findings related to the assumption of “full cooperation and commitment of sector stakeholder staff” demonstrate that this assumption has not always held true – both at the Minister level and utlity mid-level management. PROJECT ACTIVITIES TO PRIORITIZE 1. Take advantage of the mid-term evaluation to revisit assumptions and the Theory of Change, given the current dynamics of commitment, capacity, and resources, fine-tune the way forward. 2. Strengthen buy-in for effective reform of the regulatory environment of the sector. 3. Corporatization through AA reform, strengthening the autonomy of water providers, and allocating the necessary time and resources to arrive at agreed-upon improvement of the AAs/AoAs. 4. Consensus on a vision for WUA mandate. 5. Create a sustainability plan to institutionalize IMF Action Plan reporting with a focus on knowledge transfer to MWI staff/units. 6. Establish Private Wells Business Unit. 7. Develop a sector-wide NRW masterplan. Figure 9. Critical Factors for Effectiveness and Sustainability. 52| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV 8. Work with stakeholders within the Ma’an and Miyahuna utility companies to agree upon a vision and identify activities related to WMI’s support to utilities,that will need to be conducted. 9. Revise approach to align activities with priorities of Communications Departments and focus on supporting initiatives that have the potential to be internalized for sustainability. 10. Rescope the BCC component to focus only on direct support to MWI in developing the national-level strategy (at the policy level) and work with each utility company to determine how best to support their WDM communication plans. 11. Meet with all partners to ensure a common understanding of WMI assistance mechanisms (TA only or TA and financial assistance). 53 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV RECOMMENDATIONS CROSS-CUTTING 1. Mid-term evaluation offers the opportune time for WMI and USAID to revisit assumptions and, given the current dynamics of commitment, capacity, and resources, fine-tune the way forward. USAID and WMI would benefit from reflecting on how changes, both as a result of modifying and improving activities and as a result of changes in the context in Jordan, impact project logic flows against program design and the Theory of Change, and ensure that assumptions still hold. 2. USAID and WMI should consider developing custom-level outcome indicators to augment existing F-indicators as a means of increasing reliability, timeliness and attribution to project interventions. For example customer billing data collected by WMI (ie at MWA, ZWA) is both timely and attributable to WMI interventions, but correlating it with NRW datagenerated one to two years back poses challenges to attribution and learning. WMI should consider using utility-level KPI data as interim data points to ensure continuous learning and activity attribution from M&E. 3. For activities that are technology related, WMI should put effort into securing funding for hardware required to operate the systems, including leveraging Fixed Amount Reimbursement Agreement (FARA) funding as a strategic resource to avoid delays in procuring hardware. 4. In order to avoid the misperception among partners that WMI will fully fund all equipment, systems, or implementation in addition to technical assistance, WMI should meet with all partners to ensure a common understanding of WMI assistance mechanisms. If initiatives require financial or material assistance that had not been anticipated and are not covered under component five, WMI should prioritize determining whether to add financial support, assist partners in identifying alternative sources of funding, or discontinue supporting the initiative and communicate the decision in writing to both management and operational-level counterparts. 5. For initiatives that involve new systems and technology, WMI should base interventions on pre￾assessments, include technical assistance for on-the-job training of partner staff, and replicate successes such as those seen with X7. 6. WMI should develop an internal strategy for gaining buy-in for its sector-wide vision for reform among each stakeholder group. The plan should set out expected roles and responsibilities for each group in light or their unique interests and resources and outline how regular communications will be maintained. 7. WMI should deepen participatory approaches with partners through regular meetings and well￾designed communication structures that include top-level and operational-level management, throughout the lifespan of each intervention. 8. WMI and USAID should thoroughly assess risk and develop a contingency plan during the inception phase of interventions to mitigate situations such as changes in staff at utilities and changes in direction at the level of WAJ/MWI that may occur during implementation. 9. USAID should consider leveraging its support of GoJ through MESC, WIT, and WMI as a way to increase timeliness of secondary data that MWI supplies for WMI indicators. GOVERNANCE 1. Finalize the scope and approach of the assistance package to PMU to help its monitoring function of company performance/data combined with conditionality to require publicizing the KPIs.Clarify the relation between WMI support and GIZ support. 2. As a cornerstone for governance reform, prioritize further corporatization through AA reforms and consider using conditionality mechanisms for adoption of revised AAs/AOAs by Miyahuna and YWC. WMI should also further explore international best practice echanisms for citizen enegagement in utility governance. 54| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV 3. Prioritize achieving consensus on a vision for changes in WUA mandate and ensure that new leadership at the JVA and Ministerial levels is continuously engaged, in light of March 2019 renewal date of WUA contracts. Continue to ensure close coordination with GIZ to ensure consistency, synergies and mutual reinforcement of interventions. 4. Until JVA clarifies direction for reform, focus on technological interventions. Once clarified, commit significant resources to the JVA pilot in the North using a staggered approach with first steps including the use ofsmart meters at farm gates and building capacity of the JVA control center to use data and GIS mapping. Until JVA clarifies its direction for reform, focus on technological aspects of the pilot in the North without committing significant resources; building capacity of the JVA control center coupled with GIS mapping. Once the vision is clarified, WMI could consider support to institutional reform of the pilot WUA, as well as an assessment of the financial gaps in the sector. 5. Develop a sustainability plan to institutionalize secretariat support for IMF Action plan reporting with a focus on knowledge transfer to MWI staff, including those from the Strategies and Policies Unit of MWI, with a gradual handover of responsibilities. Ensure that linkages to the National Water Information System are in place (including utility level data) as part of the sustainability plan for data use and integration in IMF reporting. 6. Focus on fleshing-out clear goals and milestones for Cost Recovery/Financial Accounting. 7. Capitalize on the high levels of buy-in and commitment to Private Wells Business Unit establishment and prioritize support to WAJ. 8. Fast-track the development of a sector-wide NRW masterplan, underway at the time of evaluation, as a matter of priority to meet the November 2018 deadline. 9. Consider supporting two rather than three PSP/PBC projects, prioritizing those sub-sectors in which WMI interventions have the most added demonstration/learning value, in close coordination with donors active in these subsectors to ensure non-duplication, coordination, synergies and sustainability. 10. Assist WAJ to identify a source of co-funding for ICTU ERP implementation(hardware/licensing). 11. Focus RBAPS efforts on building leadership capacity within the Policies and Strategy Unit and building technical capacity that cascades down into business units for each entity in the sector. Support amendment of NWIS to cover utility level data and ensure that interventions related to data systems strengthening at ICTU are well-integrated into with policy development processes, including the monitoring of the implementation of the RBAPs once adopted. 12. Assign a full-time WMI specialist and ensure sufficient internet capabilities to maximize effective use and incorporation of remote sensing data, and coordinate with the World Bank to ensure synergies in support. 13. WMI should replicate training to expand the water audit team and provide a WSP expert to support knowledge transfer within MWI. For continued WAJ buy-in,WMI should ensure that work in WSP reinforces existingJordanian standards and regulations in WSP, and allay (mis)perceptions of replacing existing standards. 14. If WMI intends to make progress in water quality governance, support to the Ministry of Environment needs to be clarified and intensified, and financial support will need to be secured to support GIS infrastructure. Also, WMI should support coordination between all the stakeholders involved in monitoring of reclaimed water quality. 55 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV UTILITY PERFORMANCE 1. Work with stakeholders within utility companies to agree upon a vision, develop a work plan for each activity, and ensure that the work plan is shared with both management and operations staff. ▪ Ensure review and agreement on action plans via signatures of both WMI and counterparts at utilities. ▪ Designate an official project management team for each activity including both WMI staff and direct counterparts at the implementation level. ▪ Each activity plan should include a description of the distribution of staff time and financial responsibilities and the timeline for implementation ▪ Develop progress indicators for each utility and share these with all relevant stakeholders. 2. In order to ensure that NRW pilots such as the double connections study, HHUs, and pulsers lead to results, WMI should secure funding for the next steps, and ensure that results are measured in order to gauge their contribution to utility-wide NRW reduction. 3. WMI should continue to support WAJ in improving energy efficiency measures in WWTPs, not only through technical assistance but also through securing financial support for implementing recommendations. 4. When creating utility-wide strategic plans, replicate to the extent possible the approach of deep engagement with utility staff that was used in the development of the Aqaba water and wastewater investment Master Plan. 5. Prioritize generating buy-in at the upper management and Board levels for interventions involving structural change such as organizational charts, job descriptions, and SOPs. CIVIC ENGAGEMENT 1. WMI should conduct an assessment to determine the real value of the internship program to both interns and employers. If determined to be of significant value, USAID should consider providing an endowment to an entity such as an engineering university to continue the program or consider wrapping the activity into another project focused on youth employment. WMI should consider supporting the GOJ in instituting an incentive schemes for the utilities to have their own internship programs. 2. When designing future BCC activities, an attempt should be made to utilize information gained through the KAP, GAAP, and other BCC studies to target and differentiate BCC activities to various audiences based on their respective roles and motivations. 3. To maximize synergies, increase linkages between awareness and BCC campaigns, and supporting replication/scaling up of the pilot interventions, after rigorously assessing their effectiveness and streamlining the design, rather than supporting activities for awareness that do not necessarily link to BCC activities. 4. In order to address the spirit of the gender equality objective, WMI should broaden the focus of gender initiatives beyond employing women and recast gender-sensitivity as a target marketing approach for BCC, sensitizing the decision-makers to better understand the important role of women as water managers, and the need for policies that reflect those differentials in water use. 5. If WMI intends to increase opportunities for women within public utilities, WMI should focus on implementation, and extend beyond counting and disaggregating to measure the impact of gender-related HR policies relative to women’s representation and engagement in decision- 56| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV making. In the second half of the project WMI should focus on advancing women’s empowerment through specific actions targeting gender power dynamics and consider targeting the internship program solely to women in order to increase gender parity in employment at public utilities. 6. Provide consultative services to MWI’s Gender Unit and MWI and utilities’ Communications Departments to unpack information in the GAAP and KAP to increase communications effectiveness through leveraging gender and youth-sensitive strategies, and motivating women and youth as agents of change. Technical assistance should include raising awareness among water sector entities about how valuable women and youth can be to achieving sector objectives. 7. Re-scope the BCC component to focus only on direct support to MWI in developing the national-level strategy (at the policy level) and work with each utility company to determine how best to support their WDM communication plans. 57 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV ANNEX I. EVALUATION STATEMENT OF WORK USAID/Water Management Initiative (WMI) Performance Evaluation Water, Resources and Environment Office (WRE) Statement of Work (SOW) I. INTRODUCTION U.S. Agency for International Development (USAID) in Jordan requests an external performance evaluation of the USAID/Water Management Initiative (WMI) Project, a five-year contract with Tetra Tech, ARD, with a total value of $28,447,544 million (plus $6 million available for equipment and tools) covering the performance period of April 1, 2016, ending on March 31, 2021. II. BACKGROUND AND ACTIVITY DESCRIPTION The purpose of the Water Management Initiative (WMI)Project is to enhance Jordan’s water and wastewater infrastructure, implement policy reform while strengthening governance, and increase water conservation. Through integral partnerships with the Government of Jordan (GOJ) and other private sector allies, USAID is well on its way towards addressing Jordan’s serious and growing water scarcity. Over the last 60 years, USAID has supported GOJ in providing regular water access to 98 percent of Jordanians and 63 percent access to sanitation. Yet, more needs to be done. Currently, there is not enough water to keep up with the steadily increasing demand. To be clear, water demand exceeds sustainable supply by about 100 percent. This growing scarcity is attributed to a higher rate of population growth, influxes in regional refugees and the changing climate. To protect Jordan’s precious water sources, WMI works to reduce water losses, improvewater sector governance, and raise the average liters of (potable) water available per capita per day. WMI activities for FY18 includes 40 interventions grouped in the 4 components and are allocated to all 5 main counterparts, i) Yarmouk Water Company (YWC), ii) Miyahuna Amman and Zarqa, iii) Aqaba Water and Ma’an, iv) MWI and WAJ and v) JVA as presented in the following table. Almost half of the interventions are related to support to the utilities work of Component 1 followed by interventions in Component 3 of Governance with approximately 1/4th of all interventions. Demand management and protection of water supply cover the remaining 1/4th of the total work. The four (4) main WMI project components are: 1- Water Supply Systems: This component is focused on water utilities management and reform mainly through support for design and implementation of Performance Improvement Plans. Under this component WMI supports the implementation of the National Action Plan to Reduce Water Sector Losses (in support of its International Monetary Fund (IMF) commitments). 2- Water Conservation and Demand Management: This component covers the development of water demand management as well as activities related to the study of opportunities for expanding treated wastewater sales. As part of this component the area of behavior change communication is included, which starts with the implementation of the Knowledge Attitudes and Practices survey. 58| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV 3- Water Sector Governance: Support water sector policy development and implementation; enhance utility internal governance and overall governance environment to improve performance; support improvement of irrigation management; develop and support sector regulation; and support the integration of strategic communications, advocacy, gender inclusion, and youth engagement within the water sector. 4- Protection of Water Supply: Develop a groundwater management framework and sustainable monitoring systems; strengthen wastewater treatment performance and compliance; and improve water quality management. To achieve short-term goals and develop longer-term reforms, WMI works in close collaboration with GOJ through the Ministry of Water and Irrigation (MWI); the Water Authority of Jordan (WAJ) and its Performance Management Unit (PMU); the Jordan Valley Authority (JVA); and water utility companies, including Miyahuna, Yarmouk Water Company (YWC), and Aqaba Water. The activity’s subcontractors include Orient Engineering Consulting and Design, which provides will provide technical and engineering expertise across components and overall sector reform advisory services. Jordan Social Marketing Center, will provide specialized expertise in outreach, communications, and behavior change, SEGURA Consulting, will support utility reform, restructuring, and performance improvement and Water for Life Solutions, key in offering expertise in water demand management, and agricultural and groundwater management. Details of the activity to be evaluated: Development Objective: USAID/Water, Resources and Environment Office Activity Title: Water Management Initiative (WMI) Award Number: AID-278-C-16-00001 Award Dates: April 1, 2016to March 31, 2021 Funding: $28,447,544 Implementing Partner: Tetra Tech, ARD. COR: Louis Qaqish Alternate COR:RaziaBaqai III. PURPOSE OF THE EVALUATION The objective of this evaluation is toprovide USAID with findings and strategic recommendations related to the effectiveness of WMI’s activities. The evaluation results will help inform USAID’s implementation decisions ofWMI’s programming. IV. OBJECTIVES AND THEORY OF CHANGE Jordan is recognized as one of the most water deprived countries in the world. Domestic water use in Jordan is among the lowest in the world, and barely meets basic household needs for sanitation, cooking, and cleaning. The influx of Syrian refugees living in host communities and refugee camps is further draining the limited water supply. In certain areas of Jordan, before refugees arrived, there was already tension over water and the public water services were strained. The Syrian crisis has particularly strained the northern governorates, where the majority of refugees are located. 59 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV To achieve WMI’s four objectives, WMI will partner with the GOJ to implement highly focused, catalyzing activities that produce tangible short-term results and set Jordan firmly on a trajectory to broader reform of the water sector. Our theory of change recognizes that sustainable, measurable improvements in Jordan’s water sector will not come about in a singular event achieved by a limited set of actors working in isolation, but require myriad incremental changes over time involving a broad set of stakeholders. Theory of Change:IF water utilities performance and organizational capacity is improved and IF, water supply and quality best practices are induced, THEN water sector management will be improved and utility staff will be able to implement international best practices to increase cost recovery, reduce non-revenue water (NRW) and administrative losses, and increase cubic meter of water saved annually from water-saving technologies or techniques. AND IF broad-based water sector reform is enhanced and IF, engagement in critical water scarcity issues is strengthened, THEN water sector governance will be improved, and decentralization of functions will be transferred to water users associations or new representative entities, people will have an improved understanding of critical water scarcity issues facing Jordan such as the cost of water and importance of conservation, and they will adopt technologies or techniques to save water or increase reuse of water; THEN measurable improvement and greater sustainability of Jordan water sector is achieved and there will be a reduction in the national average NRW and policies, regulations and administrative procedures relating to water and natural resources will be in place and implemented. Critical assumptions include: • Full cooperation and commitment of sector stakeholder staff; • Access to necessary water sector documents and data; • Community members are responsive to behavior change campaigns related to water conservation; and • New technologies and practices are adopted by stakeholders. Among activity indicators, the core key performance indicators include: 1) Amount of water provided or saved increased a. Percentage of non-revenue water (NRW) (by utility) b. Water conservation improved: number of adoptions of technologies or techniques to save water or increase reuse of water c. Cubic meters of water provided or saved as a result of U.S. Government (USG) assistance 2) Water sector management and governance improved a. Number of action plans implemented for water security, integrated water resource management, and/or water source protection as a result of USG assistance b. Percent change in cost recovery of water sector (disaggregated by utility) c. Capacity built: Number of people educated on tools, approaches, and/or methods for water security, integrated water resource management, and/or water source protection as a result of USG assistance 3) Accountable, sustainable management of water and natural resources increased 60| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV a. Number of policies, regulations and administrative procedures relating to water and natural resources in development, passed, or being implemented as a result of USG assistance b. Percentage of NRW (national average) V. EVALUATION QUESTIONS In addressing the stated purpose, the evaluation will explicitly answer the following: Effectiveness 1. What have been the achievements and deficiencies in each of the components listed above? 2. How did the strategy and implementation approach enhance or weaken achievement of the anticipated tasks? 3. How were the main gender and youth constraints in the project addressed? Sustainability: 1. What technical results of the project can USAID expect to be sustainable (and which will be difficult to sustain)? 2. What lasting benefits can be expected in terms of USAID’s and the project’s relationships with the main project counterparts? 3. What recommendations could be given to ensure the sustainability of the interventions? Learning: 1. Which incomplete project activities should be prioritized for the remaining project period of performance? 2. Were there any alternatives or unexpected causes of documented results or outcomes? 3. Were there strategies or practices implemented that should be replicated in future? 4. What are the activities/sub-activities that need to be revised or enhanced in the next work-plan in order to boost project performance results? 5. Are there activities that should be changed for the remainder of the project? VI. EVALUATION DESIGN AND METHODOLOGY The external evaluation will use quantitative and qualitative approaches to assess how effective the Activity was in meeting its objectives and to provide recommendations on how to make future interventions with similar objectives more effective and more sustainable. Data collection will cover interviews with USAID staff, Government of Jordan stakeholders, activity staff and beneficiaries by component, and site visits to the utilities and ministries. VII. EVALUATION TEAM COMPOSITION 61 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV In order to meet the requirements of team composition, ensure data quality, and contribute to building capacity of local evaluation specialists, the following is suggested for team composition, or ensuring that the team has the below mentioned skills: 1- Evaluation team leader specialized in technical assistance projects in the water sector. 2- Water utility expert. 3- Financial specialist. 4- Communication and outreach specialist. 5- Policy, Governance, and institutionalspecialist. 6- Water demand and conservation expert. VIII. PERFORMANCE PERIOD The evaluation will be conducted from April 2018through July 2018, with thefinal report submitted by August 2018. IX. DELIVERABLES AND TIMELINE Action Date/Timing Evaluation process starts; initiate discussions with USAID staff February 1, 2018 Finalize SOW; hire evaluation team February – March 2018 Inception meeting between evaluation team, USAID staff, and WMI staff March 18-22, 2018 Development and finalization of evaluation design and data collection tools; submit to USAID for approval March 25 – April 5, 2018 Field work; data collection and information gathering; meetings with stakeholder (MWI, WAJ, JVA, Water utilities) April 8- May 17, 2018 Check-in meetings with USAID April - May, 2018 Presentation of initial findings to WMI and USAID May 20- 31, 2018 Submission of the evaluation draft report June 20, 2018 USAID and WMI provide comments and feedback on draft evaluation report July 11, 2018 Submission of final evaluation report August8, 2018 62| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV ANNEXII: EVALUATION DESIGN AND METHODOLOGY Table 1contains the original evaluation questions from the Scope of Work (SOW) for this Midterm Review (MTR) and suggested regrouping: Table 1: Evaluation questions From the SOW Regrouping Effectiveness 1. What have been the achievements and deficiencies in each of the components listed above? 2. How did the strategy and implementation approach enhance or weaken achievement of the anticipated tasks? 3. How were the main gender and youth constraints in the project addressed? 1. What have been the achievements and deficiencies in each of the components and what were the factors affecting it? In particular: 1a) Were the achievements in line with the plans? If not why? - How did the strategy and implementation approach enhance or weaken achievement of the anticipated tasks? Were there any alternatives or unexpected causes of documented results or outcomes? - Were the deliverables of sufficient quality? 1b) How were the main gender and youth constraints in the project addressed? 2. What are the activities/sub-activities that need to be revised or enhanced in the next work-plan in order to boost project performance results? What changes need to be made in terms of modalities of work/approaches? Sustainability: 63 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV From the SOW Regrouping 1. What technical results of the project can USAID expect to be sustainable (and which will be difficult to sustain)? 2. What lasting benefits can be expected in terms of USAID’s and the project’s relationships with the main project counterparts? 3. What recommendations could be given to ensure the sustainability of the interventions? 3. What technical results of the project can USAID expect to be sustainable (and which will be difficult to sustain)? 3.a Is there an adequate sustainability plan and is it being well implemented? 3.b How strong is the national buy-in and the institutional arrangements to promote it? 3.c What is the likelihood of technical, financial and programmatic sustainability? What lasting benefits can be expected in terms of USAID’s and the project’s relationships with the main project counterparts? 4. What recommendations could be given to ensure the sustainability of the interventions? Learning 1. Which incomplete project activities should be prioritized for the remaining project period of performance? 2. Were there any alternatives or unexpected causes of documented results or outcomes? 4. Weretherestrategiesorpractice simplementedthatshouldberepl icatedinfuture? 5. What are the activities/sub￾activities that need to be revised or enhanced in the next work-plan in order to boost project performance results? 6. Are there activities that should be changed for the remainder of the project? 5. Which incomplete project activities should be prioritized for the remaining project period of performance? 6. Were there strategies or practices implemented that should be replicated in future? 64| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV From the SOW Regrouping In answering the evaluation questions particular emphasis was be placed on: • Technical assistance to strengthen the GOJ’s reform, policy development and implementation, and capacity-building efforts; • Effectiveness of strategies and approaches in activities completed to date in achieving overall WMI objectives of water sector reform, NRW reductions, increased water conservation, improved utility efficiency, as a result of the interventions; • Sustainability of activities supported by the WMI Activity; • Learning to be applied to the remainder of the Activity implementation; • The objectives as stated earlier, with the particular emphasis on the potential for and risks to sustainability and learning; and • Its vision for TOC (both original and to-be-revised version) Data Collection Methods The evaluation team conducted a desk review, key informant interviews (KIIs), focus group discussions (FGDs) and checklists (for the company employees, please see point D below). This allowed team members to probe further into issues raised by stakeholders. A. Desk Review (Q1, Q2, Q3, Q4, Q5, Q6): An in-depth review was conducted of WMI contract and modifications, Activity Monitoring and Evaluation Plan (AMEP) and indicators, work plans, Annual and Quarterly Reports, WMI management documents, research and special studies conducted by WMI, training curricula, and databases of information maintained by WMI. Desk Review also included the review of key third party documents, including those of the (a) Government (related strategies, programs and alike) as well as (b) international and bilateral aid organizations (including those of the World Bank, IFC, GIZ, KfW and the EU) B. In-Depth Interviews (Q1, Q2, Q3, Q4, Q5, Q6): Around 100 interviews were planned to be conducted with individuals from government, the USAID, USAID projects. WMI, Government (MWI, WAJ, JVA, Ministry of Environment), international (e.g., GIZ) and local partners (schools, Chambers of Commerce, ACWUA (Arab Countries Water Utilities Association), etc.). Interviewees were selected based on participation and implementation in key interventions under evaluation, as well as sector expertise. The in-depth interviews were guided by semi-structured questions covering the evaluation topics to capture perspectives and activities of each informant category, revolving around the evaluation questions relevant to each stakeholder. The guides were designed to preserve the potential for a relatively free￾flowing conversation, while creating a standardized format to facilitate a reliable, comparative analysis of data pertaining to the evaluation questions for triangulation of information from multiple stakeholder perspectives. While questions were based on the evaluation’s overarching questions, they varied depending upon the identity of the informants. For example, questions asked of some informants might not have been asked to others due to varying levels of involvement in the various interventions or knowledge of specific interventions, while relative importance of questions varied by the type of stakeholder. The Interview guides took into account the need to capture gender differentials. Where possible and relevant, answers were sex disaggregated. Interview guides were tailored to each of the stakeholder clusters, with each interview lasting 45-60 minutes. Most interviews were conducted in Arabic; some interviews conducted by the Team Leader were conducted in English/Arabic with translation. Tools were tested and refined as necessary. To take advantage of the extensive knowledge about water sector that exists within WMI and to kick-start the 65 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV data gathering process, the evaluation began by conducting in-depth meetings with WMI staff. In addition to WMI, the evaluation team consulted with the USAID Water team and implementing partners with significant ongoing water sector programming. Table 2 describes the sampling plan as well as the actual number of interviews and FGDs per category. C. Focus Group Discussions (Q1, Q2, Q3, Q4, Q5, Q6): Group discussions were planned to be conducted to explore issues in-depth, to triangulate data, and to solicit the input of inter alia: Government officials, staff of water companies, private businesses, and other stakeholders. 1. WUAs: 1 FGD including a sampling of WUAs that attended the project-sponsored WUA event 2. Billing and Collection staff: 3 FGDs on Billing and Collection in 3 locations, covered by 3 companies- YWC, ZWA and MWA. In Ma’an this was planned to cover also the component on Behavior Change Campaign. 3. Local businesspeople:1 FGD among the businesses related to WUE and WDM 4. Interns: 1 FGD with interns from YWC and ZWA 5. Communications staff: 1 FGD with staff from YWC and ZWA D. Checklist: Among the employees of water companies to assess whether institutional arrangements are in place to ensure that the employees are aware of the new policies, know where to find these, and whether they are using them. All people-level questions will be sex disaggregated. The team will also identify the questions that will require examination of gender specific or gender differential effects. Table 2: Sampling Type Agency Number of KIIs Number of FGDs Planned Actual Planned Actual Government Officials MWI Up to 10 9 0 0 WAJ Up to 17 13 0 0 JVA Up to 7 4 0 0 Ministry of Environment 1 1 0 0 Greater Amman Municipality 1 1 0 0 IT Strategy Team 4 0 (IT Strategy Team members overlapped with original KII sample) 0 0 WDM Task Force 5-10 0 (WDM Task Force has been dissolved) 0 0 USAID and Implementing Partners (IPs) USAID Water (EDE DO) 1 1 0 0 USAID Cities project 1 1 0 0 USAID WIT project 1 1 0 0 International Organizations IMF 1 0 (KII with WB instead) 0 0 IFC 1 1 0 0 KfW 1 1 0 0 GIZ 1 4 0 0 AFD 0 1 0 0 World Bank 0 1 0 0 66| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV Water Companies ZWA Up to 7 5 1 (interns) 1 (billing & collections dpts) 1 (communications) 1 (interns) 0 (billing & collections dpts): The breadth of WMI activities with these dpts did not justify a focus group. 0 (comms): There were no interventions around communications at ZWA. YWC Up to 17 18 1 (interns) 1 (billing & collections dpts) 1 (communications) 1 (interns) 0 (billing & collections dpts): There were no activities centered on these dpts at YWC, and the checklist already covered X7 and ERP use. 0 (comms): The staff did not attend the planned FGD and reported there was very limited WMI involvement. Ma’an Water Authority Up to 4 5 1 (billing & collections depts) 1 Miyahuna Water Company Up to 7 8 0 0 Aqaba Water Company Up to 5 4 0 0 Non-Governmental Organizations (NGOs) ACWUA 2 2 0 0 JREDS 1 1 0 0 Water User Associations (WUA) Attendees of WMI WUA event 0 0 1 1 Chambers of Commerce Businesses WUE and WDM 0 1 (Amman CoC) 1 0 (There was very little interaction with CoCs) WMI and Subcontractors Tetra Tech, Segura, JSMC 12 10 0 0 Table 3describes data collection methods. Table 3: Data Collection Methods 67 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV Evaluation Questions Data Collection Methods KIIs FGDs Desk review Checklist 1. What have been the achievements and deficiencies in each of the components and what were the factors affecting it? x x x x 1a Were the achievements in line with the plans? If not why? - How did the strategy and implementation approach enhance or weaken achievement of the anticipated tasks? Were there any alternatives or unexpected causes of documented results or outcomes? - Were the deliverables of sufficient quality? x x x x 1b How were the main gender and youth constraints in the project addressed? x x x 2 What are the activities/sub-activities that need to be revised or enhanced in the next work-plan in order to boost project performance results? What changes need to be made in terms of modalities of work/approaches? x x x 3 What technical results of the project can USAID could be expected to be sustainable (and which will be difficult to sustain)? 3a Is there an adequate sustainability plan and is it being well implemented? x x 3b How strong is the national buy-in and the institutional arrangements to promote it? x x x 3c What is the likelihood of technical, financial and programmatic sustainability? What lasting benefits can be expected in terms of USAID’s and the project’s relationships with the main project counterparts? x x x 4 What recommendations could be given to ensure the sustainability of the interventions? x x 68| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV Evaluation Questions Data Collection Methods KIIs FGDs Desk review Checklist 5, Which incomplete project activities should be prioritized for the remaining project period of performance? x x 6. Were there strategies or practices implemented that should be replicated in future? x x x Data Analysis Methods Data analysis was structured by the evaluation questions, and applied the following data analysis methods: (see Table 4) Table 4: Data Analysis Methods Evaluation Questions Comparison Analysis Q1: Q2: Q3: Q4 Frequency Response/ Trend Pattern Analysis Q1; Q2; Q3; Q4; Q5; Q6 Triangulation Q1; Q2; Q3; Q4; Q5; Q6 As the issue of sustainability of the partner institutions as well as learning were of particular importance to this evaluation, questions that explored these have been integrated into all FGDs, and interviews. Conclusions regarding sustainability and learning resulted from triangulation of a variety of data sources. 69 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV ANNEX III: BIBLIOGRAPHY Dahiyat, Eng. Iyad. “Available Solutions for Hosting Oracle Application Suite for WAJ.” Received by Mr. Jose Valdez - COP, Ministry of Water and Irrigation, 3 May 2018, Amman, Jordan. Directorate of Media and Water Awareness Annual. “Annual Report Water Authority of Jordan – Jordan Valley Authority.” 2014. Dr. Saqer S. Al-Salem. “Indicator - Name of the O&M Improvement: AL-Fuhais Wastewater Treatment Plant /Balqa Governorate/WAJ.” 23 Sept. 2017. “Fiscal Reform and Public Financial Management - Public Private Partnerships in Jordan.” 20 Apr. 2018. Human, Resources. “Human Resources System for Employees Yarmouk Water Company.” Yarmouk Water Company, Irbid. (undated) Information and Communication Technology Unit. “IT Roadmap.” (undated) IRG, USAID. “Institutional Support & Strengthening Program.” Nov. 2015. Irrigation, Ministry of Water and, and USAID. “MWI’S Action Plan to reduce water sector losses (IMF commitments).” Le Grand Hotel, Amman, Jordan, 21 Mar. 2018. Jordan Valley Authority, USAID GIZ. “Future Vision of the Participatory Management of Irrigation Water in the Jordan Valley.” Dead Sea Hotel, Jordan, 30 Mar. 2018. Millennium Challenge Account Jordan. “Monitoring and Evaluation Plan.” 6 Mar. 2012. Ministry of planning and international cooperation. “Jordan Response Plan 2015 for the Syria crisis.” 17 Dec. 2014. Ministry of Water and Irrigation. “Executive Plan Information and Outreach.” 2016. Ministry of Water and Irrigation. “National Water Strategy 2016 - 2025.” Ministry of Water and Irrigation. “Structural Benchmark Action Plan to Reduce Water Sector Losses.” 2017. Ministry of Water and Irrigation. “Structural Benchmark Action Plan to Reduce Water Sector Losses.” Mar. 2017. Miyahuna. “Business Plan 2013-2017.” Naser, Hazem. “Presentations to HE Minister.” 5 Dec. 2017. “Result-Based Action Plan (RBAP) for The Water Sector Policies under the National Water Strategy.” 2016. Shammout, Fawwaz. YWC Oracle Update Project Status Report. LeadingEdge Technology and YWC, 2017, pp. 1–2, YWC Oracle Update Project Status Report. 70| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV Tetra, Tech. Report on Available Solutions for Hosting Oracle Applications Suite – WAJ & YWC. WMI - USAID, 2018, pp. 1–27, Report on Available Solutions for Hosting Oracle Applications Suite – WAJ & YWC. United States Agency for International Development, Chemonics International Inc. “Improving water and sanitation services in the Mena region (IWSMR).” Sept. 2015. UNDP-SIWI Water Governance Facility & UNICEF. WASH and Accountability: Explaining the Concept. January 2015.http://www.watergovernance.org/resources/accountability-in-wash-explaining-the-concept/ USAID/WMI. “Action Plan for a Phased Water-Use Efficiency Program - Final.” Sept. 2017. USAID/WMI. “Action Plan for Operational Improvements - Ma’an Water Administration Draft.” Feb. 2018. USAID/WMI. “Activity Monitoring, Evaluation, and Learning Plan (AMELP).” Dec. 2016. USAID/WMI. “Al-Fuhais WWTP Situation Analysis.” Feb. 2016. USAID/WMI. “Al-Me’Rad/West Jerash WWTP Situation Analysis – Draft.” May 2017. USAID/WMI. “Annual Performance Report FY 2016–FY 2017 (April 1, 2016 – September 30, 2017) Quarterly Progress Report No. 6 FY 2017, Q4 (July 1 – September 30, 2017) – Volume 1.” USAID/WMI “Annual Performance Report FY 2016–FY 2017 (April 1, 2016 – September 30, 2017) Quarterly Progress Report No. 6 FY2017, Q4 (July 1–30 September, 2017) - Volume 2 - Attachments.” USAID/WMI. “Annual Work Plan – April-September 2016.” USAID/WMI. “Annual Work Plan, October 2016 – September 2017 (FY17).” Sept. 2016. USAID/WMI. “Annual Work Plan, October 2017–September 2018 (FY18).” Sept. 2017. USAID/WMI. “Assessment of Current Operations in Ma’an Water Administration – Final.” July 2017. USAID/WMI. “Assessment of Groundwater Monitoring Practices Report.” Aug. 2017. USAID/WMI. “Assessment Study for Pulsers Technology in Ma’an (Pilot Report) - Draft.” Feb. 2018. USAID/WMI. “Baqa’a WWTP Situation Analysis Report - DRAFT.” June 2017. USAID/WMI. “Evaluation of Assignment Agreements against International Best Practices.” Sept. 2016. USAID/WMI. “Evaluation of the Current Meter Reading & Violation Management System (MRVM) - Draft.” Feb. 2018. USAID/WMI. “Evaluation of Water Quality & Reclaimed Water Monitoring Practices in The Ministry of Environment.” Aug. 2017. USAID/WMI. “Gender Analysis and Action Plan.” June 2016. 71 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV USAID/WMI. “Governance Analysis of Miyahuna and Yarmouk Water Company – Findings and Recommendations.” Nov. 2017. USAID/WMI. “IMF Action Plan - Implementation Strategy to Support MWI.” Sept. 2017. USAID/WMI. “Ma’an WWTP Situation Analysis.” Apr. 2018. USAID/WMI. “Quarterly Progress Report No. 7 – Volume 1 FY2018, Q1 (October 1 to December 31, 2017).” USAID/WMI. “Quarterly Progress Report No. 7 FY 2018, Q1 (October 1 – December 31, 2017) – Volume 2.” Jan. 2018. USAID/WMI. “Quarterly Progress Report No. 8 – Volume 1 FY2018, Q2 (January 1 to March 31, 2018).” Apr. 2018. USAID/WMI. “Recommended Modifications on the Assignment Agreement and Articles of Associations for Miyahuna and Yarmouk Water Company (YWC).” Apr. 2018. USAID/WMI. “Review of Existing Water Safety Planning Practices in Jordan Final Report.” Dec. 2016. USAID/WMI. “Toward National Water Safety ‘WSP’ Planning: An enabling environment to support sustainable WSP scale-up.” May 2018. USAID/WMI. “Water Management Initiative Quarterly Progress Report No. 8 – Volume I1 FY2018, Q2 (January 1 to March 31, 2018).” USAID/WMI. “WMI Communication Plan.” Mar. 2018. USAID/WMI. “Workshop and Report Presenting Wastewater Reuse Demand and Opportunities by Private Sector in at Least Miyahuna and Zarqa Water Supply Service Areas.” Jan. 2018. USAID/WMI. “Youth Analysis and Action Plan – Activity 3.5.1.” Mar. 2017. USAID/WMI and, WMI. “Pre-Feasibility Analysis to Establish an Operation and Maintenance Public Entity to Manage Water Conveyance and Distribution in the Jordan Valley.” Dec. 2017. USAID/WMI. “Component 3: Water Sector Governance 3.2: Utility Assignment Agreements Developed/ Modified.” May 2018. USAID/WMI. “Implementation of a Population-Based Knowledge, Attitudes and Practices (KAP) Baseline Assessment Study Final KAP Study Findings Report.” Jan. 2018. USAID/WMI. “Survey Findings of Shobak Water Consumption.” (undated) USAID, Ministry of Water and Irrigation. “Improving Operational Processes in Ma'an Water Management.” 2017. USAID/WMI. “Developing Modern and Strategic Internal Audit Function Al Yarmouk Water Company - Gap Analysis Report.” (undated) 72| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV USAID/WMI. “Intervention 1.2.3: Support Implementation of PBC and PSP to Improve O&M Services.” Mar. 2018. USAID/WMI. “Milestone of Performing Internal Audit Improvements in Yarmouk Water Company.” (undated) White, Edward. “Managing the Public-Private Partnerships Project Cycle - PPP Contract Management & Performance Monitoring & Group Exercise.” (undated) White, Edward. “Managing the Public-Private Partnerships Project Cycle - Definition, Rationale, Key Features & Structuring Models for PPP.” (undated) White, Edward. “Managing the Public-Private Partnerships Project Cycle - Group Exercise -PPP Procurement.” (undated) White, Edward “Managing the Public-Private Partnerships Project Cycle - Group Exercise - PPP Feasibility Analysis & Risk Allocation.” (undated) White, Edward. “Managing the Public-Private Partnerships Project Cycle - Managing PPP Feasibility Studies & Case Studies.” (undated) White, Edward. “Managing the Public-Private Partnerships Project Cycle - Overview and Introduction.” (undated) White, Edward. “Managing the Public-Private Partnerships Project Cycle - Phase 3 - Managing PPP Tenders & Procurement.” (undated) White, Edward. “Managing the Public-Private Partnerships Project Cycle - PPP Project Identification Selection & Project Concept Paper Development.” (undated) WMI Sr. M& E specialist, Dania Husseini. “Indicator Data - Athruh Debt Collection Campaign.” Sept. 2017. WMI, Dr. Saqer S. Al-Salem. “Indicator - Name of the O&M Improvement: AL-Ekader Wastewater Treatment Plant /YWC.” 23 Sept. 2017. WMI, Dr. Saqer S. Al-Salem. “Indicator - Name of the O&M Improvement: Al-Me’Rad Wastewater Treatment Plant /YWC.” 25 Mar. 2017. USAID/WMI. “IT Strategy for the Water Sector.” (undated) USAID/WMI. “Options for Improved Private Wells Management.” Nov. 2017. WMI. “Annual Work Plan FY 18 Presentation to Evaluation Team.” 23 Apr. 2018. WMI. “General Information About the Selected 25 Wells - Sites Visit Report.” Apr. 2017. WMI. “Report on Review of Existing Communication Strategy & Identification of Areas of Intervention Linked to MWI and Utilities Priorities – Activity 2.2.2.1.” June 2017. 73 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV WMI/Expert on Wastewater process, biosolids and reuse. “Indicator: CCAO1: Number of People Educated on Tools, Approaches, and/or Methods for Water Security, Integrated Water Resource Management, and/or Water Source Protection as a Result of USG Assistance (M-PMP 3.3.2.1.a) (F HL.8.3-1).” 20 Oct. 2017. Yarmouk, Company Water. “Audit Charter of the Internal Audit Unit Yarmouk Water Company.” Sept. 2017. Yarmouk, Company Water. “Charter of the Audit Committee.” Irbid, Irbid, Dec. 2017. Yarmouk, Company Water. “Directory of Internal Audit Procedures of Al Yarmouk Water Company.” Amman, Jordan. (undated) “Yarmouk Water Company Request for Proposal.” Yarmouk Water Company, Irbid. (undated) Yarmouk Water Company. Fixed Assets Policy. Yarmouk Water Company, 2018, pp. 1–23, Fixed Assets Policy. Yarmouk Water Company. “Fixed Asset Policy.” Mar. 2018. Author unknown, document undated “Road Map for Empowering MWI Gender Unit and Gender Mainstreaming Efforts in the Water Sector.” “Gender Policy Outline.” “Reflections on the Results of Key Informant Interviews with Males and Females in YWC.” 74| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV ANNEX IV: LIST OF PERSONS INTERVIEWED Organization Title Name Type of meeting Date of meeting Phone no./email Gender Ministry of Water and Irrigation (MWI) MWI General Secretary Ali Sobuh KII 5/8/18 775690020 M MWI Project Manager Susan Kilani KII 5/9/18 795600505 F MWI Previous Project Manager AkramRabadi KII 5/17/18 795565307 M MWI Media and Awareness Unit Adnan Zoubi KII 5/21/18 777534549 M MWI General Secretary Assistant for Strategic Planning WaelElayyan KII 5/21/18 798060574 M MWI International Finance and Cooperation Unit Mohammed Aldwairi KII 6/4/18 775744046 M MWI Women's Studies Unit Lulu Masalha KII 5/14/18 775653611 F MWI Water Demand Directorate FatenShabban KII 5/14/18 06-5652265 Ext:1027 F MWI Private Sector Participation Expert QusayQuteishat KII 6/4/18 795532606 M Water Authority of Jordan (WAJ) WAJ Secretary General Iyad Dahiyat KII 6/7/18 796860000 M WAJ Managerial Affairs MajedQutaishat KII 5/23/18 777666422 M WAJ Financial Affairs FirasAlazzam KII 5/10/18 799019990 M WAJ Institutional Performance Improvement Directorate TahaniJabasini KII 5/23/18 Tahani_Jabasini@mwi.gov.jo F WAJ Manager of WAJ PMU Mohammed Waqfi KII 5/10/18 77283587 M 75 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV WAJ GIZ PMU Eng. Udo Kachel KII 6/6/18 797133832 M WAJ Laboratories and Quality Affairs Ahmed Ulaimat KII 6/7/18 777785251 M WAJ Water Basin Affairs KhairHaddidi KII 5/9/18 777495600 M WAJ Communications and ICT SuhaAfaneh KII 5/1/18 796589111 F WAJ Communications and ICT WisamRabadi KII 5/9/18 795530409 M WAJ PR and Media Relations Omar Salameh KII 5/8/18 77744295 M WAJ Head of WWTPs section Mohammad Kabashneh KII 6/5/18 065683100 M WAJ ICTU Head of Decision Support Unit Ibtesam Al Saleh. KII 5/1/21 065683100 F Jordan Valley Authority (JVA) JVA Acting Secretary General Ali Kouz KII 5/23/18 797828282 M JVA Polices and Institutional Performance Development Khalil Alabsi KII 5/13/18 799035691 M JVA Water Users Association Administration Khaled Qusous KII 5/14/18 0795571246 M JVA Assitant for Northern and Middle Ghors GhassanObidat KII 5/13/18 ghassan_obidat@mwi.gov.jo M Ministry of Environment MoEnv Monitoring and Evaluation Sayyed Saleh KII 5/21/18 795827205 M Yarmouk Water Company (YWC) YWC General Manger Hasan Hazaemeh KII 5/6/18 795444439 M 76| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV YWC Head of Accounting Dept Khaled Rifai KII 5/14/18 080022977 M YWC Head of Expenditures Dept Mahmoud Berham KII 5/14/18 080022977 M YWC HR Director Khaled Zou'bi KII 5/23/18 795106101 M YWC HR: Training Department AfafSabbagh KII 5/16/18 779914149 F YWC HR: Communication ReemaHamouri KII 5/15/18 775636869 F YWC Technical Affairs Director Ashraf Bataineh KII 5/14/18 799601971 M YWC Commercial Director Ahmad Gharaibeh KII 5/15/18 799603768 M YWC IT manager Mahmoud Zou'bi KII 5/7/18 796001750 M YWC Water Production Manager Ahmad Shikhah KII 5/16/18 798861681 M YWC Procurement Manager Hussein Abunijim KII 5/16/18 796625958 M YWC Internal Audit Manager MwafaqMalkawi KII 5/7/18 799625925 M YWC Customer Service Control Unit Manager SamerZou'bi KII 5/7/18 798502205 M YWC Jerash &Ajloun Water Director Montaser Momani KII 5/9/18 799166005 M YWC Irbid Customer Service Manager IssamJaradat KII 5/15/18 799574985 M YWC Jerash O&M Manager Marwan Ayasrah KII 5/9/18 798594816 M YWC Ajloun Customer Service Manager ZiadSmadi KII 5/9/18 798918073 M YWC Commerical Center Customer Lana Zaareer KII 5/16/18 778481843 F 77 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV Zarqa Water ZWA General Director JiriesDababneh KII 5/21/18 799071111 M ZWA Administration & Accounting Department Subhi Abu Al-Roos KII 5/21/18 Contacted through AmerMatalqah M ZWA Administration & Accounting Department Mohammad Tara'an KII 5/21/18 Contacted through AmerMatalqah M ZWA Technical Affairs Department AseelHijazeen KII 5/21/18 Contacted through AmerMatalqah F ZWA Customer Service Director AmerMatalqah KII 5/21/18 799034385 M Miyahuna Miyahuna Chief Executive Officer Ghazi Khaleel KII 5/24/18 799500095 M Miyahuna Customer Service Mohamad Ouran KII 5/26/18 799034336 M Miyahuna Production and Quality HaithamAlkilani KII 5/29/18 799034291 M Miyahuna Operations Mohamad Kharabsheh KII 5/29/18 799034458 M Miyahuna Financial Affairs Bahaa Baghdadi KII 5/29/18 799034263 M Miyahuna Human Resources NisrineNabulsi KII 5/24/18 799034815 F Miyahuna Employee Affairs Dina Al-Shareef KII 5/24/18 799034963 F Miyahuna Information Technology Muhammad Al￾Qudah KII 5/23/18 795162299 M Aqaba Water Company (AWC) AWC General Manager Khaled Obaideen KII 5/8/18 770444000 M AWC Planning and Technical Support Unit YaserHanaqteh KII 5/8/18 777627940 M 78| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV AWC Financial Administrative Affairs Unit Alaa el Dein Mahmoud KII 5/8/18 775444450 M AWC IT Ahmed Abu Soud KII 5/8/18 799069699 M Ma'an Water MWA Operations Directorate Bilal Dahiyat KII 5/8/18 772423608 M MWA Customer Services Ahmed AbdAddayem KII 5/8/18 775435335 M MWA Operations Directorate Waleed Qaraleh KII 5/8/18 776188289 M MWA GIS Administrator Iman Ababneh KII 5/8/18 779881533 F MWA Non-Revenue Water Emad Al-Tahoni KII 5/8/18 32131082 (for the MWA Office) M Other GIZ Team Leader Johannes Stork KII 5/15/18 799111607 M GIZ Senior Technical Adviser NayefHammad KII 5/10/18 795076657 M GIZ Component Manager Mai Dergham KII 5/7/18 795155696 F GIZ Water Portfolio Lead Frauke Neumann￾Silkow KII 6/7/18 795987183 F ACWUA Secretary General of ACWUA KhaldounKhashman KII 5/31/18 779050888 M ACWUA Capacity Building Ahmad Azzam KII 5/31/18 777425555 M JREDS: Royal Marine Conservation Society Executive Director Ehab Eid KII 5/6/18 775244204 M IFC Principal Investment Officer - IFC MENA Julie Carles KII (over skype) 6/5/18 jcarles@ifc.org F 79 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV KfW Senior Program Coordinator NisreenHaddadin KII 5/16/18 795453172 F AFD Project Officer Atika Ben Aid KII 5/9/18 065004333 F World Bank Lead Water and Sanitation Specialist, WB-IFC MENA SuhailJme'an KII (over skype) 6/5/18 sjmean@worldbank.org M Amman Chamber of Commerce - Rose Smadi KII 5/16/18 796207422 F Greater Amman Municipality - HaneenAyoub KII 5/16/18 798166460 F WMI Staff WMI Utility Mgmt Specialist –Yarmouk Naeem Saleh KII 5/17/18 naeem.saleh@jordan-wmi.org M Consultant YWC X7 Consultant Ahmad Sadouni KII 5/17/18 Ahmad.Sadouni@Jordan￾WMI.org M WMI Chief of Party Jose Valdez KII 5/26/18 jose.valdez@jordan-wmi.org M WMI Deputy Chief of Party George Rizkallah KII 5/26/18 george.rizkallah@tetratech.com M Consultant Consultant for Assignment Agreements Kamal Zoubi KII 5/23/18 zoubikamal@yahoo.com M Segura Governance Consultant Alejandro Deeb KII (over skype) 5/6/18 adeeb@segura-co.net M Segura Financial Management expert Paul Cumiskey KII (over skype) 5/1/31 pauljcumiskey@outlook.com M Segura Founder and CEO Jorge Segura KII (over skype) 6/1/18 js@segura-co.net M WMI Governance Advisor Tamer Al-Assa'd KII 5/23/18 tamer.alassad@jordan￾wmi.org ? 80| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV WMI Water Tech & Engineering Lead Saddam Khleifat KII 5/23/18 saddam.khleifat@jordan￾wmi.org M W4Life Water Demand Mgmt Specialist Noor Esoh KII 5/23/18 noor.esoh@jordan-wmi.org F JSMC Behavior Change Specialist MahaDergham KII 5/22/18 maha.dergham@jordan-wmi.org ? WMI Youth, Gender, Vul. Pops Specialist ShadiaNassar KII 5/22/18 shadia.nassar@jordan-wmi.org ? WMI Utility Commercialization and Finance Specialist Iyad Al-Qassir KII 5/6/18 iyad.alqassir@jordan-wmi.org M WMI Monitoring, Evaluation and Learning Specialist MaramBarqawi KII Maram.barqawi@jordan-wmi.org F USAID & Implementing Partners Water DO Team Project Manager/ WMI COR Louis Qaqish KII 5/29/18 lqaqish@usaid.gov M USAID Cities project Community Government Engagement Team Leader Ban Saraf KII 6/1/18 bsaraf@jordancities.org M USAID FRP Public Private Partnership Advisor Scott B Jazynka KII 6/1/18 sjazynka@delloite.com M USAID WIT Project DCOP RaedNimri KII 5/15/18 772000820 M Ma'an Focus Group MWA Director of Customer Services Directorate Ahmad Abu-Allah Abdu - AlDayem FGD 5/28/18 775435335 M MWA Jabi Same'elAtteyeh FGD 5/28/18 32131082 M MWA Jabi Ahmad Jamal Ahmad FGD 5/28/18 32131082 M 81 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV MWA Jabi Khaled Mansour FGD 5/28/18 32131082 M MWA Data Entry Saleh Ahmad Abu￾Odeh FGD 5/28/18 32131082 M MWA Head of Collection Unit Ayed Mahmoud Ahmad Al￾Khawaldeh FGD 5/28/18 32131082 M MWA N/A Ibrahim Mahmoud Mefleh Abdu-Al￾Dayem FGD 5/28/18 32131082 M MWA Center Jabi Mohammad Mousa Al-Tahan FGD 5/28/18 32131082 M MWA Head of Collection Department Mohammad Abdu￾Aldayem FGD 5/28/18 32131082 M MWA Jabi ZiadMarzouq Al￾Talhouni FGD 5/28/18 32131082 M Interns Focus Group WMI Intern Dareen Abu Ghoush FGD 5/23/18 799258769 F Miyahuna, ZWA General Civil Engineer ShorouqKhawaldeh FGD 5/23/18 798040253 F MWI/WAJ Institutional Development Bashar Alkhlaifat FGD 5/23/18 799784547 F MWI/WAJ Institutional Development Saja Abu Al￾Hommos FGD 5/23/18 798513666 F Miyahuna, ZWA Civil Engineer Amani Alsayyed FGD 5/23/18 796632313 M WUA Focus Group WUA 33 Head of WUA 33 MawafKrayem Abdallah FGD 5/28/18 788689832 M Al-Baqura WUA Head of Al-Baqura WUA parcel Ali Al-Hussein FGD 5/28/18 785600919 M 82| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV WUA Pump 41 Head of WUA Pump 41 Zaki Mohammad Al-Rababaa' FGD 5/28/18 797013360 M TOTAL Male - 90 Female - 27 Total - 117 83 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV ANNEX V: WMI LOGIC MODEL AS OF Q2 2018 AO 1: Water Sector Management Improved 1.1 Percent change in cost recovery of water sector (disaggregated by utility) (M-PMP 3.3.2.b)  1.2 Percentage of NRW (by utility) (M-PMP 3.3.1.a) 1.3 Cubic meters of water saved annually from water-saving technologies or techniques as a result of USG assistance (M-PMP 3.3.1.b)  AO 2: Water Sector Governance Improved 2.1 Number of action plans implemented for water security, integrated water resource management, and/or water source protection as a result of USG assistance (M-PMP 3.3.2.a) (F HL.8.3-2) 2.2 Number of legal instruments drafted, proposed or adopted with USG assistance designed to promote gender equality or non-discrimination against women or girls at the national or sub-national level (M-PMP 4.2.a) (F Gender-1) WMI Goal: Measurable improvement and greater sustainability of Jordan water sector is achieved 1. Percentage of NRW (National Average) (M-PMP 3.3.a) 2. Number of policies, regulations and administrative procedures relating to water and natural resources in development, passed, or being implemented as a result of USG assistance (M-PMP 3.3.b)  3. Cubic meters of water provided or saved as a result of USG assistance (M-PMP 3.3.c) Sub-AO 1.1: Performance and organizational capacity improved 1.1.1 Number of operation and maintenance improvements implemented as a result of USG assistance 1.1.2Number of employees who pass the certification program in water related topicsas a result of USG assistance 1.1.3 Number of private sector participation transactionsas a result of USG assistance IR 3.3: Accountable, Sustainable Management of Water Resources Increased Sub-AO 1.2: Water supply &quality best practices induced 1.2.1Number of implementations of technology or systems to improve the monitoring of the ground water abstraction as a result of USG assistance 1.2.2 Number of water utility supply systems that apply the WQRM framework for safe drinking water supply and preventive risk management approachesas a result of USG assistance 1.2.3 Number of adoptions of technologies or techniques to save water or increase reuse of water (M-PMP 3.3.1.2.a)  GOAL: Improved Prosperity, Accountability, and Equality for a Stable, Democratic Jordan Sub-AO 2.2: Engagement in critical water scarcity issues strengthened 2.2.1 Number of people reached by water-related BCC and social marketing campaigns 2.2.2 Number of people who have improved understanding of critical water scarcity issues facing Jordan, including the cost of water and importance of conservation 2.2.3 Number of initiatives undertaken by GOJ water entities, CSOs (including Media outlets), community groups or beneficiaries to promote water conservation as a result of USG assistance DO 3: Social Sector Quality Improved Sub-AO 2.1: Broad-based water sector reform is enhanced 2.1.1 Number of approved best practices to improve the water sector governance as a result of USG assistance 2.1.2 Number of functions transferred from JVA to WUAs or new representative entities as a result of USG assistance Crosscutting Indicators CCAO1: Number of people educated on tools, approaches, and/or methods for water security, integrated water resource management, and/or water source protection as a result of USG assistance (M-PMP 3.3.2.1.a) (F HL.8.3-1)  Context Indicator Liters of water per capita per day Context Indicator Number of employees per 1000 subscribers (context) 84| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV 85 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV ANNEX VI: INTERVIEW TOOLS 1. KII Guide for IMF Name: Date: Title: Gender: Duration of Involvement with WMI: 1. How has WMI built the ministry capacities to handle the preparation of the reports? What has been the most important area of support in capacity building? In what areas could capacity building be strengthened? What would WMI’s role in that be? 2. To the extent that you are aware and involved in the activities, could you please elaborate on the following activities supported by WMI? ACTIVITY How relevant has this activity been to advance IMF action plan objectives? How effective? (How useful has the activity been to you? Has it contributed to positive change?) Why? WMI Strategy to support MWI in implementing the IMF Action Plan WMI Support to preparation of the quarterly reports Estimating the cost of hosting Syrians on the water sector of Jordan Communication workshop for the IMF action plan Implementation of Performance-Based Contracting and PSP to Improve O&M Services 86| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV Developing the Master Workplan for NRW reduction in Jordan Training and technical support in NRW reduction Efforts to improve Financial Management, Determination of Cost and Subsidy of Services Support to improve financial accounting and fixed asset accounting policies Supporting WAJ in FARA accreditation Support the development of the IT master plan, WAJ IT support, GIS and Control Room 3. Do you have any recommendations to that could be undertaken by the WMI project that could further advance compliance or capacity to support IMF action plan objectives? 87 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV 2. KII Guide for GIZ Name: Date: Title: Gender: Duration of Involvement with WMI: 1. Could you elaborate on the coordination between WMI and GIZ in the water sector? Prompt using specific interventions listed below. ACTIVITY How relevant has this activity been to advance objectives? Why? Why not? Task force and eight policy technical committees formulated Hardware needed for X7 implementation in YWC Defining the role of the WUAs in the management of the distribution system 2. In your opinion which of the WMI approaches are most important to achieve water reform objectives? Why? 3. In your opinion, in the coming two years, what measures can improve promotion and adoption of respective activities supported by WMI that can help promote the reform agenda? 4. To the extent that you are aware, which WMI supported activities/pilots/approaches do you consider as most effective to replicate? Why? 88| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV 5. Do you have any recommendations to that could be undertaken by the WMI project that could further advance the joint aims of WMI and GIZ? 3. KII Guide for MWI Name: Date: Title: Gender: Duration of Involvement with WMI: Effectiveness 89 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV 1. I understand the WMI project has been working closely with you. Which of the USAID WMI activities were you involved in? (Prompt from the list below if needed). In your opinion how effective were they in helping to accomplish the objectives of the MWI in water sector reform? Which in your opinion were most effective? Least effective? Why do you think so? Is there anything that could have been done differently? Please elaborate. Activity Engagement In which subcomponent? How relevant Why? What should have the project done differently? How effective? (How useful has the activity been to you? Has it contributed to positive change?) Why? What do you think will be the outcomes and benefits as a result of these activities? (relevant , to some extent, not relevant Define general sector strategy Water Sector Policy (e.g. policy actions plans, NRW strategy, PSP strategy,etc) Water policy action plans NRW Strategy PSP Strategy WDM Strategy supporting MWI commitment along with IMF Action Plan Economic regulation 90| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV Utility Assignment Agreements Developed/ Modified (including CABs) Communication and outreach strategy and action plan Gender and Youth participation Update urban Water Demand Forecasting Create opportunities for treated wastewater sale to the private sector Demand Management Action Plan for Water Conservation and Increased Reuse/ Water Use Efficiency (WUE) Water monitoring, meter testing, GIS integration, regulations Water Safety Plans Support to Women Studies Unit 2. In your opinion, in the coming two years, what measures can improve promotion and adoption of respective activities supported by WMI that can help promote the reform agenda? In your opinion, are there additional or different measures that could be taken to support the momentum for reform? Activity Possible new measures 91 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV 3. In relation to WMI strategies and approaches (modalities of work), which approaches have been helpful to you? Which have helped strengthen achievements? Elaborate on how and why? Which of the WMI approaches have been least helpful to you? Have any weakened achievements? Elaborate on how and why? (prompt: for example engagement of stakeholders, needs assessments, ensuring buy-in, sufficient training, consultation) 4. Certain activities were changed during the course of the project either entirely or partially, or in terms of the approaches: what in your view were the reasons and how will such changes affect the WMI outcomes? 5. WMI supported a gender, youth assessment and KAP survey: are you aware of it? If yes, do you see a use of it in your work? Have you used it already? How? Sustainability 6. Did you participate in the design and planning the respective measures related to implementation of the WMI project support to the MWI? In your opinion, is the design or approach suitable for lasting benefits? If yes, why? If not, what would you change? 7. Does your ministry/unit have sufficient technical/financial/human capacity and know-how in developing and/or implementing respective strategies for WMI supported activities? If yes, what was WMI role in it? If not then why and what is needed to address the gaps? 8. How do you see your role in relation to continuing the activities/momentum for reform supported by WMI? For example, is there a sustainability plan in place? Can you elaborate? 9. How have connections and relations built with WMI support or with WMI helped support your performance or reform agenda? Can you elaborate on how these relationships have helped/hindered your work? 10. What recommendation would you give to ensure the sustainability of the interventions and by whom? (Prompt: financial support, technical assistance, increased institutional buy-in, increased coordination) 92| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV 11. How do you see WMI supporting active engagement of end users and community members for good governance and social accountability? Is WMI support sufficient in this regard? What else could be done or done differently? 12. How have WMI interventions contributed to the achievement of IWRM? What in the way of technical assistance or other kind of support is needed to meet those objectives? Whose role is it? Learning 13. In your opinion what activities are the most important to focus on in the coming two years to ensure meeting of objectives? Please elaborate on why and how these should be prioritized? 14. Which WMI supported activities/pilots/approaches do you consider as most effective to replicate? Is the MWI already planning replication? If yes, for which and how? 15. What are your top most concerns about the sector reform? In your opinion, with respect to achieving water sector reform, are there issues that need to be better addressed to achieve water reform objectives and by whom? Please elaborate. How could this be better addressed? Is there a role in filling these gaps for WMI in technical assistance? Other areas? Other actors? What role would that be? 93 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV 4. KII Guide for WAJ Name: Date: Title: Gender: Duration of Involvement with WMI: Effectiveness 1. I understand the WMI project has been working closely with you. Which of the USAID WMI activities were you involved in? (Prompt from the list below if needed). In your opinion how effective were they in helping to accomplish the objectives of the MWI in water sector reform? Which in your opinion were most effective? Least effective? Why do you think so? Is there anything that could have been done differently? Please elaborate. Activity Engagement In which subcomponent? How relevant? Why? What should have the project done differently? How effective? (How useful has the activity been to you? Has it contributed to positive change?) Why? What do you think will be the outcomes and benefits as a result of these activities? (relevant , to some extent, not relevant) Bulk water management Water monitoring, meter testing, GIS integration, regulations (please reflect on smart meter, remote sensing 94| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV and GIS; upgrading LYMS system; amendment of the by￾law and enforcement mechanism) Development & Support of Enforcement Mechanism for the GW By-law (please reflect on preparation of the draft amendment of the by￾law among others) Rationalize groundwater monitoring and management functions (please reflect on Improve groundwater monitoring standards, identify the legal, technical, financial, social and behavioral factors for sustainability of groundwater monitoring, assessment, and management services among others) Creation of a Business Unit for Private Wells Improving Wastewater Treatment Performance (Please reflect on the assessment of the WWTPs creation of wastewater committee, among others) WHO Water Safety Plans in 5 Systems Support in relation to IT systems (Please reflect on: establishing the IT Strategy Team, IT strategy, WAJ ICTU roadmap, the Assessment of GIS Capacity in the Water Sector, GIS survey, 95 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV report on Available Solutions for Hosting Oracle Applications Suite for WAJ and YWC. Miyahuna Business Process Review (BPR), X7 Deployment etc) Develop communication materials (Please reflect on the development of a physical model representing the Water Cycle, developed a schematic for the Water Supply System, among others) Debt management strategy and implementation (example Ma’anAmiri law enforcement, new meters for utilities, etc) Planning and implementing PIP for utilities (Yarmouk water company PIP, etc) 2. In your opinion, in the coming two years, what measures can improve promotion and adoption of respective activities supported by WMI that can help promote the reform agenda? Activity Possible new measures 3. Certain activities were changed during the course of the project either entirely or partially, or in terms of the approaches: what in your view were the reasons and how would such changes affect the WMI outcomes? 96| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV 4. Do you think that WMI support to more active involvement of women and youth in the sector was sufficient? Please elaborate. What areas advanced involvement? What, if anything, could be improved or done to advance women and youth involvement? 5. What measures can improve promotion and adoption of respective policies that WMI supports that you are engaged in/aware of? Activity Possible new measures 6. In relation to WMI strategies and approaches (modalities of work), which approaches have been helpful to you? Which have helped strengthen achievements? Elaborate on how and why? Which of the WMI approaches have been least helpful to you? Have any weakened achievements? Elaborate on how and why? (prompt: for example engagement of stakeholders, needs assessments, ensuring buy-in, sufficient training, consultation) Sustainability 7. Did you participate in the design and planning the respective measures related to implementation of the WMI support to the WAJ? In your opinion, is the design or approach suitable for lasting benefits? If yes, why? If not, what would you change? 8. Does the staff of your unit have sufficient technical/financial/human capacity and know-how in developing and/or implementing respective strategies for WMI supported activities? If yes, what was WMI role in it? If not then why and what is needed to address the gaps? 9. How do you see the role of WAJ in relation to sustaining the improvements achieved in utilities performance? Does WAJ have the capacity to monitor and follow up the improvements? 10. How likely it is that the USAID MWI –supported policies and activities supporting WAJ will be sustainable after the project end and why? Please reflect, inter alia on the existence of qualified staff, financial resources, support by the government, likelihood of the adoption of the laws and by laws, potential plans to enforce the implementation of WSP in all water utilities in Jordan, etc) 97 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV 11. How have connections and relations built with WMI support or with WMI helped support your performance or reform agenda? Can you elaborate on how these relationships have helped/hindered your work? 12. How do you see WAJ supporting active engagement of end users and community members for good governance and social accountability? Is WMI support sufficient in this regard? What else could be done or done differently? Learning 13. In your opinion what activities are the most important to focus on in the coming years to ensure meeting of objectives? Please elaborate on why and how these should be prioritized? Activity Suggestions for the areas of enhancement/disengagement 14. Which WMI supported activities/pilots/approaches do you consider as most effective to replicate? Is the WAJ already planning replication? If yes, for which and how? 15. What are your top most concerns about the sector reform? In your opinion, with respect to achieving water sector reform, are there issues that need to be better addressed to achieve water reform objectives and by whom? Please elaborate. How could this be better addressed? Is there a role in filling these gaps for WMI in technical assistance? Other areas? Other actors? What role would that be? 98| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV 5. KII Guide for JVA Name: Date: Title: Gender: Duration of Involvement with WMI: Effectiveness 1. I understand the WMI project has been working closely with you. Which of the USAID WMI activities were you involved in? (Prompt from the list below if needed). In your opinion how effective were they in helping to accomplish the objectives of the MWI in water sector reform? Which in your opinion were most effective? Least effective? Why do you think so? Is there anything that could have been done differently? Please elaborate. Activity Engagement In which subcomponent? How relevant Why? What should have the project done differently? How effective? (How useful has the activity been to you? Has it contributed to positive change?) Why? What do you think will be the outcomes and benefits as a result of these activities? (relevant , to some extent, not relevant) Design, implementation and support of new JVA distribution company Support residual JVA and water users association 99 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV (Support JVA reform and governance improvement, Develop the irrigation regulatory framework) Measurement, Analysis and Recommendations of Water Losses Reduction in King Abdullah Canal Support Capacity building needs assessment for JVA staff Support capacity building and governance strengthening for WUAs 2. In relation to WMI strategies and approaches (modalities of work), which approaches have been helpful to you? Which have helped strengthen achievements? Elaborate on how and why? Which of the WMI approaches have been least helpful to you? Have any weakened achievements? Elaborate on how and why? (prompt: for example engagement of stakeholders, needs assessments, ensuring buy-in, sufficient training, consultation) 3. Certain activities were changed during the course of the project either entirely or partially, or in terms of the approaches: what in your view were the reasons and how would such changes affect the WMI outcomes? 4. WMI supported a gender, youth assessment and KAP survey: are you aware of it? If yes, do you see a use of it in your work? Have you used it already? How? Sustainability 5. Did you participate in the design and planning the respective measures related to implementation of the WMI project support to the MWI? In your opinion, is the design or approach suitable for lasting benefits? If yes, why? If not, what would you change? 100| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV 6. Does JVA have sufficient technical/financial/human capacity and know-how in developing and/or implementing respective strategies for WMI supported activities? If yes, what was WMI role in it? If not then why and what is needed to address the gaps? 7. How do you see your role in relation to continuing the activities/momentum for reform supported by WMI? For example, is there a sustainability plan in place? Can you elaborate? 8. How likely are the activities supported by WMI to be sustained by JVA? In particular: • How would you assess the potential for sustainability of the work now started supporting WUAs? Are there any risks to sustainability? If so,, what are they? What should be done to mitigate the risks? • Would you be able to continue training on new measurement method of losses (related to KAC) for your employees? If yes then how? If no, then why not? Intervention How likely is the sustainability? Why? Recommendation (likely, not sure, unsustainable) 1 Design, implementation and support of new JVA distribution company 2 Support residual JVA and water users association (Support JVA reform and governance improvement, Develop the irrigation regulatory framework 3 Measurement, Analysis and Recommendations of Water Losses Reduction in King Abdulla Canal 101 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV 4 Support Capacity building needs assessment for JVA staff 5 Support capacity building and governance strengthening for WUAs Learning 9. In your opinion what activities are the most important to focus on in the coming two years to ensure meeting of objectives? Please elaborate on why and how these should be prioritized? Activity Suggestions for the areas of enhancement/disengagement 10. Which WMI supported activities/pilots/approaches do you consider as most effective to replicate? Is the JVA already planning replication? If yes, for which and how? 11. What are your top most concerns about the sector reform? In your opinion, with respect to achieving water sector reform, are there issues that need to be better addressed to achieve water reform objectives and by whom? Please elaborate. How could this be better addressed? Is there a role in filling these gaps for WMI in technical assistance? Other areas? Other actors? What role would that be? 102| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV 6. KII Guide for Ministry of Environment Name: Date: Title: Gender: Duration of Involvement with WMI: Effectiveness 1. I understand the WMI project has been working closely with you. Were you involved in the WMI support to the Ministry of Environment to Enhance Reclaimed Water Quality Monitoring following activity? If yes, which activities were you involved in? (Prompt from the list below if needed). In your opinion how effective were they in helping to accomplish the objectives of the MWI in water sector reform? Which in your opinion were most effective? Least effective? Why do you think so? Is there anything that could have been done differently? Please elaborate. Activity Engagement In which subcomponent? How relevant Why? What should have the project done differently? How effective? (How useful has the activity been to you? Has it contributed to positive change?) Why? What do you think will be the outcomes and benefits as a result of these activities? (relevant , to some extent, not relevant) Support of GIS unit Training program on GIS Assessment of existing capacities and systems related 103 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV to water quality protection Enhance Wastewater￾Related Enforcement Capacity and Minimize Management Burden. 2. Does the USAID WMI support to GIS unit help the Ministry of Environment to improve the current practices in reclaimed water quality monitoring? If yes, how? If no, what could to be done differently? 3. In relation to WMI strategies and approaches (modalities of work), which approaches have been helpful to you? Which have helped strengthen achievements? Elaborate on how and why? Which of the WMI approaches have been least helpful to you? Have any weakened achievements? Elaborate on how and why? (prompt: for example engagement of stakeholders, needs assessments, ensuring buy-in, sufficient training, consultation) 4. Are you planning to establish join server/system with the Ministry of Water and Irrigation to share data and information about the reclaimed water quality? Sustainability 5. Did you participate in the design and planning the respective measures related to implementation of the WMI support to the MoEnv? In your opinion, is the design or approach suitable for lasting benefits? If yes, why? If not, what would you change? 6. Does your ministry/unit have sufficient technical/financial/human capacity and know-how in developing and/or implementing respective strategies for WMI supported activities? If yes, what was WMI role in it? If not, then why and what is needed to address the gaps? 104| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV 7. How have connections and relations built with WMI support or with WMI helped support your performance or reform agenda? Can you elaborate on how these relationships have helped/hindered your work? 8. Do you have a plan to implement the MWI recommendations? If yes, what has been most helpful to you? If no, what are obstacles to implementation of recommendations? 9. What recommendation would you give to ensure the sustainability of the interventions and by whom? (Prompt: financial support, technical assistance, increased institutional buy-in, increased coordination) 10. Does MoEnv have enough technical and financial capacity to run and follow up the GIS unit? 11. What recommendations could be given to ensure the sustainability of the GIS interventions? Learning 12. In your opinion what activities are the most important to focus on in the coming two years to ensure meeting of objectives with respect to the MoEnv role in advancing WMI-supported activities? Please elaborate on why and how these should be prioritized? 13. Which WMI-supported activities/pilots/approaches do you consider as most effective to replicate? Is the MoEnv already planning replication? If yes, for which and how? 105 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV 7. KII Guide for ZWA Name: Date: Title: Gender: Duration of Involvement with WMI: Effectiveness 1. Are you aware of the PIP? Were you involved in its development and implementation? How? Has the PIP been helpful to you? How? (Probe on suggested areas to change and why) 2. Which interventions/activities were you involved in (prompt using the list below If needed). In your opinion how effective were they in helping to accomplish the objectives of the PIP? Elaborate on how it was effective and why; or why it was not effective. Intervention Role How relevant were WMI interventions in helping your company accomplish the goals of its PIP? If they were relevant then how? And if not - then Why? If not, then why? What were key challenges [constraints, obstacles] in implementing the PIP, and what caused these challenges? How were these challenges managed and how can they be addressed in the future? (relevant , to some extent, not relevant) 1 Billing and Collection 2 NRW 3. In terms of WMI interventions supporting your institution (as above), can you elaborate on how these interventions have helped support change desired in the areas listed below? How effective was the WMI support in these areas, and what were the strongest factors contributing to their successful implementation? Any challenges? 106| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV Intervention How effective? (How useful has the activity been to you? Has it contributed to positive change?) Key achievements or challenges Factors (external and internal) a) not effective b) somewhat effective c) very effective 1 Billing and Collection 2 NRW 4. Is there a gender focal point and policy at Zarqa? Are you planning to have one? Is there a gender HR policy in place? Was it supported by WMI? Is it in use? If yes, what are the outcomes and what do you see as next steps to advance gender equity? 5. In relation to Water Demand Management, what are the constraints/challenges you face in WDM adoption (improved water use/management for water savings) at your company? Is the action plan in place sufficient to cover these challenges and constraints? Elaborate. 6. How do you ensure the link with customers? Are there established mechanisms of cooperation with CBOs/CSOs for example? Would you support a CAB? If not- why? 7. What do you think about the internship program? Are you planning to hire any of the interns? How does the internship approach serve your management needs? Are you planning to have your own internship program and if not why? 8. Are you aware of USAID WMI assistance to MWI in terms of policy support? Were you part of discussions on policy? What aspects of reform are most important to you and your institution? Why? Are these being addressed? Is there anything else that should be tackled or addressed differently? Elaborate. 9. How have connections and relations built with WMI support and with WMI helped support your institution’s performance? Can you elaborate on how these relationships have helped/hindered your work? 107 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV Sustainability 10. Which of the improvements accomplished through cooperation with WMI until now is most likely to be sustained or adopted? And why (please reflect on policy, as well as company level constraints, among others)? Which will be difficult to sustain? 11. Of the interventions achieved to date, have any been integrated into operations or sustainability planning to maintain the improvements achieved so far over the course of the project? Has this plan been successfully implemented? Intervention How likely is the sustainability? Why? Recommendation (likely, not sure, unsustainable) 1 Billing and Collection 2 NRW 12. In terms of relationship building for improved performance and reform, how would you envision relations that could support those aims? How could USAID/WMI support or strengthen that? Learning 13. In your opinion, which incomplete project activities should be prioritized for completion during the remaining project period of performance? Are there any incomplete project activities that should be stopped? And why? Activity Suggestions for the areas of enhancement/disengagement? 108| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV 14. Were there strategies or practices implemented that should be replicated? And why? (Probe on missing elements to be incorporated in these strategies and practices before replication, factors contributing to strength) 109 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV 8. KII Guide for YWC Name: Date: Title: Gender: Duration of Involvement with WMI: Effectiveness 1. Are you aware of the PIP? Were you involved in its development and implementation? How? Has the PIP been helpful to you? How? (Probe on suggested areas to change and why?) 2. Which interventions/activities were you involved in (prompt using the list below if needed). In your opinion how effective were they in helping to accomplish the objectives of the PIP? Elaborate on how it was effective and why or why not it was ineffective. Intervention Role How relevant were WMI interventions in helping YWC accomplish the goals of its PIP? If they were relevant then how? And if not - then Why? If, then why? What were key challenges [constraints, obstacles] in implementing the PIP, and what caused these challenges? How were these challenges managed and how can they be addressed in the future? (relevant , to some extent, not relevant) 1 NRW 2 Collection improvements 3 ERP upgrade 4 Fixed Asset identification 110| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV 5 FARA facility preparation 6 HR 7 X7 8 Business planning￾9 Complaint response mechanism 3. In terms of WMI interventions supporting your institution (as above), can you elaborate on how these interventions have helped support change desired in the areas listed below? How effective was the WMI support in these areas, and what were the strongest factors contributing to their successful implementation? Any challenges? Intervention How effective? Key achievements or challenges Factors (external and internal) a) not effective; (b) somewhat (c) very effective 1 NRW 2 Collection improvements 3 ERP upgrade 4 Fixed Asset identification 5 FARA facility preparation 6 HR 7 X7 8 Business planning 9 Complaint response mechanism 111 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV 4. Can you elaborate on the governance review and the gender studies? How did this evolve? How useful were these? How has the company utilized them? Or planning to? Now that the studies are ready, what do you plan to do with them? 5. Is there HR policy on gender approved at your company? If yes, is the policy in use? What are the outcomes? What do you see as a next step to advance gender equity? 6. In relation to Water Demand Management, what are the constraints/challenges you face in ensuring WDM adoption (improved water use/management for water savings) at your company? Is the action plan in place sufficient to cover these challenges and constraints? Elaborate. 7. How do you ensure the link with customers? Are there established mechanisms of cooperation with CBOs/CSOs for example? Would you support a CAB? If not- why? 8. What do you think about the internship program? Are you planning to hire any of the interns? How does the internship approach serve your management needs? Are you planning to have your own internship program and if not why? 9. Are you aware of the WMI assistance to MWI in terms of policy support? Were you part of discussions on policy? What aspects of reform are most important to you and your institution? Why? Are these being addressed? Is there anything else that should be tackled or addressed differently? Elaborate. 10. How have connections and relations built with WMI support and with WMI helped support your institution’s performance? Can you elaborate on how these relationships have helped/hindered your work? 11. In relation to improving Wastewater Treatment Performance: did YWC participate in the assessment of the WWTPs? • If yes, how? How would you describe the assessment’s consideration of the current conditions? Its consideration of potential options for improvement? • If not, why? 112| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV • Do you think the WWTP will contribute to improved performance? How? What factors will support improved performance? Why? Sustainability 12. Which of the improvements accomplished through cooperation with WMI until now is most likely to be sustained or adopted? And why (please reflect on policy, as well as company level constraints, among others)? Which will be difficult to sustain? 13. Of the interventions achieved to date, have any been integrated into operations or sustainability planning to maintain the improvements achieved so far over the course of the project? Has this plan been successfully implemented? Intervention How likely is the sustainability? Why? Recommendation likely, not sure, unsustainable) 1 NRW 2 Collection improvements 3 ERP upgrade 4 Fixed Asset identification 5 FARA facility preparation 6 HR 7 X7 8 Business planning￾9 Complaint response mechanism 113 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV 14. In terms of relationship building for improved performance and reform, how would you envision relations that could support those aims? How could USAID/WMI support or strengthen that? Learning 15. In your opinion, which incomplete project activities should be prioritized for faster completion during the remaining project period of performance? Are there any incomplete project activities that should be stopped? And why? Activity Suggestions for the areas of enhancement/disengagement 16. Were there strategies or practices implemented that should be replicated? And why? (Probe on missing elements to be incorporated in these strategies and practices before replication, and factors of contributing to strength of performance) 114| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV 9. KII Guide for Ma’an Water Authority (MWA) Name: Date: Title: Gender: Duration of Involvement with WMI: Effectiveness 1. I understand the WMI project has been working closely with you. Could you elaborate on your role within the WMI program of support to Ma’an Water Administration? 2. Which interventions/activities were you involved in (prompt using the list below if needed). In your opinion how effective were they in helping to accomplish the objectives of increased utility efficiency and performance? Elaborate on how it was effective and why; or why it was not effective. Intervention Role How relevant for MWA? If they were relevant then how? And if not -then Why? How effective? (How useful has the activity been to you? Has it contributed to positive change?) Key Achievements and challenges Factors (internal and external) contributing to challenges or achievements (relevant , to some extent, not relevant) 1 Operational Assessment Report 2 Allocating 1000 new customer meters 115 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV 3 Replacement of bulk meters at 20 wells 4 Development of a hydraulic model for a main system in Shoubak 5 Debt collection campaign 6 Customer meter replacement campaign. 7 Work with Aqaba Water to update the billing reporting system at Ma’an. 8 Training for all customer service staff at Ma’an Water Administration 9 implementation of an archiving system for Ma’an Customer files 10 support in preparing and distribution of Barcodes for water meters for efficient reading time. 11 Geographic Information System (GIS): 116| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV Capacity building program 12 New Technologies : 12 (a) Pulsers 12 (b) HHUs 12 (c) New water meters and automated reading modules on water production wells 3. In relation to Water Demand Management, what are the constraints/challenges you face in ensuring WDM adoption (improved water use/management for water savings) at your company? Is the action plan in place sufficient to cover these challenges and constraints? Elaborate. 4. Did you benefit from WMI support in company governance review? If yes, what were the outcomes and what do you see as a next step? 5. Is there HR policy on gender approved at your company? If yes, is the policy in use? What are the outcomes? What do you see as a next step to advance gender equity? 6. How do you ensure the link with customers? Are there established mechanisms of cooperation with CBOs/CSOs for example? Would you support a CAB? If not- why? 7. What do you think about the internship program? Are you planning to hire any of the interns? How does the internship approach serve your management needs? Are you planning to have your own internship program? 117 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV 8. Are you aware of WMI assistance to MWI on policy support and reform? Have you been involved in policy level discussions? What aspects of reform are most important to you and your institution? Why? Are these being addressed? Is there anything that WMI could support or address differently? 9. How have connections and relations built with WMI support and with WMI helped support your institution’s performance? Can you elaborate on how these relationships have helped/hindered your work? 10. In relation to improving Wastewater Treatment Performance: did MWA participate in the assessment of the WWTPs? • If yes, how? How would you describe the assessment’s consideration of the current conditions? Its consideration of potential options for improvement? • If not, why? • Do you think the WWTP will contribute to improved performance? How? What factors will support improved performance? Why? Sustainability 11. Is MWA ready to be transferred to AWC? If yes, what was the contribution of WMI, of any? And if not, then why? 12. Which of the improvements accomplished through cooperation with WMI until now is most likely to be sustained or adopted? And why (please reflect on policy, as well as company level constraints, among others)? Which will be difficult to sustain? 13. Of the interventions achieved to date, have any been integrated into operations or sustainability planning to maintain the improvements achieved so far over the course of the project? Has this plan been successfully implemented? Intervention How likely is the sustainability? Why? Recommendation 118| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV (likely, not sure, unsustainable) 1 Operational Assessment Report 2 Allocating 1000 new customer meters 3 Replacement of bulk meters at 20 wells 4 Development of a hydraulic model for a main system in Shoubak 5 Debt collection campaign 6 Customer meter replacement campaign. 7 Work with Aqaba Water to update the billing reporting system at Ma’an. 8 Training for all customer service staff at Ma’an Water Administration 9 implementation of an archiving system for Ma’an Customer files 10 support in preparing and distribution of Barcodes for water 119 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV meters for efficient reading time. 11 Geographic Information System (GIS): Capacity building program 12 New Technologies : 12 (a) Pulsers 12 (b) HHUs 12 (c) New water meters and automated reading modules on water production wells 14. In terms of relationship building for improved performance and reform, how would you envision relations that could support those aims? How could USAID/WMI support or strengthen that? Learning 15. Which incomplete project activities should be prioritized for completion during the remaining project period of performance? Are there any incomplete project activities that should be stopped? And why? Activity Suggestions for the areas of enhancement/disengagement? 16. Were there strategies or practices implemented that should be replicated? And why? Are there any missing elements to be incorporated in these strategies and practices before replication? 10. 120| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV KII Guide for Aqaba Water Company (AWC) Name: Date: Title: Gender: Duration of Involvement with WMI: Effectiveness 1. I understand the WMI project has been working closely with you. Could you elaborate on your role within the WMI program of support to Aqaba Water Company? 2. Which interventions/activities were you involved in (prompt using the list below if needed). In your opinion how effective were they in helping to accomplish the objectives of increased utility efficiency and performance? Elaborate on how it was effective and why or why not it was ineffective. Intervention Role How relevant? If they were relevant then how? And if not - then Why? How effective? (How useful has the activity been to you? Has it contributed to positive change?) Key Achievements and challenges Factors contributing to challenges or achievements (relevant , to some extent, not relevant) 1 Aqaba Water and Wastewater Investment Master Plan 2 Support Aqaba Water Performance Improvement 3 Upgrade of ERC 121 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV 4 Twinning Program 3. In relation to Water Demand Management, what are the constraints/challenges you face in ensuring WDM adoption (improved water use/management for water savings) at your company? Is the action plan in place sufficient to cover these challenges and constraints? 4. Did you benefit from WMI support in company governance review? If yes, what were the outcomes and what do you see as a next step? 5. Is there HR policy on gender approved at your company? If yes, is the policy in use? What are the outcomes? What do you see as a next step to advance gender equity? 6. How do you ensure the link with customers? Are there established mechanisms of cooperation with CBOs/CSOs for example? Would you support a CAB? If not- why? 7. Are you aware of WMI assistance to MWI in terms of policy support and reform? Were you part of discussions on policy? What aspects of reform are most important to you and your institution? Why? Are these being addressed? Is there anything else that should be tackled or addressed differently? 8. How have connections and relations built with WMI support and with WMI helped support your institution’s performance? Can you elaborate on how these relationships have helped/hindered your work? 9. In relation to improving Wastewater Treatment Performance: did AWC participate in the assessment of the WWTPs? • If yes, how? How would you describe the assessment’s consideration of the current conditions? Its consideration of potential options for improvement? • If not, why? • Do you think the WWTP will contribute to improved performance? How? What factors will support improved performance? Why? Sustainability 10. Does AWC have the capacity to include MWA to its mandate and service area? (a) Technical______________ (b) Financial _______________ 11. Does AWC have strong and clear communication channel with MWA? 12. Which of the improvements accomplished through cooperation with WMI until now is most likely to be sustained or adopted? And why (please reflect on policy, as well as company level constraints, among others)? Which will be difficult to sustain? 13. Of the interventions achieved to date, have any been integrated into operations or sustainability planning to maintain the improvements achieved so far over the course of the project? Has this plan been successfully implemented? Intervention How likely is the sustainability? Why? Recommendation 122| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV (likely, not sure, unsustainable) 1 Aqaba Water and Wastewater Investment Master Plan 2 Support Aqaba Water Performance Improvement 3 Upgrade of ERC 4 Twinning Program 14. In terms of relationship building for improved performance and reform, how would you envision relations that could support those aims? How could USAID/WMI support or strengthen that? Learning 15. In your opinion, which incomplete project activities should be prioritized for completion during the remaining project period of performance? Are there any incomplete project activities that should be stopped? And why? Activity Suggestions for the areas of enhancement/disengagement 16. Were there strategies or practices implemented that should be replicated? And why? (Probe on missing elements to be incorporated in these strategies and practices before replication, factors of contributing to strength of performance) 123 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV 11. KII Guide for Miyahuna Name: Date: Title: Gender: Duration of Involvement with WMI: Effectiveness 1. Are you aware of the PIP? Were you involved in its development and implementation? How? Has the PIP been helpful to you? How? (Probe on suggested areas to change and why) 2. Which interventions/activities were you involved in (prompt using the list below if needed). In your opinion how effective were they in helping to accomplish the objectives of the PIP? Elaborate on how it was effective and why; or why it was not effective. Intervention Role How relevant were WMI interventions in helping your company accomplish the goals of its PIP? If they were relevant then how? And if not - then Why? If , then why? What were key challenges [constraints, obstacles] in implementing the PIP, and what caused these challenges? How were these challenges managed and how can they be addressed in the future? (relevant , to some extent, not relevant) 1 Billing and Collection 2 NRW 3 Miyahuna Madaba Merger 3. In terms of WMI interventions supporting your institution (as above), can you elaborate on how these interventions have helped support change desired in the areas listed below? How effective was the WMI support in these areas, and what were the strongest factors contributing to their successful implementation? Any challenges? 124| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV Intervention How effective? Key achievements or challenges Factors (external and internal) a) not effective b) somewhat effective c) very effective 1 Billing and Collection 2 NRW 3 Miyahuna Madaba Merger 4. Can you elaborate on the governance review and the gender studies? How did this evolve? How useful were these? How has the company utilized them? Or planning to? Now that the studies are ready, what do you plan to do with them? 5. Is there HR policy on gender approved at your company? If yes, what were the outcomes and what do you see as a next step? 6. In relation to Water Demand Management, what are the constraints/challenges you face in WDM adoption (improved water use/management for water savings) at your company? Is the action plan in place sufficient to cover these challenges and constraints? Elaborate. 7. How do you ensure the link with customers? Are there established mechanisms of cooperation with CBOs/CSOs for example? Would you support a CAB? If not- why? 8. What do you think about the internship program? Are you planning to hire any of the interns? How does the internship approach serve your management needs? Are you planning to have your own internship program and if not why? 9. Are you aware about the WMI assistance to MWI in terms of policy support? Were you part of discussions on policy? What aspects of reform are most important to you and your institution? Why? Are these being addressed? Is there anything else that should be tackled or addressed differently? Elaborate. 10. How have connections and relations built with WMI support and with WMI helped support your institution’s performance? Can you elaborate on how these relationships have helped/hindered your work? 11. In relation to improving Wastewater Treatment Performance:, did your company participate in the assessment of the WWTPs? • If yes, how? How would you describe the assessment’s consideration of the current conditions? Its consideration of potential options for improvement? • If not, why? • Do you think the WWTP will contribute to improved performance? How? What factors will support improved performance? Why? Sustainability 12. Which of the improvements accomplished through cooperation with WMI until now is most likely to be sustained or adopted? And why (please reflect on policy, as well as company level constraints, among others)? Which will be difficult to sustain? 125 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV 13. Of the interventions achieved to date, have any been integrated into operations or sustainability planning to maintain the improvements achieved so far over the course of the project? Has this plan been successfully implemented? Intervention How likely is the sustainability? Why? Recommendation (likely, not sure, unsustainable) 1 Billing and Collection 2 NRW 3 Miyahuna Madaba Merger (for Miyahuna Manager only) 14. In terms of relationship building for improved performance and reform, how would you envision relations that could support those aims? How could USAID/WMI support or strengthen that? Learning 15. In your opinion, which incomplete project activities should be prioritized for completion during the remaining project period of performance? Are there any incomplete project activities that should be stopped? And why? Activity Suggestions for the areas of enhancement/disengagement 16. Were there strategies or practices implemented that should be replicated? And why? (Probe on missing elements to be incorporated in these strategies and practices before replication, factors of contributing to strength of performance) 126| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV 12. KII Guide for school management Name: Date: Title: Gender: Duration of Involvement with WMI: 1. Where you involved in WMI activities in your school? What was your role? 2. Did the WMI training and other activities, in your view, meet a need for awareness of water issues for the students? 3. Did you have a chance to be involved in the design of activities? Were the activities appropriately matched to your students needs? If yes, how so? If not, what should have been done differently? 4. In your opinion, how effective were these training and other associated activities? a. Do you think the students learned and internalized new ideas? If yes, then how? If not then why? b. Do you think that they already display changed behavior? If yes, then how? If not then why? c. Are you getting any feedback from the parents? If yes, what kind? 5. Did you incorporate these training into a regular and annual school activity? If yes, then how? And if not then why? 6. What do you think about the potential for replicating this in other schools? If this was done: a. What elements should change and why? b. What should be the role of the Ministry of Education? 13. KII Guide for ACWUA Name: Date: Title: Gender: Duration of Involvement with WMI: 1. I understand you have been working closely with WMI. Could you elaborate on your collaboration with WMI? (If needed prompt from activity list below). 2. How would you assess the importance and usefulness of these activities to advancing your work and water use efficiency? ACTIVITY How relevant has this activity been to advance objectives? How effective? (How useful has the activity been to ACWUA? Has it contributed to positive change?) Why? Why not? 127 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV Non-Revenue Water (NRW) reduction plan Knowledge sharing workshop and Water utility management training 3. Have WMI activities built your capacities? If so, how? 4. Do you have any recommendations to that could be undertaken by the WMI project that could further advance ACWUA’s work in water use efficiency, utility performance, and water demand management? 14. KII Guide for WDM Task Force Members Name: Date: Title: Gender: Duration of Involvement with WMI: 1. Could you elaborate on the activities you participated in with WMI support? ACTIVITY How relevant has this activity been to advance objectives? How effective? (How useful has the activity been to you? Has it contributed to positive change?) Why? Why not? Advisory Work Training 2. In your opinion, as a result of the training, does the staff of the unit have capacity and know how in developing and implementing WDM strategy? If yes, what was most effective in building capacity? If not, why not? What is needed? 3. Which WMI supported activities/pilots/approach/training do you consider as most effective to replicate? Are you planning replication? If yes, for which and how? 4. In your opinion what activities are the most important to focus on in the coming years to ensure meeting of objectives? Please elaborate on why and how these should be prioritized? 5. What is the previous initiatives/ projects implemented in this domain? What are the lessons learned from them? 128| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV 6. In your opinion are there any other activities/measures can improve WDM promotion and adoption? Is there a role for WMI in supporting those? 7. Did you participate in the design and planning the action plan? 8. Who are the main stakeholders in this plan? Please elaborate on coordination.How have connections and relations built with WMI support or with WMI helped support your performance/action plan agenda? Can you elaborate on how these relationships have helped/hindered your work? 9. Which pilots in the pilot are most likely to be replicated? Why? 10. In your opinion, does the action plan inclusive and how does it addresses women and youth inclusion? 15. KII Guide for WAJ ICTU/ Control Unit Name: Date: Title: Gender: Duration of Involvement with WMI: 1. Are you satisfied with the advisory work and training provided by the WMI? If yes, why? If no, why? 2. Which training did you participate in? o By IT expert for the initial training? o By Information and Communication Technology Unit (ICTU) Expert? o By IT Operational Specialist expert o in the training modules for GIS for managers 3. Was the training useful and effective? Do you use the learning in your work? If yes, please give one example, and if not, then why? 4. How relevant and efficient were the activities aimed at developing the following documents? o IT strategy o Available Solutions for Hosting Oracle Applications Suite for WAJ o Assessment of GIS Capacity in the Water Sector Interviewer: probe into the following: (a) Were these activities timely? (b) Where did the idea originate? And (c) Were the processes of development smooth and efficient? 5. Were you personally satisfied with the quality of the documents? If not then why? 6. Is the WAJ ICTU roadmap (high level strategy) formally adopted? If not how likely its formal adoption soon and effective implementation? 7. How advanced is the work on GIS standards for the infrastructural architecture, water utility data, and management practices? 8. Did WAJ incorporate the Unit on ICT in its organizational structure? 9. Did it adopt the recommendations? If yes, what were the outcomes? And if not, then why? 10. WMI is now developing GIS standards for the infrastructural architecture, water utility data, and management practices. Do you think this will help for better data and information exchange among WAJ and water utilities? 129 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV 11. WMI has supported WAJ and water utilities to get a GIS Enterprise License Agreement (ELA) as a strategic solution for the sector. How important was this support and why? 12. How regularly does the WAJ ICTU meet? Do you think the meeting frequency is sufficient or there is a need for more often meetings?Does this support enhance the cooperation/ relations between WAJ and the water utilities? 13. Any recommendations for WMI for the remaining time? 130| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV FGDs 1. FGD for Water USER Associations: 1 FGD Date: Names of participants by gender: 1. What do you see as the most significant challenges in ensuring good performance of water user associations? 2. What do you think about the idea of an O&M company? Was this discussed with you? Who should manage O&M function in your opinion? Why? 3. What do you think about taking over all the O&M functions and responsibilities from the Pumping Stations to farms intakes? What are the most significant challenges in this regard? Why? 4. Do you receive (plan to) sufficient support in capacity building from JVA? Elaborate. Any other recommendations? 5. Do you receive (plan to) sufficient support from WMI or other agencies/projects? Elaborate. Any other recommendations? 2. FGD for Teachers and Parents of one of the schools 1 FGD Date: Names of participants by gender: 7. Did the WMI training and other activities, in your view, meet a need for awareness of water issues for the students? INTRODUCTION: This FGD is organized as part of the midterm review of WMI. We received your contacts through the WMI, but this FGD is organized independently from it as well as the project. We would appreciate your feedback which will then help to improve the design of the project in relation to help improving billing and collection in water companies. INTRODUCTION: This FGD is organized as part of the midterm review of WMI. We received your contacts through the WMI, but this FGD is organized independently from it as well as the project. We would appreciate your feedback which will then help to improve the design of the project in relation to help improving billing and collection in water companies. 131 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV 8. Did you have a chance to be involved in the design of activities? Were the activities appropriately matched to your students needs? If yes, how so? If not, what should have been done differently? 9. In your opinion, how effective were these training and other associated activities? d. Do you think the students learned and internalized new ideas? If yes, then how? If not then why? e. Do you think that they already display changed behavior? If yes, then how? If not then why? 1. What could have been done differently? 2. Should this subject be part of curricula? At which grade? 3. FGD FOR BILLING AND COLLECTION (3): customer service manager, Jabis (Meter readers and Money collectors), IT managers, communication specialists, data quality controllers In 3 areas: Erbid (for YWC) and Zarka city (for ZWA) and Shoubak (for Ma’an) Date: Names of participants by gender: HHUs (Ma’an only) 1. How did the use of the HHUs affect the effectiveness of your work? Please reflect on reading/billing/collection time, ease of reading, convenience, precision, etc. 2. Are you fully satisfied with the HHUs or would you have recommendations for improvement? 3. Were you satisfied with the training you received related to the use of HHUs? Do you have any recommendations for improvement? 4. Who would train newly hired staff (as Jabis)? Do you have a plan to train other staff in using new staff on the use of HHUs? 5. Are the HHUs costly to operate and maintain? In your view would their usage require financial resources, which would be hard to secure? 6. Did you face any resistance from the customers during the reading of the water meters using the HHUs? Billing and Collection (all 3) 7. Planning: Was there any “Annual Billing and Collection Management Plan” that address the different aspects of collection including risk assessment? If yes, how did the WMI support in that? Please elaborate. INTRODUCTION: ThisFGD is organized as part of the midterm review of WMI. We received your contacts through the Water Company, but this FGD is organized independently from it as well as the project. We would appreciate your feedback which will then help to improve the design of the project in relation to help improving billing and collection in water companies. 132| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV 8. Monitoring & controlling implementation: How was the billing and collection plan monitored and controlled during its course of action and what additional resources, tools and procedures put in place to improve the billing and collection rates? 9. Influencing factors: What were the main factors that influenced the improvement/ obstructing the billing and collection; and what was the role of WMI in bring that? How efficient was the utility management in supporting that: • Additional resources, new tools/ equipment/ procedures, linking between staff performance and incentive scheme • Change in the method/ approach of calculating the billing and collection performance • Obstacles: delay of six months due to legal debate in YWC. 10. Customer database updates and data quality: What approved processes/ procedures put in place to sustain continuous updates of customers' database? (online, front office, field staff, etc.) and how was the data quality managed during the whole process (reading, consumption rates, bill value,..) 11. Billing & collection trend (Collection value per quarter in 2015, 2016 and 2017). How was the collection trend during the project lifetime? What do you propose to improve future performance? BEHAVIOUR CHANGE CAMPAIGN (MA’AN ONLY): 12. What do you think about the campaign aimed at customers with debts? DO you think that using the brochures was effective? • If yes, how?Did it result in the HHs with highest debts (over 1500 JOD) settling their debts to avoid Amiri Law? Did it result in the HHs with high debts (500-1000 JOD) settling their debts? • If not, why not? • What would have been more effective? 13. Did the households with debts identify the specific in-house water inefficient practices that resulted in high water bills to your knowledge? If yes, how widespread was this and If not, why not? What should have been done differently? 133 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV 4. FGD for BUSINESSES (through AMMAN and ZARKA Chambers of commerce) 1 FGD Date: Names of participants by gender: WUE 3. Are you aware of WUE activities promoted by WMI, targeting private businesses? • If yes, do you think they are sufficient and effective? What could have been done differently? • What should the WMI do in the remaining 2 years? • What expectations do you have from the Government? Recycled water reuse 4. Are you aware of the activities promoted by WMI, targeting private businesses to promote the reuse of recycled water? • If yes, do you think they are sufficient and effective? What could have been done differently? • What should the WMI do in the remaining 2 years? • What expectations do you have from the Government? Mobile APP 5. What do you think about having a Water waste reporting application? Have you heard about such an idea? If yes: • Would you use it? If yes, then how? If not then why? • Do you have any recommendations for the project related to this? INTRODUCTION: This FGD is organized as part of the midterm review of WMI. We received your contacts through the WMI, but this FGD is organized independently from it as well as the project. We would appreciate your feedback which will then help to improve the design of the project in relation to help improving billing and collection in water companies. 134| WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION USAID.GOV 5. FGD for INTERNS 1 FGD Date: Names of participants by gender: 1. How does the new internship Program differ from the older version based on what you know? 2. What do you think about the selection criteria and scopes of work? 3. Why did you apply for the internship? Did it meet your expectations? If yes, then how and if not, then why? ‘ 4. Does it give you an opportunity to gain hands-on experience? If yes, please mention a few key things you have learned? If not, then why, in your view? 5. Did you get mentoring support from the experienced employees? 6. How did you hear about the program? Do you think that the program is known to wider scope of potential applicants? What could be done differently? 7. Was the application and selection process adequate in your view? What could have been done differently? 8. What do you think of doing after it? 9. Do you see a potential career path through this internship? Were you approached by the sector entities for a job? Have you approached them for a job? 10. What should still be done differently: any remaining recommendations? 6. FGD for COMMUNICATIONS EXPERTS 1 FGD Date: Names of participants by gender: 1. What is the role of communication unit in your agency regarding to overall MWI communication strategy? 2. How does WMI support help you in performing your activities? INTRODUCTION: This FGD is organized as part of the midterm review of WMI. We received your contacts through the WMI, but this FGD is organized independently from it as well as the project. We would appreciate your feedback which will then help to improve the design of the project in relation to help improving billing and collection in water companies. INTRODUCTION: This FGD is organized as part of the midterm review of WMI, but this FGD is organized independently from it as well as the project. We would appreciate your feedback which will then help to improve the design of the project in relation to help improving billing and collection in water companies. 135 | WATER MANAGEMENT INITIATIVE (WMI) MID-TERM EVALUATION REPORT USAID.GOV 3. Does the staff of the unit have capacity and know how in developing and implementing the communication strategy? If yes, is it a result of training received? If not what kind of trainings are needed? 4. What are the constraints/challenges facing the unit in implementing the strategy? 5. In your opinion what other activities/measures can improve the ministry communication efforts? 6. Did you participate in the design and planning of the strategy and the action plan? Were all stakeholders involved? Is there coordination and collaboration amongst stakeholders on the strategy and action plan? Between your unit and stakeholders? Please elaborate. 7. Is their coordination between the different departments in the ministry and the utilities to implement the strategy? 8. Are the strategy and the action plan inclusive? How does it addresses women and youth inclusion? 9. Is the access/sharing of the needed information and data is available for different stakeholders? 10. Are the communication channels and the role and responsibilities of the different stakeholders are clear and useful?